Foundry - Cross-server MCP / plugin instruction override

Foundry Mcp Cross Server Override

Query

let overrideMarkers = dynamic([
    "ignore the description","ignore your tool description",
    "instead use the","instead invoke","use the tool from",
    "override the","switch to the","use server","forget the previous tool",
    "this tool is deprecated, use","do not use this tool","call the function from",
    "the real tool is","disregard the schema","actually invoke","route to server"
]);
let markerRegex = strcat("(", strcat_array(overrideMarkers, "|"), ")");
  let protectedContent = union isfuzzy=true
    (datatable(OperationId:string, SpanId:string, ProtectedArgs:string, ProtectedResult:string)[]),
    (AppGenAIContent
    | project OperationId = TraceId, SpanId,
      ProtectedArgs = tostring(ToolCallArguments),
      ProtectedResult = tostring(ToolCallResult));
  let dependencyTools =
    AppDependencies
    | where TimeGenerated > ago(1h)
    | extend
        Agent      = tostring(Properties["gen_ai.agent.name"]),
        ConvId     = tostring(Properties["gen_ai.conversation.id"]),
        Model      = tostring(Properties["gen_ai.request.model"]),
        ToolName   = tolower(tostring(Properties["gen_ai.tool.name"])),
        ToolServer = tolower(tostring(coalesce(
                        Properties["gen_ai.tool.server_url"],
                        Properties["mcp.server.url"],
                        Properties["server_url"],
                        Properties["gen_ai.tool.server_label"],
                        Properties["mcp.server.label"],
                        Properties["server_label"],
                        Properties["gen_ai.tool.server"],
                        Properties["mcp.server.name"],
                        Properties["microsoft.agent.tool.server"], Target, ""))),
        ToolDesc   = tolower(tostring(coalesce(
                        Properties["gen_ai.tool.description"],
                        Properties["microsoft.agent.tool.description"],
                        Properties["mcp.tool.description"], ""))),
          LegacyArgs   = tostring(Properties["gen_ai.tool.call.arguments"]),
          LegacyResult = tostring(Properties["gen_ai.tool.call.result"]),
          SpanId = Id
        | where isnotempty(ConvId) and isnotempty(ToolServer)
        | project TimeGenerated, OperationId, SpanId, Agent, ConvId, Model,
          ToolName, ToolServer, ToolDesc, LegacyArgs, LegacyResult;
      let toolCalls =
        dependencyTools
        | join kind=leftouter protectedContent on OperationId, SpanId
        | extend ToolArgs = tolower(coalesce(ProtectedArgs, LegacyArgs)),
          ToolResult = tolower(coalesce(ProtectedResult, LegacyResult));
let convServers =
    toolCalls
    | summarize ServerCount = dcount(ToolServer),
                Servers     = make_set(ToolServer, 8)
        by ConvId
    | where ServerCount >= 2;
let suspect =
    toolCalls
    | extend Body = strcat(ToolDesc, " ", ToolArgs, " ", ToolResult)
    | where Body matches regex markerRegex
    | extend Marker = extract(markerRegex, 1, Body)
    | summarize OverrideHits = count(),
                Markers   = make_set(Marker, 8),
                Tools     = make_set(ToolName, 16),
                AnyAgent  = take_any(Agent),
                AnyModel  = take_any(Model),
                FirstSeen = min(TimeGenerated),
                LastSeen  = max(TimeGenerated)
        by ConvId;
suspect
| join kind=inner convServers on ConvId
| where OverrideHits >= 1
| extend AccountName = iff(isempty(AnyAgent), "unknown-agent", AnyAgent)
| project LastSeen, AccountName, Agent = AnyAgent, Model = AnyModel, ConvId,
          OverrideHits, ServerCount, Servers, Tools, Markers, FirstSeen
| order by OverrideHits desc

Explanation

This query is designed to detect a specific type of malicious activity involving cross-server manipulation in a system that uses multiple servers for processing tasks. Here's a simplified breakdown of what the query does:

Purpose: The query aims to identify instances where a malicious server attempts to override the behavior of a tool from another server within the same conversation. This is considered an abuse pattern and can be a security threat.
Detection Method:
- The query looks for specific phrases (override markers) that indicate an attempt to override tool instructions. These phrases are searched within tool descriptions, arguments, or results.
- It requires that the conversation involves at least two different servers, indicating cross-server interaction.
Data Sources:
- The query uses data from Application Insights, specifically focusing on AppDependencies and AppGenAIContent data types.
Process:
- It first gathers potential override markers.
- It then collects data on tool calls, including their descriptions, arguments, and results.
- It identifies conversations that involve multiple servers.
- It searches for the override markers in the collected data.
- If any markers are found, it logs details such as the number of override hits, involved tools, and servers.
Output:
- The query outputs metadata about the detected incidents, including the time they were first and last seen, the number of servers involved, and the tools and markers detected.
Severity and Response:
- The severity of this detection is marked as high, indicating a significant security concern.
- If such an incident is detected, an alert is generated, and an incident is created for further investigation.
Tactics and Techniques:
- The query is associated with tactics like Initial Access, Defense Evasion, Execution, and Lateral Movement.
- It relates to specific techniques such as T1574 (Hijack Execution Flow) and T1195 (Supply Chain Compromise).

Overall, this query is part of a security monitoring system that helps detect and respond to potential cross-server manipulation threats in a multi-server environment.