Foundry - Agent grounded on untrusted tool / RAG source

Foundry Untrusted Tool Source

Query

let trusted =
    _GetWatchlist('FoundryTrustedToolSources')
    | project SourceUri = tolower(tostring(column_ifexists('SourceUri', '')))
    | where isnotempty(SourceUri);
  let protectedContent = union isfuzzy=true
    (datatable(OperationId:string, SpanId:string, ProtectedArgs:string, ProtectedResult:string)[]),
    (AppGenAIContent
    | project OperationId = TraceId, SpanId,
      ProtectedArgs = tostring(ToolCallArguments),
      ProtectedResult = tostring(ToolCallResult));
  AppDependencies
| where isnotempty(Properties["gen_ai.tool.name"])
    or isnotempty(Properties["gen_ai.tool.call.arguments"])
| extend
    Agent      = tostring(Properties["gen_ai.agent.name"]),
    Model      = tostring(Properties["gen_ai.request.model"]),
    ConvId     = tostring(Properties["gen_ai.conversation.id"]),
    ToolName   = tostring(Properties["gen_ai.tool.name"]),
    ToolType   = tostring(Properties["gen_ai.tool.type"]),
      LegacyArgs   = tostring(Properties["gen_ai.tool.call.arguments"]),
      LegacyResult = tostring(Properties["gen_ai.tool.call.result"]),
      SpanId = Id
    | join kind=leftouter protectedContent on OperationId, SpanId
    | extend ToolArgs = coalesce(ProtectedArgs, LegacyArgs),
      ToolResult = coalesce(ProtectedResult, LegacyResult)
| extend SourceUri = tolower(extract(@"https?://([A-Za-z0-9.\-]+)", 1, strcat(ToolArgs, " ", ToolResult)))
| where isnotempty(SourceUri)
| join kind=leftanti trusted on SourceUri
| summarize
    UntrustedHits    = count(),
    UntrustedSources = make_set(SourceUri, 64),
    Tools            = make_set(ToolName, 16),
    ToolTypes        = make_set(ToolType, 8),
    FirstSeen        = min(TimeGenerated),
    LastSeen         = max(TimeGenerated)
    by Agent, Model, ConvId
| extend AccountName = iff(isempty(Agent), "unknown-agent", Agent)
| project
    LastSeen, AccountName, Agent, Model, ConvId, UntrustedHits,
    UntrustedSources, Tools, ToolTypes, FirstSeen
| order by UntrustedHits desc

Explanation

This query is designed to detect potential security threats in a system using Foundry or Agent services. Here's a simplified breakdown of what it does:

Purpose: The query identifies when a tool call fetches content from an untrusted source. This is important for detecting attacks where malicious content is planted in a location that the agent retrieves from, potentially leading to unauthorized access or data manipulation.
Trusted Sources: It uses a watchlist called "FoundryTrustedToolSources" to determine which sources are trusted. If a source is not on this list, it is considered untrusted.
Data Sources: The query pulls data from two main sources: AppDependencies and AppGenAIContent. These contain information about tool calls and their results.
Process:
- It extracts URLs from tool arguments and results.
- It checks if these URLs are in the list of trusted sources.
- If a URL is not trusted, it is flagged.
Output: The query summarizes the findings by counting how many times untrusted sources were accessed, listing these sources, and providing details about the tools and agents involved.
Severity and Alerts: The severity of this detection is set to "Medium". If any untrusted sources are detected, an alert is triggered, and an incident is created for further investigation.
Frequency: The query runs every hour and looks at data from the past hour.
Tactics and Techniques: It relates to tactics like Initial Access and Defense Evasion, and techniques such as T1195 (Supply Chain Compromise) and T1199 (Trusted Relationship).

In essence, this query helps maintain the security of a system by ensuring that only approved sources are used for tool calls, thereby preventing potential security breaches through untrusted content.