Foundry - Sensitive tool / capability invoked by agent

Foundry Sensitive Tool Invocation

Query

let sensitiveTools = dynamic([
    "code_interpreter", "code-interpreter", "python", "shell", "bash",
    "cmd", "powershell", "file_search", "file-search", "filesystem",
    "http", "fetch", "browser", "email", "sendmail", "sql", "database"
]);
AppDependencies
| where isnotempty(Properties["gen_ai.tool.name"])
    or isnotempty(Properties["gen_ai.tool.type"])
| extend
    Agent    = tostring(Properties["gen_ai.agent.name"]),
    Model    = tostring(Properties["gen_ai.request.model"]),
    ConvId   = tostring(Properties["gen_ai.conversation.id"]),
    ToolName = tolower(tostring(Properties["gen_ai.tool.name"])),
    ToolType = tolower(tostring(Properties["gen_ai.tool.type"]))
| where ToolName has_any (sensitiveTools) or ToolType has_any (sensitiveTools)
| summarize
    Invocations = count(),
    Tools       = make_set(ToolName, 16),
    ToolTypes   = make_set(ToolType, 16),
    FirstSeen   = min(TimeGenerated),
    LastSeen    = max(TimeGenerated)
    by Agent, Model, ConvId
| extend AccountName = iff(isempty(Agent), "unknown-agent", Agent)
| project
    LastSeen, AccountName, Agent, Model, ConvId, Invocations,
    Tools, ToolTypes, FirstSeen
| order by Invocations desc

Explanation

This query is designed to monitor and alert on potentially risky activities performed by agents using a system called Foundry. It specifically looks for instances where an agent uses high-impact tools or capabilities, such as code interpreters, shell commands, file system access, HTTP requests, database queries, or email sending. These actions are considered sensitive because they can be exploited by attackers after a successful injection attack.

Here's a simple breakdown of the query:

Sensitive Tools List: It defines a list of tools and capabilities considered sensitive, such as "python", "shell", "http", "sql", etc.
Data Source: It uses data from the AppDependencies table, which is part of the Application Insights data connector.
Filtering: The query filters records where either the tool name or type is not empty and matches any of the sensitive tools listed.
Data Extraction: It extracts relevant information like the agent's name, the model used, conversation ID, tool name, and tool type.
Aggregation: It summarizes the data to count the number of invocations, list the tools and tool types used, and record the first and last time these actions were seen.
Output: The results are ordered by the number of invocations, and it projects key details like the last seen time, agent name, model, conversation ID, and the tools used.
Alerting: If any sensitive tool usage is detected, an alert is triggered. The alert includes information about the agent and the tools used, and incidents are created for further investigation.
Incident Management: Incidents are grouped by the account (agent) and can be reopened if similar activity is detected within a 6-hour lookback period.

Overall, this query helps security teams identify and respond to potentially malicious activities involving sensitive tools used by agents, which could indicate an attack or misuse of the system.