KQL Search

Query Details

Copilot Studio - MCP server and tool inventory

Agent Mcp Inventory

Query

let lookback = 30d;
let Legacy = AppDependencies
| where TimeGenerated > ago(lookback)
| where AppRoleName == "Microsoft Copilot Studio"
| where Name has "MCP" or Target has "mcp"
| project TimeGenerated, OperationId, Success, DurationMs,
    Agent = "", Conversation = tostring(Properties["conversationId"]),
    Tool = Name, ToolType = "MCP - connector telemetry", Target = tostring(Target),
    Source = "Agent-level connector";
let Modern = AppDependencies
| where TimeGenerated > ago(lookback)
| where tostring(Properties["gen_ai.operation.name"]) == "execute_tool"
| extend Tool = tostring(Properties["gen_ai.tool.name"]),
         ToolType = tostring(Properties["gen_ai.tool.type"])
| where Tool has "mcp" or ToolType has "mcp"
| project TimeGenerated, OperationId, Success, DurationMs,
    Agent = tostring(Properties["gen_ai.agent.name"]),
    Conversation = tostring(Properties["gen_ai.conversation.id"]),
    Tool, ToolType, Target = tostring(Target), Source = "Environment ExecuteTool";
union Legacy, Modern
| summarize Calls = count(), Failures = countif(Success == false),
    Agents = make_set_if(Agent, isnotempty(Agent), 20),
    Conversations = dcountif(Conversation, isnotempty(Conversation)),
    AvgMs = round(avg(DurationMs), 0), P95Ms = round(percentile(DurationMs, 95), 0),
    FirstSeen = min(TimeGenerated), LastSeen = max(TimeGenerated)
    by Tool, ToolType, Target, Source
| order by Calls desc

Explanation

This query is designed to inventory and analyze data related to Microsoft Copilot Studio's MCP (Microsoft Cloud Platform) server and tool usage over the past 30 days. It combines two sets of data: "Legacy" and "Modern."

  1. Legacy Data:

    • It filters records from the AppDependencies table where the AppRoleName is "Microsoft Copilot Studio" and either the Name or Target contains "MCP".
    • It extracts relevant fields such as TimeGenerated, OperationId, Success, DurationMs, and others, labeling the source as "Agent-level connector".

  2. Modern Data:

    • It filters records where the operation name is "execute_tool" and either the tool name or type contains "mcp".
    • It extracts similar fields as the Legacy data, labeling the source as "Environment ExecuteTool".

  3. Combining Data:

    • The query combines both datasets using a union operation.
    • It then summarizes the combined data by counting the number of calls, failures, unique agents, and conversations.
    • It calculates average and 95th percentile durations, and identifies the first and last seen timestamps for each tool, tool type, target, and source.

  4. Output:

    • The results are ordered by the number of calls in descending order.

The purpose of this query is to provide insights into the usage patterns and performance of MCP tools, helping to populate a watchlist and identify responsible parties before enforcing rules. It is tagged with various identifiers for tracking and categorization purposes.