KQL Search

Query Details

Microsoft 365 Copilot - Trace-level looping anomaly

Copilot Trace Level Anomaly

Query

CopilotActivity
| where RecordType == "CopilotInteraction"
| extend ThreadId=tostring(LLMEventData.ThreadId), InteractionMessages=array_length(LLMEventData.Messages), InteractionPlugins=array_length(LLMEventData.AISystemPlugin), InteractionResources=array_length(LLMEventData.AccessedResources)
| where isnotempty(ThreadId)
| summarize Interactions=count(), TotalMessages=sum(InteractionMessages), TotalPlugins=sum(InteractionPlugins), TotalResources=sum(InteractionResources), FirstSeen=min(TimeGenerated), LastSeen=max(TimeGenerated), ActorName=take_any(ActorName), SrcIpAddr=take_any(SrcIpAddr), TenantId=take_any(TenantId) by ThreadId, AgentId, AgentName, ActorUserId
| extend DurationMin=datetime_diff('minute', LastSeen, FirstSeen)
| extend MessagesPerMinute=iff(DurationMin > 0, todouble(TotalMessages) / todouble(DurationMin), todouble(TotalMessages))
| where TotalMessages > 100 or (Interactions > 50 and DurationMin < 60) or (MessagesPerMinute > 5 and DurationMin >= 5)
| extend TimeGenerated=LastSeen, Signal="TRACE-LEVEL LOOPING ANOMALY"
| project TimeGenerated, Signal, AgentId, AgentName, ActorName, ActorUserId, ThreadId, SrcIpAddr, TenantId, Interactions, TotalMessages, TotalPlugins, TotalResources, DurationMin, MessagesPerMinute, FirstSeen, LastSeen
| order by TotalMessages desc, MessagesPerMinute desc

Explanation

This query is designed to detect unusual activity patterns in Microsoft 365 Copilot interactions, which could indicate potential misuse or anomalies. Here's a simplified breakdown:

  1. Purpose: The query identifies threads in Microsoft 365 Copilot that show signs of excessive activity, which might suggest an anomaly or misuse.

  2. Criteria for Detection:

    • A thread with more than 100 messages.
    • More than 50 interactions within an hour.
    • A sustained message rate of over five messages per minute for at least five minutes.

  3. Data Source: It uses data from the MicrosoftCopilot connector, specifically looking at CopilotActivity records.

  4. Process:

    • It filters interactions to focus on those with a valid thread ID.
    • It calculates the total number of messages, plugins, and resources used in each thread.
    • It determines the duration of the interaction and calculates the average messages per minute.
    • It flags threads that meet any of the specified criteria for high activity.

  5. Output: The query outputs details such as the time of the anomaly, the agent and actor involved, thread ID, source IP address, and various counts related to the interaction.

  6. Severity and Response: The severity is set to medium, and it triggers an alert if any such anomaly is detected. The alert can lead to the creation of an incident, which can be grouped by account or cloud application for further investigation.

  7. Additional Information:

    • The query runs every hour and checks data from the past hour.
    • It is part of a scheduled task and is tagged for easy identification and categorization.

Overall, this query helps in monitoring and identifying potentially suspicious or unintended high-volume activities in Microsoft 365 Copilot, which could be indicative of misuse or a system anomaly.