Copilot Studio - Goal hijacking / goal drift across a conversation

Copilot Studio Goal Drift Between Turns

Query

let benignFirstWords = dynamic([
    "summarise","summarize","translate","explain","help me","tell me",
    "draft","write a","compose","describe","what is","how does",
    "example","tutorial","define","outline","brainstorm","rephrase",
    "improve the wording","proofread","check the grammar"
]);
let sensitiveIntent = dynamic([
    "delete","drop","grant","revoke","admin","password","secret","key",
    "token","credential","database","table","customer","payment",
    "invoice","payroll","exfil","send to","upload to","exec","execute",
    "run command","ssh","sudo","privilege","impersonate"
]);
let sensitiveConnectors = dynamic([
    "office365","sendemail","sendmail","outlook","exchange",
    "http","webhook","azuread","azure","sql","dataverse",
    "sharepoint","onedrive","powershell","function","automate",
    "logicapp","graph","keyvault","blob","storage"
]);
let firstMsg =
    AppEvents
    | where TimeGenerated > ago(2h)
    | where Name == "BotMessageReceived"
    | extend ConvId = tostring(Properties["conversationId"]),
             Text   = tolower(tostring(Properties["text"]))
    | where isnotempty(Text)
    | summarize arg_min(TimeGenerated, Text, UserId, ClientIP, ChannelId = tostring(Properties["channelId"])) by ConvId
    | extend FirstInput      = substring(Text, 0, 1024),
             BenignIntent    = Text has_any (benignFirstWords),
             SensitiveIntent = Text has_any (sensitiveIntent);
let acts =
    AppDependencies
    | where TimeGenerated > ago(2h)
    | where AppRoleName == "Microsoft Copilot Studio" or DependencyType == "Connector"
    | extend ConvId   = tostring(Properties["conversationId"]),
             MatchKey = tolower(strcat(Name, " ", Target))
    | where MatchKey has_any (sensitiveConnectors)
    | summarize SensitiveCalls    = count(),
                Connectors        = make_set(Name, 16),
                DistinctConnectors = dcount(Name),
                Targets           = make_set(Target, 25),
                FirstSeen         = min(TimeGenerated),
                LastSeen          = max(TimeGenerated)
            by ConvId;
firstMsg
| join kind=inner acts on ConvId
| where SensitiveCalls >= 3
        and BenignIntent == true
        and SensitiveIntent == false
| extend AccountName = iff(isempty(UserId), "unknown-agent", UserId)
| project LastSeen, AccountName, UserId, ConvId, ChannelId, ClientIP,
          SensitiveCalls, DistinctConnectors, Connectors, Targets, FirstInput, FirstSeen
| order by SensitiveCalls desc

Explanation

This query is designed to detect a specific type of security issue called "goal hijacking" or "goal drift" in conversations involving Microsoft Copilot Studio. Here's a simplified breakdown:

Purpose: The query identifies situations where a conversation starts with a benign user request (like summarizing or translating) but ends up making multiple sensitive connector calls (such as accessing email, databases, or cloud services).
Detection Method:
- Benign Start: The conversation begins with a message that matches a list of non-sensitive intents (e.g., "summarize," "translate") and does not contain any sensitive keywords (e.g., "delete," "admin").
- Sensitive Activity: Despite the benign start, the conversation results in three or more calls to sensitive connectors (e.g., Office 365, SQL, Azure).
Data Sources: The query uses data from Application Insights, specifically looking at application events and dependencies.
Frequency and Period: The query runs every hour and looks back over the past two hours.
Severity and Tactics: The issue is considered medium severity and relates to tactics like execution and defense evasion.
Output: If a conversation meets the criteria, it logs details such as the user account, client IP, number of sensitive calls, and the initial user message.
Incident Management: If such an event is detected, it creates an incident for further investigation.

Overall, this query helps identify potential security risks where a seemingly harmless conversation could lead to unauthorized or sensitive actions, possibly due to indirect influences or vulnerabilities.

Details

David Alonso

Released: June 8, 2026

Tables

AppEventsAppDependencies

Keywords

AppEventsAppDependenciesApplicationInsightsBotMessageReceivedMicrosoftCopilotStudioConversationIdUserIdClientIPChannelIdSensitiveCallsDistinctConnectorsConnectorsTargetsFirstInputFirstSeenAccountName

Operators

letdynamictolowertostringisnotemptysummarizearg_minextendsubstringhas_anyagowherejoinkind=inneriffisemptyprojectorder bydesccountmake_setdcountminmax

Severity

Medium

Tactics

ExecutionDefenseEvasion

MITRE Techniques

T1059 T1556

Frequency: PT1H

Period: PT2H

Actions

GitHub

KQL Search