KQL Search

Query Details

Custom Agents - Active-content syntax in recorded output

Custom Unsafe Active Content Output

Query

let Evidence=materialize(CustomAgentInlineContent(1h));
Evidence
| where OutputText matches regex @'(?i)(<script(?:\s|>)|javascript\s*:|data\s*:\s*text/html|<iframe(?:\s|>)|<object(?:\s|>)|<embed(?:\s|>)|on(?:error|load|click)\s*=|document\.cookie|window\.location\s*=)'
| summarize arg_min(EventTime, *) by _ResourceId, OperationId, AgentKey
| extend Signal='Inline output contains active-content syntax'
| extend TimeGenerated=EventTime, ResourceId=_ResourceId, TraceId=OperationId,
    EventCount=tolong(column_ifexists('EventCount', 1)),
    SuccessfulCalls=tolong(column_ifexists('SuccessfulCalls', 0)),
    FailedCalls=tolong(column_ifexists('FailedCalls', 0))
| project TimeGenerated, ResourceId, Agent, AgentId, AgentKey, AppRoleName,
    TraceId, SpanId, Conversation, Model, Tool, TargetHost, ServerDisplay,
    Signal, EventCount, SuccessfulCalls, FailedCalls,
    TriggerSpanId=tostring(column_ifexists('TriggerSpanId', '')),
    TriggerTime=todatetime(column_ifexists('TriggerTime', datetime(null))),
    TriggerTool=tostring(column_ifexists('TriggerTool', '')),
    TriggerToolResult=tostring(column_ifexists('TriggerToolResult', '')),
    TriggerServerDisplay=tostring(column_ifexists('TriggerServerDisplay', '')),
    TriggerTargetHost=tostring(column_ifexists('TriggerTargetHost', '')),
    RiskyOutputKind=tostring(column_ifexists('RiskyOutputKind', '')),
    RelatedToolResults=tolong(column_ifexists('RelatedToolResults', 0)),
    MatchedIntentPrompt=tostring(column_ifexists('IntentPrompt', '')),
    MatchedIntentSpanId=tostring(column_ifexists('IntentSpanId', ''))
| join kind=leftouter (Evidence
    | project ResourceId=_ResourceId, TraceId=OperationId, SpanId, AgentKey, Prompt, Output, ToolArguments, ToolResult, InputMessages, OutputMessages,
    EvidenceSource, PromptEvidenceOrigin, OutputEvidenceOrigin, PromptEvidenceSpanId, OutputEvidenceSpanId,
    InputParseState, OutputParseState, ToolArgumentsState, ToolResultState)
    on ResourceId, TraceId, SpanId, AgentKey
| extend Prompt=coalesce(MatchedIntentPrompt, Prompt),
    PromptEvidenceOrigin=iff(isnotempty(MatchedIntentPrompt), 'Matched intent span: user role', PromptEvidenceOrigin),
    PromptEvidenceSpanId=coalesce(MatchedIntentSpanId, PromptEvidenceSpanId)
| extend PreviewCharacters=toint(min_of(160, max_of(0, (1500 - string_size(tostring(bag_pack(
    'AgentId', AgentId, 'AgentKey', AgentKey, 'TraceId', TraceId, 'SpanId', SpanId,
    'Conversation', Conversation, 'ResourceId', ResourceId, 'Signal', Signal)))) / 12)))
| extend PromptPreview=substring(Prompt, 0, PreviewCharacters),
    OutputPreview=substring(Output, 0, PreviewCharacters)
| extend EvidencePreviewStatus=case(PreviewCharacters == 0, 'Metadata exhausted preview budget; inspect full query results',
    strlen(Prompt) > PreviewCharacters or strlen(Output) > PreviewCharacters, 'Inline evidence; previews shortened; full text in query results',
    isempty(Prompt) and isempty(Output), 'No prompt/output captured or parsed; not a protection verdict',
    'Inline evidence; full recorded text in query results')
| project TimeGenerated, ResourceId, Agent, AgentId, AgentKey, AppRoleName,
    TraceId, SpanId, Conversation, Model, Tool, TargetHost, ServerDisplay,
    Signal, EventCount, SuccessfulCalls, FailedCalls, Prompt, Output, ToolArguments, ToolResult, InputMessages, OutputMessages,
    EvidenceSource, PromptEvidenceOrigin, OutputEvidenceOrigin, PromptEvidenceSpanId, OutputEvidenceSpanId,
    InputParseState, OutputParseState, ToolArgumentsState, ToolResultState,
    PromptPreview, OutputPreview, EvidencePreviewStatus, TriggerSpanId, TriggerTime, TriggerTool, TriggerToolResult,
    TriggerServerDisplay, TriggerTargetHost, RiskyOutputKind, RelatedToolResults
| order by TimeGenerated desc

Explanation

This query is part of a scheduled alert in Azure Sentinel that aims to detect potentially risky active-content syntax in the output of custom agents. Here's a simplified breakdown:

  1. Purpose: The query identifies and flags instances where scripts, iframes, or other active-content syntax appear in the inline output of custom agents. This is not evidence of execution, just detection of the syntax.

  2. Severity: The alert is classified with a "Medium" severity level.

  3. Data Source: It uses data from Application Insights, specifically the "AppRequests" and "AppDependencies" data types.

  4. Frequency and Period: The query runs every hour and looks back over the past 14 days.

  5. Trigger: The alert triggers if any instances are found (threshold greater than 0).

  6. Query Logic:

    • It first collects evidence of inline content over the past hour.
    • It searches for specific patterns in the output text that match active-content syntax using a regular expression.
    • It summarizes and extends the data with additional information like event time, resource ID, and operation ID.
    • It projects various fields for output, including previews of the prompt and output text.
    • It joins additional evidence details to enrich the results.

  7. Output: The results include details like the time generated, resource ID, agent information, and a preview of the prompt and output. It also provides a status on the evidence preview.

  8. Entity Mappings: The query maps results to Azure resources and cloud applications based on specific fields.

  9. Custom Details: It includes custom details like agent ID, trace ID, and a preview of the prompt and output.

  10. Incident Configuration: If the alert triggers, it creates an incident and groups alerts by agent key and resource ID.

  11. Version and Tags: The query is version 1.2.0 and tagged with relevant identifiers like "Sentinel-As-Code" and "Custom-Agents".

Overall, this query is designed to monitor and alert on potentially risky content in the outputs of custom agents, helping to identify and mitigate security risks.