KQL Search

Query Details

Custom Agents - Secret-shaped value in recorded model output

Custom Secret Shaped Output

Query

let Evidence=materialize(CustomAgentInlineContent(1h));
Evidence
| where OutputText matches regex @'(-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----|(?:AKIA|ASIA)[A-Z0-9]{16}|gh[pousr]_[A-Za-z0-9]{36,}|sk-[A-Za-z0-9_-]{20,})'
| summarize arg_min(EventTime, *) by _ResourceId, OperationId, AgentKey
| extend Signal='Inline output contains a credential-shaped pattern'
| extend TimeGenerated=EventTime, ResourceId=_ResourceId, TraceId=OperationId,
    EventCount=tolong(column_ifexists('EventCount', 1)),
    SuccessfulCalls=tolong(column_ifexists('SuccessfulCalls', 0)),
    FailedCalls=tolong(column_ifexists('FailedCalls', 0))
| project TimeGenerated, ResourceId, Agent, AgentId, AgentKey, AppRoleName,
    TraceId, SpanId, Conversation, Model, Tool, TargetHost, ServerDisplay,
    Signal, EventCount, SuccessfulCalls, FailedCalls,
    TriggerSpanId=tostring(column_ifexists('TriggerSpanId', '')),
    TriggerTime=todatetime(column_ifexists('TriggerTime', datetime(null))),
    TriggerTool=tostring(column_ifexists('TriggerTool', '')),
    TriggerToolResult=tostring(column_ifexists('TriggerToolResult', '')),
    TriggerServerDisplay=tostring(column_ifexists('TriggerServerDisplay', '')),
    TriggerTargetHost=tostring(column_ifexists('TriggerTargetHost', '')),
    RiskyOutputKind=tostring(column_ifexists('RiskyOutputKind', '')),
    RelatedToolResults=tolong(column_ifexists('RelatedToolResults', 0)),
    MatchedIntentPrompt=tostring(column_ifexists('IntentPrompt', '')),
    MatchedIntentSpanId=tostring(column_ifexists('IntentSpanId', ''))
| join kind=leftouter (Evidence
    | project ResourceId=_ResourceId, TraceId=OperationId, SpanId, AgentKey, Prompt, Output, ToolArguments, ToolResult, InputMessages, OutputMessages,
    EvidenceSource, PromptEvidenceOrigin, OutputEvidenceOrigin, PromptEvidenceSpanId, OutputEvidenceSpanId,
    InputParseState, OutputParseState, ToolArgumentsState, ToolResultState)
    on ResourceId, TraceId, SpanId, AgentKey
| extend Prompt=coalesce(MatchedIntentPrompt, Prompt),
    PromptEvidenceOrigin=iff(isnotempty(MatchedIntentPrompt), 'Matched intent span: user role', PromptEvidenceOrigin),
    PromptEvidenceSpanId=coalesce(MatchedIntentSpanId, PromptEvidenceSpanId)
| extend PreviewCharacters=toint(min_of(160, max_of(0, (1500 - string_size(tostring(bag_pack(
    'AgentId', AgentId, 'AgentKey', AgentKey, 'TraceId', TraceId, 'SpanId', SpanId,
    'Conversation', Conversation, 'ResourceId', ResourceId, 'Signal', Signal)))) / 12)))
| extend PromptPreview=substring(Prompt, 0, PreviewCharacters),
    OutputPreview=substring(Output, 0, PreviewCharacters)
| extend EvidencePreviewStatus=case(PreviewCharacters == 0, 'Metadata exhausted preview budget; inspect full query results',
    strlen(Prompt) > PreviewCharacters or strlen(Output) > PreviewCharacters, 'Inline evidence; previews shortened; full text in query results',
    isempty(Prompt) and isempty(Output), 'No prompt/output captured or parsed; not a protection verdict',
    'Inline evidence; full recorded text in query results')
| project TimeGenerated, ResourceId, Agent, AgentId, AgentKey, AppRoleName,
    TraceId, SpanId, Conversation, Model, Tool, TargetHost, ServerDisplay,
    Signal, EventCount, SuccessfulCalls, FailedCalls, Prompt, Output, ToolArguments, ToolResult, InputMessages, OutputMessages,
    EvidenceSource, PromptEvidenceOrigin, OutputEvidenceOrigin, PromptEvidenceSpanId, OutputEvidenceSpanId,
    InputParseState, OutputParseState, ToolArgumentsState, ToolResultState,
    PromptPreview, OutputPreview, EvidencePreviewStatus, TriggerSpanId, TriggerTime, TriggerTool, TriggerToolResult,
    TriggerServerDisplay, TriggerTargetHost, RiskyOutputKind, RelatedToolResults
| order by TimeGenerated desc

Explanation

This query is designed to detect patterns in text output from custom agents that resemble credentials, such as private keys or access keys. It does not confirm the validity or misuse of these credentials but flags their presence for further investigation. The query runs every hour and looks back over the past 14 days of data from Application Insights, specifically focusing on application requests and dependencies.

Here's a simplified breakdown of what the query does:

  1. Data Collection: It collects data from custom agents over the last hour.
  2. Pattern Matching: It searches for text patterns that look like credentials (e.g., private keys, AWS keys, GitHub tokens) in the output.
  3. Summarization: It summarizes the findings by the resource, operation, and agent involved, marking them with a signal indicating a credential-like pattern was found.
  4. Data Enrichment: It adds additional context to the findings, such as event time, resource ID, and counts of successful and failed calls.
  5. Projection: It selects specific fields to display, including previews of the prompt and output text, and a status message about the evidence preview.
  6. Joining Data: It joins additional evidence data to enrich the results with more details about the prompts and outputs.
  7. Ordering: It orders the results by the time they were generated, showing the most recent first.
  8. Entity Mapping: It maps the results to Azure resources and cloud applications for easier identification.
  9. Custom Details and Grouping: It includes custom details for alerts and groups incidents based on specific criteria, creating an incident if a potential issue is detected.

The query is currently disabled and has a high severity level, indicating its importance in monitoring for potential credential exposure.