KQL Search

Query Details

Custom Agents - Secret-shaped value in observed tool arguments

Custom Secrets In Tool Arguments

Query

let Evidence=materialize(CustomAgentInlineContent(1h));
Evidence
| where IsTool and ToolArguments matches regex @'(-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----|(?:AKIA|ASIA)[A-Z0-9]{16}|gh[pousr]_[A-Za-z0-9]{36,}|sk-[A-Za-z0-9_-]{20,})'
| summarize arg_min(EventTime, *) by _ResourceId, OperationId, AgentKey
| extend Signal='Observed tool arguments contain a credential-shaped pattern'
| extend TimeGenerated=EventTime, ResourceId=_ResourceId, TraceId=OperationId,
    EventCount=tolong(column_ifexists('EventCount', 1)),
    SuccessfulCalls=tolong(column_ifexists('SuccessfulCalls', 0)),
    FailedCalls=tolong(column_ifexists('FailedCalls', 0))
| project TimeGenerated, ResourceId, Agent, AgentId, AgentKey, AppRoleName,
    TraceId, SpanId, Conversation, Model, Tool, TargetHost, ServerDisplay,
    Signal, EventCount, SuccessfulCalls, FailedCalls,
    TriggerSpanId=tostring(column_ifexists('TriggerSpanId', '')),
    TriggerTime=todatetime(column_ifexists('TriggerTime', datetime(null))),
    TriggerTool=tostring(column_ifexists('TriggerTool', '')),
    TriggerToolResult=tostring(column_ifexists('TriggerToolResult', '')),
    TriggerServerDisplay=tostring(column_ifexists('TriggerServerDisplay', '')),
    TriggerTargetHost=tostring(column_ifexists('TriggerTargetHost', '')),
    RiskyOutputKind=tostring(column_ifexists('RiskyOutputKind', '')),
    RelatedToolResults=tolong(column_ifexists('RelatedToolResults', 0)),
    MatchedIntentPrompt=tostring(column_ifexists('IntentPrompt', '')),
    MatchedIntentSpanId=tostring(column_ifexists('IntentSpanId', ''))
| join kind=leftouter (Evidence
    | project ResourceId=_ResourceId, TraceId=OperationId, SpanId, AgentKey, Prompt, Output, ToolArguments, ToolResult, InputMessages, OutputMessages,
    EvidenceSource, PromptEvidenceOrigin, OutputEvidenceOrigin, PromptEvidenceSpanId, OutputEvidenceSpanId,
    InputParseState, OutputParseState, ToolArgumentsState, ToolResultState)
    on ResourceId, TraceId, SpanId, AgentKey
| extend Prompt=coalesce(MatchedIntentPrompt, Prompt),
    PromptEvidenceOrigin=iff(isnotempty(MatchedIntentPrompt), 'Matched intent span: user role', PromptEvidenceOrigin),
    PromptEvidenceSpanId=coalesce(MatchedIntentSpanId, PromptEvidenceSpanId)
| extend PreviewCharacters=toint(min_of(160, max_of(0, (1500 - string_size(tostring(bag_pack(
    'AgentId', AgentId, 'AgentKey', AgentKey, 'TraceId', TraceId, 'SpanId', SpanId,
    'Conversation', Conversation, 'ResourceId', ResourceId, 'Signal', Signal)))) / 12)))
| extend PromptPreview=substring(Prompt, 0, PreviewCharacters),
    OutputPreview=substring(Output, 0, PreviewCharacters)
| extend EvidencePreviewStatus=case(PreviewCharacters == 0, 'Metadata exhausted preview budget; inspect full query results',
    strlen(Prompt) > PreviewCharacters or strlen(Output) > PreviewCharacters, 'Inline evidence; previews shortened; full text in query results',
    isempty(Prompt) and isempty(Output), 'No prompt/output captured or parsed; not a protection verdict',
    'Inline evidence; full recorded text in query results')
| project TimeGenerated, ResourceId, Agent, AgentId, AgentKey, AppRoleName,
    TraceId, SpanId, Conversation, Model, Tool, TargetHost, ServerDisplay,
    Signal, EventCount, SuccessfulCalls, FailedCalls, Prompt, Output, ToolArguments, ToolResult, InputMessages, OutputMessages,
    EvidenceSource, PromptEvidenceOrigin, OutputEvidenceOrigin, PromptEvidenceSpanId, OutputEvidenceSpanId,
    InputParseState, OutputParseState, ToolArgumentsState, ToolResultState,
    PromptPreview, OutputPreview, EvidencePreviewStatus, TriggerSpanId, TriggerTime, TriggerTool, TriggerToolResult,
    TriggerServerDisplay, TriggerTargetHost, RiskyOutputKind, RelatedToolResults
| order by TimeGenerated desc

Explanation

This query is designed to detect potential security risks by identifying credential-like patterns in the arguments of tools that have been executed. Here's a simplified breakdown:

  1. Purpose: The query aims to find patterns resembling credentials (like private keys or access tokens) in the arguments of tools that have been run. This is not necessarily proof of unauthorized access but indicates that sensitive information might be exposed in plaintext.

  2. Severity: The severity level is set to "High", indicating that the findings are considered serious.

  3. Data Source: It uses data from Application Insights, specifically the "AppRequests" and "AppDependencies" data types.

  4. Frequency and Period: The query runs every hour and looks back over the past 14 days.

  5. Detection Logic:

    • It first collects evidence of tool executions within the last hour.
    • It filters these to find those where the tool arguments match specific patterns that resemble credentials.
    • It summarizes the findings, highlighting the earliest occurrence of each pattern.

  6. Output: The query generates a detailed report including the time of the event, resource ID, tool details, and a preview of the arguments and outputs. It also provides a status on the evidence preview, indicating if the full text is available or if it has been shortened.

  7. Entity Mapping: The results are mapped to Azure resources and cloud applications for easier identification.

  8. Incident Management: If an incident is detected, it will create an alert and potentially an incident, grouping related alerts based on specific criteria.

  9. Version and Tags: The query is versioned at 1.2.0 and tagged for easy identification and categorization within Sentinel.

Overall, this query is part of a security monitoring strategy to detect and alert on potential exposure of sensitive information in tool arguments, helping to prevent credential leakage.