Foundry - Sensitive data / secrets in agent output

Foundry Sensitive Data In Output

Query

union isfuzzy=true
(AppDependencies
| extend ContentKey = strcat(OperationId, ":", Id),
    Agent = tostring(Properties["gen_ai.agent.name"]),
    Model = tostring(Properties["gen_ai.request.model"]),
    ConvId = tostring(Properties["gen_ai.conversation.id"]),
    ProjectId = tostring(Properties["microsoft.foundry.project.id"]),
    Output = tostring(Properties["gen_ai.output.messages"])),
(AppGenAIContent
| extend ContentKey = strcat(TraceId, ":", SpanId), Agent = tostring(AgentName),
    Model = tostring(ModelName), ConvId = tostring(Attributes["gen_ai.conversation.id"]),
    ProjectId = tostring(Attributes["microsoft.foundry.project.id"]),
    Output = tostring(OutputMessages))
| where isnotempty(Output)
| summarize arg_max(TimeGenerated, *) by ContentKey
| extend
    HasAwsKey     = Output matches regex @"AKIA[0-9A-Z]{16}",
    HasPrivateKey = Output contains "-----BEGIN" and Output contains "PRIVATE KEY-----",
    HasJwt        = Output matches regex @"eyJ[A-Za-z0-9_\-]{10,}\.[A-Za-z0-9_\-]{10,}\.[A-Za-z0-9_\-]{10,}",
    HasCreditCard = Output matches regex @"\b(?:\d[ \-]?){13,16}\b",
    EmailCount    = array_length(extract_all(@"([A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,})", Output))
| where HasAwsKey or HasPrivateKey or HasJwt or HasCreditCard or EmailCount >= 10
| extend Signal = strcat(
    iff(HasAwsKey, "AWSAccessKey;", ""),
    iff(HasPrivateKey, "PrivateKey;", ""),
    iff(HasJwt, "JWT;", ""),
    iff(HasCreditCard, "CreditCardLike;", ""),
    iff(EmailCount >= 10, strcat("BulkEmails(", tostring(EmailCount), ");"), ""))
| extend AccountName = iff(isempty(Agent), "unknown-agent", Agent)
| project
    TimeGenerated, Signal, AccountName, Agent, Model, ProjectId,
    ConvId, EmailCount
| order by TimeGenerated desc

Explanation

This query is designed to detect and alert on potentially sensitive data being exposed in the output of a Foundry or Agent Service. It looks for specific patterns that indicate the presence of sensitive information, such as:

AWS access keys
PEM private-key blocks
JSON Web Tokens (JWTs)
Credit card-like number sequences
A large number of distinct email addresses (10 or more) in a single output

The query works by examining data from two sources: AppDependencies and AppGenAIContent. It combines these data sources and checks the output for the presence of the sensitive patterns mentioned above using regular expressions.

If any of these patterns are found, the query generates an alert with metadata about the incident, including the time it was generated, the type of sensitive data detected (e.g., AWSAccessKey, PrivateKey, etc.), and information about the agent or model involved.

The query runs every hour and checks data from the past hour. If it finds any matches, it triggers an alert with a high severity level. The alert is then grouped by account, and incidents are created for further investigation.

This query is part of a scheduled task and is tagged with relevant keywords for easy identification and categorization.