Kubernetes Abuse of Secret by Unusual User Agent

name: Kubernetes Abuse of Secret by Unusual User Agent
id: 096ab390-05ca-462c-884e-343acd5b9240
version: 10
creation_date: '2023-12-20'
modification_date: '2026-05-13'
author: Patrick Bareiss, Splunk
status: production
type: Anomaly
description: The following analytic detects unauthorized access or misuse of Kubernetes Secrets by unusual user agents. It leverages Kubernetes Audit logs to identify anomalies in access patterns by analyzing the source of requests based on user agents. This activity is significant for a SOC because Kubernetes Secrets store sensitive information like passwords, OAuth tokens, and SSH keys, making them critical assets. If confirmed malicious, this activity could lead to unauthorized access to sensitive systems or data, potentially resulting in significant security breaches and exfiltration of critical information.
data_source:
    - Kubernetes Audit
search: |-
    `kube_audit` objectRef.resource=secrets verb=get
      | search NOT `kube_allowed_user_agents`
      | fillnull
      | stats count
        BY objectRef.name objectRef.namespace objectRef.resource
           requestReceivedTimestamp requestURI responseStatus.code
           sourceIPs{} stage user.groups{}
           user.uid user.username userAgent
           verb
      | rename sourceIPs{} as src_ip, user.username as user
      | `kubernetes_abuse_of_secret_by_unusual_user_agent_filter`
how_to_implement: The detection is based on data that originates from Kubernetes Audit logs. Ensure that audit logging is enabled in your Kubernetes cluster. Kubernetes audit logs provide a record of the requests made to the Kubernetes API server, which is crucial for monitoring and detecting suspicious activities. Configure the audit policy in Kubernetes to determine what kind of activities are logged. This is done by creating an Audit Policy and providing it to the API server. Use the Splunk OpenTelemetry Collector for Kubernetes to collect the logs. This doc will describe how to collect the audit log file https://github.com/signalfx/splunk-otel-collector-chart/blob/main/docs/migration-from-sck.md. When you want to use this detection with AWS EKS, you need to enable EKS control plane logging https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html. Then you can collect the logs from Cloudwatch using the AWS TA https://splunk.github.io/splunk-add-on-for-amazon-web-services/CloudWatchLogs/.
known_false_positives: No false positives have been identified at this time.
references:
    - https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/
intermediate_findings:
    entities:
        - field: user
          type: user
          score: 20
          message: Access of Kubernetes secret $objectRef.name$ from unusual user agent $userAgent$ by $user$
threat_objects:
    - field: src_ip
      type: ip_address
analytic_story:
    - Kubernetes Security
asset_type: Kubernetes
mitre_attack_id:
    - T1552.007
product:
    - Splunk Enterprise
    - Splunk Enterprise Security
    - Splunk Cloud
category: cloud
security_domain: network