Nodes

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches list-nodes: list nodes, watch-nodes: watch nodes, create-nodes: create nodes, update-nodes: update nodes, patch-nodes: patch nodes
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.
• GKE Multi-Resource Discovery source medium: Adversaries who land credentials in a GKE cluster—or abuse an over-privileged token, often map the environment before exfiltration or privilege escalation. A practical first pass is to learn where workloads run, how the cluster is partitioned, and what RBAC exists at namespace vs cluster scope. Rapid get/list traffic across many distinct API resource kinds that answer those questions (namespaces, workloads, roles, cluster-wide roles) is a common setup and orientation pattern for both interactive attackers and automated recon scripts. This rule highlights that cross-resource burst from a single client fingerprint within a one-minute bucket when both cluster-layout and RBAC resource kinds are touched, so analysts can separate routine automation from potential discovery ahead of follow-on actions.T1613↳ also matches list-nodes: list nodes
  Investigation guide

  Triage and analysis#

  Investigating GKE Multi-Resource Discovery#

  The rule groups GKE audit get/list events on namespaces, nodes, pods, configmaps, serviceaccounts, roles, rolebindings, clusterroles, and clusterrolebindings into one-minute windows per client.user.email, source.ip, and user_agent.original. It alerts when five or more distinct resource kinds appear and the burst includes both cluster-layout kinds (namespaces, pods, or nodes) and RBAC kinds. Allowed and denied authorizations are both included: failures still signal probing.

  Possible investigation steps#

  • Review Esql.enumerated_resources, Esql.enumerated_namespaces, and Esql.enumerated_resource_names for ordering and targeted APIs.
  • Confirm whether source.ip and user_agent.original match expected admin or automation clients.
  • Correlate with follow-on secret reads, RoleBinding changes, pod exec, or unusual user agents from the same actor.

  False positive analysis#

  • Documented platform sync jobs that read layout and RBAC together; exclude known service accounts after validation.
  • Upgrade or install windows that briefly query many resource kinds; correlate with change records.

  Response and remediation#

  • If malicious, revoke or rotate the implicated credentials, tighten RBAC, and inspect for data access or persistence established after the burst.

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-nodes: get nodes, watch-nodes: watch nodes, create-nodes: create nodes, update-nodes: update nodes, patch-nodes: patch nodes
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.
• GKE Multi-Resource Discovery source medium: Adversaries who land credentials in a GKE cluster—or abuse an over-privileged token, often map the environment before exfiltration or privilege escalation. A practical first pass is to learn where workloads run, how the cluster is partitioned, and what RBAC exists at namespace vs cluster scope. Rapid get/list traffic across many distinct API resource kinds that answer those questions (namespaces, workloads, roles, cluster-wide roles) is a common setup and orientation pattern for both interactive attackers and automated recon scripts. This rule highlights that cross-resource burst from a single client fingerprint within a one-minute bucket when both cluster-layout and RBAC resource kinds are touched, so analysts can separate routine automation from potential discovery ahead of follow-on actions.T1613↳ also matches get-nodes: get nodes
  Investigation guide

  Triage and analysis#

  Investigating GKE Multi-Resource Discovery#

  The rule groups GKE audit get/list events on namespaces, nodes, pods, configmaps, serviceaccounts, roles, rolebindings, clusterroles, and clusterrolebindings into one-minute windows per client.user.email, source.ip, and user_agent.original. It alerts when five or more distinct resource kinds appear and the burst includes both cluster-layout kinds (namespaces, pods, or nodes) and RBAC kinds. Allowed and denied authorizations are both included: failures still signal probing.

  Possible investigation steps#

  • Review Esql.enumerated_resources, Esql.enumerated_namespaces, and Esql.enumerated_resource_names for ordering and targeted APIs.
  • Confirm whether source.ip and user_agent.original match expected admin or automation clients.
  • Correlate with follow-on secret reads, RoleBinding changes, pod exec, or unusual user agents from the same actor.

  False positive analysis#

  • Documented platform sync jobs that read layout and RBAC together; exclude known service accounts after validation.
  • Upgrade or install windows that briefly query many resource kinds; correlate with change records.

  Response and remediation#

  • If malicious, revoke or rotate the implicated credentials, tighten RBAC, and inspect for data access or persistence established after the burst.

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-nodes: get nodes, list-nodes: list nodes, create-nodes: create nodes, update-nodes: update nodes, patch-nodes: patch nodes
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-nodes: get nodes, list-nodes: list nodes, watch-nodes: watch nodes, update-nodes: update nodes, patch-nodes: patch nodes
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-nodes: get nodes, list-nodes: list nodes, watch-nodes: watch nodes, create-nodes: create nodes, patch-nodes: patch nodes
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-nodes: get nodes, list-nodes: list nodes, watch-nodes: watch nodes, create-nodes: create nodes, update-nodes: update nodes
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Elastic #

• Azure AKS Kubelet Proxy to Command Execution Endpoint source high: Detects use of the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet command-execution endpoints (run, exec, attach, portforward, cri). Unlike benign monitoring that scrapes /metrics and /stats, a request to these endpoints executes commands inside a pod on the node, the core of the kubeletctl and Peirates lateral-movement technique. Even a GET to /exec is command execution because the Kubelet maps the WebSocket upgrade handshake to the RBAC get verb, so nodes/proxy GET is sufficient for remote code execution.T1210, T1609↳ also matches create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, patch-nodes-proxy: patch nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS Kubelet Proxy to Command Execution Endpoint#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. This rule keys on the Kubelet endpoint in azure.platformlogs.properties.log.requestURI: /proxy/run, /proxy/exec, /proxy/attach, /proxy/portforward, and /proxy/cri execute commands in a pod on the target node, whereas monitoring uses /metrics and /stats. Any hit here is high-signal command execution regardless of the acting identity, including a stolen monitoring service-account token.

  Possible investigation steps#

  • Confirm the endpoint and target in azure.platformlogs.properties.log.requestURI (the path encodes /<endpoint>/<namespace>/<pod>/<container>) and the acting identity in azure.platformlogs.properties.log.user.username.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP and the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal.
  • Determine what the target pod runs and what secrets or tokens it exposes, and whether the identity should reach the Kubelet at all.
  • Correlate with prior recon (/proxy/pods, /proxy/runningpods) and follow-on RBAC changes, secret reads, or token requests from the same identity.

  False positive analysis#

  • Direct use of Kubelet exec endpoints via nodes/proxy is uncommon; validate any administrative debugging tool that proxies exec and exclude verified operators.

  Response and remediation#

  • Treat as active command execution on a node. Revoke the acting identity's tokens, isolate the affected node and pods, and rotate credentials reachable from them.
  • Review the RBAC that granted nodes/proxy and remove it from workload identities that do not require it.
  • Note that direct Kubelet access on port 10250 bypasses the API server and is not in kube-audit; a confirmed proxy exec may indicate broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Azure AKS API Server Proxying Request to Kubelet source medium: Detects a non-system identity using the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet. Proxying through the API server reaches the Kubelet API to enumerate pods or run commands on nodes, a lateral-movement and privilege-escalation vector (kubeletctl, Peirates). Node, control-plane, and kube-system service account identities that routinely proxy for monitoring are excluded, so remaining matches, including compromised workload service accounts, are surfaced for review.T1210, T1609↳ also matches create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, patch-nodes-proxy: patch nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS API Server Proxying Request to Kubelet#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. The nodes/proxy subresource tunnels a request through the API server to a node's Kubelet. An identity with this permission can enumerate pods (/proxy/pods, /proxy/runningpods) or run commands (/proxy/run, /proxy/exec) on nodes without direct network access to the Kubelet port. This rule excludes the node, control-plane, and kube-system service-account identities that routinely proxy for monitoring, so remaining matches are workload identities or users that should rarely, if ever, reach the Kubelet.

  Possible investigation steps#

  • Identify the acting identity in azure.platformlogs.properties.log.user.username and whether a workload service account or user should reach the Kubelet at all. Compromised pod service accounts (system:serviceaccount:<ns>:<name>) are the primary vehicle for this technique.
  • Inspect the proxied Kubelet endpoint in azure.platformlogs.properties.log.requestURI. /metrics and /stats are monitoring; /pods and /runningpods are reconnaissance; /run, /exec, /attach, and /portforward are command execution and warrant immediate escalation.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP, while the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal. In-cluster pivots show only internal addresses, so absence of an external IP does not clear the event.
  • Review the target node in azure.platformlogs.properties.log.objectRef.name and correlate with subsequent activity on that node's workloads (secret reads, RBAC changes, new pods).

  False positive analysis#

  • Monitoring and log-collection agents outside kube-system proxy to the Kubelet on every scrape and will match repeatedly; add targeted exclusions for verified monitoring service accounts and namespaces.

  Response and remediation#

  • If unauthorized, revoke the acting identity's tokens and review the RBAC that granted nodes/proxy.
  • Inspect the target node for command execution, dropped tooling, or credential theft, and rotate credentials reachable from affected pods.
  • Note that direct Kubelet access on port 10250 bypasses the API server and does not appear in kube-audit; treat a confirmed proxy abuse as possible evidence of broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• GKE API Server Proxying Request to Kubelet source medium: Detects non-system identities using the GKE nodes/proxy API to reach a node's Kubelet through the API server. The nodes/proxy subresource allows any principal with this permission to call the Kubelet API without direct node network access or Kubelet TLS certificates. Through this path an attacker can list pod specs (including environment secrets), read Kubelet configuration, retrieve container logs, and access running pod metadata on the target node. Monitoring endpoints such as metrics, healthz, and stats/summary are excluded to reduce noise from observability tooling.T1550, T1550.001, T1611, T1613↳ also matches create-nodes-proxy: create nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating GKE API Server Proxying Request to Kubelet#

  Review client.user.email, source.ip, and user_agent.original to determine who initiated the proxy request. Examine gcp.audit.resource_name and event.action to identify which Kubelet path was accessed after /proxy/.

  Possible investigation steps#

  • Check the proxied Kubelet path for attacker intent:
    • /proxy/pods — pod spec enumeration, including environment variable secrets
    • /proxy/exec or /proxy/run — command execution inside containers on that node
    • /proxy/configz — Kubelet configuration and authentication settings
    • /proxy/runningpods — active workload enumeration
    • /proxy/containerLogs — log harvesting for leaked credentials
  • Identify how the principal obtained nodes/proxy permission by reviewing RBAC bindings.
  • Correlate with TokenRequest activity from the same actor shortly before the proxy call.
  • Review whether the same principal proxied multiple nodes in a short window.

  False positive analysis#

  • Monitoring agents that scrape paths other than the excluded metrics/health endpoints may match. Add approved paths or identities after baselining.
  • Cluster admin tools that inspect node health via the proxy API can match during maintenance windows.

  Response and remediation#

  • Review and remove unauthorized RBAC granting nodes/proxy.
  • If /proxy/pods was accessed, rotate secrets and credentials that may have been exposed via environment variables on that node.
  • If /proxy/exec or /proxy/run was accessed, treat the node as compromised and isolate it.
  • Restrict nodes/proxy to infrastructure automation only.

Show 1 more (4 total) on the rules page

Elastic #

• Azure AKS Kubelet Proxy to Command Execution Endpoint source high: Detects use of the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet command-execution endpoints (run, exec, attach, portforward, cri). Unlike benign monitoring that scrapes /metrics and /stats, a request to these endpoints executes commands inside a pod on the node, the core of the kubeletctl and Peirates lateral-movement technique. Even a GET to /exec is command execution because the Kubelet maps the WebSocket upgrade handshake to the RBAC get verb, so nodes/proxy GET is sufficient for remote code execution.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, update-nodes-proxy: update nodes/proxy, patch-nodes-proxy: patch nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS Kubelet Proxy to Command Execution Endpoint#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. This rule keys on the Kubelet endpoint in azure.platformlogs.properties.log.requestURI: /proxy/run, /proxy/exec, /proxy/attach, /proxy/portforward, and /proxy/cri execute commands in a pod on the target node, whereas monitoring uses /metrics and /stats. Any hit here is high-signal command execution regardless of the acting identity, including a stolen monitoring service-account token.

  Possible investigation steps#

  • Confirm the endpoint and target in azure.platformlogs.properties.log.requestURI (the path encodes /<endpoint>/<namespace>/<pod>/<container>) and the acting identity in azure.platformlogs.properties.log.user.username.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP and the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal.
  • Determine what the target pod runs and what secrets or tokens it exposes, and whether the identity should reach the Kubelet at all.
  • Correlate with prior recon (/proxy/pods, /proxy/runningpods) and follow-on RBAC changes, secret reads, or token requests from the same identity.

  False positive analysis#

  • Direct use of Kubelet exec endpoints via nodes/proxy is uncommon; validate any administrative debugging tool that proxies exec and exclude verified operators.

  Response and remediation#

  • Treat as active command execution on a node. Revoke the acting identity's tokens, isolate the affected node and pods, and rotate credentials reachable from them.
  • Review the RBAC that granted nodes/proxy and remove it from workload identities that do not require it.
  • Note that direct Kubelet access on port 10250 bypasses the API server and is not in kube-audit; a confirmed proxy exec may indicate broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Azure AKS API Server Proxying Request to Kubelet source medium: Detects a non-system identity using the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet. Proxying through the API server reaches the Kubelet API to enumerate pods or run commands on nodes, a lateral-movement and privilege-escalation vector (kubeletctl, Peirates). Node, control-plane, and kube-system service account identities that routinely proxy for monitoring are excluded, so remaining matches, including compromised workload service accounts, are surfaced for review.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, update-nodes-proxy: update nodes/proxy, patch-nodes-proxy: patch nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS API Server Proxying Request to Kubelet#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. The nodes/proxy subresource tunnels a request through the API server to a node's Kubelet. An identity with this permission can enumerate pods (/proxy/pods, /proxy/runningpods) or run commands (/proxy/run, /proxy/exec) on nodes without direct network access to the Kubelet port. This rule excludes the node, control-plane, and kube-system service-account identities that routinely proxy for monitoring, so remaining matches are workload identities or users that should rarely, if ever, reach the Kubelet.

  Possible investigation steps#

  • Identify the acting identity in azure.platformlogs.properties.log.user.username and whether a workload service account or user should reach the Kubelet at all. Compromised pod service accounts (system:serviceaccount:<ns>:<name>) are the primary vehicle for this technique.
  • Inspect the proxied Kubelet endpoint in azure.platformlogs.properties.log.requestURI. /metrics and /stats are monitoring; /pods and /runningpods are reconnaissance; /run, /exec, /attach, and /portforward are command execution and warrant immediate escalation.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP, while the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal. In-cluster pivots show only internal addresses, so absence of an external IP does not clear the event.
  • Review the target node in azure.platformlogs.properties.log.objectRef.name and correlate with subsequent activity on that node's workloads (secret reads, RBAC changes, new pods).

  False positive analysis#

  • Monitoring and log-collection agents outside kube-system proxy to the Kubelet on every scrape and will match repeatedly; add targeted exclusions for verified monitoring service accounts and namespaces.

  Response and remediation#

  • If unauthorized, revoke the acting identity's tokens and review the RBAC that granted nodes/proxy.
  • Inspect the target node for command execution, dropped tooling, or credential theft, and rotate credentials reachable from affected pods.
  • Note that direct Kubelet access on port 10250 bypasses the API server and does not appear in kube-audit; treat a confirmed proxy abuse as possible evidence of broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• GKE API Server Proxying Request to Kubelet source medium: Detects non-system identities using the GKE nodes/proxy API to reach a node's Kubelet through the API server. The nodes/proxy subresource allows any principal with this permission to call the Kubelet API without direct node network access or Kubelet TLS certificates. Through this path an attacker can list pod specs (including environment secrets), read Kubelet configuration, retrieve container logs, and access running pod metadata on the target node. Monitoring endpoints such as metrics, healthz, and stats/summary are excluded to reduce noise from observability tooling.T1550, T1550.001, T1611, T1613↳ also matches get-nodes-proxy: get nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating GKE API Server Proxying Request to Kubelet#

  Review client.user.email, source.ip, and user_agent.original to determine who initiated the proxy request. Examine gcp.audit.resource_name and event.action to identify which Kubelet path was accessed after /proxy/.

  Possible investigation steps#

  • Check the proxied Kubelet path for attacker intent:
    • /proxy/pods — pod spec enumeration, including environment variable secrets
    • /proxy/exec or /proxy/run — command execution inside containers on that node
    • /proxy/configz — Kubelet configuration and authentication settings
    • /proxy/runningpods — active workload enumeration
    • /proxy/containerLogs — log harvesting for leaked credentials
  • Identify how the principal obtained nodes/proxy permission by reviewing RBAC bindings.
  • Correlate with TokenRequest activity from the same actor shortly before the proxy call.
  • Review whether the same principal proxied multiple nodes in a short window.

  False positive analysis#

  • Monitoring agents that scrape paths other than the excluded metrics/health endpoints may match. Add approved paths or identities after baselining.
  • Cluster admin tools that inspect node health via the proxy API can match during maintenance windows.

  Response and remediation#

  • Review and remove unauthorized RBAC granting nodes/proxy.
  • If /proxy/pods was accessed, rotate secrets and credentials that may have been exposed via environment variables on that node.
  • If /proxy/exec or /proxy/run was accessed, treat the node as compromised and isolate it.
  • Restrict nodes/proxy to infrastructure automation only.

Show 1 more (4 total) on the rules page

Elastic #

• Azure AKS Kubelet Proxy to Command Execution Endpoint source high: Detects use of the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet command-execution endpoints (run, exec, attach, portforward, cri). Unlike benign monitoring that scrapes /metrics and /stats, a request to these endpoints executes commands inside a pod on the node, the core of the kubeletctl and Peirates lateral-movement technique. Even a GET to /exec is command execution because the Kubelet maps the WebSocket upgrade handshake to the RBAC get verb, so nodes/proxy GET is sufficient for remote code execution.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, patch-nodes-proxy: patch nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS Kubelet Proxy to Command Execution Endpoint#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. This rule keys on the Kubelet endpoint in azure.platformlogs.properties.log.requestURI: /proxy/run, /proxy/exec, /proxy/attach, /proxy/portforward, and /proxy/cri execute commands in a pod on the target node, whereas monitoring uses /metrics and /stats. Any hit here is high-signal command execution regardless of the acting identity, including a stolen monitoring service-account token.

  Possible investigation steps#

  • Confirm the endpoint and target in azure.platformlogs.properties.log.requestURI (the path encodes /<endpoint>/<namespace>/<pod>/<container>) and the acting identity in azure.platformlogs.properties.log.user.username.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP and the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal.
  • Determine what the target pod runs and what secrets or tokens it exposes, and whether the identity should reach the Kubelet at all.
  • Correlate with prior recon (/proxy/pods, /proxy/runningpods) and follow-on RBAC changes, secret reads, or token requests from the same identity.

  False positive analysis#

  • Direct use of Kubelet exec endpoints via nodes/proxy is uncommon; validate any administrative debugging tool that proxies exec and exclude verified operators.

  Response and remediation#

  • Treat as active command execution on a node. Revoke the acting identity's tokens, isolate the affected node and pods, and rotate credentials reachable from them.
  • Review the RBAC that granted nodes/proxy and remove it from workload identities that do not require it.
  • Note that direct Kubelet access on port 10250 bypasses the API server and is not in kube-audit; a confirmed proxy exec may indicate broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Azure AKS API Server Proxying Request to Kubelet source medium: Detects a non-system identity using the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet. Proxying through the API server reaches the Kubelet API to enumerate pods or run commands on nodes, a lateral-movement and privilege-escalation vector (kubeletctl, Peirates). Node, control-plane, and kube-system service account identities that routinely proxy for monitoring are excluded, so remaining matches, including compromised workload service accounts, are surfaced for review.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, patch-nodes-proxy: patch nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS API Server Proxying Request to Kubelet#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. The nodes/proxy subresource tunnels a request through the API server to a node's Kubelet. An identity with this permission can enumerate pods (/proxy/pods, /proxy/runningpods) or run commands (/proxy/run, /proxy/exec) on nodes without direct network access to the Kubelet port. This rule excludes the node, control-plane, and kube-system service-account identities that routinely proxy for monitoring, so remaining matches are workload identities or users that should rarely, if ever, reach the Kubelet.

  Possible investigation steps#

  • Identify the acting identity in azure.platformlogs.properties.log.user.username and whether a workload service account or user should reach the Kubelet at all. Compromised pod service accounts (system:serviceaccount:<ns>:<name>) are the primary vehicle for this technique.
  • Inspect the proxied Kubelet endpoint in azure.platformlogs.properties.log.requestURI. /metrics and /stats are monitoring; /pods and /runningpods are reconnaissance; /run, /exec, /attach, and /portforward are command execution and warrant immediate escalation.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP, while the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal. In-cluster pivots show only internal addresses, so absence of an external IP does not clear the event.
  • Review the target node in azure.platformlogs.properties.log.objectRef.name and correlate with subsequent activity on that node's workloads (secret reads, RBAC changes, new pods).

  False positive analysis#

  • Monitoring and log-collection agents outside kube-system proxy to the Kubelet on every scrape and will match repeatedly; add targeted exclusions for verified monitoring service accounts and namespaces.

  Response and remediation#

  • If unauthorized, revoke the acting identity's tokens and review the RBAC that granted nodes/proxy.
  • Inspect the target node for command execution, dropped tooling, or credential theft, and rotate credentials reachable from affected pods.
  • Note that direct Kubelet access on port 10250 bypasses the API server and does not appear in kube-audit; treat a confirmed proxy abuse as possible evidence of broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Kubernetes API Server Proxying Request to Kubelet source medium: Detects non-system identities using the Kubernetes nodes/proxy API to proxy requests through the API server directly to a node's Kubelet. The nodes/proxy subresource allows any principal with this RBAC permission to reach the Kubelet API on any worker node without needing direct network access or Kubelet TLS certificates. Through this proxy path, an attacker can list all pod specifications including environment variable secrets, read Kubelet configuration and PKI material, retrieve container logs, and access running pod metadata across all workloads on the target node. Monitoring and health check endpoints such as /metrics, /healthz, and /stats are excluded to reduce noise from legitimate observability tooling.T1550, T1550.001, T1611, T1613↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, patch-nodes-proxy: patch nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Kubernetes API Server Proxying Request to Kubelet#

  Review the user.name, source.ip, and user_agent.original fields to determine who initiated the proxy request and from where. Examine kubernetes.audit.requestURI to identify which Kubelet endpoint was proxied — the path after /proxy/ maps directly to the Kubelet API path the attacker accessed.

  Possible investigation steps#

  • Check the proxied Kubelet path in requestURI to determine attacker intent:
    • /proxy/pods — pod spec enumeration including environment variable secrets
    • /proxy/exec or /proxy/run — command execution inside containers on that node
    • /proxy/configz — Kubelet configuration and authentication settings
    • /proxy/runningpods — active workload enumeration
    • /proxy/containerLogs — log harvesting for leaked credentials
  • Identify how the principal obtained nodes/proxy permission by reviewing RBAC bindings
  • Check if a ServiceAccount token was created shortly before via the TokenRequest API - this indicates the attacker minted a token specifically for this access.
  • Review whether the same principal accessed multiple nodes via proxy in a short window, which indicates systematic lateral movement across the cluster.

  False positive analysis#

  • Prometheus, Datadog, and other monitoring agents scrape /metrics and /stats via nodes/proxy. These endpoints are excluded by default. If additional monitoring paths generate noise, add them to the requestURI exclusion.
  • Cluster administration tools that inspect node health via the proxy API can match. Correlate with change management windows and verify the source identity.

  Response and remediation#

  • Immediately review the RBAC role granting nodes/proxy permission and determine if the binding is authorized. Remove unauthorized bindings.
  • If /proxy/pods was accessed, assume all environment variable secrets on that node are compromised. Rotate affected credentials, API keys, and database passwords.
  • If /proxy/exec or /proxy/run was accessed, treat the target node as compromised. Isolate the node, review running containers for unauthorized modifications, and check for persistence mechanisms.
  • Audit all ClusterRoles for nodes/proxy permission — this is a powerful privilege that should be restricted to infrastructure automation only, never granted to application service accounts.

Elastic #

• Azure AKS Kubelet Proxy to Command Execution Endpoint source high: Detects use of the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet command-execution endpoints (run, exec, attach, portforward, cri). Unlike benign monitoring that scrapes /metrics and /stats, a request to these endpoints executes commands inside a pod on the node, the core of the kubeletctl and Peirates lateral-movement technique. Even a GET to /exec is command execution because the Kubelet maps the WebSocket upgrade handshake to the RBAC get verb, so nodes/proxy GET is sufficient for remote code execution.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS Kubelet Proxy to Command Execution Endpoint#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. This rule keys on the Kubelet endpoint in azure.platformlogs.properties.log.requestURI: /proxy/run, /proxy/exec, /proxy/attach, /proxy/portforward, and /proxy/cri execute commands in a pod on the target node, whereas monitoring uses /metrics and /stats. Any hit here is high-signal command execution regardless of the acting identity, including a stolen monitoring service-account token.

  Possible investigation steps#

  • Confirm the endpoint and target in azure.platformlogs.properties.log.requestURI (the path encodes /<endpoint>/<namespace>/<pod>/<container>) and the acting identity in azure.platformlogs.properties.log.user.username.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP and the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal.
  • Determine what the target pod runs and what secrets or tokens it exposes, and whether the identity should reach the Kubelet at all.
  • Correlate with prior recon (/proxy/pods, /proxy/runningpods) and follow-on RBAC changes, secret reads, or token requests from the same identity.

  False positive analysis#

  • Direct use of Kubelet exec endpoints via nodes/proxy is uncommon; validate any administrative debugging tool that proxies exec and exclude verified operators.

  Response and remediation#

  • Treat as active command execution on a node. Revoke the acting identity's tokens, isolate the affected node and pods, and rotate credentials reachable from them.
  • Review the RBAC that granted nodes/proxy and remove it from workload identities that do not require it.
  • Note that direct Kubelet access on port 10250 bypasses the API server and is not in kube-audit; a confirmed proxy exec may indicate broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Azure AKS API Server Proxying Request to Kubelet source medium: Detects a non-system identity using the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet. Proxying through the API server reaches the Kubelet API to enumerate pods or run commands on nodes, a lateral-movement and privilege-escalation vector (kubeletctl, Peirates). Node, control-plane, and kube-system service account identities that routinely proxy for monitoring are excluded, so remaining matches, including compromised workload service accounts, are surfaced for review.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS API Server Proxying Request to Kubelet#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. The nodes/proxy subresource tunnels a request through the API server to a node's Kubelet. An identity with this permission can enumerate pods (/proxy/pods, /proxy/runningpods) or run commands (/proxy/run, /proxy/exec) on nodes without direct network access to the Kubelet port. This rule excludes the node, control-plane, and kube-system service-account identities that routinely proxy for monitoring, so remaining matches are workload identities or users that should rarely, if ever, reach the Kubelet.

  Possible investigation steps#

  • Identify the acting identity in azure.platformlogs.properties.log.user.username and whether a workload service account or user should reach the Kubelet at all. Compromised pod service accounts (system:serviceaccount:<ns>:<name>) are the primary vehicle for this technique.
  • Inspect the proxied Kubelet endpoint in azure.platformlogs.properties.log.requestURI. /metrics and /stats are monitoring; /pods and /runningpods are reconnaissance; /run, /exec, /attach, and /portforward are command execution and warrant immediate escalation.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP, while the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal. In-cluster pivots show only internal addresses, so absence of an external IP does not clear the event.
  • Review the target node in azure.platformlogs.properties.log.objectRef.name and correlate with subsequent activity on that node's workloads (secret reads, RBAC changes, new pods).

  False positive analysis#

  • Monitoring and log-collection agents outside kube-system proxy to the Kubelet on every scrape and will match repeatedly; add targeted exclusions for verified monitoring service accounts and namespaces.

  Response and remediation#

  • If unauthorized, revoke the acting identity's tokens and review the RBAC that granted nodes/proxy.
  • Inspect the target node for command execution, dropped tooling, or credential theft, and rotate credentials reachable from affected pods.
  • Note that direct Kubelet access on port 10250 bypasses the API server and does not appear in kube-audit; treat a confirmed proxy abuse as possible evidence of broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Kubernetes API Server Proxying Request to Kubelet source medium: Detects non-system identities using the Kubernetes nodes/proxy API to proxy requests through the API server directly to a node's Kubelet. The nodes/proxy subresource allows any principal with this RBAC permission to reach the Kubelet API on any worker node without needing direct network access or Kubelet TLS certificates. Through this proxy path, an attacker can list all pod specifications including environment variable secrets, read Kubelet configuration and PKI material, retrieve container logs, and access running pod metadata across all workloads on the target node. Monitoring and health check endpoints such as /metrics, /healthz, and /stats are excluded to reduce noise from legitimate observability tooling.T1550, T1550.001, T1611, T1613↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, delete-nodes-proxy: delete nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Kubernetes API Server Proxying Request to Kubelet#

  Review the user.name, source.ip, and user_agent.original fields to determine who initiated the proxy request and from where. Examine kubernetes.audit.requestURI to identify which Kubelet endpoint was proxied — the path after /proxy/ maps directly to the Kubelet API path the attacker accessed.

  Possible investigation steps#

  • Check the proxied Kubelet path in requestURI to determine attacker intent:
    • /proxy/pods — pod spec enumeration including environment variable secrets
    • /proxy/exec or /proxy/run — command execution inside containers on that node
    • /proxy/configz — Kubelet configuration and authentication settings
    • /proxy/runningpods — active workload enumeration
    • /proxy/containerLogs — log harvesting for leaked credentials
  • Identify how the principal obtained nodes/proxy permission by reviewing RBAC bindings
  • Check if a ServiceAccount token was created shortly before via the TokenRequest API - this indicates the attacker minted a token specifically for this access.
  • Review whether the same principal accessed multiple nodes via proxy in a short window, which indicates systematic lateral movement across the cluster.

  False positive analysis#

  • Prometheus, Datadog, and other monitoring agents scrape /metrics and /stats via nodes/proxy. These endpoints are excluded by default. If additional monitoring paths generate noise, add them to the requestURI exclusion.
  • Cluster administration tools that inspect node health via the proxy API can match. Correlate with change management windows and verify the source identity.

  Response and remediation#

  • Immediately review the RBAC role granting nodes/proxy permission and determine if the binding is authorized. Remove unauthorized bindings.
  • If /proxy/pods was accessed, assume all environment variable secrets on that node are compromised. Rotate affected credentials, API keys, and database passwords.
  • If /proxy/exec or /proxy/run was accessed, treat the target node as compromised. Isolate the node, review running containers for unauthorized modifications, and check for persistence mechanisms.
  • Audit all ClusterRoles for nodes/proxy permission — this is a powerful privilege that should be restricted to infrastructure automation only, never granted to application service accounts.

Elastic #

• Azure AKS Kubelet Proxy to Command Execution Endpoint source high: Detects use of the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet command-execution endpoints (run, exec, attach, portforward, cri). Unlike benign monitoring that scrapes /metrics and /stats, a request to these endpoints executes commands inside a pod on the node, the core of the kubeletctl and Peirates lateral-movement technique. Even a GET to /exec is command execution because the Kubelet maps the WebSocket upgrade handshake to the RBAC get verb, so nodes/proxy GET is sufficient for remote code execution.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, patch-nodes-proxy: patch nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS Kubelet Proxy to Command Execution Endpoint#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. This rule keys on the Kubelet endpoint in azure.platformlogs.properties.log.requestURI: /proxy/run, /proxy/exec, /proxy/attach, /proxy/portforward, and /proxy/cri execute commands in a pod on the target node, whereas monitoring uses /metrics and /stats. Any hit here is high-signal command execution regardless of the acting identity, including a stolen monitoring service-account token.

  Possible investigation steps#

  • Confirm the endpoint and target in azure.platformlogs.properties.log.requestURI (the path encodes /<endpoint>/<namespace>/<pod>/<container>) and the acting identity in azure.platformlogs.properties.log.user.username.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP and the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal.
  • Determine what the target pod runs and what secrets or tokens it exposes, and whether the identity should reach the Kubelet at all.
  • Correlate with prior recon (/proxy/pods, /proxy/runningpods) and follow-on RBAC changes, secret reads, or token requests from the same identity.

  False positive analysis#

  • Direct use of Kubelet exec endpoints via nodes/proxy is uncommon; validate any administrative debugging tool that proxies exec and exclude verified operators.

  Response and remediation#

  • Treat as active command execution on a node. Revoke the acting identity's tokens, isolate the affected node and pods, and rotate credentials reachable from them.
  • Review the RBAC that granted nodes/proxy and remove it from workload identities that do not require it.
  • Note that direct Kubelet access on port 10250 bypasses the API server and is not in kube-audit; a confirmed proxy exec may indicate broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Azure AKS API Server Proxying Request to Kubelet source medium: Detects a non-system identity using the AKS (Azure Kubernetes Service) API server nodes/proxy subresource to reach a node's Kubelet. Proxying through the API server reaches the Kubelet API to enumerate pods or run commands on nodes, a lateral-movement and privilege-escalation vector (kubeletctl, Peirates). Node, control-plane, and kube-system service account identities that routinely proxy for monitoring are excluded, so remaining matches, including compromised workload service accounts, are surfaced for review.T1210, T1609↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, patch-nodes-proxy: patch nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Azure AKS API Server Proxying Request to Kubelet#

  AKS kube-audit events are carried under the flattened azure.platformlogs.properties.log.* subtree and share the ARM operation event.action: Microsoft.ContainerService/managedClusters/diagnosticLogs/Read. The nodes/proxy subresource tunnels a request through the API server to a node's Kubelet. An identity with this permission can enumerate pods (/proxy/pods, /proxy/runningpods) or run commands (/proxy/run, /proxy/exec) on nodes without direct network access to the Kubelet port. This rule excludes the node, control-plane, and kube-system service-account identities that routinely proxy for monitoring, so remaining matches are workload identities or users that should rarely, if ever, reach the Kubelet.

  Possible investigation steps#

  • Identify the acting identity in azure.platformlogs.properties.log.user.username and whether a workload service account or user should reach the Kubelet at all. Compromised pod service accounts (system:serviceaccount:<ns>:<name>) are the primary vehicle for this technique.
  • Inspect the proxied Kubelet endpoint in azure.platformlogs.properties.log.requestURI. /metrics and /stats are monitoring; /pods and /runningpods are reconnaissance; /run, /exec, /attach, and /portforward are command execution and warrant immediate escalation.
  • Evaluate azure.platformlogs.properties.log.sourceIPs. This is an array; for an externally operated attack the first element is the operator's real client IP, while the trailing entry is the internal API-server/konnectivity hop (172.31.x). A first entry that is not the cluster's own egress is a strong signal. In-cluster pivots show only internal addresses, so absence of an external IP does not clear the event.
  • Review the target node in azure.platformlogs.properties.log.objectRef.name and correlate with subsequent activity on that node's workloads (secret reads, RBAC changes, new pods).

  False positive analysis#

  • Monitoring and log-collection agents outside kube-system proxy to the Kubelet on every scrape and will match repeatedly; add targeted exclusions for verified monitoring service accounts and namespaces.

  Response and remediation#

  • If unauthorized, revoke the acting identity's tokens and review the RBAC that granted nodes/proxy.
  • Inspect the target node for command execution, dropped tooling, or credential theft, and rotate credentials reachable from affected pods.
  • Note that direct Kubelet access on port 10250 bypasses the API server and does not appear in kube-audit; treat a confirmed proxy abuse as possible evidence of broader Kubelet access.
  • Collect kube-audit and identity artifacts per incident response procedures.
• Kubernetes API Server Proxying Request to Kubelet source medium: Detects non-system identities using the Kubernetes nodes/proxy API to proxy requests through the API server directly to a node's Kubelet. The nodes/proxy subresource allows any principal with this RBAC permission to reach the Kubelet API on any worker node without needing direct network access or Kubelet TLS certificates. Through this proxy path, an attacker can list all pod specifications including environment variable secrets, read Kubelet configuration and PKI material, retrieve container logs, and access running pod metadata across all workloads on the target node. Monitoring and health check endpoints such as /metrics, /healthz, and /stats are excluded to reduce noise from legitimate observability tooling.T1550, T1550.001, T1611, T1613↳ also matches get-nodes-proxy: get nodes/proxy, create-nodes-proxy: create nodes/proxy, update-nodes-proxy: update nodes/proxy, patch-nodes-proxy: patch nodes/proxy
  Investigation guide

  Triage and analysis#

  Investigating Kubernetes API Server Proxying Request to Kubelet#

  Review the user.name, source.ip, and user_agent.original fields to determine who initiated the proxy request and from where. Examine kubernetes.audit.requestURI to identify which Kubelet endpoint was proxied — the path after /proxy/ maps directly to the Kubelet API path the attacker accessed.

  Possible investigation steps#

  • Check the proxied Kubelet path in requestURI to determine attacker intent:
    • /proxy/pods — pod spec enumeration including environment variable secrets
    • /proxy/exec or /proxy/run — command execution inside containers on that node
    • /proxy/configz — Kubelet configuration and authentication settings
    • /proxy/runningpods — active workload enumeration
    • /proxy/containerLogs — log harvesting for leaked credentials
  • Identify how the principal obtained nodes/proxy permission by reviewing RBAC bindings
  • Check if a ServiceAccount token was created shortly before via the TokenRequest API - this indicates the attacker minted a token specifically for this access.
  • Review whether the same principal accessed multiple nodes via proxy in a short window, which indicates systematic lateral movement across the cluster.

  False positive analysis#

  • Prometheus, Datadog, and other monitoring agents scrape /metrics and /stats via nodes/proxy. These endpoints are excluded by default. If additional monitoring paths generate noise, add them to the requestURI exclusion.
  • Cluster administration tools that inspect node health via the proxy API can match. Correlate with change management windows and verify the source identity.

  Response and remediation#

  • Immediately review the RBAC role granting nodes/proxy permission and determine if the binding is authorized. Remove unauthorized bindings.
  • If /proxy/pods was accessed, assume all environment variable secrets on that node are compromised. Rotate affected credentials, API keys, and database passwords.
  • If /proxy/exec or /proxy/run was accessed, treat the target node as compromised. Isolate the node, review running containers for unauthorized modifications, and check for persistence mechanisms.
  • Audit all ClusterRoles for nodes/proxy permission — this is a powerful privilege that should be restricted to infrastructure automation only, never granted to application service accounts.

Elastic #

• Kubernetes Multi-Resource Discovery source medium: Adversaries who land credentials in a cluster—or abuse an over-privileged token—often map the environment before exfiltration or privilege escalation. A practical first pass is to learn where workloads run, how the cluster is partitioned, and what RBAC exists at namespace vs cluster scope. Rapid get/list traffic across distinct API resource kinds that answer those questions (namespaces, workloads, roles, cluster-wide roles) is a common setup and orientation pattern for both interactive attackers and automated recon scripts. It is less typical for steady-state controllers, which usually touch a narrow set of resources repeatedly. This rule highlights that cross-resource burst from a single client fingerprint within a one-minute bucket so analysts can separate routine automation from potential discovery and permission reconnaissance ahead of follow-on actions.T1613
  Investigation guide

  Triage and analysis#

  Investigating Kubernetes Multi-Resource Discovery#

  The rule groups Kubernetes audit get/list events on namespaces, pods, roles, and clusterroles into one-minute windows per user.name, source.ip, user_agent.original, and flags windows where three or more distinct resource types appear. That combination is consistent with someone sketching cluster layout and privilege structure rather than touching a single resource type. Allowed and denied authorizations are both included: failures still signal probing and validate which object types the caller attempted to reach.

  Possible investigation steps#

  • Pivot on Esql.time_interval and the same identity or IP in raw audit logs to see ordering (e.g. namespaces first, then roles, then pods) and whether calls succeeded.
  • Review Esql.decisions and namespaces touched; correlate with RBAC for that identity to see if scope matches a known job or breaks least-privilege expectations.
  • Hunt for follow-on activity: secret/configmap reads, rolebinding changes, pod exec, anonymous or unusual user agents.
  • Baseline automation: CI, GitOps, and some monitoring agents can read several resource types during sync; exclude known service accounts or source networks if noisy.

  False positive analysis#

  • Platform operators or runbooks that reconcile RBAC and workload state may legitimately span these resource types in a short window; tune by user, IP allowlist, or user agent when documented.
  • Some installers briefly query namespaces, pods, and roles during upgrades—correlate with change windows.

  Response and remediation#

  • If malicious, revoke or rotate the implicated credentials, review and tighten RBAC, and inspect for data access or persistence established after the burst.