CronJobs

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches list-cronjobs: list cronjobs, watch-cronjobs: watch cronjobs, create-cronjobs: create cronjobs, update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-cronjobs: get cronjobs, watch-cronjobs: watch cronjobs, create-cronjobs: create cronjobs, update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-cronjobs: get cronjobs, list-cronjobs: list cronjobs, create-cronjobs: create cronjobs, update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Sigma #

• Kubernetes CronJob/Job Modification source medium: Detects when a Kubernetes CronJob or Job is created or modified. A Kubernetes Job creates one or more pods to accomplish a specific task, and a CronJob creates Jobs on a recurring schedule. An adversary can take advantage of this Kubernetes object to schedule Jobs to run containers that execute malicious code within a cluster, allowing them to achieve persistence.↳ also matches update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs, delete-cronjobs: delete cronjobs

Elastic #

• GKE Unusual Sensitive Workload Modification source low: Detects the first occurrence of create or patch activity against sensitive GKE workloads (DaemonSets, Deployments, or CronJobs) from an unusual combination of user agent, source IP, and user identity, which may indicate privilege escalation or unauthorized access within the cluster.T1098, T1098.006↳ also matches patch-cronjobs: patch cronjobs
  Investigation guide

  Triage and analysis#

  Investigating GKE Unusual Sensitive Workload Modification#

  This new-terms rule alerts on the first create or patch of a DaemonSet, Deployment, or CronJob from a new combination of user_agent.original, source.ip, and client.user.email.

  Possible investigation steps#

  • Review the audit request for image, command, service account, and privileged settings changes.
  • Attribute the actor to its backing identity and validate whether the source network is expected.
  • Correlate with RBAC, secret, or exec activity from the same identity.

  False positive analysis#

  • Legitimate on-call changes from new workstations or updated kubectl versions are common in lab clusters.

  Response and remediation#

  • Roll back unauthorized workload changes, revoke the credential used, and tighten RBAC on workload controllers.
• Unusual Kubernetes Sensitive Workload Modification source low: Detects the creation or modification of several sensitive workloads, such as DaemonSets, Deployments, or CronJobs, by an unusual user agent, source IP and username, which may indicate privilege escalation or unauthorized access within the cluster.T1098, T1098.006↳ also matches patch-cronjobs: patch cronjobs
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Unusual Kubernetes Sensitive Workload Modification#

  This rule detects allowed create or patch activity against sensitive Kubernetes workloads (DaemonSets, Deployments, CronJobs) coming from an unusual combination of client, network origin, and user identity, which can signal stolen credentials, privilege escalation, or unauthorized control of cluster execution. Attackers commonly patch an existing Deployment to inject a new container or init container that runs with elevated privileges and pulls a remote payload, then rely on the workload controller to redeploy it across the environment.

  Possible investigation steps#

  • Retrieve the full audit event for the change and compare it to the most recent prior modification of the same workload to identify what was altered (e.g., image, command/args, env/secret refs, volumes, serviceAccount, securityContext, hostPath/hostNetwork, privileged settings).
  • Attribute the action to a real identity by tracing the Kubernetes user to its backing cloud/IAM identity or kubeconfig/cert and validate whether the access path (SSO, token, service account, CI/CD runner) and source network location are expected for that operator.
  • Determine blast radius by listing other recent creates/patches by the same identity and from the same origin across namespaces, and check for follow-on actions such as creating RBAC bindings, secrets, or additional controllers.
  • Inspect the affected workload’s rollout status and pod specs to confirm whether new pods were created, then review container images, pull registries, and runtime behavior for indicators of compromise (unexpected network egress, crypto-mining, credential access, or exec activity).
  • Validate the change against an approved deployment workflow by correlating with GitOps/CI commit history and change tickets, and if unapproved, contain by scaling down/rolling back the workload and revoking the credential or token used.

  False positive analysis#

  • A legitimate on-call engineer performs an emergency kubectl create/patch to a Deployment/CronJob/DaemonSet from a new workstation, VPN egress IP, or updated kubectl version, producing an unusual user_agent/source IP/username combination despite being authorized.
  • A routine automation path changes (e.g., CI runner or service account rotated/migrated to a new node pool or network segment) and continues applying standard workload updates, causing the same create/patch activity to appear anomalous due to the new origin and client identity.

  Response and remediation#

  • Immediately pause impact by scaling the modified Deployment/CronJob to zero or deleting the new DaemonSet and stopping any active rollout while preserving the altered manifest for evidence.
  • Roll back the workload to the last known-good version from GitOps/CI or prior ReplicaSet/Job template, then redeploy only after verifying container images, init containers, commands, serviceAccount, and privileged/host settings match the approved baseline.
  • Revoke and rotate the credential used for the change (user token/cert or service account token), invalidate related kubeconfigs, and review/remove any newly created RBAC bindings, secrets, or service accounts tied to the same actor.
  • Quarantine affected nodes and pods for analysis by cordoning/draining nodes that ran the new pods and collecting pod logs, container filesystem snapshots, and network egress details to identify payloads and persistence.
  • Escalate to the incident response/on-call security team immediately if the change introduced privileged containers, hostPath mounts, hostNetwork, new external images/registries, or any unexpected DaemonSet creation across multiple nodes.
  • Harden by enforcing admission controls to restrict privileged settings and sensitive namespaces, requiring changes via approved automation identities, and tightening RBAC so only designated deployment controllers can create/patch DaemonSets, Deployments, and CronJobs.
• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-cronjobs: get cronjobs, list-cronjobs: list cronjobs, watch-cronjobs: watch cronjobs, update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Show 2 more (5 total) on the rules page

Splunk #

• Kubernetes Cron Job Creation source: The following analytic detects the creation of a Kubernetes cron job, which is a task scheduled to run automatically at specified intervals. It identifies this activity by monitoring Kubernetes Audit logs for the creation events of cron…T1053, T1053.007

Panther #

• Kubernetes CronJob Created or Modified source informational: This detection monitors for creation or modification of CronJobs in Kubernetes clusters. Attackers may create or modify scheduled jobs to achieve cluster persistence, execute malicious code on a schedule, or maintain backdoor access to compromised clusters. This detection works across AWS EKS, Azure AKS, and GCP GKE clusters.T1053.003↳ also matches update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs
• Kubernetes Cron Job Created or Modified source medium linked query: Kubernetes Cron Job Created or Modified: This detection monitor for any modifications or creations of a cron job. Attackers may create or modify an existing scheduled job in order to achieve cluster persistence.↳ also matches update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs
• Kubernetes Cron Job Created or Modified source: This detection monitor for any modifications or creations of a cron job. Attackers may create or modify an existing scheduled job in order to achieve cluster persistence.↳ also matches update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs

Sigma #

• Kubernetes CronJob/Job Modification source medium: Detects when a Kubernetes CronJob or Job is created or modified. A Kubernetes Job creates one or more pods to accomplish a specific task, and a CronJob creates Jobs on a recurring schedule. An adversary can take advantage of this Kubernetes object to schedule Jobs to run containers that execute malicious code within a cluster, allowing them to achieve persistence.↳ also matches create-cronjobs: create cronjobs, patch-cronjobs: patch cronjobs, delete-cronjobs: delete cronjobs

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-cronjobs: get cronjobs, list-cronjobs: list cronjobs, watch-cronjobs: watch cronjobs, create-cronjobs: create cronjobs, patch-cronjobs: patch cronjobs
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.

Panther #

• Kubernetes CronJob Created or Modified source informational: This detection monitors for creation or modification of CronJobs in Kubernetes clusters. Attackers may create or modify scheduled jobs to achieve cluster persistence, execute malicious code on a schedule, or maintain backdoor access to compromised clusters. This detection works across AWS EKS, Azure AKS, and GCP GKE clusters.T1053.003↳ also matches create-cronjobs: create cronjobs, patch-cronjobs: patch cronjobs
• Kubernetes Cron Job Created or Modified source medium linked query: Kubernetes Cron Job Created or Modified: This detection monitor for any modifications or creations of a cron job. Attackers may create or modify an existing scheduled job in order to achieve cluster persistence.↳ also matches create-cronjobs: create cronjobs, patch-cronjobs: patch cronjobs
• Kubernetes Cron Job Created or Modified source: This detection monitor for any modifications or creations of a cron job. Attackers may create or modify an existing scheduled job in order to achieve cluster persistence.↳ also matches create-cronjobs: create cronjobs, patch-cronjobs: patch cronjobs

Sigma #

• Kubernetes CronJob/Job Modification source medium: Detects when a Kubernetes CronJob or Job is created or modified. A Kubernetes Job creates one or more pods to accomplish a specific task, and a CronJob creates Jobs on a recurring schedule. An adversary can take advantage of this Kubernetes object to schedule Jobs to run containers that execute malicious code within a cluster, allowing them to achieve persistence.↳ also matches create-cronjobs: create cronjobs, update-cronjobs: update cronjobs, delete-cronjobs: delete cronjobs

Elastic #

• Direct Interactive Kubernetes API Request by Unusual Utilities source low: This rule leverages a combination of Defend for Containers and Kubernetes audit logs to detect the execution of direct interactive Kubernetes API requests via unusual utilities. An adversary may need to execute direct interactive Kubernetes API requests to gain access to the Kubernetes API server or other resources within the cluster. These requests are often used to enumerate the Kubernetes API server or other resources within the cluster, and may indicate an attempt to move laterally within the cluster.T1059, T1059.004, T1069, T1087, T1609, T1610↳ also matches get-cronjobs: get cronjobs, list-cronjobs: list cronjobs, watch-cronjobs: watch cronjobs, create-cronjobs: create cronjobs, update-cronjobs: update cronjobs
  Investigation guide

  Triage and analysis#

  Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

  Investigating Direct Interactive Kubernetes API Request by Unusual Utilities#

  This rule detects interactive commands executed inside containers that use atypical utilities to hit the Kubernetes API, paired with near-simultaneous API activity on pods, secrets, service accounts, roles/bindings, or pod exec/attach/log/portforward. It surfaces hands-on-keyboard discovery and lateral movement using custom scripts that evade common tool allowlists; for example, an intruder opens a shell in a pod, uses Python to query the in-cluster API to list secrets, then triggers pods/exec to pivot into another workload.

  Possible investigation steps#

  • Identify the implicated pod, container image, and executing service account, then quickly review its RBAC bindings and effective permissions to determine blast radius.
  • Inspect the container’s interactive session context by pulling recent command lines, shell history, environment variables, and mounted service account tokens, and look for custom scripts or binaries issuing HTTP requests.
  • Correlate nearby Kubernetes audit entries tied to the same principal and pod to map accessed resources and verbs, noting any exec/attach/portforward or sensitive object interactions across namespaces.
  • Review network activity from the pod to the API server and any in-pod proxies, including DNS lookups and outbound connections, to spot nonstandard clients or tunneling behavior.
  • If suspicious, isolate the pod or node, capture runtime artifacts (e.g., process memory or HTTP client traffic), revoke and rotate the service account credentials, and verify image provenance and integrity.

  False positive analysis#

  • An operator interactively attaches to a pod and uses a Python REPL or bash with /dev/tcp to call the in-cluster API for routine troubleshooting (e.g., list pods, read ConfigMaps, or run selfsubjectaccessreviews), producing normal audit entries that match the rule signature.
  • A correlation artifact arises when two namespaces have pods with the same name: one pod starts an interactive shell while another independently performs get/list/watch calls, and the 1-second sequence keyed only on pod-name links the unrelated events.

  Response and remediation#

  • Immediately isolate the implicated pod that issued direct API calls using a nonstandard utility by applying a deny-all egress NetworkPolicy in its namespace (including to kubernetes.default.svc:443), terminating the interactive session, and scaling its owning Deployment/Job/StatefulSet to zero replicas.
  • Before teardown, capture a runtime snapshot of the container and node including the binary or script used to query the API (e.g., files under /tmp or /dev/tcp usage), shell history, environment, and the mounted service account token and CA bundle at /var/run/secrets/kubernetes.io/serviceaccount/.
  • Revoke access by removing the service account’s RoleBindings/ClusterRoleBindings, deleting all pods that mount that service account to force token rotation, rotating any Secrets and ConfigMaps that were read or created during the window, and deleting any unauthorized Jobs, CronJobs, or Deployments created by the same principal.
  • Restore workloads from a known-good image digest, re-enable the Deployment only after image scan and integrity checks pass, and monitor subsequent Kubernetes audit logs for pods/exec, portforward, and access to secrets across the affected namespaces.
  • Escalate to incident response leadership and consider cluster-wide containment if audit logs show create/patch of ClusterRoleBindings, access to secrets outside the workload’s namespace, or use of pods/exec to pivot into other nodes or system namespaces such as kube-system.
  • Harden access by enforcing least-privilege RBAC that denies pods/exec and attach for application service accounts, setting automountServiceAccountToken: false on workloads that do not need it, restricting egress to the API server with NetworkPolicies, and requiring just-in-time break-glass roles for interactive access.
• GKE Sensitive RBAC Change Followed by Workload Modification source medium: Detects when the same GKE identity creates or modifies a Role or ClusterRole with high-risk permissions (wildcard access, RBAC escalation verbs, or access to secrets / privileged APIs) and also creates or patches a DaemonSet, Deployment, or CronJob within five minutes. This correlation is consistent with RBAC-based privilege escalation followed by payload deployment.T1098, T1098.006↳ also matches create-cronjobs: create cronjobs
  Investigation guide

  Triage and analysis#

  Investigating GKE Sensitive RBAC Change Followed by Workload Modification#

  This ES|QL rule correlates two successful GKE audit behaviors from the same client.user.email within five minutes:

  1. Role or ClusterRole create/update/patch that grants high-risk permissions (wildcards, escalate / bind / impersonate, secret read, or privileged API resources such as pods/exec and serviceaccounts/token)
  2. DaemonSet, Deployment, or CronJob create or patch after the sensitive RBAC change

  Esql.rbac_to_workload_minutes is the gap from the latest sensitive RBAC event to the earliest workload modification in the lookback window.

  Possible investigation steps#

  • Review Esql.event_action_values and Esql.gcp_audit_resource_name_values for the Role/ClusterRole and workload objects touched.
  • Inspect Esql.user_agent_original_values and Esql.source_ip_values for unexpected clients or networks.
  • Check for RoleBinding or ClusterRoleBinding activity around the same identity and time window.
  • Correlate with secret access, pod exec, or token creation from the same actor.

  False positive analysis#

  • GitOps pipelines that manage both RBAC manifests and workloads in one sync cycle.
  • Platform bootstrap that patches built-in roles and reconciles addon workloads.

  Response and remediation#

  • Roll back unauthorized Role/ClusterRole and workload changes, revoke the actor's credentials, and tighten who can mutate RBAC and sensitive workloads.
• GKE Unusual Sensitive Workload Modification source low: Detects the first occurrence of create or patch activity against sensitive GKE workloads (DaemonSets, Deployments, or CronJobs) from an unusual combination of user agent, source IP, and user identity, which may indicate privilege escalation or unauthorized access within the cluster.T1098, T1098.006↳ also matches create-cronjobs: create cronjobs
  Investigation guide

  Triage and analysis#

  Investigating GKE Unusual Sensitive Workload Modification#

  This new-terms rule alerts on the first create or patch of a DaemonSet, Deployment, or CronJob from a new combination of user_agent.original, source.ip, and client.user.email.

  Possible investigation steps#

  • Review the audit request for image, command, service account, and privileged settings changes.
  • Attribute the actor to its backing identity and validate whether the source network is expected.
  • Correlate with RBAC, secret, or exec activity from the same identity.

  False positive analysis#

  • Legitimate on-call changes from new workstations or updated kubectl versions are common in lab clusters.

  Response and remediation#

  • Roll back unauthorized workload changes, revoke the credential used, and tighten RBAC on workload controllers.

Show 2 more (5 total) on the rules page

Panther #

• Kubernetes CronJob Created or Modified source informational: This detection monitors for creation or modification of CronJobs in Kubernetes clusters. Attackers may create or modify scheduled jobs to achieve cluster persistence, execute malicious code on a schedule, or maintain backdoor access to compromised clusters. This detection works across AWS EKS, Azure AKS, and GCP GKE clusters.T1053.003↳ also matches create-cronjobs: create cronjobs, update-cronjobs: update cronjobs
• Kubernetes Cron Job Created or Modified source medium linked query: Kubernetes Cron Job Created or Modified: This detection monitor for any modifications or creations of a cron job. Attackers may create or modify an existing scheduled job in order to achieve cluster persistence.↳ also matches create-cronjobs: create cronjobs, update-cronjobs: update cronjobs
• Kubernetes Cron Job Created or Modified source: This detection monitor for any modifications or creations of a cron job. Attackers may create or modify an existing scheduled job in order to achieve cluster persistence.↳ also matches create-cronjobs: create cronjobs, update-cronjobs: update cronjobs

Sigma #

• Kubernetes CronJob/Job Modification source medium: Detects when a Kubernetes CronJob or Job is created or modified. A Kubernetes Job creates one or more pods to accomplish a specific task, and a CronJob creates Jobs on a recurring schedule. An adversary can take advantage of this Kubernetes object to schedule Jobs to run containers that execute malicious code within a cluster, allowing them to achieve persistence.↳ also matches create-cronjobs: create cronjobs, update-cronjobs: update cronjobs, patch-cronjobs: patch cronjobs