Detection rules › Kusto

GreyNoise TI Map IP Entity to CommonSecurityLog

Severity
medium
Time window
14d
Group by
CS_ipEntity, IndicatorId, TI_ipEntity
Source
github.com/Azure/Azure-Sentinel

This query maps GreyNoise IP indicators of compromise (IOCs) from threat intelligence (TI), by searching for matches in CommonSecurityLog.

MITRE ATT&CK coverage

TacticTechniques
Command & ControlT1071 Application Layer Protocol

Rule body kusto

id: e50657d7-8bca-43ff-a647-d407fae440d6
name: GreyNoise TI Map IP Entity to CommonSecurityLog
description: |
  This query maps GreyNoise IP indicators of compromise (IOCs) from threat intelligence (TI), by searching for matches in CommonSecurityLog.
severity: Medium
requiredDataConnectors:
  - connectorId: ThreatIntelligence
    dataTypes:
      - ThreatIntelligenceIndicator
  - connectorId: CEF
    dataTypes:
      - CommonSecurityLog
  - connectorId: CefAma
    dataTypes:
      - CommonSecurityLog
  - connectorId: GreyNoise2SentinelAPI
    dataTypes:
      - ThreatIntelligenceIndicator
queryFrequency: 4h
queryPeriod: 14d
triggerOperator: gt
triggerThreshold: 0
tactics:
  - CommandAndControl
relevantTechniques:
  - T1071
query: |
  let IPRegex = '[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}';
  let dt_lookBack = 1h; // Look back 1 hour for CommonSecurityLog events
  let ioc_lookBack = 14d; // Look back 14 days for threat intelligence indicators
  // Fetch threat intelligence indicators related to IP addresses
  let IP_Indicators = ThreatIntelligenceIndicator
    | where TimeGenerated >= ago(ioc_lookBack)
    | summarize LatestIndicatorTime = arg_max(TimeGenerated, *) by IndicatorId
    | where Active == true and ExpirationDateTime > now()
    | where SourceSystem == 'GreyNoise'
    | where isnotempty(NetworkIP) or isnotempty(EmailSourceIpAddress) or isnotempty(NetworkDestinationIP) or isnotempty(NetworkSourceIP)
    | extend TI_ipEntity = iff(isnotempty(NetworkIP), NetworkIP, NetworkDestinationIP)
    | extend TI_ipEntity = iff(isempty(TI_ipEntity) and isnotempty(NetworkSourceIP), NetworkSourceIP, TI_ipEntity)
    | extend TI_ipEntity = iff(isempty(TI_ipEntity) and isnotempty(EmailSourceIpAddress), EmailSourceIpAddress, TI_ipEntity)
    | where ipv4_is_private(TI_ipEntity) == false and  TI_ipEntity !startswith "fe80" and TI_ipEntity !startswith "::" and TI_ipEntity !startswith "127.";
  // Perform a join between IP indicators and CommonSecurityLog events
  IP_Indicators
    // Use innerunique to keep performance fast and result set low, as we only need one match to indicate potential malicious activity that needs investigation
    | join kind=innerunique (
        CommonSecurityLog
        | where TimeGenerated >= ago(dt_lookBack)
        | extend MessageIP = extract(IPRegex, 0, Message)
        | extend CS_ipEntity = iff(isnotempty(SourceIP), SourceIP, DestinationIP)
        | extend CS_ipEntity = iff(isempty(CS_ipEntity) and isnotempty(MessageIP), MessageIP, CS_ipEntity)
        | extend CommonSecurityLog_TimeGenerated = TimeGenerated
    )
    on $left.TI_ipEntity == $right.CS_ipEntity
    // Filter out logs that occurred after the expiration of the corresponding indicator
    | where CommonSecurityLog_TimeGenerated < ExpirationDateTime
    // Group the results by IndicatorId and CS_ipEntity, and keep the log entry with the latest timestamp
    | summarize CommonSecurityLog_TimeGenerated = arg_max(CommonSecurityLog_TimeGenerated, *) by IndicatorId, CS_ipEntity
    // Select the desired output fields
    | project timestamp = CommonSecurityLog_TimeGenerated, SourceIP, DestinationIP, MessageIP, Message, DeviceVendor, DeviceProduct, IndicatorId, ThreatType, ExpirationDateTime, ConfidenceScore, TI_ipEntity, CS_ipEntity, LogSeverity, DeviceAction, Type
entityMappings:
  - entityType: IP
    fieldMappings:
      - identifier: Address
        columnName: CS_ipEntity
version: 1.0.2
kind: Scheduled

Stages and Predicates

Parameters

let IPRegex = '[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}';
let dt_lookBack = 1h;
let ioc_lookBack = 14d;

The stages below define let IP_Indicators (the rule's main pipeline source).

Stage 1: source

ThreatIntelligenceIndicator

Stage 2: where

| where TimeGenerated >= ago(ioc_lookBack)

Stage 3: summarize

| summarize LatestIndicatorTime = arg_max(TimeGenerated, *) by IndicatorId

Stage 4: where

| where Active == true and ExpirationDateTime > now()

Stage 5: where

| where SourceSystem == 'GreyNoise'

Stage 6: where

| where isnotempty(NetworkIP) or isnotempty(EmailSourceIpAddress) or isnotempty(NetworkDestinationIP) or isnotempty(NetworkSourceIP)

Stage 7: extend (3 consecutive steps)

| extend TI_ipEntity = iff(isnotempty(NetworkIP), NetworkIP, NetworkDestinationIP)
| extend TI_ipEntity = iff(isempty(TI_ipEntity) and isnotempty(NetworkSourceIP), NetworkSourceIP, TI_ipEntity)
| extend TI_ipEntity = iff(isempty(TI_ipEntity) and isnotempty(EmailSourceIpAddress), EmailSourceIpAddress, TI_ipEntity)
TI_ipEntity =
ifisnotempty(NetworkIP)NetworkIP
elseNetworkDestinationIP

Stage 8: where

| where ipv4_is_private(TI_ipEntity) == false and  TI_ipEntity !startswith "fe80" and TI_ipEntity !startswith "::" and TI_ipEntity !startswith "127."

The stages below run on IP_Indicators (the outer pipeline).

Stage 9: join

IP_Indicators
| join kind=innerunique (
      CommonSecurityLog
      | where TimeGenerated >= ago(dt_lookBack)
      | extend MessageIP = extract(IPRegex, 0, Message)
      | extend CS_ipEntity = iff(isnotempty(SourceIP), SourceIP, DestinationIP)
      | extend CS_ipEntity = iff(isempty(CS_ipEntity) and isnotempty(MessageIP), MessageIP, CS_ipEntity)
      | extend CommonSecurityLog_TimeGenerated = TimeGenerated
  )
  on $left.TI_ipEntity == $right.CS_ipEntity

Stage 10: where

| where CommonSecurityLog_TimeGenerated < ExpirationDateTime

Stage 11: summarize

| summarize CommonSecurityLog_TimeGenerated = arg_max(CommonSecurityLog_TimeGenerated, *) by IndicatorId, CS_ipEntity

Stage 12: project

| project timestamp = CommonSecurityLog_TimeGenerated, SourceIP, DestinationIP, MessageIP, Message, DeviceVendor, DeviceProduct, IndicatorId, ThreatType, ExpirationDateTime, ConfidenceScore, TI_ipEntity, CS_ipEntity, LogSeverity, DeviceAction, Type

Exclusions

Top-level NOT(...) conjuncts: predicates this rule actively suppresses.

FieldKindExcluded values
TI_ipEntitycidr_match10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, 127.0.0.0/8
TI_ipEntitystarts_with127.
TI_ipEntitystarts_with::
TI_ipEntitystarts_withfe80

Indicators

Each row is a field, operator, and value that the rule matches. The corpus column counts how many other rules in the catalog look for the same combination: high numbers point to widely-used, community-vetted indicators. Blank or 1 shows that the indicator is specific to this rule.

FieldKindValues
Activeeq
  • true transforms: cased
CommonSecurityLog_TimeGeneratedlt
  • ExpirationDateTime transforms: cased
EmailSourceIpAddressis_not_null
  • (no value, null check)
NetworkDestinationIPis_not_null
  • (no value, null check)
NetworkIPis_not_null
  • (no value, null check)
NetworkSourceIPis_not_null
  • (no value, null check)
SourceSystemeq
  • GreyNoise transforms: cased

Output fields

Fields the rule emits when it matches. Chronicle authors list these in the outcome block; they appear on the detection and $risk_score drives alerting. Sentinel / Defender XDR rules build them up through project / summarize / extend stages. Sentinel maps these into alert fields via entityMappings and customDetails; Defender XDR custom detections surface them as alert fields directly.

FieldSource
CS_ipEntityproject
ConfidenceScoreproject
DestinationIPproject
DeviceActionproject
DeviceProductproject
DeviceVendorproject
ExpirationDateTimeproject
IndicatorIdproject
LogSeverityproject
Messageproject
MessageIPproject
SourceIPproject
TI_ipEntityproject
ThreatTypeproject
Typeproject
timestampproject