TI Map URL Entity to Syslog Data

Severity: medium
Time window: 14d
Group by: IndicatorId, Url
Source: github.com/Azure/Azure-Sentinel

'This query identifies any URL indicators of compromise (IOCs) from threat intelligence (TI) by searching for matches in Syslog data.'

MITRE ATT&CK coverage

Tactic	Techniques
Command & Control	`T1071` Application Layer Protocol

Rule body

id: b31037ea-6f68-4fbd-bab2-d0d0f44c2fcf
name: TI Map URL Entity to Syslog Data
description: |
  'This query identifies any URL indicators of compromise (IOCs) from threat intelligence (TI) by searching for matches in Syslog data.'
severity: Medium
requiredDataConnectors:
  - connectorId: Syslog
    dataTypes:
      - Syslog
  - connectorId: ThreatIntelligence
    dataTypes:
      - ThreatIntelligenceIndicator
  - connectorId: ThreatIntelligenceTaxii
    dataTypes:
      - ThreatIntelligenceIndicator
  - connectorId: MicrosoftDefenderThreatIntelligence
    dataTypes:
      - ThreatIntelligenceIndicator
queryFrequency: 1h
queryPeriod: 14d
triggerOperator: gt
triggerThreshold: 0
tactics:
  - CommandAndControl
relevantTechniques:
  - T1071
query: |
  let dt_lookBack = 1h;
  let ioc_lookBack = 14d;
  ThreatIntelligenceIndicator
  // Picking up only IOC's that contain the entities we want
  | where isnotempty(Url)
  | where TimeGenerated >= ago(ioc_lookBack)
  | summarize LatestIndicatorTime = arg_max(TimeGenerated, *) by IndicatorId
  | where Active == true and ExpirationDateTime > now()
  // using innerunique to keep perf fast and result set low, we only need one match to indicate potential malicious activity that needs to be investigated
  | join kind=innerunique (
    Syslog
    | where TimeGenerated >= ago(dt_lookBack)
    // Extract URL from the Syslog message but only take messages that include URLs
    | extend Url = extract("(http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\\(\\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+)", 1,SyslogMessage)
    | where isnotempty(Url)
    | extend Syslog_TimeGenerated = TimeGenerated
  ) on Url
  | where Syslog_TimeGenerated < ExpirationDateTime
  | summarize Syslog_TimeGenerated  = arg_max(Syslog_TimeGenerated , *) by IndicatorId, Url
  | project timestamp = Syslog_TimeGenerated, Description, ActivityGroupNames, IndicatorId, ThreatType, ExpirationDateTime, ConfidenceScore, SyslogMessage, Computer, ProcessName, Url, HostIP
entityMappings:
  - entityType: Host
    fieldMappings:
      - identifier: HostName
        columnName: Computer
  - entityType: IP
    fieldMappings:
      - identifier: Address
        columnName: HostIP
  - entityType: URL
    fieldMappings:
      - identifier: Url
        columnName: Url
version: 1.2.6
kind: Scheduled

Stages and Predicates

Parameters

let dt_lookBack = 1h;
let ioc_lookBack = 14d;

Stage 1: `source`

ThreatIntelligenceIndicator

Stage 2: `where`

| where isnotempty(Url)

Stage 3: `where`

| where TimeGenerated >= ago(ioc_lookBack)

Stage 4: `summarize`

| summarize LatestIndicatorTime = arg_max(TimeGenerated, *) by IndicatorId

Stage 5: `where`

| where Active == true and ExpirationDateTime > now()

Stage 6: `join`

| join kind=innerunique (
  Syslog
  | where TimeGenerated >= ago(dt_lookBack)
  | extend Url = extract("(http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\\(\\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+)", 1,SyslogMessage)
  | where isnotempty(Url)
  | extend Syslog_TimeGenerated = TimeGenerated
) on Url

Stage 7: `where`

| where Syslog_TimeGenerated < ExpirationDateTime

Stage 8: `summarize`

| summarize Syslog_TimeGenerated  = arg_max(Syslog_TimeGenerated , *) by IndicatorId, Url

Stage 9: `project`

| project timestamp = Syslog_TimeGenerated, Description, ActivityGroupNames, IndicatorId, ThreatType, ExpirationDateTime, ConfidenceScore, SyslogMessage, Computer, ProcessName, Url, HostIP

Indicators

These rows show field, operator, and value matches.

Field	Kind	Values	Search
`Active`	eq	`true`	`field:"Active" kind:eq value:"true"`
`Syslog_TimeGenerated`	cross_field_compare	`ExpirationDateTime` transforms: `op:lt`	`field:"Syslog_TimeGenerated" kind:cross_field_compare value:"ExpirationDateTime"`
`Url`	is_not_null	(no value, null check)	`field:"Url" kind:is_not_null`

Output fields

These fields are emitted when the rule matches.

Field	Source
`ActivityGroupNames`	`project`
`Computer`	`project`
`ConfidenceScore`	`project`
`Description`	`project`
`ExpirationDateTime`	`project`
`HostIP`	`project`
`IndicatorId`	`project`
`ProcessName`	`project`
`SyslogMessage`	`project`
`ThreatType`	`project`
`Url`	`project`
`timestamp`	`project`

TI Map URL Entity to Syslog Data

MITRE ATT&CK coverage

Rule body

Stages and Predicates

Parameters

Stage 1: source

Stage 2: where

Stage 3: where

Stage 4: summarize

Stage 5: where

Stage 6: join

Stage 7: where

Stage 8: summarize

Stage 9: project