Detection rules › Kusto

Anomaly found in Network Session Traffic (ASIM Network Session schema)

Status
available
Severity
medium
Time window
14d
Source
github.com/Azure/Azure-Sentinel

'The rule identifies anomalous pattern in network session traffic based on previously seen data, different Device Action, Network Protocol, Network Direction or overall volume. The rule utilize ASIM normalization, and is applied to any source which supports the ASIM Network Session schema. This rule leverages log summaries generated by a Summary Rule or Summarized Playbook. If no such summaries are available, the rule falls back to direct analysis using ASIM function.'

MITRE ATT&CK coverage

Rule body kusto

id: cd6def0d-3ef0-4d55-a7e3-faa96c46ba12
name: Anomaly found in Network Session Traffic (ASIM Network Session schema)
description: |
  'The rule identifies anomalous pattern in network session traffic based on previously seen data, different Device Action, Network Protocol, Network Direction or overall volume. The rule utilize [ASIM](https://aka.ms/AboutASIM) normalization, and is applied to any source which supports the ASIM Network Session schema. This rule leverages log summaries generated by a Summary Rule or Summarized Playbook. If no such summaries are available, the rule falls back to direct analysis using ASIM function.'
severity: Medium
status: Available 
tags:
  - Schema: ASimNetworkSessions
    SchemaVersion: 0.2.4
requiredDataConnectors:
  - connectorId: AWSS3
    dataTypes:
      - AWSVPCFlow
  - connectorId: MicrosoftThreatProtection
    dataTypes:
      - DeviceNetworkEvents
  - connectorId: SecurityEvents
    dataTypes:
      - SecurityEvent
  - connectorId: WindowsSecurityEvents
    dataTypes:
      - SecurityEvent
  - connectorId: WindowsForwardedEvents
    dataTypes:
      - WindowsEvent
  - connectorId: Zscaler
    dataTypes:
      - CommonSecurityLog
  - connectorId: MicrosoftSysmonForLinux
    dataTypes:
      - Syslog
  - connectorId: PaloAltoNetworks
    dataTypes:
      - CommonSecurityLog
  - connectorId: AzureMonitor(VMInsights)
    dataTypes:
      - VMConnection
  - connectorId: AzureFirewall
    dataTypes:
      - AzureDiagnostics
  - connectorId: AzureNSG
    dataTypes:
      - AzureDiagnostics
  - connectorId: CiscoASA
    dataTypes:
      - CommonSecurityLog
  - connectorId: CiscoAsaAma
    dataTypes:
      - CommonSecurityLog
  - connectorId: Corelight
    dataTypes:
      - Corelight_CL
  - connectorId: AIVectraStream
    dataTypes:
      - VectraStream
  - connectorId: CheckPoint
    dataTypes:
      - CommonSecurityLog
  - connectorId: Fortinet
    dataTypes:
      - CommonSecurityLog
  - connectorId: CiscoMeraki
    dataTypes:
      - Syslog
      - CiscoMerakiNativePoller

queryFrequency: 1d
queryPeriod: 14d
triggerOperator: gt
triggerThreshold: 0
tactics:
  - CommandAndControl
  - Discovery
  - Exfiltration
  - LateralMovement
relevantTechniques:
  - T1095
  - T1071
  - T1046
  - T1030
  - T1210
query: |
  let min_t = ago(14d);
  let max_t = now();
  let dt = 1d;
  let fieldForDvcAction = "DvcAction";
  let fieldForNetworkDirection = "NetworkDirection";
  let fieldForNetworkProtocol = "NetworkProtocol";
  let AnomalyThreshold = 2.5;
  let eps = materialize (_Im_NetworkSession
      | project TimeGenerated
      | where TimeGenerated > ago(5m)
      | count
      | extend Count = Count / 300);
  let maxSummarizedTime = toscalar (
      union isfuzzy=true
          (
          NetworkSummary_Protocol_CL
          | where EventTime > min_t
          | summarize max_TimeGenerated=max(EventTime)
          | extend max_TimeGenerated = datetime_add('minute', 20, max_TimeGenerated)
          ),
          (
          NetworkCustomAnalytics_protocol_CL
          | where EventTime_t > min_t
          | summarize max_TimeGenerated=max(EventTime_t)
          | extend max_TimeGenerated = datetime_add('minute', 20, max_TimeGenerated)
          ),
          (
          print(min_t)
          | project max_TimeGenerated = print_0
          )
      | summarize maxTimeGenerated = max(max_TimeGenerated) 
      );
  let NetworkSummary_Protocol = materialize(
      union isfuzzy=true 
          (
          NetworkSummary_Protocol_CL
          | where EventTime > ago(1d)
          | project v = int(2)
          ),
          (
          print int(1) 
          | project v = print_0
          )
      | summarize maxv = max(v)
      | extend
          NetworkSummary_Protocol_Exists = (maxv > 1),
          NetworkCustomAnalyticsExists = false
      );
  let NetworkCustomAnalytics = materialize(
      union isfuzzy=true 
          (
          NetworkCustomAnalytics_protocol_CL
          | where EventTime_t > ago(1d) 
          | project v = int(2)
          ),
          (
          print int(1) 
          | project v = print_0
          )
      | summarize maxv = max(v)
      | extend
          NetworkCustomAnalyticsExists = (maxv > 1),
          NetworkSummary_Protocol_Exists = false
      );
  let allData = union isfuzzy=true 
          (
          (datatable(
      exists: int,
      NetworkSummary_Protocol_Exists: bool,
      NetworkCustomAnalyticsExists: bool
  )[
      1, false, false
  ]
          | where toscalar(eps) > 500
          | join (NetworkSummary_Protocol) on NetworkSummary_Protocol_Exists
          | join (NetworkCustomAnalytics) on NetworkCustomAnalyticsExists)
          | join (
              _Im_NetworkSession(starttime=todatetime(ago(3d)), endtime=now())
              | where TimeGenerated > maxSummarizedTime
              | summarize Count=count()
                  by
                  NetworkProtocol,
                  DstPortNumber,
                  DstAppName,
                  NetworkDirection,
                  DvcAction,
                  bin(TimeGenerated, 10m)
              | extend
                  EventTime = TimeGenerated,
                  Count = toint(Count),
                  DstPortNumber = toint(DstPortNumber),
                  exists=int(1)
              )
              on exists
          | project-away exists*, maxv, NetworkSummary_Protocol_Exists*, NetworkCustomAnalyticsExists*
          ),
          (
          (datatable(
      exists: int,
      NetworkSummary_Protocol_Exists: bool,
      NetworkCustomAnalyticsExists: bool
  )[
      1, false, false
  ]
          | where toscalar(eps) <= 500
          | join (NetworkSummary_Protocol) on NetworkSummary_Protocol_Exists
          | join (NetworkCustomAnalytics) on NetworkCustomAnalyticsExists
          )
          | join (
              _Im_NetworkSession(starttime=todatetime(ago(4d)), endtime=now())
              | where TimeGenerated > maxSummarizedTime
              | summarize Count=count()
                  by
                  NetworkProtocol,
                  DstPortNumber,
                  DstAppName,
                  NetworkDirection,
                  DvcAction,
                  bin(TimeGenerated, 10m)
              | extend
                  EventTime = TimeGenerated,
                  Count = toint(Count),
                  DstPortNumber = toint(DstPortNumber),
                  exists=int(1)
              )
              on exists
          | project-away exists*, maxv, NetworkSummary_Protocol_Exists*, NetworkCustomAnalyticsExists*
          ),
          (
          (datatable(exists: int, NetworkSummary_Protocol_Exists: bool)[1, false]
          | join (NetworkSummary_Protocol) on NetworkSummary_Protocol_Exists
          )
          | join (
              NetworkCustomAnalytics_protocol_CL
              | where EventTime_t > min_t
              | project
                  NetworkProtocol=NetworkProtocol_s,
                  DstPortNumber=DstPortNumber_d,
                  DstAppName=DstAppName_s,
                  NetworkDirection=NetworkDirection_s,
                  DvcAction=DvcAction_s,
                  Count=count__d,
                  EventTime=EventTime_t,
                  TimeGenerated,
                  Type
              | extend Count = toint(Count), DstPortNumber = toint(DstPortNumber), exists=int(1)
              )
              on exists
          | project-away exists*, maxv, NetworkSummary_Protocol_Exists*, NetworkCustomAnalyticsExists*
          ),
          (
          NetworkSummary_Protocol_CL
          | where EventTime > min_t
          | project
              NetworkProtocol,
              DstPortNumber,
              DstAppName,
              NetworkDirection,
              DvcAction,
              Count=count_,
              EventTime,
              TimeGenerated,
              Type
          | extend Count = toint(Count), DstPortNumber = toint(DstPortNumber)
          )
  ;
  let findVolumneBasedAnomaly = allData
      | make-series total=sum(Count) on EventTime from min_t to max_t step dt
      | extend (anomalies, score, baseline) = series_decompose_anomalies(total, AnomalyThreshold, -1, 'linefit') 
      | mv-expand anomalies, score, baseline, EventTime, total
      | extend
          anomalies = toint(anomalies),
          score = toint(score),
          baseline = toint(baseline),
          EventTime = todatetime(EventTime),
          total = tolong(total)
      | where EventTime >= ago(1d)
      | where score >= 2 * AnomalyThreshold
  ;
  let findAnomalies = (field: string) {
      allData
      | where isnotempty(column_ifexists(field, ""))
      | make-series total=sum(Count) on EventTime from min_t to max_t step dt by column_ifexists(field, "")
      | extend (anomalies, score, baseline) = series_decompose_anomalies(total, AnomalyThreshold, -1, 'linefit')
      | mv-expand anomalies, score, baseline, EventTime, total
      | extend
          anomalies = toint(anomalies),
          score = toint(score),
          baseline = toint(baseline),
          EventTime = todatetime(EventTime),
          total = tolong(total)
      | where EventTime >= ago(1d)
      | where score >= 2 * AnomalyThreshold
  };
  union
      findAnomalies(fieldForDvcAction),
      findAnomalies(fieldForNetworkDirection),
      findAnomalies(fieldForNetworkProtocol),
      findVolumneBasedAnomaly
  | extend
      anomalyFieldType  = case (
                          isnotempty(column_ifexists(fieldForDvcAction, "")),
                          "DvcAction",
                          isnotempty(column_ifexists(fieldForNetworkDirection, "")),
                          "NetworkDirection",
                          isnotempty(column_ifexists(fieldForNetworkProtocol, "")),
                          "NetworkProtocol",
                          "TotalVolume"
                      ),
      anomalyFieldValue  = case (
                          isnotempty(column_ifexists(fieldForDvcAction, "")),
                          column_ifexists(fieldForDvcAction, ""),
                          isnotempty(column_ifexists(fieldForNetworkDirection, "")),
                          column_ifexists(fieldForNetworkDirection, ""),
                          isnotempty(column_ifexists(fieldForNetworkProtocol, "")),
                          column_ifexists(fieldForNetworkProtocol, ""),
                          "Overall"
                      )
eventGroupingSettings:
  aggregationKind: AlertPerResult
customDetails:
  AnomalyFieldType: anomalyFieldType
  AnomalyFieldValue: anomalyFieldValue
  Score: score

alertDetailsOverride:
  alertDisplayNameFormat: Anomaly was observed with {{anomalyFieldValue}} Traffic
  alertDescriptionFormat: 'Based on past data, anomaly was observed in {{anomalyFieldValue}} Traffic with a score of {{score}}.'
version: 1.3.0
kind: Scheduled

Stages and Predicates

Parameters

let min_t = ago(14d);
let max_t = now();
let dt = 1d;
let fieldForDvcAction = "DvcAction";
let fieldForNetworkDirection = "NetworkDirection";
let fieldForNetworkProtocol = "NetworkProtocol";
let AnomalyThreshold = 2.5;

Let binding: eps

let eps = materialize (_Im_NetworkSession
    | project TimeGenerated
    | where TimeGenerated > ago(5m)
    | count
    | extend Count = Count / 300);

Let binding: maxSummarizedTime

let maxSummarizedTime = toscalar (
    union isfuzzy=true
        (
        NetworkSummary_Protocol_CL
        | where EventTime > min_t
        | summarize max_TimeGenerated=max(EventTime)
        | extend max_TimeGenerated = datetime_add('minute', 20, max_TimeGenerated)
        ),
        (
        NetworkCustomAnalytics_protocol_CL
        | where EventTime_t > min_t
        | summarize max_TimeGenerated=max(EventTime_t)
        | extend max_TimeGenerated = datetime_add('minute', 20, max_TimeGenerated)
        ),
        (
        print(min_t)
        | project max_TimeGenerated = print_0
        )
    | summarize maxTimeGenerated = max(max_TimeGenerated) 
    );

Derived from min_t.

Let binding: NetworkSummary_Protocol

let NetworkSummary_Protocol = materialize(
    union isfuzzy=true 
        (
        NetworkSummary_Protocol_CL
        | where EventTime > ago(1d)
        | project v = int(2)
        ),
        (
        print int(1) 
        | project v = print_0
        )
    | summarize maxv = max(v)
    | extend
        NetworkSummary_Protocol_Exists = (maxv > 1),
        NetworkCustomAnalyticsExists = false
    );

Let binding: NetworkCustomAnalytics

let NetworkCustomAnalytics = materialize(
    union isfuzzy=true 
        (
        NetworkCustomAnalytics_protocol_CL
        | where EventTime_t > ago(1d) 
        | project v = int(2)
        ),
        (
        print int(1) 
        | project v = print_0
        )
    | summarize maxv = max(v)
    | extend
        NetworkCustomAnalyticsExists = (maxv > 1),
        NetworkSummary_Protocol_Exists = false
    );

Let binding: allData

let allData = union isfuzzy=true 
        (
        (datatable(
    exists: int,
    NetworkSummary_Protocol_Exists: bool,
    NetworkCustomAnalyticsExists: bool
)[
    1, false, false
]
        | where toscalar(eps) > 500
        | join (NetworkSummary_Protocol) on NetworkSummary_Protocol_Exists
        | join (NetworkCustomAnalytics) on NetworkCustomAnalyticsExists)
        | join (
            _Im_NetworkSession(starttime=todatetime(ago(3d)), endtime=now())
            | where TimeGenerated > maxSummarizedTime
            | summarize Count=count()
                by
                NetworkProtocol,
                DstPortNumber,
                DstAppName,
                NetworkDirection,
                DvcAction,
                bin(TimeGenerated, 10m)
            | extend
                EventTime = TimeGenerated,
                Count = toint(Count),
                DstPortNumber = toint(DstPortNumber),
                exists=int(1)
            )
            on exists
        | project-away exists*, maxv, NetworkSummary_Protocol_Exists*, NetworkCustomAnalyticsExists*
        ),
        (
        (datatable(
    exists: int,
    NetworkSummary_Protocol_Exists: bool,
    NetworkCustomAnalyticsExists: bool
)[
    1, false, false
]
        | where toscalar(eps) <= 500
        | join (NetworkSummary_Protocol) on NetworkSummary_Protocol_Exists
        | join (NetworkCustomAnalytics) on NetworkCustomAnalyticsExists
        )
        | join (
            _Im_NetworkSession(starttime=todatetime(ago(4d)), endtime=now())
            | where TimeGenerated > maxSummarizedTime
            | summarize Count=count()
                by
                NetworkProtocol,
                DstPortNumber,
                DstAppName,
                NetworkDirection,
                DvcAction,
                bin(TimeGenerated, 10m)
            | extend
                EventTime = TimeGenerated,
                Count = toint(Count),
                DstPortNumber = toint(DstPortNumber),
                exists=int(1)
            )
            on exists
        | project-away exists*, maxv, NetworkSummary_Protocol_Exists*, NetworkCustomAnalyticsExists*
        ),
        (
        (datatable(exists: int, NetworkSummary_Protocol_Exists: bool)[1, false]
        | join (NetworkSummary_Protocol) on NetworkSummary_Protocol_Exists
        )
        | join (
            NetworkCustomAnalytics_protocol_CL
            | where EventTime_t > min_t
            | project
                NetworkProtocol=NetworkProtocol_s,
                DstPortNumber=DstPortNumber_d,
                DstAppName=DstAppName_s,
                NetworkDirection=NetworkDirection_s,
                DvcAction=DvcAction_s,
                Count=count__d,
                EventTime=EventTime_t,
                TimeGenerated,
                Type
            | extend Count = toint(Count), DstPortNumber = toint(DstPortNumber), exists=int(1)
            )
            on exists
        | project-away exists*, maxv, NetworkSummary_Protocol_Exists*, NetworkCustomAnalyticsExists*
        ),
        (
        NetworkSummary_Protocol_CL
        | where EventTime > min_t
        | project
            NetworkProtocol,
            DstPortNumber,
            DstAppName,
            NetworkDirection,
            DvcAction,
            Count=count_,
            EventTime,
            TimeGenerated,
            Type
        | extend Count = toint(Count), DstPortNumber = toint(DstPortNumber)
        );

Derived from min_t, eps, maxSummarizedTime, NetworkSummary_Protocol, NetworkCustomAnalytics.

Let binding: findVolumneBasedAnomaly

let findVolumneBasedAnomaly = allData
    | make-series total=sum(Count) on EventTime from min_t to max_t step dt
    | extend (anomalies, score, baseline) = series_decompose_anomalies(total, AnomalyThreshold, -1, 'linefit') 
    | mv-expand anomalies, score, baseline, EventTime, total
    | extend
        anomalies = toint(anomalies),
        score = toint(score),
        baseline = toint(baseline),
        EventTime = todatetime(EventTime),
        total = tolong(total)
    | where EventTime >= ago(1d)
    | where score >= 2 * AnomalyThreshold;

Derived from min_t, max_t, dt, AnomalyThreshold, allData.

Let binding: findAnomalies

let findAnomalies = (field: string) {
    allData
    | where isnotempty(column_ifexists(field, ""))
    | make-series total=sum(Count) on EventTime from min_t to max_t step dt by column_ifexists(field, "")
    | extend (anomalies, score, baseline) = series_decompose_anomalies(total, AnomalyThreshold, -1, 'linefit')
    | mv-expand anomalies, score, baseline, EventTime, total
    | extend
        anomalies = toint(anomalies),
        score = toint(score),
        baseline = toint(baseline),
        EventTime = todatetime(EventTime),
        total = tolong(total)
    | where EventTime >= ago(1d)
    | where score >= 2 * AnomalyThreshold
};

Derived from min_t, max_t, dt, AnomalyThreshold, allData.

union (4 sources)

Each leg below queries one source; the rule matches if any leg does. Sources: findAnomalies(fieldForDvcAction), findAnomalies(fieldForNetworkDirection), findAnomalies(fieldForNetworkProtocol), findVolumneBasedAnomaly

Leg 1: findAnomalies(fieldForDvcAction)

Leg 2: findAnomalies(fieldForNetworkDirection)

Leg 3: findAnomalies(fieldForNetworkProtocol)

Leg 4: findVolumneBasedAnomaly

Applied to the combined result

| extend
    anomalyFieldType  = case (
                        isnotempty(column_ifexists(fieldForDvcAction, "")),
                        "DvcAction",
                        isnotempty(column_ifexists(fieldForNetworkDirection, "")),
                        "NetworkDirection",
                        isnotempty(column_ifexists(fieldForNetworkProtocol, "")),
                        "NetworkProtocol",
                        "TotalVolume"
                    ),
    anomalyFieldValue  = case (
                        isnotempty(column_ifexists(fieldForDvcAction, "")),
                        column_ifexists(fieldForDvcAction, ""),
                        isnotempty(column_ifexists(fieldForNetworkDirection, "")),
                        column_ifexists(fieldForNetworkDirection, ""),
                        isnotempty(column_ifexists(fieldForNetworkProtocol, "")),
                        column_ifexists(fieldForNetworkProtocol, ""),
                        "Overall"
                    )

Indicators

Each row is a field, operator, and value that the rule matches. The corpus column counts how many other rules in the catalog look for the same combination: high numbers point to widely-used, community-vetted indicators. Blank or 1 shows that the indicator is specific to this rule.

FieldKindValues
EventTimegt
  • min_t transforms: cased
EventTime_tgt
  • min_t transforms: cased
TimeGeneratedgt
  • maxSummarizedTime transforms: cased
scorege
  • 5 transforms: cased

Output fields

Fields the rule emits when it matches. Chronicle authors list these in the outcome block; they appear on the detection and $risk_score drives alerting. Sentinel / Defender XDR rules build them up through project / summarize / extend stages. Sentinel maps these into alert fields via entityMappings and customDetails; Defender XDR custom detections surface them as alert fields directly.

FieldSource
totalextend
anomaliesextend
baselineextend
scoreextend
EventTimeextend
anomalyFieldTypeextend
anomalyFieldValueextend