Using Unified Basic Threat Intel Datasets with Observe

Observe has created the following resource sets from these Indicators of Compromise (IOC) types from the main Threat Resource Lists.

Use these unified lists with your data fields that suit each IOC value type, and use standardized returned field names.

Using Each Unified Dataset

Unified Hash Threatlist

Threat Intel IOC file hash values can be useful when matching against data sources that provide the hash of suspicious files.

You need a log dataset that includes processes executed or downloaded by your systems, such as Endpoint Detection and Response (EDR) or Deep Packet Inspection (DPI) logs. The Observe Host Monitoring integration can provide this data.

The Process Hash Search dashboard uses this dataset.

Unified Hosts-Domains Threatlist

Threat Intel IOC suspicious domains can be matched against data sources that provide a domain such as email addresses, web activity logs, etc.

Note

  • Observe strongly recommends enabling the Majestic Million poller.

  • This Dataset filters against the Majestic Million resource set, the top 50K domains of the resource set, to reduce match noise against well known domains.

You need a logs dataset that includes host interactions by your systems, such as network or proxy logs. The Observe Host Monitoring or AWS integration can use also provide this data.

Datasets that include domains or URLs visited by your systems, such as proxy server or EDR logs can also be useful.

Unified IPv4 Threatlist

IPv4 addresses provide a very common data value such as the source of login activity. Matching against Threat Intel IOC lists of IPs can help identify suspicious activity by IPv4 address alone without additional context can result in alert fatigue.

You need a logs dataset that includes IPv4 addresses interacted with by your systems, such as network or proxy logs. Observe’s Host Monitoring or AWS integration can use this data.

The IP Search and IPv6 Search dashboards use this dataset.

Unified URL Threatlist

Threat Intel with known bad URLs can be useful for matching against an organization’s Web access activity data. This data can also contain the host value from the URL extracted and compared to Host-Domain IOC lists.

You need a logs dataset that includes domains or URLs visited by your systems, such as proxy server or EDR logs.

Commonly Returned Fields

Using standard returned field names can make downstream processing simpler and provide common references for Monitor alerts.

  • tip_match - The matched value in the Unified table

  • tip_provider - The original resource name.

  • tip_provider_id - If present, the UID of the IOC in the original resource data

  • tip_source - If present, the source of the IOC per the original resource data or the original resource provider name.

  • tip_severity - If present, assigned severity terms. Defaults to informational if no severity is provided.

  • tip_tags - If present, contextual tags from the original resource data.

  • tip_tlp - All public resources are assigned a TLP of clear by default.

  • tip_category - If present, a contextual field from the original resource data for the IOC value.

An Example Join Using OPAL

// extract host from url
extract_regex url, /\/(?P<url_host>[^\/:]+)[:\/]/

join on(url_host=@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_domain),
  tip_match:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_domain,
  tip_provider:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_provider,
  tip_provider_id:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_provider_url,
  tip_severity:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_severity,
  tip_tags:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_tags,
  tip_source:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_source,
  tip_tlp:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_tlp,
  tip_category:@"Threat Intel Basic/Unified Hosts-Domains Threatlist".tip_category,
  tip_match_field:"url_host"

Traffic Light Protocol (TLP)

By default, Observe matches the free public IOCs to the clear sharing rating per the CISA TLP guide. Assigning TLP ratings to IOCs can assist with how you handle matches within your organization.

Traffic Light Protocol Clear Designation

Figure 1 - Traffic Light Protocol Designation

Severity Scale

Observe defaults the free public IOCs to informational severity as they do not provide a severity value. If handling a 0-10 scale, Observe recommends that you map them to the terms.

value

severity

0-1

informational

2-3

low

4-6

medium

7-9

high

10

critical