ThreatPipes can be used offensively, i.e. as part of a black-box penetration test to gather information about the target or defensively to identify what information your organisation is freely providing for attackers to use against you.
In either case, ThreatPipes works the same way for both attacker and defender.
All data collection by scans in ThreatPipes modularised.
Each module typically queries a third-party service for intelligence on a target (e.g. haveibeenpwned).
When a module discovers a piece of data, that data is transmitted to all other modules that are watching for that data type for processing. Those modules will then act on that piece of data to identify new data, and in turn generate new events for other modules which may be interested, and so on.
Visually it looks like this:
The red node is the seed target and black nodes are intelligence generated from the target by modules.
By building up a complete threat graph, you discover both directly related intelligence on the target (original threat) but also any affiliates to that potential threat.
Sometimes it's easier to explain this concept with a worked example, so here's one for you.
The DNS Resolver module identifies an IP address associated with a target, notifying all interested modules watching for IP related events.
One of those interested modules is the RIPE module, that takes the IP address and identifies the netblock it is a part of, the BGP ASN and so on.
Meanwhile, as each event is generated to a module, it is also recorded in the ThreatPipes database for reporting and viewing.
Each module is classified into one or more of the following use cases:
footprint: Understand what information this target exposes to the Internet.
investigate: Best for when you suspect the target to be malicious but need more information.
passive: When you don't want the target to even suspect they are being investigated.
Modules might have multiple classifications.
For example, the abuse.ch module checks if a host/domain, IP or netblock is malicious according to abuse.ch is classified into two use cases; "investigate" and "passive".
All modules are classified into categories that describe the type of service they relate to (e.g. Reputation Systems, DNS, Public Registries).
For example, the Whois module that searches Whois servers for domain names and netblocks is classified into the category; "Public Registries".
ThreatPipes modules are driven by events. Each module listen to and consumes data elements to generate new events (intelligence) about these data elements, which in turn other modules listen to and consume, and so on.
For example, the Name Extractor module listens and consumers the data element types
EMAILADDR events for identifying human names (
HUMAN_NAME data elements). In turn, the Accounts module, and many others, listen for
HUMAN_NAME data elements to gather more intelligence.
Data element types
The data elements are classified into one of the following types:
entities: e.g. IP addresses, hostnames, sub-domains, domains
sub-entities: e.g. port numbers, URLs, software installed
descriptors: (of entities) e.g. malicious, physical location information
data: (of entities) e.g. web page content, port banners, raw DNS records
To help explain the differences between data elements, here an example; the data element "Email Address" is an entity, the data element "Hacked Email Address" is a descriptor, and the "Search Engine's Web Content" that might have uncovered the email is "data".
ThreatPipes ships with over 150 default modules . It is also possible to install custom modules developed by third parties to work with ThreatPipes.