Using the API

You can start ThreatPipes scans and download the data they produce programatically.

About the API

ThreatPipes includes a basic API to start scans and download completed scan data.

The API is particularly useful if you want to integrate ThreatPipes with other tools in your security stack (SIEMs, IR, etc.).

Authenticating

All request to the API require a valid API key.

You can obtain your API key by logging into ThreatPipes and navigating to your user profile.

Starting scans

You can use the newscan endpoint to start a new scan.

https://IP:PORT/newscan?

The following can be passed to the endpoint as arguments:

  • apikey [Required]: A valid API key for the user.

  • scan_name [Required]: A name to identify the scan.

  • scan_target [Required]: The target you want to scan. Must be a valid target type. Available target types below.

  • scan method [1 Required]:

    • usecase: Use case of scan. Must be a single, valid usecase. OR;

    • mods: Modules to be used for scan. Can be multiple valid module(s). OR;

    • types: Data types you want returned by the scan. Can be multiple valid type(s). OR;

    • cat: Module categories to be used for the scan. Can be multiple valid categories(s). OR;

    • profile: Scan profile slug to be used for the scan. Must be a single, valid scan profile.

  • response [Optional]: For programmatic request, you can request a json success message. Default (if nothing specified) is html.

Scan methods

You must pass one method to start the scan. Either by use case (usecase) OR scan profile (profile) OR module category (cat) OR data types (types) OR specific modules (mods).

You can only pass a single usecase or profile parameter.

You can pass multiple mods or cat or types parameter.

You can view a full list of parameter values in the ThreatPipes UI on the Start Scan page.

You can read more about scan methods, and the options available, in the User Guide here.

Examples

By usecase (usecase)

Must be only one usecase (e.g. passive)

Start a scan for target threatpipes.com with all modules (usecase=all):

https://IP:PORT/newscan?scan_target=threatpipes.com&scan_name=threatpipes_example&usecase=all&api_key=MY_API_KEY

By module (mods)

Must be one or more modules (e.g. tpp_abusech)

Start a scan for target threatpipes.com with tpp_abusech and tpp_accounts modules:

https://IP:PORT/newscan?scan_target=threatpipes.com&scan_name=threatpipes_example&mods=tpp_abusech&mods=tpp_accounts&api_key=MY_API_KEY

By module category (cat)

Must be one or more module category (e.g. Public%20Registries)

Start a scan for target threatpipes.com with modules with categories Public%20Registries and cat=Real%20World

https://IP:PORT/newscan?scan_target=threatpipes.com&scan_name=threatpipes_example&cat=Public%20Registries&cat=Real%20World&api_key=MY_API_KEY

By data type (types)

Must be one or more data element type (e.g. IP_ADDRESS)

Start a scan for target threatpipes.com for data types DOMAIN_NAME and EMAILADDR:

https://IP:PORT/newscan?scan_target=threatpipes.com&scan_name=threatpipes_example&types=EMAILADDR&types=DOMAIN_NAME&api_key=MY_API_KEY

By custom scan profile (profile)

Must be only one scan profile (e.g. my-custom-profile)

Start a scan for target threatpipes.com with my-custom-profile:

https://IP:PORT/newscan?scan_target=threatpipes.com&scan_name=threatpipes_example&profile=my-custom-profile&api_key=MY_API_KEY

Get JSON responses (response)

Optional.

Start a scan for target threatpipes.com with all modules (usecase=all) enabled and provide a response=JSON:

https://IP:PORT/newscan?scan_target=threatpipes.com&scan_name=threatpipes_example&usecase=all&response=json&api_key=MY_API_KEY

Will provide response:

{result:"success", description:"scan started", scanid:"a632b922-15e2-4213-9373-6e436c232290/"}

Downloading scan results

You can use the getscan endpoint to download scan data in a supported data format.

https://IP:PORT/getscan?

The following arguments can be passed to the endpoint:

  • apikey [Required]: A valid API key for the user.

  • scan_id [Required]: ThreatPipes scan ID. Can be found in the UI for the scan.

  • data_type [Required]: CSV, JSON, or GEXF

It is only possible to download scans in finished or aborted state. You will receive an error if you try to download data from a running scan.

scan_id

You can get a scan ID in two ways:

  • in ThreatPipes web by navigating to: Scans > Scan > Scan Settings

  • in the response when starting a scan using the /newscan endpoint with the response=json

data_type

Currently supports following data formats:

  • JSON (json)

  • CSV (csv)

  • GEXF (gexf)

Successful responses will be printed in one of the following formats available

json

http://IP:PORT/getscan?scan_id=SCANID&data_type=json&api_key=YOUR_API_KEY

csv

http://IP:PORT/getscan?scan_id=SCANID&data_type=csv&api_key=YOUR_API_KEY

gexf

http://IP:PORT/getscan?scan_id=SCANID&data_type=gexf&api_key=YOUR_API_KEY

Errors

If your scan does not start, you will get an error informing you of the reason in an error message. For example:

{"scan_id": "62ab2f5a-6db9-4c7c-9d38-13d3cc6b1022", "result": "error", "description": "Invalid API key"}