ThreatPipes will show the intelligence gathered in near real-time meaning you don't have to wait for the scan to complete before analysing the results.
From the moment you click ‘Run Scan’, you will be taken to a screen for monitoring your scan.
At this point you can explore the dashboards for the scan to analyse the results as they're generated.
The starting point for any analysis. Get a quick overview of what the scan has currently returned, and drill-down onto anything that looks of interest.
Easily see the relationships between data elements uncovered by a scan
Analyse the individual data element types uncovered by the scan.
Drill down to view individual data elements for detailed analysis when something interesting has been found.
You can use the search box to perform a search using a regular expression.
Note on search
The search function will search using filters already applied. For example, if in a specific data element type view, the search will only consider that data element.
You can search as follows:
Exact value: Non-wildcard searching for a specific value. For example, search for 404 within the HTTP Status Code section to see all pages that were not found.
Pattern matching: Search for simple wildcards to find patterns. For example, search for *:22 within the Open TCP Port section to see all instances of port 22 open.
Regular expression searches: Encapsulate your string in ‘/’ to search by regular expression. For example, search for ‘/\d+.\d+.\d+.\d+/’ to find anything looking like an IP address in your scan results.
The fields displayed are explained as follows:
Checkbox field: Use this to set/unset fields as false positive. Once at least one is checked, click the orange False Positive button above to set/unset the record.
Risky results: flags results identified as high risk
Data Element: The data the module was able to obtain about your target.
Source Data Element: The data the module received as the basis for its data collection.
Source Module: The module that identified this data.
Identified: When the data was identified by the module.
Note on false positives
Records can only be set to false positive once a scan has finished running. This is because setting a record to false positive also results in all child data elements being set to false positive. This obviously cannot be done if the scan is still running and can thus lead to an inconsistent state in the back-end. The UI will prevent you from doing so.
The result of a record being set to false positive, aside from the indicator in the data table view and exports, is that such data will not be shown in the node graphs.
Note on graphs
You can also visualise the results in graphical formal. The following graphs can be used to visualise data elements:
Bubble (by data element)
Bubble (by source data element)
If the scan is part of an investigation, or you want to add it to an investigation you can navigate to the scans investigation tab and selecting the investigation you want to add or remove the investigation from.
A scan can trigger one or more correlation rule.
You can view a list of correlation rules under: Settings > Global settings.
By drilling down on the triggered correlation rule, you can see the data elements uncovered by the scan that caused the rule to trigger.
Details all the specifics of the scan, including the settings used to start it, the global ThreatPipes configuration, and the module configurations at the time the scan started.
This view is particularly useful for analysing scans others have executed to understand how they were setup and for times when global settings were different to those currently configured (e.g. TOR enabled / disabled).
Useful for troubleshooting when errors occurs or when you want to see how the scan is progressing in near real-time.
Use the dropdown log type filter and plaintext search bar at the top of the page to filter the logs for the scan.
The Explore tab (and graph) allows you to perform lookups on data elements for manual scans.
You can download data produced by a ThreatPipes scan to analyse using other tools.
Results can be downloaded in the following formats:
To download all scan data in the UI:
navigate to the scan list view
select the scan you want to download using the checkboxes
click the download button and select your chosen data format.
On the scan browse view you can download certain data elements in CSV format.
By selecting the download button, you can download the data elements included in the current view.
You can also select one or more data elements using the checkboxes and selecting the download button to download only those data elements in CSV format.
It is possible to download both a .
.gexf version of scan results.
.png download option downloads an image file of the currently rendered graph view, under the graph tab. Download the image by clicking the picture icon.
.gexf download will provide a .gexf file of the scan network graph you can upload to other tools for further analysis.