spiderfoot
SpiderFoot automates OSINT for threat intelligence and mapping your attack surface.
An open-source OSINT automation tool that takes a starting point like a domain, IP, or email and queries 200+ data sources to map what information about it is publicly exposed on the internet.
SpiderFoot is an open-source OSINT (open-source intelligence) automation tool — OSINT means gathering information from publicly available sources. It is designed to help security professionals and organizations understand what information about them is exposed on the internet, or to gather intelligence on a target during a penetration test or security assessment.
The way it works is by taking a starting point — which can be an IP address, domain name, email address, phone number, username, person's name, Bitcoin address, or network range — and then running it through over 200 modules that each query different data sources. These modules feed their results back into each other in a chain: discovering a domain might reveal email addresses, which then get checked in data breach databases, which might reveal associated usernames, which then get searched on social media platforms. The tool extracts hosts, sub-domains, email addresses, social media accounts, files exposed in cloud storage buckets, threat intelligence blacklist hits, and more.
It provides both a web-based interface (launched by running a local web server) and a command-line mode. Results can be exported in CSV, JSON, or GEXF formats and are stored in a local SQLite database for custom querying. A YAML-based correlation engine lets you define rules to automatically detect patterns of interest across the collected data.
It is written in Python 3 and licensed under MIT. You would use it when you need to map out your organization's internet footprint, investigate potential threats, or gather reconnaissance data during authorized security testing. The full README is longer than what was provided.
Where it fits
- Map your organization's internet footprint to find exposed subdomains, email addresses, and cloud storage buckets before attackers do.
- Run authorized penetration test reconnaissance to gather intelligence on a target domain, IP address, or email.
- Detect if your company's credentials appear in data breach databases by starting a scan from a known email address.
- Build custom correlation rules in YAML to automatically flag suspicious patterns across collected intelligence data.