2021-02-15 01:27:18 +00:00
# Open-source Intelligence (OSINT)
2018-01-18 03:45:47 +00:00
Open-source intelligence (OSINT) is data collected from open source and publicly available sources. The following are a few OSINT resources and references:
2020-05-22 05:45:22 +00:00
## Passive Recon Tools:
2020-08-27 00:09:25 +00:00
- [AMass ](https://github.com/OWASP/Amass )
2020-07-29 01:17:17 +00:00
- [Exiftool ](https://www.sno.phy.queensu.ca/~phil/exiftool/ )
- [ExtractMetadata ](http://www.extractmetadata.com )
- [Findsubdomains ](https://findsubdomains.com/ )
- [FOCA ](https://elevenpaths.com )
- [IntelTechniques ](https://inteltechniques.com )
2018-08-07 20:33:50 +00:00
- [Maltego ](https://www.paterva.com/web7/ )
2019-11-14 18:05:20 +00:00
- [Recon-NG ](https://github.com/lanmaster53/recon-ng )
2018-08-07 20:33:50 +00:00
- [Scrapy ](https://scrapy.org )
- [Screaming Frog ](https://www.screamingfrog.co.uk )
2020-07-29 01:17:17 +00:00
- [Shodan ](https://shodan.io )
- [SpiderFoot ](http://spiderfoot.net )
- [theHarvester ](https://github.com/laramies/theHarvester )
- [Visual SEO Studio ](https://visual-seo.com/ )
2018-08-07 20:40:04 +00:00
- [Web Data Extractor ](http://www.webextractor.com )
2020-07-29 01:17:17 +00:00
- [Xenu ](http://home.snafu.de )
2021-02-24 01:29:10 +00:00
- [ParamSpider ](https://github.com/devanshbatham/ParamSpider )
2020-05-22 05:45:22 +00:00
2021-03-18 18:51:48 +00:00
2020-05-22 05:45:22 +00:00
## Open Source Threat Intelligence
- [GOSINT ](https://github.com/ciscocsirt/gosint ) - a project used for collecting, processing, and exporting high quality indicators of compromise (IOCs). GOSINT allows a security analyst to collect and standardize structured and unstructured threat intelligence.
- [Awesome Threat Intelligence ](https://github.com/santosomar/awesome-threat-intelligence ) - A curated list of awesome Threat Intelligence resources. This is a great resource and I try to contribute to it.
2021-06-18 22:48:37 +00:00
### Website Exploration and "Google Hacking"
- censys : https://censys.io
- Certficate Search: https://crt.sh/
- ExifTool: https://www.sno.phy.queensu.ca/~phil/exiftool
- Google Hacking Database (GHDB): https://www.exploit-db.com/google-hacking-database
- Google Transparency Report: https://transparencyreport.google.com/https/certificates
- Huge TLS/SSL certificate DB with advanced search: https://certdb.com
- netcraft: https://searchdns.netcraft.com
- SiteDigger: http://www.mcafee.com/us/downloads/free-tools/sitedigger.aspx
- Spyse: https://spyse.com
2020-08-11 19:28:55 +00:00
2021-06-18 22:48:37 +00:00
### Data Breach Query Tools
- BaseQuery: https://github.com/g666gle/BaseQuery
- Buster: https://github.com/sham00n/buster
- h8mail: https://github.com/khast3x/h8mail
- LeakLooker: https://github.com/woj-ciech/LeakLooker
- PwnDB: https://github.com/davidtavarez/pwndb
- Scavenger: https://github.com/rndinfosecguy/Scavenger
- WhatBreach: https://github.com/Ekultek/WhatBreach
2020-08-11 19:28:55 +00:00
### IP address and DNS Lookup Tools
2020-05-22 05:45:22 +00:00
- [bgp ](https://bgp.he.net/ )
- [Bgpview ](https://bgpview.io/ )
- [DataSploit (IP Address Modules) ](https://github.com/DataSploit/datasploit/tree/master/ip )
- [Domain Dossier ](https://centralops.net/co/domaindossier.aspx )
- [Domaintoipconverter ](http://domaintoipconverter.com/ )
- [Googleapps Dig ](https://toolbox.googleapps.com/apps/dig/ )
- [Hurricane Electric BGP Toolkit ](https://bgp.he.net/ )
- [ICANN Whois ](https://whois.icann.org/en )
- [Massdns ](https://github.com/blechschmidt/massdns )
- [Mxtoolbox ](https://mxtoolbox.com/BulkLookup.aspx )
- [Ultratools ipv6Info ](https://www.ultratools.com/tools/ipv6Info )
- [Viewdns ](https://viewdns.info/ )
- [Umbrella (OpenDNS) Popularity List ](http://s3-us-west-1.amazonaws.com/umbrella-static/index.html )
2021-06-18 22:48:37 +00:00
### Social Media
2020-08-11 19:28:55 +00:00
* A tool to scrape LinkedIn: https://github.com/dchrastil/TTSL
* cree.py http://ilektrojohn.github.com/creepy
2021-06-18 22:48:37 +00:00
### Whois
2020-08-11 19:28:55 +00:00
WHOIS information is based upon a tree hierarchy. ICANN (IANA) is the authoritative registry for all of the TLDs and is a great starting point for all manual WHOIS queries.
2021-06-18 22:48:37 +00:00
- ICANN: http://www.icann.org
- IANA: http://www.iana.com
- NRO: http://www.nro.net
- AFRINIC: http://www.afrinic.net
- APNIC: http://www.apnic.net
- ARIN: http://ws.arin.net
- LACNIC: http://www.lacnic.net
- RIPE: http://www.ripe.net
2020-08-11 19:28:55 +00:00
### BGP looking glasses
2021-06-18 22:48:37 +00:00
- BGP4: http://www.bgp4.as/looking-glasses
- BPG6: http://lg.he.net/
2020-08-11 19:28:55 +00:00
### DNS
2023-06-30 14:12:58 +00:00
- dnsenum - https://code.google.com/p/dnsenum
- dnsmap: https://code.google.com/p/dnsmap
- dnsrecon: https://www.darkoperator.com/tools-and-scripts
- dnstracer: https://www.mavetju.org/unix/dnstracer.php
- dnswalk: https://sourceforge.net/projects/dnswalk
2021-06-18 22:48:37 +00:00
## The OSINT Framework
- [OSINT Framework ](https://osintframework.com )
2020-08-11 19:28:55 +00:00
2021-02-15 01:27:18 +00:00
## Dark Web OSINT Tools
### Dark Web Search Engine Tools
- [Ahmia Search Engine ](https://ahmia.fi ) and [their GitHub repo ](https://github.com/ahmia/ahmia-site )
- [DarkSearch ](https://darksearch.io ) and their [GitHub repo ](https://github.com/thehappydinoa/DarkSearch )
- [Katana ](https://github.com/adnane-X-tebbaa/Katana )
- [OnionSearch ](https://github.com/megadose/OnionSearch )
2021-02-23 18:49:01 +00:00
- [Search Engines for Academic Research ](https://www.itseducation.asia/deep-web.htm )
2021-02-24 01:29:10 +00:00
- [DarkDump ](https://github.com/josh0xA/darkdump )
2021-02-15 01:27:18 +00:00
### Tools to Obtain Information of .onion Links
- [H-Indexer ](http://jncyepk6zbnosf4p.onion/onions.html )
- [Hunchly ](https://www.hunch.ly/darkweb-osint )
- [Tor66 Fresh Onions ](http://tor66sewebgixwhcqfnp5inzp5x5uohhdy3kvtnyfxc2e5mxiuh34iid.onion/fresh )
### Tools to scan onion links
- [Onioff ](https://github.com/k4m4/onioff )
- [Onion-nmap ](https://github.com/milesrichardson/docker-onion-nmap )
- [Onionscan ](https://github.com/s-rah/onionscan )
### Tools to Crawl Dark Web Data
- [TorBot ](https://github.com/DedSecInside/TorBot )
- [TorCrawl ](https://github.com/MikeMeliz/TorCrawl.py )
- [OnionIngestor ](https://github.com/danieleperera/OnionIngestor )
### Other Great Intelligence Gathering Sources and Tools
2021-03-18 18:51:48 +00:00
- Resources from Pentest-standard.org - http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#Intelligence_Gathering
2018-08-07 20:34:22 +00:00
2020-08-11 19:28:55 +00:00
### Active Recon
2021-03-18 18:51:48 +00:00
- Tons of references to scanners and vulnerability management software for active reconnaissance - http://www.pentest-standard.org/index.php/PTES_Technical_Guidelines#Vulnerability_Analysis