Mirrors/sherlock
2
0
Fork 0
mirror of https://github.com/sherlock-project/sherlock synced 2024-11-22 03:43:02 +00:00
Code Issues Projects Releases Packages Wiki Activity

Refresh documentation

Browse source
This commit is contained in:
Paul Pfeister 2024-05-15 15:20:26 -04:00
parent f8e3bd7e29
commit 379725e796
5 changed files with 230 additions and 108 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

31
CONTRIBUTING.md
View file

@ -1,31 +0,0 @@
# How To Contribute To Sherlock
First off, thank you for the help!
There are many ways to contribute. Here is some high level grouping.
## Adding New Sites
Please look at the Wiki entry on
[adding new sites](https://github.com/sherlock-project/sherlock/wiki/Adding-Sites-To-Sherlock)
to understand the issues.
Any new sites that are added need to have a username that has been claimed, and one
that is unclaimed documented in the site data. This allows the regression tests
to ensure that everything is working.
It is required that a contributor test any new sites by either running the full tests, or running
a site-specific query against the claimed and unclaimed usernames.
It is not required that a contributor run the
[site_list.py](https://github.com/sherlock-project/sherlock/blob/master/site_list.py)
script.
If there are performance problems with a site (e.g. slow to respond, unreliable uptime, ...), then
the site may be removed from the list. The
[removed_sites.md](https://github.com/sherlock-project/sherlock/blob/master/removed_sites.md)
file contains sites that were included at one time in Sherlock, but had to be removed for
one reason or another.
## Adding New Functionality
Please ensure that the content on your branch passes all tests before submitting a pull request.

0
CODE_OF_CONDUCT.md → docs/CODE_OF_CONDUCT.md
View file

88
docs/CONTRIBUTING.md Normal file
View file

@ -0,0 +1,88 @@
<p align=center>
<br>
<a href="https://sherlock-project.github.io/" target="_blank"><img src="https://user-images.githubusercontent.com/27065646/53551960-ae4dff80-3b3a-11e9-9075-cef786c69364.png"/></a>
<br>
</p>
<p align="center">
<a href="https://github.com/sherlock-project/sherlock">Home</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="https://github.com/sherlock-project/sherlock#installation">Installation</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="https://github.com/sherlock-project/sherlock#usage">Usage</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="https://github.com/sherlock-project/sherlock/docs/install.md#docker">Docker</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<strong><a href="https://github.com/sherlock-project/sherlock/docs/CONTRIBUTING.md">Contributing</a></strong>
</p>
# How to contribute to Sherlock
We would love to have you help us with the development of Sherlock. Each and every contribution is greatly valued!
Here are some things we would appreciate your help on:
1. [Adding targets](#adding-targets)
1. [Cleaning up existing targets](#removing-targets)
1. [Restoring previously removed targets](#restoring-targets)
## Adding targets
Please look at the Wiki entry on [adding new sites][wiki_new_sites] to understand the issues.
All new sites that are added to Sherlock need to have an existing (already claimed) username included in their definition. The linked Wiki page describes this in more detail. This inclusion allows us to run unit tests and prevent regression.
Contributors are *required* to test any new sites for both false positives and false negatives. Contributors are *encouraged* to run unit tests as well.
Contributors do not have to run the [site_list.py](/site_list.py) script, as it's ran automagically on master after each manifest change.
## Removing targets
If there are performance problems with a site (e.g. slow to respond, unreliable uptime, ...), then
the site may be removed from the list. The [removed_sites.md][file_removed_md] file contains sites that were included at one time in Sherlock, but had to be removed for one reason or another.
If a site has *occasional* performance problems, but is otherwise accurate, it may be preferable to add a test to weed out false positives rather than removing it.
## Restoring targets
Likely our biggest backlog. If you can propose a functional query that complies with [#Adding targets](#adding-targets) that would shrink our [removed sites list][file_removed_md], that would be greatly appreciated.
## Adding New Functionality
Contributors that would like to add a feature to Sherlock should open an new [issue][issues_new], proposing their idea. Indicate that you would like to make a Pull Request for said feature.
Creating an Issue prior to opening a PR helps with tracking, discussions, and avoids hurt feelings if for whatever reason we don't feel that a feature is compatible with the project.
Please ensure that the content on your branch passes all tests before submitting a pull request.
# Coverage and Unit Tests
Thank you for contributing to Sherlock!
Before creating a pull request with new development, please run the tests
to ensure that everything is working great. It would also be a good idea to run the tests
before starting development to distinguish problems between your
environment and the Sherlock software.
The following is an example of the command line to run all the tests for
Sherlock. This invocation hides the progress text that Sherlock normally
outputs, and instead shows the verbose output of the tests.
```console
$ cd sherlock/sherlock
$ python3 -m unittest tests.all --verbose
```
Unfortunately, some of the sites that Sherlock checks are not always reliable, so it is common
to get response problems. Any problems in connection will show up as warnings in the tests instead of true errors.
If some sites are failing due to connection problems (site is down, in maintenance, etc) you can exclude them from tests by creating a `tests/.excluded_sites` file with a list of sites to ignore (one site name per line).
## Coverage for new features
Contributors that add new features are *encouraged* make an attempt at creating unit tests for them, as well. Not all contributions are suitable for unit tests, but when it's doable, it helps prevent regression.
<!-- Reference Links -->
[wiki_new_sites]: https://github.com/sherlock-project/sherlock/wiki/Adding-Sites-To-Sherlock
[file_removed_md]: /removed_sites.md
[issues_new]: https://github.com/sherlock-project/sherlock/issues/new/choose

124
README.md → docs/README.md
View file

@ -2,18 +2,18 @@
<br> <br>
<a href="https://sherlock-project.github.io/" target="_blank"><img src="https://user-images.githubusercontent.com/27065646/53551960-ae4dff80-3b3a-11e9-9075-cef786c69364.png"/></a> <a href="https://sherlock-project.github.io/" target="_blank"><img src="https://user-images.githubusercontent.com/27065646/53551960-ae4dff80-3b3a-11e9-9075-cef786c69364.png"/></a>
<br> <br>
<span>Hunt down social media accounts by username across <a href="https://github.com/sherlock-project/sherlock/blob/master/sites.md">social networks</a></span> <span>Hunt down social media accounts by username across <a href="https://github.com/sherlock-project/sherlock/blob/master/sites.md">400+ social networks</a></span>
<br> <br>
</p> </p>
<p align="center"> <p align="center">
<a href="#installation">Installation</a> <a href="https://github.com/sherlock-project/sherlock#installation">Installation</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="#usage">Usage</a> <a href="https://github.com/sherlock-project/sherlock#usage">Usage</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="#docker-notes">Docker Notes</a> <a href="https://github.com/ppfeister/sherlock/blob/feature/docu/docs/install.md#docker">Docker</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="#contributing">Contributing</a> <a href="https://github.com/ppfeister/sherlock/blob/feature/docu/docs/CONTRIBUTING.md">Contributing</a>
</p> </p>
<p align="center"> <p align="center">
@ -24,9 +24,22 @@
## Installation ## Installation
```console [![PyPI - Version](https://img.shields.io/pypi/v/sherlock-project?logo=PyPi&label=PyPI&color=darkgreen)][ext_pypi] [![Docker Image Version](https://img.shields.io/docker/v/sherlock/sherlock?sort=semver&logo=docker&label=Docker&color=darkgreen)][docs_docker] [![homebrew version](https://img.shields.io/homebrew/v/sherlock?logo=Homebrew&color=darkgreen)][ext_brew]
pip install git+https://github.com/sherlock-project/sherlock.git
```
| Method | Command | Notes |
| - | - | - |
| pypi | `pipx install sherlock-project` | `pip` may be used in place of `pipx` |
| brew | `brew install sherlock` | Community supported |
| docker | `docker pull sherlock/sherlock` | |
### Alternative guides and methods
- [See all alternative guides][docs_install]
- [Python package][docs_py]
- [Docker container][docs_docker]
## Usage ## Usage
@ -80,88 +93,28 @@ optional arguments:
``` ```
To search for only one user: To search for only one user:
``` ```bash
python3 sherlock user123 sherlock user123
``` ```
To search for more than one user: To search for more than one user:
``` ```bash
python3 sherlock user1 user2 user3 sherlock user1 user2 user3
``` ```
Accounts found will be stored in an individual text file with the corresponding username (e.g ```user123.txt```). Accounts found will be stored in an individual text file with the corresponding username (e.g ```user123.txt```).
## Anaconda (Windows) Notes
If you are using Anaconda in Windows, using `python3` might not work. Use `python` instead. ## How to contribute to Sherlock
## Docker Notes
If docker is installed you can build an image and run this as a container.
```
docker build -t mysherlock-image .
```
Once the image is built, sherlock can be invoked by running the following:
```
docker run --rm -t mysherlock-image user123
```
Use the following command to access the saved results:
```
docker run --rm -t -v "$PWD/results:/opt/sherlock/results" mysherlock-image -o /opt/sherlock/results/text.txt user123
```
Docker is instructed to create (or use) the folder `results` in the current working directory and to mount it at `/opt/sherlock/results` on the docker container by using the ```-v "$PWD/results:/opt/sherlock/results"``` options. `Sherlock` is instructed to export the result using the `-o /opt/sherlock/results/text.txt` option.
### Using `docker-compose`
You can use the `docker-compose.yml` file from the repository and use this command:
```
docker-compose run sherlock -o /opt/sherlock/results/text.txt user123
```
## Contributing
We would love to have you help us with the development of Sherlock. Each and every contribution is greatly valued! We would love to have you help us with the development of Sherlock. Each and every contribution is greatly valued!
Here are some things we would appreciate your help on: Here are some things we would appreciate your help on:
- Addition of new site support ¹ - [Adding targets][docs_contrib_adding_targets]
- Bringing back site support of [sites that have been removed](removed_sites.md) in the past due to false positives - [Cleaning up existing targets][docs_contrib_removing_targets]
- [Restoring previously removed targets][docs_contrib_restoring_targets]
[1] Please look at the Wiki entry on [adding new sites](https://github.com/sherlock-project/sherlock/wiki/Adding-Sites-To-Sherlock) Head over to our __[Contributing][docs_contrib]__ page for additional info.
to understand the issues.
## Tests
Thank you for contributing to Sherlock!
Before creating a pull request with new development, please run the tests
to ensure that everything is working great. It would also be a good idea to run the tests
before starting development to distinguish problems between your
environment and the Sherlock software.
The following is an example of the command line to run all the tests for
Sherlock. This invocation hides the progress text that Sherlock normally
outputs, and instead shows the verbose output of the tests.
```console
$ cd sherlock/sherlock
$ python3 -m unittest tests.all --verbose
```
Note that we do currently have 100% test coverage. Unfortunately, some of
the sites that Sherlock checks are not always reliable, so it is common
to get response problems. Any problems in connection will show up as
warnings in the tests instead of true errors.
If some sites are failing due to connection problems (site is down, in maintenance, etc)
you can exclude them from tests by creating a `tests/.excluded_sites` file with a
list of sites to ignore (one site name per line).
## Star History ## Star History
@ -175,3 +128,20 @@ list of sites to ignore (one site name per line).
MIT © Sherlock Project<br/> MIT © Sherlock Project<br/>
Original Creator - [Siddharth Dushantha](https://github.com/sdushantha) Original Creator - [Siddharth Dushantha](https://github.com/sdushantha)
<!-- Reference Links -->
[docs_install]: /docs/install.md
[docs_docker]: /docs/install.md#docker
[docs_docker_dockerhub]: /docs/install.md#docker
[docs_docker_compose]: /docs/install.md#using-compose
[docs_docker_source]: /docs/install.md#build-image-from-source-useful-for-contributors
[docs_py]: /docs/install.md#python
[docs_py_build]: /docs/install.md#build-python-package-from-source-useful-for-contributors
[docs_contrib]: /docs/CONTRIBUTING.md
[docs_contrib_adding_targets]: /docs/CONTRIBUTING.md#adding-targets
[docs_contrib_removing_targets]: /docs/CONTRIBUTING.md#removing-targets
[docs_contrib_restoring_targets]: /docs/CONTRIBUTING.md#restoring-targets
[ext_pypi]: https://pypi.org/project/sherlock-project/
[ext_brew]: https://formulae.brew.sh/formula/sherlock

95
docs/install.md Normal file
View file

@ -0,0 +1,95 @@
<p align=center>
<br>
<a href="https://sherlock-project.github.io/" target="_blank"><img src="https://user-images.githubusercontent.com/27065646/53551960-ae4dff80-3b3a-11e9-9075-cef786c69364.png"/></a>
<br>
</p>
<p align="center">
<strong><a href="https://github.com/sherlock-project/sherlock">Home</a></strong>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<strong><a href="#">Installation</a></strong>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="https://github.com/sherlock-project/sherlock#usage">Usage</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="#docker">Docker</a>
&nbsp;&nbsp;&nbsp;|&nbsp;&nbsp;&nbsp;
<a href="https://github.com/sherlock-project/sherlock/docs/CONTRIBUTING.md">Contributing</a>
</p>
# Alternative install methods
1. __[Python package](#python)__
1. [Build from source](#build-and-install-package-from-source)
1. __[Docker Container](#docker)__
1. [via Docker Hub (one off)](#docker)
1. [via Docker Hub (compose)](#using-compose)
1. [Local image with Dockerfile](#build-image-from-source-alternative-to-docker-hub)
<h2>
Python
<a href="https://pypi.org/project/sherlock-project/"><img align="right" alt="PyPI - Version" src="https://img.shields.io/pypi/v/sherlock-project?style=for-the-badge&logo=PyPI&label=PyPI&color=darkgreen"></a>
</h2>
```bash
# pipx is recommended, but pip may suffice if pipx is unavailable
pipx install sherlock-project
```
### Build python package from source (useful for contributors)
```bash
# pipx is recommended, but pip may suffice if pipx is unavailable
git clone https://github.com/sherlock-project/sherlock.git
cd sherlock
pipx install .
```
<h2>
Docker
<a href="https://hub.docker.com/r/sherlock/sherlock"><img align="right" alt="Docker Image Version" src="https://img.shields.io/docker/v/sherlock/sherlock?sort=semver&style=for-the-badge&logo=docker&label=Docker&color=darkgreen"></a>
</h2>
> [!NOTE]
> Sherlock doesn't yet have context detection. It's recommended that Docker containers be ran with option `-o /opt/sherlock/results/{user123}.txt` (replace {user123}) when an output file is desired at the mounted volume (as seen in the compose).
>
> This has no effect on stdout, which functions as expected out of the box.
```bash
# One-off searches
docker run --rm -t sherlock/sherlock user123
# If you need to save the output file... (modify as needed)
# Output file will land in ${pwd}/results
docker run --rm -t -v "$PWD/results:/opt/sherlock/results" sherlock/sherlock -o /opt/sherlock/results/text.txt user123
```
```bash
# At any time, you may update the image via this command
docker pull sherlock/sherlock
```
### Using compose
```yml
version: "3"
services:
sherlock:
container_name: sherlock
image: sherlock/sherlock
volumes:
- ./sherlock/:/opt/sherlock/results/
```
```bash
docker compose run sherlock user123
```
### Build image from source (useful for contributors)
```bash
# Assumes ${pwd} is repository root
docker build -t sherlock .
docker run --rm -t sherlock user123
```