Mirrors/sherlock
2
0
Fork 0
mirror of https://github.com/sherlock-project/sherlock synced 2024-12-02 00:29:15 +00:00
Code Issues Projects Releases Packages Wiki Activity
820 commits 13 branches 2 tags 27 MiB
Commit graph

188 commits

Author SHA1 Message Date
SebastianElvis
29d89921d0 add proxy functionality 2019-01-12 12:55:19 +08:00
Yahya SayadArbabi
ed196b7466 bump version 2019-01-10 17:23:55 +03:30
Yahya SayadArbabi
1c5e07645e Merge branch 'Made_parameter_for_site_name_not_case_sensitive' 2019-01-10 17:22:56 +03:30
Yahya SayadArbabi
2247f556f2 bump version 2019-01-10 17:17:17 +03:30
Master
b245a6e5cc fixed wrong condition place 2019-01-09 19:46:02 +02:00
Master
34a0db9cd8 implemented #115: make parameter for site name not case sensitive 2019-01-09 19:43:00 +02:00
Tejasvi (Teju) Nareddy
d0ecb70d84
Merge pull request #109 from nareddyt/response-time-metrics 
Response time metrics
 2019-01-08 19:08:47 -05:00
Yahya SayadArbabi
5ed2c9a740 bump version 2019-01-09 01:03:11 +03:30
Tejasvi Nareddy
038d531483 doc: response time 
- Document new feature for response times
- Refactor
 2019-01-07 12:04:55 -05:00
Tejasvi Nareddy
4c3967fd6d feat: expose response times 
- Add hook to FuturesSession to calculate absolute response time (ms)
- In verbose mode, display response times for each request
- Always outputs response times to CSV
 2019-01-06 22:33:36 -05:00
Yahya SayadArbabi
7e5bac0cb8 bump version 2019-01-06 14:02:35 +03:30
Yahya SayadArbabi
18ee78f9e0 Merge branch 'masta2898/master' 2019-01-06 14:01:45 +03:30
Yahya SayadArbabi
c165952ca8 bump version 2019-01-06 13:55:50 +03:30
Christopher K. Hoadley
33e8beb5b4 Add command line option to only run a report on specified sites (as opposed to all of them). Move loading of JSON file out of the query logic proper: we need to keep the database and the query logic separate anyway for future changes, so this is a first step in the refactoring. Update readme file with latest information. 2019-01-05 22:52:53 -06:00
Christopher K. Hoadley
4596f7121e Specify the encoding to be UTF-8 for the csv file. The Rajče.net site was causing this to fail. 2019-01-05 20:39:56 -06:00
Master
5dc147e39a efficient usage of GET request only for github 2019-01-05 23:05:23 +02:00
Yahya SayadArbabi
81d15482f7 bump version 2019-01-06 00:30:32 +03:30
Master
69cdbe83f2 fix #95: head request may cause 301 redirect which can be interpreted wrong i.e. github's cause 2019-01-05 19:45:19 +02:00
Master
1e50c92d8b update from upstream 2019-01-05 18:50:01 +02:00
Yahya SayadArbabi
0d85703093 bump version 2019-01-05 16:57:31 +03:30
QuantumSheep
2e4b3a32ad Stops opening the file at each writing 2019-01-05 13:43:03 +01:00
Yahya SayadArbabi
f27d2b2aed bump version 2019-01-05 11:14:13 +03:30
Yahya SayadArbabi
cc22d88679 bump version 2019-01-05 09:38:28 +03:30
nil0x42
2791e1553a
Fix incorrect file save notification 
When username argument was provided with uppercase chars, sherlock
said that file was saved into "UserName.txt" instead of "username.txt" (fname var)
 2019-01-04 23:59:58 +00:00
Yahya SayadArbabi
683ca99316 bump version 2019-01-05 01:20:41 +03:30
Yahya SayadArbabi
de1d0d643c bump version 2019-01-04 19:34:00 +03:30
Yahya SayadArbabi
a63c14071b Merge branch 'Use_Colorama_syntax_for_better_readability' 2019-01-04 19:33:25 +03:30
Yahya SayadArbabi
30eb9c76ec bump version 2019-01-04 11:51:07 +03:30
Yahya SayadArbabi
287305a12c bump version 2019-01-04 11:42:00 +03:30
Yahya SayadArbabi
3b5263c5bd Add Wikipedia 2019-01-04 11:02:19 +03:30
Jen-Chieh
df1f35bc88 Use Colorama for better readability. 2019-01-04 13:17:23 +08:00
Andrew Yatzkan
2d50409428 Changed error code from 404 to not 2XX 2019-01-03 18:04:52 -06:00
Andrew Yatzkan
ab99ee016e Made .txt file case insensitive 2019-01-03 17:44:24 -06:00
Yahya SayadArbabi
ffbd5b3d4b Fix versioning 2019-01-04 01:20:59 +03:30
Yahya SayadArbabi
992e22059a bump version 2019-01-04 01:18:10 +03:30
Yahya SayadArbabi
f4c490d25e run "isort" 2019-01-04 01:16:55 +03:30
Yahya SayadArbabi
853008c075 Merge branch 'jcs-PR/color' 2019-01-04 01:05:26 +03:30
Yahya SayadArbabi
f984b447c0 Run "isort" 2019-01-04 01:05:12 +03:30
Jonas Tranberg
e51da5b712 If only status_code is needed use HEAD instead of GET request to avoid body download 2019-01-03 18:39:43 +01:00
Yahya SayadArbabi
e6256e70a6 Bump version 2019-01-03 17:44:35 +03:30
jenchieh
43f4e893f3 Initialize Colorama and import it. 2019-01-03 18:18:04 +08:00
Jon Ander Oribe
005e9591c1
Adding a counter of resources 
At the end of the process we can see how many results we have
 2019-01-02 16:43:38 +01:00
Yahya SayadArbabi
f7daae14e3 Merge branch 'cclauss_patch-3' 2019-01-02 13:25:47 +03:30
Yahya SayadArbabi
85c80952c9 Merge branch 'simplify_print_error' 2019-01-02 13:18:54 +03:30
Christopher K. Hoadley
448f072e36 Merge branch 'pr/n49_cblument' 2018-12-31 09:59:15 -06:00
Chris Blumentritt
ce53663331 Use os.path.join to build file path for data file 
fixes #47
 2018-12-31 09:46:42 -06:00
Christopher K. Hoadley
8090a96c57 Fix merge problem in pull request #43. Previously, there was just one for loop that iterated thru all of the sites. But, with the addition of the parallel functionality, there are now two for loops. The dictionary changes were not done in the second loop, which caused bogus results. 2018-12-31 06:08:20 -06:00
cclauss
05cbeb5c2c
Add a shebang line: #! /usr/bin/env python3 2018-12-31 08:32:35 +01:00
cclauss
c0ac0d9f9d
Update sherlock.py 2018-12-31 07:44:27 +01:00
cclauss
96c3ce7625
Simplify print_error() 2018-12-31 07:40:05 +01:00
cclauss
3200eb3ad5
Run isort on the imports 
[isort](https://github.com/timothycrosley/isort/blob/develop/README.rst) cleans up the imports for readability.
 2018-12-31 07:27:54 +01:00
Chris Blumentritt
cf7ec54f23 Script can now load the data.json file when invoked from different directory 
If the script is run from a different directory than the where it
resides then it would fail to load because it could not find data.json.
fixes #47
 2018-12-30 23:52:23 -06:00
Christopher K. Hoadley
f9d59270a3 Merge branch 'pr/n43_cclauss'. Use new pattern for getting "urlMain" data as well. 2018-12-30 18:22:30 -06:00
Christopher K. Hoadley
e2c4dbf1ef Merge branch 'pr/n4_svisser' 2018-12-30 17:46:02 -06:00
cclauss
28f023a339
Use dict.get() on optional parameters 2018-12-30 23:39:08 +01:00
cclauss
60555eeda7
Use dict.items() to simplify interaction with dict (again) 
#20 again...
 2018-12-30 23:22:29 +01:00
Siddharth Dushantha
03c41ee555
Version bump 
From now on, the version will be the date and the format is from big to small. Year, Month, Day. 
Just like youtube-dl does it.
 2018-12-30 21:39:45 +01:00
Siddharth Dushantha
57a4448264
modified the comment about the user agent #41 2018-12-30 21:25:04 +01:00
Christopher K. Hoadley
55b6ffa5ba Simeon Visser noticed that "enforce utf-8 encoding when we open data.json" (#8) got undone in the previous merge. Include it again. 2018-12-30 11:16:15 -06:00
Tejasvi Nareddy
78a3451c03 Merge remote-tracking branch 'upstream/master' into async-requests 
# Conflicts:
#	.gitignore
#	requirements.txt
#	sherlock.py
 2018-12-30 00:10:31 -05:00
mpieters93
51ce9fc10b
Merge branch 'master' into tor 2018-12-29 15:59:30 +01:00
Christopher K. Hoadley
d9e600d05d Add support for writing to Comma-Separated Values (CSV) File with all of the results. Unfortunately, the request text is so long that it is not visible in spreadsheet programs. So, I am leaving this off for now. 2018-12-28 22:32:30 -06:00
Christopher K. Hoadley
b17072b3cd Change sherlock method to return dictionary summarizing all results. This will allow the caller to have raw information for other reports. 2018-12-28 22:28:47 -06:00
Mike Pieters
feef27b2e4 Removed accidental whitespace 2018-12-29 02:45:19 +01:00
Mike Pieters
49f65021bb Adds option to use new TOR circuit for each request 2018-12-29 02:31:31 +01:00
Mike Pieters
b2cb0506a3 Adds option to connect over TOR 2018-12-29 01:50:25 +01:00
Siddharth Dushantha
6fcc62f500 new image, did some PEP8, and organized the code 2018-12-28 20:15:41 +01:00
Mike Pieters
8659c0fbe1 Altered regex_check logic 2018-12-28 17:39:10 +01:00
Christopher K. Hoadley
ed8513fd06 Update Command Line Interface: Accept List Of User Names, Add Version 
Revamp command line interface.  Allow user to pass a list of user names on the command line.  Update readme to illustrate actual usage.

Pass debug option in argument list instead of depending on the global variable.  This should probably all be reworked to use the logging module.
 2018-12-26 21:54:25 -06:00
Christopher K. Hoadley
4067991b59 Replace noPeriod attribute in JSON file with a regexCheck attribute. If this does not exist in the JSON file, then there will be a default regular expression that will be used. By default, the allowed user name pattern requires that the first character is alphabetic, while the following characters are either alphanumeric or ".", "_", or "-". Also, specifically indicate that the user name is invalid in the feedback. 2018-12-26 19:41:11 -06:00
Siddharth Dushantha
94a58614fb
Merge pull request #15 from shijuleon/master 
Refactor error printing code
 2018-12-26 19:00:34 +01:00
shijuleon
804c5d3fc8 Refactor error printing code 2018-12-26 22:55:28 +05:30
Christopher K. Hoadley
dd8d36741d Fix bug that incorrectly said that all user names were not supported for any site that would not allow a dot in the user name. 
The check for the dot should only happen if the user has a dot in the name.
 2018-12-26 11:13:18 -06:00
shijuleon
69480d82fb Removed banner from help 2018-12-26 21:01:53 +05:30
Shiju Leon
58dff89e5e
Merge branch 'master' into master 2018-12-26 19:22:15 +05:30
Master
b59dc55631 cleanup, fixed handling connection error exception 2018-12-26 15:16:49 +02:00
Siddharth Dushantha
930ffd19db
fixed styling on the ascii art 2018-12-26 13:45:38 +01:00
Siddharth Dushantha
52d1c09c21
Merge pull request #8 from ronaldseoh/enforce-encoding 
enforce utf-8 encoding when we open data.json
 2018-12-26 13:42:31 +01:00
Siddharth Dushantha
41a44be6cd fixed issue #9 2018-12-26 13:29:26 +01:00
shijuleon
ff2f42cc38 Better error printing. Debug mode 2018-12-26 15:40:09 +05:30
shijuleon
7f65018746 Add argparse. Styling in error messages 2018-12-26 13:56:03 +05:30
Siddharth Dushantha
ad23efc785 fixed #2, added new method for response_url and gitignore 2018-12-26 09:22:37 +01:00
Ronald Seoh
9c12834217 enforce utf-8 encoding when we open data.json 2018-12-26 10:34:47 +09:00
Simeon Visser
10582b629e Ensure raw file is automatically closed 2018-12-25 20:17:39 +00:00
Tejasvi Nareddy
971a88dd1e Asynchronous requests 
Improves performance greatly
 2018-12-25 14:19:21 -05:00
Tejasvi Nareddy
8ed7a39b86 Reformat file 2018-12-25 13:14:39 -05:00
shijuleon
dbfd332ca5 HTTP requests error handling, adding cmdline params 2018-12-25 21:31:01 +05:30
Siddharth Dushantha
f9dd54df2b
Add files via upload 2018-12-24 15:31:34 +01:00