Mirrors/sherlock
2
0
Fork 0
mirror of https://github.com/sherlock-project/sherlock synced 2024-11-27 14:20:37 +00:00
Code Issues Projects Releases Packages Wiki Activity
916 commits 13 branches 2 tags 27 MiB
Commit graph

205 commits

Author SHA1 Message Date
Christopher K. Hoadley
76b6a579bb Update site list and ranking. Update version. 2019-07-17 07:19:58 -05:00
Siddharth Dushantha
b1b561981a
removed the unnecessary banner 2019-07-17 10:31:42 +02:00
Siddharth Dushantha
723ca405e5
Merge pull request #226 from sp1rs/invalid-url-on-print-found-flag 
Don't show invalid/error social site if --print-found flag is enabled.
 2019-07-16 15:42:31 +02:00
Christopher K. Hoadley
8bb9386aca Update version. 2019-07-16 06:40:16 -05:00
Christopher K. Hoadley
77d62bca10 Fix Error When csv Is Written When Username Is Illegal For A Site 
This problem happens during the write to the CSV.  There are various results that are gathered during the query for the existence of a given site, and they are always expected to exist in the results dictionary.  However, if a given user name has an illegal format for a given site, then the query is not even attempted.  Which leaves some keys in the dictionary undefined.

The solution is to define these keys even in the case where the username has an illegal format.
 2019-07-16 06:35:57 -05:00
Christopher K. Hoadley
cc163ce0ef Update site list and ranking. Update version. 2019-07-14 11:02:36 -05:00
Christopher K. Hoadley
5731a7bafc Update site list and ranking. Update version. 2019-07-14 02:53:58 -05:00
Christopher K. Hoadley
e45e0d0c18 Merge branch 'pr/n225_sp1rs'. Update site list and readme. 
# Conflicts:
#	sites.md
 2019-07-13 09:56:41 -05:00
Christopher K. Hoadley
e9d7a45ca0 Update site list and ranking. Update version. 2019-07-13 09:32:35 -05:00
sp1rs
ed08f979f0 Don't show invalid/error social site if --print-found flag is enabled. 2019-07-12 00:47:25 +05:30
Christopher K. Hoadley
485676fc36 Update site list and ranking. Update version. 2019-07-10 23:05:18 -05:00
Christopher K. Hoadley
3276e97327 Update site information. Fix ResearchGate test username for unclaimed. The old value did not pass the regular expression check. 2019-06-25 23:34:41 -05:00
Christopher K. Hoadley
7f2d3520d2 Update site list and ranking. Update version. 2019-06-08 09:26:26 -05:00
Christopher K. Hoadley
63fae2902e Update site list and ranking. Update version. 2019-05-28 07:27:35 -05:00
Christopher K. Hoadley
f96febcd3a Update version information. 2019-05-22 07:41:35 -05:00
Christopher K. Hoadley
75c3a1622f Update version to document the great f-string renaissance of 2019. :) 2019-05-15 07:41:57 -05:00
Christopher K. Hoadley
3b6cb1d4d9 Try to back out "Fix stray f's that break on Linux" #198. The user was getting a syntax error from the format strings (probably from using an old version of Python). But, blindly removing the f-strings broke the actual functionality. 2019-05-15 07:39:39 -05:00
snehankekre
92d39f855a Fix stray f's that break on Linux 2019-04-10 14:45:55 +05:30
Christopher K. Hoadley
a8f1891028 Update list of supported sites and ranking. Update version number. 2019-04-07 20:51:47 -05:00
Christopher K. Hoadley
ed08886982 Update rank and version information. 2019-04-07 11:26:11 -05:00
Christopher K. Hoadley
66f19f8c7b Update ranking results. Update version information. 2019-03-31 11:07:59 -05:00
Christopher K. Hoadley
76578d83a7 Add "urlProbe" to data area. While the results for YouNow work after putting the API URL in the "url" area, the printed results do not display properly. So, I needed to introduce the concept of a probe URL separate from the actual user URL. 2019-03-31 10:59:24 -05:00
Yahya SayadArbabi
55b1fa5c44 bump version 2019-03-08 16:11:25 +03:30
Yahya SayadArbabi
34e72a0622 Resolve conflicts 2019-03-08 13:32:38 +03:30
Yahya SayadArbabi
ed3dddde65 bump version 2019-03-08 13:31:15 +03:30
Yahya SayadArbabi
6c5656a84b bump version 2019-03-08 13:25:38 +03:30
Jen-Chieh
972d7dceea Org code for removing all the purpose print functions. 2019-03-08 13:40:58 +08:00
cclauss
47fac5943f
Use compound if statement rather than "if and" 
This approach is a bit strange to read the first few time but has advantages in that it is easier to get right and faster to execute.

The interactions between `and`, `or`, `not`, and `()` can get confusing for new coders.  For example, the code line 308 does not match the comment on 307 and I believe that in this case the comment correct and the code is wrong (for values < 200) because it is missing parens.  I believe that __200 <= status_code < 300__ produces the correct results in a readable (semi-)intuitive code.
```
>>> for status_code in (-1, 1, 199, 200, 201, 299, 300, 301, 1000):
...     print(not status_code >= 300 or status_code < 200,
...           not (status_code >= 300 or status_code < 200),
...           200 <= status_code < 300)
...
True False False
True False False
True False False
True True True
True True True
True True True
False False False
False False False
False False False
```
 2019-03-07 23:57:56 +01:00
Yahya SayadArbabi
bbc056cd1d Bump version 2019-02-27 11:48:37 +03:30
Tait Hoyem
74eefe02ca Rename option to --print-found 2019-02-26 16:54:49 +00:00
Tait Hoyem
2750538e2e Add option to not show 'Not Found!' messages. 2019-02-26 04:35:52 +00:00
Christopher K. Hoadley
0f8e734d78 Bump version. 2019-02-21 07:15:44 -06:00
Christopher K. Hoadley
4fa8de4758 Update version. 2019-02-16 19:49:21 -06:00
Yahya SayadArbabi
a27c0a855e bump version 2019-02-16 04:00:24 +03:30
Christopher K. Hoadley
ea90e7fa95 Bump version and add link to Wiki on procedure to add new sites. 2019-02-02 21:24:46 -06:00
Yahya SayadArbabi
8a87513a0d bump version 2019-02-02 17:49:46 +03:30
Yahya SayadArbabi
800431b424 bump version 2019-02-01 21:57:40 +03:30
Christopher K. Hoadley
7fb6d26cc7 Previous code was failing the flake8 tests because the random module was not imported. 2019-01-30 18:37:03 -06:00
Yahya SayadArbabi
f69be05803 Rebase & bump version 2019-01-29 10:11:18 +03:30
BlucyBlue
465f4c85c3 Typo in printout when reading proxies from file. 2019-01-29 10:10:31 +03:30
BlucyBlue
9f523365f7 Finally importing load_proxies module. 2019-01-29 10:10:31 +03:30
BlucyBlue
8587d1a835 If the ProxyError gets raised in the 'get_response' function, the request will be tried with another proxy selected from the 'proxy_list' global var. New parameter 'retry_no' is the number of retries that will be made before throwing a final ProxyError. 2019-01-29 10:10:31 +03:30
BlucyBlue
6bf8358342 Set new parameter 'retry_no' of the 'get_response' function to 3 (can be changed). This will be used if retrying a ProxyError. 2019-01-29 10:10:31 +03:30
BlucyBlue
855f154d9b If the 'proxy_list' we select a random member and pass it as the proxy to the session. If the list is empty, the proxy parameter will be set to arg.proxy, which defaults to None if the user did not pass an individual proxy as well. 2019-01-29 10:10:31 +03:30
BlucyBlue
2accdcafea If the user selected --check_proxies option along with --proxy_list option, proxies loaded from the .csv file are checked using the check_proxies function from the load_proxies module. Proxies which pass the test are stored in the proxy_list global var. 2019-01-29 10:10:06 +03:30
BlucyBlue
6cc4e22898 If the user selected --proxy_list option, we attempt to read proxies from the csv, and store the list in global var proxy_list. 2019-01-29 10:10:06 +03:30
BlucyBlue
bd683022b3 Exception is raised if both a single proxy and the proxy_list are used. As needed, this can be changed to merging the single proxy with the proxy list, but seems a bit unnecessary at this time. 2019-01-29 10:10:06 +03:30
BlucyBlue
dc32d473e0 Exception will now be raised if etiher a single proxy or proxy_list options are used along with Tor. 2019-01-29 10:10:06 +03:30
BlucyBlue
c5e06b068e Added two new arguments, '--proxy_list'/'-pl' and '--check_proxies'/'-cp', for users to activate options of reading proxies from a document (at this time, only .csv is supported), and check their anonimity before using them. 2019-01-29 10:10:06 +03:30
BlucyBlue
166d224423 First change to 'sherlock.py' for use of load_proxies module. Global variable proxy_list is created, and by default set to an empty list. This variable will store proxies from a proxy list (if this option is used), and will enable different threads to access proxies at the same time. 2019-01-29 10:09:33 +03:30
Yahya SayadArbabi
67108071e5 bump version 2019-01-29 10:02:54 +03:30
Yahya SayadArbabi
011df7af55 bump version 2019-01-27 13:57:03 +03:30
ptalmeida
8b681158bc small corrections to rank sort 2019-01-25 17:36:38 +00:00
ptalmeida
78ade00dee Update outdated REAME.md 2019-01-25 15:10:03 +00:00
ptalmeida
5d972a3138 add --rank -r option to sherlock 2019-01-25 15:05:38 +00:00
George Tsomlektsis
0e6b8d0dca Added optional parameters for outputting files and folders. 2019-01-24 21:59:06 +02:00
George Tsomlektsis
f511faab23 Added the ability to load external json files. 2019-01-24 21:16:49 +02:00
Yahya SayadArbabi
7a93f73b23 bump version 2019-01-23 23:03:01 +03:30
Christopher K. Hoadley
a18065c1a7 Bump version. Update usage results in readme. 2019-01-22 20:55:10 -06:00
Christopher K. Hoadley
a538c07ab3 Merge branch 'pr/n133_cclauss' 2019-01-22 20:51:32 -06:00
Christopher K. Hoadley
65b38592c4 Change "response_url" detection strategy completely. 
Previously, there was a problem with sites that redirect an attempt to view a non-existing username to the main site. For example, if you try to go to https://devrant.com/users/dfoxxxxxxxxx (a user name that does not exist), then we get a redirect to the https://devrant.com/ root of the site. But, the "response_url" checking algorithm was only looking for the configured error URL being included in the response.  So, these sites always indicated that the username was not found.

Update the "response_url" detection method so that the request does not allow redirects. If we get a 200 response of some type, then the username has been found. However, if we get something like a 302, then we know that the username was not found as we are being redirected.

This whole method seems fragile, but I did exhaustively test all of the supported sites, and they all work.  So, this change is clearly an improvement.
 2019-01-22 20:37:05 -06:00
cclauss
0b3c5f2ba0
Fix the top of the deerstalker 
https://en.m.wikipedia.org/wiki/Deerstalker
 2019-01-22 20:15:13 +01:00
cclauss
a380e48057
Make BANNER an r'string' and move it up front 
Sherlock has a cool logo so this PR moves it to the top of sherlock.py to show it off when potential contributors read our source.  It also makes a BANNER an __r'string'__ so we do not need backslashes to escape characters and the ascii art is more what-you-see-is-what-you-get.  This also suppresses [a bunch of flake8 complaints](https://travis-ci.com/TheYahya/sherlock/jobs/172028552#L490-L503) about _invalid escape sequences_.  Finally this PR runs isort on the imports for readability.  Standard Lib imports come first, etc.
 2019-01-22 18:55:03 +01:00
Christopher K. Hoadley
4c6bb61483 Update version number. This version does have basic tests. 2019-01-21 19:27:08 -06:00
Christopher K. Hoadley
ee4cd8e14b While running tests, I was getting a ResourceWarning about an unclosed file. The result file was not being closed, so it was a valid warning. Close the file to avoid this warning. 2019-01-21 16:30:07 -06:00
Yahya SayadArbabi
d4e29b1327 bump version 2019-01-19 19:24:02 +03:30
Yahya SayadArbabi
ff9e239928 bump version 2019-01-15 00:32:30 +03:30
SebastianElvis
29d89921d0 add proxy functionality 2019-01-12 12:55:19 +08:00
Yahya SayadArbabi
ed196b7466 bump version 2019-01-10 17:23:55 +03:30
Yahya SayadArbabi
1c5e07645e Merge branch 'Made_parameter_for_site_name_not_case_sensitive' 2019-01-10 17:22:56 +03:30
Yahya SayadArbabi
2247f556f2 bump version 2019-01-10 17:17:17 +03:30
Master
b245a6e5cc fixed wrong condition place 2019-01-09 19:46:02 +02:00
Master
34a0db9cd8 implemented #115: make parameter for site name not case sensitive 2019-01-09 19:43:00 +02:00
Tejasvi (Teju) Nareddy
d0ecb70d84
Merge pull request #109 from nareddyt/response-time-metrics 
Response time metrics
 2019-01-08 19:08:47 -05:00
Yahya SayadArbabi
5ed2c9a740 bump version 2019-01-09 01:03:11 +03:30
Tejasvi Nareddy
038d531483 doc: response time 
- Document new feature for response times
- Refactor
 2019-01-07 12:04:55 -05:00
Tejasvi Nareddy
4c3967fd6d feat: expose response times 
- Add hook to FuturesSession to calculate absolute response time (ms)
- In verbose mode, display response times for each request
- Always outputs response times to CSV
 2019-01-06 22:33:36 -05:00
Yahya SayadArbabi
7e5bac0cb8 bump version 2019-01-06 14:02:35 +03:30
Yahya SayadArbabi
18ee78f9e0 Merge branch 'masta2898/master' 2019-01-06 14:01:45 +03:30
Yahya SayadArbabi
c165952ca8 bump version 2019-01-06 13:55:50 +03:30
Christopher K. Hoadley
33e8beb5b4 Add command line option to only run a report on specified sites (as opposed to all of them). Move loading of JSON file out of the query logic proper: we need to keep the database and the query logic separate anyway for future changes, so this is a first step in the refactoring. Update readme file with latest information. 2019-01-05 22:52:53 -06:00
Christopher K. Hoadley
4596f7121e Specify the encoding to be UTF-8 for the csv file. The Rajče.net site was causing this to fail. 2019-01-05 20:39:56 -06:00
Master
5dc147e39a efficient usage of GET request only for github 2019-01-05 23:05:23 +02:00
Yahya SayadArbabi
81d15482f7 bump version 2019-01-06 00:30:32 +03:30
Master
69cdbe83f2 fix #95: head request may cause 301 redirect which can be interpreted wrong i.e. github's cause 2019-01-05 19:45:19 +02:00
Master
1e50c92d8b update from upstream 2019-01-05 18:50:01 +02:00
Yahya SayadArbabi
0d85703093 bump version 2019-01-05 16:57:31 +03:30
QuantumSheep
2e4b3a32ad Stops opening the file at each writing 2019-01-05 13:43:03 +01:00
Yahya SayadArbabi
f27d2b2aed bump version 2019-01-05 11:14:13 +03:30
Yahya SayadArbabi
cc22d88679 bump version 2019-01-05 09:38:28 +03:30
nil0x42
2791e1553a
Fix incorrect file save notification 
When username argument was provided with uppercase chars, sherlock
said that file was saved into "UserName.txt" instead of "username.txt" (fname var)
 2019-01-04 23:59:58 +00:00
Yahya SayadArbabi
683ca99316 bump version 2019-01-05 01:20:41 +03:30
Yahya SayadArbabi
de1d0d643c bump version 2019-01-04 19:34:00 +03:30
Yahya SayadArbabi
a63c14071b Merge branch 'Use_Colorama_syntax_for_better_readability' 2019-01-04 19:33:25 +03:30
Yahya SayadArbabi
30eb9c76ec bump version 2019-01-04 11:51:07 +03:30
Yahya SayadArbabi
287305a12c bump version 2019-01-04 11:42:00 +03:30
Yahya SayadArbabi
3b5263c5bd Add Wikipedia 2019-01-04 11:02:19 +03:30
Jen-Chieh
df1f35bc88 Use Colorama for better readability. 2019-01-04 13:17:23 +08:00
Andrew Yatzkan
2d50409428 Changed error code from 404 to not 2XX 2019-01-03 18:04:52 -06:00
Andrew Yatzkan
ab99ee016e Made .txt file case insensitive 2019-01-03 17:44:24 -06:00