Commit graph

1421 commits

Author SHA1 Message Date
ahrav
dfb7e9a405
Add hash v2 to results proto. (#693) 2022-08-04 16:37:15 -07:00
Bill Rich
7273dc9058
Archive decoder (#683)
* Archive decoder

* Fix reader handling

* Seek error handling

* Add tests

* Fix extra empty chunk

* Sync chunk size
2022-08-02 20:36:21 -07:00
ahrav
21e1ff4a8a
Fix the order to correctly match the params in NewGit. (#676) 2022-07-28 13:23:45 -07:00
Ariel Ril
4b0315c90a
added support for gitlab subgroups (#664) 2022-07-26 13:37:37 -07:00
trufflesteeeve
176552b07a
Fix commit attribution, git tests, and run make protos (#667)
* Update dependency to fix commit attribution, fix git tests

* Run make protos to match code with current proto definitions
2022-07-25 11:44:15 -04:00
Ariel Ril
2515fbeebe
added http protocol support for gitlab repositories (#662) 2022-07-20 10:35:24 -07:00
trufflesteeeve
96106563a9
Remove git fragment trace (#656)
The fragment trace was a bit too verbose even at the trace level. We may
want to trace the file being chunked or something like that, but not the
entire diff.
2022-07-14 13:13:23 -04:00
Dustin Decker
2d3ddad076
Pr/589 (#654)
* added common regex patterns for detectors

* For HexPattern

* enhancements

* used parseInt

* enhancement

* enhanced regex for email and subdomain

* enhancement for email pattern

* update pattern and detector

Co-authored-by: Roxanne Tampus <roxannetampus02@gmail.com>
2022-07-12 08:34:02 -07:00
roxanne-tampus
e9f503a083
added new detectors (#639)
* added new detectors

* added gemini
2022-07-08 08:19:03 -07:00
Dustin Decker
c2426df4d6
Disable sentiment verifier and update teams webhook to avoid being stateful (#646) 2022-07-07 16:55:45 -07:00
Miccah
c4ca7d7c8b
Mark detector tests with a build flag (#613) 2022-07-07 10:27:21 -07:00
Miccah
8e6289e70c
Test numeric password redaction (#644) 2022-07-07 10:13:50 -07:00
ahrav
5ac54ac107
[THOG-531] - remove ending word boundary AWS detector (#637)
* Remove the ending word boundary for the AWS dectector. This will prevent missing secrets that end with / due to it not being ASCII.

* Update regex to be more strict.
2022-07-07 10:10:33 -07:00
Dustin Decker
85ed11519f
fix npd for incompatible sources on json legacy output (#623) 2022-06-21 09:50:55 -07:00
trufflesteeeve
d1a81afbb5
Only include one result per AWS Key ID, preferably verified (#619)
Also ignore unverified results that match hashes, because they are
probably just hashes.
2022-06-17 16:22:36 -04:00
trufflesteeeve
e793f4a5e6
Properly count the number of repos after a github scan resume (#625) 2022-06-17 16:21:22 -04:00
trufflesteeeve
10f4d02c31
Allow gitlab to resume from encoded resume info (#611) 2022-06-17 11:45:17 -04:00
Dustin Decker
2178f1f42e reword and fix error logging 2022-06-13 16:14:22 -07:00
trufflesteeeve
e123e9f177
Cleanup individual repositories after scanning (#614) 2022-06-10 14:00:50 -04:00
Dustin Decker
9bcddbc45a
Change GHE org enum to use since ID instead of pages (#618)
* Change GHE org enum to use since ID instead of pages

* fix logging
2022-06-09 15:09:13 -07:00
trufflesteeeve
26bf1664eb
Move aws detector back to ID checking against secrets, make ID the redacted secret (#617) 2022-06-09 15:06:08 -07:00
Dustin Decker
8051b03bbf
improve debug logging for GHE enum (#615) 2022-06-08 13:56:07 -07:00
Dustin Decker
1a12a25f4d
Enumerate all visible orgs in GHE (#612) 2022-06-07 09:24:31 -07:00
Dustin Decker
e3bbf293e2
Fix NPD on mutex (#609)
* Fix NPD on mutex

* fix test
2022-06-06 17:20:27 -07:00
Miccah
9074006695
Fix bug in GitHub unit test mocking (#608) 2022-06-06 16:58:34 -07:00
trufflesteeeve
fd79a367f1
Allow github to resume from encoded resume info (#601) 2022-06-06 12:08:57 -04:00
ahrav
59fc54b94a
[Thog-441] create public event monitoring source (#599)
* Add source for public event monitoring.

* Update protos.

* Add source for public event monitoring.

* Update metadata.

* Update proto.

* Update proto.

* Update pb files.
2022-06-03 14:46:31 -07:00
Miccah
fc18a5ae0c
Bug fix and add authentication in shallow clone (#595) 2022-05-31 20:45:28 -05:00
Miccah
d7d5db9e78
Replace AccessToken protobuf with a simpler string type (#594) 2022-05-31 14:28:07 -05:00
Dustin Decker
c52545a0d3
Increase default timeout for SaneHttpClient, but fail early. (#587)
* Increase default timeout for SaneHttpClient, but fail early.

* Don't change default for client with timeout
2022-05-26 11:08:38 -07:00
ahrav
143fa333f3
[THOG-403] Fix var naming. (#586)
* remove profililing.

* rename uri to URI.

* change var declaration.
2022-05-25 15:23:20 -07:00
Dustin Decker
43955b9a38
Use GetCallerIdentity for AWS verification (#585) 2022-05-25 14:45:28 -07:00
Miccah
6fa2171a22
Refactor Engine to wait for workers in a Finish method (#581)
* Refactor Engine to wait for workers in a Finish method

This should allow the engine to run multiple concurrent scans if
desired before shutting down.

Additionally, this commit refactors some of the printing logic to the
output package.

* Fix tests
2022-05-25 11:35:44 -05:00
Miccah
67ad2f2247
Shallow clone if --since-commit is provided (#564)
* Shallow clone if --since-commit is provided

* Set the user before constructing args

* Fix vbout detector

* Address PR comments

* Use a better name for timestamp
* Use net.URL.String method for the remote path
2022-05-24 10:49:03 -05:00
roxanne-tampus
80c6579226
Updated aws detector (#579) 2022-05-24 05:07:54 -07:00
Marlon
cd8fdec3aa
added codemagic detector (#480) 2022-05-23 20:20:33 -07:00
Dustin Decker
235a60a850
add more detectors (#577) 2022-05-23 18:39:49 -07:00
roxanne-tampus
da872f5c27
modified Alibaba detector to use standard library (#568)
* added alibaba detector

* enhancement

* enhancement and ran mod tidy

* fixed
2022-05-23 18:37:01 -07:00
roxanne-tampus
66dc7efac0
Added postbacks detector (#551)
* Added postbacks detector

* enhancement
2022-05-23 18:17:52 -07:00
roxanne-tampus
83dfed9257
Added zipcodebase detector (#550)
* Added zipcodebase detector

* regex enhancement
2022-05-23 18:15:49 -07:00
roxanne-tampus
75972a6f81
Added zenrows detector (#548) 2022-05-23 18:14:52 -07:00
roxanne-tampus
d5e034fddd
Added tefter detector (#547)
* Added tefter detector

* enhancement
2022-05-23 18:14:01 -07:00
roxanne-tampus
3bbc9ef394
Added diggernaut detector (#546) 2022-05-23 18:13:12 -07:00
roxanne-tampus
f509261727
added convertapi detector (#545) 2022-05-23 18:12:22 -07:00
roxanne-tampus
d5f8ddd804
Added collect2 detector (#544)
* added collect2 detector

* enhancement as suggested
2022-05-23 18:11:32 -07:00
roxanne-tampus
c9f1f61c49
Added cloudconvert detector (#543)
* added cloudconvert detector

* updated regex
2022-05-23 18:10:02 -07:00
Dustin Decker
509ae79143 add missing import 2022-05-23 15:48:50 -07:00
ahrav
2051fe14ff
remove profililing. (#567) 2022-05-23 11:05:39 -07:00
valerie gale
9d1c1e958a
enhancements in amplitude detector (#575)
* enhancements in amplitude detector

* enhancements in amplitude detector
2022-05-23 09:20:35 -07:00
Dustin Decker
4d3c2d70e5
use go 1.18 (#566)
* use go 1.18 in CI

* require go 1.18

* use latest linter

* rename ci check

* improve regex issues identified by codeql
2022-05-19 09:01:50 -07:00