Mirrors/trufflehog
2
0
Fork 0
mirror of https://github.com/trufflesecurity/trufflehog.git synced 2024-11-10 15:14:38 +00:00
Code Issues Projects Releases Packages Wiki Activity
2705 commits 139 branches 257 tags 147 MiB
Commit graph

242 commits

Author SHA1 Message Date
Alexandr Marchenko
b29b78c10d
filesystem support for exclude and include filters (2nd attemp) (#1033) 
* fix filter issue - empty lines should be ignored

* filesystem support for filter exclude

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-01-26 09:33:45 -08:00
ahrav
319ae64a02
[chore] - Small cleanup of CircleCi source (#1028) 
* Small cleanup of CircleCi source.

* address comments.

* Add context to methods as first param.
 2023-01-17 09:36:18 -08:00
Bill Rich
430d5c764c
Rename and export isGitSource (#1016) 2023-01-10 12:51:58 -08:00
Bill Rich
8b2e1d36cf
Copy metadata for line number aware sources (#1011) 
* Copy metadata for line number aware sources

* Improve style
 2023-01-10 09:35:44 -08:00
Dustin Decker
5f6143f09a
Add Circle CI source (#997) 
* Add Circle CI source

* remove SHA1 line

* remove trim
 2023-01-05 21:44:37 -08:00
ahrav
936a139596
Allow using a glob for include list. (#977) 
* Allow using a glob for include list.

* Update command flag.

* Make comment more clear.

* update comment.

* Allow scanning repo and org at the same time.
 2022-12-16 13:28:16 -08:00
Bill Rich
335ce85ce4
Export line number code (#962) 2022-12-06 15:31:15 -08:00
Thiago Lages de Alencar
9757c339d9
Fix error message (#933) 2022-11-20 05:31:11 -08:00
Dustin Decker
ae4b387448
add LDAP detector (#896) 2022-11-18 19:45:11 -08:00
Dustin Decker
b18edef01a
Enable skipping of particular key IDs (#930) 
* Enable skipping of particular key IDs

* update test
 2022-11-18 09:09:40 -08:00
Jessica
3d501975e4
Add filter as scan option to gitlab module's git scan (#919) 2022-11-15 13:02:37 -08:00
Bill Rich
d3b24fa592
Replace plain decoder with utf8 (#922) 2022-11-15 09:36:01 -08:00
kstilwell
ecd25784f5
Adding Shopify detector (#875) 
* Fixes/work based on testing

* Remove some commented code

* Change how verification happens and grab additional information

* Address linter warnings.

* add shopify detector to default detectors.

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2022-11-08 16:21:57 -08:00
ahrav
dd141fb55f
[oc-147] - Add context to all git methods (#901) 
* Add context to all git methods.

* remove logrus.

* Add ctx.

* Address comments.

* Add error to clone failing.

* Return error.
 2022-11-03 16:36:52 -07:00
Dustin Decker
a7fc12240f
Do local URI verification, while attempting to defuse SSRF (#879) 
* simplify monogo pattern

* do URI verification locally, while attempting to defuse SSRF

* test SSRF defuse

* simplify err check logic per linter recommendation

* split up detectors

* address comments

* remove unused var
 2022-11-01 17:27:24 -07:00
ahrav
fe029b1098
[THOG-793] - Return all unverified results (#856) 
* Remove the check to filter and return only a single unverified result.

* Revert "Remove the check to filter and return only a single unverified result."

This reverts commit 494e432803.

* Add new CLI flag to filter unverified results.
 2022-10-31 09:36:10 -07:00
Bill Rich
034ca4fb5b
Add bytes counter to scans (#876) 2022-10-27 12:54:22 -07:00
Alexandr Marchenko
60464da3ce
proposal: SqlServer connection string detector (#867) 
* sqlserver added to detectors.proto

* make protos

* boilerplate detector generated

* wireup

* initial
 2022-10-26 07:46:13 -07:00
Bill Rich
958266ea84
Run chunker in pipeline (#859) 
* Run chunker in pipeline

* Move ChunkSize and PeekSize to source package.

* Use new Chunk and Peek size location
 2022-10-24 13:57:27 -07:00
Ankush Goel
d29357c9d4
added npm detector (#841) 2022-10-13 06:04:02 -07:00
Dustin Decker
85467538f6
remove faulty detector (#836) 2022-10-07 09:20:44 -07:00
ahrav
128002885a
Add decoder type to results. (#835) 2022-10-06 11:55:07 -07:00
Miccah
2bc4985061
Add SSH config option for the git source (#830) 
* Add SSH config option for the git source

The auth message is empty since we use the git binary underneath to
handle the SSH authentication.

* Import digitaloceanv2
 2022-09-28 20:40:01 +02:00
Mildred Bernardo
ad4b9406a7
Added digitaloceanv2 detector (#829) 
* Added digitaloceanv2 detector

* import detector

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-09-28 09:56:35 -07:00
ahrav
db42bcf2a2
[OC-103] - Add Gemini detector (#800) 
* Add Gemini detector.

* Add regex and test code for Gemini detector.

* Remove else.

* Add commentary.

* Address comments.

* Use regular else.

* Make nice and complicated.

* use regular detection pattern.

* Add detector to default detectors.
 2022-09-26 11:48:48 -07:00
Bill Rich
e3107ad6bb
Move head and base normalization to source (#818) 2022-09-23 08:58:45 -07:00
ahrav
f2f5b6dba8
Fix recover. (#817) 2022-09-22 15:47:42 -07:00
ahrav
92f40c2031
[THOG-709] - Recover from detector panics (#810) 2022-09-22 07:01:10 -07:00
Dustin Decker
e8f1bb9127 Import NGC detector 2022-09-21 16:26:58 -07:00
Bill Rich
509cf8b6fa
Use headref and check empty commits for base (#815) 2022-09-21 16:04:01 -07:00
Dustin Decker
fcd580406e
A few improvements (#809) 
* Run integration tests

* Update examples

* Import mongodb
 2022-09-19 13:23:25 -07:00
Dustin Decker
fa9479100e
Add common sentry recover library and add into goroutines (#738) 
* Add common sentry recover library and add into goroutines

* fix nits
 2022-08-29 11:45:37 -07:00
Dustin Decker
2452e93a80
Import 27 new detectors (#737) 2022-08-26 12:35:06 -07:00
ahrav
73f9d3f0a0
[chore] - Use config struct instead of pointer for engine scans. (#709) 
* Use a config struct instead of pointer when scanning engine sources.

* use config.
 2022-08-12 09:56:24 -07:00
ahrav
dcc102a81c
[Thog-371] Utilize config struct for engine scans (#700) 
* Use a config struct when scanning and engine source.

* fix tests.

* Move test_helpers to the sources pkg.

* Handle ScanGit error in tests.

* adderss comments.

* Use functional options.

* Remove temp var.

* Add better var names for the setup functions for each config.

* Remove unused var.

* fix error logs.

* fix error logs.

* single line.

* remove blank lines.
 2022-08-10 10:11:13 -07:00
ahrav
30ebe84e3e
[THOG-608] - Fix linter errors. (#701) 
* Fix linter errors.

* Fix gist adding test.

* Update test string for mock JSON reply.

* Remove if.
 2022-08-09 19:20:02 -07:00
trufflesteeeve
176552b07a
Fix commit attribution, git tests, and run make protos (#667) 
* Update dependency to fix commit attribution, fix git tests

* Run make protos to match code with current proto definitions
 2022-07-25 11:44:15 -04:00
Dustin Decker
c2426df4d6
Disable sentiment verifier and update teams webhook to avoid being stateful (#646) 2022-07-07 16:55:45 -07:00
Miccah
6fa2171a22
Refactor Engine to wait for workers in a Finish method (#581) 
* Refactor Engine to wait for workers in a Finish method

This should allow the engine to run multiple concurrent scans if
desired before shutting down.

Additionally, this commit refactors some of the printing logic to the
output package.

* Fix tests
 2022-05-25 11:35:44 -05:00
Dustin Decker
235a60a850
add more detectors (#577) 2022-05-23 18:39:49 -07:00
Dustin Decker
606dadfbb1
Import more detectors (#563) 2022-05-17 18:56:48 -07:00
ahrav
d2605354fe
[THOG-332 ]Remove TokenSource interface from the init method of Source. (#539) 
* Remove TokenSource interface from the init method of Source.

* Remove proto message.

* Remove proto message.

* Fix tests.

* Fix filesystem test.
 2022-05-13 14:35:06 -07:00
ahrav
b0d79180f6
[THOG-314] Add new parameter to the Init method for the source interface. (#529) 
* Add new parameter to the Init method for the source interface.

* Add Oauth Token service.

* remove .test file.

* remove .test file.

* Fix param spelling.

* fix tests with new param in init

* Add missing gock lib.
 2022-05-10 11:11:43 -07:00
ahrav
e12432cef8
[THOG-315] Replace bytes.buffer with strings.builder. (#533) 
* Replace bytes.buffer with string.builder.

* Remove profiling.

* Remove detector changes.

* ignore .test files.

* fix detectors removed.
 2022-05-09 17:02:46 -07:00
Bill Rich
6ee5c57968
Fix overlooked nil (#523) 
* Fix overlooked nil

* Use default case
 2022-05-04 15:45:12 -07:00
Bill Rich
c78120e56f
Syslog source (#500) 
* Add syslog source

* only load cert/key with tls

* Cleanup

* Linting

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
 2022-05-04 15:08:11 -07:00
Bill Rich
62bb3c57d3
Keep first line number for each chunk (#520) 2022-05-04 14:11:10 -07:00
Dustin Decker
d217a517c0
Cleanup packages and include more detectors (#521) 
* clean up duplicate packages

* include more detectors
 2022-05-04 09:27:42 -07:00
Dustin Decker
913c75db15 Add 12 more detecotrs 2022-04-21 15:31:17 -07:00
Dustin Decker
9622932d81
Add 12 detectors (#432) 2022-04-19 12:32:00 -07:00
Dustin Decker
972108aea4
21 new detectors (#347) 2022-04-08 16:07:39 -07:00
Dustin Decker
8d2dd624e4 fetcher controls fetch interval, prevent fetching loop 2022-04-03 21:13:39 -07:00
Dustin Decker
a1dfcde9a6
address detector issues (#123) 2022-04-01 18:51:41 -07:00
Miccah
78b344d7b8
Add --include-members flag to scan all members of an organization (#118) 
* Add --member-repos flag to scan all members of an organization

* Move flag to the githubScan sub-command

Co-authored-by: Miccah Castorina <miccah.castorina@trufflesec.com>
 2022-04-01 18:22:37 -07:00
ahrav
cedb3393d1
[THOG-128] Code cleanup/ OSS onboarding (#117) 
* Small amount of code clean up.

* Rename sem to concurrency for better readability and to remove an extra comment.

* fix stashing issue.

Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
 2022-04-01 16:47:27 -07:00
Bill Rich
c2b4f0bc39
Include line numbers in git metadata (#97) 
* Include line numbers in git metadata

* Update tests for fragments and line numbers
 2022-03-22 09:27:15 -07:00
Bill Rich
0ea3b93238
Use git cmd for clone (#96) 
* Use git command to clone

* Check for git command when cloning
 2022-03-22 09:26:58 -07:00
trufflesteeeve
d33551e7dc
Add midise detector (#88) 2022-03-17 13:38:05 -07:00
Bill Rich
e8234c3514
Remove unused code and add git binary check (#80) 2022-03-14 17:47:18 -07:00
Dustin Decker
72c13c4b8f
Update detector process and readme (#79) 2022-03-14 17:27:14 -07:00
Bill Rich
0ee34a5be6
Use gitleaks git cmd stream (#75) 
* Switch to git cmd stream

* Fix rebase issues
 2022-03-14 17:12:58 -07:00
Bill Rich
6486c18565
Add s3 support to CLI (#76) 
* Add s3 support to CLI

* Clean up comments

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-03-14 17:07:07 -07:00
Bill Rich
2cb5b27d3b
Add GitLab support to CLI (#78) 
* Add GitLab support to CLI

* Update cli message

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-03-14 17:05:15 -07:00
Bill Rich
8d1db04c36
Add filesystem scan to CLI (#77) 
* Add filesystem scan to CLI

* Fix linter errors

* Update message

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-03-14 17:04:19 -07:00
Dustin Decker
43de9c3604 Update defaults 2022-03-11 17:25:31 -08:00
Dustin Decker
b4ac103d11 Add snifftest 2022-03-11 10:24:36 -08:00
Dustin Decker
b0e0c26137 update detectors 2022-03-11 10:24:36 -08:00
Bill Rich
55b839fc5a
Only scan files that changed from base to head (#68) 
* Only scan files that changed from base to head

* Acknowledge ignored errors
 2022-03-08 08:48:00 -08:00
Bill Rich
bdbc7e80d1
Use sync.Map to avoid concurrent map writes (#66) 2022-03-05 22:42:06 -08:00
Bill Rich
665b0bf928 Add timestamp (#61) 
* Add timestamp to sources

* Include timestamp in sources
 2022-03-04 08:39:23 -08:00
Bill Rich
9f378b8cb3 Resolve a ref as arg for --since_commit (#57) 2022-03-04 08:39:23 -08:00
Dustin Decker
6f32b108a3 Disable GetEmails 2022-03-04 08:39:22 -08:00
Bill Rich
c144630c54 Add parent results to ignore list (#47) 
* Add parent results to ignore list

* Force concurrency to 1 when base commit is set
 2022-03-04 08:39:22 -08:00
Dustin Decker
1da3e0f723 turn off getemail scanner 2022-03-04 08:39:22 -08:00
Bill Rich
30034f5d28 Use metadata specific to source type (#48) 2022-03-04 08:39:22 -08:00
Bill Rich
56dc1b109c Check commit order, dedupe results, and support using a head commit. (#44) 
* Check commit order and support using a head commit.

* Only apply dedupe to git bases source
 2022-03-04 08:39:22 -08:00
Dustin Decker
c20e9f4732 improvements 2022-03-04 08:39:17 -08:00
Dustin Decker
77418fb3f8 module v3 2022-02-15 18:54:47 -08:00
Dustin Decker
79496b8142 Add context timeout to scanners 2022-02-15 18:54:47 -08:00
Dustin Decker
ee99f0c9a2 Record avg detector time 2022-02-15 18:54:47 -08:00
Dustin Decker
152ef6d4e1 add include forks option (#37) 2022-02-15 18:54:47 -08:00
Dustin Decker
a96de1a2cd update docs and support multi platform docker images 2022-02-15 18:54:47 -08:00
Dustin Decker
e15fa3a5be helpful logging 2022-02-15 18:54:47 -08:00
Bill Rich
206b99704b Change log order and path filtering. 2022-02-15 18:54:47 -08:00
Bill Rich
28ed0c3b7c Complete support for existing git scan flags (#13) 
* Add `since_commit` to git scan

* Support `max_depth` option for git scan

* Use new options in github and gitlab sources

* Address review feedback

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
 2022-02-15 18:54:47 -08:00
Dustin Decker
089b5c9063 improved plain output 2022-02-15 18:54:47 -08:00
Dustin Decker
03ead2f7ed Integrate GitHub source 2022-02-15 18:54:47 -08:00
Dustin Decker
533f6d1e20 updates 2022-02-15 18:54:43 -08:00
Dustin Decker
d6ffadb1ee
Initial docs and release automation (#5) 2022-01-18 16:59:18 -08:00
Bill Rich
a70937bfe6
Support remote git repos using https (#9) 
Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
 2022-01-14 16:07:45 -08:00
Bill Rich
1422cc303a
Include and exclude paths args for gitscan (#6) 
* include and exclude paths gitscan args

Add support for include_paths and exclude_paths arguments when scanning
git sources.

* Improve variable name

Co-authored-by: Bill Rich <hrich@Bills-MacBook-Pro.local>
 2022-01-14 12:40:50 -08:00
Dustin Decker
4218c39d99
Initial CLI w/ partially implemented Git source and demo detector (#1) 2022-01-13 12:02:24 -08:00