Mirrors/trufflehog
2
0
Fork 0
mirror of https://github.com/trufflesecurity/trufflehog.git synced 2024-11-14 00:47:21 +00:00
Code Issues Projects Releases Packages Wiki Activity
3154 commits 138 branches 258 tags 165 MiB
Commit graph

289 commits

Author SHA1 Message Date
Brandon Yan
8fad5fff79
add dockerhub scanner (#1496) 
* add dockerhub scanner

* clean

* clean and fix regex logic and tests

* check length of userMatches before access

* Use camelcase.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2023-07-19 09:26:28 -07:00
Brandon Yan
17b90b32d0
add couchbase scanner to defaults (#1497) 
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2023-07-16 17:45:56 -07:00
Brandon Yan
9af31f00a9
add envoy api key scanner (#1482) 
* add envoy api key scanner

* Use detectors4.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2023-07-16 16:46:28 -07:00
ahrav
a9213a1103
[chore] - Update loop to switch. (#1487) 
* Update loop to switch.

* remove unused fxn.
 2023-07-12 15:47:43 -07:00
Zachary Rice
b48ac24c46
Dedupe results (#1479) 
* init 4 dedupin

* use raw rather than rawv2

* rm comment

* comments

* nits

* clean up and use rawv2 too

* add decoder order test
 2023-07-11 15:48:00 -05:00
Zachary Rice
d4972313ff
remove old detector (#1474) 2023-07-10 13:02:19 -05:00
Zachary Rice
0bdd513d88
additional similarity check for base64 and plain (#1462) 
* additional similarity check for base64 and plain

* use bytes equal

* move logic into util function
 2023-07-10 10:12:59 -05:00
Richard Gomez
23757dbe0a
remove image4 detector (#1461) 2023-07-06 12:56:09 -07:00
Zachary Rice
18a70b64bb
Introduce trufflehog:ignore tag feature (#1433) 
* init ignore

* cleanup and add test

* update readme
 2023-06-29 08:45:56 -05:00
roxanne-tampus
00920984e3
added opsgenie detector (#650) 
* added opsgenie detector

* update interface and import

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-06-27 16:43:25 -07:00
Miccah
945c27cb82
Fix docker source to return any chunk errors (#1429) 2023-06-26 12:12:46 -05:00
Brendan Shaklovitz
da5301ea1e
Exit with non-zero exit code on chunk source error (#1286) 
* Exit with non-zero exit code on chunk source error

* Exit with a non-zero exit code whenever we hit an error getting
  chunks. Previously the error would be logged but trufflehog would exit
  with a 0 (success) status code.

* fix gcs test

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: ahrav <ahravdutta02@gmail.com>
 2023-06-26 11:39:57 -05:00
Dustin Decker
e856a6890d
🎉 Add Docker image scanning 🎉 (#1412) 
* Add Docker source

* Add metrics

* Add test

* Add debugging, address PR comments, fix path output

* review suggestions
 2023-06-22 08:02:25 -07:00
Zachary Rice
74ffbd2878
add a custom detector check for logging duplicate detector (#1394) 
* add a custom detector check for logging duplicate detector

* use pb type
 2023-06-13 14:49:21 -05:00
Zubair Khan
dfb1a0cd38
Add DocuSign detector (#1382) 
* init

* look for client id and client secret, encode them for basis auth

* add tests

* test without checking the contents of response

* confirm access_token exists

* cleanup test

* explain in code that an undocumented grant_type is used

* remove use of deprecated ioutil, remove dead code, return errors instead of just logging

* directly pull access token

* update error text, remove redundant body close()

* import new detector into defaults
 2023-06-08 13:34:50 -04:00
ahrav
1da7720912
Replace context.TODO. (#1349) 2023-05-19 11:09:51 -07:00
ahrav
e81b908e07
Add buildkitev2 detector for newer tokens. (#1341) 2023-05-15 12:58:36 -07:00
vickygoel
4c04bbbe85
added pulumi cloud Access token detector (#1295) 
* added pulumi cloud Access token detector

* removed accidentally committed tokens

* added the databricks token detection

* made recommended changes

* added supabase management api token

* nuget api key detector

* added aiven.io token detector

* added prefect.io api key detector

* update protos.

---------

Co-authored-by: Developer <garg47294+1@gmail.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2023-05-11 09:08:48 -07:00
Brendan Shaklovitz
584db86031
Support line numbers in filesystem source (#1297) 2023-05-09 08:02:34 -07:00
Brendan Shaklovitz
fad34d4dc6
git worktree scanning fix for #827 (#1315) 
* Fix worktree scan by setting EnableDotGitCommonDir

* Change `PlainOpenOptions` to set `EnableDotGitCommonDir` to true.
  In every current usage of this function, it is on an already-cloned
  repository, so it should always be valid to have this set. By doing
  so, it should fix some issues with worktrees.

* Remove unused go.mod replace directives

* Remove replace directives for libraries that are not in use.
 2023-05-09 08:00:47 -07:00
Jason Solis
c13c56283d
add tineswebhook detector (#1304) 2023-05-01 07:48:58 -07:00
Miccah
b1675194ca
Implement EndpointCustomizer (#1291) 
* Implement EndpointCustomizer

Add the EndpointCustomizer interface and EndpointSetter convenience struct,
implement EndpointCustomizer for github and gitlab detectors, and add
parsing, verification, and applying user-supplied configuration.

* Check error from SetEndpoints

* Rename variable for clarity
 2023-04-27 12:23:50 -05:00
Miccah
5a86c18302
Fix include and exclude detector logic (#1267) 
* Fix include and exclude detector logic

* Fix test

* Add more clarifying comments
 2023-04-26 10:49:54 -05:00
Shabbir B
6f801f64c7
Added a new detector for percy.io (#1284) 
* Feature: Added a new detector for percy.io

* Updated variable name

---------

Co-authored-by: ahrav <ahravdutta02@gmail.com>
 2023-04-25 13:18:34 -07:00
ahrav
cec1543894
Add utf16 decoder proto. (#1276) 2023-04-20 15:25:36 -07:00
Miccah
dfc5a9f5db
[chore] Log possible duplicate detectors (#1266) 
* [chore] Log possible duplicate detectors

* Fix typos
 2023-04-18 10:36:00 -05:00
Zachary Rice
1c89e79c2d
Remove toLower call on decoded chunk (#1254) 
* remove to lower on decoded data

* clean up
 2023-04-14 07:29:32 -05:00
iamjpotts
b3d917f9c7
Resolve #1167 by adding support for the AWS_SESSION_TOKEN (#1170) 
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg

* fix error message

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-04-03 14:56:43 -07:00
ahrav
0052f60090
Allow for custom verifier (#1070) 
* allow for custom verifier.

* Update engine.

* use custom detectors.

* set cap.

* Update verifiers.

* Remove nil check.

* resolved nit

* handle uppercase values

* updating missing url logs

* adding more descriptive variable names

* updating logs to use correct variables

* Removing toLower for urls

* if else nits

* Adding versioning for github and gitlab

---------

Co-authored-by: ahmed <ahmed.zahran@trufflesec.com>
Co-authored-by: ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d <13666360+0x1@users.noreply.github.com>
 2023-03-29 12:26:39 -07:00
Zachary Rice
fb9ae75661
Support for exclude globs at the git log level (#1202) 
* init

* seems to be working

* better comment

* rm conditional

* Add more context to exclude-globs description
 2023-03-28 10:46:03 -05:00
garg472
3e4496156c
added new detectors and fixed mesibo detector (#1166) 
* added new detectors and fixed mesibo detector

* added bscscan.com API detector

* added coinmarketcap detector

* update alchemy

* update blocknative

* update bscscan test

* update cmc test

* update tests

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-03-16 18:57:08 -07:00
Yassine Ilmi
d382d5cb1c
Add OpenAI API Tokens detector (#1142) 
* Add OpenAI Detector

* Add OpenAI Detector tests

* Add OpenAI Detector to defaults.go

* Removing references to github detector in tests
 2023-03-16 17:58:22 -07:00
ahrav
c617bd7a4e
Add resuming capability to GCS source (#1161) 
* Add resuming capability to GCS source.

* Handle no auth scans.

* complete resume logic

* Use custom function type.

* remove functions.

* linter.

* fix test.

* fix test.

* Handle concurrent map writes.

* use string as CLI flag for include/exclude.

* handle emtpy buckets.

* Handle enumeration on initial job run.

* Rename stats to attributes.

* remove redundant return.

* If test fails due to 400, that is fine, it's expected.

* Add unauth GCS source type.

* comments.

* update proto.

* Use short flag.

* address comments.
 2023-03-16 17:53:42 -07:00
ahrav
6193509098
add support for json service account and service account file. (#1185) 2023-03-16 13:04:36 -07:00
Zachary Rice
f0b6b5d0d9
add a break statement when iterating through keywords (#1184) 2023-03-15 16:51:03 -05:00
ahrav
cbf299aa77
Add gcs scanning integration (#1153) 
* Setup for GCS scanning.

* Update GCS engine w/ projectID req.

* Add concurrency field to gcsManager.

* add errgroup to gcsManager.

* Update gcs manager.

* Use defautl ADC.

* use ADC.'

* Add TOOD.

* add log to iterator completion.

* use a BinaryReader instead of concrete object for channel type.

* initial test for Chunks.

* Add tests for chunking objects.

* Add concurrency.

* update metadata to include content type and acls.

* Add object reading code.

* Add integration test.

* Add entrypoint.

* Add removed wg.Wait().

* remove dead code.

* remove build.

* Remove period from file extension.

* remove used.

* Add comment.

* Setup for GCS scanning.

* Update GCS engine w/ projectID req.

* Add concurrency field to gcsManager.

* add errgroup to gcsManager.

* Update gcs manager.

* Use defautl ADC.

* use ADC.'

* Add TOOD.

* add log to iterator completion.

* use a BinaryReader instead of concrete object for channel type.

* initial test for Chunks.

* Add tests for chunking objects.

* Add concurrency.

* update metadata to include content type and acls.

* Add object reading code.

* Add integration test.

* Add entrypoint.

* Add removed wg.Wait().

* remove dead code.

* remove build.

* remove used.

* Add file type for objects.

* Add check for file type and size.

* Add default file size.

* Add additinoal auth options and remaining CLI flags.

* Handle errors in go routines.

* Handle resuming for buckets.

* Remove redundant words in comment.

* remove ok check on bool check.

* remove extra blank line.

* Add return if handler handles chunk.

* Add comment.

* remove extra blank line.

* cleanup comment.

* Add comment.

* move up fxn.

* go mod tidy.

* Add exclusion to perf testing buckets.

* Handle blocking the channel.

* remove unused const.

* fix tests.

* fix tests.

* Handle gcs manger options better.

* update fxn name.

* Remove arg name.

* ignore buckets in gcsManager test.

* fix test.

* propulate gsManagerOpts.

* inline err check.

* Add readme.

* update readme spelling.

* fix test.
 2023-03-07 17:32:04 -08:00
Zachary Rice
4777b77ec6
Keyword optimization (#1144) 
* init

* ignore trufflehog binary and added comment

* remove unused keywords in chunk, better comment

* remove keywords from engine struct
 2023-03-02 11:32:37 -06:00
Miccah
dd39848709
Add ability to include and exclude detectors (#1106) 
* Add ability to include and exclude detectors

* Trim space before checking for empty items

* Explicitly check for integer overflow

* Use strconv.ParseInt instead of strconv.Atoi

* Address comments
 2023-02-27 16:46:45 -06:00
Miccah
c5b4d6f28b
Support file scanning in filesystem source (#1030) 
* Rename directories to paths

* Generate protos

* Add file scanning support to filesystem source

* Add directories back to filesystem proto

* Generate protos

* Combine paths and directories from in source

* Add filesystem filter

* Address comments
 2023-02-27 12:15:05 -06:00
SAYGIN Metin
f2139a7615
Github filter support for exclude and include (#1087) 
* test

* Add missing head and base hash back.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2023-02-14 08:40:53 -08:00
trufflesteeeve
4f13090c01
Remove duplicated detectors (#1092) 
In this case just Heroku and LinearAPI. But this includes the Moonclerck
detector, which appears to be a typo that got turned into a separate
detector type.

Co-authored-by: zubairk14 <zubair.khan@trufflesec.com>
 2023-02-13 11:44:19 -05:00
Dustin Decker
0c66d30c1f
Revert "Make detectors configurable (#1084)" (#1097) 
This reverts commit 67784f6928.
 2023-02-11 08:12:13 -08:00
ahrav
67784f6928
Make detectors configurable (#1084) 
* Make detectors configurable.

* remove redundant check.

* add number of detectors.

* update comment.

* remove reflect.

* inline key.

* replace name w/ type.

* remove temp var.

* fix test name.

* fix engine start.

* add filter unverified to engine.

* reorder engine args.

* Address comments.

* Add include and exclude.

* update comments.

* add comment.

* add comment.
 2023-02-10 16:30:38 -08:00
ahrav
c5c8d10d28
[chore] - Remove monolithic config struct (#1091) 
* REmove monolithic config struct.

* fix broken test.
 2023-02-10 12:43:00 -08:00
Miccah
d317ddb51a
[chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources (#1089) 
* [chore] Remove logrus from circleci, filesystem, gitlab, and s3 sources

* Address comments
 2023-02-10 11:02:55 -06:00
Miccah
0ce72ccda3
[chore] Remove logrus from github source (#1086) 
* [chore] Remove logrus from github source

* Fix handleRateLimit test

* Fix tests
 2023-02-09 18:02:04 -06:00
Miccah
58e8c1e4ac
[chore] Remove logrus from engine package (#1085) 2023-02-09 16:55:19 -06:00
Alexandr Marchenko
b29b78c10d
filesystem support for exclude and include filters (2nd attemp) (#1033) 
* fix filter issue - empty lines should be ignored

* filesystem support for filter exclude

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-01-26 09:33:45 -08:00
ahrav
319ae64a02
[chore] - Small cleanup of CircleCi source (#1028) 
* Small cleanup of CircleCi source.

* address comments.

* Add context to methods as first param.
 2023-01-17 09:36:18 -08:00
Bill Rich
430d5c764c
Rename and export isGitSource (#1016) 2023-01-10 12:51:58 -08:00
Bill Rich
8b2e1d36cf
Copy metadata for line number aware sources (#1011) 
* Copy metadata for line number aware sources

* Improve style
 2023-01-10 09:35:44 -08:00
Dustin Decker
5f6143f09a
Add Circle CI source (#997) 
* Add Circle CI source

* remove SHA1 line

* remove trim
 2023-01-05 21:44:37 -08:00
ahrav
936a139596
Allow using a glob for include list. (#977) 
* Allow using a glob for include list.

* Update command flag.

* Make comment more clear.

* update comment.

* Allow scanning repo and org at the same time.
 2022-12-16 13:28:16 -08:00
Bill Rich
335ce85ce4
Export line number code (#962) 2022-12-06 15:31:15 -08:00
Thiago Lages de Alencar
9757c339d9
Fix error message (#933) 2022-11-20 05:31:11 -08:00
Dustin Decker
ae4b387448
add LDAP detector (#896) 2022-11-18 19:45:11 -08:00
Dustin Decker
b18edef01a
Enable skipping of particular key IDs (#930) 
* Enable skipping of particular key IDs

* update test
 2022-11-18 09:09:40 -08:00
Jessica
3d501975e4
Add filter as scan option to gitlab module's git scan (#919) 2022-11-15 13:02:37 -08:00
Bill Rich
d3b24fa592
Replace plain decoder with utf8 (#922) 2022-11-15 09:36:01 -08:00
kstilwell
ecd25784f5
Adding Shopify detector (#875) 
* Fixes/work based on testing

* Remove some commented code

* Change how verification happens and grab additional information

* Address linter warnings.

* add shopify detector to default detectors.

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2022-11-08 16:21:57 -08:00
ahrav
dd141fb55f
[oc-147] - Add context to all git methods (#901) 
* Add context to all git methods.

* remove logrus.

* Add ctx.

* Address comments.

* Add error to clone failing.

* Return error.
 2022-11-03 16:36:52 -07:00
Dustin Decker
a7fc12240f
Do local URI verification, while attempting to defuse SSRF (#879) 
* simplify monogo pattern

* do URI verification locally, while attempting to defuse SSRF

* test SSRF defuse

* simplify err check logic per linter recommendation

* split up detectors

* address comments

* remove unused var
 2022-11-01 17:27:24 -07:00
ahrav
fe029b1098
[THOG-793] - Return all unverified results (#856) 
* Remove the check to filter and return only a single unverified result.

* Revert "Remove the check to filter and return only a single unverified result."

This reverts commit 494e432803.

* Add new CLI flag to filter unverified results.
 2022-10-31 09:36:10 -07:00
Bill Rich
034ca4fb5b
Add bytes counter to scans (#876) 2022-10-27 12:54:22 -07:00
Alexandr Marchenko
60464da3ce
proposal: SqlServer connection string detector (#867) 
* sqlserver added to detectors.proto

* make protos

* boilerplate detector generated

* wireup

* initial
 2022-10-26 07:46:13 -07:00
Bill Rich
958266ea84
Run chunker in pipeline (#859) 
* Run chunker in pipeline

* Move ChunkSize and PeekSize to source package.

* Use new Chunk and Peek size location
 2022-10-24 13:57:27 -07:00
Ankush Goel
d29357c9d4
added npm detector (#841) 2022-10-13 06:04:02 -07:00
Dustin Decker
85467538f6
remove faulty detector (#836) 2022-10-07 09:20:44 -07:00
ahrav
128002885a
Add decoder type to results. (#835) 2022-10-06 11:55:07 -07:00
Miccah
2bc4985061
Add SSH config option for the git source (#830) 
* Add SSH config option for the git source

The auth message is empty since we use the git binary underneath to
handle the SSH authentication.

* Import digitaloceanv2
 2022-09-28 20:40:01 +02:00
Mildred Bernardo
ad4b9406a7
Added digitaloceanv2 detector (#829) 
* Added digitaloceanv2 detector

* import detector

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-09-28 09:56:35 -07:00
ahrav
db42bcf2a2
[OC-103] - Add Gemini detector (#800) 
* Add Gemini detector.

* Add regex and test code for Gemini detector.

* Remove else.

* Add commentary.

* Address comments.

* Use regular else.

* Make nice and complicated.

* use regular detection pattern.

* Add detector to default detectors.
 2022-09-26 11:48:48 -07:00
Bill Rich
e3107ad6bb
Move head and base normalization to source (#818) 2022-09-23 08:58:45 -07:00
ahrav
f2f5b6dba8
Fix recover. (#817) 2022-09-22 15:47:42 -07:00
ahrav
92f40c2031
[THOG-709] - Recover from detector panics (#810) 2022-09-22 07:01:10 -07:00
Dustin Decker
e8f1bb9127 Import NGC detector 2022-09-21 16:26:58 -07:00
Bill Rich
509cf8b6fa
Use headref and check empty commits for base (#815) 2022-09-21 16:04:01 -07:00
Dustin Decker
fcd580406e
A few improvements (#809) 
* Run integration tests

* Update examples

* Import mongodb
 2022-09-19 13:23:25 -07:00
Dustin Decker
fa9479100e
Add common sentry recover library and add into goroutines (#738) 
* Add common sentry recover library and add into goroutines

* fix nits
 2022-08-29 11:45:37 -07:00
Dustin Decker
2452e93a80
Import 27 new detectors (#737) 2022-08-26 12:35:06 -07:00
ahrav
73f9d3f0a0
[chore] - Use config struct instead of pointer for engine scans. (#709) 
* Use a config struct instead of pointer when scanning engine sources.

* use config.
 2022-08-12 09:56:24 -07:00
ahrav
dcc102a81c
[Thog-371] Utilize config struct for engine scans (#700) 
* Use a config struct when scanning and engine source.

* fix tests.

* Move test_helpers to the sources pkg.

* Handle ScanGit error in tests.

* adderss comments.

* Use functional options.

* Remove temp var.

* Add better var names for the setup functions for each config.

* Remove unused var.

* fix error logs.

* fix error logs.

* single line.

* remove blank lines.
 2022-08-10 10:11:13 -07:00
ahrav
30ebe84e3e
[THOG-608] - Fix linter errors. (#701) 
* Fix linter errors.

* Fix gist adding test.

* Update test string for mock JSON reply.

* Remove if.
 2022-08-09 19:20:02 -07:00
trufflesteeeve
176552b07a
Fix commit attribution, git tests, and run make protos (#667) 
* Update dependency to fix commit attribution, fix git tests

* Run make protos to match code with current proto definitions
 2022-07-25 11:44:15 -04:00
Dustin Decker
c2426df4d6
Disable sentiment verifier and update teams webhook to avoid being stateful (#646) 2022-07-07 16:55:45 -07:00
Miccah
6fa2171a22
Refactor Engine to wait for workers in a Finish method (#581) 
* Refactor Engine to wait for workers in a Finish method

This should allow the engine to run multiple concurrent scans if
desired before shutting down.

Additionally, this commit refactors some of the printing logic to the
output package.

* Fix tests
 2022-05-25 11:35:44 -05:00
Dustin Decker
235a60a850
add more detectors (#577) 2022-05-23 18:39:49 -07:00
Dustin Decker
606dadfbb1
Import more detectors (#563) 2022-05-17 18:56:48 -07:00
ahrav
d2605354fe
[THOG-332 ]Remove TokenSource interface from the init method of Source. (#539) 
* Remove TokenSource interface from the init method of Source.

* Remove proto message.

* Remove proto message.

* Fix tests.

* Fix filesystem test.
 2022-05-13 14:35:06 -07:00
ahrav
b0d79180f6
[THOG-314] Add new parameter to the Init method for the source interface. (#529) 
* Add new parameter to the Init method for the source interface.

* Add Oauth Token service.

* remove .test file.

* remove .test file.

* Fix param spelling.

* fix tests with new param in init

* Add missing gock lib.
 2022-05-10 11:11:43 -07:00
ahrav
e12432cef8
[THOG-315] Replace bytes.buffer with strings.builder. (#533) 
* Replace bytes.buffer with string.builder.

* Remove profiling.

* Remove detector changes.

* ignore .test files.

* fix detectors removed.
 2022-05-09 17:02:46 -07:00
Bill Rich
6ee5c57968
Fix overlooked nil (#523) 
* Fix overlooked nil

* Use default case
 2022-05-04 15:45:12 -07:00
Bill Rich
c78120e56f
Syslog source (#500) 
* Add syslog source

* only load cert/key with tls

* Cleanup

* Linting

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
 2022-05-04 15:08:11 -07:00
Bill Rich
62bb3c57d3
Keep first line number for each chunk (#520) 2022-05-04 14:11:10 -07:00
Dustin Decker
d217a517c0
Cleanup packages and include more detectors (#521) 
* clean up duplicate packages

* include more detectors
 2022-05-04 09:27:42 -07:00
Dustin Decker
913c75db15 Add 12 more detecotrs 2022-04-21 15:31:17 -07:00
Dustin Decker
9622932d81
Add 12 detectors (#432) 2022-04-19 12:32:00 -07:00
Dustin Decker
972108aea4
21 new detectors (#347) 2022-04-08 16:07:39 -07:00
Dustin Decker
8d2dd624e4 fetcher controls fetch interval, prevent fetching loop 2022-04-03 21:13:39 -07:00
Dustin Decker
a1dfcde9a6
address detector issues (#123) 2022-04-01 18:51:41 -07:00
Miccah
78b344d7b8
Add --include-members flag to scan all members of an organization (#118) 
* Add --member-repos flag to scan all members of an organization

* Move flag to the githubScan sub-command

Co-authored-by: Miccah Castorina <miccah.castorina@trufflesec.com>
 2022-04-01 18:22:37 -07:00
ahrav
cedb3393d1
[THOG-128] Code cleanup/ OSS onboarding (#117) 
* Small amount of code clean up.

* Rename sem to concurrency for better readability and to remove an extra comment.

* fix stashing issue.

Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
 2022-04-01 16:47:27 -07:00
Bill Rich
c2b4f0bc39
Include line numbers in git metadata (#97) 
* Include line numbers in git metadata

* Update tests for fragments and line numbers
 2022-03-22 09:27:15 -07:00
Bill Rich
0ea3b93238
Use git cmd for clone (#96) 
* Use git command to clone

* Check for git command when cloning
 2022-03-22 09:26:58 -07:00
trufflesteeeve
d33551e7dc
Add midise detector (#88) 2022-03-17 13:38:05 -07:00
Bill Rich
e8234c3514
Remove unused code and add git binary check (#80) 2022-03-14 17:47:18 -07:00
Dustin Decker
72c13c4b8f
Update detector process and readme (#79) 2022-03-14 17:27:14 -07:00
Bill Rich
0ee34a5be6
Use gitleaks git cmd stream (#75) 
* Switch to git cmd stream

* Fix rebase issues
 2022-03-14 17:12:58 -07:00
Bill Rich
6486c18565
Add s3 support to CLI (#76) 
* Add s3 support to CLI

* Clean up comments

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-03-14 17:07:07 -07:00
Bill Rich
2cb5b27d3b
Add GitLab support to CLI (#78) 
* Add GitLab support to CLI

* Update cli message

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-03-14 17:05:15 -07:00
Bill Rich
8d1db04c36
Add filesystem scan to CLI (#77) 
* Add filesystem scan to CLI

* Fix linter errors

* Update message

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-03-14 17:04:19 -07:00
Dustin Decker
43de9c3604 Update defaults 2022-03-11 17:25:31 -08:00
Dustin Decker
b4ac103d11 Add snifftest 2022-03-11 10:24:36 -08:00
Dustin Decker
b0e0c26137 update detectors 2022-03-11 10:24:36 -08:00
Bill Rich
55b839fc5a
Only scan files that changed from base to head (#68) 
* Only scan files that changed from base to head

* Acknowledge ignored errors
 2022-03-08 08:48:00 -08:00
Bill Rich
bdbc7e80d1
Use sync.Map to avoid concurrent map writes (#66) 2022-03-05 22:42:06 -08:00
Bill Rich
665b0bf928 Add timestamp (#61) 
* Add timestamp to sources

* Include timestamp in sources
 2022-03-04 08:39:23 -08:00
Bill Rich
9f378b8cb3 Resolve a ref as arg for --since_commit (#57) 2022-03-04 08:39:23 -08:00
Dustin Decker
6f32b108a3 Disable GetEmails 2022-03-04 08:39:22 -08:00
Bill Rich
c144630c54 Add parent results to ignore list (#47) 
* Add parent results to ignore list

* Force concurrency to 1 when base commit is set
 2022-03-04 08:39:22 -08:00
Dustin Decker
1da3e0f723 turn off getemail scanner 2022-03-04 08:39:22 -08:00
Bill Rich
30034f5d28 Use metadata specific to source type (#48) 2022-03-04 08:39:22 -08:00
Bill Rich
56dc1b109c Check commit order, dedupe results, and support using a head commit. (#44) 
* Check commit order and support using a head commit.

* Only apply dedupe to git bases source
 2022-03-04 08:39:22 -08:00
Dustin Decker
c20e9f4732 improvements 2022-03-04 08:39:17 -08:00
Dustin Decker
77418fb3f8 module v3 2022-02-15 18:54:47 -08:00
Dustin Decker
79496b8142 Add context timeout to scanners 2022-02-15 18:54:47 -08:00
Dustin Decker
ee99f0c9a2 Record avg detector time 2022-02-15 18:54:47 -08:00
Dustin Decker
152ef6d4e1 add include forks option (#37) 2022-02-15 18:54:47 -08:00
Dustin Decker
a96de1a2cd update docs and support multi platform docker images 2022-02-15 18:54:47 -08:00
Dustin Decker
e15fa3a5be helpful logging 2022-02-15 18:54:47 -08:00
Bill Rich
206b99704b Change log order and path filtering. 2022-02-15 18:54:47 -08:00
Bill Rich
28ed0c3b7c Complete support for existing git scan flags (#13) 
* Add `since_commit` to git scan

* Support `max_depth` option for git scan

* Use new options in github and gitlab sources

* Address review feedback

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
 2022-02-15 18:54:47 -08:00
Dustin Decker
089b5c9063 improved plain output 2022-02-15 18:54:47 -08:00
Dustin Decker
03ead2f7ed Integrate GitHub source 2022-02-15 18:54:47 -08:00
Dustin Decker
533f6d1e20 updates 2022-02-15 18:54:43 -08:00
Dustin Decker
d6ffadb1ee
Initial docs and release automation (#5) 2022-01-18 16:59:18 -08:00
Bill Rich
a70937bfe6
Support remote git repos using https (#9) 
Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
 2022-01-14 16:07:45 -08:00
Bill Rich
1422cc303a
Include and exclude paths args for gitscan (#6) 
* include and exclude paths gitscan args

Add support for include_paths and exclude_paths arguments when scanning
git sources.

* Improve variable name

Co-authored-by: Bill Rich <hrich@Bills-MacBook-Pro.local>
 2022-01-14 12:40:50 -08:00
Dustin Decker
4218c39d99
Initial CLI w/ partially implemented Git source and demo detector (#1) 2022-01-13 12:02:24 -08:00