Dustin Decker
10b6e2898d
Increase log level of engine messages ( #1576 )
2023-07-28 14:30:43 -07:00
Zachary Rice
1a1977f7e6
case insensitive ( #1547 )
2023-07-25 17:01:15 -05:00
Zachary Rice
85f363f093
init ( #1538 )
2023-07-24 19:09:57 -05:00
Dustin Decker
fab80445d1
continue scanning on detector / decoder panic ( #863 )
2023-07-24 07:34:43 -07:00
Miccah
91c5472876
Implement SourceManager basics ( #1515 )
...
* Implement SourceManager basics
* Rename identifiers and add a default headlessAPI implementation
* Rewrite to use SourceInitFunc
* Update variable name to accurately reflect its value
2023-07-21 15:20:25 -05:00
Miccah
a613bbb979
[chore] Remove parent manipulation in context package ( #1525 )
...
The ability to set the parent allowed creating context cycles which
shouldn't be allowed, or at the very least have unintuitive behavior.
2023-07-21 13:51:51 -05:00
Miccah
e8b5e3cea3
Revert "[chore] Remove parent setting / getting in Context wrapper ( #1516 )" ( #1519 )
...
This reverts commit 8ec5e4916c
2023-07-20 23:31:28 -05:00
Miccah
8ec5e4916c
[chore] Remove parent setting / getting in Context wrapper ( #1516 )
...
* [chore] Remove parent setting / getting in Context wrapper
* Keep the cancellable context from errgroup
2023-07-20 13:33:09 -05:00
Brandon Yan
8fad5fff79
add dockerhub scanner ( #1496 )
...
* add dockerhub scanner
* clean
* clean and fix regex logic and tests
* check length of userMatches before access
* Use camelcase.
---------
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-19 09:26:28 -07:00
Brandon Yan
17b90b32d0
add couchbase scanner to defaults ( #1497 )
...
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-16 17:45:56 -07:00
Brandon Yan
9af31f00a9
add envoy api key scanner ( #1482 )
...
* add envoy api key scanner
* Use detectors4.
---------
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-16 16:46:28 -07:00
ahrav
a9213a1103
[chore] - Update loop to switch. ( #1487 )
...
* Update loop to switch.
* remove unused fxn.
2023-07-12 15:47:43 -07:00
Zachary Rice
b48ac24c46
Dedupe results ( #1479 )
...
* init 4 dedupin
* use raw rather than rawv2
* rm comment
* comments
* nits
* clean up and use rawv2 too
* add decoder order test
2023-07-11 15:48:00 -05:00
Zachary Rice
d4972313ff
remove old detector ( #1474 )
2023-07-10 13:02:19 -05:00
Zachary Rice
0bdd513d88
additional similarity check for base64 and plain ( #1462 )
...
* additional similarity check for base64 and plain
* use bytes equal
* move logic into util function
2023-07-10 10:12:59 -05:00
Richard Gomez
23757dbe0a
remove image4 detector ( #1461 )
2023-07-06 12:56:09 -07:00
Zachary Rice
18a70b64bb
Introduce trufflehog:ignore tag feature ( #1433 )
...
* init ignore
* cleanup and add test
* update readme
2023-06-29 08:45:56 -05:00
roxanne-tampus
00920984e3
added opsgenie detector ( #650 )
...
* added opsgenie detector
* update interface and import
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-06-27 16:43:25 -07:00
Miccah
945c27cb82
Fix docker source to return any chunk errors ( #1429 )
2023-06-26 12:12:46 -05:00
Brendan Shaklovitz
da5301ea1e
Exit with non-zero exit code on chunk source error ( #1286 )
...
* Exit with non-zero exit code on chunk source error
* Exit with a non-zero exit code whenever we hit an error getting
chunks. Previously the error would be logged but trufflehog would exit
with a 0 (success) status code.
* fix gcs test
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: ahrav <ahravdutta02@gmail.com>
2023-06-26 11:39:57 -05:00
Dustin Decker
e856a6890d
🎉 Add Docker image scanning 🎉 ( #1412 )
...
* Add Docker source
* Add metrics
* Add test
* Add debugging, address PR comments, fix path output
* review suggestions
2023-06-22 08:02:25 -07:00
Zachary Rice
74ffbd2878
add a custom detector check for logging duplicate detector ( #1394 )
...
* add a custom detector check for logging duplicate detector
* use pb type
2023-06-13 14:49:21 -05:00
Zubair Khan
dfb1a0cd38
Add DocuSign detector ( #1382 )
...
* init
* look for client id and client secret, encode them for basis auth
* add tests
* test without checking the contents of response
* confirm access_token exists
* cleanup test
* explain in code that an undocumented grant_type is used
* remove use of deprecated ioutil, remove dead code, return errors instead of just logging
* directly pull access token
* update error text, remove redundant body close()
* import new detector into defaults
2023-06-08 13:34:50 -04:00
ahrav
1da7720912
Replace context.TODO. ( #1349 )
2023-05-19 11:09:51 -07:00
ahrav
e81b908e07
Add buildkitev2 detector for newer tokens. ( #1341 )
2023-05-15 12:58:36 -07:00
vickygoel
4c04bbbe85
added pulumi cloud Access token detector ( #1295 )
...
* added pulumi cloud Access token detector
* removed accidentally committed tokens
* added the databricks token detection
* made recommended changes
* added supabase management api token
* nuget api key detector
* added aiven.io token detector
* added prefect.io api key detector
* update protos.
---------
Co-authored-by: Developer <garg47294+1@gmail.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-05-11 09:08:48 -07:00
Brendan Shaklovitz
584db86031
Support line numbers in filesystem source ( #1297 )
2023-05-09 08:02:34 -07:00
Brendan Shaklovitz
fad34d4dc6
git worktree scanning fix for #827 ( #1315 )
...
* Fix worktree scan by setting EnableDotGitCommonDir
* Change `PlainOpenOptions` to set `EnableDotGitCommonDir` to true.
In every current usage of this function, it is on an already-cloned
repository, so it should always be valid to have this set. By doing
so, it should fix some issues with worktrees.
* Remove unused go.mod replace directives
* Remove replace directives for libraries that are not in use.
2023-05-09 08:00:47 -07:00
Jason Solis
c13c56283d
add tineswebhook detector ( #1304 )
2023-05-01 07:48:58 -07:00
Miccah
b1675194ca
Implement EndpointCustomizer ( #1291 )
...
* Implement EndpointCustomizer
Add the EndpointCustomizer interface and EndpointSetter convenience struct,
implement EndpointCustomizer for github and gitlab detectors, and add
parsing, verification, and applying user-supplied configuration.
* Check error from SetEndpoints
* Rename variable for clarity
2023-04-27 12:23:50 -05:00
Miccah
5a86c18302
Fix include and exclude detector logic ( #1267 )
...
* Fix include and exclude detector logic
* Fix test
* Add more clarifying comments
2023-04-26 10:49:54 -05:00
Shabbir B
6f801f64c7
Added a new detector for percy.io ( #1284 )
...
* Feature: Added a new detector for percy.io
* Updated variable name
---------
Co-authored-by: ahrav <ahravdutta02@gmail.com>
2023-04-25 13:18:34 -07:00
ahrav
cec1543894
Add utf16 decoder proto. ( #1276 )
2023-04-20 15:25:36 -07:00
Miccah
dfc5a9f5db
[chore] Log possible duplicate detectors ( #1266 )
...
* [chore] Log possible duplicate detectors
* Fix typos
2023-04-18 10:36:00 -05:00
Zachary Rice
1c89e79c2d
Remove toLower call on decoded chunk ( #1254 )
...
* remove to lower on decoded data
* clean up
2023-04-14 07:29:32 -05:00
iamjpotts
b3d917f9c7
Resolve #1167 by adding support for the AWS_SESSION_TOKEN ( #1170 )
...
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg
* fix error message
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-03 14:56:43 -07:00
ahrav
0052f60090
Allow for custom verifier ( #1070 )
...
* allow for custom verifier.
* Update engine.
* use custom detectors.
* set cap.
* Update verifiers.
* Remove nil check.
* resolved nit
* handle uppercase values
* updating missing url logs
* adding more descriptive variable names
* updating logs to use correct variables
* Removing toLower for urls
* if else nits
* Adding versioning for github and gitlab
---------
Co-authored-by: ahmed <ahmed.zahran@trufflesec.com>
Co-authored-by: ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d <13666360+0x1@users.noreply.github.com>
2023-03-29 12:26:39 -07:00
Zachary Rice
fb9ae75661
Support for exclude globs at the git log level ( #1202 )
...
* init
* seems to be working
* better comment
* rm conditional
* Add more context to exclude-globs description
2023-03-28 10:46:03 -05:00
garg472
3e4496156c
added new detectors and fixed mesibo detector ( #1166 )
...
* added new detectors and fixed mesibo detector
* added bscscan.com API detector
* added coinmarketcap detector
* update alchemy
* update blocknative
* update bscscan test
* update cmc test
* update tests
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-03-16 18:57:08 -07:00
Yassine Ilmi
d382d5cb1c
Add OpenAI API Tokens detector ( #1142 )
...
* Add OpenAI Detector
* Add OpenAI Detector tests
* Add OpenAI Detector to defaults.go
* Removing references to github detector in tests
2023-03-16 17:58:22 -07:00
ahrav
c617bd7a4e
Add resuming capability to GCS source ( #1161 )
...
* Add resuming capability to GCS source.
* Handle no auth scans.
* complete resume logic
* Use custom function type.
* remove functions.
* linter.
* fix test.
* fix test.
* Handle concurrent map writes.
* use string as CLI flag for include/exclude.
* handle emtpy buckets.
* Handle enumeration on initial job run.
* Rename stats to attributes.
* remove redundant return.
* If test fails due to 400, that is fine, it's expected.
* Add unauth GCS source type.
* comments.
* update proto.
* Use short flag.
* address comments.
2023-03-16 17:53:42 -07:00
ahrav
6193509098
add support for json service account and service account file. ( #1185 )
2023-03-16 13:04:36 -07:00
Zachary Rice
f0b6b5d0d9
add a break statement when iterating through keywords ( #1184 )
2023-03-15 16:51:03 -05:00
ahrav
cbf299aa77
Add gcs scanning integration ( #1153 )
...
* Setup for GCS scanning.
* Update GCS engine w/ projectID req.
* Add concurrency field to gcsManager.
* add errgroup to gcsManager.
* Update gcs manager.
* Use defautl ADC.
* use ADC.'
* Add TOOD.
* add log to iterator completion.
* use a BinaryReader instead of concrete object for channel type.
* initial test for Chunks.
* Add tests for chunking objects.
* Add concurrency.
* update metadata to include content type and acls.
* Add object reading code.
* Add integration test.
* Add entrypoint.
* Add removed wg.Wait().
* remove dead code.
* remove build.
* Remove period from file extension.
* remove used.
* Add comment.
* Setup for GCS scanning.
* Update GCS engine w/ projectID req.
* Add concurrency field to gcsManager.
* add errgroup to gcsManager.
* Update gcs manager.
* Use defautl ADC.
* use ADC.'
* Add TOOD.
* add log to iterator completion.
* use a BinaryReader instead of concrete object for channel type.
* initial test for Chunks.
* Add tests for chunking objects.
* Add concurrency.
* update metadata to include content type and acls.
* Add object reading code.
* Add integration test.
* Add entrypoint.
* Add removed wg.Wait().
* remove dead code.
* remove build.
* remove used.
* Add file type for objects.
* Add check for file type and size.
* Add default file size.
* Add additinoal auth options and remaining CLI flags.
* Handle errors in go routines.
* Handle resuming for buckets.
* Remove redundant words in comment.
* remove ok check on bool check.
* remove extra blank line.
* Add return if handler handles chunk.
* Add comment.
* remove extra blank line.
* cleanup comment.
* Add comment.
* move up fxn.
* go mod tidy.
* Add exclusion to perf testing buckets.
* Handle blocking the channel.
* remove unused const.
* fix tests.
* fix tests.
* Handle gcs manger options better.
* update fxn name.
* Remove arg name.
* ignore buckets in gcsManager test.
* fix test.
* propulate gsManagerOpts.
* inline err check.
* Add readme.
* update readme spelling.
* fix test.
2023-03-07 17:32:04 -08:00
Zachary Rice
4777b77ec6
Keyword optimization ( #1144 )
...
* init
* ignore trufflehog binary and added comment
* remove unused keywords in chunk, better comment
* remove keywords from engine struct
2023-03-02 11:32:37 -06:00
Miccah
dd39848709
Add ability to include and exclude detectors ( #1106 )
...
* Add ability to include and exclude detectors
* Trim space before checking for empty items
* Explicitly check for integer overflow
* Use strconv.ParseInt instead of strconv.Atoi
* Address comments
2023-02-27 16:46:45 -06:00
Miccah
c5b4d6f28b
Support file scanning in filesystem source ( #1030 )
...
* Rename directories to paths
* Generate protos
* Add file scanning support to filesystem source
* Add directories back to filesystem proto
* Generate protos
* Combine paths and directories from in source
* Add filesystem filter
* Address comments
2023-02-27 12:15:05 -06:00
SAYGIN Metin
f2139a7615
Github filter support for exclude and include ( #1087 )
...
* test
* Add missing head and base hash back.
---------
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-02-14 08:40:53 -08:00
trufflesteeeve
4f13090c01
Remove duplicated detectors ( #1092 )
...
In this case just Heroku and LinearAPI. But this includes the Moonclerck
detector, which appears to be a typo that got turned into a separate
detector type.
Co-authored-by: zubairk14 <zubair.khan@trufflesec.com>
2023-02-13 11:44:19 -05:00
Dustin Decker
0c66d30c1f
Revert "Make detectors configurable ( #1084 )" ( #1097 )
...
This reverts commit 67784f6928
2023-02-11 08:12:13 -08:00
