Mirrors/trufflehog
2
0
Fork 0
mirror of https://github.com/trufflesecurity/trufflehog.git synced 2024-11-10 07:04:24 +00:00
Code Issues Projects Releases Packages Wiki Activity
3150 commits 139 branches 257 tags 147 MiB
Commit graph

208 commits

Author SHA1 Message Date
Cody Rose
b803a0f701
Report indeterminacy in AWS verifier (#1480) 2023-07-11 15:50:31 -04:00
Zubair Khan
4334af4d34
scan GitHub PR and issue comments (#1435) 
* issue comment scanning

* save progress

* test

* test for pr comment and issue comment

* add pagination support

* linter stuff

* make linter happy

* remove debug log

* readd logging

* github issue resolved

* var const block and handle rate limit

* remove magic number

* make gitURLParse a public function to use more generally

* fix test bug

* make comment scanning OPT-IN
 2023-07-11 15:13:33 -04:00
Cody Rose
87058dd7fa
Add new verification error message field (#1463) 2023-07-10 11:15:40 -04:00
Zubair Khan
f52946b996
Add Couchbase Detector (#1385) 
* init

* add detector type

* rotate leaked credentials

* tighten up username pattern

* isolated prefixregex as overrriding new line stuff

* passwordPat working now

* add username test

* fix edge case

* cleanup

* make linter happy

* make linter happy rd 2

* skip error logging

* fix test

* add password regex helper func

* make test more robust

* cleanup PR

* remove comments

* clarify prepend rationale
 2023-06-26 14:37:10 -04:00
Dustin Decker
e856a6890d
🎉 Add Docker image scanning 🎉 (#1412) 
* Add Docker source

* Add metrics

* Add test

* Add debugging, address PR comments, fix path output

* review suggestions
 2023-06-22 08:02:25 -07:00
Zubair Khan
dfb1a0cd38
Add DocuSign detector (#1382) 
* init

* look for client id and client secret, encode them for basis auth

* add tests

* test without checking the contents of response

* confirm access_token exists

* cleanup test

* explain in code that an undocumented grant_type is used

* remove use of deprecated ioutil, remove dead code, return errors instead of just logging

* directly pull access token

* update error text, remove redundant body close()

* import new detector into defaults
 2023-06-08 13:34:50 -04:00
Dustin Decker
4250773e92
GitHub basic auth (#1337) 2023-05-15 22:04:42 -07:00
vickygoel
4c04bbbe85
added pulumi cloud Access token detector (#1295) 
* added pulumi cloud Access token detector

* removed accidentally committed tokens

* added the databricks token detection

* made recommended changes

* added supabase management api token

* nuget api key detector

* added aiven.io token detector

* added prefect.io api key detector

* update protos.

---------

Co-authored-by: Developer <garg47294+1@gmail.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2023-05-11 09:08:48 -07:00
Brendan Shaklovitz
584db86031
Support line numbers in filesystem source (#1297) 2023-05-09 08:02:34 -07:00
Miccah
6699ccd2b5
Generate protos (#1329) 2023-05-04 12:26:41 -05:00
Jason Solis
c13c56283d
add tineswebhook detector (#1304) 2023-05-01 07:48:58 -07:00
Brendan Shaklovitz
10902f802a
Add max object size flag for s3 bucket scanning (#1294) 
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-04-26 15:39:43 -07:00
ahrav
622700b6ec
update proto to allow for ignoring projects. (#1289) 2023-04-26 07:30:43 -07:00
Shabbir B
6f801f64c7
Added a new detector for percy.io (#1284) 
* Feature: Added a new detector for percy.io

* Updated variable name

---------

Co-authored-by: ahrav <ahravdutta02@gmail.com>
 2023-04-25 13:18:34 -07:00
ahrav
cec1543894
Add utf16 decoder proto. (#1276) 2023-04-20 15:25:36 -07:00
Zubair Khan
6dd24d17d4
Switch Endpoint Field to Client ID (#1270) 
* no longer using endpoint, but do need client id

* use oauth2 credential type
 2023-04-19 16:54:37 -04:00
ahrav
5b2b434a8a
Allow multipel team IDs. (#1259) 2023-04-19 11:47:25 -07:00
ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
1c1f034468
Adding Google drive to MetaData proto (#1264) 2023-04-19 10:49:40 -04:00
ahrav
531e3ef6c3
Add team name to proto. (#1258) 2023-04-14 09:07:23 -07:00
Miccah
562f3aaa2a
Generate protos (#1250) 2023-04-10 17:15:53 -05:00
Zubair Khan
2c1d8fbba4
THOG-920/add oss proto (#1240) 
* add sharepoint source proto

* create sharepoint oss protos

* add email field, remove oauth2 type, update token to refresh_token

* rename path to link

* restore clientcredentials

* restore s3 and confluence proto changes from make command
 2023-04-10 10:30:47 -07:00
iamjpotts
b3d917f9c7
Resolve #1167 by adding support for the AWS_SESSION_TOKEN (#1170) 
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg

* fix error message

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-04-03 14:56:43 -07:00
ahrav
dfc38a135a
Add oauth2 cred as auth type for Teams. (#1221) 2023-03-29 19:37:41 -07:00
ahrav
b8467ee17e
Add Oauth creds to GCS. (#1212) 2023-03-27 10:29:21 -07:00
garg472
3e4496156c
added new detectors and fixed mesibo detector (#1166) 
* added new detectors and fixed mesibo detector

* added bscscan.com API detector

* added coinmarketcap detector

* update alchemy

* update blocknative

* update bscscan test

* update cmc test

* update tests

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2023-03-16 18:57:08 -07:00
ahrav
c617bd7a4e
Add resuming capability to GCS source (#1161) 
* Add resuming capability to GCS source.

* Handle no auth scans.

* complete resume logic

* Use custom function type.

* remove functions.

* linter.

* fix test.

* fix test.

* Handle concurrent map writes.

* use string as CLI flag for include/exclude.

* handle emtpy buckets.

* Handle enumeration on initial job run.

* Rename stats to attributes.

* remove redundant return.

* If test fails due to 400, that is fine, it's expected.

* Add unauth GCS source type.

* comments.

* update proto.

* Use short flag.

* address comments.
 2023-03-16 17:53:42 -07:00
ahrav
6193509098
add support for json service account and service account file. (#1185) 2023-03-16 13:04:36 -07:00
ahrav
17e8d7030e
Add unauth GCS source type. (#1178) 2023-03-13 16:54:45 -07:00
ahrav
cbf299aa77
Add gcs scanning integration (#1153) 
* Setup for GCS scanning.

* Update GCS engine w/ projectID req.

* Add concurrency field to gcsManager.

* add errgroup to gcsManager.

* Update gcs manager.

* Use defautl ADC.

* use ADC.'

* Add TOOD.

* add log to iterator completion.

* use a BinaryReader instead of concrete object for channel type.

* initial test for Chunks.

* Add tests for chunking objects.

* Add concurrency.

* update metadata to include content type and acls.

* Add object reading code.

* Add integration test.

* Add entrypoint.

* Add removed wg.Wait().

* remove dead code.

* remove build.

* Remove period from file extension.

* remove used.

* Add comment.

* Setup for GCS scanning.

* Update GCS engine w/ projectID req.

* Add concurrency field to gcsManager.

* add errgroup to gcsManager.

* Update gcs manager.

* Use defautl ADC.

* use ADC.'

* Add TOOD.

* add log to iterator completion.

* use a BinaryReader instead of concrete object for channel type.

* initial test for Chunks.

* Add tests for chunking objects.

* Add concurrency.

* update metadata to include content type and acls.

* Add object reading code.

* Add integration test.

* Add entrypoint.

* Add removed wg.Wait().

* remove dead code.

* remove build.

* remove used.

* Add file type for objects.

* Add check for file type and size.

* Add default file size.

* Add additinoal auth options and remaining CLI flags.

* Handle errors in go routines.

* Handle resuming for buckets.

* Remove redundant words in comment.

* remove ok check on bool check.

* remove extra blank line.

* Add return if handler handles chunk.

* Add comment.

* remove extra blank line.

* cleanup comment.

* Add comment.

* move up fxn.

* go mod tidy.

* Add exclusion to perf testing buckets.

* Handle blocking the channel.

* remove unused const.

* fix tests.

* fix tests.

* Handle gcs manger options better.

* update fxn name.

* Remove arg name.

* ignore buckets in gcsManager test.

* fix test.

* propulate gsManagerOpts.

* inline err check.

* Add readme.

* update readme spelling.

* fix test.
 2023-03-07 17:32:04 -08:00
Miccah
c5b4d6f28b
Support file scanning in filesystem source (#1030) 
* Rename directories to paths

* Generate protos

* Add file scanning support to filesystem source

* Add directories back to filesystem proto

* Generate protos

* Combine paths and directories from in source

* Add filesystem filter

* Address comments
 2023-02-27 12:15:05 -06:00
ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
ee5b028c67
Adding initial protos for Google Drive scanner (#1121) 2023-02-22 10:04:46 -05:00
Dustin Decker
26c455d302
add more confluence options (#1105) 2023-02-13 13:58:02 -08:00
ahrav
3f30216356
Add incclude exclude spaces for confluence source. (#1073) 2023-02-06 08:49:45 -08:00
ahrav
e85411b59f
Add file to confluence proto. (#1049) 2023-01-26 17:34:01 -08:00
ahrav
2088f030f9
Add location to Teams metadata. (#1034) 2023-01-23 08:12:05 -08:00
ahrav
936a139596
Allow using a glob for include list. (#977) 
* Allow using a glob for include list.

* Update command flag.

* Make comment more clear.

* update comment.

* Allow scanning repo and org at the same time.
 2022-12-16 13:28:16 -08:00
Miccah
861ad057c7
Implement CustomRegex detector (#950) 
* Remove verifying successRanges because it is unused in webhook

* Move custom_detectors validation code into its own file

* Initial implementation of custom regex detector

Secret verification is done via webhook.

* Add CustomRegex detector type

* Add upper bound to permutation

* Return early if the context is canceled

* Add headers from configuration

* Add detector name as a key in the JSON body

* Implement faster algorithm for productIndices
 2022-12-14 10:26:53 -06:00
Miccah
4409210b87
Add custom detectors configuration parsing (#927) 
* Add custom_detectors proto

* Generate proto code

* Create custom_detectors package

Also create protoyaml package to test YAML unmarshalling the
configuration.

* Simplify custom_detectors proto by removing connection

* Generate proto code

* Update custom_detectors parsing tests
 2022-11-21 15:10:38 -06:00
Jessica
6e25664a52
add rambbitmq detector (#936) 
* add rambbitmq detector

* use fixed length redaction

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-11-21 10:47:41 -08:00
Dustin Decker
ae4b387448
add LDAP detector (#896) 2022-11-18 19:45:11 -08:00
kstilwell
ecd25784f5
Adding Shopify detector (#875) 
* Fixes/work based on testing

* Remove some commented code

* Change how verification happens and grab additional information

* Address linter warnings.

* add shopify detector to default detectors.

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
 2022-11-08 16:21:57 -08:00
trufflesteeeve
e8cd2e7fae
Add Unknown visibility level (#902) 2022-11-04 14:28:20 -04:00
Dustin Decker
3a143f095b
add option to skip tls verification for confluence (#900) 2022-11-03 13:10:01 -07:00
Dustin Decker
a7fc12240f
Do local URI verification, while attempting to defuse SSRF (#879) 
* simplify monogo pattern

* do URI verification locally, while attempting to defuse SSRF

* test SSRF defuse

* simplify err check logic per linter recommendation

* split up detectors

* address comments

* remove unused var
 2022-11-01 17:27:24 -07:00
Alexandr Marchenko
60464da3ce
proposal: SqlServer connection string detector (#867) 
* sqlserver added to detectors.proto

* make protos

* boilerplate detector generated

* wireup

* initial
 2022-10-26 07:46:13 -07:00
ahrav
c203eef86f
[THOG-767] - Ignore Bitbucket and Gitlab repos (#852) 
* Add messages to BB and Gitlab source protos to allow ignoring repos.

* remove unsued field in struct.j

* Fix casing.
 2022-10-18 14:14:04 -07:00
ahrav
cea2a23c56
[THOG-768] - Add ignore repo list to Github proto (#843) 
* Add ignore repo list to Github proto.

* Add proto.

* Add missing proto.
 2022-10-11 15:41:33 -07:00
ahrav
128002885a
Add decoder type to results. (#835) 2022-10-06 11:55:07 -07:00
Miccah
2bc4985061
Add SSH config option for the git source (#830) 
* Add SSH config option for the git source

The auth message is empty since we use the git binary underneath to
handle the SSH authentication.

* Import digitaloceanv2
 2022-09-28 20:40:01 +02:00
Mildred Bernardo
80dcfbe9db
Added DigitalOceanV2 detector (#828) 2022-09-27 17:51:10 -07:00
trufflesteeeve
02310a64f3
Add token auth to JIRA proto (#824) 2022-09-27 15:39:51 -04:00
Bill Rich
1c00014051
Include public/private in github metadata (#812) 
* Include public/private in github metadata

* CR feedback

* Fix typos and naming
 2022-09-26 14:55:46 -07:00
Joseph Lucas
b02cf7e032
Adding detector for Nvidia NGC (#797) 
* template

* minimum viable regex

* valid api 401

* passing tests

* snake to camelcase
 2022-09-20 08:20:18 -07:00
Miccah
59d6d29c02
Add location to Slack and Confluence metadata (#802) 
Location indicates where in the source a secret was found. For example,
Slack could be in a message or in an attachment.
 2022-09-16 22:58:14 +02:00
trufflesteeeve
57e46f9b76
Add SlackRealtime proto message (#803) 2022-09-16 16:49:51 -04:00
ahrav
c4492b1fdc
Add support for MongoDB detector. (#793) 
* Add support for MongoDB detector.

* Remove extra line.

* Remove unused arg.

* Add context around found secret test.

* Remove unused arg.
 2022-09-15 05:47:09 -07:00
ahrav
e9599db240
[THOG-690] - Add visibility to slack metadata proto message (#796) 2022-09-15 05:01:45 -07:00
roxanne-tampus
90da460fa1
added new detector (#765) 2022-08-31 17:54:23 -07:00
Mildred Bernardo
4c3c103b62
added new detectors (#761) 2022-08-31 11:50:33 -07:00
Marlon
a35786dccd
fix and make_protos (#757) 2022-08-30 17:13:04 -07:00
roxanne-tampus
fa2d6b90cd
added new detectors (#743) 2022-08-29 16:44:11 -07:00
Marlon
098d4a9e7d
added appointed scanner (#425) 
* added appointed scanner

* fix comment

* fix comment

* fix comment

* fix issue

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-08-25 10:40:35 -07:00
Max Thomson
e9f4cf99e5
Add Honeycomb detector (#687) 
* Add Honeycomb detector

* Update pattern

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-08-24 16:50:31 -07:00
trufflesteeeve
25082e63a6
Add the slack client token to the slack tokens proto (#717) 2022-08-16 16:12:46 -04:00
ahrav
dfb7e9a405
Add hash v2 to results proto. (#693) 2022-08-04 16:37:15 -07:00
trufflesteeeve
176552b07a
Fix commit attribution, git tests, and run make protos (#667) 
* Update dependency to fix commit attribution, fix git tests

* Run make protos to match code with current proto definitions
 2022-07-25 11:44:15 -04:00
roxanne-tampus
e9f503a083
added new detectors (#639) 
* added new detectors

* added gemini
 2022-07-08 08:19:03 -07:00
ahrav
59fc54b94a
[Thog-441] create public event monitoring source (#599) 
* Add source for public event monitoring.

* Update protos.

* Add source for public event monitoring.

* Update metadata.

* Update proto.

* Update proto.

* Update pb files.
 2022-06-03 14:46:31 -07:00
Miccah
d7d5db9e78
Replace AccessToken protobuf with a simpler string type (#594) 2022-05-31 14:28:07 -05:00
ahrav
d2605354fe
[THOG-332 ]Remove TokenSource interface from the init method of Source. (#539) 
* Remove TokenSource interface from the init method of Source.

* Remove proto message.

* Remove proto message.

* Fix tests.

* Fix filesystem test.
 2022-05-13 14:35:06 -07:00
ahrav
b0d79180f6
[THOG-314] Add new parameter to the Init method for the source interface. (#529) 
* Add new parameter to the Init method for the source interface.

* Add Oauth Token service.

* remove .test file.

* remove .test file.

* Fix param spelling.

* fix tests with new param in init

* Add missing gock lib.
 2022-05-10 11:11:43 -07:00
Bill Rich
c78120e56f
Syslog source (#500) 
* Add syslog source

* only load cert/key with tls

* Cleanup

* Linting

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
 2022-05-04 15:08:11 -07:00
Marlon
48a0c28d33
added new protos (#495) 
* added new protos

* fix comment
 2022-05-02 09:23:09 -07:00
Dustin Decker
28d5396e61
Pr/371 (#490) 
* added paydirtyapp scanner

* change paydirtyapp to paydirtapp

Co-authored-by: Marlon Pamisa <marlonpamisa@gmail.com>
 2022-04-28 23:39:35 -07:00
Dustin Decker
40a2d8c9f4
Pr/478 (#489) 
* added nightfall detector

* fix protos and improve pattern

Co-authored-by: Mildred Tosoc <mildredtosoc@gmail.com>
 2022-04-28 23:11:48 -07:00
Marlon
5aaa60e418
added new protos (#445) 2022-04-21 21:46:56 -07:00
roxanne-tampus
0dedefdd1b
Added new detectors (#443) 2022-04-21 20:56:07 -07:00
roxanne-tampus
3e0e1da232
Renamed GTmetrix detector (#436) 
* added new protos

* added new detectors

* Renamed mispelled detector
 2022-04-21 18:02:05 -07:00
Dustin Decker
272dacaed3
Recharge payments detector Pr/381 (#430) 
* Add RechargePayments to detectors

* First pass at code and tests for RechargePayments detector

* Running make protos

* Fixes based on running tests

Co-authored-by: Kevin Stilwell <kevin.stilwell@gmail.com>
 2022-04-18 21:51:27 -07:00
Marlon
3e25996c08
fix formcraft proto (#427) 2022-04-18 10:56:01 -07:00
Marlon
6d3f27b89f
added new protos (#412) 2022-04-15 08:19:33 -07:00
roxanne-tampus
0971db82f3
Added new detectors (#400) 
* added new protos

* added new detectors
 2022-04-14 17:18:34 -07:00
Mildred Bernardo
313ab5df22
added new protos (#394) 
* Merge branch 'protos' of https://github.com/ladybug0125/trufflehog into protos

* make protos

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
 2022-04-13 23:01:11 -07:00
ahrav
ad82a8e52c
Update the Teams message proto to include access token auth. (#389) 
* Update the Teams message proto to include access token auth.

* Add team_id field to the Teams message proto.
 2022-04-12 10:55:17 -07:00
Miccah
c6b537feb3
Change Artifactory credential to AccessToken (#387) 
* Change Artifactory credential to AccessToken

* Generate protos
 2022-04-12 10:54:45 -05:00
Marlon
b33376fc16
Feature/add protos1 (#364) 2022-04-10 07:49:46 -07:00
Bill Rich
bf5c757cd3
Add head and base support to github (#346) 2022-04-08 14:32:54 -07:00
roxanne-tampus
62d08fc7d4
Added mockaroo scanner (#320) 
* added new protos

* added mockaroo & updated mispelled detector
 2022-04-08 14:13:49 -07:00
Dustin Decker
37fce75c4a generate protos 2022-04-08 00:20:48 -07:00
roxanne-tampus
d25638979f
added new protos (#318) 2022-04-07 23:00:09 -07:00
Dustin Decker
ea516711ae
update protos image and fix it for linux/amd64 (#316) 2022-04-07 18:10:23 -07:00
Miccah
3c4a152e6e
Add JFrog Artifactory protobuf source and metadata (#293) 
* Add JFrog Artifactory protobuf source and metadata

* Add generated protobuf code

* Update metadata to include an email

* Generate protobuf code
 2022-04-05 16:38:48 -07:00
ahrav
24dacc67fb
Add source protos for Microsoft Teams scanner integration. (#284) 2022-04-04 10:48:43 -07:00
Bill Rich
0744a54aa7
Use ignoreList instead of denyChannel (#98) 2022-03-22 09:27:35 -07:00
Bill Rich
c2b4f0bc39
Include line numbers in git metadata (#97) 
* Include line numbers in git metadata

* Update tests for fragments and line numbers
 2022-03-22 09:27:15 -07:00
Bill Rich
239a9422c4
Add DenyChannel to slack source (#90) 2022-03-18 10:38:20 -07:00
trufflesteeeve
d33551e7dc
Add midise detector (#88) 2022-03-17 13:38:05 -07:00
Dustin Decker
4b64f9a377 fix protos 2022-03-15 19:48:38 -07:00
Dustin Decker
43de9c3604 Update defaults 2022-03-11 17:25:31 -08:00
Bill Rich
665b0bf928 Add timestamp (#61) 
* Add timestamp to sources

* Include timestamp in sources
 2022-03-04 08:39:23 -08:00
trufflesteeeve
fa57727370 190 Add spaces scope to confluence - default to all spaces (#62) 2022-03-04 08:39:23 -08:00
Dustin Decker
7f7e2665a0 Change BasicAuthWithHeader to just Header 2022-03-04 08:39:22 -08:00
Dustin Decker
8b500219f8 Add new credential type for Jenkins 2022-03-04 08:39:22 -08:00
Dustin Decker
c20e9f4732 improvements 2022-03-04 08:39:17 -08:00
Dustin Decker
77418fb3f8 module v3 2022-02-15 18:54:47 -08:00
Dustin Decker
152ef6d4e1 add include forks option (#37) 2022-02-15 18:54:47 -08:00
Dustin Decker
533f6d1e20 updates 2022-02-15 18:54:43 -08:00
Dustin Decker
4218c39d99
Initial CLI w/ partially implemented Git source and demo detector (#1) 2022-01-13 12:02:24 -08:00