Commit graph

125 commits

Author SHA1 Message Date
Dylan Ayrey
f13fe36ae2
adding azure storage detector (#1840)
* adding azure storage detector

* Fix variable name

* Escape regex

* fix test fields and update expected status code

---------

Co-authored-by: counter <counter@counters-MacBook-Air.local>
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-10-02 13:45:45 -07:00
joeleonjr
699547b7d3
consolidated pr and issue descr/comment flags (#1827) 2023-09-27 15:54:02 -04:00
Ankush Goel
faf46175e4
added Web3 Storage detector (#1789)
* added Web3 Storage detector

* fixed the regex

* removed test and disabled token
2023-09-27 12:09:39 -05:00
joeleonjr
1e42dae734
added PR and Issue body scanning (#1816)
* added PR and Issue body scanning; adjusted CLI args to fit

* removed print statement from debugging

* removed exclude-commits; adjusted CLI flags

* minor changes to match main branch

* fixing logic

* updating README for --issues and --prs
2023-09-26 12:25:48 -04:00
Marwan Sulaiman
3aa5369608
Add Tailscale detector (#1719)
* Add tailscale detector

* PR feedback: match on first element
2023-09-07 19:11:17 -07:00
s.shivasurya
6695cf1dce
added sourcegraph token verification detection (#1730) 2023-08-31 08:47:13 -07:00
Zubair Khan
519646342e
add snowflake detector (#1653)
Detect Snowflake secrets (compound URI of account, username, password) and enrich Secret Result with account and databases that the secret has access to.
2023-08-24 13:29:58 -04:00
Mike Vanbuskirk
64dd49f9ce
add role assumption for s3 source (#1477)
* add role assumption for s3 source

* refactor role assumption to repeatable string

user can pass array of roles to assume

* refactor s3 chunks to handle passed roleARNs

* add role-session name

use timestamp to make dynamic

* add docstring for rolearn strings()

* make sure role ars are passed into source

* refactor role assumption functionality

break s3 bucket scanning into sep. function

* add log check on assume role

* fix role iteration

- Make sure s3 struct is populated with roles
- add separate new client instantiation for role-based access
- iterates through each role

* add comment

* protobuf revert for merge

* re-run make proto

* lint cleanup

* cleanup TODOs

* drop redundant switch case in assumerole client

* use less verbose 'ctx' designator

* breakout functionality from Chunks

- separate functions for:
- enumerating buckets to scan
- scanning objects within the buckets

* remake protobuf defs

* allow scan to continue on single bucket err

* add readme docs

* minor fixups
2023-08-17 20:30:20 -04:00
Zubair Khan
62d359eba4
add salesforce detector (#1608)
* setup

* update time out case to return detector result

* fix

* remove unneeded comment

* remove debug print

* cleanup

* more robust error handling

* reflect new detector template changes

* fixes

* mark response body check err as indeterminate
2023-08-16 10:42:04 -04:00
Zubair Khan
ea6e8b6bb5
add huggingface detector (#1621)
* init huggingface detector

* completed test
2023-08-14 14:22:04 -04:00
Bill Rich
0c7ed19270
Github Oauth2 verification (#1584)
* Github Oauth2 verification

* Use prefix and include RawV2

* Make gh_oauth2 a new detector

* Remove unused struct

* Remove versioner

* Remove unused code
2023-08-02 11:16:40 -07:00
Richard Gomez
e0faac8d1c
Fix runtime error when scanning Gist comments (#1552)
* fix(github): fix runtime error from gist comments

* fix(github): add flag to scan Gist comments
2023-07-31 08:57:42 -05:00
Zubair Khan
9f3809f19e
gdrive proto change (#1566) 2023-07-28 10:38:51 -04:00
ahrav
ade5d91d5c
Add azure repos protos. (#1559) 2023-07-26 19:53:10 -07:00
Brandon Yan
8fad5fff79
add dockerhub scanner (#1496)
* add dockerhub scanner

* clean

* clean and fix regex logic and tests

* check length of userMatches before access

* Use camelcase.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-19 09:26:28 -07:00
Zubair Khan
be549a7287
add thog enterprise detector for web keys (#1448)
* saving progress

* proto changes

* run make protos

* verify response, add test case

* resolve linter warning about unescaped . in regex pattern

* resolve overlapping proto number
2023-07-18 09:53:12 -04:00
Brandon Yan
9af31f00a9
add envoy api key scanner (#1482)
* add envoy api key scanner

* Use detectors4.

---------

Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-07-16 16:46:28 -07:00
Cody Rose
b803a0f701
Report indeterminacy in AWS verifier (#1480) 2023-07-11 15:50:31 -04:00
Zubair Khan
4334af4d34
scan GitHub PR and issue comments (#1435)
* issue comment scanning

* save progress

* test

* test for pr comment and issue comment

* add pagination support

* linter stuff

* make linter happy

* remove debug log

* readd logging

* github issue resolved

* var const block and handle rate limit

* remove magic number

* make gitURLParse a public function to use more generally

* fix test bug

* make comment scanning OPT-IN
2023-07-11 15:13:33 -04:00
Cody Rose
87058dd7fa
Add new verification error message field (#1463) 2023-07-10 11:15:40 -04:00
Zubair Khan
f52946b996
Add Couchbase Detector (#1385)
* init

* add detector type

* rotate leaked credentials

* tighten up username pattern

* isolated prefixregex as overrriding new line stuff

* passwordPat working now

* add username test

* fix edge case

* cleanup

* make linter happy

* make linter happy rd 2

* skip error logging

* fix test

* add password regex helper func

* make test more robust

* cleanup PR

* remove comments

* clarify prepend rationale
2023-06-26 14:37:10 -04:00
Dustin Decker
e856a6890d
🎉 Add Docker image scanning 🎉 (#1412)
* Add Docker source

* Add metrics

* Add test

* Add debugging, address PR comments, fix path output

* review suggestions
2023-06-22 08:02:25 -07:00
Zubair Khan
dfb1a0cd38
Add DocuSign detector (#1382)
* init

* look for client id and client secret, encode them for basis auth

* add tests

* test without checking the contents of response

* confirm access_token exists

* cleanup test

* explain in code that an undocumented grant_type is used

* remove use of deprecated ioutil, remove dead code, return errors instead of just logging

* directly pull access token

* update error text, remove redundant body close()

* import new detector into defaults
2023-06-08 13:34:50 -04:00
Dustin Decker
4250773e92
GitHub basic auth (#1337) 2023-05-15 22:04:42 -07:00
vickygoel
4c04bbbe85
added pulumi cloud Access token detector (#1295)
* added pulumi cloud Access token detector

* removed accidentally committed tokens

* added the databricks token detection

* made recommended changes

* added supabase management api token

* nuget api key detector

* added aiven.io token detector

* added prefect.io api key detector

* update protos.

---------

Co-authored-by: Developer <garg47294+1@gmail.com>
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-05-11 09:08:48 -07:00
Brendan Shaklovitz
584db86031
Support line numbers in filesystem source (#1297) 2023-05-09 08:02:34 -07:00
Miccah
6699ccd2b5
Generate protos (#1329) 2023-05-04 12:26:41 -05:00
Jason Solis
c13c56283d
add tineswebhook detector (#1304) 2023-05-01 07:48:58 -07:00
Brendan Shaklovitz
10902f802a
Add max object size flag for s3 bucket scanning (#1294)
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-26 15:39:43 -07:00
ahrav
622700b6ec
update proto to allow for ignoring projects. (#1289) 2023-04-26 07:30:43 -07:00
Shabbir B
6f801f64c7
Added a new detector for percy.io (#1284)
* Feature: Added a new detector for percy.io

* Updated variable name

---------

Co-authored-by: ahrav <ahravdutta02@gmail.com>
2023-04-25 13:18:34 -07:00
ahrav
cec1543894
Add utf16 decoder proto. (#1276) 2023-04-20 15:25:36 -07:00
Zubair Khan
6dd24d17d4
Switch Endpoint Field to Client ID (#1270)
* no longer using endpoint, but do need client id

* use oauth2 credential type
2023-04-19 16:54:37 -04:00
ahrav
5b2b434a8a
Allow multipel team IDs. (#1259) 2023-04-19 11:47:25 -07:00
ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
1c1f034468
Adding Google drive to MetaData proto (#1264) 2023-04-19 10:49:40 -04:00
ahrav
531e3ef6c3
Add team name to proto. (#1258) 2023-04-14 09:07:23 -07:00
Miccah
562f3aaa2a
Generate protos (#1250) 2023-04-10 17:15:53 -05:00
Zubair Khan
2c1d8fbba4
THOG-920/add oss proto (#1240)
* add sharepoint source proto

* create sharepoint oss protos

* add email field, remove oauth2 type, update token to refresh_token

* rename path to link

* restore clientcredentials

* restore s3 and confluence proto changes from make command
2023-04-10 10:30:47 -07:00
iamjpotts
b3d917f9c7
Resolve #1167 by adding support for the AWS_SESSION_TOKEN (#1170)
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg

* fix error message

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-03 14:56:43 -07:00
ahrav
dfc38a135a
Add oauth2 cred as auth type for Teams. (#1221) 2023-03-29 19:37:41 -07:00
ahrav
b8467ee17e
Add Oauth creds to GCS. (#1212) 2023-03-27 10:29:21 -07:00
garg472
3e4496156c
added new detectors and fixed mesibo detector (#1166)
* added new detectors and fixed mesibo detector

* added bscscan.com API detector

* added coinmarketcap detector

* update alchemy

* update blocknative

* update bscscan test

* update cmc test

* update tests

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-03-16 18:57:08 -07:00
ahrav
c617bd7a4e
Add resuming capability to GCS source (#1161)
* Add resuming capability to GCS source.

* Handle no auth scans.

* complete resume logic

* Use custom function type.

* remove functions.

* linter.

* fix test.

* fix test.

* Handle concurrent map writes.

* use string as CLI flag for include/exclude.

* handle emtpy buckets.

* Handle enumeration on initial job run.

* Rename stats to attributes.

* remove redundant return.

* If test fails due to 400, that is fine, it's expected.

* Add unauth GCS source type.

* comments.

* update proto.

* Use short flag.

* address comments.
2023-03-16 17:53:42 -07:00
ahrav
6193509098
add support for json service account and service account file. (#1185) 2023-03-16 13:04:36 -07:00
ahrav
17e8d7030e
Add unauth GCS source type. (#1178) 2023-03-13 16:54:45 -07:00
ahrav
cbf299aa77
Add gcs scanning integration (#1153)
* Setup for GCS scanning.

* Update GCS engine w/ projectID req.

* Add concurrency field to gcsManager.

* add errgroup to gcsManager.

* Update gcs manager.

* Use defautl ADC.

* use ADC.'

* Add TOOD.

* add log to iterator completion.

* use a BinaryReader instead of concrete object for channel type.

* initial test for Chunks.

* Add tests for chunking objects.

* Add concurrency.

* update metadata to include content type and acls.

* Add object reading code.

* Add integration test.

* Add entrypoint.

* Add removed wg.Wait().

* remove dead code.

* remove build.

* Remove period from file extension.

* remove used.

* Add comment.

* Setup for GCS scanning.

* Update GCS engine w/ projectID req.

* Add concurrency field to gcsManager.

* add errgroup to gcsManager.

* Update gcs manager.

* Use defautl ADC.

* use ADC.'

* Add TOOD.

* add log to iterator completion.

* use a BinaryReader instead of concrete object for channel type.

* initial test for Chunks.

* Add tests for chunking objects.

* Add concurrency.

* update metadata to include content type and acls.

* Add object reading code.

* Add integration test.

* Add entrypoint.

* Add removed wg.Wait().

* remove dead code.

* remove build.

* remove used.

* Add file type for objects.

* Add check for file type and size.

* Add default file size.

* Add additinoal auth options and remaining CLI flags.

* Handle errors in go routines.

* Handle resuming for buckets.

* Remove redundant words in comment.

* remove ok check on bool check.

* remove extra blank line.

* Add return if handler handles chunk.

* Add comment.

* remove extra blank line.

* cleanup comment.

* Add comment.

* move up fxn.

* go mod tidy.

* Add exclusion to perf testing buckets.

* Handle blocking the channel.

* remove unused const.

* fix tests.

* fix tests.

* Handle gcs manger options better.

* update fxn name.

* Remove arg name.

* ignore buckets in gcsManager test.

* fix test.

* propulate gsManagerOpts.

* inline err check.

* Add readme.

* update readme spelling.

* fix test.
2023-03-07 17:32:04 -08:00
Miccah
c5b4d6f28b
Support file scanning in filesystem source (#1030)
* Rename directories to paths

* Generate protos

* Add file scanning support to filesystem source

* Add directories back to filesystem proto

* Generate protos

* Combine paths and directories from in source

* Add filesystem filter

* Address comments
2023-02-27 12:15:05 -06:00
ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
ee5b028c67
Adding initial protos for Google Drive scanner (#1121) 2023-02-22 10:04:46 -05:00
Dustin Decker
26c455d302
add more confluence options (#1105) 2023-02-13 13:58:02 -08:00
ahrav
3f30216356
Add incclude exclude spaces for confluence source. (#1073) 2023-02-06 08:49:45 -08:00