Commit graph

1770 commits

Author SHA1 Message Date
Abdul Basit
a43d451c4d
[analyze] Add Analyzer for SourceGraph (#3173)
* implement analyzer interface for sourcegraph

* created permission for sourcegraph
test for sourcegraph.
added email in resource metadata.

* handling of missing keys in map

* linked sourcegraph detector to analyzer

* update the fullyqualidied name of resource to make it unique.
updated the test.

* add current user email in metadata

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:40:47 -07:00
Abdul Basit
4cab071032
[analyze] Add Analyzer for Asana (#3139)
* impelmented analyzer interface with data models for Asana

* add unit test for asana analyzer

* link asana detector with analyzer

* added permission for asana
linked detector with only positive cases.

* to make test cleaner moved want json in external file.
Moreover without sorting test will not be able to compare.

* use general functions to avoid code duplication.
optimize app permission making logic.

* [fix] assigned bindings to results.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:37:45 -07:00
Abdul Basit
f235b8a442
[analyze] Add Analyzer for Slack (#3207)
* implement analyzer interface for slack

* slack analyzer adjusted for new changes in main, unit test added

* link detector with analyzer for slack

* added generated permissions for slack analyzer

* generate permission fix, keep dot in permissions intact

* removed scope from permission and put it metadata.

* [chore]
moved expected output of test in json file to neat the code.
added team id in fully qualified name of user resource.
check permissions before adding it in bindings.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:33:06 -07:00
Abdul Basit
5d7e6fc2fa
[analyze] Add Analyze interface for Mailchimp (#3225)
* implement analyzer interface for mailchimp

* link detector with analyzer for mailchimp

* fix analyzer type

* add mailchimp analyzer test

* [chore] appended string to make fulllyqualifiedname as unique.

* [chore]
moved expected output of test in json file to neat the code.
removed PII information in metadata.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-05 16:33:47 -07:00
Abdul Basit
899f59fbb4
[analyze] Add Analyzer for Sendgrid (#3174)
* implement analyzer interface for sendgrid

* add unit test for sendgrid analyzer

* fixed sendgrid detector.
linked analyzer with sendgrid detector.
handling if key not found in map

* category as resource.
if subcategory is present then subcategory will become resource with parent category

* corrected test and remove hardcoded boolean for printing in sendgrid

* incorporate code refactoring and suggestion for FullyQualifiedName for subcategory by Miccah.

* generate permissions for sendgrid analyzer

* [NIT] rather than updating the global scopes variable, ProcessPermission will return new list of categories with Permission and eliminate those which are not in generated one.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-04 15:50:32 -07:00
Abdul Basit
d6e1627f16
[analyze] Add Analyzer for Opsgenie (#3181)
* implement analyzer interface for opsgenie and add unit tests

* Add analyzer interface for opsgenie

linked detector  with analyzers
fixed test cases.

* generate permissions for opsgenie and change scope names to lowercase for consistency

* fixed unboundedresources slice issue.
username as fullqualifiedname

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-04 15:50:19 -07:00
Abdul Basit
5ce1578a6f
[analyze] Add analyzer for Postman (#3180)
* implement analyzer interface for postman and add unit test

* analyzer interface inplementation for postman

linked detector with analyzer for postman
add permission for postman

* [fix] linter in postman

* considered Miccah comments about fullyqualifiedName and code refactoring.

* moved want result to expected output file.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-04 15:40:12 -07:00
0x1
7eb5b5b12c
add rotation links (#3257) 2024-09-04 10:45:04 -04:00
Dustin Decker
6bbb683ead
Reduce high freq keywords (#3265) 2024-09-03 16:58:46 -07:00
Shreyas Sriram
15faaba61c
Add Robinhood Crypto detector (#3254)
* Add Robinhood Crypto detector

* Address comment - use single keyword
2024-08-29 14:05:52 -07:00
Hon
247b56ad0b
update rotation guide link for teams (#3248) 2024-08-26 14:34:18 -07:00
Cody Rose
f39a5254ff
Customize results cleaning (using smuggled interface) (#3235)
We have identified some cases in which the results "cleaning" logic (the logic that eliminates superfluous results) should not run. In order to allow this, we need to expose the cleaning logic to the engine. This PR does so by doing these things:

- Create a CustomResultsCleaner interface that can be implemented by detectors that want to use custom cleaning logic
- Implement this interface for the aws and awssessionkey detectors (and remove their previous invocation of their custom cleaning logic)
- Modify the engine to invoke this logic (conditionally)

This PR also removes the "custom" cleaning logic for the opsgenie, razorpay, and twilio detectors, because it was added erroneously.

This is an alternative implementation of #3233.
2024-08-21 09:42:20 -04:00
Dustin Decker
fe5624c709
Improve domain / url handling in detectors (#3221)
* Strip path and params and use new client

* update clients

* additional client updates

* revert client
2024-08-15 11:34:28 -07:00
0x1
8cf1ec2824
remove two letter keyword (#3210) 2024-08-13 09:09:36 -05:00
Miccah
1df83f79ef
[analyze] Separate SID from token in twilio analyzer (#3177)
* [analyze] Separate SID from token in twilio analyzer

* Fix test

* Set sid in detector
2024-08-05 17:46:57 -07:00
Richard Gomez
f335d486ef
Update Zulip detector (#2897)
* fix(zulip): prevent false positives

* update extra data

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-08-04 09:30:15 -07:00
Dustin Decker
88359addc5
update pattern (#3167) 2024-08-04 09:12:09 -07:00
ahrav
0a3451a1ba
[bug] - Create a new context with timeout per request (#3163)
* Create a new context with timeout per request

* match timeout

* use context timeout

* reduce timeout
2024-08-02 14:46:37 -07:00
ahrav
c549b5bd15
[bug] - add context timeout to ssh verification (#3161)
* add context timeout to ssh verification

* fix test
2024-08-02 12:39:50 -07:00
Abdul Basit
04a13385a8
Add Analyzers interface for HuggingFace (#3140)
* implemented analyzer interface with data models for HuggingFace

* correct test for huggingface due to new addition of key in detection result.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-08-02 08:20:11 -07:00
Abdul Basit
c1645e8c27
Add Analyzers interface for Square (#3141)
* implement analyzer interface for square

* linked detector with analyzer for square
fix test for square.

* code refactoring

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-08-02 08:00:25 -07:00
Dustin Decker
605d037e45
add twilio analyze relationships (#3148)
* add twilio analyze relationships

* unused struct
2024-08-01 17:04:44 -07:00
Abdul Basit
843120427d
Add Analyze interface to Stripe (#3132)
* implement analyzer interface for stripe

* consider cateogry as unbound resource if there is no permission with it.

* check for key existence in map.
pass on analysis info from Stripe detector.
test change to remove analysis info.

* remove Valid boolean from metadata of analyzer result

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-08-01 08:23:06 -07:00
Dustin Decker
a3d3565248
Add new canary ID (#3117) 2024-07-30 20:44:58 -07:00
Cody Rose
ed8bc501e5
Use non-canary credentials for AWS tests (#3109)
The AWS detector verifies credentials in a weird hacky way to work around some non-obvious STS behavior. This workaround does not work for canary tokens, so I updated the test secrets to use non-canary tokens. This PR updates the tests to match the secrets file changes.
2024-07-29 10:11:27 -04:00
Dustin Decker
c048487739
Support openai project and fine grained tokens (#3112) 2024-07-26 15:31:17 -07:00
Miccah
9d089c2188
[analyze] Implement Analyzer interface for github (#3110)
* [analyze] Implement Analyzer interface for github

* Make github repo and user enumeration configurable

* Add AnalysisInfo to github detector

* Use AnalyzeAndPrintPermissions from the CLI
2024-07-26 14:47:03 -07:00
Miccah
551dc6578c
[chore] Fix Versioner interface for twitter (#3104) 2024-07-25 14:17:02 -07:00
Miccah
2424683923
Analyze (#3099)
* Add POC analyze sub-command

* Address lint errors

* [chore] Embed scopes at compile time

* [chore] Move subcommand check up to prevent printing metrics

* added http logging to most analyzers

* Use custom RoundTripper with default http.Client

* Create framework of interfaces, structs, and protos

* Merge main

* Add AnalysisInfo to detectors.Result

* Hide analyze subcommand

* Update gen_proto.sh

* Update protos

* Make protos

* Update analyzer data types

* Rename argument to credentialInfo

---------

Co-authored-by: Joe Leon <joe.leon@trufflesec.com>
2024-07-25 12:06:05 -07:00
Abdul Basit
7c0c2e9ff3
fixed crash issue if data array is empty (#3091) 2024-07-23 09:36:27 -07:00
trufflesteeeve
c01428d107
Remove onwater detector (#3088) 2024-07-22 17:00:32 -04:00
Abdul Basit
5b64e1e5a1
implemented a netsuite detector (#3068)
* implemented a netsuite detector

* implemented the netsuite detector with modified test.

* clean up go.sum by running `go mod tidy`

* implemented a netsuite detector

* implemented the netsuite detector with modified test.

* clean up go.sum by running `go mod tidy`

* Incorporated suggestion by Ahrav
	- optimized nonce generation logic.
	- use string builder as compare to concatenation.

* fix go.sum

* fix import

* fix

---------

Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2024-07-22 12:37:18 -07:00
ahrav
7a36e89c61
add verify check (#3079) 2024-07-19 18:09:26 -07:00
Abdul Basit
c20ca0d6a1
Added Twitter v2 Detector (#3016)
* implemented a newer version of twitter included a test.
moved old twitter detector to as V1.

* added version information in twitter test
2024-07-18 12:15:22 -04:00
Ankush Goel
e2e3b870b6
Atlassian Token Detector (#3065)
* atlassian token

* Update atlassian.go

Fixed version

* fixed regex
2024-07-17 12:44:39 -07:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
2a626c4dab
update package name (#3020) 2024-07-09 16:51:46 -05:00
dylanTruffle
47535830c4
Elevenlabs detector (#3023)
* adding v1 eleven labs

* updating elevenlabs to support old and new version

* fixing status codes

* lint fixes

* adding test for v2

* adding test for v1

* return err

---------

Co-authored-by: Dylan Ayrey <dylan@Dylans-MacBook-Pro.local>
Co-authored-by: āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d <13666360+0x1@users.noreply.github.com>
2024-07-03 10:53:20 -04:00
Richard Gomez
b883645856
fix(detectors): avoid race (#3028) 2024-07-02 10:36:20 -05:00
Patrick Kaeding
4e04687712
update LaunchDarkly detector to use the caller-identity API instead of the tokens API, and instantiating an SDK (#3018) 2024-07-01 12:20:37 -07:00
Shreyas Sriram
e9206c66bb
Add endorlabs detector (#3015)
* Add endorlabs detector

* Remove unrelated changes

* Addrss comments

* remove prefix regex

---------

Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2024-06-26 21:28:19 -07:00
orionooooo
b9dd330365
added "example" (#3010) 2024-06-25 14:01:22 -05:00
Abdul Basit
dddeca5224
Adding Larksuite Detectors + Tests (#3008)
* implemented larksuite detectores for tokens and api keys.
test implemented for larksuite token based detectors.

* implemented test for larksuiteapikey detector

* load credentials from GCP secret manager for larksuite api keys
2024-06-24 11:05:56 -05:00
ahrav
347e8a6683
[feat] - Add Option to Retain False Positives During Detection (#2967)
* provide a mechanism to retain false positive findings

* update

* reorganzie

* revert comment

* update test

* typo

* fix test

* fix test

* update

* update
2024-06-18 09:40:21 -07:00
ahrav
a0618348fe
[fix] - implement MaxSecretSizeProvider for auth0managementapitoken detector (#2953)
* updates

* remove ey as a keyword
2024-06-14 14:21:17 -07:00
Cody Rose
15719c2c4f
Fix integration tests (#2970)
The recent engine changes (#2887) broke some integration tests. This PR fixes them. I also fixed an unrelated linter issue.
2024-06-14 14:30:18 -04:00
Richard Gomez
2964b3b2d2
feat(detectors): log falsepositive reason (#2969) 2024-06-14 08:26:05 -07:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
7bf3a9b5e2
adding eraser ai detector (#2961)
* adding eraser ai detector

* add eraser to defaults
2024-06-14 10:10:37 -04:00
ahrav
cb072603dc
Modularize scanning engine (#2887)
* POC: Modularize scanning engine.

* fix typo

* update interface name

* fix tests

* update test

* fix moar tests

* fix bug

* fixes.

* fix merge

* add detector verification overrides

* handle --no-verification flag

* support fp

* add test

* update name

* filter

* update test

* explicit use of detector

* updates
2024-06-13 13:47:09 -07:00
Richard Gomez
4addd81e29
test: fix compile errors (#2964) 2024-06-13 08:22:25 -07:00
Abdul Basit
cb4d332cbf
adding twitter + Consumer key detector (#2963)
* updated the twitter regex.

* updated regex for bearer token.

* clean up the code for existing twitter detector
added and Implemented new detector for twitter consumer key & secrets with test.
proto generated.

* string updated.

* written test for twitter consumer key detector

* reverted the file to avoid conflicts

* corrected the regex library in twitter detector
2024-06-13 09:32:24 -04:00