* Implemented gitlab inclusion globbing.
Included test.
* implemented two new flags for gitlab scan, includeRepo and excludeRepo to support globbing.
Apply globbing filter when repos is not provided.
* implemented integration test for inclusion globbing
remove test to check errors if globs are invalid.
* made changes to support glob compile errors.
modified changes to support glob compilation errors.
* removed unused context from few functions.
* Implemented a box detector with test cases.
* corrected comments
* remove generic keyword for box detector
remove PII details of user.
* Added Box Oauth detector
Implemented description for Box detectors.
Separated out test for Box detectors.
* removed user information from ExtraData.
---------
Co-authored-by: 0x1 <13666360+0x1@users.noreply.github.com>
* Added cisco meraki apikey detector
* addressed the comments
* handled api response and saving orgs data in extra data
* fixed linter
---------
Co-authored-by: Zachary Rice <zachary.rice@trufflesec.com>
Since Endpoints() isn't a defined interface, we are testing an
implementation detail of EndpointSetter. If that function changes in
anyway, the test will now fail instead of skipping every detector and
passing.
* [fix] Correctly initialize detectors with cloud endpoint customization
We were only initializing if the detector was configured with a custom
endpoint, but not in the default case.
* Add test
* Fix gitlab.v2 detector
* Fixed the checks for local exported data
* Fixed the check for local export files
* Fixed the check for local export files
* Fixed the check for local export files
* Merge branch 'main' into th-899-postman-panic-issue
* minor changes in the tests
* test update
* test
There is a scenario in which results filtration is known to cause problems, and this PR disables it in that scenario. (It should cause problems more generally, but lacking any concrete cases of that, I want to tread lightly.)
We have identified some cases in which the results "cleaning" logic (the logic that eliminates superfluous results) should not run. In order to allow this, we need to expose the cleaning logic to the engine. This PR does so by doing these things:
- Create a CustomResultsCleaner interface that can be implemented by detectors that want to use custom cleaning logic
- Implement this interface for the aws and awssessionkey detectors (and remove their previous invocation of their custom cleaning logic)
- Modify the engine to invoke this logic (conditionally)
This PR also removes the "custom" cleaning logic for the opsgenie, razorpay, and twilio detectors, because it was added erroneously.
This is an alternative implementation of #3233.
If a detector ignores the configured timeout it is probably because of I/O blocking, which degrades the efficiency of the detector worker pool when it happens a lot. In the worst case, a detector that fully hangs will zombify its worker, causing really bad performance problems. When this happens, we don't really have a good way to notice other than seeing scan throughput drop suspiciously. This PR adds explicit logging when detection takes longer than it should so we have a better chance of catching this.
(This problem theoretically can spring up anywhere, in any worker, but the detector fleet is vast, uses network I/O, and is implemented by a much larger group of people, so this sort of problem is much more likely to slip into detector implementations than anywhere else in the codebase. We could generalize this mechanism, but I don't want to make that investment before seeing if this smaller change captures the information we need.)
* alpha feature for scanning hidden commits on github
* improvements re: git operations
* lint updates
* updating with exec block due to no gh token
* reworked logic into new source
* fixed collisions threshold flag input
* fixed IOutil issues
* removed additions from GH config
---------
Co-authored-by: Joe Leon <joe.leon@trufflesec.com>
* implemented a netsuite detector
* implemented the netsuite detector with modified test.
* clean up go.sum by running `go mod tidy`
* implemented a netsuite detector
* implemented the netsuite detector with modified test.
* clean up go.sum by running `go mod tidy`
* Incorporated suggestion by Ahrav
- optimized nonce generation logic.
- use string builder as compare to concatenation.
* fix go.sum
* fix import
* fix
---------
Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
* adding v1 eleven labs
* updating elevenlabs to support old and new version
* fixing status codes
* lint fixes
* adding test for v2
* adding test for v1
* return err
---------
Co-authored-by: Dylan Ayrey <dylan@Dylans-MacBook-Pro.local>
Co-authored-by: āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d <13666360+0x1@users.noreply.github.com>
* initial spike on hf
* added in user and org enum
* adding huggingface source
* updated with lint suggestions
* updated readme
* addressing resources that require org approval to access
* removing unneeded code
* updating with new error msg for 403
* deleted unused code + added resource check in main
* implemented larksuite detectores for tokens and api keys.
test implemented for larksuite token based detectors.
* implemented test for larksuiteapikey detector
* load credentials from GCP secret manager for larksuite api keys
* updated the twitter regex.
* updated regex for bearer token.
* clean up the code for existing twitter detector
added and Implemented new detector for twitter consumer key & secrets with test.
proto generated.
* string updated.
* written test for twitter consumer key detector
* reverted the file to avoid conflicts
* corrected the regex library in twitter detector
