Commit graph

2681 commits

Author SHA1 Message Date
Richard Gomez
3077215eef
fix(mongodb): ignore invalid URLs (#3440) 2024-11-07 08:18:21 -08:00
Richard Gomez
53d01ad219
fix(rabbitmq): add dial timeout (#3421) 2024-11-07 08:17:58 -08:00
ahrav
c345df9951
add config option for s3 resumption (#3563) 2024-11-06 18:30:40 -08:00
Kashif Khan
9095845958
added pattern test cases for detectors starting with b (#3559)
* added pattern test cases for detectors starting with b

* last brick
2024-11-06 15:45:47 -06:00
Kashif Khan
8897ed2a66
added pattern test cases for all detectors starting with Alphabet a (#3539)
* added pattern test cases

* some more test cases with some fixes too

* improved apimetrics regex and added test cases

* added more pattern test cases

* added more pattern test cases

* last brick of pattern test cases for a Alphabet
2024-11-06 15:35:33 -06:00
Abdul Basit
d073ad69ca
validate if twitter services are more than one before accessing it. (#3565) 2024-11-06 07:47:41 -08:00
ahrav
bf89c89e09
[bug] - Correct Line Number Calculation (#3550)
* correclty report line number

* add addiitonal test cases
2024-11-05 08:53:18 -08:00
Kashif Khan
3f8ddd39a2
set verification error if failed to decode body (#3560) 2024-11-05 08:00:54 -08:00
Dustin Decker
5ca4a17a4c
Add owner to github tokens (#3558) 2024-11-04 14:43:30 -08:00
ahrav
fa871b0c1e
[feat] - Add Weights and Biases detector (#3551)
* add Weights and biases detector

* add to default detector list
2024-11-04 10:27:46 -08:00
ahrav
6923843bc4
[chore] - minor cleanup S3 source (#3554)
* cleanup s3 source

* revert
2024-11-04 08:54:30 -08:00
Cody Rose
e6d786a7d9
stop logging all GitLab projects (#3541)
#3532 stopped logging one huge GItLab slice, but there's a second one! We should stop logging it for the same reason.
2024-11-01 12:27:44 -04:00
Kashif Khan
bd5e287425
fixed gitlab extradata overwriting (#3537) 2024-10-31 07:14:39 -05:00
Zachary Rice
4d355d414e
gcp cred not set (#3535)
* stopgap i guess

* slightly smarter
2024-10-30 16:39:31 -05:00
Kashif Khan
84ec670228
standardize email pattern (#3524)
* standardize email pattern

* added email pattern test cases

* added negative test cases for email pattern
2024-10-30 11:39:07 -05:00
Cody Rose
65e913e2c8
stop logging detailed group info (#3532)
Co-authored-by: Abdul Basit <basit.mussani@gmail.com>
2024-10-30 10:04:33 -04:00
Abdul Basit
9b2cef55c0
[analyze] Add Analyzer interface for Gitlab (#3232)
* implement analyzer interface for gitlab

* generated permissions, added unit test for gitlab analyzer

* revert deletion of scopes.go

* appending domain in resource names

* [chore]
moved expected output of test in json file to neat the code.

* updated the test for gitlab analyzer
to make more unique FullyQualifiedName, Ids are added for resources.

* remove unnecessary metadata field and fix github -> gitlab

* extract user id from access token json, make user as resource

* link analyzer with gitlab v2 detector

* fixed code breaking changes due to analyzer protobuf removal.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-10-30 18:46:50 +05:00
Abdul Basit
f4670aaab1
[feat] Gitlab inclusion globbing (#3500)
* Implemented gitlab inclusion globbing.
Included test.

* implemented two new flags for gitlab scan, includeRepo and excludeRepo to support globbing.
Apply globbing filter when repos is not provided.

* implemented integration test for inclusion globbing
remove test to check errors if globs are invalid.

* made changes to support glob compile errors.
modified changes to support glob compilation errors.

* removed unused context from few functions.
2024-10-30 18:23:34 +05:00
Sahil Silare
e81ff7630c
feat: added v3 API version for the detector captaindatago (#3484)
* feat: added `v3` API version for the detector `captaindatago`

* chore: renamed captaindata to captaindatav2

* fix: fixed v3 detector to follow new pattern

* feat: added versioner impl

Signed-off-by: Sahil Silare <sahilsilare@gmail.com>

* feat: changed the name from v2 -> v1 and v3 -> v2

Signed-off-by: Sahil Silare <sahilsilare@gmail.com>

---------

Signed-off-by: Sahil Silare <sahilsilare@gmail.com>
Co-authored-by: Kashif Khan <70996046+kashifkhan0771@users.noreply.github.com>
2024-10-30 10:40:49 +05:00
Dustin Decker
0e7bce28e6
update aws descriptions (#3529) 2024-10-29 21:10:52 -07:00
Zachary Rice
b44dc24494
enforce timeout on circleci test (#3528)
* enforce timeout

* bump to 10s, 3s seemed too aggro
2024-10-29 15:59:02 -05:00
Cody Rose
a136e31d83
Redact more source credentials (#3526)
This PR implements global log redaction for the credentials of most other source types. It doesn't redact for sources that don't load their credentials with Init as a way to keep the PR simple - we can do those separately.
2024-10-29 12:42:20 -04:00
Cody Rose
f42f63271b
Create global log redaction capability (#3522)
Some source use client libraries that can emit errors that contain sensitive information - in particular, git-facing libraries that embed tokens into repository URLs. This PR introduces a way of redacting them - starting with GitLab (where we've seen this most recently), but in theory extensible to other sources as needed.

This implementation uses a custom zap core; this might also be possible with a custom zap encoder, but I didn't test it out.

(The deleted core.go file was entirely unused.)
2024-10-29 09:44:07 -04:00
Kashif Khan
f164ebcd88
Handle custom detector response and include in extra data (#3411)
* Handle custom detector response and include in extra data

* Added todo

* fixed panic

* simplicity is always good

* limit the response to 200 chars

* results should print now in output
2024-10-28 13:09:14 -05:00
Sahil Silare
c6aa491201
fix: fixed validation logic for calendarific (#3480) 2024-10-28 11:26:14 -05:00
Cody Rose
ad15888a33
Move DecoderType into ResultWithMetadata #3502
Result.DecoderType is only ever used by ResultWithMetadata (via its embedded Result). This unnecessarily complicates the relationship between the types and adds some warts to #3457, so this PR moves DecoderType directly into the only struct which actually uses it.
2024-10-28 12:20:42 -04:00
Kashif Khan
97fac39885
Addeded 403 account block status code handling for gitlab (#3471)
* Addeded 403 account block status code handling for gitlab

* resolved comments

* removed unmarshalling logic
2024-10-28 11:12:30 -05:00
Kashif Khan
6a367ab763
updated gcpapplicationdefaultcredentials detector results with RawV2 (#3499)
* updated results with RawV2

* censored the refresh token in result
2024-10-28 08:35:04 -05:00
Sahil Silare
b48f748be0
fix: fixed verifcation pattern logic for bulksms (#3478)
* fix: fixed verifcation pattern logic for `bulksms`

* fix: addressed review comments

* fix: added check to break the loop if a combination matches

* fix: fixed key verification logic

Signed-off-by: Sahil Silare <sahilsilare@gmail.com>

---------

Signed-off-by: Sahil Silare <sahilsilare@gmail.com>
2024-10-25 10:47:06 +05:00
Helio Machado
9cf0a4cb28
Extend algoliaadminkey with additional checks (#3459) 2024-10-24 14:16:43 -05:00
Sahil Silare
1aa1871f9a
fix: added correct api endpoint for verification & logic for Aeroworkflow (#3435)
* fix: added correct api endpoint for verification

* fix: fixed pattern detection logic in aeroworkflow
2024-10-24 12:16:39 -05:00
ahrav
01a2a4a3cc
remove debug log (#3505) 2024-10-24 09:40:23 -07:00
Cody Rose
4cc10d3da1
delete unused code (#3504) 2024-10-24 10:59:30 -05:00
Sahil Silare
c37edcd72e
fix: added correct verification endpoint & validation logic for alegra (#3437)
* fix: added correct verification endpoint for alegra

* fix: fixed email regex

* fix: added correct tests and validation

* fix: fixed alegra tests
2024-10-24 17:13:14 +05:00
Richard Gomez
06e37caa04
chore: re-order log context fields (#3430) 2024-10-23 10:53:15 -04:00
Sahil Silare
d78239c9f5
feat: validation & verification fix for apiscience to apimetrics (#3475) 2024-10-22 11:31:32 -05:00
Sahil Silare
3f9ba20a1c
fix: fixed validation logic for cannyio (#3482) 2024-10-22 11:20:12 -05:00
ahrav
0b64fa0805
update error messages (#3490)
Co-authored-by: Abdul Basit <basit.mussani@gmail.com>
2024-10-22 07:54:55 -07:00
ahrav
2d1dc73c9e
[fix] - Inadvertent s3 body close (#3491)
* fix object retrival

* update comment
2024-10-21 19:54:06 -07:00
Bill Rich
abecab0d8f
Remove proto (#3489)
Only used for tracking analyzer types
2024-10-21 17:16:26 -07:00
Cody Rose
3ab60865c8
Extract FP logic correctly at other call site #3476 2024-10-21 09:51:53 -04:00
ahrav
6e055ea578
increase timeout to 30s (#3422)
* increase timeout to 30s

* add debug

* use context logger and include size in logs

* close reader

* address comments
2024-10-18 16:13:03 -07:00
Damanpreet Singh
88b8c862a6
Update yousign detector endpoints to check againt prod and staging urls (#3426) 2024-10-18 15:47:42 -05:00
Sahil Silare
f7252c525a
fix: fixed autoklose verification endpoint (#3447)
* fix: fixed autoklose verification endpoint

* fix: addressed review comments

* fix: addressed review comments

* fix: added guard checks for 401 and 200

* fix: added code
2024-10-18 15:32:17 -05:00
Sahil Silare
3499df65f2
fix: fixed verification logic & endpoint for AyrShare (#3452)
* fix: fixed verification endpoint for AyrShare

* fix: changed verification endpoint for ayrshare

* fix: fixed error handling
2024-10-18 15:26:24 -05:00
ahrav
871a2b0f06
fix timeout (#3460) 2024-10-17 12:53:40 -07:00
Kashif Khan
e6c43ca863
ignore https as false postive for slackwebhook detector (#3425) 2024-10-16 05:53:06 -07:00
JonZeolla
4ea311dea9
feat: add github comments timeframe filtering (fixes #3388) (#3390)
* feat: add github comments timeframe filtering

* fixup and generate protos

* Cleanup

---------

Co-authored-by: Zachary Rice <zachary.rice@trufflesec.com>
2024-10-15 15:13:36 -04:00
ahrav
bf38b8480c
[fix] - resource leak (#3402)
* fix resource leak

* add comment

* use errors.Join

* address error wrapping
2024-10-15 12:11:45 -07:00
Abdul Basit
aa17b8eba4
[detector] Implemented Box Detector (#3242)
* Implemented a box detector with test cases.

* corrected comments

* remove generic keyword for box detector
remove PII details of user.

* Added Box Oauth detector
Implemented description for Box detectors.
Separated out test for Box detectors.

* removed user information from ExtraData.

---------

Co-authored-by: 0x1 <13666360+0x1@users.noreply.github.com>
2024-10-15 08:42:37 -05:00