Commit graph

3433 commits

Author SHA1 Message Date
Ahrav Dutta
1aebf110d3 pass ctx 2024-09-23 15:00:42 -07:00
renovate[bot]
f8f2485c6d
fix(deps): update module github.com/xanzy/go-gitlab to v0.109.0 (#3318)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-23 14:31:55 -07:00
Abdul Basit
3e46b3f221
[Analyzer] Test and generated permissions for HuggingFace, Square & Stripe (#3294)
* stripe analyzer unit test

* add huggingface analyzer unit test

* add permissions.yaml for huggingface and fix in analyzer

* square permissions generated

* permissions generated for stripe

* change permissions to lowercase

* skip unknown permissions for square and stripe

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-23 12:17:20 -07:00
Miccah
2f3a410e38
Implement SourceUnitEnumChunker for GitHub (#3298)
* Implement SourceUnitEnumChunker for GitHub

This change refactors the internal scan method to introduce a scanRepo
method to perform the actual scan.

* Export unit fields so the values are captured in the report

* Add comment for scanRepo

* Break out ensureRepoInfoCache into a method

* Update comments and check errors

* Ensure that the repoInfoCache contains the repo during ChunkUnit

* Add integration test for ChunkUnit

* Move s.scanOptions initialization to Init()
2024-09-23 10:56:55 -07:00
renovate[bot]
764db68116
fix(deps): update module github.com/wasilibs/go-re2 to v1.7.0 (#3317)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-23 09:29:31 -07:00
renovate[bot]
49c78c9724
fix(deps): update module github.com/schollz/progressbar/v3 to v3.16.0 (#3315)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-23 09:16:37 -07:00
Zachary Rice
b78be6d935
hit em w/ a min (#3316) 2024-09-23 11:07:39 -05:00
renovate[bot]
6c7c1ae206
fix(deps): update module github.com/snowflakedb/gosnowflake to v1.11.1 (#3313)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-23 07:46:07 -07:00
ahrav
97fd2f80ee
[fix] - Add Size Method to BufferedReadSeeker and Refactor Context Timeout Handling in HandleFile (#3307) 2024-09-23 06:21:07 -07:00
renovate[bot]
c33ab21ea6
fix(deps): update module github.com/sendgrid/sendgrid-go to v3.16.0+incompatible (#3312)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-22 10:57:58 -07:00
renovate[bot]
6b7c137e11
fix(deps): update module github.com/schollz/progressbar/v3 to v3.15.0 (#3311)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-21 15:14:53 -07:00
renovate[bot]
3d1cb2ce5e
fix(deps): update module github.com/getsentry/sentry-go to v0.29.0 (#3310)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-21 13:12:41 -07:00
renovate[bot]
e29785dbaf
fix(deps): update module github.com/elastic/go-elasticsearch/v8 to v8.15.0 (#3309)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-21 10:54:20 -07:00
Miccah
77dc2720a8
Update GitHub enumeration to report unique filtered values (#3292)
The reported values should match the values populated in s.repos.
2024-09-18 14:30:10 -07:00
Miccah
b2da2a6a5c
[analyze] Add client filter to detect successful unsafe HTTP requests (#3305)
* Move analyzer client to its own file

* Add analyzer client filter to detect successful unsafe HTTP requests

* Close response body in test
2024-09-18 10:31:21 -07:00
renovate[bot]
1b59a5ecf2
fix(deps): update module github.com/prometheus/client_golang to v1.20.4 (#3303)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-17 09:03:13 -07:00
renovate[bot]
3d72f53524
fix(deps): update module cloud.google.com/go/secretmanager to v1.14.1 (#3301)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-17 08:52:00 -07:00
Miccah
401bc4687b
Instrument GitHub source with a ChunkReporter (#3296) 2024-09-16 09:57:09 -07:00
renovate[bot]
661984cc9c
fix(deps): update golang.org/x/exp digest to 701f63a (#3291)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-15 12:00:03 -07:00
Dustin Decker
7e78ca385f
Add user agent suffix feature flag (#3297)
* Add user agent suffix feature flag

* unecessary concat
2024-09-13 15:20:43 -07:00
Miccah
213bf7e4fd
Fix GitHub analyzer panic on empty organization name (#3295) 2024-09-13 11:31:31 -07:00
tiaoxizhan
07dc95dd58
Fix slice initialization error (#3293) 2024-09-13 06:14:51 -07:00
Abdul Basit
dc9c9a30b3
[analyze] Add analyzer interface for Shopify (#3226)
* implement analyzer interface for shopify

* fixed shopify analyzer according to new code and generated permissions

* shopify analyzer test added

* [chore]
- key validations
- linked analyzer with detectors

* [chore]
- moved redundant initialize to global.

* [chore]
moved expected output of test in json file to neat the code.

* [Fixes]
- Fixed permission and category resource issue in shopify analyzer
- corrected test for shopify analyzer

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-12 11:53:42 -07:00
Abdul Basit
57e58123f1
[analyze] Add Analyzer for Mailgun (#3206)
* implement analyzer interface, add unit test and link with detector for mailgun

* [chore] moved expected output of test in json file to neat the code.
corrected variable name for test in detector bucket

* append domain id in fully qualified name of domain resources

* [Fixes]
domains will be added as resource in bindings and permissions.
updated the test.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-12 11:48:46 -07:00
Abdul Basit
b0318a9edb
[analyze] Add Analyzer for MySQL (#3193)
* implement analyzer interface for mysql

* add integration test for mysql analyzer

* linked detectors with analyzers for jdbc and mysql
validation for connection string in analyzer

* refactored secretInfoToAnalyzerResult func

* generated permissions for mysql analyzer

* [chore]
- optimization in execution flow
- use test-container library for analyze test.

* added host in secret info struct
simplified the mysql test due to huge structure

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-12 11:48:34 -07:00
Miccah
e89190f3ed
Instrument GitHub source with a UnitReporter (#3284)
* Fix GitHub integration test

* Instrument GitHub source with a UnitReporter

The reporter is currently unused, but is the first step to support
scanning while enumerating.

* Update GitHub unit tests
2024-09-12 10:28:37 -07:00
renovate[bot]
0cb872307c
fix(deps): update module github.com/prometheus/client_golang to v1.20.3 (#3279)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-12 08:35:34 -07:00
dylanTruffle
029f03e7bd
adding pypi v1 support (#3289)
Co-authored-by: Dylan Ayrey <dxa4481@rit.edu>
2024-09-11 21:39:01 -07:00
dylanTruffle
d201e54305
adding pypi detector (#3287)
* adding pypi detector

* update test and use helper

---------

Co-authored-by: Dylan Ayrey <dxa4481@rit.edu>
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-09-11 19:17:17 -07:00
Dustin Decker
70c6bb5634
feature flag additional refs (#3282) 2024-09-10 15:51:41 -07:00
Richard Gomez
b7411d2922
Clarify "no decoder found for chunk" log message (#3001)
* chore(engine): clarify trace log message

* chore(engine): fix merge conflicts
2024-09-10 13:58:40 -05:00
Zachary Rice
2fb90295ce
update aha keyword (#3281) 2024-09-10 13:07:27 -05:00
ahrav
5dad5a738b
[chore] - remove unused method and function (#3089)
* remove unused method

* delete more unused stuff
2024-09-10 12:35:49 -05:00
Ankush Goel
0feca62469
Jira Email fix (#3061) 2024-09-10 12:27:55 -05:00
Valentin B.
075f8c67a5
fix(git): config normalization for git sources (#3278)
When normalizing the git source config, the base and head refs should be normalized to commit hashes, in case
a branch or tag name was used. The `resolveAndSetCommit` function was returning a boolean value which should
indicate whether the input ref was changed from its original value. While this is in itself not a problem,
the caller (`normalizeConfig`) was using this boolean as an error marker, and returning early in case of `false`.

This meant that if the config was already containing a commit hash for the base or head ref, `resolveAndSetCommit`
would set the flag to `false` and `normalizeConfig` would early return erreneously. This caused the logic to find
the ancestor commit to be skipped which caused the bug in the issue #3220.

Since the `resolveAndSetCommit` function was only used in `normalizeConfig`, the signature has been changed to only
return the commit object and an error. The check for early return in `normalizeConfig` now instead relies on the
commit object being `nil` to indicate a failure to resolve the ref.

Refs: #3220
2024-09-10 12:12:57 -05:00
Daniel Teixeira
f24f62832b
Add detector for Nvidia NGC Personal Keys (#3280)
* Add detector for Nvidia NGC Personal Keys

* Update nvapi.go to use `nvapi-` as the keyword
2024-09-10 08:36:33 -07:00
Nash
17f6c98119
GitHub source logger clean up (#3269)
* GitHub source logger clean up

* applied pr comments

* applied pr comments

* applied pr comments

* applied PR review comments
2024-09-09 15:44:56 -04:00
renovate[bot]
8a4d62c670
fix(deps): update module github.com/felixge/fgprof to v0.9.5 (#3277)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-09 09:05:12 -07:00
renovate[bot]
8cc7a40fa0
fix(deps): update golang.org/x/exp digest to e7e105d (#3202)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-09-08 11:05:07 -07:00
Miccah
bc2d00710a
[chore] Skip analyzer tests in CI (#3270) 2024-09-06 14:09:13 -07:00
Abdul Basit
93d09c78b4
[analyze] Add Analyzer for Postgres (#3192)
* implement analyzer interface for postgres

* added unit test for postgres analyzer

* refactored code in postgres analyzer

* generate permissions for postgres analyzer

* renamed variable

* [chore] corrected the variable name.

* appended hostname to distinguish the resources.
updated the test.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:42:55 -07:00
Abdul Basit
a43d451c4d
[analyze] Add Analyzer for SourceGraph (#3173)
* implement analyzer interface for sourcegraph

* created permission for sourcegraph
test for sourcegraph.
added email in resource metadata.

* handling of missing keys in map

* linked sourcegraph detector to analyzer

* update the fullyqualidied name of resource to make it unique.
updated the test.

* add current user email in metadata

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:40:47 -07:00
Abdul Basit
4cab071032
[analyze] Add Analyzer for Asana (#3139)
* impelmented analyzer interface with data models for Asana

* add unit test for asana analyzer

* link asana detector with analyzer

* added permission for asana
linked detector with only positive cases.

* to make test cleaner moved want json in external file.
Moreover without sorting test will not be able to compare.

* use general functions to avoid code duplication.
optimize app permission making logic.

* [fix] assigned bindings to results.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:37:45 -07:00
Abdul Basit
f235b8a442
[analyze] Add Analyzer for Slack (#3207)
* implement analyzer interface for slack

* slack analyzer adjusted for new changes in main, unit test added

* link detector with analyzer for slack

* added generated permissions for slack analyzer

* generate permission fix, keep dot in permissions intact

* removed scope from permission and put it metadata.

* [chore]
moved expected output of test in json file to neat the code.
added team id in fully qualified name of user resource.
check permissions before adding it in bindings.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:33:06 -07:00
Abdul Basit
c449129d75
[analyze] Improve SquareUp analyzer and Implemented test (#3231)
* square analyzer fix  assign team members to unbounded resources - unit test for square analyzer

* refactoring

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-06 12:32:52 -07:00
Abdul Basit
5d7e6fc2fa
[analyze] Add Analyze interface for Mailchimp (#3225)
* implement analyzer interface for mailchimp

* link detector with analyzer for mailchimp

* fix analyzer type

* add mailchimp analyzer test

* [chore] appended string to make fulllyqualifiedname as unique.

* [chore]
moved expected output of test in json file to neat the code.
removed PII information in metadata.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-05 16:33:47 -07:00
Abdul Basit
becc2c4010
[analyze] Add analyze interface for Bitbucket (#3224)
* impelmented analyzer interface with data models for Bitbucket

* Added bitbucket permissions in yaml
add ids to keep resources distinguishable.

* [chore]
moved expected output of test in json file to neat the code.
corrected the fully qualified name for repository resources.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-05 16:04:35 -07:00
Abdul Basit
899f59fbb4
[analyze] Add Analyzer for Sendgrid (#3174)
* implement analyzer interface for sendgrid

* add unit test for sendgrid analyzer

* fixed sendgrid detector.
linked analyzer with sendgrid detector.
handling if key not found in map

* category as resource.
if subcategory is present then subcategory will become resource with parent category

* corrected test and remove hardcoded boolean for printing in sendgrid

* incorporate code refactoring and suggestion for FullyQualifiedName for subcategory by Miccah.

* generate permissions for sendgrid analyzer

* [NIT] rather than updating the global scopes variable, ProcessPermission will return new list of categories with Permission and eliminate those which are not in generated one.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-04 15:50:32 -07:00
Abdul Basit
d6e1627f16
[analyze] Add Analyzer for Opsgenie (#3181)
* implement analyzer interface for opsgenie and add unit tests

* Add analyzer interface for opsgenie

linked detector  with analyzers
fixed test cases.

* generate permissions for opsgenie and change scope names to lowercase for consistency

* fixed unboundedresources slice issue.
username as fullqualifiedname

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-04 15:50:19 -07:00
Abdul Basit
5ce1578a6f
[analyze] Add analyzer for Postman (#3180)
* implement analyzer interface for postman and add unit test

* analyzer interface inplementation for postman

linked detector with analyzer for postman
add permission for postman

* [fix] linter in postman

* considered Miccah comments about fullyqualifiedName and code refactoring.

* moved want result to expected output file.

---------

Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-09-04 15:40:12 -07:00