* Fix GitHub integration test
* Instrument GitHub source with a UnitReporter
The reporter is currently unused, but is the first step to support
scanning while enumerating.
* Update GitHub unit tests
When normalizing the git source config, the base and head refs should be normalized to commit hashes, in case
a branch or tag name was used. The `resolveAndSetCommit` function was returning a boolean value which should
indicate whether the input ref was changed from its original value. While this is in itself not a problem,
the caller (`normalizeConfig`) was using this boolean as an error marker, and returning early in case of `false`.
This meant that if the config was already containing a commit hash for the base or head ref, `resolveAndSetCommit`
would set the flag to `false` and `normalizeConfig` would early return erreneously. This caused the logic to find
the ancestor commit to be skipped which caused the bug in the issue #3220.
Since the `resolveAndSetCommit` function was only used in `normalizeConfig`, the signature has been changed to only
return the commit object and an error. The check for early return in `normalizeConfig` now instead relies on the
commit object being `nil` to indicate a failure to resolve the ref.
Refs: #3220
* implement analyzer interface for postgres
* added unit test for postgres analyzer
* refactored code in postgres analyzer
* generate permissions for postgres analyzer
* renamed variable
* [chore] corrected the variable name.
* appended hostname to distinguish the resources.
updated the test.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* implement analyzer interface for sourcegraph
* created permission for sourcegraph
test for sourcegraph.
added email in resource metadata.
* handling of missing keys in map
* linked sourcegraph detector to analyzer
* update the fullyqualidied name of resource to make it unique.
updated the test.
* add current user email in metadata
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* impelmented analyzer interface with data models for Asana
* add unit test for asana analyzer
* link asana detector with analyzer
* added permission for asana
linked detector with only positive cases.
* to make test cleaner moved want json in external file.
Moreover without sorting test will not be able to compare.
* use general functions to avoid code duplication.
optimize app permission making logic.
* [fix] assigned bindings to results.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* implement analyzer interface for slack
* slack analyzer adjusted for new changes in main, unit test added
* link detector with analyzer for slack
* added generated permissions for slack analyzer
* generate permission fix, keep dot in permissions intact
* removed scope from permission and put it metadata.
* [chore]
moved expected output of test in json file to neat the code.
added team id in fully qualified name of user resource.
check permissions before adding it in bindings.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* square analyzer fix assign team members to unbounded resources - unit test for square analyzer
* refactoring
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* implement analyzer interface for mailchimp
* link detector with analyzer for mailchimp
* fix analyzer type
* add mailchimp analyzer test
* [chore] appended string to make fulllyqualifiedname as unique.
* [chore]
moved expected output of test in json file to neat the code.
removed PII information in metadata.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* impelmented analyzer interface with data models for Bitbucket
* Added bitbucket permissions in yaml
add ids to keep resources distinguishable.
* [chore]
moved expected output of test in json file to neat the code.
corrected the fully qualified name for repository resources.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* implement analyzer interface for sendgrid
* add unit test for sendgrid analyzer
* fixed sendgrid detector.
linked analyzer with sendgrid detector.
handling if key not found in map
* category as resource.
if subcategory is present then subcategory will become resource with parent category
* corrected test and remove hardcoded boolean for printing in sendgrid
* incorporate code refactoring and suggestion for FullyQualifiedName for subcategory by Miccah.
* generate permissions for sendgrid analyzer
* [NIT] rather than updating the global scopes variable, ProcessPermission will return new list of categories with Permission and eliminate those which are not in generated one.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* implement analyzer interface for opsgenie and add unit tests
* Add analyzer interface for opsgenie
linked detector with analyzers
fixed test cases.
* generate permissions for opsgenie and change scope names to lowercase for consistency
* fixed unboundedresources slice issue.
username as fullqualifiedname
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
* implement analyzer interface for postman and add unit test
* analyzer interface inplementation for postman
linked detector with analyzer for postman
add permission for postman
* [fix] linter in postman
* considered Miccah comments about fullyqualifiedName and code refactoring.
* moved want result to expected output file.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
The previous implementation of targeted file scanning pulled patches out of commit data, which didn't work for binary files (because GitHub doesn't return patches for them). This PR changes the system to always just download the requested file and scan it, which means we get binary file support.
* Fixed the checks for local exported data
* Fixed the check for local export files
* Fixed the check for local export files
* Fixed the check for local export files
* Merge branch 'main' into th-899-postman-panic-issue
* minor changes in the tests
* test update
* test
The GitHub source generates chunks for targeted scans differently than it does for "normal" scans. One difference was the presence of leading + and - characters, which can interfere with detection in some cases.
There is a scenario in which results filtration is known to cause problems, and this PR disables it in that scenario. (It should cause problems more generally, but lacking any concrete cases of that, I want to tread lightly.)
We have identified some cases in which the results "cleaning" logic (the logic that eliminates superfluous results) should not run. In order to allow this, we need to expose the cleaning logic to the engine. This PR does so by doing these things:
- Create a CustomResultsCleaner interface that can be implemented by detectors that want to use custom cleaning logic
- Implement this interface for the aws and awssessionkey detectors (and remove their previous invocation of their custom cleaning logic)
- Modify the engine to invoke this logic (conditionally)
This PR also removes the "custom" cleaning logic for the opsgenie, razorpay, and twilio detectors, because it was added erroneously.
This is an alternative implementation of #3233.
Many analyzer tests require GCP credentials, which the community does
not have access to. It's best to ignore these tests, which would
otherwise immediately fail for unrelated community contributions.
