Miccah
160fd830dd
Remove capturing the cancel callstack in the context package ( #1595 )
...
* Fix race condition in context package
* Remove capturing the cancel callstack
2023-08-01 21:34:00 -05:00
Zubair Khan
0ad46381d9
tighten up regex pattern for timezoneapi ( #1591 )
...
* tighten up regex pattern
* add response body check
2023-08-01 17:30:29 -04:00
Zachary Rice
91aa75679b
broken link ( #1592 )
2023-08-01 16:13:22 -05:00
Zachary Rice
b906a51d5c
updating github templates ( #1587 )
...
* updating github templates
* address miccahs comments
* remove community note comment
2023-08-01 15:25:33 -05:00
Zachary Rice
3c79b6b133
Remove CI Badge ( #1590 )
2023-08-01 13:45:46 -05:00
ahrav
78d06658ca
Dont return in loop. ( #1589 )
2023-08-01 10:29:01 -07:00
Miccah
69021f59c5
Refactor git source to allow ScanOptions and use source in engine ( #1518 )
...
* Refactor git source to allow ScanOptions and use source in engine
Refactor the Chunks method of the git Source to call out to two helper
methods: scanRepos and scanDirs which scans s.conn.Repositories and
s.conn.Directories respectively. The only notable change in behavior is
that a credential is no longer necessary if there are no
s.conn.Repositories to scan.
* Preserve ScanGit functionality of not cleaning up temporary files
2023-08-01 09:52:02 -05:00
ahrav
b8c43ea58f
Fix VirusTotal deetector ( #1585 )
2023-08-01 05:41:18 -07:00
Zubair Khan
a4b1fb7752
create hidden debug flag to disable overseer ( #1582 )
...
* add in new debug flag
* keep localdev local
2023-07-31 22:03:59 -04:00
Cody Rose
7d2f126411
add tri-state verification to mongodb detector ( #1575 )
2023-07-31 18:23:35 -04:00
ahrav
5043fc8756
[bug] - Fix unlocking an unlocked mutex ( #1583 )
...
* use correct mutext.
* remove unused fxn.
2023-07-31 14:06:41 -07:00
ahrav
661c6b47b7
[bug] - fix shodan detector ( #1579 )
...
* fix shodan detector.
* fix import order.
2023-07-31 11:12:52 -07:00
ahrav
eb00d0d4e1
[bug] - fix data races ( #1577 )
...
* fix data race.
* Add test and fix additional data race.
* address comments.
2023-07-31 11:12:38 -07:00
ahrav
406ce7bc55
Replace magic strings with const ( #1568 )
...
* Add normalize repo for azure.
* remove zero values from test cases.
* use const.
* remove azure logic.
2023-07-31 11:12:26 -07:00
ahrav
5e7a6ca11c
Concurrent detection ( #1580 )
...
* Run detection on each chunk concurrently.
* Add printer functionality.
* Add logic for dedupe.
* cleanup.
* Moddify number of notifier workers.
* Add comment.
* move consts into fxn.
* buffer resutls chan.
* fix test.
* address comments.
* return an error from Finish.
* fix test.
* fix test.
* linter.
* check err.
* address comments.
2023-07-31 11:12:08 -07:00
Miccah
b54683acb9
gitparse: Use an object for currentDiff ( #1573 )
...
* gitparse: Use an object for currentDiff instead of a pointer
* gitparse: Use an object for currentCommit instead of a pointer
* Revert "gitparse: Use an object for currentCommit instead of a pointer"
This reverts commit c5f0708b4a
2023-07-31 11:39:14 -05:00
Miccah
32e3f1f015
Fix pubnub regular expression ( #1565 )
...
One of the sub-groups of the UUIDv4 was missing the characters 0-9.
2023-07-31 11:37:25 -05:00
Miccah
a07b6664f8
Support fatal errors in job reports ( #1562 )
...
* Support fatal errors in job reports
* WIP: JobReporter and JobInspector
* WIP: JobReportHook and JobReportRef
* Add ChunkError type and asyncRun helper method
* Rename JobReport to JobProgress
* Return a closed channel from Done when the JobProgress is nil
* Comment catchFirstFatal function
2023-07-31 11:28:30 -05:00
Cody Rose
61bee6c8b1
Identify transient AWS verification failures ( #1563 )
...
It turns out that GetCallerIdentity returns a surprising quantity of transient, false-negative 403 responses that carry the SignatureDoesNotMatch error reason. I don't know why this is happening, but their transient nature makes them indeterminate verification failures and they should be flagged as such. The AWS detector has therefore been modified to specifically look for the InvalidClientTokenId error reason in 403 responses and mark all other responses as indeterminate.
In addition to the functional changes this PR contains some updates to the test code that allow us to test them.
2023-07-31 12:06:11 -04:00
Cody Rose
ad57de50cd
Do not nest transports for Github installation client ( #1564 )
...
#1454 modified one of the Github enumeration code paths in a way that broke an integration test by causing one client's transport to be used for the construction of a different client, causing authentication failures. This saves the original transport for use, fixing the test.
2023-07-31 11:31:16 -04:00
Richard Gomez
e0faac8d1c
Fix runtime error when scanning Gist comments ( #1552 )
...
* fix(github): fix runtime error from gist comments
* fix(github): add flag to scan Gist comments
2023-07-31 08:57:42 -05:00
Miccah
070014f380
Initialize the default logger to output to stderr ( #1569 )
2023-07-31 03:26:19 -05:00
Dustin Decker
10b6e2898d
Increase log level of engine messages ( #1576 )
2023-07-28 14:30:43 -07:00
Miccah
6bd48583ae
Fix gitparse from panicking on a nil-pointer ( #1570 )
2023-07-28 11:15:02 -05:00
Cody Rose
431d26f5fa
move false positive check in alchemy detector ( #1532 )
...
This PR makes the Alchemy detector run its known false positive check even if verification is disabled. This isn't the most important detector but it's the template for new ones so getting a good pattern nailed down is important.
Moving the check allowed me to rewrite the determinacy logic to hopefully be more clear.
2023-07-28 11:36:02 -04:00
Zubair Khan
9f3809f19e
gdrive proto change ( #1566 )
2023-07-28 10:38:51 -04:00
Miccah
e391e89f3e
Initial implementation of JobReport with SourceManager usage ( #1557 )
...
* Initial implementation of JobReport with SourceManager usage
* Limit concurrent units
* Only save the last JobReport per handle
2023-07-27 10:49:56 -05:00
Zachary Rice
3897454dbb
add merge support ( #1561 )
2023-07-27 09:24:49 -05:00
ahrav
ade5d91d5c
Add azure repos protos. ( #1559 )
2023-07-26 19:53:10 -07:00
Dustin Decker
84d4bf0d51
Override broken dependency version ( #1558 )
2023-07-26 16:13:02 -07:00
Richard Gomez
f925da7cea
fix(mongodb): detect CosmoDB access keys ( #1511 )
...
https://learn.microsoft.com/en-us/microsoft-365/compliance/sit-defn-azure-cosmos-db-account-access-key?view=o365-worldwide
2023-07-26 16:50:12 -05:00
dependabot[bot]
e0011ac08e
Bump github.com/go-git/go-git/v5 from 5.8.0 to 5.8.1 ( #1554 )
...
Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git ) from 5.8.0 to 5.8.1.
- [Release notes](https://github.com/go-git/go-git/releases )
- [Commits](https://github.com/go-git/go-git/compare/v5.8.0...v5.8.1 )
---
updated-dependencies:
- dependency-name: github.com/go-git/go-git/v5
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-26 14:13:12 -07:00
dependabot[bot]
7a29f8a514
Bump github.com/bradleyfalzon/ghinstallation/v2 from 2.4.0 to 2.6.0 ( #1503 )
...
Bumps [github.com/bradleyfalzon/ghinstallation/v2](https://github.com/bradleyfalzon/ghinstallation ) from 2.4.0 to 2.6.0.
- [Release notes](https://github.com/bradleyfalzon/ghinstallation/releases )
- [Commits](https://github.com/bradleyfalzon/ghinstallation/compare/v2.4.0...v2.6.0 )
---
updated-dependencies:
- dependency-name: github.com/bradleyfalzon/ghinstallation/v2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-26 09:03:55 -07:00
Richard Gomez
46823f77c9
feat(github): clarify comment log statement ( #1553 )
2023-07-26 09:40:30 -05:00
Miccah
10f0963bc9
Add SourceManager tests for Run and Wait methods ( #1530 )
...
* Miscellaneous SourceManager updates
* Own the chunks channel instead of accepting it as an input
* Add Chunks and Wait methods
* Fix bug in Enroll so it actually returns the handle
* Add context.Context parameter to the SourceInitFunc type
* Add SourceManager tests for Run and Wait methods
* Rename man variables to mgr
2023-07-26 00:48:28 -05:00
dependabot[bot]
35b332c498
Bump github.com/google/go-containerregistry from 0.14.0 to 0.15.2 ( #1504 )
...
Bumps [github.com/google/go-containerregistry](https://github.com/google/go-containerregistry ) from 0.14.0 to 0.15.2.
- [Release notes](https://github.com/google/go-containerregistry/releases )
- [Changelog](https://github.com/google/go-containerregistry/blob/main/.goreleaser.yml )
- [Commits](https://github.com/google/go-containerregistry/compare/v0.14.0...v0.15.2 )
---
updated-dependencies:
- dependency-name: github.com/google/go-containerregistry
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bill Rich <bill.rich@gmail.com>
2023-07-25 20:33:18 -07:00
dependabot[bot]
c814ae4d83
Bump github.com/xanzy/go-gitlab from 0.86.0 to 0.88.0 ( #1522 )
...
Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab ) from 0.86.0 to 0.88.0.
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go )
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.86.0...v0.88.0 )
---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Bill Rich <bill.rich@gmail.com>
2023-07-25 20:31:29 -07:00
Richard Gomez
2290954b02
fix(github): use apiEndpoint for basic or no auth ( #1454 )
2023-07-25 20:03:08 -07:00
Richard Gomez
f48a635c34
feat: update gitparse logic ( #1486 )
2023-07-25 17:52:34 -05:00
Miccah
91cbca941a
Fix data race in context wrapper library ( #1546 )
2023-07-25 17:09:36 -05:00
Zachary Rice
1a1977f7e6
case insensitive ( #1547 )
2023-07-25 17:01:15 -05:00
Bill Rich
f39303495a
Add commitsScanned metrics ( #1533 )
...
* Add commitsScanned metrics
* Just keep commit count
2023-07-25 11:31:01 -07:00
ahrav
b5b01d3eba
[chore] - optimize chunker ( #1535 )
...
* Use chunkbytes that includes the size of peek.
* linter.
* continue.
* add TotalChunkSize const.
2023-07-24 19:30:29 -07:00
Zachary Rice
85f363f093
init ( #1538 )
2023-07-24 19:09:57 -05:00
Dustin Decker
302c3ce8d1
Downgrade couchbase dep ( #1543 )
...
* update couchbase dep
* downgrade couchbase dep
2023-07-24 17:00:13 -07:00
Dustin Decker
fd6c281589
update couchbase dep ( #1540 )
2023-07-24 16:27:11 -07:00
Miccah
93c561f324
Add match boundary to okta regular expressions ( #1531 )
2023-07-24 10:52:50 -05:00
Dustin Decker
fab80445d1
continue scanning on detector / decoder panic ( #863 )
2023-07-24 07:34:43 -07:00
ahrav
9e0a2e9ddd
[chore] - Remove password info from log ( #1528 )
...
* Remove password info from log.
* update.
* one more.
2023-07-22 20:25:45 -07:00
Miccah
69515bb7ac
Correctly route pprof endpoint ( #1527 )
2023-07-21 22:45:27 -05:00
