Commit graph

1993 commits

Author SHA1 Message Date
Dustin Decker
4086895249
add scripts to benchmark and plot performance across tags (#1293)
* add scripts to benchmark and plot performance across tags

* missing newline

* fmt
2023-04-26 15:43:23 -07:00
Brendan Shaklovitz
10902f802a
Add max object size flag for s3 bucket scanning (#1294)
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-26 15:39:43 -07:00
Aman Sakhuja
2a3f8942ee
Fixed contentfulpersonalaccesstoken regex (#1199) 2023-04-26 14:32:36 -07:00
Shabbir B
d1cbc54fc6
Updated BrowserStack detector endpoint (#1290)
Updated endpoint
2023-04-26 08:59:24 -07:00
Miccah
5a86c18302
Fix include and exclude detector logic (#1267)
* Fix include and exclude detector logic

* Fix test

* Add more clarifying comments
2023-04-26 10:49:54 -05:00
ahrav
622700b6ec
update proto to allow for ignoring projects. (#1289) 2023-04-26 07:30:43 -07:00
ahrav
15ed428e28
update jira detector. (#1288) 2023-04-25 17:26:51 -07:00
Shabbir B
6f801f64c7
Added a new detector for percy.io (#1284)
* Feature: Added a new detector for percy.io

* Updated variable name

---------

Co-authored-by: ahrav <ahravdutta02@gmail.com>
2023-04-25 13:18:34 -07:00
Dustin Decker
3485a6dab1
improve sqlserver detection and testing (#1285)
* improve sqlserver detection and testing

* add data source keyword
2023-04-25 11:00:37 -07:00
ahrav
34f5db64ae
Small optimizations for the base64 decoder (#1278)
* Small optimizations.

* remove unnecessary timer reset.

* remove blank lines.

* remove test file.

* Move b64 character mapping creation to init.
2023-04-24 11:27:07 -07:00
dependabot[bot]
91bd843ba7
Bump cloud.google.com/go/storage from 1.29.0 to 1.30.1 (#1282)
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.29.0 to 1.30.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.29.0...spanner/v1.30.1)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:37:48 -07:00
dependabot[bot]
584ab043a2
Bump github.com/envoyproxy/protoc-gen-validate from 0.10.1 to 1.0.0 (#1280)
Bumps [github.com/envoyproxy/protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate) from 0.10.1 to 1.0.0.
- [Release notes](https://github.com/envoyproxy/protoc-gen-validate/releases)
- [Changelog](https://github.com/bufbuild/protoc-gen-validate/blob/main/.goreleaser.yaml)
- [Commits](https://github.com/envoyproxy/protoc-gen-validate/compare/v0.10.1...v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/envoyproxy/protoc-gen-validate
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:27:10 -07:00
dependabot[bot]
7578a3a56e
Bump google.golang.org/api from 0.118.0 to 0.119.0 (#1279)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.118.0 to 0.119.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.118.0...v0.119.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-24 10:12:12 -07:00
ahrav
800ac30ea0
optimize base64 decoder. (#1277) 2023-04-20 20:36:46 -07:00
ahrav
abdff53d5d
optimize utf-8 decoder (#1275)
* optimize utf-8 decoder.

* remove string conversion.
2023-04-20 16:52:34 -07:00
Yassine Ilmi
a002ba9a75
Add RawV2 Results to the JSON Output (#1273)
* Add RawV2 to JSON Output

* Adding RawV2 results to Azure, Datadog and GCP Detectors
2023-04-20 16:31:53 -07:00
ahrav
cec1543894
Add utf16 decoder proto. (#1276) 2023-04-20 15:25:36 -07:00
Dustin Decker
e217e2fbfd
Ensure multipart credentials are deduplicated correctly (#1271)
* Ensure multipart credentials are deduplicated correctly

* update tests
2023-04-20 15:07:59 -07:00
ahrav
4116a24b1c
Add utf16 decoder (#1274)
* Add utf16 decoder.

* Add test for utf-8.

* Remove else if.

* optimize to use a single loop.
2023-04-20 15:07:49 -07:00
Bill Rich
a6902ae9cb
Add configurable detectors (#1139)
* JDBC detector ignore patterns

* Remove newline

---------

Co-authored-by: Bill Rich <bill.rich@trufflesec.com>
2023-04-20 11:44:28 -07:00
Zubair Khan
6dd24d17d4
Switch Endpoint Field to Client ID (#1270)
* no longer using endpoint, but do need client id

* use oauth2 credential type
2023-04-19 16:54:37 -04:00
ahrav
5b2b434a8a
Allow multipel team IDs. (#1259) 2023-04-19 11:47:25 -07:00
ah̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
1c1f034468
Adding Google drive to MetaData proto (#1264) 2023-04-19 10:49:40 -04:00
dependabot[bot]
661e62f006
Bump github.com/xanzy/go-gitlab from 0.81.0 to 0.83.0 (#1268)
Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.81.0 to 0.83.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.81.0...v0.83.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-18 08:48:26 -07:00
dependabot[bot]
92244019a9
Bump github.com/lib/pq from 1.10.7 to 1.10.8 (#1260)
Bumps [github.com/lib/pq](https://github.com/lib/pq) from 1.10.7 to 1.10.8.
- [Release notes](https://github.com/lib/pq/releases)
- [Commits](https://github.com/lib/pq/compare/v1.10.7...v1.10.8)

---
updated-dependencies:
- dependency-name: github.com/lib/pq
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-18 08:36:17 -07:00
Miccah
dfc5a9f5db
[chore] Log possible duplicate detectors (#1266)
* [chore] Log possible duplicate detectors

* Fix typos
2023-04-18 10:36:00 -05:00
dependabot[bot]
f5ecbc857b
Bump github.com/xanzy/go-gitlab from 0.81.0 to 0.82.0 (#1262)
Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.81.0 to 0.82.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.81.0...v0.82.0)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-18 08:34:01 -07:00
dependabot[bot]
42975c6d1e
Bump google.golang.org/api from 0.114.0 to 0.118.0 (#1261)
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.114.0 to 0.118.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.114.0...v0.118.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-18 08:33:14 -07:00
Bill Rich
0507f0eb87
Only add detectors once (#1265) 2023-04-17 14:10:13 -07:00
ahrav
531e3ef6c3
Add team name to proto. (#1258) 2023-04-14 09:07:23 -07:00
Dustin Decker
97ce27153a
[]bytes were being logged as b64ed string (#1255) 2023-04-14 06:43:26 -07:00
Zachary Rice
1c89e79c2d
Remove toLower call on decoded chunk (#1254)
* remove to lower on decoded data

* clean up
2023-04-14 07:29:32 -05:00
ahrav
461f1a631e
[chore] - use hex encode vs base64 (#1256)
* use hex encode vs base64.

* fix tests.
2023-04-13 19:16:06 -07:00
ahrav
2fbf86a6ab
Use md5 hash for resuming key (#1203)
* Add in-memory caching lib, used by the GCS source.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* Fix static check.

* Add test for NewWithData.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* update comment.

* update comments.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* Include md5 hash to the object struct.

* remove unused dep.

* address comments.

* Add exists method.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* rebase.

* fix test.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* rebase.

* rebase.

* split encode resume by comma.

* update comment.

add comment for shouldCache.

remove redundant return.

* use md5 instead of name.

* update tests.

* Include md5 hash to the object struct.

* use md5 instead of name.

* update tests.

* Use a persistable cache.

* fix merge.

* fix merge.

* Include md5 hash to the object struct.

* use md5 instead of name.

* update tests.

* use md5 instead of name.

* update progress tests.

* use name for log message.

* remove slice operation.
2023-04-13 18:26:45 -07:00
Zachary Rice
458c79165a
fix extra log messages (#1253)
* fix extra log messages

* add small test, move flag to isindex
2023-04-13 09:53:21 -05:00
Dustin Decker
1db22599af
update circle test because workflows expire and need re-running (#1251) 2023-04-10 16:21:19 -07:00
Miccah
562f3aaa2a
Generate protos (#1250) 2023-04-10 17:15:53 -05:00
Zubair Khan
2c1d8fbba4
THOG-920/add oss proto (#1240)
* add sharepoint source proto

* create sharepoint oss protos

* add email field, remove oauth2 type, update token to refresh_token

* rename path to link

* restore clientcredentials

* restore s3 and confluence proto changes from make command
2023-04-10 10:30:47 -07:00
dependabot[bot]
f1cb933a5a
Bump go.mongodb.org/mongo-driver from 1.11.3 to 1.11.4 (#1247)
Bumps [go.mongodb.org/mongo-driver](https://github.com/mongodb/mongo-go-driver) from 1.11.3 to 1.11.4.
- [Release notes](https://github.com/mongodb/mongo-go-driver/releases)
- [Commits](https://github.com/mongodb/mongo-go-driver/compare/v1.11.3...v1.11.4)

---
updated-dependencies:
- dependency-name: go.mongodb.org/mongo-driver
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 10:29:48 -07:00
dependabot[bot]
256ae20079
Bump github.com/bradleyfalzon/ghinstallation/v2 from 2.2.0 to 2.3.0 (#1245)
Bumps [github.com/bradleyfalzon/ghinstallation/v2](https://github.com/bradleyfalzon/ghinstallation) from 2.2.0 to 2.3.0.
- [Release notes](https://github.com/bradleyfalzon/ghinstallation/releases)
- [Commits](https://github.com/bradleyfalzon/ghinstallation/compare/v2.2.0...v2.3.0)

---
updated-dependencies:
- dependency-name: github.com/bradleyfalzon/ghinstallation/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 10:11:46 -07:00
dependabot[bot]
5fce8f9f18
Bump golang.org/x/crypto from 0.7.0 to 0.8.0 (#1244)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/golang/crypto/releases)
- [Commits](https://github.com/golang/crypto/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 10:09:40 -07:00
dependabot[bot]
304abb9ca9
Bump golang.org/x/oauth2 from 0.6.0 to 0.7.0 (#1243)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/oauth2/releases)
- [Commits](https://github.com/golang/oauth2/compare/v0.6.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 10:01:46 -07:00
dependabot[bot]
a8c27b9510
Bump golang.org/x/text from 0.8.0 to 0.9.0 (#1246)
Bumps [golang.org/x/text](https://github.com/golang/text) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/golang/text/releases)
- [Commits](https://github.com/golang/text/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/text
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-10 10:00:15 -07:00
ahrav
c451f9daf8
Use persistable cache for GCS progress tracking (#1204)
* Add in-memory caching lib, used by the GCS source.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* Fix static check.

* Add test for NewWithData.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* update comment.

* update comments.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* remove unused dep.

* address comments.

* Add exists method.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* rebase.

* fix test.

* Use cache for tracking progress for the GCS source.

* fix merge issue.

* fix merge issue.

* fix test.

* rebase.

* rebase.

* split encode resume by comma.

* Use a persistable cache.

* fix merge.

* fix merge.

* Add progress as part of the cache given it will be the persistence layer.

* Add test for making sure the cache doesn't persist when the increment value is not met.

* fix tests.
2023-04-10 07:55:00 -07:00
ahrav
f107e1b497
Use defautl endpoints when no custom verifier provided. (#1242) 2023-04-06 08:35:01 -07:00
iamjpotts
b3d917f9c7
Resolve #1167 by adding support for the AWS_SESSION_TOKEN (#1170)
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg

* fix error message

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-03 14:56:43 -07:00
Dustin Decker
20d5683199
fix linting step (#1235) 2023-04-03 13:21:58 -07:00
Dustin Decker
58222610ff
readme improvements (#1234)
* readme improvements

* Add logos

* simplify points

* update headings and order

* fix heading sizes
2023-04-03 10:29:57 -07:00
Miccah
3932486ea9
Add lint for exporting loop references (#1232) 2023-04-03 09:26:32 -05:00
Dustin Decker
8f10938bf7
forager requires direct access to gitparse.FromReader (#1233) 2023-04-02 17:54:43 -07:00