Commit graph

3167 commits

Author SHA1 Message Date
Richard Gomez
5216142960
refactor(cache): use generics (#2930) 2024-06-06 13:08:00 -04:00
ahrav
ea9f8ace9f
[chore] - address comments (#2920)
* address comments

* fix test

* address comments

* update comments

* fix tests

* lint

* do the thing
2024-06-06 07:58:08 -07:00
Richard Gomez
40fa304a3a
feat(git): improve scan logging (#2923) 2024-06-06 05:12:59 -04:00
ahrav
bef4a46b65
[fix] - Correctly calculate EntireSpanChunkCalculator span (#2924)
* fix bug when calculating the span for the entire span chunk calculator

* fix rename
2024-06-05 19:21:57 -07:00
ahrav
babe48fdd1
remove stutter in naming (#2926) 2024-06-05 18:36:08 -07:00
Hon
4b3f834859
Update Jenkins in tui (#2925) 2024-06-05 18:34:01 -07:00
ahrav
b0fd70c0ff
continue on error (#2921) 2024-06-05 15:20:55 -07:00
Dustin Decker
d8e7fa983d
Go should be installed before codeql initializes (#2919) 2024-06-05 16:46:58 -04:00
ahrav
ce1ce29b90
[feat] - Optimize detector performance by reducing data passed to regex (#2812)
* optimize maching detetors

* update method name

* updates

* update naming

* updates

* update comment

* updates

* remove testcase

* update default match len to 512

* update

* update test

* add support for multpart cred provider

* add ability to scan entire chunk

* encapsulate matches logic within FindDetectorMatches

* use []byte directly

* nil chunk data

* use []byte

* set hidden flag to true

* remove

* [refactor] - multi part detectors (#2906)

* Detectors beginning w/ a

* Detectors beginning w/ b

* Detectors beginning w/ c

* Detectors beginning w/ d

* Detectors beginning w/ e

* Detectors beginning w/ f

* Detectors beginning w/ f&g

* fix

* Detectors beginning w/ i-l

* Detectors beginning w/ m-p

* Detectors beginning w/ r-s

* Detectors beginning w/ t

* Detectors beginning w/ u-z

* revert alconst

* remaining fixes

* lint

* [feat] - Add Support for `compareDetectionStrategies` Mode (#2918)

* Detector comparison mode

* remove else

* return error if results dont match

* update default hidden flag to not scan entire chunks

* fix tests

* enhance encapsulation by including methods on DetectorMatch to handle merging and extracting

* remove space

* fix

* update detector

* updates

* remove else

* run comparison concurrently
2024-06-05 13:28:19 -07:00
ahrav
aa91acc86b
[feat] - Introduce channelmetrics Package for Channel Metrics Collection (#2889)
* Add pkg for channel metrics

* Use map for different channel metrics

* go mod tidy

* use CacnellableWrite
2024-06-05 12:00:48 -07:00
Hon
c1a2019d5b
Add flag to get information if trufflehog being ran from TUI (#1644)
* Add flag to get information if trufflehog being ran from TUI

Co-authored-by: mcastorina <m.castorina93@gmail.com>

* Always use version.BuildVersion

---------

Co-authored-by: mcastorina <m.castorina93@gmail.com>
2024-06-05 10:07:50 -07:00
Richard Gomez
024b219dfb
feat(openai): add project and service account keys (#2863)
Co-authored-by: āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d <13666360+0x1@users.noreply.github.com>
2024-06-05 11:11:50 -04:00
Richard Gomez
4d2c8c6e11
refactor(github): improve wiki err handling (#2917) 2024-06-05 08:06:01 -04:00
Hon
3d7f9d96f1
Add elasticsearch to tui (#2915)
Co-authored-by: mcastorina <m.castorina93@gmail.com>
2024-06-04 14:54:39 -07:00
renovate[bot]
96a1eda308
fix(deps): update module github.com/microsoft/go-mssqldb to v1.7.2 (#2912)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-04 15:25:26 -04:00
renovate[bot]
84fd4c0fae
fix(deps): update module github.com/aws/aws-sdk-go to v1.53.15 (#2911)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-04 07:38:15 -07:00
Dustin Decker
ef410873f2
Add Jenkins scanning (#2892)
* add jenkins

* whoops

* adding unauthenticated jenkins scanning

* update docs

---------

Co-authored-by: Joe Leon <joe.leon@trufflesec.com>
2024-06-04 07:13:14 -04:00
Miccah
c86b423c61
[chore] Always log git repositories being scanned (#2909) 2024-06-03 18:02:34 -07:00
jinjiadu
911ea4d678
chore: fix some comments (#2903)
Signed-off-by: jinjiadu <jinjiadu@aliyun.com>
2024-06-03 18:26:33 -04:00
renovate[bot]
2940a3514d
fix(deps): update module github.com/azure/go-autorest/autorest/azure/auth to v0.5.13 (#2902)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-03 13:39:12 -07:00
Abdul Basit
e5a5b63845
integration testing for mongodb. (#2907)
* integration testing for mongodb.

* add more test cases.
remove dead code/comments
2024-06-03 12:59:25 -07:00
renovate[bot]
fbff476a52
fix(deps): update module github.com/hashicorp/go-retryablehttp to v0.7.7 (#2904)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-03 12:37:39 -07:00
renovate[bot]
eb41756eb6
fix(deps): update module github.com/aws/aws-sdk-go to v1.53.14 (#2900)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-06-01 09:55:39 -07:00
Richard Gomez
9053d8f4de
refactor(github): enumerateWithToken flow & tests (#2880) 2024-05-31 15:53:44 -05:00
Abdul Basit
2b3284e650
Redis integration test (#2901)
* implemented redis integration test using docker container

* rename the function and use the redis:7.0 image
2024-05-31 11:59:06 -05:00
Richard Gomez
5575514174
fix(falsepositives): remove 'www' (#2896) 2024-05-31 11:37:27 -04:00
renovate[bot]
155c83c433
fix(deps): update module github.com/charmbracelet/bubbletea to v0.26.4 (#2885)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-31 10:56:33 -04:00
Abdul Basit
ddd861d4c7
consistent image of MSSQL for integration testing. (#2898) 2024-05-31 10:56:02 -04:00
Abdul Basit
d93c428b54
Update metadata for DataDog for API + APPKey (#2879)
* put emails from response in metadata for datadog.

* removed unused type in user structure.

* filter user information based on verified, is service account and disabled boolean
also include organization detail if it is available in response.
2024-05-31 10:50:23 -04:00
renovate[bot]
568fea8110
fix(deps): update golang.org/x/exp digest to fd00a4e (#2899)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-31 07:42:08 -07:00
renovate[bot]
20014e4339
chore(deps): update alpine docker tag to v3.20 (#2874)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-31 10:21:01 -04:00
Hon
793231370e
Add postman to tui (#2895) 2024-05-29 16:07:23 -07:00
James Telfer
0024b6ce77
feat: support docker image history scanning (#2882)
* feat: support docker image history scanning

* refactor: collapse error handling into return

Style suggestion from review feedback.

* fix: associate layers with history entries

Where possible, add the associated layer to the history entry record. This may help tracing any issues discovered.

This also changes the entry reference format to `image-metadata:history:%d:created-by` which _may_ be more self-explanatory.
2024-05-28 14:07:43 -07:00
Abdul Basit
18b81013b8
Added extra data for LaunchDarkly (#2836)
* added extra data for LaunchDarkly
- Token type like api or sdk
- name and role of First token in response of api-tokens
- total token counts associated with api-token

* renamed total_count to total_token_count

* updated & renamed fields of metadata for launchdarkly, specifically for API based tokens
2024-05-28 14:08:53 -04:00
renovate[bot]
48b570d909
fix(deps): update module github.com/gabriel-vasile/mimetype to v1.4.4 (#2890)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-28 07:59:18 -07:00
renovate[bot]
5f96f3c5c5
fix(deps): update module github.com/elastic/go-elasticsearch/v8 to v8.13.1 (#2886)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-28 07:49:00 -07:00
renovate[bot]
a591f3986c
fix(deps): update golang.org/x/exp digest to 4c93da0 (#2883)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-27 05:44:30 -07:00
renovate[bot]
3460629c3f
fix(deps): update module cloud.google.com/go/secretmanager to v1.13.1 (#2884)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-27 05:44:04 -07:00
renovate[bot]
45246c3fea
fix(deps): update module github.com/go-logr/logr to v1.4.2 (#2869)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-26 08:23:01 -07:00
renovate[bot]
3632349bb3
fix(deps): update module github.com/aws/aws-sdk-go to v1.53.10 (#2871)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-25 07:57:35 -07:00
Richard Gomez
5102e3ae11
test(github): fix some errors (#2774) 2024-05-24 13:03:41 -07:00
Richard Gomez
e53f5bd5c5
Improve handling of Gist URLs (#2653)
* feat(github): handle ghes gists

* fix(github): handle all gist URLs

* refactor(github): helper func to check gist urls
2024-05-24 08:36:30 -07:00
Charlie Gunyon
311494e86e
Elastic adapter (#2727)
* Add stub source and elastic API funcs

* Spawn workers and ship chunks

* Now successfully detects a credential

- Added tests
- Added some documentation comments
- Threaded the passed context through to all the API requests

* Linting fixes

* Add integration tests and resolve some bugs they uncovered

* Logstash -> Elasticsearch

* Add support for --index-pattern

* Add support for --query-json

* Use structs instead of string building to construct a search body

* Support --since-timestamp

* Implement additional authentication methods

* Fix some small bugs

* Refactoring to support --best-effort-scan

* Finish implementation of --best-effort-scan

* Implement scan catch-up

* Finish connecting support for nodes CLI arg

* Add some integration tests around the catchup mechanism

* go mod tidy

* Fix some linting issues

* Remove some debugging Prints

* Move off of _doc

* Remove informational Printf and add informational logging

* Remove debugging logging

* Copy the index from the outer loop as well

* Don't burn up the ES API with rapid requests if there's no work to do in subsequent scans

* No need to export UnitOfWork.AddSearch

* Use a better name for the range query variable when building the timestamp range clause in searches

* Replace some unlocking defers with explicit unlocks to make the synchronized part of the code clearer

* found -> ok

* Remove superfluous buildElasticClient method

---------

Co-authored-by: Charlie Gunyon <charlie@spectral.energy>
2024-05-24 09:38:20 -05:00
Richard Gomez
1441289d41
fix(github): scan user repos (#2814) 2024-05-23 09:40:40 -05:00
Cody Rose
f7214cfee3
Log reasons for GitLab repo exclusion (#2875)
We have some evidence that some GitLab repos are getting incorrectly ignored, but it's not clear why this is happening, so this PR adds some more logging to the relevant code.
2024-05-23 09:08:36 -04:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
b9ea2f5d4e
adding Groq detector (#2873)
* adding Groq detector

* using prefix as keyword
2024-05-22 15:46:14 -04:00
ahrav
fccf7c9a41
[chore] - Use http.NewRequestWithContext (#2870) 2024-05-22 06:58:12 -07:00
renovate[bot]
0a3a62be0e
fix(deps): update module github.com/aws/aws-sdk-go to v1.53.6 (#2867)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-05-21 10:15:28 -07:00
Abdul Basit
4b10b8a009
made changes in organization regex for azure devops. (#2866) 2024-05-20 15:02:12 -05:00
Abdul Basit
5dff334ffa
Update azure storage extra data (#2808)
Blob service of Azure storage returns containers name in response. From that, containers name is added in extra data.
2024-05-20 13:42:54 -04:00