Commit graph

2825 commits

Author SHA1 Message Date
ahrav
5c6ce693c1
[feat] - Make skipping binaries configurable (#2226)
* Make skipping binaries configurable

* remove ioutil

* fix

* address comments

* address comments

* use multi-reader

* remove print

* use const

* fix test

* fix my stupidness
2023-12-15 11:46:27 -08:00
Miccah
78b5a95342
[chore] Prevent panic when ChunkError has a nil Unit (#2227) 2023-12-15 11:11:28 -08:00
Richard Gomez
b0fab16ad4
chore: don't run test workflow in forks (#2221) 2023-12-14 16:48:48 -08:00
Richard Gomez
b3040b1227
fix(github): remove unused 'members' var (#2202) 2023-12-14 11:53:24 -08:00
Miccah
f6bbc59bf6
Check for SourceUnit support dynamically in the SourceManager (#2205)
* Check for SourceUnit support dynamically in the SourceManager

* Only call the function if we can use source units
2023-12-14 11:48:15 -08:00
Richard Gomez
06b137fd93
fix(gitlab): check for valid JSON (#2218) 2023-12-14 11:22:06 -08:00
ahrav
d8cb65833c
Avoid reading decompressed data into memory (#2196) 2023-12-14 11:00:11 -08:00
Richard Gomez
e72fdb62e4
fix(gitparse): don't trim filename (#2201) 2023-12-14 08:29:46 -08:00
Richard Gomez
22ae6a7a8f
fix(giturl): encode '%' in path (#2214) 2023-12-14 08:06:09 -08:00
Richard Gomez
f38b6a27b6
build: upgrade bodgit/sevenzip to v1.4.5 (#2215) 2023-12-14 07:48:39 -08:00
Richard Gomez
07d9d1c5ef
Fix emoji in README (#2217) 2023-12-14 07:37:58 -08:00
dependabot[bot]
e8db0f2cd0
Bump github.com/docker/docker (#2213)
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 24.0.0+incompatible to 24.0.7+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v24.0.0...v24.0.7)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-13 16:13:44 -08:00
Miccah
4db20e29f8
Update metabase verification to check for a valid JSON response (#2210)
* Update metabase verification to check for a valid JSON response

* added test tokens + cleanup

---------

Co-authored-by: ahmed <ahmed.zahran@trufflesec.com>
2023-12-13 12:18:56 -08:00
Miccah
84b7461796
[chore] Remove unnecessary string conversion in tefter detector (#2209) 2023-12-13 11:39:16 -08:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
6987507534
fix and refactor browserstack detector (#2208)
* fix and refactor browserstack detector
2023-12-12 16:14:31 -05:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
5e3ea1a8f2
Fix azurestorage detector (#2207)
* bugfix + cleanup - update azurestorage detector raw string to use key instead of id
2023-12-12 16:07:09 -05:00
Miccah
9f6a47da3f
[chore] Remove omitempty tags on JobProgressMetrics and UnitMetrics (#2204) 2023-12-12 10:02:56 -08:00
Mike Vanbuskirk
53f060a08e
Add disk buffer tempfile cleanup (#2130)
* add tempfile creation

- break PID retrieval into sep. function

* add tmpfile cleanup func

* add file cleanup to main cleanup func

* refactor file logic to only return name string

* add temp buffer naming to gcs

* add temp buffer naming to s3

* add temp buffer naming to filesystem

* add temp buffer naming to git

* consolidate cleanup functions

- have single function handle both files and dirs
- remove interface(not needed with a single func implementation)
- change calls to `New(...)` to reflect config implementation
- simplify automation in main.go
- update disk-buffer-reader dependency

* integrate changes from pr #2133

* merge main

* checkout from main to revert conflict issues

* re-add buffer logic to git

* interface no longer needed

* move string format to global const

---------

Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2023-12-11 18:31:50 -05:00
Cody Rose
405f356071
Use bad json in slackwebhooks (#2193)
* add rotation guides to SlackWebhook tests

* begin cleaning up tests

* have slack webhook detector use malformed json

* update test secrets

---------

Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2023-12-11 15:04:55 -08:00
ahrav
61c7d52a43
[bug] - close file after reading (#2203)
* close file after reading

* inline return
2023-12-11 15:04:30 -08:00
Richard Gomez
d1a2d9e832
chore: propagate log context to handlers (#2191) 2023-12-10 10:30:11 -08:00
Richard Gomez
6c5fc2f212
feat(privatekey): run checks concurrently (#2139) 2023-12-10 10:11:17 -08:00
ahrav
331336dc0a
[fixup] - skip files in the archive handler (#2195) 2023-12-08 20:23:32 -08:00
ahrav
2728e514d2
move logic to main Chunks method (#2194) 2023-12-08 14:51:24 -08:00
ahrav
2a7813929b
add metrics for gitlab (#2190) 2023-12-08 09:50:09 -08:00
ahrav
4b31b39d6b
[chore] - Refactor common code into a separate function (#2179)
* Refactor common code into a separate function

* rename vars

* make sure to set the scanOptions fields

* address comments
2023-12-08 08:44:35 -08:00
Cody Rose
ee6923a241
Remove java archives from ignored extensions (#2188)
A previous commit (5d0196957f) added .jar/.war/.ear files to the ignored extensions list, but these are archive files that we can scan, so we shouldn't exclude them.
2023-12-07 15:19:56 -05:00
ahrav
b75991850a
[chore] - Compile regex once (#2176)
* move regex compilation out of the fxn

* missed a spot

* merge main
2023-12-07 07:26:27 -08:00
ahrav
f772fd8b44
update regex (#2184) 2023-12-06 17:04:38 -08:00
Dustin Decker
3167dde8a1
Deprecate some detectors (#2186) 2023-12-06 16:57:55 -08:00
ahrav
0595a3baac
allow targets for the source manager (#2182)
* allow targets to the source manager

* use targets
2023-12-06 16:38:35 -08:00
ahrav
c6e9b8ff64
use https for verification endpoints (#2185) 2023-12-06 16:06:04 -08:00
ahrav
e6bc7f4451
remove unnecessary Git cmd check (#2175) 2023-12-06 13:38:34 -08:00
ahrav
cb81f7d11a
[feat] - Remove go-git dependency (#2174)
* remove use of go-git for binary files

* fix it

* use limit reader

* fix comment

* fix test

* address comments

* address comments

* address comments
2023-12-06 13:38:01 -08:00
ahrav
990274b596
Skip trying to determine MIME type for directories (#2178) 2023-12-06 12:00:18 -08:00
dylanTruffle
96aa50d119
fixing how to rotate URL (#2183) 2023-12-06 11:59:21 -08:00
Bill Rich
a031de12de
Use forked sevenzip (#2180) 2023-12-05 15:38:20 -08:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
11394ea318
[thog-1548] add auto redaction for verification errors (#2106)
* Updating VerificationError to have auto redaction logic
* find/replace error
2023-12-05 08:57:52 -05:00
renovate[bot]
692582fbd9
fix(deps): update module github.com/google/go-github/v42 to v57 (#2172)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-04 16:55:03 -08:00
renovate[bot]
16cf858495
chore(deps): update google-github-actions/auth action to v2 (#2171)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-04 16:46:56 -08:00
ahrav
13da76d357
skip files we can't scan (#2170) 2023-12-04 13:37:11 -08:00
renovate[bot]
dbfd9a7092
fix(deps): update module google.golang.org/api to v0.152.0 (#2169)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-04 13:25:28 -08:00
ahrav
996a11dcc0
[chore] - remove deprecated types (#2168)
* remove deprecated types

* missed one
2023-12-04 13:23:58 -08:00
renovate[bot]
08b58aa549
fix(deps): update module golang.org/x/oauth2 to v0.15.0 (#2167)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-04 13:11:37 -08:00
renovate[bot]
5d0023603c
fix(deps): update module github.com/aws/aws-sdk-go to v1.48.12 (#2166)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-04 12:50:20 -08:00
renovate[bot]
83cd276ce5
fix(deps): update module github.com/xanzy/go-gitlab to v0.94.0 (#2165)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-04 12:28:37 -08:00
renovate[bot]
b5b8223b80
fix(deps): update module github.com/trufflesecurity/disk-buffer-reader to v0.2.1 (#2163)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-04 11:25:52 -08:00
Cody Rose
5d0196957f
Ignore images and binaries (#2162)
This PR expands the list of excluded file extensions to contain images and other binary files. These files can technically contain secrets, but need decoding to properly be handled, and we don't have any such decoding yet. Down the road if we want to add it we can.
2023-12-04 13:25:29 -05:00
ahrav
37d9e5eedf
[chore] - Increase pagination limit (#2154)
* increae pagination limit

* rename
2023-12-04 10:14:46 -08:00
renovate[bot]
32d81500ee
fix(deps): update module github.com/google/go-containerregistry to v0.17.0 (#2160)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-12-03 13:28:26 -08:00