Bill Rich
ceff786db4
Skip all binaries ( #2256 )
...
* Skip all binaries
* Remove noop
* Drop handlerOpts
2023-12-22 12:01:07 -08:00
Dustin Decker
7d93adc1d0
Add skip archive support ( #2257 )
2023-12-22 11:55:23 -08:00
Dustin Decker
f699f60e89
use walk dir for tmp cleanup ( #2255 )
2023-12-22 07:52:58 -08:00
ahrav
39f0310f1f
[fixup] - Refactor to Pass Reader for Binary Diffs and Archived Data; Optimize /tmp Directory Cleanup ( #2253 )
2023-12-22 07:41:54 -08:00
Cody Rose
9c8674777c
Dedupe some source log keys ( #2250 )
...
The source manager attaches some context keys, but in certain circumstances, they're already present, resulting in duplicate keys. This PR changes the attachment to be conditional. It also adds some new log messages to track source startup progress.
2023-12-21 10:11:52 -08:00
ahrav
07ae9ec870
Fix goroutine leak ( #2251 )
2023-12-20 21:09:05 -08:00
ahrav
28212c9a82
[chore] - lower logging level ( #2249 )
...
* lower logging level
* move pid len check outside loop
2023-12-20 09:50:32 -08:00
ahrav
4a66dddd81
[chore] - add additional binary extensions to skip ( #2235 )
...
* add additional binary extensions to skip
* remove whl
2023-12-20 06:57:23 -08:00
ahrav
f5d0f3f366
use snake_case for naming ( #2238 )
2023-12-20 06:57:00 -08:00
ahrav
5848f5b8d6
[bug] - Bug archive handler memory leak ( #2247 )
2023-12-20 06:16:58 -08:00
Dustin Decker
6b90a96ca0
Add missing import ( #2246 )
2023-12-19 16:40:31 -08:00
Richard Gomez
69d5e0c993
fix(snowflake): avoid extraneous attempts ( #2057 )
...
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-12-19 16:04:08 -08:00
Richard Gomez
97afd570ae
feat(github): update extradata ( #2219 )
2023-12-19 16:01:07 -08:00
Mike Vanbuskirk
adba91da50
Adds basic if/else check if pid slice is empty ( #2244 )
...
* adds basic if/else check for pid slice
* use continue instead of if
2023-12-18 22:24:49 -05:00
ahrav
64c7365364
add secretID to chunk ( #2242 )
2023-12-18 15:27:49 -08:00
Richard Gomez
eeb4dbd304
feat(shortcut): use tri-state verification ( #2211 )
2023-12-17 15:47:14 -08:00
Richard Gomez
ded8e459bd
feat(huggingface): enhance extradata ( #2222 )
2023-12-17 14:29:45 -08:00
Richard Gomez
69a70a3374
fix(myfreshworks): check for valid JSON ( #2212 )
2023-12-17 10:26:38 -08:00
Miccah
88281bc354
[chore] Add skip_binaries field to AzureRepos proto message ( #2232 )
...
* [chore] Add skip_binaries field to AzureRepos proto message
* Make protos
2023-12-15 12:23:46 -08:00
ahrav
5c6ce693c1
[feat] - Make skipping binaries configurable ( #2226 )
...
* Make skipping binaries configurable
* remove ioutil
* fix
* address comments
* address comments
* use multi-reader
* remove print
* use const
* fix test
* fix my stupidness
2023-12-15 11:46:27 -08:00
Miccah
78b5a95342
[chore] Prevent panic when ChunkError has a nil Unit ( #2227 )
2023-12-15 11:11:28 -08:00
Richard Gomez
b3040b1227
fix(github): remove unused 'members' var ( #2202 )
2023-12-14 11:53:24 -08:00
Miccah
f6bbc59bf6
Check for SourceUnit support dynamically in the SourceManager ( #2205 )
...
* Check for SourceUnit support dynamically in the SourceManager
* Only call the function if we can use source units
2023-12-14 11:48:15 -08:00
Richard Gomez
06b137fd93
fix(gitlab): check for valid JSON ( #2218 )
2023-12-14 11:22:06 -08:00
ahrav
d8cb65833c
Avoid reading decompressed data into memory ( #2196 )
2023-12-14 11:00:11 -08:00
Richard Gomez
e72fdb62e4
fix(gitparse): don't trim filename ( #2201 )
2023-12-14 08:29:46 -08:00
Richard Gomez
22ae6a7a8f
fix(giturl): encode '%' in path ( #2214 )
2023-12-14 08:06:09 -08:00
Miccah
4db20e29f8
Update metabase verification to check for a valid JSON response ( #2210 )
...
* Update metabase verification to check for a valid JSON response
* added test tokens + cleanup
---------
Co-authored-by: ahmed <ahmed.zahran@trufflesec.com>
2023-12-13 12:18:56 -08:00
Miccah
84b7461796
[chore] Remove unnecessary string conversion in tefter detector ( #2209 )
2023-12-13 11:39:16 -08:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
6987507534
fix and refactor browserstack detector ( #2208 )
...
* fix and refactor browserstack detector
2023-12-12 16:14:31 -05:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
5e3ea1a8f2
Fix azurestorage detector ( #2207 )
...
* bugfix + cleanup - update azurestorage detector raw string to use key instead of id
2023-12-12 16:07:09 -05:00
Miccah
9f6a47da3f
[chore] Remove omitempty tags on JobProgressMetrics and UnitMetrics ( #2204 )
2023-12-12 10:02:56 -08:00
Mike Vanbuskirk
53f060a08e
Add disk buffer tempfile cleanup ( #2130 )
...
* add tempfile creation
- break PID retrieval into sep. function
* add tmpfile cleanup func
* add file cleanup to main cleanup func
* refactor file logic to only return name string
* add temp buffer naming to gcs
* add temp buffer naming to s3
* add temp buffer naming to filesystem
* add temp buffer naming to git
* consolidate cleanup functions
- have single function handle both files and dirs
- remove interface(not needed with a single func implementation)
- change calls to `New(...)` to reflect config implementation
- simplify automation in main.go
- update disk-buffer-reader dependency
* integrate changes from pr #2133
* merge main
* checkout from main to revert conflict issues
* re-add buffer logic to git
* interface no longer needed
* move string format to global const
---------
Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2023-12-11 18:31:50 -05:00
Cody Rose
405f356071
Use bad json in slackwebhooks ( #2193 )
...
* add rotation guides to SlackWebhook tests
* begin cleaning up tests
* have slack webhook detector use malformed json
* update test secrets
---------
Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2023-12-11 15:04:55 -08:00
ahrav
61c7d52a43
[bug] - close file after reading ( #2203 )
...
* close file after reading
* inline return
2023-12-11 15:04:30 -08:00
Richard Gomez
d1a2d9e832
chore: propagate log context to handlers ( #2191 )
2023-12-10 10:30:11 -08:00
Richard Gomez
6c5fc2f212
feat(privatekey): run checks concurrently ( #2139 )
2023-12-10 10:11:17 -08:00
ahrav
331336dc0a
[fixup] - skip files in the archive handler ( #2195 )
2023-12-08 20:23:32 -08:00
ahrav
2728e514d2
move logic to main Chunks method ( #2194 )
2023-12-08 14:51:24 -08:00
ahrav
2a7813929b
add metrics for gitlab ( #2190 )
2023-12-08 09:50:09 -08:00
ahrav
4b31b39d6b
[chore] - Refactor common code into a separate function ( #2179 )
...
* Refactor common code into a separate function
* rename vars
* make sure to set the scanOptions fields
* address comments
2023-12-08 08:44:35 -08:00
Cody Rose
ee6923a241
Remove java archives from ignored extensions ( #2188 )
...
A previous commit (5d0196957f
2023-12-07 15:19:56 -05:00
ahrav
b75991850a
[chore] - Compile regex once ( #2176 )
...
* move regex compilation out of the fxn
* missed a spot
* merge main
2023-12-07 07:26:27 -08:00
ahrav
f772fd8b44
update regex ( #2184 )
2023-12-06 17:04:38 -08:00
Dustin Decker
3167dde8a1
Deprecate some detectors ( #2186 )
2023-12-06 16:57:55 -08:00
ahrav
0595a3baac
allow targets for the source manager ( #2182 )
...
* allow targets to the source manager
* use targets
2023-12-06 16:38:35 -08:00
ahrav
c6e9b8ff64
use https for verification endpoints ( #2185 )
2023-12-06 16:06:04 -08:00
ahrav
e6bc7f4451
remove unnecessary Git cmd check ( #2175 )
2023-12-06 13:38:34 -08:00
ahrav
cb81f7d11a
[feat] - Remove go-git dependency ( #2174 )
...
* remove use of go-git for binary files
* fix it
* use limit reader
* fix comment
* fix test
* address comments
* address comments
* address comments
2023-12-06 13:38:01 -08:00
ahrav
990274b596
Skip trying to determine MIME type for directories ( #2178 )
2023-12-06 12:00:18 -08:00
