Dustin Decker
a00ffe9522
Allow multiple domains for Forager ( #2400 )
2024-02-08 07:08:30 -08:00
ahrav
bbf1decb39
prevent concurrent map writes ( #2399 )
2024-02-07 17:45:06 -08:00
Ryan Jacobchick
7296bcdc5d
Allow CLI version pinning in GHA ( #2397 ) ( #2398 )
...
* Allow CLI version pinning in GHA (#2397 )
* prevent segfault in test-community
2024-02-07 16:58:04 -06:00
Richard Gomez
b3ff12d1e9
Fix handling of GitHub ratelimit information ( #2041 )
...
This is a follow-up to #1912 , which used the headers from the response to determine rate-limiting information, instead of using the values from RateLimitError.Rate. Although that logic seemed solid, I discovered that it did not work in some circumstances. This lead to the "unexpected" path more often than intended, and periodic instances where requests would be made before the ratelimit was refreshed.
2024-02-07 09:11:12 -05:00
ahrav
7b492a690a
[feat] - use diff chan ( #2387 )
...
* use diff chan
* address comments
* add comment
* address comments
* use old ordering
* add correct author line
* Add required *Commit arg to newDiff
* address comments
2024-02-06 10:06:10 -08:00
ahrav
843334222c
[not-fixup] - Reduce memory consumption for Buffered File Writer ( #2377 )
...
* correctly use the buffered file writer
* use value from source
* reorder fields
* use only the DetectorKey as a map field
* correctly use the buffered file writer
* use value from source
* reorder fields
* add tests and update
* Fix issue with buffer slices growing
* fix test
* fix
* add singleton
* use shared pool
* optimize
* rename and cleanup
* use correct calculation to grow buffer
* only grow if needed
* address comments
* remove unused
* remove
* rip out Grow
* address coment
* use 2k default buffer
* update comment allow large buffers to be garbage collected
2024-02-06 09:22:25 -08:00
Richard Gomez
8104611d6e
fix: case-insensitive ext check ( #2383 )
2024-02-06 10:13:53 -05:00
dylanTruffle
901c851698
tightening opsgenie detection and verification ( #2389 )
...
Co-authored-by: Dylan Ayrey <dylan@Dylans-MacBook-Pro.local>
2024-02-05 17:31:09 -08:00
Miccah
01c9ac7b59
Fix binary file hanging bug in git sources ( #2388 )
...
Waiting for the sub-command will block until all of `stdout` has been
read. In some cases, we return early due to failed chunking without
reading all of the data, and thus, get stuck waiting for the command to
finish. Closing the pipe will ensure `Wait` does not block on that I/O.
2024-02-05 15:28:49 -08:00
ahrav
135cc3eb69
[fixup] - correctly use the buffered file writer ( #2373 )
...
* correctly use the buffered file writer
* use value from source
* reorder fields
* use only the DetectorKey as a map field
* address comments and use factory function
* fix optional params
* remove commented out code
2024-02-05 10:43:55 -08:00
ahrav
28d079bdad
use only the DetectorKey as a map field ( #2374 )
2024-02-05 06:53:08 -08:00
ahrav
a22874f9f0
[feat] - concurently scan the filesystem source ( #2364 )
...
* concurently scan the filesystem source
Co-authored-by: Miccah Castorina <m.castorina93@gmail.com>
* fix test
* update test
* remove return
* use error not info
* address comment
---------
Co-authored-by: Miccah Castorina <m.castorina93@gmail.com>
2024-02-03 10:49:14 -08:00
Miccah
27b30e65ed
[chore] Cleanup GitLab source errors ( #2345 )
...
* [chore] Cleanup GitLab source errors
* Ungroup compile time interface checks and revert error message
2024-02-02 20:00:34 -08:00
ahrav
382990a6bd
[bug] - use DetectorKey as the key in the detectorKeysWithResults map ( #2366 )
...
* use DetectorKey as the key in the map
* nil check
* update comment
2024-02-02 13:43:56 -08:00
Mike Vanbuskirk
f6546ffaf5
Add s3 credential validation ( #2362 )
...
* add string non-empty validation to AWS creds
* clean up import spacing
* syntax fixup
* change to non-empty validation only
* convert to lower snake_case
- https://protobuf.dev/programming-guides/style/#message-field-names
2024-02-02 12:49:46 -05:00
ahrav
b2074ad05d
Polite Verification ( #2356 )
...
* draft reverify chunks
* remove
* remove
* reduce dupe map cap
* do not verify chunk
* cli arg and use val for dupe lut
* remove counter
* skipp empty results]
* working on test and normalizing val for comparison
* forgot to save file
* optimize normalize
* reuse map
* remove print
* use levenshtein distance to check dupes
* forgot to leave in emptying map
* use slice
* small tweak
* comment
* use bytes
* praise
* use ctx logger
* add len check
* add comments
* use 8x concurrency for reverifier workers
* revert worker count
* use more workers
* process result directly for any collisions
* continue after decoder match for reverifying
* use map
* use map
* otimization and fix the bug.
* revert worker count
* better option naming
* handle identical secrets in chunks
* update comment
* update comment
* fix test
* use DetecotrKey
* rm out of scope tests and testdata
* rename all reverification elements
* don't re-write map entry
* use correct key
* rename worker, remove log val
* test likelydupe, add eq detector check in loop
* add test
* add comment
* add test
* Set verification error
* Update tests
---------
Co-authored-by: Zachary Rice <zachary.rice@trufflesec.com>
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-02-02 09:29:18 -08:00
Dustin Decker
c2ae31d060
Make AzureDevopsPersonalAccessToken verification more robust ( #2359 )
...
* Make AzureDevopsPersonalAccessToken verification more robust
* fix snifftest
2024-02-01 08:40:44 -08:00
roxanne-tampus
143e275272
update azure test files to check rawV2 ( #2353 )
2024-01-31 08:36:52 -08:00
Miccah
24d0680f5c
[chore] Add filesystem integration test ( #2358 )
2024-01-31 08:27:57 -08:00
Richard Gomez
8e90c4e669
Scan GitHub wikis #2233
2024-01-31 10:52:24 -05:00
Marlon
91d6496a76
added flyio protos ( #2357 )
...
* added flyio protos
* added builtwith proto
---------
Co-authored-by: root <root@ubuntutruffle.myguest.virtualbox.org>
2024-01-31 07:02:06 -08:00
ahrav
9867ce8eb8
Allow for configuring the buffered file writer ( #2319 )
...
* Write large diffs to tmp files
* address comments
* Move bufferedfilewriter to own pkg
* update test
* swallow write err
* use buffer pool
* use size vs len
* use interface
* fix test
* update comments
* fix test
* Allow for configuring the buffered file writer
* remove unused
* add missing method
* remove
* remove unused
* move parser and commit struct closer to where they are used
* linter change
* fix snifftest
* address comments
* add more kvp pairs to error
* fix test
* update
* add back missing metadata fields
* address comments
* remove bufferedfile writer
* fix
* address comments
* use unint8
* update interface
* adjust interface
* fix tests
* make linter happy
* fix finalize
* address comments
* update test
* address comments
* lint
* remove guard
* fix test
* fix
* add TODO
* fix tests
2024-01-30 12:51:58 -08:00
ahrav
7c59ff95d5
[feat] - tmp file diffs ( #2306 )
...
* Write large diffs to tmp files
* address comments
* Move bufferedfilewriter to own pkg
* update test
* swallow write err
* use buffer pool
* use size vs len
* use interface
* fix test
* update comments
* fix test
* remove unused
* remove
* remove unused
* move parser and commit struct closer to where they are used
* linter change
* add more kvp pairs to error
* fix test
* update
* address comments
* remove bufferedfile writer
* address comments
* adjust interface
* fix finalize
* address comments
* lint
* remove guard
* fix
* add TODO
2024-01-30 12:30:51 -08:00
Miccah
6824eb41ea
Fix filesystem enumeration ignore paths bug ( #2355 )
2024-01-30 12:21:37 -08:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
7ece4c3e66
Detectors Updates 1 for Tristate Verification ( #2187 )
...
* updating alibaba
* updating agora
* updating aeroworkflow
* updating aha
* updating artifactory
* updating abbysale
* updating abstract
* updating abuseipdb
* updating accuweather
* updating adafruitio
* updating adzuna
* cleanup on abuseipdb
* cleanup on aha
* cleanup on abuseipdb
* cleanup on aeroworkflow
* cleanup on adzuna
* cleanup on accuweather
* cleanup/refactor
* update token pattern to be explicitly 73char (old) or 64char (new)
* comment to clarify 403 on Aha
* mocking out verified case for aha + adding inactive account test
* using contact response instead of gock
* update 403 to be determinate
2024-01-30 12:20:56 -05:00
Richard Gomez
232032410c
feat(detectors): update template ( #2342 )
2024-01-29 21:21:23 -08:00
Dustin Decker
453792de76
Azure function key is throwing FPs ( #2352 )
...
* Merge branch 'main' of github.com:trufflesecurity/trufflehog
* AZF throwing FPs
* update snifftest script
2024-01-29 15:32:12 -05:00
Dustin Decker
7befefd369
Improve fp ignore logic ( #2351 )
...
* forgot field change
* use aho corasick for filter
* reduce wordlist sensitivity
2024-01-29 11:28:46 -08:00
roxanne-tampus
303e191f38
added azuresearchquerykey detector ( #2349 )
...
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-29 10:19:51 -08:00
roxanne-tampus
06b7ebf824
added azuresearchadminkey detector ( #2348 )
...
* added azuresearchadminkey detector
* additional update
* update import
* fix raw fields for new detectors and entropy check
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-29 09:55:46 -08:00
roxanne-tampus
83dc986b07
added azurefunctionkey detector ( #2337 )
...
* added azurefunctionkey detector
* update raw field to include url
* clean up and added prefix on key pattern
* update bench script
* update imports, snifftest, and gen proto
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-29 09:20:33 -08:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
fa1c5fa583
updates to plain and json printing to include verification error ( #2335 )
2024-01-29 11:57:28 -05:00
faktas2
76fcdae3a0
Add the new MaxMind license key format ( #2181 )
...
* Add the new MaxMind license key format
* feedback
* reorg rules
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-26 11:49:47 -08:00
Dustin Decker
3a6cfd9d97
Prevent print or logging in detectors ( #2341 )
...
* Prevent print or logging in detectors
* mount repo
* update job name
2024-01-26 11:39:41 -08:00
ahrav
41cfec4e1c
make sure to close connections after testing ( #2343 )
2024-01-26 09:24:06 -08:00
Dustin Decker
f078df43eb
Fix test ( #2339 )
2024-01-25 23:15:37 -08:00
Zubair Khan
76057618ba
add tri-state verification to yelp ( #1736 )
...
* init yelp tri state
* fix detector and test
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-25 22:28:16 -08:00
Richard Gomez
38eb5d08e7
Improve GitHub scan logging ( #2220 )
...
* feat(github): improve scan logging
* Move metric
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-25 22:11:01 -08:00
Richard Gomez
c6f73a7d10
Update DockerHub detector logic ( #2266 )
...
* feat(dockerhub): update logic
* updates
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-25 22:02:36 -08:00
Richard Gomez
d4a0645c29
Add Google oauth2 token detector ( #2274 )
...
* feat(google): add oauth2 token detector
* update pr
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-25 21:32:03 -08:00
ahrav
f209b04d5d
add priority semaphore ( #2336 )
2024-01-24 16:43:56 -08:00
joeleonjr
792266afa9
updating doppler logic ( #2329 )
...
* updating doppler logic
* added json response struct
2024-01-24 12:20:33 -05:00
roxanne-tampus
47c6539a41
added azuredevopspersonalaccesstoken detector ( #2315 )
...
* added azuredevopspersonalaccesstoken detector
* fix comment
* update raw field to include all parts of the credential
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-01-23 20:46:51 -08:00
Miccah
4c698fc1e8
Walk directories in filesystem source enumeration ( #2313 )
...
* Walk directories in filesystem source enumeration
* Ignore all directories instead of just the root
* Fix bug with multiple directories
* Skip filesystem TestEnumerate
* Update filesystem enumeration test to create files and folders
2024-01-23 14:57:38 -08:00
ahrav
e44802a31d
[feat] - Replace regexp pkg w/ go-re2 in detectors ( #2324 )
...
* update detectors to use go-re regex library replacement
* update go mod and sum
* add tests with invalid utf-8
* revert
2024-01-23 13:16:22 -08:00
Cody Rose
80f2696ae0
Update Gitlab repo count in tests #2333
2024-01-23 15:04:11 -05:00
Cody Rose
e0a1b11a8e
Narrow Postgres detector to only look for URIs ( #2314 )
2024-01-23 14:42:31 -05:00
joeleonjr
05d189c977
fixing incorrect acct num id for some aws keys ( #2332 )
2024-01-23 13:27:50 -05:00
joeleonjr
443ef98e41
updating detector logic for zenscrape ( #2316 )
...
* updating detector logic for zenscrape
* updating to use status endpoint
2024-01-22 16:57:51 -05:00
Miccah
2d96b89554
Add prometheus metrics to measure hook execution time ( #2312 )
...
* Add prometheus metrics to measure hook execution time
* Move metrics to separate file and reduce buckets
2024-01-22 11:47:45 -08:00
ahrav
383f8a1f67
[chore] - reduce test time ( #2321 )
...
* reduce test time
* remove commented out code
2024-01-22 09:40:32 -08:00
ahrav
d3d551d24e
[chore] - Update Chunk struct comment ( #2317 )
...
* update comment to include information on the importance of struct ordering
* more cute tricks
* remove cute tricks
2024-01-20 13:31:27 -08:00
ahrav
8380e1713e
save 8 bytes per chunk ( #2310 )
2024-01-18 13:20:06 -08:00
ahrav
b0fd951652
[chore] - Add regex and keyword for api_org tokens ( #2240 )
...
* Add regex and keyword for api_org tokens.
* handle org token auth struct
* update keywords
2024-01-16 15:25:26 -08:00
Miccah
c5af979aee
Assume unauthenticated github scans have public visibility ( #2308 )
2024-01-16 14:57:06 -08:00
ahrav
a1dc660f41
[fixup ] - Allow ssh cloning with AWS Code Commit ( #2307 )
2024-01-16 11:55:17 -08:00
roxanne-tampus
d6419a8ab2
added azure protos ( #2304 )
2024-01-15 06:59:47 -08:00
Dustin Decker
2596331bca
Disable recently added postgres detector because it it too sensitive ( #2303 )
2024-01-13 19:52:57 -08:00
Cody Rose
b03cc30263
Individuate archive tests #2293
2024-01-12 09:39:48 -05:00
ahrav
651beff492
[feat] - Allow for the use of include/exclude path files for filesystem scans ( #2297 )
...
* Allow for the use of include/exclude path files for filesystem scans
* remove oopsie
2024-01-11 15:41:50 -08:00
ahrav
9408425cc6
[chore] - small updates ( #2288 )
...
* small updates
* fix logic
* simplify fxn
* remove errors
* use strings.EqualFold
2024-01-11 14:27:10 -08:00
ahrav
677238c96c
Extend memory cache ( #2275 )
...
* Extend memory cache to allow for configuring custom expiration and purge interval
* use any for value type
* fix test
* fix test
* address comments
* address
* make new construct more clear
* reduce duplication
* fix test
2024-01-11 08:20:37 -08:00
dylanTruffle
3b4518cbab
adding postgres detector ( #2108 )
...
* adding postgres detector
---------
Co-authored-by: Chair <chair@Chairs-MacBook-Pro.local>
Co-authored-by: ahmed <ahmed.zahran@trufflesec.com>
2024-01-10 16:19:45 -05:00
ahrav
fb927e011b
update test ( #2283 )
2024-01-10 09:56:21 -08:00
Richard Gomez
04bf244f38
fix(signable): ignore common false positives ( #2230 )
2024-01-09 17:53:25 -08:00
Richard Gomez
38f36475de
fix(parseur): ignore false positives ( #2229 )
2024-01-09 17:50:30 -08:00
ahrav
b2c0d50ba8
[chore] - update docs for pre-commit ( #2280 )
...
* update docs for pre-commit
* fix spacing
2024-01-09 15:41:27 -08:00
David
24a09bc37d
1833 Fix syslog udp ( #1835 )
...
* # 1183 - Update syslog UDP listener deadline
* #1833 - Update syslog UDP listener deadline v2
* #1833 - Update syslog UDP listener deadline v3
2024-01-08 09:59:48 -08:00
Cody Rose
fd8e5e9df6
Wrap temp deletion err #2277
2024-01-08 12:48:58 -05:00
Shivasurya
e5289ac8e4
Updated trufflehog sourcegraph secret format ( #2254 )
2024-01-06 08:48:41 -08:00
NIKHIL PANWAR
7365dba9c9
Update stripe detector regex ( #2261 )
2024-01-06 08:43:39 -08:00
Miccah
5e78ad402e
[chore] Add test to check all versioned detectors are non-zero ( #2272 )
2024-01-05 12:06:59 -08:00
Richard Gomez
241e153dfb
fix(gitparse): handle fromFileLine edge case ( #2206 )
2024-01-04 14:53:08 -08:00
Dustin Decker
1cc41e2c75
Use directory iterator instead of walkdir ( #2260 )
...
* Use directory iterator instead of walkdir
* pr comments
2023-12-22 22:45:27 -08:00
Bill Rich
78d8dd3abf
Add handlerOpts back ( #2258 )
2023-12-22 12:11:59 -08:00
Bill Rich
ceff786db4
Skip all binaries ( #2256 )
...
* Skip all binaries
* Remove noop
* Drop handlerOpts
2023-12-22 12:01:07 -08:00
Dustin Decker
7d93adc1d0
Add skip archive support ( #2257 )
2023-12-22 11:55:23 -08:00
Dustin Decker
f699f60e89
use walk dir for tmp cleanup ( #2255 )
2023-12-22 07:52:58 -08:00
ahrav
39f0310f1f
[fixup] - Refactor to Pass Reader for Binary Diffs and Archived Data; Optimize /tmp Directory Cleanup ( #2253 )
2023-12-22 07:41:54 -08:00
Cody Rose
9c8674777c
Dedupe some source log keys ( #2250 )
...
The source manager attaches some context keys, but in certain circumstances, they're already present, resulting in duplicate keys. This PR changes the attachment to be conditional. It also adds some new log messages to track source startup progress.
2023-12-21 10:11:52 -08:00
ahrav
07ae9ec870
Fix goroutine leak ( #2251 )
2023-12-20 21:09:05 -08:00
ahrav
28212c9a82
[chore] - lower logging level ( #2249 )
...
* lower logging level
* move pid len check outside loop
2023-12-20 09:50:32 -08:00
ahrav
4a66dddd81
[chore] - add additional binary extensions to skip ( #2235 )
...
* add additional binary extensions to skip
* remove whl
2023-12-20 06:57:23 -08:00
ahrav
f5d0f3f366
use snake_case for naming ( #2238 )
2023-12-20 06:57:00 -08:00
ahrav
5848f5b8d6
[bug] - Bug archive handler memory leak ( #2247 )
2023-12-20 06:16:58 -08:00
Dustin Decker
6b90a96ca0
Add missing import ( #2246 )
2023-12-19 16:40:31 -08:00
Richard Gomez
69d5e0c993
fix(snowflake): avoid extraneous attempts ( #2057 )
...
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-12-19 16:04:08 -08:00
Richard Gomez
97afd570ae
feat(github): update extradata ( #2219 )
2023-12-19 16:01:07 -08:00
Mike Vanbuskirk
adba91da50
Adds basic if/else check if pid slice is empty ( #2244 )
...
* adds basic if/else check for pid slice
* use continue instead of if
2023-12-18 22:24:49 -05:00
ahrav
64c7365364
add secretID to chunk ( #2242 )
2023-12-18 15:27:49 -08:00
Richard Gomez
eeb4dbd304
feat(shortcut): use tri-state verification ( #2211 )
2023-12-17 15:47:14 -08:00
Richard Gomez
ded8e459bd
feat(huggingface): enhance extradata ( #2222 )
2023-12-17 14:29:45 -08:00
Richard Gomez
69a70a3374
fix(myfreshworks): check for valid JSON ( #2212 )
2023-12-17 10:26:38 -08:00
Miccah
88281bc354
[chore] Add skip_binaries field to AzureRepos proto message ( #2232 )
...
* [chore] Add skip_binaries field to AzureRepos proto message
* Make protos
2023-12-15 12:23:46 -08:00
ahrav
5c6ce693c1
[feat] - Make skipping binaries configurable ( #2226 )
...
* Make skipping binaries configurable
* remove ioutil
* fix
* address comments
* address comments
* use multi-reader
* remove print
* use const
* fix test
* fix my stupidness
2023-12-15 11:46:27 -08:00
Miccah
78b5a95342
[chore] Prevent panic when ChunkError has a nil Unit ( #2227 )
2023-12-15 11:11:28 -08:00
Richard Gomez
b3040b1227
fix(github): remove unused 'members' var ( #2202 )
2023-12-14 11:53:24 -08:00
Miccah
f6bbc59bf6
Check for SourceUnit support dynamically in the SourceManager ( #2205 )
...
* Check for SourceUnit support dynamically in the SourceManager
* Only call the function if we can use source units
2023-12-14 11:48:15 -08:00
Richard Gomez
06b137fd93
fix(gitlab): check for valid JSON ( #2218 )
2023-12-14 11:22:06 -08:00
ahrav
d8cb65833c
Avoid reading decompressed data into memory ( #2196 )
2023-12-14 11:00:11 -08:00
Richard Gomez
e72fdb62e4
fix(gitparse): don't trim filename ( #2201 )
2023-12-14 08:29:46 -08:00
Richard Gomez
22ae6a7a8f
fix(giturl): encode '%' in path ( #2214 )
2023-12-14 08:06:09 -08:00
Miccah
4db20e29f8
Update metabase verification to check for a valid JSON response ( #2210 )
...
* Update metabase verification to check for a valid JSON response
* added test tokens + cleanup
---------
Co-authored-by: ahmed <ahmed.zahran@trufflesec.com>
2023-12-13 12:18:56 -08:00
Miccah
84b7461796
[chore] Remove unnecessary string conversion in tefter detector ( #2209 )
2023-12-13 11:39:16 -08:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
6987507534
fix and refactor browserstack detector ( #2208 )
...
* fix and refactor browserstack detector
2023-12-12 16:14:31 -05:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
5e3ea1a8f2
Fix azurestorage detector ( #2207 )
...
* bugfix + cleanup - update azurestorage detector raw string to use key instead of id
2023-12-12 16:07:09 -05:00
Miccah
9f6a47da3f
[chore] Remove omitempty tags on JobProgressMetrics and UnitMetrics ( #2204 )
2023-12-12 10:02:56 -08:00
Mike Vanbuskirk
53f060a08e
Add disk buffer tempfile cleanup ( #2130 )
...
* add tempfile creation
- break PID retrieval into sep. function
* add tmpfile cleanup func
* add file cleanup to main cleanup func
* refactor file logic to only return name string
* add temp buffer naming to gcs
* add temp buffer naming to s3
* add temp buffer naming to filesystem
* add temp buffer naming to git
* consolidate cleanup functions
- have single function handle both files and dirs
- remove interface(not needed with a single func implementation)
- change calls to `New(...)` to reflect config implementation
- simplify automation in main.go
- update disk-buffer-reader dependency
* integrate changes from pr #2133
* merge main
* checkout from main to revert conflict issues
* re-add buffer logic to git
* interface no longer needed
* move string format to global const
---------
Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2023-12-11 18:31:50 -05:00
Cody Rose
405f356071
Use bad json in slackwebhooks ( #2193 )
...
* add rotation guides to SlackWebhook tests
* begin cleaning up tests
* have slack webhook detector use malformed json
* update test secrets
---------
Co-authored-by: Ahrav Dutta <ahrav.dutta@trufflesec.com>
2023-12-11 15:04:55 -08:00
ahrav
61c7d52a43
[bug] - close file after reading ( #2203 )
...
* close file after reading
* inline return
2023-12-11 15:04:30 -08:00
Richard Gomez
d1a2d9e832
chore: propagate log context to handlers ( #2191 )
2023-12-10 10:30:11 -08:00
Richard Gomez
6c5fc2f212
feat(privatekey): run checks concurrently ( #2139 )
2023-12-10 10:11:17 -08:00
ahrav
331336dc0a
[fixup] - skip files in the archive handler ( #2195 )
2023-12-08 20:23:32 -08:00
ahrav
2728e514d2
move logic to main Chunks method ( #2194 )
2023-12-08 14:51:24 -08:00
ahrav
2a7813929b
add metrics for gitlab ( #2190 )
2023-12-08 09:50:09 -08:00
ahrav
4b31b39d6b
[chore] - Refactor common code into a separate function ( #2179 )
...
* Refactor common code into a separate function
* rename vars
* make sure to set the scanOptions fields
* address comments
2023-12-08 08:44:35 -08:00
Cody Rose
ee6923a241
Remove java archives from ignored extensions ( #2188 )
...
A previous commit (5d0196957f
2023-12-07 15:19:56 -05:00
ahrav
b75991850a
[chore] - Compile regex once ( #2176 )
...
* move regex compilation out of the fxn
* missed a spot
* merge main
2023-12-07 07:26:27 -08:00
ahrav
f772fd8b44
update regex ( #2184 )
2023-12-06 17:04:38 -08:00
Dustin Decker
3167dde8a1
Deprecate some detectors ( #2186 )
2023-12-06 16:57:55 -08:00
ahrav
0595a3baac
allow targets for the source manager ( #2182 )
...
* allow targets to the source manager
* use targets
2023-12-06 16:38:35 -08:00
ahrav
c6e9b8ff64
use https for verification endpoints ( #2185 )
2023-12-06 16:06:04 -08:00
ahrav
e6bc7f4451
remove unnecessary Git cmd check ( #2175 )
2023-12-06 13:38:34 -08:00
ahrav
cb81f7d11a
[feat] - Remove go-git dependency ( #2174 )
...
* remove use of go-git for binary files
* fix it
* use limit reader
* fix comment
* fix test
* address comments
* address comments
* address comments
2023-12-06 13:38:01 -08:00
ahrav
990274b596
Skip trying to determine MIME type for directories ( #2178 )
2023-12-06 12:00:18 -08:00
dylanTruffle
96aa50d119
fixing how to rotate URL ( #2183 )
2023-12-06 11:59:21 -08:00
āh̳̕mͭͭͨͩ̐e̘ͬ́͋ͬ̊̓͂d
11394ea318
[thog-1548] add auto redaction for verification errors ( #2106 )
...
* Updating VerificationError to have auto redaction logic
* find/replace error
2023-12-05 08:57:52 -05:00
ahrav
13da76d357
skip files we can't scan ( #2170 )
2023-12-04 13:37:11 -08:00
ahrav
996a11dcc0
[chore] - remove deprecated types ( #2168 )
...
* remove deprecated types
* missed one
2023-12-04 13:23:58 -08:00
Cody Rose
5d0196957f
Ignore images and binaries ( #2162 )
...
This PR expands the list of excluded file extensions to contain images and other binary files. These files can technically contain secrets, but need decoding to properly be handled, and we don't have any such decoding yet. Down the road if we want to add it we can.
2023-12-04 13:25:29 -05:00
ahrav
37d9e5eedf
[chore] - Increase pagination limit ( #2154 )
...
* increae pagination limit
* rename
2023-12-04 10:14:46 -08:00
Dustin Decker
07dc123840
update forager types ( #2159 )
2023-12-03 13:16:16 -08:00
ahrav
c34efc3cf9
make empty slice delcration consistent ( #2144 )
2023-12-01 11:03:44 -08:00
ahrav
279f915799
[chore] - fix error comparisons ( #2142 )
...
* fix error comparisons
* fix imports
2023-12-01 08:32:41 -08:00
ahrav
52ffab1034
[chore] - fix import name clashes ( #2143 )
...
* fix import name clashes
* fix missing var
2023-12-01 06:53:15 -08:00
Dustin Decker
a367f9ce34
Fix azure panic when invalid URL is constructed ( #2137 )
2023-11-30 11:33:04 -08:00
ahrav
8880c2e005
fixup cleantemp ( #2136 )
2023-11-30 09:39:30 -08:00
Miccah
e498c80b3d
Fix nil pointer dereference when checking if a unit IsFinished ( #2135 )
2023-11-29 14:19:31 -08:00
Miccah
7ecd43ab1e
[chore] Minor cleanup of source_manager.go ( #2134 )
2023-11-29 11:08:25 -08:00
Dustin Decker
363ccab316
Simplify temp dir cleaning ( #2133 )
...
* Simplify temp dir cleaning
* rename vars
* add test
* update test
2023-11-28 16:42:17 -08:00
Dustin Decker
ede0c39589
Add new auth method to source ( #2132 )
2023-11-28 10:58:11 -08:00
Zachary Rice
d552222385
add extradata nil check and use make ( #2129 )
...
* add extradata nil check and use make
* remove some lines
2023-11-28 09:45:37 -06:00
Miccah
78219a27b3
Call Finish in SourceManager after the semaphore is released ( #2121 )
2023-11-24 13:22:08 -08:00
Richard Gomez
024aa056b9
chore(github): add a newline between titles and bodies ( #2124 )
2023-11-23 16:14:28 -08:00
Richard Gomez
1f502fd42c
feat(github): scan issue & pr titles ( #1899 )
2023-11-22 19:15:27 -08:00
ahrav
0e6e1dce3f
use camelcase var names ( #2123 )
2023-11-22 09:09:04 -08:00
Oleksandr Redko
7d10e2540e
Remove unused functions ( #2122 )
2023-11-22 06:58:16 -08:00
Dustin Decker
a7d330a2a5
import missing detectors ( #2119 )
2023-11-21 10:30:11 -08:00
Dustin Decker
75e869faff
Fix forks and repos counter, add metric for orgs enumerated ( #2118 )
2023-11-21 08:52:33 -08:00
Richard Gomez
62c628fb52
feat(telegram): add username to extradata ( #2100 )
2023-11-20 14:00:09 -08:00
