Miccah
97f8a4834b
Add metrics for command invocation ( #3185 )
2024-08-13 08:50:36 -07:00
0x1
8cf1ec2824
remove two letter keyword ( #3210 )
2024-08-13 09:09:36 -05:00
renovate[bot]
e9f8123776
fix(deps): update module cloud.google.com/go/secretmanager to v1.13.6 ( #3208 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-10 09:33:28 -07:00
Cody Rose
9718ec6a51
Capture decoding time metric ( #3209 )
...
We're trying to track down some slowness.
2024-08-09 15:19:16 -04:00
renovate[bot]
f2c7bb93be
fix(deps): update module github.com/google/go-containerregistry to v0.20.2 ( #3184 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-09 08:02:50 -07:00
ahrav
a966a47b63
[bug] - Correctly Handle Large Files in BufferedReadSeeker ( #3203 )
...
* handle large files correctly
* return if http get fails
2024-08-08 12:07:45 -07:00
Cody Rose
239f35921d
Log when a detector ignores the timeout ( #3201 )
...
If a detector ignores the configured timeout it is probably because of I/O blocking, which degrades the efficiency of the detector worker pool when it happens a lot. In the worst case, a detector that fully hangs will zombify its worker, causing really bad performance problems. When this happens, we don't really have a good way to notice other than seeing scan throughput drop suspiciously. This PR adds explicit logging when detection takes longer than it should so we have a better chance of catching this.
(This problem theoretically can spring up anywhere, in any worker, but the detector fleet is vast, uses network I/O, and is implemented by a much larger group of people, so this sort of problem is much more likely to slip into detector implementations than anywhere else in the codebase. We could generalize this mechanism, but I don't want to make that investment before seeing if this smaller change captures the information we need.)
2024-08-08 14:58:23 -04:00
renovate[bot]
713521c433
fix(deps): update module go.mongodb.org/mongo-driver to v1.16.1 ( #3197 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-08 11:00:59 -07:00
Miccah
39f5f547e1
[analyze] Fix double-print in postgres analyzer ( #3199 )
...
* [analyze] Fix double-print in postgres analyzer
* Continue on error in github analyzer
2024-08-07 16:10:43 -07:00
renovate[bot]
d0726eb949
fix(deps): update module golang.org/x/net to v0.28.0 ( #3187 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-07 12:12:12 -07:00
Miccah
bf2afc9751
[analyze] Deduplicate finegrained GitHub permissions ( #3196 )
2024-08-07 11:22:29 -07:00
Dustin Decker
fc4829a387
Fixes for a few finegrained token issues ( #3194 )
...
* Fixes a few finegrained issues
* remove some code
2024-08-07 07:48:00 -07:00
Miccah
8b37ae11ca
[analyze] Add basic section to README ( #3190 )
2024-08-07 07:26:01 -07:00
Miccah
7730fc826b
[analyze] Bandaid solution for occasional slow startups ( #3191 )
...
* [analyze] Bandaid solution for occasional slow startups
* Speed up shutdown
* Add link to upstream issue
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-08-06 22:24:58 -07:00
Hon
ab8c843fec
Analyzer capitalization ( #3188 )
...
* capitalization
* Lowercase analyze labels for the subcommand
* Canonicalize input and lowercase when matching command
* add warning
---------
Co-authored-by: Miccah Castorina <m.castorina93@gmail.com>
2024-08-06 17:00:40 -07:00
Miccah
a8777fcad9
[analyze] Add analyze option to main TUI and unhide subcommand ( #3186 )
...
This is currently a one-way operation. Once you select "analyze" you
cannot get back to the main menu.
2024-08-06 15:30:50 -07:00
renovate[bot]
8c6f852a9c
fix(deps): update module golang.org/x/text to v0.17.0 ( #3183 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-06 11:10:22 -07:00
renovate[bot]
8ea60861ba
fix(deps): update module golang.org/x/crypto to v0.26.0 ( #3182 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-06 11:10:00 -07:00
Dustin Decker
29de521ed0
Improve finegrained token support ( #3179 )
2024-08-05 18:55:05 -07:00
Miccah
a373f6bd78
[chore] Use custom HTTP client in sendgrid analyzer ( #3178 )
2024-08-05 17:47:37 -07:00
Miccah
1df83f79ef
[analyze] Separate SID from token in twilio analyzer ( #3177 )
...
* [analyze] Separate SID from token in twilio analyzer
* Fix test
* Set sid in detector
2024-08-05 17:46:57 -07:00
Miccah
59fccbcf3f
Analyze TUI ( #3172 )
...
* Setup TUI entrypoint
* Setup key type selector and form pages
* Add basic confirmation component
* Add basic list selector for analyzer type
* Add form page
* Remove quit confirmation
* Add styles
* Add input text redaction
* Add log file input to form
* Fix some bugs and race conditions
* Remove unused code
* Fix filtering bug
2024-08-05 15:00:46 -07:00
renovate[bot]
b8cbb4dc72
fix(deps): update module github.com/envoyproxy/protoc-gen-validate to v1.1.0 ( #3176 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-05 13:29:28 -07:00
Cody Rose
f26b502c2e
Auth GitHub in Init ( #3131 )
...
The GitHub source currently applies its authentication configuration as the first step of enumeration. This is incompatible with both targeted scans and scan job reports, and also means that authentication logic has to be duplicated into the validation flow. This PR moves it into Init so that it's available to targeted scans and, eventually, unit-specific scans. This also allows us to remove the copy of the old logic that was in Validate.
As part of the work I've also cleaned up the integration test suite. (Several of them were apparently disabled back when they ran on every push, but now that we're not doing that, we can re-enable them.)
2024-08-05 15:13:29 -04:00
Dustin Decker
c2e5506b95
Change log verbosity for detection errors ( #3171 )
2024-08-04 20:47:41 -07:00
renovate[bot]
38db52ec1f
fix(deps): update github.com/tailscale/depaware digest to 585336c ( #3166 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-04 10:17:22 -07:00
renovate[bot]
41a4b0839c
fix(deps): update module golang.org/x/sync to v0.8.0 ( #3169 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-04 10:16:49 -07:00
renovate[bot]
4b75ab8c63
fix(deps): update module golang.org/x/oauth2 to v0.22.0 ( #3168 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-04 09:32:13 -07:00
Richard Gomez
f335d486ef
Update Zulip detector ( #2897 )
...
* fix(zulip): prevent false positives
* update extra data
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2024-08-04 09:30:15 -07:00
Dustin Decker
88359addc5
update pattern ( #3167 )
2024-08-04 09:12:09 -07:00
Miccah
37a130fb58
[analyze] Use permission enum values in openai analyzer ( #3165 )
2024-08-02 16:20:45 -07:00
ahrav
0a3451a1ba
[bug] - Create a new context with timeout per request ( #3163 )
...
* Create a new context with timeout per request
* match timeout
* use context timeout
* reduce timeout
2024-08-02 14:46:37 -07:00
Miccah
f939572a43
[analyze] Fix off-by-one error in generated data structures ( #3162 )
...
* [analyze] Fix off-by-one error in generated data structures
* Generate data structures
* Fix finegrained checks
2024-08-02 14:22:22 -07:00
renovate[bot]
6ddae129b5
fix(deps): update module github.com/schollz/progressbar/v3 to v3.14.6 ( #3158 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-02 14:19:23 -07:00
joeleonjr
4498c4be7c
Update README.md ( #3160 )
2024-08-02 14:18:36 -07:00
ahrav
c549b5bd15
[bug] - add context timeout to ssh verification ( #3161 )
...
* add context timeout to ssh verification
* fix test
2024-08-02 12:39:50 -07:00
ahrav
29613220b0
[chore] - log detector type on error ( #3159 )
...
* log detector type on error
* update error message
* update log
* update message
2024-08-02 10:54:59 -07:00
ahrav
ddb7211ded
[chore] - set custom transport for the Docker client ( #3156 )
...
* set custom transport for docker
* fix lint
2024-08-02 08:51:59 -07:00
Abdul Basit
04a13385a8
Add Analyzers interface for HuggingFace ( #3140 )
...
* implemented analyzer interface with data models for HuggingFace
* correct test for huggingface due to new addition of key in detection result.
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-08-02 08:20:11 -07:00
joeleonjr
f927076483
quick patch for cfor enumeration ( #3155 )
...
Co-authored-by: Joe Leon <joe.leon@trufflesec.com>
2024-08-02 11:12:43 -04:00
renovate[bot]
fe9ac9d0bf
fix(deps): update module google.golang.org/api to v0.190.0 ( #3146 )
...
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-08-02 08:09:40 -07:00
Abdul Basit
c1645e8c27
Add Analyzers interface for Square ( #3141 )
...
* implement analyzer interface for square
* linked detector with analyzer for square
fix test for square.
* code refactoring
---------
Co-authored-by: Abdul Basit <abasit@folio3.com>
2024-08-02 08:00:25 -07:00
ahrav
170f6ab624
enable mutex and block profiler ( #3154 )
2024-08-02 07:56:09 -07:00
Miccah
eccbca730d
[fix] Always configure the engine with the default detectors ( #3152 )
...
If detectors are not wanted by a user, they can be filtered out via
the `--include-detectors` or `--exclude-detectors` flag.
2024-08-02 07:48:39 -07:00
Dustin Decker
05e4635824
Add progress bar to CFOR ( #3151 )
...
* Add progress bar to CFOR
* unused vars
* explicitly ignore progress errors
* removed print statements
* use stderr
---------
Co-authored-by: joeleonjr <20135619+joeleonjr@users.noreply.github.com>
Co-authored-by: Joe Leon <joe.leon@trufflesec.com>
2024-08-02 07:43:59 -07:00
ahrav
fba1a8b410
[perf] - Leverage pgzip for Parallel decompression ( #3149 )
2024-08-02 04:11:10 -07:00
joeleonjr
7d606e2480
CFOR Commit Scanner ( #3145 )
...
* alpha feature for scanning hidden commits on github
* improvements re: git operations
* lint updates
* updating with exec block due to no gh token
* reworked logic into new source
* fixed collisions threshold flag input
* fixed IOutil issues
* removed additions from GH config
---------
Co-authored-by: Joe Leon <joe.leon@trufflesec.com>
2024-08-01 23:04:20 -04:00
Miccah
38e844f968
[chore] Only set default detectors if none are provided ( #3147 )
2024-08-01 17:15:06 -07:00
Dustin Decker
605d037e45
add twilio analyze relationships ( #3148 )
...
* add twilio analyze relationships
* unused struct
2024-08-01 17:04:44 -07:00
ahrav
b193febab5
[chore] - move automaxprocs to init ( #3143 )
...
* move automaxprocs to init
* revert
2024-08-01 11:31:03 -07:00
