Commit graph

1793 commits

Author SHA1 Message Date
Miccah
4e774d1f01
Define SourceUnit chunking interface (#1484)
* Define SourceUnit chunking interface

* Refactor to use a ChunkReporter interface

* Rename shadowed err to scanErr
2023-07-13 14:11:43 -05:00
Miccah
4b7f94dea1
Rewrite SourceUnitEnumerator to use UnitReporter instead of a channel (#1485) 2023-07-13 13:48:33 -05:00
ahrav
a9213a1103
[chore] - Update loop to switch. (#1487)
* Update loop to switch.

* remove unused fxn.
2023-07-12 15:47:43 -07:00
Richard Gomez
1594fddf05
feat(git): include line in github & gitlab links (#1466) 2023-07-11 20:02:27 -07:00
Zachary Rice
b48ac24c46
Dedupe results (#1479)
* init 4 dedupin

* use raw rather than rawv2

* rm comment

* comments

* nits

* clean up and use rawv2 too

* add decoder order test
2023-07-11 15:48:00 -05:00
Cody Rose
a123d5c5e1
do not report 403s as indeterminate in AWS detector (#1481) 2023-07-11 16:35:23 -04:00
Cody Rose
b803a0f701
Report indeterminacy in AWS verifier (#1480) 2023-07-11 15:50:31 -04:00
Zubair Khan
4334af4d34
scan GitHub PR and issue comments (#1435)
* issue comment scanning

* save progress

* test

* test for pr comment and issue comment

* add pagination support

* linter stuff

* make linter happy

* remove debug log

* readd logging

* github issue resolved

* var const block and handle rate limit

* remove magic number

* make gitURLParse a public function to use more generally

* fix test bug

* make comment scanning OPT-IN
2023-07-11 15:13:33 -04:00
Miccah
5c0ffda618
Define SourceUnit enumeration interface (#1428)
* Add CancellableWrite helper function

* Create SourceUnitEnumerator interface and EnumerationResult struct

* Implement SourceUnitEnumerator for the filesystem Source

* Omit explicit zero values
2023-07-10 15:05:40 -05:00
Zachary Rice
d4972313ff
remove old detector (#1474) 2023-07-10 13:02:19 -05:00
Cody Rose
87058dd7fa
Add new verification error message field (#1463) 2023-07-10 11:15:40 -04:00
Zachary Rice
0bdd513d88
additional similarity check for base64 and plain (#1462)
* additional similarity check for base64 and plain

* use bytes equal

* move logic into util function
2023-07-10 10:12:59 -05:00
Zubair Khan
b38857edb4
fix missing api key, tighten up regex pattern, use response body check (#1438) 2023-07-06 16:35:52 -04:00
Richard Gomez
23757dbe0a
remove image4 detector (#1461) 2023-07-06 12:56:09 -07:00
Zachary Rice
a99d89d711
fix typo (#1452) 2023-07-05 14:14:18 -05:00
Zachary Rice
8a508e6bcd
Add missing keywords for sqlserver (#1449) 2023-07-05 11:12:19 -05:00
Zachary Rice
452734adc8
remove head from git diff command, rename unstaged to staged (#1439) 2023-06-29 15:33:30 -05:00
Zachary Rice
18a70b64bb
Introduce trufflehog:ignore tag feature (#1433)
* init ignore

* cleanup and add test

* update readme
2023-06-29 08:45:56 -05:00
roxanne-tampus
00920984e3
added opsgenie detector (#650)
* added opsgenie detector

* update interface and import

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-06-27 16:43:25 -07:00
Zubair Khan
d6375ba921
verify response body with expected keywords (#1419)
* verify response body with expected keywords

* remove debug log

* add extra test case

* migrate from ioutil to io

* close body and only check for one keyword

* cleanup
2023-06-27 11:46:15 -04:00
Miccah
8ea49de490
Don't return on okta credential failed verification (#1432) 2023-06-27 09:21:39 -05:00
Zachary Rice
4a77688097
use stringer again for now (#1430) 2023-06-26 14:33:54 -05:00
trufflesteeeve
11bff81def
Use url redaction in git (#1399)
Co-authored-by: Zachary Rice <zachary.rice@trufflesec.com>
2023-06-26 13:56:08 -05:00
Zubair Khan
f52946b996
Add Couchbase Detector (#1385)
* init

* add detector type

* rotate leaked credentials

* tighten up username pattern

* isolated prefixregex as overrriding new line stuff

* passwordPat working now

* add username test

* fix edge case

* cleanup

* make linter happy

* make linter happy rd 2

* skip error logging

* fix test

* add password regex helper func

* make test more robust

* cleanup PR

* remove comments

* clarify prepend rationale
2023-06-26 14:37:10 -04:00
Miccah
945c27cb82
Fix docker source to return any chunk errors (#1429) 2023-06-26 12:12:46 -05:00
Brendan Shaklovitz
da5301ea1e
Exit with non-zero exit code on chunk source error (#1286)
* Exit with non-zero exit code on chunk source error

* Exit with a non-zero exit code whenever we hit an error getting
  chunks. Previously the error would be logged but trufflehog would exit
  with a 0 (success) status code.

* fix gcs test

---------

Co-authored-by: Dustin Decker <dustin@trufflesec.com>
Co-authored-by: ahrav <ahravdutta02@gmail.com>
2023-06-26 11:39:57 -05:00
Chris Atkin
6e6895b48e
Update Slack webhook error text for verification (#1427)
This updates the matched error text to determine the verified status of a Slack webhook, as this has been updated on Slack's API.
2023-06-26 08:44:17 -07:00
Zubair Khan
cd67f6bf16
prevent www from being a key to prevent fp (#1418) 2023-06-25 11:55:11 -04:00
Dustin Decker
eeefde1ec9
Ensure results are collected correctly when verification is off, and dedupe twilio (#1420) 2023-06-23 14:14:08 -07:00
Miccah
f3152b6885
Implement SourceUnitUnmarshaller for all sources (#1416)
* Implement CommonSourceUnitUnmarshaller

* Add SourceUnitUnmarshaller to all sources using

All sources, with the exception of git, will use the CommonSourceUnit as
they only contain a single type of unit to scan.

* Fix method comments to adhere to Go's style guide
2023-06-23 11:15:51 -05:00
Dustin Decker
e856a6890d
🎉 Add Docker image scanning 🎉 (#1412)
* Add Docker source

* Add metrics

* Add test

* Add debugging, address PR comments, fix path output

* review suggestions
2023-06-22 08:02:25 -07:00
dillonstreator
648ef3b52c
fix spelling errors (#1413) 2023-06-21 07:15:28 -07:00
Zubair Khan
0c3410c5cd
add new key pat for mailgun detector (#1375)
* add new detector key pat for mailgun

* resolve mailgun issue

* remove unused tokenPat and commented strings import

* fix closing bracket issue
2023-06-20 19:14:56 -04:00
Zachary Rice
4938d67e66
Custom detector name (#1400)
* hacky way to add detector name to output

* set name in custom detectors
2023-06-20 13:55:31 -05:00
Miccah
e12f0f84a1
Setup SourceUnit interface (#1393)
* Test: Asymmetrical unmarshal API

* Test: Symmetric marshal API

* Revert "Test: Symmetric marshal API"

This reverts commit f51c64a797.

* Cleanup test example and add SourceUnitUnmarshaller interface

* Add CommonSourceUnit implementation

* Update comments

* Remove UnmarshalJSON
2023-06-16 10:38:28 -05:00
Bill Rich
401688d0a8
Add Validator interface and example (#1397)
* Add Validator interface and example

* Close sockets and improve error messages

* Remove duplicate error

* Use var declaration so err slice can be nil
2023-06-15 08:24:32 -07:00
Bill Rich
6d9ae7acbb
Make trace error message so newlines aren't escaped (#1396) 2023-06-14 17:24:31 -07:00
Miccah
fb76eaf17b
Use heuristic to choose the most likely UTF-16 decoded string (#1381)
* Use heuristic to choose the most likely UTF-16 decoded string

* Assume ASCII and include valid BE and LE bytes

* Remove unused code

* Assume ASCII and return nil when not utf16

---------

Co-authored-by: bill-rich <bill.rich@gmail.com>
2023-06-13 17:00:40 -07:00
Zachary Rice
74ffbd2878
add a custom detector check for logging duplicate detector (#1394)
* add a custom detector check for logging duplicate detector

* use pb type
2023-06-13 14:49:21 -05:00
Bill Rich
c2e3e7d53a
Split files instead of using ReadAll (#1387)
* Split files instead of using ReadAll

* Remove dup chunk

* Actually break out of loop
2023-06-12 14:09:05 -07:00
Dustin Decker
ca1947291b
Update sqlserver redaction, deduplication, and URI redaction (#1369)
* Update sqlserver redaction, deduplication, and URI redaction

* don't use pointer
2023-06-09 11:06:54 -07:00
Zachary Rice
c28c70b399
fix new git file plus plus plus bug (#1386) 2023-06-08 18:29:11 -05:00
Zubair Khan
dfb1a0cd38
Add DocuSign detector (#1382)
* init

* look for client id and client secret, encode them for basis auth

* add tests

* test without checking the contents of response

* confirm access_token exists

* cleanup test

* explain in code that an undocumented grant_type is used

* remove use of deprecated ioutil, remove dead code, return errors instead of just logging

* directly pull access token

* update error text, remove redundant body close()

* import new detector into defaults
2023-06-08 13:34:50 -04:00
ahrav
ce4a1fd7e6
[chore] - fix test (#1383)
* fix test.

* fix import order.

* fix twilio test.
2023-06-06 18:58:00 -07:00
Dustin Decker
572cb0e5dc Loosen up version check for git 2023-06-01 12:17:48 -07:00
Dustin Decker
183037ab34
Check that git meets version requirements (#1373) 2023-06-01 09:41:06 -07:00
ahrav
8b7c50825e
update detector regex. (#1368) 2023-06-01 08:16:18 -07:00
Dustin Decker
5358ed776b
fix mockaroo fps (#1370)
* fix mockaroo fps

* fix test
2023-05-30 20:58:41 -07:00
Dustin Decker
c8944825de
Surface missing git as an error during initialization (#1362) 2023-05-26 15:23:08 -07:00
Tim Strazzere
cbfbf5335e
Add Data member to ResultsMetadata struct. (#1358)
When a Result is emitted, it should include
the `chunk.Data []byte` so that we can utilize
the blob of data which caused the result.

This makes it so something catching the results
does not have to maintain a collection of chunks
to correlate the two together.
2023-05-24 09:21:41 -07:00