Commit graph

39 commits

Author SHA1 Message Date
Miccah
01c9ac7b59
Fix binary file hanging bug in git sources (#2388)
Waiting for the sub-command will block until all of `stdout` has been
read. In some cases, we return early due to failed chunking without
reading all of the data, and thus, get stuck waiting for the command to
finish. Closing the pipe will ensure `Wait` does not block on that I/O.
2024-02-05 15:28:49 -08:00
ahrav
135cc3eb69
[fixup] - correctly use the buffered file writer (#2373)
* correctly use the buffered file writer

* use value from source

* reorder fields

* use only the DetectorKey as a map field

* address comments and use factory function

* fix optional params

* remove commented out code
2024-02-05 10:43:55 -08:00
ahrav
9867ce8eb8
Allow for configuring the buffered file writer (#2319)
* Write large diffs to tmp files

* address comments

* Move bufferedfilewriter to own pkg

* update test

* swallow write err

* use buffer pool

* use size vs len

* use interface

* fix test

* update comments

* fix test

* Allow for configuring the buffered file writer

* remove unused

* add missing method

* remove

* remove unused

* move parser and commit struct closer to where they are used

* linter change

* fix snifftest

* address comments

* add more kvp pairs to error

* fix test

* update

* add back missing metadata fields

* address comments

* remove bufferedfile writer

* fix

* address comments

* use unint8

* update interface

* adjust interface

* fix tests

* make linter happy

* fix finalize

* address comments

* update test

* address comments

* lint

* remove guard

* fix test

* fix

* add TODO

* fix tests
2024-01-30 12:51:58 -08:00
ahrav
7c59ff95d5
[feat] - tmp file diffs (#2306)
* Write large diffs to tmp files

* address comments

* Move bufferedfilewriter to own pkg

* update test

* swallow write err

* use buffer pool

* use size vs len

* use interface

* fix test

* update comments

* fix test

* remove unused

* remove

* remove unused

* move parser and commit struct closer to where they are used

* linter change

* add more kvp pairs to error

* fix test

* update

* address comments

* remove bufferedfile writer

* address comments

* adjust interface

* fix finalize

* address comments

* lint

* remove guard

* fix

* add TODO
2024-01-30 12:30:51 -08:00
ahrav
383f8a1f67
[chore] - reduce test time (#2321)
* reduce test time

* remove commented out code
2024-01-22 09:40:32 -08:00
Richard Gomez
241e153dfb
fix(gitparse): handle fromFileLine edge case (#2206) 2024-01-04 14:53:08 -08:00
Richard Gomez
e72fdb62e4
fix(gitparse): don't trim filename (#2201) 2023-12-14 08:29:46 -08:00
Bill Rich
00a00ef651
Fix binary handling (#1999) 2023-10-26 10:07:02 -07:00
Savely Krasovsky
d062834997
initial support for bare repositories (#1499)
* feat: initial support for bare repositories

* feat: use concatenation instead of formatting and os.Getenv instead of os.Environ

Signed-off-by: Savely Krasovsky <savely@krasovs.ky>

* fix: go-git update with pre-receive hooks fix

Signed-off-by: Savely Krasovsky <savely@krasovs.ky>

* fix: remove info about pre-receive hook from README.md for now

Signed-off-by: Savely Krasovsky <savely@krasovs.ky>

* fix: don't scan staged while using --bare option, fixes to make it work with the latest master

Signed-off-by: Savely Krasovsky <savely@krasovs.ky>

* fix: small refactor according to #1518

Signed-off-by: Savely Krasovsky <savely@krasovs.ky>

---------

Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
2023-08-03 11:23:41 -05:00
Miccah
b54683acb9
gitparse: Use an object for currentDiff (#1573)
* gitparse: Use an object for currentDiff instead of a pointer

* gitparse: Use an object for currentCommit instead of a pointer

* Revert "gitparse: Use an object for currentCommit instead of a pointer"

This reverts commit c5f0708b4a.
2023-07-31 11:39:14 -05:00
Miccah
6bd48583ae
Fix gitparse from panicking on a nil-pointer (#1570) 2023-07-28 11:15:02 -05:00
Zachary Rice
3897454dbb
add merge support (#1561) 2023-07-27 09:24:49 -05:00
Richard Gomez
f48a635c34
feat: update gitparse logic (#1486) 2023-07-25 17:52:34 -05:00
Zachary Rice
452734adc8
remove head from git diff command, rename unstaged to staged (#1439) 2023-06-29 15:33:30 -05:00
Zachary Rice
c28c70b399
fix new git file plus plus plus bug (#1386) 2023-06-08 18:29:11 -05:00
ahrav
1da7720912
Replace context.TODO. (#1349) 2023-05-19 11:09:51 -07:00
Bill Rich
f2924f3061
Make sure context lines are properly handled (#1331)
* Make sure context lines are properly handled

* Fix git test to account for context change
2023-05-05 12:51:27 -07:00
ahrav
714c480931
Add log to track git log size (#1325)
* Add log to track git log size.

* Add calc for large commits and last commit.
2023-05-02 16:36:39 -07:00
Zachary Rice
458c79165a
fix extra log messages (#1253)
* fix extra log messages

* add small test, move flag to isindex
2023-04-13 09:53:21 -05:00
Dustin Decker
8f10938bf7
forager requires direct access to gitparse.FromReader (#1233) 2023-04-02 17:54:43 -07:00
Zachary Rice
fb9ae75661
Support for exclude globs at the git log level (#1202)
* init

* seems to be working

* better comment

* rm conditional

* Add more context to exclude-globs description
2023-03-28 10:46:03 -05:00
ahrav
aa47e5e248
Only scanned staged git changes. (#1143) 2023-03-01 08:58:36 -08:00
Bill Rich
ae2d510ced
Gitparse message fix (#1125)
* Fix messages being reused

* Add comment about change.
2023-02-23 15:20:54 -08:00
Bill Rich
f1582aafa9
Drop tabs for filenames with spaces (#1115) 2023-02-16 17:15:32 -08:00
Bill Rich
9158dcaa80
Correctly parse most filenames with ' and ' (#1113) 2023-02-16 14:11:35 -08:00
Miccah
161e499142
[chore] Remove logrus from trufflehog (#1095)
* [chore] Remove logrus from trufflehog

* Minor fixes

* Fix logFatal call

* Fix logrus call
2023-02-14 17:00:07 -06:00
Bill Rich
b37080e6a5
Add max commit size (#1079)
* Add max commit size

* Use common.IsDone

* Use breaks instead of return
2023-02-07 15:25:00 -08:00
Bill Rich
af6e3f8fdf
Pull gitparse config options out of pkg consts (#1072)
* Pull gitparse config options out of pkg consts.

* Adjust naming
2023-02-04 13:19:23 -08:00
Bill Rich
00ebb2ed64
Full git log when targeting base merge commit (#1044)
* Full git log when targeting merge commits

* Full log is needed whenever base is specified.
2023-01-26 09:17:54 -08:00
Bill Rich
ac1dd23d37
Limit diff size to prevent out of control memory use. (#1035)
* Limit diff size to prevent out of control memory use.

* Group consts
2023-01-23 10:14:10 -08:00
Miccah
4aab7b7276
Buffer commit log processing (#845)
Some very large commits take a lot of time to process, which we can make
progress on while we are scanning the contents of other commits.
2022-10-12 14:55:08 -05:00
ahrav
92f40c2031
[THOG-709] - Recover from detector panics (#810) 2022-09-22 07:01:10 -07:00
Bill Rich
912d8e461d
Add context so to avoid splitting creds. (#791)
* Add context so to avoid splitting creds.

* Add context newlines to expected results
2022-09-09 15:00:33 -07:00
Bill Rich
3fe916fe1e
add tests (#785) 2022-09-08 21:46:12 -07:00
Bill Rich
41936169c7
Use gitparse for unstaged changes. (#775) 2022-09-03 18:01:36 -07:00
Bill Rich
65a7855713
Split parsing from reader (#774) 2022-09-02 11:02:38 -07:00
Dustin Decker
fa9479100e
Add common sentry recover library and add into goroutines (#738)
* Add common sentry recover library and add into goroutines

* fix nits
2022-08-29 11:45:37 -07:00
ahrav
20cdcbc970
[bug] - Fix the starting index value for plus line check. (#734)
* Fix the starting index value for plus line check.

* Set the correct source type for notifications.

* Reset old value.

* Fix the starting index value for plus line check.

* Fix len check.

* Reset old value.

* Add tests.

* Update tests.

* Update tests.
2022-08-25 10:45:35 -07:00
Bill Rich
a0d44a39f1
Use trufflesec git parser (#729)
* Use trufflesec git parser.

* wip

* Fix line numbers and linter feedback
2022-08-23 13:29:20 -07:00