Commit graph

100 commits

Author SHA1 Message Date
Bill Rich
f39303495a
Add commitsScanned metrics (#1533)
* Add commitsScanned metrics

* Just keep commit count
2023-07-25 11:31:01 -07:00
ahrav
9e0a2e9ddd
[chore] - Remove password info from log (#1528)
* Remove password info from log.

* update.

* one more.
2023-07-22 20:25:45 -07:00
Richard Gomez
1594fddf05
feat(git): include line in github & gitlab links (#1466) 2023-07-11 20:02:27 -07:00
Zubair Khan
4334af4d34
scan GitHub PR and issue comments (#1435)
* issue comment scanning

* save progress

* test

* test for pr comment and issue comment

* add pagination support

* linter stuff

* make linter happy

* remove debug log

* readd logging

* github issue resolved

* var const block and handle rate limit

* remove magic number

* make gitURLParse a public function to use more generally

* fix test bug

* make comment scanning OPT-IN
2023-07-11 15:13:33 -04:00
Zachary Rice
452734adc8
remove head from git diff command, rename unstaged to staged (#1439) 2023-06-29 15:33:30 -05:00
Zachary Rice
4a77688097
use stringer again for now (#1430) 2023-06-26 14:33:54 -05:00
trufflesteeeve
11bff81def
Use url redaction in git (#1399)
Co-authored-by: Zachary Rice <zachary.rice@trufflesec.com>
2023-06-26 13:56:08 -05:00
Miccah
f3152b6885
Implement SourceUnitUnmarshaller for all sources (#1416)
* Implement CommonSourceUnitUnmarshaller

* Add SourceUnitUnmarshaller to all sources using

All sources, with the exception of git, will use the CommonSourceUnit as
they only contain a single type of unit to scan.

* Fix method comments to adhere to Go's style guide
2023-06-23 11:15:51 -05:00
dillonstreator
648ef3b52c
fix spelling errors (#1413) 2023-06-21 07:15:28 -07:00
Dustin Decker
572cb0e5dc Loosen up version check for git 2023-06-01 12:17:48 -07:00
Dustin Decker
183037ab34
Check that git meets version requirements (#1373) 2023-06-01 09:41:06 -07:00
Dustin Decker
c8944825de
Surface missing git as an error during initialization (#1362) 2023-05-26 15:23:08 -07:00
ahrav
1da7720912
Replace context.TODO. (#1349) 2023-05-19 11:09:51 -07:00
Brendan Shaklovitz
fad34d4dc6
git worktree scanning fix for #827 (#1315)
* Fix worktree scan by setting EnableDotGitCommonDir

* Change `PlainOpenOptions` to set `EnableDotGitCommonDir` to true.
  In every current usage of this function, it is on an already-cloned
  repository, so it should always be valid to have this set. By doing
  so, it should fix some issues with worktrees.

* Remove unused go.mod replace directives

* Remove replace directives for libraries that are not in use.
2023-05-09 08:00:47 -07:00
Bill Rich
f2924f3061
Make sure context lines are properly handled (#1331)
* Make sure context lines are properly handled

* Fix git test to account for context change
2023-05-05 12:51:27 -07:00
Brendan Shaklovitz
be4147a24e
Output git timestamps as UTC times (#1323) 2023-05-03 11:47:00 -05:00
ahrav
67972683ea
[chore] - format log msg (#1299)
* format log msg.

* snake.

* lowercase repo.
2023-04-27 17:14:00 -07:00
ahrav
a2266b4e28
add additional logging (#1298)
* add additional logging.

* update test.

* remove continue.

* address comments.
2023-04-27 16:48:04 -07:00
Dustin Decker
97ce27153a
[]bytes were being logged as b64ed string (#1255) 2023-04-14 06:43:26 -07:00
Zachary Rice
fb9ae75661
Support for exclude globs at the git log level (#1202)
* init

* seems to be working

* better comment

* rm conditional

* Add more context to exclude-globs description
2023-03-28 10:46:03 -05:00
Miccah
ef9488c77d
[chore] Log git output on error (#1180) 2023-03-15 15:32:29 -05:00
Tim Walter
a7abd6231d
Fix git commit date string formatting (#1181) 2023-03-14 22:39:12 -05:00
ahrav
aa47e5e248
Only scanned staged git changes. (#1143) 2023-03-01 08:58:36 -08:00
Miccah
6209a80ce1
[chore] Address more linter errors (#1134)
* Address lint errors in detectors

* Update deprecated ioutil call
2023-02-28 10:00:41 -06:00
Miccah
4efe5313f4
[chore] Address lint errors (#1133)
* Update strings.Title to cases.Title

* Migrate go-genproto to google-cloud-go

See: https://github.com/googleapis/google-cloud-go/blob/main/migration.md

* Check error in test

* Check error from sem.Acquire

* Remove unused code
2023-02-27 21:03:47 -06:00
Miccah
0ce72ccda3
[chore] Remove logrus from github source (#1086)
* [chore] Remove logrus from github source

* Fix handleRateLimit test

* Fix tests
2023-02-09 18:02:04 -06:00
Miccah
1f0fd91205
Skip repo and continue scanning when encountering an error (#1080) 2023-02-08 11:33:01 -06:00
Bill Rich
af6e3f8fdf
Pull gitparse config options out of pkg consts (#1072)
* Pull gitparse config options out of pkg consts.

* Adjust naming
2023-02-04 13:19:23 -08:00
Bill Rich
00ebb2ed64
Full git log when targeting base merge commit (#1044)
* Full git log when targeting merge commits

* Full log is needed whenever base is specified.
2023-01-26 09:17:54 -08:00
Yassine Ilmi
d720c0c0f3
Switch to retryableHttpClient for GitHub AuthN API Client + More Logs (#995)
* Adding missing flags to Readme

* Use retryableHttpClient by default for GitHub

* Adding repoUrl for scanning time log

* Use WithField instead of WithFields

* Updating README with lasted --help output
2023-01-09 09:21:56 -08:00
ahrav
009756dce6
add proto that was missing. (#986) 2022-12-23 13:27:07 -08:00
Miccah
7ac7fdae44
Add more logging for git sources (#974) 2022-12-13 17:51:57 -06:00
Bill Rich
33d32d2de4
Don't scan the --since-commit target (#960) 2022-12-06 13:24:27 -08:00
Bill Rich
f1ec9e74eb
Close files to clean up tmp files (#940) 2022-11-22 13:13:34 -08:00
Bill Rich
79cae3b82b
Add newlines when file is split (#937) 2022-11-22 09:01:39 -08:00
Miccah
86f9e1288f
Initialize scan options if given a nil pointer (#924) 2022-11-15 17:01:59 -06:00
ahrav
dd141fb55f
[oc-147] - Add context to all git methods (#901)
* Add context to all git methods.

* remove logrus.

* Add ctx.

* Address comments.

* Add error to clone failing.

* Return error.
2022-11-03 16:36:52 -07:00
Bill Rich
965279421c
Support common ssh repo format (#878)
* Try ssh repo format

* Add tests
2022-10-28 11:56:03 -07:00
Bill Rich
ab71b93f7d
Add context to handler (#877)
* Add context to handler

* Return rather than break out of select
2022-10-28 08:57:55 -07:00
Bill Rich
d7d614cc5f
Copy buffer bytes (#864) 2022-10-25 09:09:47 -07:00
Bill Rich
958266ea84
Run chunker in pipeline (#859)
* Run chunker in pipeline

* Move ChunkSize and PeekSize to source package.

* Use new Chunk and Peek size location
2022-10-24 13:57:27 -07:00
Bill Rich
3d5f697f9a
Use line aware chunking for git. (#858) 2022-10-24 13:00:03 -07:00
Dustin Decker
64ace363af Change commit to trace level logging 2022-10-24 08:59:52 -07:00
Miccah
2bc4985061
Add SSH config option for the git source (#830)
* Add SSH config option for the git source

The auth message is empty since we use the git binary underneath to
handle the SSH authentication.

* Import digitaloceanv2
2022-09-28 20:40:01 +02:00
Dustin Decker
752c848640
Show clone path for git repos (#823) 2022-09-26 14:36:55 -07:00
Bill Rich
e3107ad6bb
Move head and base normalization to source (#818) 2022-09-23 08:58:45 -07:00
Bill Rich
509cf8b6fa
Use headref and check empty commits for base (#815) 2022-09-21 16:04:01 -07:00
Bill Rich
912d8e461d
Add context so to avoid splitting creds. (#791)
* Add context so to avoid splitting creds.

* Add context newlines to expected results
2022-09-09 15:00:33 -07:00
ahrav
7ba583ca40
[THOG-681] - Handle errors sources (#783)
* Handle errors w/ github source.

* Fix loop var captured by func literal.

* Fix loop var captured by func literal.

* Set completed progress if the scan completes with no errors.

* Set progress to 100% if the scope and iteration are both 0.

* Fix commentary.

* Fix test.

* Return after the defer to os.RemoveAll.

* Fix unauth scan.

* Inline range loop.

* update tests for partial scan completion with errors. Ensure correct progress is set.

* Update progress for all sources.

* Update github test.

* Address comments.
2022-09-07 19:40:37 -07:00
Bill Rich
41936169c7
Use gitparse for unstaged changes. (#775) 2022-09-03 18:01:36 -07:00