Dustin Decker
5f6143f09a
Add Circle CI source ( #997 )
...
* Add Circle CI source
* remove SHA1 line
* remove trim
2023-01-05 21:44:37 -08:00
ahrav
009756dce6
add proto that was missing. ( #986 )
2022-12-23 13:27:07 -08:00
ahrav
936a139596
Allow using a glob for include list. ( #977 )
...
* Allow using a glob for include list.
* Update command flag.
* Make comment more clear.
* update comment.
* Allow scanning repo and org at the same time.
2022-12-16 13:28:16 -08:00
Bill Rich
36ca2601e0
Add s3 object count to trace logs ( #975 )
...
* Add s3 object count to trace logs
* fix debug level
2022-12-13 16:46:09 -08:00
Miccah
7ac7fdae44
Add more logging for git sources ( #974 )
2022-12-13 17:51:57 -06:00
ahrav
26befdd1ec
[bug] - Handle error when scanning s3 bucket. ( #969 )
...
* Handle error when scanning s# bucket.
* move wait outside loop.
* Add logging.
* revert changes.
* remove.
* revert.
2022-12-12 10:10:06 -08:00
Dustin Decker
7de9bdd12d
Support globbing with ignore repos ( #967 )
2022-12-09 12:10:42 -08:00
ahrav
a72b9feb35
Only scan org with --org flag. ( #931 )
2022-12-06 16:18:48 -08:00
Bill Rich
33d32d2de4
Don't scan the --since-commit target ( #960 )
2022-12-06 13:24:27 -08:00
Bill Rich
1a1c2e275e
Change chunker test source ( #959 )
...
* Change chunker test source
* Emit chunk if the size isn't 0
2022-12-06 12:45:08 -08:00
Bill Rich
9f99ee470d
Integration test fixes ( #956 )
...
* Adjust repo count for new app
* Fix chunk test count
2022-12-06 08:42:24 -08:00
Bill Rich
f1ec9e74eb
Close files to clean up tmp files ( #940 )
2022-11-22 13:13:34 -08:00
Bill Rich
79cae3b82b
Add newlines when file is split ( #937 )
2022-11-22 09:01:39 -08:00
Dustin Decker
28dd25beeb
S3 scanner improvements ( #938 )
2022-11-21 19:15:26 -08:00
Miccah
86f9e1288f
Initialize scan options if given a nil pointer ( #924 )
2022-11-15 17:01:59 -06:00
Jessica
3d501975e4
Add filter as scan option to gitlab module's git scan ( #919 )
2022-11-15 13:02:37 -08:00
ahrav
dd141fb55f
[oc-147] - Add context to all git methods ( #901 )
...
* Add context to all git methods.
* remove logrus.
* Add ctx.
* Address comments.
* Add error to clone failing.
* Return error.
2022-11-03 16:36:52 -07:00
ahrav
fe1e475a04
Prevent concurrent read and writes to visibility map. ( #892 )
2022-11-01 16:20:59 -07:00
Bill Rich
965279421c
Support common ssh repo format ( #878 )
...
* Try ssh repo format
* Add tests
2022-10-28 11:56:03 -07:00
Bill Rich
ab71b93f7d
Add context to handler ( #877 )
...
* Add context to handler
* Return rather than break out of select
2022-10-28 08:57:55 -07:00
Bill Rich
d7d614cc5f
Copy buffer bytes ( #864 )
2022-10-25 09:09:47 -07:00
Bill Rich
958266ea84
Run chunker in pipeline ( #859 )
...
* Run chunker in pipeline
* Move ChunkSize and PeekSize to source package.
* Use new Chunk and Peek size location
2022-10-24 13:57:27 -07:00
Bill Rich
3d5f697f9a
Use line aware chunking for git. ( #858 )
2022-10-24 13:00:03 -07:00
Dustin Decker
64ace363af
Change commit to trace level logging
2022-10-24 08:59:52 -07:00
ahrav
46bc010165
Add tests for including github repos. ( #854 )
2022-10-21 07:56:36 -07:00
trufflesteeeve
fb56b9f713
Check rate limit when getting github user ( #855 )
...
Also, don't fetch a github user or their token when both are known. This
currently only affects the Github Token auth type. Github App
installations will continually fetch tokens every time we clone a repo.
In the future we should check the `ExpiresAt` field of the Github App
token and determine if we need to fetch a new one at that point.
2022-10-20 18:14:28 -04:00
ahrav
029519eb01
[THOG-767] ignore gitlab repos ( #853 )
...
* Add ability to ignore repos.
* use std library slices.Contains.
* Add tests.
* Remove zero values from test.
2022-10-19 13:55:44 -07:00
ahrav
2d6aadcb46
[THOG-774] - GitHub ignore repo full name ( #848 )
...
* Use github repo full name.
* fix tests.
2022-10-14 09:20:49 -07:00
ahrav
04c9bb535e
[THOG-768] - Add ability to skip scanning Github repos ( #846 )
...
* Add ability to skip scanning Github repos.
* remove old change.
* rename method.
2022-10-12 16:28:24 -07:00
Dustin Decker
785cead43e
Ignore URIs where the password is redacted ( #842 )
...
Only `*`s in the password is a redacted basic auth URI.
2022-10-11 14:18:52 -07:00
Miccah
2bc4985061
Add SSH config option for the git source ( #830 )
...
* Add SSH config option for the git source
The auth message is empty since we use the git binary underneath to
handle the SSH authentication.
* Import digitaloceanv2
2022-09-28 20:40:01 +02:00
Miccah
891996f546
Do not fail scanning if we cannot enumerate gists ( #826 )
2022-09-27 20:59:10 +02:00
Bill Rich
1c00014051
Include public/private in github metadata ( #812 )
...
* Include public/private in github metadata
* CR feedback
* Fix typos and naming
2022-09-26 14:55:46 -07:00
Dustin Decker
97a73710de
403 on listing user gist should not fail org scan ( #822 )
2022-09-26 14:37:25 -07:00
Dustin Decker
752c848640
Show clone path for git repos ( #823 )
2022-09-26 14:36:55 -07:00
Bill Rich
e3107ad6bb
Move head and base normalization to source ( #818 )
2022-09-23 08:58:45 -07:00
ahrav
92f40c2031
[THOG-709] - Recover from detector panics ( #810 )
2022-09-22 07:01:10 -07:00
trufflesteeeve
63fcf33ce6
Fix improper github org member pagination ( #814 )
...
I'm not sure I fully understand why this issue exists. But I think the
short version is this: When we attempted to paginate users, we would set
a variable's Page value. But that variable appears to not actually be a
pointer, despite being added as one. It probably has to do with how
struct embedding works. Either way, if we make the overall options
variable the whole thing, and update its embedded struct with our page
variable, everything works out.
2022-09-21 16:22:42 -07:00
Bill Rich
509cf8b6fa
Use headref and check empty commits for base ( #815 )
2022-09-21 16:04:01 -07:00
Dustin Decker
335e676caa
Provide user when during private clones with token and fix integration tests ( #811 )
2022-09-19 15:53:21 -07:00
Bill Rich
593f1e6754
Include apiClient in Github source ( #804 )
2022-09-19 14:31:48 -07:00
trufflesteeeve
945de06858
Fix include-members not working on github ( #773 )
2022-09-12 13:26:38 -04:00
Bill Rich
912d8e461d
Add context so to avoid splitting creds. ( #791 )
...
* Add context so to avoid splitting creds.
* Add context newlines to expected results
2022-09-09 15:00:33 -07:00
Dustin Decker
ecfdb0105b
Provide correct username for app cloning and add integration test ( #786 )
2022-09-08 17:41:53 -07:00
Dustin Decker
80b247286b
Improve GitHub debug logging ( #784 )
...
* close bodies early
* add more debug logging to github
* fix nil check
* Add nil checks for response
2022-09-08 12:23:40 -07:00
ahrav
7ba583ca40
[THOG-681] - Handle errors sources ( #783 )
...
* Handle errors w/ github source.
* Fix loop var captured by func literal.
* Fix loop var captured by func literal.
* Set completed progress if the scan completes with no errors.
* Set progress to 100% if the scope and iteration are both 0.
* Fix commentary.
* Fix test.
* Return after the defer to os.RemoveAll.
* Fix unauth scan.
* Inline range loop.
* update tests for partial scan completion with errors. Ensure correct progress is set.
* Update progress for all sources.
* Update github test.
* Address comments.
2022-09-07 19:40:37 -07:00
Bill Rich
41936169c7
Use gitparse for unstaged changes. ( #775 )
2022-09-03 18:01:36 -07:00
Bill Rich
d11ce27f33
Use correct reader in filesystem source ( #756 )
2022-08-30 10:24:52 -07:00
Dustin Decker
fa9479100e
Add common sentry recover library and add into goroutines ( #738 )
...
* Add common sentry recover library and add into goroutines
* fix nits
2022-08-29 11:45:37 -07:00
Bill Rich
0ddd49a1b8
Use file handler and common chunker ( #707 )
2022-08-23 16:35:52 -07:00
