Miccah
e12f0f84a1
Setup SourceUnit interface ( #1393 )
...
* Test: Asymmetrical unmarshal API
* Test: Symmetric marshal API
* Revert "Test: Symmetric marshal API"
This reverts commit f51c64a797
2023-06-16 10:38:28 -05:00
Bill Rich
401688d0a8
Add Validator interface and example ( #1397 )
...
* Add Validator interface and example
* Close sockets and improve error messages
* Remove duplicate error
* Use var declaration so err slice can be nil
2023-06-15 08:24:32 -07:00
Brendan Shaklovitz
10902f802a
Add max object size flag for s3 bucket scanning ( #1294 )
...
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-26 15:39:43 -07:00
iamjpotts
b3d917f9c7
Resolve #1167 by adding support for the AWS_SESSION_TOKEN ( #1170 )
...
* Resolve #1167 by adding support for the AWS_SESSION_TOKEN environment variable and adding a --session-token cli arg
* fix error message
---------
Co-authored-by: Dustin Decker <dustin@trufflesec.com>
2023-04-03 14:56:43 -07:00
Zachary Rice
fb9ae75661
Support for exclude globs at the git log level ( #1202 )
...
* init
* seems to be working
* better comment
* rm conditional
* Add more context to exclude-globs description
2023-03-28 10:46:03 -05:00
ahrav
c617bd7a4e
Add resuming capability to GCS source ( #1161 )
...
* Add resuming capability to GCS source.
* Handle no auth scans.
* complete resume logic
* Use custom function type.
* remove functions.
* linter.
* fix test.
* fix test.
* Handle concurrent map writes.
* use string as CLI flag for include/exclude.
* handle emtpy buckets.
* Handle enumeration on initial job run.
* Rename stats to attributes.
* remove redundant return.
* If test fails due to 400, that is fine, it's expected.
* Add unauth GCS source type.
* comments.
* update proto.
* Use short flag.
* address comments.
2023-03-16 17:53:42 -07:00
ahrav
cbf299aa77
Add gcs scanning integration ( #1153 )
...
* Setup for GCS scanning.
* Update GCS engine w/ projectID req.
* Add concurrency field to gcsManager.
* add errgroup to gcsManager.
* Update gcs manager.
* Use defautl ADC.
* use ADC.'
* Add TOOD.
* add log to iterator completion.
* use a BinaryReader instead of concrete object for channel type.
* initial test for Chunks.
* Add tests for chunking objects.
* Add concurrency.
* update metadata to include content type and acls.
* Add object reading code.
* Add integration test.
* Add entrypoint.
* Add removed wg.Wait().
* remove dead code.
* remove build.
* Remove period from file extension.
* remove used.
* Add comment.
* Setup for GCS scanning.
* Update GCS engine w/ projectID req.
* Add concurrency field to gcsManager.
* add errgroup to gcsManager.
* Update gcs manager.
* Use defautl ADC.
* use ADC.'
* Add TOOD.
* add log to iterator completion.
* use a BinaryReader instead of concrete object for channel type.
* initial test for Chunks.
* Add tests for chunking objects.
* Add concurrency.
* update metadata to include content type and acls.
* Add object reading code.
* Add integration test.
* Add entrypoint.
* Add removed wg.Wait().
* remove dead code.
* remove build.
* remove used.
* Add file type for objects.
* Add check for file type and size.
* Add default file size.
* Add additinoal auth options and remaining CLI flags.
* Handle errors in go routines.
* Handle resuming for buckets.
* Remove redundant words in comment.
* remove ok check on bool check.
* remove extra blank line.
* Add return if handler handles chunk.
* Add comment.
* remove extra blank line.
* cleanup comment.
* Add comment.
* move up fxn.
* go mod tidy.
* Add exclusion to perf testing buckets.
* Handle blocking the channel.
* remove unused const.
* fix tests.
* fix tests.
* Handle gcs manger options better.
* update fxn name.
* Remove arg name.
* ignore buckets in gcsManager test.
* fix test.
* propulate gsManagerOpts.
* inline err check.
* Add readme.
* update readme spelling.
* fix test.
2023-03-07 17:32:04 -08:00
Miccah
c5b4d6f28b
Support file scanning in filesystem source ( #1030 )
...
* Rename directories to paths
* Generate protos
* Add file scanning support to filesystem source
* Add directories back to filesystem proto
* Generate protos
* Combine paths and directories from in source
* Add filesystem filter
* Address comments
2023-02-27 12:15:05 -06:00
SAYGIN Metin
f2139a7615
Github filter support for exclude and include ( #1087 )
...
* test
* Add missing head and base hash back.
---------
Co-authored-by: Ahrav Dutta <ahravdutta02@gmail.com>
2023-02-14 08:40:53 -08:00
ahrav
c5c8d10d28
[chore] - Remove monolithic config struct ( #1091 )
...
* REmove monolithic config struct.
* fix broken test.
2023-02-10 12:43:00 -08:00
ahrav
936a139596
Allow using a glob for include list. ( #977 )
...
* Allow using a glob for include list.
* Update command flag.
* Make comment more clear.
* update comment.
* Allow scanning repo and org at the same time.
2022-12-16 13:28:16 -08:00
ahrav
7ba583ca40
[THOG-681] - Handle errors sources ( #783 )
...
* Handle errors w/ github source.
* Fix loop var captured by func literal.
* Fix loop var captured by func literal.
* Set completed progress if the scan completes with no errors.
* Set progress to 100% if the scope and iteration are both 0.
* Fix commentary.
* Fix test.
* Return after the defer to os.RemoveAll.
* Fix unauth scan.
* Inline range loop.
* update tests for partial scan completion with errors. Ensure correct progress is set.
* Update progress for all sources.
* Update github test.
* Address comments.
2022-09-07 19:40:37 -07:00
Dustin Decker
fa9479100e
Add common sentry recover library and add into goroutines ( #738 )
...
* Add common sentry recover library and add into goroutines
* fix nits
2022-08-29 11:45:37 -07:00
Bill Rich
5ad3bbde37
Use pointer to config ( #715 )
2022-08-16 09:15:25 -07:00
ahrav
73f9d3f0a0
[chore] - Use config struct instead of pointer for engine scans. ( #709 )
...
* Use a config struct instead of pointer when scanning engine sources.
* use config.
2022-08-12 09:56:24 -07:00
ahrav
dcc102a81c
[Thog-371] Utilize config struct for engine scans ( #700 )
...
* Use a config struct when scanning and engine source.
* fix tests.
* Move test_helpers to the sources pkg.
* Handle ScanGit error in tests.
* adderss comments.
* Use functional options.
* Remove temp var.
* Add better var names for the setup functions for each config.
* Remove unused var.
* fix error logs.
* fix error logs.
* single line.
* remove blank lines.
2022-08-10 10:11:13 -07:00
ahrav
d2605354fe
[THOG-332 ]Remove TokenSource interface from the init method of Source. ( #539 )
...
* Remove TokenSource interface from the init method of Source.
* Remove proto message.
* Remove proto message.
* Fix tests.
* Fix filesystem test.
2022-05-13 14:35:06 -07:00
ahrav
b0d79180f6
[THOG-314] Add new parameter to the Init method for the source interface. ( #529 )
...
* Add new parameter to the Init method for the source interface.
* Add Oauth Token service.
* remove .test file.
* remove .test file.
* Fix param spelling.
* fix tests with new param in init
* Add missing gock lib.
2022-05-10 11:11:43 -07:00
steeeve
a770f643df
Add placeholder for encoded resume info in SetProgressComplete
2022-03-24 12:43:36 -04:00
Bill Rich
e8234c3514
Remove unused code and add git binary check ( #80 )
2022-03-14 17:47:18 -07:00
Bill Rich
c144630c54
Add parent results to ignore list ( #47 )
...
* Add parent results to ignore list
* Force concurrency to 1 when base commit is set
2022-03-04 08:39:22 -08:00
Dustin Decker
c20e9f4732
improvements
2022-03-04 08:39:17 -08:00
Dustin Decker
77418fb3f8
module v3
2022-02-15 18:54:47 -08:00
Dustin Decker
4218c39d99
Initial CLI w/ partially implemented Git source and demo detector ( #1 )
2022-01-13 12:02:24 -08:00
