* add exportable validate function for github
* update validator
* use the context
* gate to prevent panic
* wrap error with context
* wrap error with context for basic auth and unauth
* add role assumption for s3 source
* refactor role assumption to repeatable string
user can pass array of roles to assume
* refactor s3 chunks to handle passed roleARNs
* add role-session name
use timestamp to make dynamic
* add docstring for rolearn strings()
* make sure role ars are passed into source
* refactor role assumption functionality
break s3 bucket scanning into sep. function
* add log check on assume role
* fix role iteration
- Make sure s3 struct is populated with roles
- add separate new client instantiation for role-based access
- iterates through each role
* add comment
* protobuf revert for merge
* re-run make proto
* lint cleanup
* cleanup TODOs
* drop redundant switch case in assumerole client
* use less verbose 'ctx' designator
* breakout functionality from Chunks
- separate functions for:
- enumerating buckets to scan
- scanning objects within the buckets
* remake protobuf defs
* allow scan to continue on single bucket err
* add readme docs
* minor fixups
With the introduction of the SourceManager, the chunks channel became
private and read-only. This provides a method to write chunks into the
channel as we transition away from needing to do that.
* Use custom context for archive handler of specialized archives.
* fix arg.
* fix test.
* use re-reader.
* use re-reader.
* Update error and comments.
* Add better error handling.
* update.
* setup
* update time out case to return detector result
* fix
* remove unneeded comment
* remove debug print
* cleanup
* more robust error handling
* reflect new detector template changes
* fixes
* mark response body check err as indeterminate
* Add handler for .deb file formats.
* Add handler for .rpm file formats.
* update.
* move logic to general archive handler.
* update const.
* Add compile time guard.
* Remove redundant parens.
* Add checks to make sure we have the tools installed to extract arhives.
* Limit size of temp file for arhive reading.
* handle nested archives.
* add comment.
* use consistent name for tempEnv -> env
* fix handler fxn signature.
* added functionality to scan docker images with digests instead of tags
* cleaned import statement
* added unit test for baseAndTag parsing + remote digest scan
Verification of an Alchemy secret requires putting the candidate secret directly into a URL. This makes the URL potentially sensitive, and if the request fails, we don't want to save it anywhere that might inadvertently get logged elsewhere - like the resulting error message. (Despite verification failing, this error message is only saved if the failure is indeterminate, which means that the secret might actually be live.)
* Init attempt at tui with bubble tea.
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Add starting and source selection options
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Rewrite models into a state machine
* Update source descriptions
* Make subpages implement tea.Model
* Rename page0 and page1 to be more descriptive
* Adjust styling and adding color consts
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Add helper generic function to call Update and type cast
* Setup plumbing for source configuration page
* Use CLI introspection for source configuration (WIP)
* Experiment with table view
* Replace table with form fields
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Change 🔒 to 💸
* Copy components from soft-serve
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Copy styles from soft-serve
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Copy common from soft-serve
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Refactor into pages
This is still a WIP, but the main structure is there.
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Trying out selector for wizard intro
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Use selector with custom View
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Change Item to be an enum
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Add link pages
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Update source select to use selector
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Delete source configure page and add blank tabs
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Add tab placeholder pages for configurationi
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Added headers and style to each tab
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Update with new sources
* Remove kingpin attribute from SourceItem
* Add basic form field and source structuring
* Hookup git form fields with an underlying textinput component
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Update forms for git and github
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Add labels per text input
* Add sources and adjust styling
* add basic trufflehog configuration page
* Add skip button to textinputs component
* Emit and handle textinputs skip/submit button commands
* Don't quit when q is pressed on the sourceConfigurePage
* Build trufflehog command based on source config vals
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* Build flags based on truffle config inputs
* Update summary section
* Add generated truffle fields
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* update summary to correctly print info
* Go back a page when escape key is pressed
* WIP run page list
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Allow running trufflehog from the run page
Co-authored-by: hxnyk <8292703+hxnyk@users.noreply.github.com>
* Add option to view help docs
Co-authored-by: mcastorina <m.castorina93@gmail.com>
* comment out unused styles and remove unused types
* Capitalize H in TruffleHog
* remove unneeded fmt.Sprintf
---------
Co-authored-by: mcastorina <m.castorina93@gmail.com>
This PR implements tri-state verification in the FTP detector. The verification timeout was made injectable to support a new test case. Some test cases that had already been broken have been fixed as well.
* Add common chunker.
* add comment.
* use better config name.
* Add common chunk reader to s3.
* Add common chunk reader to git, gcs, circleci.
* revert gcs.
* revert gcs.
* fix chunker.
* revert gcs.
* update cancellablewrite.
* revert impl.
* update to remove totalsize.
* Fix my goof.
* Use unified struct in chunkreader.
* return err instead of logging and returning.
* rename error to err.
* only send single ChunkResult even if there is an error and chunkBytes.
* fix logic.
* Add SourceManager to Engine struct
* Update Engine methods to use the SourceManager
* Fix GCS test
The original was testing that `Init()` errors weren't surfaced in
`Finish()`, but the `SourceManager` changed that behavior.
* JobProgress race fixes
* Add contextual values
* Remove unused code
* Add debug logs
* Rename WithConcurrency to WithConcurrentSources
* Always forward chunks to the output chunks channel
This PR implements tri-state verification for the LDAP detector. This implementation looks for network errors to explicitly flag as indeterminate, rather than authentication errors to explicitly flag as determinate; this is because the error that occurs from authentication failures doesn't appear to have its own type and I didn't want to have to match on the error message text.
* feat: initial support for bare repositories
* feat: use concatenation instead of formatting and os.Getenv instead of os.Environ
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: go-git update with pre-receive hooks fix
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: remove info about pre-receive hook from README.md for now
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: don't scan staged while using --bare option, fixes to make it work with the latest master
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* fix: small refactor according to #1518
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
---------
Signed-off-by: Savely Krasovsky <savely@krasovs.ky>
* Github Oauth2 verification
* Use prefix and include RawV2
* Make gh_oauth2 a new detector
* Remove unused struct
* Remove versioner
* Remove unused code
* Refactor git source to allow ScanOptions and use source in engine
Refactor the Chunks method of the git Source to call out to two helper
methods: scanRepos and scanDirs which scans s.conn.Repositories and
s.conn.Directories respectively. The only notable change in behavior is
that a credential is no longer necessary if there are no
s.conn.Repositories to scan.
* Preserve ScanGit functionality of not cleaning up temporary files
