Support Windows Directory Resolver
Add function that converts windows to posix functionality
Add function that converts posix to windows
Add build tags to remove windows developer environment errors
redact carriage return specific windows issues
Signed-off-by: Christopher Phillips <christopher.phillips@anchore.com>
* Fix CPE generation when the generated CPE contains invalid characters
Currently syft seems to generate invalid CPEs which do not
conform with the official CPE spec. This is because the underlying
nvdtools library is not a completely spec compliant implementation
and has some interesting bugs/issues.
The following are the list of issues I have encountered with nvdtools:
1. It parses strings which are not CPEs incorrectly as valid CPEs. This
messes up our filter function which is supposed to filter out any
incorrect CPEs we generate. In order to fix this, I have introduced
a new regex in the NewCPE function which follows the upstream spec and
filters out any incorrect CPEs.
2. Introduce wfn.WFNize for any cpe attributes we infer from packages.
This ensures that we are escaping and quoting any special characters
before putting them into CPEs. Note that nvdtools has yet another bug
in the WFNize function, specifically the "addSlashesAt" part of the
function which stops the loop as soon as it encounters ":" a valid
character for a WFN attribute after quoting, but the way nvdtools
handles it causes it to truncate strings that container ":". As a result
strings like "prefix:1.2" which would have been quoted as "prefix\:1.2"
end up becoming "prefix" instead causing loss of information and
incorrect CPEs being generated. As a result in such cases, we remove out
strings containing ":" in any part entirely for now. This is similar
to the way we were handling CPE filtering in the past with http urls as
vendor strings
3. Add special handling for version which contain ":" due to epochs in
debian and rpm. In this case, we strip out the parts before ":" i.e.
the epoch and only output the actual function. This ensures we are not
discarding valid version strings due to pt #.2.
In the future we should look at moving to a more spec compliant cpe
parsing library to avoid such shenanigans.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Remove WFNize for input strings
WFNize seems to not be part of the standard as per
https://pkg.go.dev/github.com/facebookincubator/nvdtools@v0.1.4/wfn#WFNize
and seems to have bugs/issues with encode/decode cycles, so I am
just removing it at this point and relying on the CPE regex to filter
out invalid CPEs for now.
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Quote the string on decode to ensure consistent CPE string generation
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add test cases for round-tripping the CPE and fix strip slashes
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add comprehensive tests for cpe parsing
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Use strings.Builder instead of byte buffer
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* add direct_url.json fields to python metadata
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* rename DirectURLOrigin struct; add stub for file
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add detection for direct_url.json
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* Add tests for direct-url information and add it to the output purl
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Update golden snapshot ids after adding new python package metadata field
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
* Add test names for packageurl tests
Signed-off-by: Sambhav Kothari <skothari44@bloomberg.net>
Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* set package ID in catalogers and improve hashing performance
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update setting ID + tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cataloging within universal binaries
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update json test fixtures
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add comments + correct 32 bit multi arch magic check
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cyclone json format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* adapt format to sbom.SBOM structure
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* cycloneDX json output with official lib
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add cycloneDX 1.3 schema output in xml
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix lints errors
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove cycloneDX 1.2 format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* update cycloneDX xml schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix cyclone according to schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use RFC 2141 URN form of uuid for serial number
add schema validation for cycloneDX 1.3 JSON output
add yajsv cli for JSON schema validation during tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod up
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go get json schema validator
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* install yajsv without mess with go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* reuse code between cycloneDX json & xml encoders
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add output options for cyclone XML
add bom.json to .gitignore
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add cyclone json format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* adapt format to sbom.SBOM structure
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* cycloneDX json output with official lib
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add cycloneDX 1.3 schema output in xml
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix lints errors
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* remove cycloneDX 1.2 format
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* update cycloneDX xml schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix cyclone according to schema
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* use RFC 2141 URN form of uuid for serial number
add schema validation for cycloneDX 1.3 JSON output
add yajsv cli for JSON schema validation during tests
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* tidying go mod up
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go get json schema validator
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* install yajsv without mess with go mod
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* reuse code between cycloneDX json & xml encoders
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* add output options for cyclone XML
add bom.json to .gitignore
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix cyclone12xml removal
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* feedback changes
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* go mod tidy
Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
* fix spdx namespace and add scheme range assertions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* validate SPDX document name from source metadata
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* comment why namespace tests only check prefix
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove power-user document shape
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add power-user specific fields to syft-json format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* port remaining spdx-json relationships to sbom model
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add coordinate set
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add SBOM file path helper
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use internal mimetype helper in go binary cataloger
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add new package-of relationship
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update json schema to v2
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* replace power-user presenter with syft-json format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix tests and linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove "package-of" relationship (in favor of "contains")
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add tests for spdx22json format encoding enhancements
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update TODO and log entries
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* introduce sbom.Descriptor
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split source.Location and create source.Coordinates for minimal path addressing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* move coordinates into separate file
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* Update syft/source/coordinates.go
Co-authored-by: Dan Luhring <luhring@users.noreply.github.com>
* migrate pkg.ID and pkg.Relationship to artifact package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* return relationships from tasks
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix more tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add artifact.Identifiable by Identity() method
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove catalog ID assignment
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust spdx helpers to use copy of packages
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* stabilize package ID relative to encode-decode format cycles
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* rename Identity() to ID()
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use zero value for nils in ID generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* enable source.Location to be identifiable
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* hoist up package relationship discovery to analysis stage
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update ownership-by-file-overlap relationship description
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add test reminders to put new relationships under test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* adjust PHP composer.lock parser function to return relationships
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* [wip] single sbom doc
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix more tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update cli tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove scope in import path
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* swap SPDX tag-value formatter to single sbom document
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* bust CLI cache
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update fixture to byte diff
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* byte for byte
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* bust the cache
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* who needs cache
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add jar for testing
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* no more bit flips
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* update apk with the delta for image and directory cases
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* restore cache workflow
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* use anchore fork of go-presenter
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* drop coverage threshold
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add new spdx tag-value format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove public presenter package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use named pipe bit when checking for piped input
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
Co-authored-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* remove existing spdxjson presenter + helpers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add new spdx22json format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add common sdpxhelpers (migrated)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use new common spdx helpers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* wire up new spdx22json format object
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove lossless syft-specific property bags
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove spdxjson decoder and validator
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add nil checks in spdx test helpers
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove empty default case
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use explicit golden snapshot
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add new cyclonedx format object
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove cyclonedx presenter
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove cyclonedx presenter call
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove dependence on golden images for format tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* wire up new formt + rename all-presenters ref
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add CLI test to ensure that all formats can be expressed as report output
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add cyclonedx version and encoding format to package name
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* optionally preserve format snapshot images
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix linting + text unit tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update build tags, ui support, and stereoscope, and release for windows support
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add new format pattern
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add syftjson format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add internal formats helper
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add SBOM encode/decode to lib API
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove json presenter + update presenter tests to use common utils
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove presenter format enum type + add formats shim in presenter helper
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add MustCPE helper for tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update usage of format enum
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add test fixtures for encode/decode tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix integration test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* migrate format detection to use reader
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* address review comments
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fixed piped input
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow pipedinput helper to raise an error
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* factor out verbosity check to function
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add query by MIME type to source.FileResolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* import stereoscope lib changes to find mime type
- add bin cataloger
- add bin parser
- add mime type go utils
- import new resolver
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
* add go std library code to unpack bin
- keep them in their own (original) files
- add note for "this code was copied from"
- comment the lines the required changing
Signed-off-by: Christopher Angelo Phillips <christopher.phillips@anchore.com>
Co-authored-by: Alex Goodman <alex.goodman@anchore.com>
* add query by MIME type to source.FileResolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* pull in stereoscope MIME type feature
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* fix tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add output to file option
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* log errors on close of the report destination
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove file option from persistent args
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update file option comments and logging
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* allow for multiple UI fallback options
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update UI select signatures + tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* update SPDX license list from 3.13 to 3.14
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove license list version from spdx snapshot unit tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* change directory resolver to ignore system runtime paths + drive by index
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add event/etui support for filesystem indexing (for dir resolver)
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add warnings for path indexing problems
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add directory resolver index tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* improve testing around directory resolver
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* renamed p var to path when not conflicting with import
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* pull docker image in CLI dir scan timeout test
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* ensure file not exist errors do not stop directory resolver indexing
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* split UI from event handling
Signed-off-by: Alex Goodman <wagoodman@gmail.com>
* add event loop tests
Signed-off-by: Alex Goodman <wagoodman@gmail.com>
* use stereoscope cleanup function during signal handling
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* correct error wrapping in packages cmd
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* migrate ui event handlers to ui package
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* clarify command worker input var + remove dead comments
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add initial spdx support
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* expose FileOwner and use in SPDX presenter
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add initial json support for SPDX
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add remaining package fields
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add spdx license list generation + tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* keep fileOwner unexported from pkg
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* restore cli test util
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add external refs to spdx tag-value format
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add golang support to CPE generation
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* use tag-value format as default "spdx" format flavor
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add tests around spdx presenters + refactor presenter tests
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* add bouncer exception for spdx tools-golang repo
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
* remove spdx model questions
Signed-off-by: Alex Goodman <alex.goodman@anchore.com>
