Mirrors/syft
2
0
Fork 0
mirror of https://github.com/anchore/syft synced 2024-11-10 06:14:16 +00:00
Code Issues Projects Releases Packages Wiki Activity

Use go 1.18 buildinfo to catalog binaries (#827)

Browse source 
* initial working version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* added build settings to pkg metadata

wip - unit tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* handle mach-O FatFiles

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support to mod replace

fixed golang catalger tests

trying GH Actions with go 1.18rc1

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* log error

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* use go-macholibre for extraction

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleaner tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add version to main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* check macho file with macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* run golangci in its own workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci workflow

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix golangci wf yml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* wip - golangci wf

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get arch from bin file headers

upgrade macholibre

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* test new stereoscope lazy reader interface

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove devel version from golang cataloger

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* switch github workflows to go1.18 stable

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add union reader interface in golang cataloger

update stereoscope

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* go mod tidy

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* simpler golangci validation

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix makefile

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* get archs refactor

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for golang version

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix go bin tests

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* golangci nolint needs a \n before package

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* cleanup

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* move golangci-lint to its own jobs again

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix ci yaml

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* add support for xcoff files

add arch assets to test bin file types

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* clean up golangci-lint config

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* nolint for xcoff

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain nolints

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* remove unused xcoff testdata assets

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* make go bin test-fixtures in docker

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* fix make clean with -f

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update json output schema

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* update schema version in test fixture

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* feedback changes

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>

* explain possible empty main module

Signed-off-by: Jonas Galvão Xavier <jonas.agx@gmail.com>
This commit is contained in:
Jonas Xavier 2022-03-16 17:07:02 -07:00 committed by GitHub
parent ee0a1d172c
commit 6ef3e45ffc
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
32 changed files with 3110 additions and 805 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
.github/workflows/release.yaml vendored
View file

@ -9,7 +9,7 @@ on:
- "v*" - "v*"
env: env:
GO_VERSION: "1.17.x" GO_VERSION: "1.18.x"
jobs: jobs:
quality-gate: quality-gate:
@ -34,6 +34,16 @@ jobs:
checkName: "Static analysis" checkName: "Static analysis"
ref: ${{ github.event.pull_request.head.sha || github.sha }} ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check static analysis results - golangci-lint
uses: fountainhead/action-wait-for-check@v1.0.0
id: static-analysis-golangci-lint
with:
token: ${{ secrets.GITHUB_TOKEN }}
# This check name is defined as the github action job name (in .github/workflows/testing.yaml)
checkName: "Static analysis - golangci-lint"
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check unit test results - name: Check unit test results
uses: fountainhead/action-wait-for-check@v1.0.0 uses: fountainhead/action-wait-for-check@v1.0.0
id: unit id: unit
@ -80,9 +90,10 @@ jobs:
ref: ${{ github.event.pull_request.head.sha || github.sha }} ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Quality gate - name: Quality gate
if: steps.static-analysis.outputs.conclusion != 'success' || steps.unit.outputs.conclusion != 'success' || steps.integration.outputs.conclusion != 'success' || steps.cli-linux.outputs.conclusion != 'success' || steps.acceptance-linux.outputs.conclusion != 'success' || steps.acceptance-mac.outputs.conclusion != 'success' if: steps.static-analysis.outputs.conclusion != 'success' || steps.static-analysis-golangci-lint.outputs.conclusion != 'success' || steps.unit.outputs.conclusion != 'success' || steps.integration.outputs.conclusion != 'success' || steps.cli-linux.outputs.conclusion != 'success' || steps.acceptance-linux.outputs.conclusion != 'success' || steps.acceptance-mac.outputs.conclusion != 'success'
run: | run: |
echo "Static Analysis Status: ${{ steps.static-analysis.conclusion }}" echo "Static Analysis Status: ${{ steps.static-analysis.conclusion }}"
echo "Static Analysis Status - Golangci-Lint: ${{ steps.static-analysis-golangci-lint.conclusion }}"
echo "Unit Test Status: ${{ steps.unit.outputs.conclusion }}" echo "Unit Test Status: ${{ steps.unit.outputs.conclusion }}"
echo "Integration Test Status: ${{ steps.integration.outputs.conclusion }}" echo "Integration Test Status: ${{ steps.integration.outputs.conclusion }}"
echo "Acceptance Test (Linux) Status: ${{ steps.acceptance-linux.outputs.conclusion }}" echo "Acceptance Test (Linux) Status: ${{ steps.acceptance-linux.outputs.conclusion }}"

58
.github/workflows/validations.yaml vendored
View file

@ -7,9 +7,49 @@ on:
pull_request: pull_request:
env: env:
GO_VERSION: "1.17.x" GO_VERSION: "1.18.x"
GO_STABLE_VERSION: true
jobs: jobs:
# TODO: remove this job once golanci-lint is compatible with g01.18+
Static-Analysis-Golanci-lint:
name: "Static analysis - golangci-lint"
runs-on: ubuntu-20.04
steps:
- uses: actions/setup-go@v2
with:
go-version: "1.17" # NOTE: please use GO_VERSION once golangci supports go1.18+
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2
- name: Restore tool cache
id: tool-cache
uses: actions/cache@v2.1.3
with:
path: ${{ github.workspace }}/.tmp
key: ${{ runner.os }}-tool-${{ hashFiles('Makefile') }}
- name: Restore go cache
id: go-cache
uses: actions/cache@v2.1.3
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ env.GO_VERSION }}-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-${{ env.GO_VERSION }}-
- name: (cache-miss) Bootstrap all project dependencies
if: steps.tool-cache.outputs.cache-hit != 'true' || steps.go-cache.outputs.cache-hit != 'true'
run: make bootstrap
- name: Bootstrap CI environment dependencies
run: make ci-bootstrap
- name: Run static analysis
run: make static-analysis-golanci-lint
Static-Analysis: Static-Analysis:
# Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline # Note: changing this job name requires making the same update in the .github/workflows/release.yaml pipeline
@ -19,6 +59,7 @@ jobs:
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2
with: with:
go-version: ${{ env.GO_VERSION }} go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@ -56,6 +97,7 @@ jobs:
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2
with: with:
go-version: ${{ env.GO_VERSION }} go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@ -92,6 +134,16 @@ jobs:
path: syft/pkg/cataloger/java/test-fixtures/java-builds/packages path: syft/pkg/cataloger/java/test-fixtures/java-builds/packages
key: ${{ runner.os }}-unit-java-cache-${{ hashFiles( 'syft/pkg/cataloger/java/test-fixtures/java-builds/packages.fingerprint' ) }} key: ${{ runner.os }}-unit-java-cache-${{ hashFiles( 'syft/pkg/cataloger/java/test-fixtures/java-builds/packages.fingerprint' ) }}
- name: Build cache key for go binary test-fixture blobs (for unit tests)
run: make go-binaries-fingerprint
- name: Restore Go binary test-fixture cache
id: unit-go-binary-cache
uses: actions/cache@v2.1.3
with:
path: syft/pkg/cataloger/golang/test-fixtures/archs/binaries
key: ${{ runner.os }}-unit-go-binaries-cache-${{ hashFiles( 'syft/pkg/cataloger/golang/test-fixtures/archs/binaries.fingerprint' ) }}
- name: Run unit tests - name: Run unit tests
run: make unit run: make unit
@ -108,6 +160,7 @@ jobs:
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2
with: with:
go-version: ${{ env.GO_VERSION }} go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@ -159,6 +212,7 @@ jobs:
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2
with: with:
go-version: ${{ env.GO_VERSION }} go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@ -231,6 +285,7 @@ jobs:
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2
with: with:
go-version: ${{ env.GO_VERSION }} go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2 - uses: actions/checkout@v2
@ -338,6 +393,7 @@ jobs:
- uses: actions/setup-go@v2 - uses: actions/setup-go@v2
with: with:
go-version: ${{ env.GO_VERSION }} go-version: ${{ env.GO_VERSION }}
stable: ${{ env.GO_STABLE_VERSION }}
- uses: actions/checkout@v2 - uses: actions/checkout@v2

13
Makefile
View file

@ -122,7 +122,12 @@ bootstrap: $(RESULTSDIR) bootstrap-go bootstrap-tools ## Download and install al
$(call title,Bootstrapping dependencies) $(call title,Bootstrapping dependencies)
.PHONY: static-analysis .PHONY: static-analysis
static-analysis: lint check-go-mod-tidy check-licenses static-analysis: check-go-mod-tidy check-licenses
# NOTE: isolating golanci-lint so it runs over go1.17 in CI, since it is not compatible with
# go1.18+
.PHONY: static-analysis-golanci-lint
static-analysis-golanci-lint: lint
.PHONY: lint .PHONY: lint
lint: ## Run gofmt + golangci lint checks lint: ## Run gofmt + golangci lint checks
@ -214,6 +219,12 @@ java-packages-fingerprint:
cd syft/pkg/cataloger/java/test-fixtures/java-builds && \ cd syft/pkg/cataloger/java/test-fixtures/java-builds && \
make packages.fingerprint make packages.fingerprint
.PHONY: go-binaries-fingerprint
go-binaries-fingerprint:
$(call title,Go binaries test fixture fingerprint)
cd syft/pkg/cataloger/golang/test-fixtures/archs && \
make binaries.fingerprint
.PHONY: fixtures .PHONY: fixtures
fixtures: fixtures:
$(call title,Generating test fixtures) $(call title,Generating test fixtures)

3
go.mod
View file

@ -9,11 +9,12 @@ require (
github.com/adrg/xdg v0.2.1 github.com/adrg/xdg v0.2.1
github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074 github.com/alecthomas/jsonschema v0.0.0-20210301060011-54c507b6f074
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3 github.com/anchore/stereoscope v0.0.0-20220315185520-25183ec78f40
github.com/antihax/optional v1.0.0 github.com/antihax/optional v1.0.0
github.com/bmatcuk/doublestar/v4 v4.0.2 github.com/bmatcuk/doublestar/v4 v4.0.2
github.com/dustin/go-humanize v1.0.0 github.com/dustin/go-humanize v1.0.0

6
go.sum
View file

@ -276,6 +276,8 @@ github.com/alexflint/go-filemutex v0.0.0-20171022225611-72bdc8eae2ae/go.mod h1:C
github.com/alexkohler/prealloc v1.0.0/go.mod h1:VetnK3dIgFBBKmg0YnD9F9x6Icjd+9cvfHR56wJVlKE= github.com/alexkohler/prealloc v1.0.0/go.mod h1:VetnK3dIgFBBKmg0YnD9F9x6Icjd+9cvfHR56wJVlKE=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg= github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=
github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf/go.mod h1:FaODhIA06mxO1E6R32JE0TL1JWZZkmjRIAd4ULvHUKk= github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf/go.mod h1:FaODhIA06mxO1E6R32JE0TL1JWZZkmjRIAd4ULvHUKk=
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb h1:iDMnx6LIjtjZ46C0akqveX83WFzhpTD3eqOthawb5vU=
github.com/anchore/go-macholibre v0.0.0-20220308212642-53e6d0aaf6fb/go.mod h1:DmTY2Mfcv38hsHbG78xMiTDdxFtkHpgYNVDPsF2TgHk=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 h1:C9W/LAydEz/qdUhx1MdjO9l8NEcFKYknkxDVyo9LAoM= github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63 h1:C9W/LAydEz/qdUhx1MdjO9l8NEcFKYknkxDVyo9LAoM=
github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63/go.mod h1:6qH8c6U/3CBVvDDDBZnPSTbTINq3cIdADUYTaVf75EM= github.com/anchore/go-rpmdb v0.0.0-20210914181456-a9c52348da63/go.mod h1:6qH8c6U/3CBVvDDDBZnPSTbTINq3cIdADUYTaVf75EM=
github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 h1:VzprUTpc0vW0nnNKJfJieyH/TZ9UYAnTZs5/gHTdAe8= github.com/anchore/go-testutils v0.0.0-20200925183923-d5f45b0d3c04 h1:VzprUTpc0vW0nnNKJfJieyH/TZ9UYAnTZs5/gHTdAe8=
@ -284,8 +286,8 @@ github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b h1:e1bmaoJfZV
github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E= github.com/anchore/go-version v1.2.2-0.20200701162849-18adb9c92b9b/go.mod h1:Bkc+JYWjMCF8OyZ340IMSIi2Ebf3uwByOk6ho4wne1E=
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b h1:YJWYt/6KQXR9JR46lLHrTTYi8rcye42tKcyjREA/hvA= github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b h1:YJWYt/6KQXR9JR46lLHrTTYi8rcye42tKcyjREA/hvA=
github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4= github.com/anchore/packageurl-go v0.1.1-0.20220314153042-1bcd40e5206b/go.mod h1:Blo6OgJNiYF41ufcgHKkbCKF2MDOMlrqhXv/ij6ocR4=
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3 h1:Kx2jlMdENAf4cVjYGYLI+fiavVhzhtmU89GUYDITJ1w= github.com/anchore/stereoscope v0.0.0-20220315185520-25183ec78f40 h1:OSyFMZgEwQW0wFyv10kEi9kmB52FQFRUQmc2H6d8LTY=
github.com/anchore/stereoscope v0.0.0-20220307154759-8a5a70c227d3/go.mod h1:XESZQTgFETDBatmyoet6XZ0zVknoIMDSAhj2INj2a5w= github.com/anchore/stereoscope v0.0.0-20220315185520-25183ec78f40/go.mod h1:OVVJ4y/L26pspeeHNvNa7GeQfoLPWxJe13iODl9x5l4=
github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8= github.com/andreyvit/diff v0.0.0-20170406064948-c7f18ee00883/go.mod h1:rCTlJbsFo29Kk6CurOXKm700vrz8f0KW0JNfpkRJY/8=
github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y= github.com/andybalholm/brotli v1.0.0/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=
github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y= github.com/andybalholm/brotli v1.0.1/go.mod h1:loMXtMfwqflxFJPmdbJO0a3KNoPuLBgiu3qAvBg8x/Y=

2
internal/constants.go
View file

@ -6,5 +6,5 @@ const (
// JSONSchemaVersion is the current schema version output by the JSON encoder // JSONSchemaVersion is the current schema version output by the JSON encoder
// This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment. // This is roughly following the "SchemaVer" guidelines for versioning the JSON schema. Please see schema/json/README.md for details on how to increment.
JSONSchemaVersion = "3.1.0" JSONSchemaVersion = "3.1.1"
) )

4
internal/formats/syftjson/test-fixtures/snapshot/TestDirectoryEncoder.golden
View file

@ -88,7 +88,7 @@
} }
}, },
"schema": { "schema": {
"version": "3.1.0", "version": "3.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.0.json" "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.1.json"
} }
} }

4
internal/formats/syftjson/test-fixtures/snapshot/TestEncodeFullJSONDocument.golden
View file

@ -184,7 +184,7 @@
} }
}, },
"schema": { "schema": {
"version": "3.1.0", "version": "3.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.0.json" "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.1.json"
} }
} }

4
internal/formats/syftjson/test-fixtures/snapshot/TestImageEncoder.golden
View file

@ -111,7 +111,7 @@
} }
}, },
"schema": { "schema": {
"version": "3.1.0", "version": "3.1.1",
"url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.0.json" "url": "https://raw.githubusercontent.com/anchore/syft/main/schema/json/schema-3.1.1.json"
} }
} }

1237
schema/json/schema-3.1.1.json Normal file
View file

File diff suppressed because it is too large Load diff

48
syft/pkg/cataloger/golang/binary_cataloger.go
View file

@ -4,10 +4,12 @@ Package golang provides a concrete Cataloger implementation for go.mod files.
package golang package golang
import ( import (
"bytes"
"fmt" "fmt"
"io"
"io/ioutil"
"github.com/anchore/syft/internal" "github.com/anchore/syft/internal"
"github.com/anchore/syft/internal/log" "github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/artifact" "github.com/anchore/syft/syft/artifact"
"github.com/anchore/syft/syft/pkg" "github.com/anchore/syft/syft/pkg"
@ -38,19 +40,51 @@ func (c *Cataloger) Catalog(resolver source.FileResolver) ([]pkg.Package, []arti
} }
for _, location := range fileMatches { for _, location := range fileMatches {
r, err := resolver.FileContentsByLocation(location) readerCloser, err := resolver.FileContentsByLocation(location)
if err != nil { if err != nil {
return pkgs, nil, fmt.Errorf("failed to resolve file contents by location=%q: %w", location.RealPath, err) log.Warnf("golang cataloger: opening file: %v", err)
continue
} }
goPkgs, err := parseGoBin(location, r, openExe) reader, err := getUnionReader(readerCloser)
if err != nil { if err != nil {
log.Warnf("could not parse possible go binary at %q: %+v", location.RealPath, err) return nil, nil, err
} }
internal.CloseAndLogError(r, location.RealPath) mods, archs := scanFile(reader, location.RealPath)
pkgs = append(pkgs, goPkgs...) internal.CloseAndLogError(readerCloser, location.RealPath)
for i, mod := range mods {
pkgs = append(pkgs, buildGoPkgInfo(location, mod, archs[i])...)
}
} }
return pkgs, nil, nil return pkgs, nil, nil
} }
func getUnionReader(readerCloser io.ReadCloser) (unionReader, error) {
reader, ok := readerCloser.(unionReader)
if ok {
return reader, nil
}
log.Debugf("golang cataloger: unable to use stereoscope file, reading entire contents")
b, err := ioutil.ReadAll(readerCloser)
if err != nil {
return nil, fmt.Errorf("unable to read contents from go binary: %w", err)
}
bytesReader := bytes.NewReader(b)
reader = struct {
io.ReadCloser
io.ReaderAt
io.Seeker
}{
ReadCloser: io.NopCloser(bytesReader),
ReaderAt: bytesReader,
Seeker: bytesReader,
}
return reader, nil
}

30
syft/pkg/cataloger/golang/binary_cataloger_test.go Normal file
View file

@ -0,0 +1,30 @@
package golang
import (
"io"
"strings"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func Test_getUnionReader_notUnionReader(t *testing.T) {
expectedContents := "this is a test"
reader := io.NopCloser(strings.NewReader(expectedContents))
// make certain that the test fixture does not implement the union reader
_, ok := reader.(unionReader)
require.False(t, ok)
actual, err := getUnionReader(reader)
require.NoError(t, err)
_, ok = actual.(unionReader)
require.True(t, ok)
b, err := io.ReadAll(actual)
require.NoError(t, err)
assert.Equal(t, expectedContents, string(b))
}

354
syft/pkg/cataloger/golang/exe.go
View file

@ -1,354 +0,0 @@
// This code was copied from the Go std library.
// https://github.com/golang/go/blob/master/src/cmd/go/internal/version/exe.go
// Copyright 2019 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//nolint
package golang
import (
"bytes"
"debug/elf"
"debug/macho"
"debug/pe"
"fmt"
"io"
"strings"
)
// An exe is a generic interface to an OS executable (ELF, Mach-O, PE, XCOFF).
type exe interface {
// Close closes the underlying file.
Close() error
// ReadData reads and returns up to size byte starting at virtual address addr.
ReadData(addr, size uint64) ([]byte, error)
// ArchName returns a string that represents the CPU architecture of the executable.
ArchName() string
// DataStart returns the writable data segment start address.
DataStart() uint64
}
// openExe opens file and returns it as an exe.
// we changed this signature from accpeting a string
// to a ReadCloser so we could adapt the code to the
// stereoscope api. We removed the file open methods.
func openExe(file io.ReadCloser) ([]exe, error) {
/*
f, err := os.Open(file)
if err != nil {
return nil, err
}
data := make([]byte, 16)
if _, err := io.ReadFull(f, data); err != nil {
return nil, err
}
f.Seek(0, 0)
*/
data, err := io.ReadAll(file)
if err != nil {
return nil, err
}
r := bytes.NewReader(data)
f := io.NewSectionReader(r, 0, int64(len(data)))
if bytes.HasPrefix(data, []byte("\x7FELF")) {
e, err := elf.NewFile(f)
if err != nil {
return nil, err
}
return []exe{&elfExe{file, e}}, nil
}
if bytes.HasPrefix(data, []byte("MZ")) {
e, err := pe.NewFile(f)
if err != nil {
return nil, err
}
return []exe{&peExe{file, e}}, nil
}
if bytes.HasPrefix(data, []byte("\xFE\xED\xFA")) || bytes.HasPrefix(data[1:], []byte("\xFA\xED\xFE")) {
e, err := macho.NewFile(f)
if err != nil {
return nil, err
}
return []exe{&machoExe{file, e}}, nil
}
// adding macho multi-architecture support (both for 64bit and 32 bit)... this case is not in the stdlib yet
if bytes.HasPrefix(data, []byte("\xCA\xFE\xBA\xBE")) || bytes.HasPrefix(data, []byte("\xCA\xFE\xBA\xBF")) {
fatExe, err := macho.NewFatFile(f)
if err != nil {
return nil, err
}
var exes []exe
for _, arch := range fatExe.Arches {
exes = append(exes, &machoExe{file, arch.File})
}
return exes, nil
}
return nil, fmt.Errorf("unrecognized executable format")
}
// elfExe is the ELF implementation of the exe interface.
// updated os to be io.ReadCloser to interopt with stereoscope
type elfExe struct {
os io.ReadCloser
f *elf.File
}
func (x *elfExe) Close() error {
return x.os.Close()
}
func (x *elfExe) ArchName() string {
return cleanElfArch(x.f.Machine)
}
func cleanElfArch(machine elf.Machine) string {
return strings.TrimPrefix(strings.ToLower(machine.String()), "em_")
}
func (x *elfExe) ReadData(addr, size uint64) ([]byte, error) {
for _, prog := range x.f.Progs {
if prog.Vaddr <= addr && addr <= prog.Vaddr+prog.Filesz-1 {
n := prog.Vaddr + prog.Filesz - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := prog.ReadAt(data, int64(addr-prog.Vaddr))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *elfExe) DataStart() uint64 {
for _, s := range x.f.Sections {
if s.Name == ".go.buildinfo" {
return s.Addr
}
}
for _, p := range x.f.Progs {
if p.Type == elf.PT_LOAD && p.Flags&(elf.PF_X|elf.PF_W) == elf.PF_W {
return p.Vaddr
}
}
return 0
}
// peExe is the PE (Windows Portable Executable) implementation of the exe interface.
type peExe struct {
os io.ReadCloser
f *pe.File
}
func (x *peExe) Close() error {
return x.os.Close()
}
func (x *peExe) ArchName() string {
// from: debug/pe/pe.go
switch x.f.Machine {
case pe.IMAGE_FILE_MACHINE_AM33:
return "amd33"
case pe.IMAGE_FILE_MACHINE_AMD64:
return "amd64"
case pe.IMAGE_FILE_MACHINE_ARM:
return "arm"
case pe.IMAGE_FILE_MACHINE_ARMNT:
return "armnt"
case pe.IMAGE_FILE_MACHINE_ARM64:
return "arm64"
case pe.IMAGE_FILE_MACHINE_EBC:
return "ebc"
case pe.IMAGE_FILE_MACHINE_I386:
return "i386"
case pe.IMAGE_FILE_MACHINE_IA64:
return "ia64"
case pe.IMAGE_FILE_MACHINE_M32R:
return "m32r"
case pe.IMAGE_FILE_MACHINE_MIPS16:
return "mips16"
case pe.IMAGE_FILE_MACHINE_MIPSFPU:
return "mipsfpu"
case pe.IMAGE_FILE_MACHINE_MIPSFPU16:
return "mipsfpu16"
case pe.IMAGE_FILE_MACHINE_POWERPC:
return "ppc"
case pe.IMAGE_FILE_MACHINE_POWERPCFP:
return "ppcfp"
case pe.IMAGE_FILE_MACHINE_R4000:
return "r4000"
case pe.IMAGE_FILE_MACHINE_SH3:
return "sh3"
case pe.IMAGE_FILE_MACHINE_SH3DSP:
return "sh3dsp"
case pe.IMAGE_FILE_MACHINE_SH4:
return "sh4"
case pe.IMAGE_FILE_MACHINE_SH5:
return "sh5"
case pe.IMAGE_FILE_MACHINE_THUMB:
return "thumb"
case pe.IMAGE_FILE_MACHINE_WCEMIPSV2:
return "wcemipsv2"
default:
return fmt.Sprintf("unknown-pe-machine-%d", x.f.Machine)
}
}
func (x *peExe) imageBase() uint64 {
switch oh := x.f.OptionalHeader.(type) {
case *pe.OptionalHeader32:
return uint64(oh.ImageBase)
case *pe.OptionalHeader64:
return oh.ImageBase
}
return 0
}
func (x *peExe) ReadData(addr, size uint64) ([]byte, error) {
addr -= x.imageBase()
for _, sect := range x.f.Sections {
if uint64(sect.VirtualAddress) <= addr && addr <= uint64(sect.VirtualAddress+sect.Size-1) {
n := uint64(sect.VirtualAddress+sect.Size) - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := sect.ReadAt(data, int64(addr-uint64(sect.VirtualAddress)))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *peExe) DataStart() uint64 {
// Assume data is first writable section.
const (
IMAGE_SCN_CNT_CODE = 0x00000020
IMAGE_SCN_CNT_INITIALIZED_DATA = 0x00000040
IMAGE_SCN_CNT_UNINITIALIZED_DATA = 0x00000080
IMAGE_SCN_MEM_EXECUTE = 0x20000000
IMAGE_SCN_MEM_READ = 0x40000000
IMAGE_SCN_MEM_WRITE = 0x80000000
IMAGE_SCN_MEM_DISCARDABLE = 0x2000000
IMAGE_SCN_LNK_NRELOC_OVFL = 0x1000000
IMAGE_SCN_ALIGN_32BYTES = 0x600000
)
for _, sect := range x.f.Sections {
if sect.VirtualAddress != 0 && sect.Size != 0 &&
sect.Characteristics&^IMAGE_SCN_ALIGN_32BYTES == IMAGE_SCN_CNT_INITIALIZED_DATA|IMAGE_SCN_MEM_READ|IMAGE_SCN_MEM_WRITE {
return uint64(sect.VirtualAddress) + x.imageBase()
}
}
return 0
}
// machoExe is the Mach-O (Apple macOS/iOS) implementation of the exe interface.
type machoExe struct {
os io.ReadCloser
f *macho.File
}
func (x *machoExe) Close() error {
return x.os.Close()
}
func (x *machoExe) ArchName() string {
return cleanMachoArch(x.f.Cpu)
}
func cleanMachoArch(cpu macho.Cpu) string {
return strings.TrimPrefix(strings.ToLower(cpu.String()), "cpu")
}
func (x *machoExe) ReadData(addr, size uint64) ([]byte, error) {
for _, load := range x.f.Loads {
seg, ok := load.(*macho.Segment)
if !ok {
continue
}
if seg.Addr <= addr && addr <= seg.Addr+seg.Filesz-1 {
if seg.Name == "__PAGEZERO" {
continue
}
n := seg.Addr + seg.Filesz - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := seg.ReadAt(data, int64(addr-seg.Addr))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *machoExe) DataStart() uint64 {
// Look for section named "__go_buildinfo".
for _, sec := range x.f.Sections {
if sec.Name == "__go_buildinfo" {
return sec.Addr
}
}
// Try the first non-empty writable segment.
const RW = 3
for _, load := range x.f.Loads {
seg, ok := load.(*macho.Segment)
if ok && seg.Addr != 0 && seg.Filesz != 0 && seg.Prot == RW && seg.Maxprot == RW {
return seg.Addr
}
}
return 0
}
/*
// xcoffExe is the XCOFF (AIX eXtended COFF) implementation of the exe interface.
type xcoffExe struct {
os *os.File
f *xcoff.File
}
func (x *xcoffExe) Close() error {
return x.os.Close()
}
func (x *xcoffExe) ReadData(addr, size uint64) ([]byte, error) {
for _, sect := range x.f.Sections {
if uint64(sect.VirtualAddress) <= addr && addr <= uint64(sect.VirtualAddress+sect.Size-1) {
n := uint64(sect.VirtualAddress+sect.Size) - addr
if n > size {
n = size
}
data := make([]byte, n)
_, err := sect.ReadAt(data, int64(addr-uint64(sect.VirtualAddress)))
if err != nil {
return nil, err
}
return data, nil
}
}
return nil, fmt.Errorf("address not mapped")
}
func (x *xcoffExe) DataStart() uint64 {
return x.f.SectionByType(xcoff.STYP_DATA).VirtualAddress
}
*/

43
syft/pkg/cataloger/golang/exe_test.go
View file

@ -1,43 +0,0 @@
package golang
import (
"debug/elf"
"debug/macho"
"testing"
"github.com/stretchr/testify/assert"
)
func Test_cleanElfArch(t *testing.T) {
tests := []struct {
machine elf.Machine
want string
}{
{
machine: elf.EM_X86_64,
want: "x86_64",
},
}
for _, test := range tests {
t.Run(test.machine.String(), func(t *testing.T) {
assert.Equalf(t, test.want, cleanElfArch(test.machine), "cleanElfArch(%v)", test.machine)
})
}
}
func Test_cleanMachoArch(t *testing.T) {
tests := []struct {
cpu macho.Cpu
want string
}{
{
cpu: macho.CpuAmd64,
want: "amd64",
},
}
for _, test := range tests {
t.Run(test.cpu.String(), func(t *testing.T) {
assert.Equalf(t, test.want, cleanMachoArch(test.cpu), "cleanMachoArch(%v)", test.cpu)
})
}
}

4
syft/pkg/cataloger/golang/internal/README.md Normal file
View file

@ -0,0 +1,4 @@
xcoff
-----
The code in this package comes from: https://github.com/golang/go/tree/master/src/internal/xcoff -- it was copied over to add support for [xcoff](https://en.wikipedia.org/wiki/XCOFF) binaries. Golang keeps this package as internal, forbidding its external use.

688
syft/pkg/cataloger/golang/internal/xcoff/file.go Normal file
View file

@ -0,0 +1,688 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package xcoff implements access to XCOFF (Extended Common Object File Format) files.
//nolint //this is an internal golang lib
package xcoff
import (
"encoding/binary"
"fmt"
"io"
"os"
"strings"
)
// SectionHeader holds information about an XCOFF section header.
type SectionHeader struct {
Name string
VirtualAddress uint64
Size uint64
Type uint32
Relptr uint64
Nreloc uint32
}
type Section struct {
SectionHeader
Relocs []Reloc
io.ReaderAt
sr *io.SectionReader
}
// AuxiliaryCSect holds information about an XCOFF symbol in an AUX_CSECT entry.
type AuxiliaryCSect struct {
Length int64
StorageMappingClass int
SymbolType int
}
// AuxiliaryFcn holds information about an XCOFF symbol in an AUX_FCN entry.
type AuxiliaryFcn struct {
Size int64
}
type Symbol struct {
Name string
Value uint64
SectionNumber int
StorageClass int
AuxFcn AuxiliaryFcn
AuxCSect AuxiliaryCSect
}
type Reloc struct {
VirtualAddress uint64
Symbol *Symbol
Signed bool
InstructionFixed bool
Length uint8
Type uint8
}
// ImportedSymbol holds information about an imported XCOFF symbol.
type ImportedSymbol struct {
Name string
Library string
}
// FileHeader holds information about an XCOFF file header.
type FileHeader struct {
TargetMachine uint16
}
// A File represents an open XCOFF file.
type File struct {
FileHeader
Sections []*Section
Symbols []*Symbol
StringTable []byte
LibraryPaths []string
closer io.Closer
}
// Open opens the named file using os.Open and prepares it for use as an XCOFF binary.
func Open(name string) (*File, error) {
f, err := os.Open(name)
if err != nil {
return nil, err
}
ff, err := NewFile(f)
if err != nil {
f.Close()
return nil, err
}
ff.closer = f
return ff, nil
}
// Close closes the File.
// If the File was created using NewFile directly instead of Open,
// Close has no effect.
func (f *File) Close() error {
var err error
if f.closer != nil {
err = f.closer.Close()
f.closer = nil
}
return err
}
// Section returns the first section with the given name, or nil if no such
// section exists.
// Xcoff have section's name limited to 8 bytes. Some sections like .gosymtab
// can be trunked but this method will still find them.
func (f *File) Section(name string) *Section {
for _, s := range f.Sections {
if s.Name == name || (len(name) > 8 && s.Name == name[:8]) {
return s
}
}
return nil
}
// SectionByType returns the first section in f with the
// given type, or nil if there is no such section.
func (f *File) SectionByType(typ uint32) *Section {
for _, s := range f.Sections {
if s.Type == typ {
return s
}
}
return nil
}
// cstring converts ASCII byte sequence b to string.
// It stops once it finds 0 or reaches end of b.
func cstring(b []byte) string {
var i int
for i = 0; i < len(b) && b[i] != 0; i++ {
}
return string(b[:i])
}
// getString extracts a string from an XCOFF string table.
func getString(st []byte, offset uint32) (string, bool) {
if offset < 4 || int(offset) >= len(st) {
return "", false
}
return cstring(st[offset:]), true
}
// NewFile creates a new File for accessing an XCOFF binary in an underlying reader.
func NewFile(r io.ReaderAt) (*File, error) {
sr := io.NewSectionReader(r, 0, 1<<63-1)
// Read XCOFF target machine
var magic uint16
if err := binary.Read(sr, binary.BigEndian, &magic); err != nil {
return nil, err
}
if magic != U802TOCMAGIC && magic != U64_TOCMAGIC {
return nil, fmt.Errorf("unrecognised XCOFF magic: 0x%x", magic)
}
f := new(File)
f.TargetMachine = magic
// Read XCOFF file header
if _, err := sr.Seek(0, os.SEEK_SET); err != nil {
return nil, err
}
var nscns uint16
var symptr uint64
var nsyms int32
var opthdr uint16
var hdrsz int
switch f.TargetMachine {
case U802TOCMAGIC:
fhdr := new(FileHeader32)
if err := binary.Read(sr, binary.BigEndian, fhdr); err != nil {
return nil, err
}
nscns = fhdr.Fnscns
symptr = uint64(fhdr.Fsymptr)
nsyms = fhdr.Fnsyms
opthdr = fhdr.Fopthdr
hdrsz = FILHSZ_32
case U64_TOCMAGIC:
fhdr := new(FileHeader64)
if err := binary.Read(sr, binary.BigEndian, fhdr); err != nil {
return nil, err
}
nscns = fhdr.Fnscns
symptr = fhdr.Fsymptr
nsyms = fhdr.Fnsyms
opthdr = fhdr.Fopthdr
hdrsz = FILHSZ_64
}
if symptr == 0 || nsyms <= 0 {
return nil, fmt.Errorf("no symbol table")
}
// Read string table (located right after symbol table).
offset := symptr + uint64(nsyms)*SYMESZ
if _, err := sr.Seek(int64(offset), os.SEEK_SET); err != nil {
return nil, err
}
// The first 4 bytes contain the length (in bytes).
var l uint32
if err := binary.Read(sr, binary.BigEndian, &l); err != nil {
return nil, err
}
if l > 4 {
if _, err := sr.Seek(int64(offset), os.SEEK_SET); err != nil {
return nil, err
}
f.StringTable = make([]byte, l)
if _, err := io.ReadFull(sr, f.StringTable); err != nil {
return nil, err
}
}
// Read section headers
if _, err := sr.Seek(int64(hdrsz)+int64(opthdr), os.SEEK_SET); err != nil {
return nil, err
}
f.Sections = make([]*Section, nscns)
for i := 0; i < int(nscns); i++ {
var scnptr uint64
s := new(Section)
switch f.TargetMachine {
case U802TOCMAGIC:
shdr := new(SectionHeader32)
if err := binary.Read(sr, binary.BigEndian, shdr); err != nil {
return nil, err
}
s.Name = cstring(shdr.Sname[:])
s.VirtualAddress = uint64(shdr.Svaddr)
s.Size = uint64(shdr.Ssize)
scnptr = uint64(shdr.Sscnptr)
s.Type = shdr.Sflags
s.Relptr = uint64(shdr.Srelptr)
s.Nreloc = uint32(shdr.Snreloc)
case U64_TOCMAGIC:
shdr := new(SectionHeader64)
if err := binary.Read(sr, binary.BigEndian, shdr); err != nil {
return nil, err
}
s.Name = cstring(shdr.Sname[:])
s.VirtualAddress = shdr.Svaddr
s.Size = shdr.Ssize
scnptr = shdr.Sscnptr
s.Type = shdr.Sflags
s.Relptr = shdr.Srelptr
s.Nreloc = shdr.Snreloc
}
r2 := r
if scnptr == 0 { // .bss must have all 0s
r2 = zeroReaderAt{}
}
s.sr = io.NewSectionReader(r2, int64(scnptr), int64(s.Size))
s.ReaderAt = s.sr
f.Sections[i] = s
}
// Symbol map needed by relocation
var idxToSym = make(map[int]*Symbol)
// Read symbol table
if _, err := sr.Seek(int64(symptr), os.SEEK_SET); err != nil {
return nil, err
}
f.Symbols = make([]*Symbol, 0)
for i := 0; i < int(nsyms); i++ {
var numaux int
var ok, needAuxFcn bool
sym := new(Symbol)
switch f.TargetMachine {
case U802TOCMAGIC:
se := new(SymEnt32)
if err := binary.Read(sr, binary.BigEndian, se); err != nil {
return nil, err
}
numaux = int(se.Nnumaux)
sym.SectionNumber = int(se.Nscnum)
sym.StorageClass = int(se.Nsclass)
sym.Value = uint64(se.Nvalue)
needAuxFcn = se.Ntype&SYM_TYPE_FUNC != 0 && numaux > 1
zeroes := binary.BigEndian.Uint32(se.Nname[:4])
if zeroes != 0 {
sym.Name = cstring(se.Nname[:])
} else {
offset := binary.BigEndian.Uint32(se.Nname[4:])
sym.Name, ok = getString(f.StringTable, offset)
if !ok {
goto skip
}
}
case U64_TOCMAGIC:
se := new(SymEnt64)
if err := binary.Read(sr, binary.BigEndian, se); err != nil {
return nil, err
}
numaux = int(se.Nnumaux)
sym.SectionNumber = int(se.Nscnum)
sym.StorageClass = int(se.Nsclass)
sym.Value = se.Nvalue
needAuxFcn = se.Ntype&SYM_TYPE_FUNC != 0 && numaux > 1
sym.Name, ok = getString(f.StringTable, se.Noffset)
if !ok {
goto skip
}
}
if sym.StorageClass != C_EXT && sym.StorageClass != C_WEAKEXT && sym.StorageClass != C_HIDEXT {
goto skip
}
// Must have at least one csect auxiliary entry.
if numaux < 1 || i+numaux >= int(nsyms) {
goto skip
}
if sym.SectionNumber > int(nscns) {
goto skip
}
if sym.SectionNumber == 0 {
sym.Value = 0
} else {
sym.Value -= f.Sections[sym.SectionNumber-1].VirtualAddress
}
idxToSym[i] = sym
// If this symbol is a function, it must retrieve its size from
// its AUX_FCN entry.
// It can happen that a function symbol doesn't have any AUX_FCN.
// In this case, needAuxFcn is false and their size will be set to 0.
if needAuxFcn {
switch f.TargetMachine {
case U802TOCMAGIC:
aux := new(AuxFcn32)
if err := binary.Read(sr, binary.BigEndian, aux); err != nil {
return nil, err
}
sym.AuxFcn.Size = int64(aux.Xfsize)
case U64_TOCMAGIC:
aux := new(AuxFcn64)
if err := binary.Read(sr, binary.BigEndian, aux); err != nil {
return nil, err
}
sym.AuxFcn.Size = int64(aux.Xfsize)
}
}
// Read csect auxiliary entry (by convention, it is the last).
if !needAuxFcn {
if _, err := sr.Seek(int64(numaux-1)*SYMESZ, os.SEEK_CUR); err != nil {
return nil, err
}
}
i += numaux
numaux = 0
switch f.TargetMachine {
case U802TOCMAGIC:
aux := new(AuxCSect32)
if err := binary.Read(sr, binary.BigEndian, aux); err != nil {
return nil, err
}
sym.AuxCSect.SymbolType = int(aux.Xsmtyp & 0x7)
sym.AuxCSect.StorageMappingClass = int(aux.Xsmclas)
sym.AuxCSect.Length = int64(aux.Xscnlen)
case U64_TOCMAGIC:
aux := new(AuxCSect64)
if err := binary.Read(sr, binary.BigEndian, aux); err != nil {
return nil, err
}
sym.AuxCSect.SymbolType = int(aux.Xsmtyp & 0x7)
sym.AuxCSect.StorageMappingClass = int(aux.Xsmclas)
sym.AuxCSect.Length = int64(aux.Xscnlenhi)<<32 | int64(aux.Xscnlenlo)
}
f.Symbols = append(f.Symbols, sym)
skip:
i += numaux // Skip auxiliary entries
if _, err := sr.Seek(int64(numaux)*SYMESZ, os.SEEK_CUR); err != nil {
return nil, err
}
}
// Read relocations
// Only for .data or .text section
for _, sect := range f.Sections {
if sect.Type != STYP_TEXT && sect.Type != STYP_DATA {
continue
}
sect.Relocs = make([]Reloc, sect.Nreloc)
if sect.Relptr == 0 {
continue
}
if _, err := sr.Seek(int64(sect.Relptr), os.SEEK_SET); err != nil {
return nil, err
}
for i := uint32(0); i < sect.Nreloc; i++ {
switch f.TargetMachine {
case U802TOCMAGIC:
rel := new(Reloc32)
if err := binary.Read(sr, binary.BigEndian, rel); err != nil {
return nil, err
}
sect.Relocs[i].VirtualAddress = uint64(rel.Rvaddr)
sect.Relocs[i].Symbol = idxToSym[int(rel.Rsymndx)]
sect.Relocs[i].Type = rel.Rtype
sect.Relocs[i].Length = rel.Rsize&0x3F + 1
if rel.Rsize&0x80 != 0 {
sect.Relocs[i].Signed = true
}
if rel.Rsize&0x40 != 0 {
sect.Relocs[i].InstructionFixed = true
}
case U64_TOCMAGIC:
rel := new(Reloc64)
if err := binary.Read(sr, binary.BigEndian, rel); err != nil {
return nil, err
}
sect.Relocs[i].VirtualAddress = rel.Rvaddr
sect.Relocs[i].Symbol = idxToSym[int(rel.Rsymndx)]
sect.Relocs[i].Type = rel.Rtype
sect.Relocs[i].Length = rel.Rsize&0x3F + 1
if rel.Rsize&0x80 != 0 {
sect.Relocs[i].Signed = true
}
if rel.Rsize&0x40 != 0 {
sect.Relocs[i].InstructionFixed = true
}
}
}
}
return f, nil
}
// zeroReaderAt is ReaderAt that reads 0s.
type zeroReaderAt struct{}
// ReadAt writes len(p) 0s into p.
func (w zeroReaderAt) ReadAt(p []byte, off int64) (n int, err error) {
for i := range p {
p[i] = 0
}
return len(p), nil
}
// Data reads and returns the contents of the XCOFF section s.
func (s *Section) Data() ([]byte, error) {
dat := make([]byte, s.sr.Size())
n, err := s.sr.ReadAt(dat, 0)
if n == len(dat) {
err = nil
}
return dat[:n], err
}
// CSect reads and returns the contents of a csect.
// func (f *File) CSect(name string) []byte {
// for _, sym := range f.Symbols {
// if sym.Name == name && sym.AuxCSect.SymbolType == XTY_SD {
// if i := sym.SectionNumber - 1; 0 <= i && i < len(f.Sections) {
// s := f.Sections[i]
// if sym.Value+uint64(sym.AuxCSect.Length) <= s.Size {
// dat := make([]byte, sym.AuxCSect.Length)
// _, err := s.sr.ReadAt(dat, int64(sym.Value))
// if err != nil {
// return nil
// }
// return dat
// }
// }
// break
// }
// }
// return nil
// }
// func (f *File) DWARF() (*dwarf.Data, error) {
// // There are many other DWARF sections, but these
// // are the ones the debug/dwarf package uses.
// // Don't bother loading others.
// var subtypes = [...]uint32{SSUBTYP_DWABREV, SSUBTYP_DWINFO, SSUBTYP_DWLINE, SSUBTYP_DWRNGES, SSUBTYP_DWSTR}
// var dat [len(subtypes)][]byte
// for i, subtype := range subtypes {
// s := f.SectionByType(STYP_DWARF | subtype)
// if s != nil {
// b, err := s.Data()
// if err != nil && uint64(len(b)) < s.Size {
// return nil, err
// }
// dat[i] = b
// }
// }
// abbrev, info, line, ranges, str := dat[0], dat[1], dat[2], dat[3], dat[4]
// return dwarf.New(abbrev, nil, nil, info, line, nil, ranges, str)
// }
// readImportID returns the import file IDs stored inside the .loader section.
// Library name pattern is either path/base/member or base/member
func (f *File) readImportIDs(s *Section) ([]string, error) {
// Read loader header
if _, err := s.sr.Seek(0, os.SEEK_SET); err != nil {
return nil, err
}
var istlen uint32
var nimpid int32
var impoff uint64
switch f.TargetMachine {
case U802TOCMAGIC:
lhdr := new(LoaderHeader32)
if err := binary.Read(s.sr, binary.BigEndian, lhdr); err != nil {
return nil, err
}
istlen = lhdr.Listlen
nimpid = lhdr.Lnimpid
impoff = uint64(lhdr.Limpoff)
case U64_TOCMAGIC:
lhdr := new(LoaderHeader64)
if err := binary.Read(s.sr, binary.BigEndian, lhdr); err != nil {
return nil, err
}
istlen = lhdr.Listlen
nimpid = lhdr.Lnimpid
impoff = lhdr.Limpoff
}
// Read loader import file ID table
if _, err := s.sr.Seek(int64(impoff), os.SEEK_SET); err != nil {
return nil, err
}
table := make([]byte, istlen)
if _, err := io.ReadFull(s.sr, table); err != nil {
return nil, err
}
offset := 0
// First import file ID is the default LIBPATH value
libpath := cstring(table[offset:])
f.LibraryPaths = strings.Split(libpath, ":")
offset += len(libpath) + 3 // 3 null bytes
all := make([]string, 0)
for i := 1; i < int(nimpid); i++ {
impidpath := cstring(table[offset:])
offset += len(impidpath) + 1
impidbase := cstring(table[offset:])
offset += len(impidbase) + 1
impidmem := cstring(table[offset:])
offset += len(impidmem) + 1
var path string
if len(impidpath) > 0 {
path = impidpath + "/" + impidbase + "/" + impidmem
} else {
path = impidbase + "/" + impidmem
}
all = append(all, path)
}
return all, nil
}
// ImportedSymbols returns the names of all symbols
// referred to by the binary f that are expected to be
// satisfied by other libraries at dynamic load time.
// It does not return weak symbols.
func (f *File) ImportedSymbols() ([]ImportedSymbol, error) {
s := f.SectionByType(STYP_LOADER)
if s == nil {
return nil, nil
}
// Read loader header
if _, err := s.sr.Seek(0, os.SEEK_SET); err != nil {
return nil, err
}
var stlen uint32
var stoff uint64
var nsyms int32
var symoff uint64
switch f.TargetMachine {
case U802TOCMAGIC:
lhdr := new(LoaderHeader32)
if err := binary.Read(s.sr, binary.BigEndian, lhdr); err != nil {
return nil, err
}
stlen = lhdr.Lstlen
stoff = uint64(lhdr.Lstoff)
nsyms = lhdr.Lnsyms
symoff = LDHDRSZ_32
case U64_TOCMAGIC:
lhdr := new(LoaderHeader64)
if err := binary.Read(s.sr, binary.BigEndian, lhdr); err != nil {
return nil, err
}
stlen = lhdr.Lstlen
stoff = lhdr.Lstoff
nsyms = lhdr.Lnsyms
symoff = lhdr.Lsymoff
}
// Read loader section string table
if _, err := s.sr.Seek(int64(stoff), os.SEEK_SET); err != nil {
return nil, err
}
st := make([]byte, stlen)
if _, err := io.ReadFull(s.sr, st); err != nil {
return nil, err
}
// Read imported libraries
libs, err := f.readImportIDs(s)
if err != nil {
return nil, err
}
// Read loader symbol table
if _, err := s.sr.Seek(int64(symoff), os.SEEK_SET); err != nil {
return nil, err
}
all := make([]ImportedSymbol, 0)
for i := 0; i < int(nsyms); i++ {
var name string
var ifile int32
var ok bool
switch f.TargetMachine {
case U802TOCMAGIC:
ldsym := new(LoaderSymbol32)
if err := binary.Read(s.sr, binary.BigEndian, ldsym); err != nil {
return nil, err
}
if ldsym.Lsmtype&0x40 == 0 {
continue // Imported symbols only
}
zeroes := binary.BigEndian.Uint32(ldsym.Lname[:4])
if zeroes != 0 {
name = cstring(ldsym.Lname[:])
} else {
offset := binary.BigEndian.Uint32(ldsym.Lname[4:])
name, ok = getString(st, offset)
if !ok {
continue
}
}
ifile = ldsym.Lifile
case U64_TOCMAGIC:
ldsym := new(LoaderSymbol64)
if err := binary.Read(s.sr, binary.BigEndian, ldsym); err != nil {
return nil, err
}
if ldsym.Lsmtype&0x40 == 0 {
continue // Imported symbols only
}
name, ok = getString(st, ldsym.Loffset)
if !ok {
continue
}
ifile = ldsym.Lifile
}
var sym ImportedSymbol
sym.Name = name
if ifile >= 1 && int(ifile) <= len(libs) {
sym.Library = libs[ifile-1]
}
all = append(all, sym)
}
return all, nil
}
// ImportedLibraries returns the names of all libraries
// referred to by the binary f that are expected to be
// linked with the binary at dynamic link time.
func (f *File) ImportedLibraries() ([]string, error) {
s := f.SectionByType(STYP_LOADER)
if s == nil {
return nil, nil
}
all, err := f.readImportIDs(s)
return all, err
}

102
syft/pkg/cataloger/golang/internal/xcoff/file_test.go Normal file
View file

@ -0,0 +1,102 @@
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package xcoff
import (
"reflect"
"testing"
)
type fileTest struct {
file string
hdr FileHeader
sections []*SectionHeader
needed []string
}
var fileTests = []fileTest{
{
"testdata/gcc-ppc32-aix-dwarf2-exec",
FileHeader{U802TOCMAGIC},
[]*SectionHeader{
{".text", 0x10000290, 0x00000bbd, STYP_TEXT, 0x7ae6, 0x36},
{".data", 0x20000e4d, 0x00000437, STYP_DATA, 0x7d02, 0x2b},
{".bss", 0x20001284, 0x0000021c, STYP_BSS, 0, 0},
{".loader", 0x00000000, 0x000004b3, STYP_LOADER, 0, 0},
{".dwline", 0x00000000, 0x000000df, STYP_DWARF | SSUBTYP_DWLINE, 0x7eb0, 0x7},
{".dwinfo", 0x00000000, 0x00000314, STYP_DWARF | SSUBTYP_DWINFO, 0x7ef6, 0xa},
{".dwabrev", 0x00000000, 0x000000d6, STYP_DWARF | SSUBTYP_DWABREV, 0, 0},
{".dwarnge", 0x00000000, 0x00000020, STYP_DWARF | SSUBTYP_DWARNGE, 0x7f5a, 0x2},
{".dwloc", 0x00000000, 0x00000074, STYP_DWARF | SSUBTYP_DWLOC, 0, 0},
{".debug", 0x00000000, 0x00005e4f, STYP_DEBUG, 0, 0},
},
[]string{"libc.a/shr.o"},
},
{
"testdata/gcc-ppc64-aix-dwarf2-exec",
FileHeader{U64_TOCMAGIC},
[]*SectionHeader{
{".text", 0x10000480, 0x00000afd, STYP_TEXT, 0x8322, 0x34},
{".data", 0x20000f7d, 0x000002f3, STYP_DATA, 0x85fa, 0x25},
{".bss", 0x20001270, 0x00000428, STYP_BSS, 0, 0},
{".loader", 0x00000000, 0x00000535, STYP_LOADER, 0, 0},
{".dwline", 0x00000000, 0x000000b4, STYP_DWARF | SSUBTYP_DWLINE, 0x8800, 0x4},
{".dwinfo", 0x00000000, 0x0000036a, STYP_DWARF | SSUBTYP_DWINFO, 0x8838, 0x7},
{".dwabrev", 0x00000000, 0x000000b5, STYP_DWARF | SSUBTYP_DWABREV, 0, 0},
{".dwarnge", 0x00000000, 0x00000040, STYP_DWARF | SSUBTYP_DWARNGE, 0x889a, 0x2},
{".dwloc", 0x00000000, 0x00000062, STYP_DWARF | SSUBTYP_DWLOC, 0, 0},
{".debug", 0x00000000, 0x00006605, STYP_DEBUG, 0, 0},
},
[]string{"libc.a/shr_64.o"},
},
}
func TestOpen(t *testing.T) {
for i := range fileTests {
tt := &fileTests[i]
f, err := Open(tt.file)
if err != nil {
t.Error(err)
continue
}
if !reflect.DeepEqual(f.FileHeader, tt.hdr) {
t.Errorf("open %s:\n\thave %#v\n\twant %#v\n", tt.file, f.FileHeader, tt.hdr)
continue
}
for i, sh := range f.Sections {
if i >= len(tt.sections) {
break
}
have := &sh.SectionHeader
want := tt.sections[i]
if !reflect.DeepEqual(have, want) {
t.Errorf("open %s, section %d:\n\thave %#v\n\twant %#v\n", tt.file, i, have, want)
}
}
tn := len(tt.sections)
fn := len(f.Sections)
if tn != fn {
t.Errorf("open %s: len(Sections) = %d, want %d", tt.file, fn, tn)
}
tl := tt.needed
fl, err := f.ImportedLibraries()
if err != nil {
t.Error(err)
}
if !reflect.DeepEqual(tl, fl) {
t.Errorf("open %s: loader import = %v, want %v", tt.file, tl, fl)
}
}
}
func TestOpenFailure(t *testing.T) {
filename := "file.go" // not an XCOFF object file
_, err := Open(filename) // don't crash
if err == nil {
t.Errorf("open %s: succeeded unexpectedly", filename)
}
}

BIN
syft/pkg/cataloger/golang/internal/xcoff/testdata/gcc-ppc32-aix-dwarf2-exec vendored Normal file
View file

Binary file not shown.

BIN
syft/pkg/cataloger/golang/internal/xcoff/testdata/gcc-ppc64-aix-dwarf2-exec vendored Normal file
View file

Binary file not shown.

373
syft/pkg/cataloger/golang/internal/xcoff/xcoff.go Normal file
View file

@ -0,0 +1,373 @@
// The code in this package comes from:
// https://github.com/golang/go/tree/master/src/internal/xcoff
// it was copied over to add support for xcoff binaries.
// Golang keeps this package as internal, forbidding its external use.
// Copyright 2018 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
//nolint // this is an internal golang lib
package xcoff
// File Header.
type FileHeader32 struct {
Fmagic uint16 // Target machine
Fnscns uint16 // Number of sections
Ftimedat int32 // Time and date of file creation
Fsymptr uint32 // Byte offset to symbol table start
Fnsyms int32 // Number of entries in symbol table
Fopthdr uint16 // Number of bytes in optional header
Fflags uint16 // Flags
}
type FileHeader64 struct {
Fmagic uint16 // Target machine
Fnscns uint16 // Number of sections
Ftimedat int32 // Time and date of file creation
Fsymptr uint64 // Byte offset to symbol table start
Fopthdr uint16 // Number of bytes in optional header
Fflags uint16 // Flags
Fnsyms int32 // Number of entries in symbol table
}
const (
FILHSZ_32 = 20
FILHSZ_64 = 24
)
const (
U802TOCMAGIC = 0737 // AIX 32-bit XCOFF
U64_TOCMAGIC = 0767 // AIX 64-bit XCOFF
)
// Flags that describe the type of the object file.
const (
F_RELFLG = 0x0001
F_EXEC = 0x0002
F_LNNO = 0x0004
F_FDPR_PROF = 0x0010
F_FDPR_OPTI = 0x0020
F_DSA = 0x0040
F_VARPG = 0x0100
F_DYNLOAD = 0x1000
F_SHROBJ = 0x2000
F_LOADONLY = 0x4000
)
// Section Header.
type SectionHeader32 struct {
Sname [8]byte // Section name
Spaddr uint32 // Physical address
Svaddr uint32 // Virtual address
Ssize uint32 // Section size
Sscnptr uint32 // Offset in file to raw data for section
Srelptr uint32 // Offset in file to relocation entries for section
Slnnoptr uint32 // Offset in file to line number entries for section
Snreloc uint16 // Number of relocation entries
Snlnno uint16 // Number of line number entries
Sflags uint32 // Flags to define the section type
}
type SectionHeader64 struct {
Sname [8]byte // Section name
Spaddr uint64 // Physical address
Svaddr uint64 // Virtual address
Ssize uint64 // Section size
Sscnptr uint64 // Offset in file to raw data for section
Srelptr uint64 // Offset in file to relocation entries for section
Slnnoptr uint64 // Offset in file to line number entries for section
Snreloc uint32 // Number of relocation entries
Snlnno uint32 // Number of line number entries
Sflags uint32 // Flags to define the section type
Spad uint32 // Needs to be 72 bytes long
}
// Flags defining the section type.
const (
STYP_DWARF = 0x0010
STYP_TEXT = 0x0020
STYP_DATA = 0x0040
STYP_BSS = 0x0080
STYP_EXCEPT = 0x0100
STYP_INFO = 0x0200
STYP_TDATA = 0x0400
STYP_TBSS = 0x0800
STYP_LOADER = 0x1000
STYP_DEBUG = 0x2000
STYP_TYPCHK = 0x4000
STYP_OVRFLO = 0x8000
)
const (
SSUBTYP_DWINFO = 0x10000 // DWARF info section
SSUBTYP_DWLINE = 0x20000 // DWARF line-number section
SSUBTYP_DWPBNMS = 0x30000 // DWARF public names section
SSUBTYP_DWPBTYP = 0x40000 // DWARF public types section
SSUBTYP_DWARNGE = 0x50000 // DWARF aranges section
SSUBTYP_DWABREV = 0x60000 // DWARF abbreviation section
SSUBTYP_DWSTR = 0x70000 // DWARF strings section
SSUBTYP_DWRNGES = 0x80000 // DWARF ranges section
SSUBTYP_DWLOC = 0x90000 // DWARF location lists section
SSUBTYP_DWFRAME = 0xA0000 // DWARF frames section
SSUBTYP_DWMAC = 0xB0000 // DWARF macros section
)
// Symbol Table Entry.
type SymEnt32 struct {
Nname [8]byte // Symbol name
Nvalue uint32 // Symbol value
Nscnum int16 // Section number of symbol
Ntype uint16 // Basic and derived type specification
Nsclass int8 // Storage class of symbol
Nnumaux int8 // Number of auxiliary entries
}
type SymEnt64 struct {
Nvalue uint64 // Symbol value
Noffset uint32 // Offset of the name in string table or .debug section
Nscnum int16 // Section number of symbol
Ntype uint16 // Basic and derived type specification
Nsclass int8 // Storage class of symbol
Nnumaux int8 // Number of auxiliary entries
}
const SYMESZ = 18
const (
// Nscnum
N_DEBUG = -2
N_ABS = -1
N_UNDEF = 0
//Ntype
SYM_V_INTERNAL = 0x1000
SYM_V_HIDDEN = 0x2000
SYM_V_PROTECTED = 0x3000
SYM_V_EXPORTED = 0x4000
SYM_TYPE_FUNC = 0x0020 // is function
)
// Storage Class.
const (
C_NULL = 0 // Symbol table entry marked for deletion
C_EXT = 2 // External symbol
C_STAT = 3 // Static symbol
C_BLOCK = 100 // Beginning or end of inner block
C_FCN = 101 // Beginning or end of function
C_FILE = 103 // Source file name and compiler information
C_HIDEXT = 107 // Unnamed external symbol
C_BINCL = 108 // Beginning of include file
C_EINCL = 109 // End of include file
C_WEAKEXT = 111 // Weak external symbol
C_DWARF = 112 // DWARF symbol
C_GSYM = 128 // Global variable
C_LSYM = 129 // Automatic variable allocated on stack
C_PSYM = 130 // Argument to subroutine allocated on stack
C_RSYM = 131 // Register variable
C_RPSYM = 132 // Argument to function or procedure stored in register
C_STSYM = 133 // Statically allocated symbol
C_BCOMM = 135 // Beginning of common block
C_ECOML = 136 // Local member of common block
C_ECOMM = 137 // End of common block
C_DECL = 140 // Declaration of object
C_ENTRY = 141 // Alternate entry
C_FUN = 142 // Function or procedure
C_BSTAT = 143 // Beginning of static block
C_ESTAT = 144 // End of static block
C_GTLS = 145 // Global thread-local variable
C_STTLS = 146 // Static thread-local variable
)
// File Auxiliary Entry
type AuxFile64 struct {
Xfname [8]byte // Name or offset inside string table
Xftype uint8 // Source file string type
Xauxtype uint8 // Type of auxiliary entry
}
// Function Auxiliary Entry
type AuxFcn32 struct {
Xexptr uint32 // File offset to exception table entry
Xfsize uint32 // Size of function in bytes
Xlnnoptr uint32 // File pointer to line number
Xendndx uint32 // Symbol table index of next entry
Xpad uint16 // Unused
}
type AuxFcn64 struct {
Xlnnoptr uint64 // File pointer to line number
Xfsize uint32 // Size of function in bytes
Xendndx uint32 // Symbol table index of next entry
Xpad uint8 // Unused
Xauxtype uint8 // Type of auxiliary entry
}
type AuxSect64 struct {
Xscnlen uint64 // section length
Xnreloc uint64 // Num RLDs
pad uint8
Xauxtype uint8 // Type of auxiliary entry
}
// csect Auxiliary Entry.
type AuxCSect32 struct {
Xscnlen int32 // Length or symbol table index
Xparmhash uint32 // Offset of parameter type-check string
Xsnhash uint16 // .typchk section number
Xsmtyp uint8 // Symbol alignment and type
Xsmclas uint8 // Storage-mapping class
Xstab uint32 // Reserved
Xsnstab uint16 // Reserved
}
type AuxCSect64 struct {
Xscnlenlo uint32 // Lower 4 bytes of length or symbol table index
Xparmhash uint32 // Offset of parameter type-check string
Xsnhash uint16 // .typchk section number
Xsmtyp uint8 // Symbol alignment and type
Xsmclas uint8 // Storage-mapping class
Xscnlenhi int32 // Upper 4 bytes of length or symbol table index
Xpad uint8 // Unused
Xauxtype uint8 // Type of auxiliary entry
}
// Auxiliary type
// const (
// _AUX_EXCEPT = 255
// _AUX_FCN = 254
// _AUX_SYM = 253
// _AUX_FILE = 252
// _AUX_CSECT = 251
// _AUX_SECT = 250
// )
// Symbol type field.
const (
XTY_ER = 0 // External reference
XTY_SD = 1 // Section definition
XTY_LD = 2 // Label definition
XTY_CM = 3 // Common csect definition
)
// Defines for File auxiliary definitions: x_ftype field of x_file
const (
XFT_FN = 0 // Source File Name
XFT_CT = 1 // Compile Time Stamp
XFT_CV = 2 // Compiler Version Number
XFT_CD = 128 // Compiler Defined Information
)
// Storage-mapping class.
const (
XMC_PR = 0 // Program code
XMC_RO = 1 // Read-only constant
XMC_DB = 2 // Debug dictionary table
XMC_TC = 3 // TOC entry
XMC_UA = 4 // Unclassified
XMC_RW = 5 // Read/Write data
XMC_GL = 6 // Global linkage
XMC_XO = 7 // Extended operation
XMC_SV = 8 // 32-bit supervisor call descriptor
XMC_BS = 9 // BSS class
XMC_DS = 10 // Function descriptor
XMC_UC = 11 // Unnamed FORTRAN common
XMC_TC0 = 15 // TOC anchor
XMC_TD = 16 // Scalar data entry in the TOC
XMC_SV64 = 17 // 64-bit supervisor call descriptor
XMC_SV3264 = 18 // Supervisor call descriptor for both 32-bit and 64-bit
XMC_TL = 20 // Read/Write thread-local data
XMC_UL = 21 // Read/Write thread-local data (.tbss)
XMC_TE = 22 // TOC entry
)
// Loader Header.
type LoaderHeader32 struct {
Lversion int32 // Loader section version number
Lnsyms int32 // Number of symbol table entries
Lnreloc int32 // Number of relocation table entries
Listlen uint32 // Length of import file ID string table
Lnimpid int32 // Number of import file IDs
Limpoff uint32 // Offset to start of import file IDs
Lstlen uint32 // Length of string table
Lstoff uint32 // Offset to start of string table
}
type LoaderHeader64 struct {
Lversion int32 // Loader section version number
Lnsyms int32 // Number of symbol table entries
Lnreloc int32 // Number of relocation table entries
Listlen uint32 // Length of import file ID string table
Lnimpid int32 // Number of import file IDs
Lstlen uint32 // Length of string table
Limpoff uint64 // Offset to start of import file IDs
Lstoff uint64 // Offset to start of string table
Lsymoff uint64 // Offset to start of symbol table
Lrldoff uint64 // Offset to start of relocation entries
}
const (
LDHDRSZ_32 = 32
LDHDRSZ_64 = 56
)
// Loader Symbol.
type LoaderSymbol32 struct {
Lname [8]byte // Symbol name or byte offset into string table
Lvalue uint32 // Address field
Lscnum int16 // Section number containing symbol
Lsmtype int8 // Symbol type, export, import flags
Lsmclas int8 // Symbol storage class
Lifile int32 // Import file ID; ordinal of import file IDs
Lparm uint32 // Parameter type-check field
}
type LoaderSymbol64 struct {
Lvalue uint64 // Address field
Loffset uint32 // Byte offset into string table of symbol name
Lscnum int16 // Section number containing symbol
Lsmtype int8 // Symbol type, export, import flags
Lsmclas int8 // Symbol storage class
Lifile int32 // Import file ID; ordinal of import file IDs
Lparm uint32 // Parameter type-check field
}
type Reloc32 struct {
Rvaddr uint32 // (virtual) address of reference
Rsymndx uint32 // Index into symbol table
Rsize uint8 // Sign and reloc bit len
Rtype uint8 // Toc relocation type
}
type Reloc64 struct {
Rvaddr uint64 // (virtual) address of reference
Rsymndx uint32 // Index into symbol table
Rsize uint8 // Sign and reloc bit len
Rtype uint8 // Toc relocation type
}
const (
R_POS = 0x00 // A(sym) Positive Relocation
R_NEG = 0x01 // -A(sym) Negative Relocation
R_REL = 0x02 // A(sym-*) Relative to self
R_TOC = 0x03 // A(sym-TOC) Relative to TOC
R_TRL = 0x12 // A(sym-TOC) TOC Relative indirect load.
R_TRLA = 0x13 // A(sym-TOC) TOC Rel load address. modifiable inst
R_GL = 0x05 // A(external TOC of sym) Global Linkage
R_TCL = 0x06 // A(local TOC of sym) Local object TOC address
R_RL = 0x0C // A(sym) Pos indirect load. modifiable instruction
R_RLA = 0x0D // A(sym) Pos Load Address. modifiable instruction
R_REF = 0x0F // AL0(sym) Non relocating ref. No garbage collect
R_BA = 0x08 // A(sym) Branch absolute. Cannot modify instruction
R_RBA = 0x18 // A(sym) Branch absolute. modifiable instruction
R_BR = 0x0A // A(sym-*) Branch rel to self. non modifiable
R_RBR = 0x1A // A(sym-*) Branch rel to self. modifiable instr
R_TLS = 0x20 // General-dynamic reference to TLS symbol
R_TLS_IE = 0x21 // Initial-exec reference to TLS symbol
R_TLS_LD = 0x22 // Local-dynamic reference to TLS symbol
R_TLS_LE = 0x23 // Local-exec reference to TLS symbol
R_TLSM = 0x24 // Module reference to TLS symbol
R_TLSML = 0x25 // Module reference to local (own) module
R_TOCU = 0x30 // Relative to TOC - high order bits
R_TOCL = 0x31 // Relative to TOC - low order bits
)

199
syft/pkg/cataloger/golang/parse_go_bin.go
View file

@ -1,26 +1,53 @@
//nolint
package golang package golang
import ( import (
"bufio" "bytes"
"debug/elf"
"debug/macho"
"debug/pe"
"errors"
"fmt" "fmt"
"io" "io"
"runtime/debug"
"strings" "strings"
"github.com/anchore/syft/internal/log"
"github.com/anchore/syft/syft/pkg" "github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/pkg/cataloger/golang/internal/xcoff"
"github.com/anchore/syft/syft/source" "github.com/anchore/syft/syft/source"
) )
const ( const GOARCH = "GOARCH"
packageIdentifier = "dep"
replaceIdentifier = "=>" var (
// errUnrecognizedFormat is returned when a given executable file doesn't
// appear to be in a known format, or it breaks the rules of that format,
// or when there are I/O errors reading the file.
errUnrecognizedFormat = errors.New("unrecognized file format")
) )
type exeOpener func(file io.ReadCloser) ([]exe, error) func makeGoMainPackage(mod *debug.BuildInfo, arch string, location source.Location) pkg.Package {
gbs := getBuildSettings(mod.Settings)
main := newGoBinaryPackage(&mod.Main, mod.GoVersion, arch, location, gbs)
main.Version = ""
if v, ok := gbs["vcs.revision"]; ok {
main.Version = v
}
return main
}
func newGoBinaryPackage(dep *debug.Module, goVersion, architecture string, location source.Location, buildSettings map[string]string) pkg.Package {
if dep.Replace != nil {
dep = dep.Replace
}
func newGoBinaryPackage(name, version, h1Digest, goVersion, architecture string, location source.Location) pkg.Package {
p := pkg.Package{ p := pkg.Package{
Name: name, FoundBy: catalogerName,
Version: version, Name: dep.Path,
Version: dep.Version,
Language: pkg.Go, Language: pkg.Go,
Type: pkg.GoModulePkg, Type: pkg.GoModulePkg,
Locations: []source.Location{ Locations: []source.Location{
@ -29,8 +56,9 @@ func newGoBinaryPackage(name, version, h1Digest, goVersion, architecture string,
MetadataType: pkg.GolangBinMetadataType, MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{ Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goVersion, GoCompiledVersion: goVersion,
H1Digest: h1Digest, H1Digest: dep.Sum,
Architecture: architecture, Architecture: architecture,
BuildSettings: buildSettings,
}, },
} }
@ -39,57 +67,128 @@ func newGoBinaryPackage(name, version, h1Digest, goVersion, architecture string,
return p return p
} }
func parseGoBin(location source.Location, reader io.ReadCloser, opener exeOpener) (pkgs []pkg.Package, err error) { // getArchs finds a binary architecture by two ways:
var exes []exe // 1) reading build info from binaries compiled by go1.18+
// it has been found that there are stdlib paths within openExe that can panic. We want to prevent this behavior // 2) reading file headers from binaries compiled by < go1.18
// bubbling up and halting execution. For this reason we try to recover from any panic and return an error. func getArchs(readers []io.ReaderAt, builds []*debug.BuildInfo) []string {
defer func() { if len(readers) != len(builds) {
if r := recover(); r != nil { log.Warnf("golang cataloger: bin parsing: number of builds and readers doesn't match")
err = fmt.Errorf("recovered from panic while parse go binary at %q: %+v", location.RealPath, r) return nil
}
}()
// Identify if bin was compiled by go
exes, err = opener(reader)
if err != nil {
return pkgs, err
} }
for _, x := range exes { if len(readers) == 0 || len(builds) == 0 {
goVersion, mod := findVers(x) log.Warnf("golang cataloger: bin parsing: %d readers and %d build info items", len(readers), len(builds))
pkgs = append(pkgs, buildGoPkgInfo(location, mod, goVersion, x.ArchName())...) return nil
} }
return pkgs, err
}
func buildGoPkgInfo(location source.Location, mod, goVersion, arch string) []pkg.Package { archs := make([]string, len(builds))
pkgsSlice := make([]pkg.Package, 0) for i, build := range builds {
scanner := bufio.NewScanner(strings.NewReader(mod)) archs[i] = getGOARCH(build.Settings)
}
// filter mod dependencies: [dep, name, version, sha] // if architecture was found via build settings return
for scanner.Scan() { if archs[0] != "" {
fields := strings.Fields(scanner.Text()) return archs
}
// must have dep, name, version for i, r := range readers {
if len(fields) < 3 { a, err := getGOARCHFromBin(r)
if err != nil {
log.Warnf("golang cataloger: bin parsing: getting arch from binary: %v", err)
continue continue
} }
name := fields[1] archs[i] = a
version := fields[2] }
h1Digest := "" return archs
// if dep is *not* vendored, it'll also have h1digest }
if len(fields) >= 4 {
h1Digest = fields[3]
}
if fields[0] == packageIdentifier { func getGOARCH(settings []debug.BuildSetting) string {
pkgsSlice = append(pkgsSlice, newGoBinaryPackage(name, version, h1Digest, goVersion, arch, location)) for _, s := range settings {
} if s.Key == GOARCH {
if fields[0] == replaceIdentifier { return s.Value
// replace the previous entry in the package slice
pkgsSlice[len(pkgsSlice)-1] = newGoBinaryPackage(name, version, h1Digest, goVersion, arch, location)
} }
} }
return pkgsSlice
return ""
}
func getGOARCHFromBin(r io.ReaderAt) (string, error) {
// Read the first bytes of the file to identify the format, then delegate to
// a format-specific function to load segment and section headers.
ident := make([]byte, 16)
if n, err := r.ReadAt(ident, 0); n < len(ident) || err != nil {
return "", fmt.Errorf("unrecognized file format: %w", err)
}
var arch string
switch {
case bytes.HasPrefix(ident, []byte("\x7FELF")):
f, err := elf.NewFile(r)
if err != nil {
return "", fmt.Errorf("unrecognized file format: %w", err)
}
arch = f.Machine.String()
case bytes.HasPrefix(ident, []byte("MZ")):
f, err := pe.NewFile(r)
if err != nil {
return "", fmt.Errorf("unrecognized file format: %w", err)
}
arch = fmt.Sprintf("%d", f.Machine)
case bytes.HasPrefix(ident, []byte("\xFE\xED\xFA")) || bytes.HasPrefix(ident[1:], []byte("\xFA\xED\xFE")):
f, err := macho.NewFile(r)
if err != nil {
return "", fmt.Errorf("unrecognized file format: %w", err)
}
arch = f.Cpu.String()
case bytes.HasPrefix(ident, []byte{0x01, 0xDF}) || bytes.HasPrefix(ident, []byte{0x01, 0xF7}):
f, err := xcoff.NewFile(r)
if err != nil {
return "", fmt.Errorf("unrecognized file format: %w", err)
}
arch = fmt.Sprintf("%d", f.FileHeader.TargetMachine)
default:
return "", errUnrecognizedFormat
}
arch = strings.Replace(arch, "EM_", "", 1)
arch = strings.Replace(arch, "Cpu", "", 1)
arch = strings.ToLower(arch)
return arch, nil
}
func getBuildSettings(settings []debug.BuildSetting) map[string]string {
m := make(map[string]string)
for _, s := range settings {
m[s.Key] = s.Value
}
return m
}
func buildGoPkgInfo(location source.Location, mod *debug.BuildInfo, arch string) []pkg.Package {
var pkgs []pkg.Package
if mod == nil {
return pkgs
}
for _, dep := range mod.Deps {
if dep == nil {
continue
}
pkgs = append(pkgs, newGoBinaryPackage(dep, mod.GoVersion, arch, location, nil))
}
// NOTE(jonasagx): this use happened originally while creating unit tests. It might never
// happen in the wild, but I kept it as a safeguard against empty modules.
var empty debug.Module
if mod.Main == empty {
return pkgs
}
main := makeGoMainPackage(mod, arch, location)
pkgs = append(pkgs, main)
return pkgs
} }

364
syft/pkg/cataloger/golang/parse_go_bin_test.go
View file

@ -1,38 +1,254 @@
package golang package golang
import ( import (
"bufio"
"io" "io"
"os"
"os/exec"
"path/filepath"
"runtime/debug"
"strconv"
"syscall"
"testing" "testing"
"github.com/anchore/syft/syft/pkg" "github.com/anchore/syft/syft/pkg"
"github.com/anchore/syft/syft/source" "github.com/anchore/syft/syft/source"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
) )
// make will run the default make target for the given test fixture path
func runMakeTarget(t *testing.T, fixtureName string) {
cwd, err := os.Getwd()
require.NoError(t, err)
fixtureDir := filepath.Join(cwd, "test-fixtures/", fixtureName)
t.Logf("Generating Fixture in %q", fixtureDir)
cmd := exec.Command("make")
cmd.Dir = fixtureDir
stderr, err := cmd.StderrPipe()
require.NoError(t, err)
stdout, err := cmd.StdoutPipe()
require.NoError(t, err)
err = cmd.Start()
require.NoError(t, err)
show := func(label string, reader io.ReadCloser) {
scanner := bufio.NewScanner(reader)
scanner.Split(bufio.ScanLines)
for scanner.Scan() {
t.Logf("%s: %s", label, scanner.Text())
}
}
go show("out", stdout)
go show("err", stderr)
if err := cmd.Wait(); err != nil {
if exiterr, ok := err.(*exec.ExitError); ok {
// The program has exited with an exit code != 0
// This works on both Unix and Windows. Although package
// syscall is generally platform dependent, WaitStatus is
// defined for both Unix and Windows and in both cases has
// an ExitStatus() method with the same signature.
if status, ok := exiterr.Sys().(syscall.WaitStatus); ok {
if status.ExitStatus() != 0 {
t.Fatalf("failed to generate fixture: rc=%d", status.ExitStatus())
}
}
} else {
t.Fatalf("unable to get generate fixture result: %+v", err)
}
}
}
func Test_getGOARCHFromBin(t *testing.T) {
runMakeTarget(t, "archs")
tests := []struct {
name string
filepath string
expected string
}{
{
name: "pe",
filepath: "test-fixtures/archs/binaries/hello-win-amd64",
// see: https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#machine-types
expected: strconv.Itoa(0x8664),
},
{
name: "elf-ppc64",
filepath: "test-fixtures/archs/binaries/hello-linux-ppc64le",
expected: "ppc64",
},
{
name: "mach-o-arm64",
filepath: "test-fixtures/archs/binaries/hello-mach-o-arm64",
expected: "arm64",
},
{
name: "linux-arm",
filepath: "test-fixtures/archs/binaries/hello-linux-arm",
expected: "arm",
},
{
name: "xcoff-32bit",
filepath: "internal/xcoff/testdata/gcc-ppc32-aix-dwarf2-exec",
expected: strconv.Itoa(0x1DF),
},
{
name: "xcoff-64bit",
filepath: "internal/xcoff/testdata/gcc-ppc64-aix-dwarf2-exec",
expected: strconv.Itoa(0x1F7),
},
}
for _, tt := range tests {
f, err := os.Open(tt.filepath)
require.NoError(t, err)
arch, err := getGOARCHFromBin(f)
require.NoError(t, err, "test name: %s", tt.name)
assert.Equal(t, tt.expected, arch)
}
}
func TestBuildGoPkgInfo(t *testing.T) { func TestBuildGoPkgInfo(t *testing.T) {
const ( const (
goCompiledVersion = "1.17" goCompiledVersion = "1.18"
archDetails = "amd64" archDetails = "amd64"
) )
buildSettings := map[string]string{
"GOARCH": "amd64",
"GOOS": "darwin",
"GOAMD64": "v1",
}
expectedMain := pkg.Package{
Name: "github.com/anchore/syft",
FoundBy: catalogerName,
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []source.Location{
{
Coordinates: source.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
BuildSettings: buildSettings,
},
}
tests := []struct { tests := []struct {
name string name string
mod string mod *debug.BuildInfo
arch string
expected []pkg.Package expected []pkg.Package
}{ }{
{ {
name: "buildGoPkgInfo parses a blank mod string and returns no packages", name: "buildGoPkgInfo parses a nil mod",
mod: "", mod: nil,
expected: make([]pkg.Package, 0), expected: []pkg.Package(nil),
}, },
{ {
name: "buildGoPkgInfo parses a populated mod string and returns packages but no source info", name: "buildGoPkgInfo parses a blank mod and returns no packages",
mod: `path github.com/anchore/syft mod github.com/anchore/syft (devel) mod: &debug.BuildInfo{},
dep github.com/adrg/xdg v0.2.1 h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic= expected: []pkg.Package(nil),
dep github.com/anchore/client-go v0.0.0-20210222170800-9c70f9b80bcf h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg= },
dep github.com/anchore/client-go v1.2.3`, {
name: "buildGoPkgInfo parses a mod without main module",
arch: archDetails,
mod: &debug.BuildInfo{
GoVersion: goCompiledVersion,
Settings: []debug.BuildSetting{
{Key: "GOARCH", Value: archDetails},
{Key: "GOOS", Value: "darwin"},
{Key: "GOAMD64", Value: "v1"},
},
Deps: []*debug.Module{
{
Path: "github.com/adrg/xdg",
Version: "v0.2.1",
Sum: "h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic=",
},
},
},
expected: []pkg.Package{ expected: []pkg.Package{
{ {
Name: "github.com/adrg/xdg", Name: "github.com/adrg/xdg",
FoundBy: catalogerName,
Version: "v0.2.1",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []source.Location{
{
Coordinates: source.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic=",
},
},
},
},
{
name: "buildGoPkgInfo parses a mod without packages",
arch: archDetails,
mod: &debug.BuildInfo{
GoVersion: goCompiledVersion,
Main: debug.Module{Path: "github.com/anchore/syft"},
Settings: []debug.BuildSetting{
{Key: "GOARCH", Value: archDetails},
{Key: "GOOS", Value: "darwin"},
{Key: "GOAMD64", Value: "v1"},
},
},
expected: []pkg.Package{expectedMain},
},
{
name: "buildGoPkgInfo parses a populated mod string and returns packages but no source info",
arch: archDetails,
mod: &debug.BuildInfo{
GoVersion: goCompiledVersion,
Main: debug.Module{Path: "github.com/anchore/syft"},
Settings: []debug.BuildSetting{
{Key: "GOARCH", Value: archDetails},
{Key: "GOOS", Value: "darwin"},
{Key: "GOAMD64", Value: "v1"},
},
Deps: []*debug.Module{
{
Path: "github.com/adrg/xdg",
Version: "v0.2.1",
Sum: "h1:VSVdnH7cQ7V+B33qSJHTCRlNgra1607Q8PzEmnvb2Ic=",
},
{
Path: "github.com/anchore/client-go",
Version: "v0.0.0-20210222170800-9c70f9b80bcf",
Sum: "h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=",
},
},
},
expected: []pkg.Package{
{
Name: "github.com/adrg/xdg",
FoundBy: catalogerName,
Version: "v0.2.1", Version: "v0.2.1",
Language: pkg.Go, Language: pkg.Go,
Type: pkg.GoModulePkg, Type: pkg.GoModulePkg,
@ -53,6 +269,7 @@ func TestBuildGoPkgInfo(t *testing.T) {
}, },
{ {
Name: "github.com/anchore/client-go", Name: "github.com/anchore/client-go",
FoundBy: catalogerName,
Version: "v0.0.0-20210222170800-9c70f9b80bcf", Version: "v0.0.0-20210222170800-9c70f9b80bcf",
Language: pkg.Go, Language: pkg.Go,
Type: pkg.GoModulePkg, Type: pkg.GoModulePkg,
@ -71,59 +288,42 @@ func TestBuildGoPkgInfo(t *testing.T) {
H1Digest: "h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=", H1Digest: "h1:DYssiUV1pBmKqzKsm4mqXx8artqC0Q8HgZsVI3lMsAg=",
}, },
}, },
{ expectedMain,
Name: "github.com/anchore/client-go",
Version: "v1.2.3",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []source.Location{
{
Coordinates: source.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
},
},
}, },
}, },
{ {
name: "buildGoPkgInfo parses a populated mod string and returns packages when a replace directive exists", name: "buildGoPkgInfo parses a populated mod string and returns packages when a replace directive exists",
mod: `path github.com/anchore/test arch: archDetails,
mod github.com/anchore/test (devel) mod: &debug.BuildInfo{
dep golang.org/x/net v0.0.0-20211006190231-62292e806868 h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k= GoVersion: goCompiledVersion,
dep golang.org/x/sys v0.0.0-20211006194710-c8a6f5223071 h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE= Main: debug.Module{Path: "github.com/anchore/syft"},
dep golang.org/x/term v0.0.0-20210927222741-03fcf44c2211 Settings: []debug.BuildSetting{
=> golang.org/x/term v0.0.0-20210916214954-140adaaadfaf h1:Ihq/mm/suC88gF8WFcVwk+OV6Tq+wyA1O0E5UEvDglI= {Key: "GOARCH", Value: archDetails},
dep github.com/anchore/client-go v1.2.3`, {Key: "GOOS", Value: "darwin"},
expected: []pkg.Package{ {Key: "GOAMD64", Value: "v1"},
{ },
Name: "golang.org/x/net", Deps: []*debug.Module{
Version: "v0.0.0-20211006190231-62292e806868", {
Language: pkg.Go, Path: "golang.org/x/sys",
Type: pkg.GoModulePkg, Version: "v0.0.0-20211006194710-c8a6f5223071",
Locations: []source.Location{ Sum: "h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE=",
{ },
Coordinates: source.Coordinates{ {
RealPath: "/a-path", Path: "golang.org/x/term",
FileSystemID: "layer-id", Version: "v0.0.0-20210927222741-03fcf44c2211",
}, Sum: "h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE=",
Replace: &debug.Module{
Path: "golang.org/x/term",
Version: "v0.0.0-20210916214954-140adaaadfaf",
Sum: "h1:Ihq/mm/suC88gF8WFcVwk+OV6Tq+wyA1O0E5UEvDglI=",
}, },
}, },
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
H1Digest: "h1:KlOXYy8wQWTUJYFgkUI40Lzr06ofg5IRXUK5C7qZt1k=",
},
}, },
},
expected: []pkg.Package{
{ {
Name: "golang.org/x/sys", Name: "golang.org/x/sys",
FoundBy: catalogerName,
Version: "v0.0.0-20211006194710-c8a6f5223071", Version: "v0.0.0-20211006194710-c8a6f5223071",
Language: pkg.Go, Language: pkg.Go,
Type: pkg.GoModulePkg, Type: pkg.GoModulePkg,
@ -132,18 +332,15 @@ func TestBuildGoPkgInfo(t *testing.T) {
Coordinates: source.Coordinates{ Coordinates: source.Coordinates{
RealPath: "/a-path", RealPath: "/a-path",
FileSystemID: "layer-id", FileSystemID: "layer-id",
}, }}},
},
},
MetadataType: pkg.GolangBinMetadataType, MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{ Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion, GoCompiledVersion: goCompiledVersion,
Architecture: archDetails, Architecture: archDetails,
H1Digest: "h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE=", H1Digest: "h1:PjhxBct4MZii8FFR8+oeS7QOvxKOTZXgk63EU2XpfJE="}},
},
},
{ {
Name: "golang.org/x/term", Name: "golang.org/x/term",
FoundBy: catalogerName,
Version: "v0.0.0-20210916214954-140adaaadfaf", Version: "v0.0.0-20210916214954-140adaaadfaf",
Language: pkg.Go, Language: pkg.Go,
Type: pkg.GoModulePkg, Type: pkg.GoModulePkg,
@ -159,28 +356,9 @@ func TestBuildGoPkgInfo(t *testing.T) {
Metadata: pkg.GolangBinMetadata{ Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion, GoCompiledVersion: goCompiledVersion,
Architecture: archDetails, Architecture: archDetails,
H1Digest: "h1:Ihq/mm/suC88gF8WFcVwk+OV6Tq+wyA1O0E5UEvDglI=", H1Digest: "h1:Ihq/mm/suC88gF8WFcVwk+OV6Tq+wyA1O0E5UEvDglI="},
},
},
{
Name: "github.com/anchore/client-go",
Version: "v1.2.3",
Language: pkg.Go,
Type: pkg.GoModulePkg,
Locations: []source.Location{
{
Coordinates: source.Coordinates{
RealPath: "/a-path",
FileSystemID: "layer-id",
},
},
},
MetadataType: pkg.GolangBinMetadataType,
Metadata: pkg.GolangBinMetadata{
GoCompiledVersion: goCompiledVersion,
Architecture: archDetails,
},
}, },
expectedMain,
}, },
}, },
} }
@ -197,30 +375,8 @@ func TestBuildGoPkgInfo(t *testing.T) {
FileSystemID: "layer-id", FileSystemID: "layer-id",
}, },
} }
pkgs := buildGoPkgInfo(location, test.mod, goCompiledVersion, archDetails) pkgs := buildGoPkgInfo(location, test.mod, test.arch)
assert.Equal(t, test.expected, pkgs) assert.Equal(t, test.expected, pkgs)
}) })
} }
} }
func Test_parseGoBin_recoversFromPanic(t *testing.T) {
freakOut := func(file io.ReadCloser) ([]exe, error) {
panic("baaahhh!")
}
tests := []struct {
name string
wantPkgs []pkg.Package
wantErr assert.ErrorAssertionFunc
}{
{
name: "recovers from panic",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
pkgs, err := parseGoBin(source.NewLocation("some/path"), nil, freakOut)
assert.Error(t, err)
assert.Nil(t, pkgs)
})
}
}

65
syft/pkg/cataloger/golang/scan_bin.go Normal file
View file

@ -0,0 +1,65 @@
//nolint
package golang
import (
"debug/buildinfo"
"io"
"runtime/debug"
macho "github.com/anchore/go-macholibre"
"github.com/anchore/syft/internal/log"
)
// unionReader is a single interface with all reading functions used by golang bin
// cataloger.
type unionReader interface {
io.Reader
io.ReaderAt
io.Seeker
io.Closer
}
// scanFile scans file to try to report the Go and module versions.
func scanFile(reader unionReader, filename string) ([]*debug.BuildInfo, []string) {
// NOTE: multiple readers are returned to cover universal binaries, which are files
// with more than one binary
readers, err := getReaders(reader)
if err != nil {
log.Warnf("golang cataloger: opening binary: %v", err)
return nil, nil
}
var builds []*debug.BuildInfo
for _, r := range readers {
bi, err := buildinfo.Read(r)
if err != nil {
log.Warnf("golang cataloger: scanning file %s: %v", filename, err)
return nil, nil
}
builds = append(builds, bi)
}
archs := getArchs(readers, builds)
return builds, archs
}
// getReaders extracts one or more io.ReaderAt objects representing binaries that can be processed (multiple binaries in the case for multi-architecture binaries).
func getReaders(f unionReader) ([]io.ReaderAt, error) {
if macho.IsUniversalMachoBinary(f) {
machoReaders, err := macho.ExtractReaders(f)
if err != nil {
log.Debugf("extracting readers: %v", err)
return nil, err
}
var readers []io.ReaderAt
for _, e := range machoReaders {
readers = append(readers, e.Reader)
}
return readers, nil
}
return []io.ReaderAt{f}, nil
}

1
syft/pkg/cataloger/golang/test-fixtures/archs/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
binaries/

29
syft/pkg/cataloger/golang/test-fixtures/archs/Makefile Normal file
View file

@ -0,0 +1,29 @@
DESTINATION=binaries
all: $(DESTINATION)/hello-mach-o-arm64 $(DESTINATION)/hello-linux-arm $(DESTINATION)/hello-linux-ppc64le $(DESTINATION)/hello-win-amd64
$(DESTINATION)/hello-mach-o-arm64:
mkdir -p $(DESTINATION)
GOARCH=arm64 GOOS=darwin ./src/build.sh $(DESTINATION)/hello-mach-o-arm64
$(DESTINATION)/hello-linux-arm:
mkdir -p $(DESTINATION)
GOARCH=arm GOOS=linux ./src/build.sh $(DESTINATION)/hello-linux-arm
$(DESTINATION)/hello-linux-ppc64le:
mkdir -p $(DESTINATION)
GOARCH=ppc64le GOOS=linux ./src/build.sh $(DESTINATION)/hello-linux-ppc64le
$(DESTINATION)/hello-win-amd64:
mkdir -p $(DESTINATION)
GOARCH=amd64 GOOS=windows ./src/build.sh $(DESTINATION)/hello-win-amd64
# we need a way to determine if CI should bust the test cache based on the source material
$(DESTINATION).fingerprint: clean
mkdir -p $(DESTINATION)
find src -type f -exec sha256sum {} \; | sort | tee /dev/stderr | tee $(DESTINATION).fingerprint
sha256sum $(DESTINATION).fingerprint
.PHONY: clean
clean:
rm -f $(DESTINATION)/*

20
syft/pkg/cataloger/golang/test-fixtures/archs/src/build.sh Executable file
View file

@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -uxe
# note: this can be easily done in a 1-liner, however circle CI does NOT allow volume mounts from the host in docker executors (since they are on remote hosts, where the host files are inaccessible)
# note: gocache override is so we can run docker build not as root in a container without permission issues
BINARY=$1
CTRID=$(docker create -e GOOS="${GOOS}" -e GOARCH="${GOARCH}" -u "$(id -u):$(id -g)" -e GOCACHE=/tmp -w /src golang:1.17 go build -o main main.go)
function cleanup() {
docker rm "${CTRID}"
}
trap cleanup EXIT
set +e
# note: pwd = parent directory (archs)
docker cp "$(pwd)/src" "${CTRID}:/"
docker start -a "${CTRID}"
docker cp "${CTRID}:/src/main" "$BINARY"

3
syft/pkg/cataloger/golang/test-fixtures/archs/src/go.mod Normal file
View file

@ -0,0 +1,3 @@
module arch/v1
go 1.17

5
syft/pkg/cataloger/golang/test-fixtures/archs/src/main.go Normal file
View file

@ -0,0 +1,5 @@
package main
func main() {
println("свобода!")
}

226
syft/pkg/cataloger/golang/version.go
View file

@ -1,226 +0,0 @@
// This code was copied from the Go std library.
// https://github.com/golang/go/blob/master/src/cmd/go/internal/version/version.go
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package version implements the ``go version'' command.
package golang
import (
"bytes"
"encoding/binary"
// "cmd/go/internal/base"
)
/*
var CmdVersion = &base.Command{
UsageLine: "go version [-m] [-v] [file ...]",
Short: "print Go version",
Long: `Version prints the build information for Go executables.
Go version reports the Go version used to build each of the named
executable files.
If no files are named on the command line, go version prints its own
version information.
If a directory is named, go version walks that directory, recursively,
looking for recognized Go binaries and reporting their versions.
By default, go version does not report unrecognized files found
during a directory scan. The -v flag causes it to report unrecognized files.
The -m flag causes go version to print each executable's embedded
module version information, when available. In the output, the module
information consists of multiple lines following the version line, each
indented by a leading tab character.
See also: go doc runtime/debug.BuildInfo.
`,
}
func init() {
CmdVersion.Run = runVersion // break init cycle
}
var (
versionM = CmdVersion.Flag.Bool("m", false, "")
versionV = CmdVersion.Flag.Bool("v", false, "")
)
func runVersion(ctx context.Context, cmd *base.Command, args []string) {
if len(args) == 0 {
// If any of this command's flags were passed explicitly, error
// out, because they only make sense with arguments.
//
// Don't error if the flags came from GOFLAGS, since that can be
// a reasonable use case. For example, imagine GOFLAGS=-v to
// turn "verbose mode" on for all Go commands, which should not
// break "go version".
var argOnlyFlag string
if !base.InGOFLAGS("-m") && *versionM {
argOnlyFlag = "-m"
} else if !base.InGOFLAGS("-v") && *versionV {
argOnlyFlag = "-v"
}
if argOnlyFlag != "" {
fmt.Fprintf(os.Stderr, "go: 'go version' only accepts %s flag with arguments\n", argOnlyFlag)
base.SetExitStatus(2)
return
}
fmt.Printf("go version %s %s/%s\n", runtime.Version(), runtime.GOOS, runtime.GOARCH)
return
}
for _, arg := range args {
info, err := os.Stat(arg)
if err != nil {
fmt.Fprintf(os.Stderr, "%v\n", err)
base.SetExitStatus(1)
continue
}
if info.IsDir() {
scanDir(arg)
} else {
scanFile(arg, info, true)
}
}
}
// scanDir scans a directory for executables to run scanFile on.
func scanDir(dir string) {
filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
if d.Type().IsRegular() || d.Type()&fs.ModeSymlink != 0 {
info, err := d.Info()
if err != nil {
if *versionV {
fmt.Fprintf(os.Stderr, "%s: %v\n", path, err)
}
return nil
}
scanFile(path, info, *versionV)
}
return nil
})
}
// isExe reports whether the file should be considered executable.
func isExe(file string, info fs.FileInfo) bool {
if runtime.GOOS == "windows" {
return strings.HasSuffix(strings.ToLower(file), ".exe")
}
return info.Mode().IsRegular() && info.Mode()&0111 != 0
}
// scanFile scans file to try to report the Go and module versions.
// If mustPrint is true, scanFile will report any error reading file.
// Otherwise (mustPrint is false, because scanFile is being called
// by scanDir) scanFile prints nothing for non-Go executables.
func scanFile(file string, info fs.FileInfo, mustPrint bool) {
if info.Mode()&fs.ModeSymlink != 0 {
// Accept file symlinks only.
i, err := os.Stat(file)
if err != nil || !i.Mode().IsRegular() {
if mustPrint {
fmt.Fprintf(os.Stderr, "%s: symlink\n", file)
}
return
}
info = i
}
if !isExe(file, info) {
if mustPrint {
fmt.Fprintf(os.Stderr, "%s: not executable file\n", file)
}
return
}
x, err := openExe(file)
if err != nil {
if mustPrint {
fmt.Fprintf(os.Stderr, "%s: %v\n", file, err)
}
return
}
defer x.Close()
vers, mod := findVers(x)
if vers == "" {
if mustPrint {
fmt.Fprintf(os.Stderr, "%s: go version not found\n", file)
}
return
}
fmt.Printf("%s: %s\n", file, vers)
if *versionM && mod != "" {
fmt.Printf("\t%s\n", strings.ReplaceAll(mod[:len(mod)-1], "\n", "\n\t"))
}
}
*/
// The build info blob left by the linker is identified by
// a 16-byte header, consisting of buildInfoMagic (14 bytes),
// the binary's pointer size (1 byte),
// and whether the binary is big endian (1 byte).
var buildInfoMagic = []byte("\xff Go buildinf:")
// findVers finds and returns the Go version and module version information
// in the executable x.
func findVers(x exe) (vers, mod string) {
// Read the first 64kB of text to find the build info blob.
text := x.DataStart()
data, err := x.ReadData(text, 64*1024)
if err != nil {
return
}
for ; !bytes.HasPrefix(data, buildInfoMagic); data = data[32:] {
if len(data) < 32 {
return
}
}
// Decode the blob.
ptrSize := int(data[14])
bigEndian := data[15] != 0
var bo binary.ByteOrder
if bigEndian {
bo = binary.BigEndian
} else {
bo = binary.LittleEndian
}
var readPtr func([]byte) uint64
if ptrSize == 4 {
readPtr = func(b []byte) uint64 { return uint64(bo.Uint32(b)) }
} else {
readPtr = bo.Uint64
}
vers = readString(x, ptrSize, readPtr, readPtr(data[16:]))
if vers == "" {
return
}
mod = readString(x, ptrSize, readPtr, readPtr(data[16+ptrSize:]))
if len(mod) >= 33 && mod[len(mod)-17] == '\n' {
// Strip module framing.
mod = mod[16 : len(mod)-16]
} else {
mod = ""
}
return vers, mod
}
// readString returns the string at address addr in the executable x.
func readString(x exe, ptrSize int, readPtr func([]byte) uint64, addr uint64) string {
hdr, err := x.ReadData(addr, uint64(2*ptrSize))
if err != nil || len(hdr) < 2*ptrSize {
return ""
}
dataAddr := readPtr(hdr)
dataLen := readPtr(hdr[ptrSize:])
data, err := x.ReadData(dataAddr, dataLen)
if err != nil || uint64(len(data)) < dataLen {
return ""
}
return string(data)
}

7
syft/pkg/golang_bin_metadata.go
View file

@ -2,7 +2,8 @@ package pkg
// GolangBinMetadata represents all captured data for a Golang Binary // GolangBinMetadata represents all captured data for a Golang Binary
type GolangBinMetadata struct { type GolangBinMetadata struct {
GoCompiledVersion string `json:"goCompiledVersion" cyclonedx:"goCompiledVersion"` BuildSettings map[string]string `json:"goBuildSettings,omitempty" cyclonedx:"goBuildSettings"`
Architecture string `json:"architecture" cyclonedx:"architecture"` GoCompiledVersion string `json:"goCompiledVersion" cyclonedx:"goCompiledVersion"`
H1Digest string `json:"h1Digest" cyclonedx:"h1Digest"` Architecture string `json:"architecture" cyclonedx:"architecture"`
H1Digest string `json:"h1Digest" cyclonedx:"h1Digest"`
} }

2
syft/source/image_squash_resolver.go
View file

@ -135,7 +135,7 @@ func (r *imageSquashResolver) RelativeFileByPath(_ Location, path string) *Locat
return &paths[0] return &paths[0]
} }
// FileContentsByLocation fetches file contents for a single file reference, irregardless of the source layer. // FileContentsByLocation fetches file contents for a single file reference, regardless of the source layer.
// If the path does not exist an error is returned. // If the path does not exist an error is returned.
func (r *imageSquashResolver) FileContentsByLocation(location Location) (io.ReadCloser, error) { func (r *imageSquashResolver) FileContentsByLocation(location Location) (io.ReadCloser, error) {
entry, err := r.img.FileCatalog.Get(location.ref) entry, err := r.img.FileCatalog.Get(location.ref)

6
test/integration/regression_go_bin_scanner_arch_test.go
View file

@ -9,9 +9,9 @@ import (
func TestRegressionGoArchDiscovery(t *testing.T) { func TestRegressionGoArchDiscovery(t *testing.T) {
const ( const (
expectedELFPkg = 3 expectedELFPkg = 4
expectedWINPkg = 3 expectedWINPkg = 4
expectedMACOSPkg = 3 expectedMACOSPkg = 4
) )
// This is a regression test to make sure the way we detect go binary packages // This is a regression test to make sure the way we detect go binary packages
// stays consistent and reproducible as the tool chain evolves // stays consistent and reproducible as the tool chain evolves