Mirrors/inspec
2
0
Fork 0
mirror of https://github.com/inspec/inspec synced 2024-11-10 15:14:23 +00:00
Code Issues Projects Releases Packages Wiki Activity

Add functional tests that try each of the cloud resource packs with bad creds

Browse source 
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
This commit is contained in:
Clinton Wolfe 2020-02-26 14:41:38 -05:00
parent dc4bc8701b
commit 2d9c443a04
12 changed files with 487 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

192
test/fixtures/profiles/cloud/test-aws/README.md vendored Normal file
View file

@ -0,0 +1,192 @@
# Example InSpec Profile For AWS
This example shows the implementation of an InSpec profile for AWS.
## Create a profile
```
$ inspec init profile --platform aws my-profile
─────────────────────────── InSpec Code Generator ───────────────────────────
Creating new profile at /Users/spaterson/my-profile
• Creating directory libraries
• Creating file README.md
• Creating directory controls
• Creating file controls/example.rb
• Creating file inspec.yml
• Creating file attributes.yml
• Creating file libraries/.gitkeep
```
## Optionally update `attributes.yml` to point to your custom VPC
```
aws_vpc_id: 'custom-vpc-id'
```
The related control will simply be skipped if this is not provided. See the [InSpec DSL documentation](https://www.inspec.io/docs/reference/dsl_inspec/) for more details on conditional execution using `only_if`.
## Run the tests
### With a VPC Identifier
With a supplied VPC identifier in `attributes.yml` both of the example controls will run. The 'aws-single-vpc-exists-check' control will only check for a VPC identifier in the currently configured AWS SDK region e.g. `eu-west-2` in the below:
```
$ cd my-profile/
$ inspec exec . -t aws:// --attrs attributes.yml
Profile: AWS InSpec Profile (my-profile)
Version: 0.1.0
Target: aws://eu-west-2
✔ aws-single-vpc-exists-check: Check to see if custom VPC exists.
✔ VPC vpc-1ea06476 should exist
✔ aws-vpcs-check: Check in all the VPCs for default sg not allowing 22 inwards
✔ EC2 Security Group sg-067cd21e928c3a2f1 should allow in {:port=>22}
✔ EC2 Security Group sg-9bb3b9f3 should allow in {:port=>22}
✔ aws-vpcs-multi-region-status-check: Check AWS VPCs in all regions have status "available"
✔ VPC vpc-6458b70d in eu-north-1 should exist
✔ VPC vpc-6458b70d in eu-north-1 should be available
✔ VPC vpc-8d1390e5 in ap-south-1 should exist
✔ VPC vpc-8d1390e5 in ap-south-1 should be available
✔ VPC vpc-07a71d6e in eu-west-3 should exist
✔ VPC vpc-07a71d6e in eu-west-3 should be available
✔ VPC vpc-021630e2e767412b5 in eu-west-2 should exist
✔ VPC vpc-021630e2e767412b5 in eu-west-2 should be available
✔ VPC vpc-1ea06476 in eu-west-2 should exist
✔ VPC vpc-1ea06476 in eu-west-2 should be available
✔ VPC vpc-169dee70 in eu-west-1 should exist
✔ VPC vpc-169dee70 in eu-west-1 should be available
✔ VPC vpc-01ac7ba0be447a1c4 in eu-west-1 should exist
✔ VPC vpc-01ac7ba0be447a1c4 in eu-west-1 should be available
✔ VPC vpc-09ff83d71da9d2b6e in eu-west-1 should exist
✔ VPC vpc-09ff83d71da9d2b6e in eu-west-1 should be available
✔ VPC vpc-0ebccac2337a90f13 in eu-west-1 should exist
✔ VPC vpc-0ebccac2337a90f13 in eu-west-1 should be available
✔ VPC vpc-c2a53da4 in eu-west-1 should exist
✔ VPC vpc-c2a53da4 in eu-west-1 should be available
✔ VPC vpc-4fb3f127 in ap-northeast-2 should exist
✔ VPC vpc-4fb3f127 in ap-northeast-2 should be available
✔ VPC vpc-0804856f in ap-northeast-1 should exist
✔ VPC vpc-0804856f in ap-northeast-1 should be available
✔ VPC vpc-ccb917ab in sa-east-1 should exist
✔ VPC vpc-ccb917ab in sa-east-1 should be available
✔ VPC vpc-0afcc60c70a30a615 in ca-central-1 should exist
✔ VPC vpc-0afcc60c70a30a615 in ca-central-1 should be available
✔ VPC vpc-20a25048 in ca-central-1 should exist
✔ VPC vpc-20a25048 in ca-central-1 should be available
✔ VPC vpc-5896143f in ap-southeast-1 should exist
✔ VPC vpc-5896143f in ap-southeast-1 should be available
✔ VPC vpc-47972220 in ap-southeast-2 should exist
✔ VPC vpc-47972220 in ap-southeast-2 should be available
✔ VPC vpc-071b6f0c69d1d0311 in eu-central-1 should exist
✔ VPC vpc-071b6f0c69d1d0311 in eu-central-1 should be available
✔ VPC vpc-807dfdeb in eu-central-1 should exist
✔ VPC vpc-807dfdeb in eu-central-1 should be available
✔ VPC vpc-0be54a71311bc362d in eu-central-1 should exist
✔ VPC vpc-0be54a71311bc362d in eu-central-1 should be available
✔ VPC vpc-f060cd8b in us-east-1 should exist
✔ VPC vpc-f060cd8b in us-east-1 should be available
✔ VPC vpc-0c3a7e116c58d714b in us-east-1 should exist
✔ VPC vpc-0c3a7e116c58d714b in us-east-1 should be available
✔ VPC vpc-047bff6c in us-east-2 should exist
✔ VPC vpc-047bff6c in us-east-2 should be available
✔ VPC vpc-93dd6ef4 in us-west-1 should exist
✔ VPC vpc-93dd6ef4 in us-west-1 should be available
✔ VPC vpc-2c0a6a55 in us-west-2 should exist
✔ VPC vpc-2c0a6a55 in us-west-2 should be available
Profile: Amazon Web Services Resource Pack (inspec-aws)
Version: 0.1.0
Target: aws://eu-west-2
No tests executed.
Profile Summary: 3 successful controls, 0 control failures, 0 controls skipped
Test Summary: 53 successful, 0 failures, 0 skipped
```
### Without Supplying a VPC Identifier
If no VPC identifier is supplied, the 'aws-single-vpc-exists-check' control is skipped and the other control runs. The `attributes.yml` file does not have to be specified to InSpec in this case.
```
$ cd my-profile/
$ inspec exec . -t aws://
Profile: AWS InSpec Profile (my-profile)
Version: 0.1.0
Target: aws://eu-west-2
↺ aws-single-vpc-exists-check: Check to see if custom VPC exists.
↺ Skipped control due to only_if condition.
✔ aws-vpcs-check: Check in all the VPCs for default sg not allowing 22 inwards
✔ EC2 Security Group sg-067cd21e928c3a2f1 should allow in {:port=>22}
✔ EC2 Security Group sg-9bb3b9f3 should allow in {:port=>22}
✔ aws-vpcs-multi-region-status-check: Check AWS VPCs in all regions have status "available"
✔ VPC vpc-6458b70d in eu-north-1 should exist
✔ VPC vpc-6458b70d in eu-north-1 should be available
✔ VPC vpc-8d1390e5 in ap-south-1 should exist
✔ VPC vpc-8d1390e5 in ap-south-1 should be available
✔ VPC vpc-07a71d6e in eu-west-3 should exist
✔ VPC vpc-07a71d6e in eu-west-3 should be available
✔ VPC vpc-021630e2e767412b5 in eu-west-2 should exist
✔ VPC vpc-021630e2e767412b5 in eu-west-2 should be available
✔ VPC vpc-1ea06476 in eu-west-2 should exist
✔ VPC vpc-1ea06476 in eu-west-2 should be available
✔ VPC vpc-169dee70 in eu-west-1 should exist
✔ VPC vpc-169dee70 in eu-west-1 should be available
✔ VPC vpc-01ac7ba0be447a1c4 in eu-west-1 should exist
✔ VPC vpc-01ac7ba0be447a1c4 in eu-west-1 should be available
✔ VPC vpc-09ff83d71da9d2b6e in eu-west-1 should exist
✔ VPC vpc-09ff83d71da9d2b6e in eu-west-1 should be available
✔ VPC vpc-0ebccac2337a90f13 in eu-west-1 should exist
✔ VPC vpc-0ebccac2337a90f13 in eu-west-1 should be available
✔ VPC vpc-c2a53da4 in eu-west-1 should exist
✔ VPC vpc-c2a53da4 in eu-west-1 should be available
✔ VPC vpc-4fb3f127 in ap-northeast-2 should exist
✔ VPC vpc-4fb3f127 in ap-northeast-2 should be available
✔ VPC vpc-0804856f in ap-northeast-1 should exist
✔ VPC vpc-0804856f in ap-northeast-1 should be available
✔ VPC vpc-ccb917ab in sa-east-1 should exist
✔ VPC vpc-ccb917ab in sa-east-1 should be available
✔ VPC vpc-0afcc60c70a30a615 in ca-central-1 should exist
✔ VPC vpc-0afcc60c70a30a615 in ca-central-1 should be available
✔ VPC vpc-20a25048 in ca-central-1 should exist
✔ VPC vpc-20a25048 in ca-central-1 should be available
✔ VPC vpc-5896143f in ap-southeast-1 should exist
✔ VPC vpc-5896143f in ap-southeast-1 should be available
✔ VPC vpc-47972220 in ap-southeast-2 should exist
✔ VPC vpc-47972220 in ap-southeast-2 should be available
✔ VPC vpc-071b6f0c69d1d0311 in eu-central-1 should exist
✔ VPC vpc-071b6f0c69d1d0311 in eu-central-1 should be available
✔ VPC vpc-807dfdeb in eu-central-1 should exist
✔ VPC vpc-807dfdeb in eu-central-1 should be available
✔ VPC vpc-0be54a71311bc362d in eu-central-1 should exist
✔ VPC vpc-0be54a71311bc362d in eu-central-1 should be available
✔ VPC vpc-f060cd8b in us-east-1 should exist
✔ VPC vpc-f060cd8b in us-east-1 should be available
✔ VPC vpc-0c3a7e116c58d714b in us-east-1 should exist
✔ VPC vpc-0c3a7e116c58d714b in us-east-1 should be available
✔ VPC vpc-047bff6c in us-east-2 should exist
✔ VPC vpc-047bff6c in us-east-2 should be available
✔ VPC vpc-93dd6ef4 in us-west-1 should exist
✔ VPC vpc-93dd6ef4 in us-west-1 should be available
✔ VPC vpc-2c0a6a55 in us-west-2 should exist
✔ VPC vpc-2c0a6a55 in us-west-2 should be available
Profile: Amazon Web Services Resource Pack (inspec-aws)
Version: 0.1.0
Target: aws://eu-west-2
No tests executed.
Profile Summary: 2 successful controls, 0 control failures, 1 control skipped
Test Summary: 52 successful, 0 failures, 1 skipped
```

2
test/fixtures/profiles/cloud/test-aws/attributes.yml vendored Normal file
View file

@ -0,0 +1,2 @@
# Below is to be uncommented and set with your AWS Custom VPC ID:
# aws_vpc_id: 'vpc-xxxxxxx'

39
test/fixtures/profiles/cloud/test-aws/controls/example.rb vendored Normal file
View file

@ -0,0 +1,39 @@
# copyright: 2018, The Authors
title "Sample Section"
aws_vpc_id = attribute("aws_vpc_id", default: "", description: "Optional AWS VPC identifier.")
# You add controls here
control "aws-single-vpc-exists-check" do # A unique ID for this control.
only_if { aws_vpc_id != "" } # Only run this control if the `aws_vpc_id` attribute is provided.
impact 1.0 # The criticality, if this control fails.
title "Check to see if custom VPC exists." # A human-readable title.
describe aws_vpc(aws_vpc_id) do # The test itself.
it { should exist }
end
end
# Plural resources can be inspected to check for specific resource details.
control "aws-vpcs-check" do
impact 1.0
title "Check in all the VPCs for default sg not allowing 22 inwards"
aws_vpcs.vpc_ids.each do |vpc_id|
describe aws_security_group(vpc_id: vpc_id, group_name: "default") do
it { should allow_in(port: 22) }
end
end
end
control "aws-vpcs-multi-region-status-check" do # A unique ID for this control.
impact 1.0 # The criticality, if this control fails.
title 'Check AWS VPCs in all regions have status "available"' # A human-readable title.
aws_regions.region_names.each do |region| # Loop over all available AWS regions
aws_vpcs(aws_region: region).vpc_ids.each do |vpc| # Find all VPCs in a single AWS region
describe aws_vpc(aws_region: region, vpc_id: vpc) do # The test itself.
it { should exist } # Confirms AWS VPC exists
it { should be_available } # Confirms AWS VPC has status "available"
end
end
end
end

22
test/fixtures/profiles/cloud/test-aws/inspec.yml vendored Normal file
View file

@ -0,0 +1,22 @@
name: test-aws
title: AWS InSpec Profile
maintainer: The Authors
copyright: The Authors
copyright_email: you@example.com
license: Apache-2.0
summary: An InSpec Compliance Profile For AWS
version: 0.1.0
inspec_version: '~> 4'
attributes:
- name: aws_vpc_id
required: false
# Below is deliberately left as a default empty string to allow the profile to run when this is not provided.
# Please see the README for more details.
default: ''
description: 'Optional Custom AWS VPC Id'
type: string
depends:
- name: inspec-aws
url: https://github.com/inspec/inspec-aws/archive/master.tar.gz
supports:
- platform: aws

56
test/fixtures/profiles/cloud/test-azure/README.md vendored Normal file
View file

@ -0,0 +1,56 @@
# Example InSpec Profile For Azure
This example shows the implementation of an InSpec profile for Azure. See [https://github.com/inspec/inspec-azure](https://github.com/inspec/inspec-azure) for details on how to configure credentials for your subscription.
## Create a profile
```
$ inspec init profile --platform azure my-profile
─────────────────────────── InSpec Code Generator ───────────────────────────
Creating new profile at /Users/spaterson/my-profile
• Creating directory libraries
• Creating file README.md
• Creating directory controls
• Creating file controls/example.rb
• Creating file inspec.yml
• Creating file libraries/.gitkeep
```
## Run the tests
```
$ cd my-profile/
$ inspec exec . -t azure://
Profile: Azure InSpec Profile (my-profile)
Version: 0.1.0
Target: azure://12345abc-987d-654e-fg21-abcdef23324r
× azure-virtual-machines-exist-check: Check resource groups to see if any VMs exist. (4 failed)
× Azure Virtual Machines should exist
expected Azure Virtual Machines to exist
× Azure Virtual Machines should exist
expected Azure Virtual Machines to exist
× Azure Virtual Machines should exist
expected Azure Virtual Machines to exist
× Azure Virtual Machines should exist
expected Azure Virtual Machines to exist
✔ Azure Virtual Machines should exist
✔ Azure Virtual Machines should exist
✔ Azure Virtual Machines should exist
Profile: Azure Resource Pack (inspec-azure)
Version: 1.2.0
Target: azure://12345abc-987d-654e-fg21-abcdef23324r
No tests executed.
Profile Summary: 0 successful controls, 1 control failure, 0 controls skipped
Test Summary: 3 successful, 4 failures, 0 skipped
```

14
test/fixtures/profiles/cloud/test-azure/controls/example.rb vendored Normal file
View file

@ -0,0 +1,14 @@
# copyright: 2018, The Authors
title "Sample Section"
# you add controls here
control "azure-virtual-machines-exist-check" do # A unique ID for this control.
impact 1.0 # The criticality, if this control fails.
title "Check resource groups to see if any VMs exist." # A human-readable title
azurerm_resource_groups.names.each do |resource_group_name| # Plural resources can be leveraged to loop across many resources
describe azurerm_virtual_machines(resource_group: resource_group_name) do
it { should exist } # The test itself.
end
end
end

14
test/fixtures/profiles/cloud/test-azure/inspec.yml vendored Normal file
View file

@ -0,0 +1,14 @@
name: test-azure
title: Azure InSpec Profile
maintainer: The Authors
copyright: The Authors
copyright_email: you@example.com
license: Apache-2.0
summary: An InSpec Compliance Profile For Azure
version: 0.1.0
inspec_version: '>= 2.2.7'
depends:
- name: inspec-azure
url: https://github.com/inspec/inspec-azure/archive/master.tar.gz
supports:
- platform: azure

66
test/fixtures/profiles/cloud/test-gcp/README.md vendored Normal file
View file

@ -0,0 +1,66 @@
# Example InSpec Profile For GCP
This example shows the implementation of an InSpec profile for GCP that depends on the [InSpec GCP Resource Pack](https://github.com/inspec/inspec-gcp). See the [README](https://github.com/inspec/inspec-gcp) for instructions on setting up appropriate GCP credentials.
## Create a profile
```
$ inspec init profile --platform gcp my-profile
Create new profile at /Users/spaterson/my-profile
* Create directory libraries
* Create file README.md
* Create directory controls
* Create file controls/example.rb
* Create file inspec.yml
* Create file attributes.yml
* Create file libraries/.gitkeep
```
## Update `attributes.yml` to point to your project
```
gcp_project_id: 'my-gcp-project'
```
## Run the tests
```
$ cd gcp-profile/
$ inspec exec . -t gcp:// --attrs attributes.yml
Profile: GCP InSpec Profile (my-profile)
Version: 0.1.0
Target: gcp://local-service-account@my-gcp-project.iam.gserviceaccount.com
✔ gcp-single-region-1.0: Ensure single region has the correct properties.
✔ Region europe-west2 zone_names should include "europe-west2-a"
✔ gcp-regions-loop-1.0: Ensure regions have the correct properties in bulk.
✔ Region asia-east1 should be up
✔ Region asia-northeast1 should be up
✔ Region asia-south1 should be up
✔ Region asia-southeast1 should be up
✔ Region australia-southeast1 should be up
✔ Region europe-north1 should be up
✔ Region europe-west1 should be up
✔ Region europe-west2 should be up
✔ Region europe-west3 should be up
✔ Region europe-west4 should be up
✔ Region northamerica-northeast1 should be up
✔ Region southamerica-east1 should be up
✔ Region us-central1 should be up
✔ Region us-east1 should be up
✔ Region us-east4 should be up
✔ Region us-west1 should be up
✔ Region us-west2 should be up
Profile: Google Cloud Platform Resource Pack (inspec-gcp)
Version: 0.5.0
Target: gcp://local-service-account@my-gcp-project.iam.gserviceaccount.com
No tests executed.
Profile Summary: 2 successful controls, 0 control failures, 0 controls skipped
Test Summary: 18 successful, 0 failures, 0 skipped
```

2
test/fixtures/profiles/cloud/test-gcp/attributes.yml vendored Normal file
View file

@ -0,0 +1,2 @@
# Below is to be uncommented and set with your GCP project ID:
# gcp_project_id: 'your-gcp-project'

27
test/fixtures/profiles/cloud/test-gcp/controls/example.rb vendored Normal file
View file

@ -0,0 +1,27 @@
# copyright: 2018, The Authors
title "Sample Section"
gcp_project_id = attribute("gcp_project_id")
# you add controls here
control "gcp-single-region-1.0" do # A unique ID for this control
impact 1.0 # The criticality, if this control fails.
title "Ensure single region has the correct properties." # A human-readable title
desc "An optional description..."
describe google_compute_region(project: gcp_project_id, name: "europe-west2") do # The actual test
its("zone_names") { should include "europe-west2-a" }
end
end
# plural resources can be leveraged to loop across many resources
control "gcp-regions-loop-1.0" do # A unique ID for this control
impact 1.0 # The criticality, if this control fails.
title "Ensure regions have the correct properties in bulk." # A human-readable title
desc "An optional description..."
google_compute_regions(project: gcp_project_id).region_names.each do |region_name| # Loop across all regions by name
describe google_compute_region(project: gcp_project_id, name: region_name) do # The test for a single region
it { should be_up }
end
end
end

19
test/fixtures/profiles/cloud/test-gcp/inspec.yml vendored Normal file
View file

@ -0,0 +1,19 @@
name: test-gcp
title: GCP InSpec Profile
maintainer: The Authors
copyright: The Authors
copyright_email: you@example.com
license: Apache-2.0
summary: An InSpec Compliance Profile For GCP
version: 0.1.0
inspec_version: '>= 2.3.5'
attributes:
- name: gcp_project_id
required: true
description: 'The GCP project identifier.'
type: string
depends:
- name: inspec-gcp
url: https://github.com/inspec/inspec-gcp/archive/master.tar.gz
supports:
- platform: gcp

34
test/functional/inspec_exec_test.rb
View file

@ -909,4 +909,38 @@ Test Summary: 2 successful, 0 failures, 0 skipped\n"
end
end
end
describe "when targeting cloud resource packs" do
let(:cloud_path) { profile_path + "/cloud/" }
let(:run_result) { run_inspec_process("exec " + cloud_profile + " " + args, env: env) }
let(:env) { {} }
describe "when targeting aws" do
let(:cloud_profile) { cloud_path + "test-aws" }
# Use log level FATAL to absorb WARNs from deprecataions and ERRORs from not having credentials set.
# An actual stacktrace then will appear as sole stderr output
let(:args) { "-t aws://fakecreds --log-level fatal " }
it "should fail to connect to aws due to lack of creds but not stacktrace" do
_(run_result.stderr).must_be_empty
end
end
describe "when targeting azure" do
let(:cloud_profile) { cloud_path + "test-azure" }
let(:args) { "-t azure://" }
it "should fail to connect to azure due to lack of creds but not stacktrace" do
_(run_result.stderr).must_be_empty
end
end
describe "when targeting gcp" do
let(:cloud_profile) { cloud_path + "test-gcp" }
let(:args) { "-t gcp:// --input gcp_project_id=fakeproject" }
let(:env) { { GOOGLE_AUTH_SUPPRESS_CREDENTIALS_WARNINGS: 1 } }
it "should fail to connect to gcp due to lack of creds but not stacktrace" do
_(run_result.stderr).must_be_empty
end
end
end
end