Commit graph

74 commits

Author SHA1 Message Date
Jared Quick
0fc83871a1
Fix functional tests. (#3572)
Signed-off-by: Jared Quick <jquick@chef.io>
2018-11-02 12:52:54 -04:00
Jared Quick
d9767884ff
Allow help args after Thor commands (#3553)
* Allow help args after thor commands.
* Add more testing and clarification.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-11-01 13:04:59 -04:00
Clinton Wolfe
ff2be7be56 FilterTable: allow Strings or Symbols as fields (#3481)
* Allow strings or symbols to be used interchangeably in filtertable criteria; improve error message
* rubocop
* Update test count

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2018-10-17 09:29:58 -04:00
Jared Quick
2bbcdbde9b
Inspec 3.0 (#3512)
* Remove deprecated yumrepo. (#3435)

* Remove deprecations for cli `--format` and metadata.rb (#3452)

* Remove deprecated database_helpers stderr/stdout methods.
Update deprecation text for processes/apache.

* Remove deprecations for `--format` and metadata.rb
Remove deprecated `format` code.
Remove deprecated code test and change json-config format test to use
reporter.
Remove deprecated metadata.rb code
Remove deprecation notice for old supports syntax.
Deprecate metadata.rb from source_reader
Remove rubocop disables as they are no longer required for this code block.
Remove deprecated legacy metadata.rb mock profiles.
Remove deprecated metadata.rb profile tests.
Remove deprecated yumrepo test.

* Allow inspec-3.0 branch to be tested.
* Allow appveyor to test inspec-3.0 branch
* Change runner tests to use reporter rather than format.
Remove deprecated `supports: linux` tests.

* Remove skip from inherited profiles from showing up in reporting (breaking change) (#3332)

* Skip loading dependency profiles if they are unsupported on the current
platform.

Skip loading dependencies if they are unsupported on the current
platform.

Wrap our log and next in a conditional checking if the platform is
supported.

Change a `if !` into a `unless`

Check if the backend is a Train Mock Connection and if so say that the
profile does support the platform.

While iterating through tests being loaded skip when the platform is
unsupported.

We now log a WARN when a profile is skipped due to unsupported platform,
so lets check that.

Modified existing test to log that there are 0 skipped tests, instead of
2.

Add functional test that loads profile-support-skip with a json reporter
to check that our controls are not loaded and that stderr contains our
warning.

* Rather than iterating through each test return before recursion if the platform is
unsupported.

* Resolve tests using a supported platform different from testing platform

Add a control to `test/unit/mock/profiles/complete-profile` that would
work on any OS with a Internet connection. This allows the profile
to execute on any OS with success. `filesystem_spec.rb` was a control
that would only work on Linux and some BSD's.

We want profile tests to consistently work across development and testing
platforms, and not get 'skipped' in some cases.  Travis-CI tests on Linux,
Inspec Dev team uses Linux and MacOS, Appveyor tests on Windows

Also Updated `file_provider_test.rb` for `complete-profile` content changes.

If you `MockLoader.load_profile` on a unsupported platform you might not
hit the usual skip. Lets handle situations where the tests array in
Profile#load_checks_params could be nil.

* Use safe navigation rather than checking if tests is nil.
Update tests to point to unsupported_inspec and account for WARN changes.
Make unsupported_inspec profile support os-family 'unsupported_inspec'

* Fix skip bug when using include/require controls. (#3487)

* Fix skip bug when using include/require controls.
* fix test and feedback.

* Remove need for UUID detection for Automate report (#3507)
* Add json metadata for skipped profiles (#3495)

* Add skip metadata to json reports
* Unify skip messages.
* Update with status field.
* Add testing.
* Fix tests.
* lint
* Add skip exit codes for profile skips.
* Update website for 3.0 launch

Add `plugins` to sidebar.
Change 2.0 -> 3.0 in slim files.
Update 3.0 features list.
* Fix comments
* Update float to numeric.
* Change Float to numeric.
* updated feature list and impact doc
* Change "What's new in InSpec 3.0" -> "Announcing InSpec 3.0"
* Bump VERSION to 3.0.0 (#3511)

* Remove 3.0 testing checks.

* Fix azure link.
2018-10-15 18:25:27 -04:00
Jerry Aldrich
20776b363d Add support for multiple descriptions for controls (#3424)
* Add support for multiple descriptions for controls

This adds the ability to specify multiple descriptions in controls.

Example:

```ruby
control 'my-control' do
  impact 1.0
  title 'My control'
  desc 'A default description'
  desc 'rational', 'I need an example'
  describe file('/tmp') do
    it { should be_directory }
  end
end
```

Many thanks to @jquick for helping me with the unit tests.
* Remove unused `descriptions` method
* Remove unused profile from test mocks
* Respond to feedback

Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-09-26 13:28:57 -04:00
Jared Quick
6445e2c6d5
Add windows functional tests (#3385)
* Add windows functional tests.
* Fix tests for 2012 server.
* Fix windows build script
* Add more functional tests for windows.
* Update comment with TODO.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-09-12 18:04:16 -04:00
Christoph Hartmann
0e0b808e9b enforce utf encoding for cli output (#3376)
* enforce utf encoding for cli output
* add profile with wrong character set

Signed-off-by: Christoph Hartmann <chris@lollyrock.com>
2018-09-10 14:37:43 -04:00
Jared Quick
30e43c294d
Remove any inspec.lock file before testing vendoring. (#3377)
Signed-off-by: Jared Quick <jquick@chef.io>
2018-09-10 10:15:16 -04:00
Jerry Aldrich
9d031053ea Various improvements to vendor command (#3286)
Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-09-06 23:28:08 -04:00
Jared Quick
76b453eee9
Allow target-id passthrough (#3320)
* Allow uuid passthrough
* Update flag to be target-id.
* Updated to use proper formatting for header.
* Fix empty line after cli banner.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-09-05 15:07:34 -04:00
Jared Quick
6e59ef176b
Populate report code for merged controls (#3264)
* Populate the code section for all profiles where we merge controls.
* Fix rubocop issues.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-08-02 14:39:11 -04:00
Miah Johnson
bfd569fe99 Ensure resources fail that target something that isn't supported (#3231)
* Use fail_resource rather than skip_resource when the platform is not
supported by the resource.

* Update tests to handle failing on unsupported platforms.
Update functional tests.

Signed-off-by: Miah Johnson <miah@chia-pet.org>
2018-07-19 08:16:54 -04:00
Clinton Wolfe
92e96ebedb Accept regexes for --controls option to inspec exec (#3179)
* Functional tests for regex control selection
* Implementation for regex-based control filtering

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2018-07-05 15:44:30 -04:00
Noel Georgi
6fe13ce1eb Updating inspec with bastion options (#3180)
* Updating inspec with bastion options as per https://github.com/inspec/train/pull/310
* Updating train pin
* Adding --password to pass the test
* Revert "Updating train pin"
* PR changes

Signed-off-by: Noel Georgi <18496730+frezbo@users.noreply.github.com>
2018-07-05 15:37:04 -04:00
Clinton Wolfe
2ac5581d32 Document exit codes for 'inspec exec' and add --no-distinct-exit option (#3178)
* Add long description to inspec exec command, mentioning exit codes
* Modify website doc builder code to use long description if available
* Functional test for --distinct-exit flag
* Implement --distinct-exit option
* Inspec shell also needs the option

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2018-06-29 08:43:34 -04:00
Jared Quick
52694d4031 Add parent_profile field in json output (#3164)
Signed-off-by: Jared Quick <jquick@chef.io>
2018-06-26 13:04:31 -04:00
Jared Quick
12890408bb
Fix control merging when overriding child controls (#3155)
* Fix the control merging issues when overriding child controls.
* Fix rubocop issue and vendor compression.
* Add in lock file for vendor profile

Signed-off-by: Jared Quick <jquick@chef.io>
2018-06-21 13:37:47 -04:00
Jared Quick
06e1aa5379 Allow custom resources to access all other resources (#3108)
Signed-off-by: Jared Quick <jquick@chef.io>
2018-06-06 11:56:51 -04:00
Clinton Wolfe
1a00853cd8
Catch exceptions in control blocks and fail the control (#2987)
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2018-04-26 15:44:16 -04:00
Jerry Aldrich
4731d95abd docs: Update example resource (#2904)
* Change `skip_resource` to use raise
* Add `supports` lines to example resource
* Change to rescue `StandardError` vs `Exception`
* Change raise to use `e.message` vs `$!`
* Remove redundant returns
* Change `File.exists?` to `File.exist?`
* Update shasum in tests

Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-04-05 08:50:49 -04:00
Jared Quick
a9127d3f6c
Create reporter directory if it does not exist. (#2798)
Signed-off-by: Jared Quick <jquick@chef.io>
2018-03-08 15:45:59 -05:00
Jared Quick
db96ee9e85
Prevent resources from loading if supports check fails (#2665)
* Prevent resources from loading if supports fail.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-02-16 15:15:53 -05:00
Jared Quick
59fd0e8775
Update reporter with breaking inspec 2.0 changes. (#2487)
* Update reporter with breaking inspec 2.0 changes.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-02-14 11:54:20 -05:00
Jared Quick
9930e40a76 Add new "reporter" system (replacement for "formatters"), support multiple reporters per run (#2464)
* Formatter and reporter refactor.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add exception and backtrace to json-min report.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add sha to json-min and include generator version for json profile.

Signed-off-by: Jared Quick <jquick@chef.io>

* Fix deprecated typo and add fallback for cli resource title.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update to build json report and clean up cli logic.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add tests for json reporter.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add cli suppress_log_output? and a fallback for invalid reporter type.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update suppress_log_output? to check if we are outputting to stdout.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update reporter cli optoins to work with json_config.

Signed-off-by: Jared Quick <jquick@chef.io>

* Refactor some safe-navigation and variable names.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add thor banner to show reporter file output syntax.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-02-08 10:06:58 +01:00
Clinton Wolfe
656423d7f2 Issue warning during check if profile name contains slash (#2231)
* Add failing unit test for deprecation warning on profiles with slashes in their name

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Issue warning during validation if name contains a slash

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Slug profile names generated from target paths to prevent breaking unit tests

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Rubocop whinges

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Update functional test watching for default profile name

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Make deprecation warning more descriptive

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Rubocop whinges

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Set title with original test path if no profile name or title provided

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Rubocop whinges

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2017-11-09 12:32:54 +01:00
Kevin Formsma
94c2e8181c Add sensitive flag to resources to restrict logging output (#2017)
* Filter check output based on sensitive flag
-Updated check in formatters to filter check output during failures based on
sensitive metadata flag
-Added functional test of output filtering
-Updated documentation with blerb on usage
* Update output format for sensitive resources

Signed-off-by: Kevin Formsma <kevin.formsma@gmail.com>

* Update color output on new test

Update the color output to match the newly-expected non-color format if there are no tests that match.

Signed-off-by: Adam Leff <adam@leff.co>
2017-08-25 16:21:49 -04:00
Adam Leff
6029a4b43d Refine the profile/test summary output of the CLI formatter (#2094)
* Refine the profile/test summary output of the CLI formatter

* The "Profile Summary" is misleading as it's not a summary of profile
  success/failure but rather the controls within the profile(s). Altered
  the output to be clear. I still like calling it the "profile summary"
  but wanted to add clarity that the numbers are about the controls.

* Made the colorized output dynamic. The success/failure will only be
  green/red if there are controls/tests that fall into that category.
  That way we are not printing red failure text when there are no
  actual failures. Fixes #1752.

* Cleaned up some grammar issues. ("1 failure" vs "1 failures")

Signed-off-by: Adam Leff <adam@leff.co>
2017-08-23 10:29:09 -04:00
Dominik Richter
9e3706aabe bugfix: enforce option values where needed (#1918)
Due to limitations in Thor it is not possible to set an argument to be both optional and its value to be mandatory. E.g. the user supplying the --password argument is optional and not always required, but whenever it is used, it requires a value. Handle options that were defined with mandatory values in a way that fails with an `ArgumentError` if the value is missing, i.e.:

```
> inspec exec examples/profile --password
ArgumentError: Please provide a value for --password. For example: --password=hello.
```

It works without `--password` or with `--password=arg`. Also handled for `--sudo-password`.

Fixes: https://github.com/chef/inspec/issues/1901
As suggested: https://github.com/chef/inspec/pull/1904

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
2017-06-12 17:33:16 -04:00
Dominik Richter
84fe398e49 bugfix: adjust localhost+sudo test output to train update
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
2017-05-30 22:40:05 -05:00
Adam Leff
04c43c6834
Make the --no-color flag work for inspec exec
The CLI formatter is not currently honoring the --no-color flag
when outputting CLI output. This change cleans up how we format
with color and properly support the flag for use cases where
color-encoding characters make the output difficult to use
(i.e. when someone redirects CLI output to a text file for
sharing with others).

Signed-off-by: Adam Leff <adam@leff.co>
2017-05-02 09:52:01 -05:00
Adam Leff
6ed5379134 Error and exit when using --sudo locally
Running `inspec exec` with --sudo locally produces unintended results
given that we cannot escalate local Ruby methods after we're already
running. --sudo is meant to only be used with remote targets. We do
not currently enforce that.

This change will print an error for the user if they attempt to use
--sudo with a local exec and exit non-zero.

Signed-off-by: Adam Leff <adam@leff.co>
2017-05-02 12:41:02 +02:00
Adam Leff
d0bc085412 Generate default profile names, fix bug when using two-or-more flat profiles
When running InSpec with multiple profiles, and two or more of the profiles
are read in using the "Flat" SourceReader (i.e. they are not actual profiles
with a metadata file like inspec.yml, but rather just a folder containing
.rb files with controls and tests in them), InSpec would throw a NilClass
error when building the necessary objects for the formatter.

The cause was in `#profile_contains_example` in the formatter code which
checks to see if the profile name is the same as the profile_id in the given
example. However, if both of those were nil, it would potentially match the
wrong Flat-read profile.

This change fixes this in two ways: refusing to match if the profile name
or example profile ID is nil, and adding a default name to a profile if
it doesn't have a title or name. This will solve the matching issue and also
clean up the formatter output so users can more easily tell what tests
are from which profile/path.

Signed-off-by: Adam Leff <adam@leff.co>
2017-02-13 13:07:41 -05:00
Franklin Webber
4695c9e7be Fixes for functional tests and better displaying dep. profiles
* Fixes an issue when specifying no profile
* Fixes an issue when displaying a profile that has included/required profiels
* Fixes an issue when specifying profiles with only metadata
* Fixes formatting for spacing to ensure it adheres to previous alignment
* Fixes issue with the Control object and the rolling up of failed
  and skipped examples.

Signed-off-by: Franklin Webber <franklin@chef.io>
2016-12-14 13:34:53 -06:00
hannah-radish
91fe22f044 Replaced Colors for output
Signed-off-by: hannah-radish <hannah@radish-life.com>
2016-11-23 20:19:18 -07:00
hannah-radish
ebb3bc729a Introduce new (awesome) CLI icons for results
Signed-off-by: Hannah Maddy hannah@radish-life.com
2016-11-23 20:06:17 -07:00
Alex Pop
5d51b7a0aa dot inspect actual value to match the expected string one 2016-10-13 07:13:51 -07:00
Alex Pop
6719cf544d add tests for cmp output 2016-10-10 05:40:22 -07:00
Victoria Jeffrey
47c6427082 print profile info before test results (inherited profiles) 2016-09-27 10:39:35 -04:00
Steven Danna
7aa4c6da8e Fix require_controls DSL method
Previous, require_controls was including all controls from the named
profile, despite the documented behavior being that it only includes
controls explicitly pulled in by the user.  The cause was two-fold:

1) A previous refactor meant that we weren't removing the rule from the
correct context, and

2) We weren't descending down the dependency tree when filtering rules.

This commit fixes the require_controls DSL method and adds a test to
help prevent future regressions.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-26 15:20:56 +02:00
Alex Pop
13da437dcc Show skip_message and correct title 2016-09-23 07:47:21 +01:00
Steven Danna
d29e8768ca Rename --no-write-lockfile to --no-create-lockfile
Signed-off-by: Steven Danna <steve@chef.io>
2016-09-22 10:08:32 +02:00
Steven Danna
2f3a916080 Always write lockfiles for local top-level profiles
This commit threads through some state related to whether or not a
profile is "local", that is whether it is a directory on disk.  If it
is, we then write out the lockfile to disk.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-22 10:08:32 +02:00
Alex Pop
e1faebd527 Include code description in the output of failed controls 2016-09-20 10:10:08 +01:00
Steven Danna
b2146d8758 Allow users to reference resources from dependencies
All resources from deps are added into the control_eval_context used by
the current profile. However, if there is a name conflict, the last
loaded resource wins. The new `require_resource` dsl method allows the
user to do the following:

    require_resource(profile: 'profile_name',
                     resource: 'other',
                    as: 'renamed')

    describe renamed do
      ...
    end

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-19 19:08:43 +02:00
Dominik Richter
6792550f8c adopt new json formatting 2016-09-19 13:45:03 +02:00
Victoria Jeffrey
ecac8ae9cb print profile summary and test summary 2016-09-18 21:53:16 -04:00
Steven Danna
be1a61f2e5 Process transitive dependencies from lock file
This is a regression introduced by the changes from string to symbol
keys in v0.34.0. It seems that our test cookbook that had a nested
dependency example wasn't actually wired up to run.

This adds a basic functional test and corrects the typo.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-15 09:41:40 +02:00
Steven Danna
f2e587f6d5
Skip controls from profile's that don't support the current platform
Any controls included from profiles that don't support our current
platform are now marked as skipped.

Fixes #1049
2016-09-14 09:57:53 +01:00
Steven Danna
8f10ee53c5 Provide inspec.yml shortcut syntax
- Allow users to elide the `name` attributes
- Assume a default source of supermarket

Fixes #1048

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-14 08:46:06 +02:00
Steven Danna
3777f06927
Remove some warnings during the test run 2016-09-09 14:14:36 +01:00