Commit graph

102 commits

Author SHA1 Message Date
Jared Quick
59fd0e8775
Update reporter with breaking inspec 2.0 changes. (#2487)
* Update reporter with breaking inspec 2.0 changes.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-02-14 11:54:20 -05:00
Jared Quick
9930e40a76 Add new "reporter" system (replacement for "formatters"), support multiple reporters per run (#2464)
* Formatter and reporter refactor.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add exception and backtrace to json-min report.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add sha to json-min and include generator version for json profile.

Signed-off-by: Jared Quick <jquick@chef.io>

* Fix deprecated typo and add fallback for cli resource title.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update to build json report and clean up cli logic.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add tests for json reporter.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add cli suppress_log_output? and a fallback for invalid reporter type.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update suppress_log_output? to check if we are outputting to stdout.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update reporter cli optoins to work with json_config.

Signed-off-by: Jared Quick <jquick@chef.io>

* Refactor some safe-navigation and variable names.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add thor banner to show reporter file output syntax.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-02-08 10:06:58 +01:00
Clinton Wolfe
656423d7f2 Issue warning during check if profile name contains slash (#2231)
* Add failing unit test for deprecation warning on profiles with slashes in their name

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Issue warning during validation if name contains a slash

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Slug profile names generated from target paths to prevent breaking unit tests

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Rubocop whinges

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Update functional test watching for default profile name

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Make deprecation warning more descriptive

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Rubocop whinges

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Set title with original test path if no profile name or title provided

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Rubocop whinges

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2017-11-09 12:32:54 +01:00
Kevin Formsma
94c2e8181c Add sensitive flag to resources to restrict logging output (#2017)
* Filter check output based on sensitive flag
-Updated check in formatters to filter check output during failures based on
sensitive metadata flag
-Added functional test of output filtering
-Updated documentation with blerb on usage
* Update output format for sensitive resources

Signed-off-by: Kevin Formsma <kevin.formsma@gmail.com>

* Update color output on new test

Update the color output to match the newly-expected non-color format if there are no tests that match.

Signed-off-by: Adam Leff <adam@leff.co>
2017-08-25 16:21:49 -04:00
Adam Leff
6029a4b43d Refine the profile/test summary output of the CLI formatter (#2094)
* Refine the profile/test summary output of the CLI formatter

* The "Profile Summary" is misleading as it's not a summary of profile
  success/failure but rather the controls within the profile(s). Altered
  the output to be clear. I still like calling it the "profile summary"
  but wanted to add clarity that the numbers are about the controls.

* Made the colorized output dynamic. The success/failure will only be
  green/red if there are controls/tests that fall into that category.
  That way we are not printing red failure text when there are no
  actual failures. Fixes #1752.

* Cleaned up some grammar issues. ("1 failure" vs "1 failures")

Signed-off-by: Adam Leff <adam@leff.co>
2017-08-23 10:29:09 -04:00
Dominik Richter
9e3706aabe bugfix: enforce option values where needed (#1918)
Due to limitations in Thor it is not possible to set an argument to be both optional and its value to be mandatory. E.g. the user supplying the --password argument is optional and not always required, but whenever it is used, it requires a value. Handle options that were defined with mandatory values in a way that fails with an `ArgumentError` if the value is missing, i.e.:

```
> inspec exec examples/profile --password
ArgumentError: Please provide a value for --password. For example: --password=hello.
```

It works without `--password` or with `--password=arg`. Also handled for `--sudo-password`.

Fixes: https://github.com/chef/inspec/issues/1901
As suggested: https://github.com/chef/inspec/pull/1904

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
2017-06-12 17:33:16 -04:00
Dominik Richter
84fe398e49 bugfix: adjust localhost+sudo test output to train update
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
2017-05-30 22:40:05 -05:00
Adam Leff
04c43c6834
Make the --no-color flag work for inspec exec
The CLI formatter is not currently honoring the --no-color flag
when outputting CLI output. This change cleans up how we format
with color and properly support the flag for use cases where
color-encoding characters make the output difficult to use
(i.e. when someone redirects CLI output to a text file for
sharing with others).

Signed-off-by: Adam Leff <adam@leff.co>
2017-05-02 09:52:01 -05:00
Adam Leff
6ed5379134 Error and exit when using --sudo locally
Running `inspec exec` with --sudo locally produces unintended results
given that we cannot escalate local Ruby methods after we're already
running. --sudo is meant to only be used with remote targets. We do
not currently enforce that.

This change will print an error for the user if they attempt to use
--sudo with a local exec and exit non-zero.

Signed-off-by: Adam Leff <adam@leff.co>
2017-05-02 12:41:02 +02:00
Adam Leff
d0bc085412 Generate default profile names, fix bug when using two-or-more flat profiles
When running InSpec with multiple profiles, and two or more of the profiles
are read in using the "Flat" SourceReader (i.e. they are not actual profiles
with a metadata file like inspec.yml, but rather just a folder containing
.rb files with controls and tests in them), InSpec would throw a NilClass
error when building the necessary objects for the formatter.

The cause was in `#profile_contains_example` in the formatter code which
checks to see if the profile name is the same as the profile_id in the given
example. However, if both of those were nil, it would potentially match the
wrong Flat-read profile.

This change fixes this in two ways: refusing to match if the profile name
or example profile ID is nil, and adding a default name to a profile if
it doesn't have a title or name. This will solve the matching issue and also
clean up the formatter output so users can more easily tell what tests
are from which profile/path.

Signed-off-by: Adam Leff <adam@leff.co>
2017-02-13 13:07:41 -05:00
Franklin Webber
4695c9e7be Fixes for functional tests and better displaying dep. profiles
* Fixes an issue when specifying no profile
* Fixes an issue when displaying a profile that has included/required profiels
* Fixes an issue when specifying profiles with only metadata
* Fixes formatting for spacing to ensure it adheres to previous alignment
* Fixes issue with the Control object and the rolling up of failed
  and skipped examples.

Signed-off-by: Franklin Webber <franklin@chef.io>
2016-12-14 13:34:53 -06:00
hannah-radish
91fe22f044 Replaced Colors for output
Signed-off-by: hannah-radish <hannah@radish-life.com>
2016-11-23 20:19:18 -07:00
hannah-radish
ebb3bc729a Introduce new (awesome) CLI icons for results
Signed-off-by: Hannah Maddy hannah@radish-life.com
2016-11-23 20:06:17 -07:00
Alex Pop
5d51b7a0aa dot inspect actual value to match the expected string one 2016-10-13 07:13:51 -07:00
Alex Pop
6719cf544d add tests for cmp output 2016-10-10 05:40:22 -07:00
Victoria Jeffrey
47c6427082 print profile info before test results (inherited profiles) 2016-09-27 10:39:35 -04:00
Steven Danna
7aa4c6da8e Fix require_controls DSL method
Previous, require_controls was including all controls from the named
profile, despite the documented behavior being that it only includes
controls explicitly pulled in by the user.  The cause was two-fold:

1) A previous refactor meant that we weren't removing the rule from the
correct context, and

2) We weren't descending down the dependency tree when filtering rules.

This commit fixes the require_controls DSL method and adds a test to
help prevent future regressions.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-26 15:20:56 +02:00
Alex Pop
13da437dcc Show skip_message and correct title 2016-09-23 07:47:21 +01:00
Steven Danna
d29e8768ca Rename --no-write-lockfile to --no-create-lockfile
Signed-off-by: Steven Danna <steve@chef.io>
2016-09-22 10:08:32 +02:00
Steven Danna
2f3a916080 Always write lockfiles for local top-level profiles
This commit threads through some state related to whether or not a
profile is "local", that is whether it is a directory on disk.  If it
is, we then write out the lockfile to disk.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-22 10:08:32 +02:00
Alex Pop
e1faebd527 Include code description in the output of failed controls 2016-09-20 10:10:08 +01:00
Steven Danna
b2146d8758 Allow users to reference resources from dependencies
All resources from deps are added into the control_eval_context used by
the current profile. However, if there is a name conflict, the last
loaded resource wins. The new `require_resource` dsl method allows the
user to do the following:

    require_resource(profile: 'profile_name',
                     resource: 'other',
                    as: 'renamed')

    describe renamed do
      ...
    end

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-19 19:08:43 +02:00
Dominik Richter
6792550f8c adopt new json formatting 2016-09-19 13:45:03 +02:00
Victoria Jeffrey
ecac8ae9cb print profile summary and test summary 2016-09-18 21:53:16 -04:00
Steven Danna
be1a61f2e5 Process transitive dependencies from lock file
This is a regression introduced by the changes from string to symbol
keys in v0.34.0. It seems that our test cookbook that had a nested
dependency example wasn't actually wired up to run.

This adds a basic functional test and corrects the typo.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-15 09:41:40 +02:00
Steven Danna
f2e587f6d5
Skip controls from profile's that don't support the current platform
Any controls included from profiles that don't support our current
platform are now marked as skipped.

Fixes #1049
2016-09-14 09:57:53 +01:00
Steven Danna
8f10ee53c5 Provide inspec.yml shortcut syntax
- Allow users to elide the `name` attributes
- Assume a default source of supermarket

Fixes #1048

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-14 08:46:06 +02:00
Steven Danna
3777f06927
Remove some warnings during the test run 2016-09-09 14:14:36 +01:00
Steven Danna
74e712854e Pass attributes from command line into profile context
We broke attributes with the dependency work. Minimally fix them. TODO:
Maximally fix them.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-07 10:29:47 +02:00
Christoph Hartmann
0a34ffef5a always display error message 2016-09-05 17:23:14 +02:00
Victoria Jeffrey
0667c334e9 fix inherited profile cli report for realz this time 2016-09-04 18:28:01 +02:00
Victoria Jeffrey
99ce09c4ac fix inherited profile cli report 2016-09-04 18:28:01 +02:00
Victoria Jeffrey
5d5aa6354d fix and add test 2016-09-01 20:39:52 -04:00
Victoria Jeffrey
d6ee153aaa print controls, then tests. print describe block header then each test 2016-08-26 10:12:56 +02:00
Kartik Null Cating-Subramanian
0f572df4be Fix integration test 2016-08-05 10:01:29 -04:00
Dominik Richter
8660d5d81c feedback round with @chris-rock 2016-06-16 20:37:51 +02:00
Dominik Richter
e3b20e88b7 provide target info in cli output 2016-06-16 12:26:46 +02:00
Dominik Richter
c34fd350cf multi-profile reporting in cli formatter 2016-06-16 00:08:50 +02:00
Dominik Richter
4fbdee84cf use utf-8 characters for default cli formatter
see https://github.com/chef/inspec/issues/532
2016-06-15 19:27:56 +02:00
Dominik Richter
f93084520f introduce cli report formatter 2016-06-15 17:11:29 +02:00
Dominik Richter
2db8d83d56 support intra-libraries file referencing + loading
solves https://github.com/chef/inspec/issues/779
2016-06-03 22:54:35 +02:00
Christoph Hartmann
8258d111ef rename json to minijson and fulljson to json 2016-05-06 13:27:42 +02:00
Dominik Richter
b14495051a prevent duplicate profile-loading
this happens when the profile is run (exec) and also interpreted (via profile.params). It will load 2 profile context calls (both via Runner) which in turn gets 2 rounds of interpreter+runner executions. This is an issue with auto-generated IDs, due to their random component, which changes in this case
2016-05-06 13:14:40 +02:00
Dominik Richter
20d08a63b5 inspec --format [json|fulljson|rspecjson] overhaul
Full rewrite of all formatters. Create a minimal JSON, a full JSON, and a fallback RSpec formatter. The latter is only needed for corner cases and should not really be used. The former 2 are for (1) running `inspec json` followed by `inspec exec` (`--format json`) and (2) running just `inspec exec --format fulljson`.
2016-05-06 13:14:40 +02:00
Dominik Richter
a809097d12 simplify full_id generation 2016-05-06 13:14:40 +02:00
Dominik Richter
2a0ccbfd76 fail on unsupported os/platform 2016-04-17 00:04:37 -04:00
Dominik Richter
14995534cd skip profiles if the platform isnt supported 2016-04-16 15:34:23 -04:00
Dominik Richter
046e6ce501 bugfix: non-profile execution with json formatter 2016-04-11 11:17:26 -04:00
Dominik Richter
fb54c4ea24 api: inspec.yml supports now adds tests w/o running
Instead of just removing all tests because of OS support, supports now acts by adding all tests to the execution context, but doesnt actually execute them. Instead tests are set to skip before they get to the actual execution context
2016-04-06 11:28:52 +02:00
Alex Pop
070c5bb0e9 update tests with 5 examples 2016-04-04 14:19:13 +01:00
Christoph Hartmann
bc3be2f302 fix functional tests 2016-03-28 01:15:48 +02:00
Dominik Richter
17840e0299 split up functional tests 2016-03-25 01:32:03 +01:00