Commit graph

53 commits

Author SHA1 Message Date
Ryan Davis
f67171fa6b Bump rspec to 3.9 and fix broken tests.
Tests are STILL brittle, but this at least lets us drop old rspec.

Fixes #4575

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-12-06 16:52:35 -08:00
Ryan Davis
bb7e69d145 Fixes for some of the inspec exec tests under Windows. 2019-11-12 14:51:56 -08:00
Ryan Davis
4242937bea Changed a bunch of profiles to use / instead of /tmp for Windows compat. 2019-11-12 14:51:56 -08:00
Ryan Davis
b1c2af209b Corresponding fixes for the move from test/unit/mock to test/fixtures.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-11-08 19:20:12 -08:00
Miah Johnson
6f460b9c3b Remove this. It slows things down.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-10-16 20:57:31 -07:00
Miah Johnson
c3eb686b51 Remove debug
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-10-16 10:15:49 -07:00
Miah Johnson
301905e8de b0rk b0rk
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-10-16 10:15:49 -07:00
Miah Johnson
41bc8fd7a0 Clean up some tests
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-10-16 10:15:49 -07:00
Miah Johnson
ae3ee6f657 Finish rename of example -> example-tmp.rb
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-10-16 10:13:24 -07:00
Miah Johnson
df7efefc9d mostly a search and replace for gordon -> example
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-10-16 10:13:24 -07:00
Clinton Wolfe
3c266655ee Refactor exec-json functional tests to reduce subprocesses
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-10-08 21:04:58 -07:00
Clinton Wolfe
ba61cd3f45 Update JSON schema to reflect new waiver keys
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-10-08 21:01:19 -07:00
Ryan Davis
07dc5e3192 First pass at cleaning deprecations for old minitest/spec-style tests.
3 files left to go, and they're behaving oddly so I'm leaving them out
in this pass. Looks like 21 deprecations left.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-10-03 13:45:19 -07:00
Ryan Davis
fc1ad67ffc More parallelization for functionals.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-09-17 11:50:31 -07:00
Ryan Davis
bcf3aa841c Skip failures from BK run.
Not sure why these failures come up in a unified run (versus my
running all test files separately). Might be test infection. Might be
environmental.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-07-24 18:59:01 -07:00
Ryan Davis
18ce4850f0 Initial pass to clean up functional testing wrt exit codes and output.
This should get windows tests running again? Hopefully? They're
running SO slowly on my vagrant box that I'm just pushing for now.
They're clean on the mac side.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-07-24 18:06:25 -07:00
Miah Johnson
a4f4fe5231 chefstyle -a; https://github.com/chef/chefstyle/pull/74
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-07-08 17:22:50 -07:00
Ryan Davis
692d6a9ab8 hand applied one that is now marked as unsafe in rubocop
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-11 18:52:38 -07:00
Miah Johnson
729177f3c6 chefstyle -a
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-11 18:52:38 -07:00
Miah Johnson
e46018a517 chefstyle -a cleanup
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-11 18:52:38 -07:00
Ryan Davis
a5309ea392 blindly applied chefstyle -a
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-11 18:52:03 -07:00
Ryan Davis
58fa148773 Stabilize windows functional testing.
Skip most everything.

After some digging, in those tests that didn't have 100% failures, of
the ~10+% passing, those tests weren't checking enough. So I skip them
too in the hopes that we improve testing across the board.

At this point, we need appveyor to be green more than we need these
tests to be fixed. If that means we skip them, so be it.

These tests will time-bomb at the end of July.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-03 23:27:26 -07:00
Miah Johnson
0aea548dd8 Update tests to use json-schema, and test something.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-01 23:17:53 -07:00
Miah Johnson
2a1b1d8e88 Remove all leading newlines.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-05-31 11:43:44 -07:00
Miah Johnson
6c19e81a06 Remove authors tags from source files.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-05-28 12:47:39 -07:00
Miah Johnson
81c9a27b37 Fix broken tests after removing magic comments.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-05-08 16:17:25 -07:00
Miah Johnson
659b4b373a Remove # encoding: utf8 magic comments
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-05-07 16:06:23 -07:00
devoptimist
2d49f39adc moved example tests into their own sub dir, and reused examples_path variable
Signed-off-by: devoptimist <sbrown@chef.io>
2019-03-18 15:15:32 +00:00
devoptimist
94d80f6c8f Decoupling test profiles from example profiles
Signed-off-by: devoptimist <sbrown@chef.io>
2019-03-14 00:32:13 +00:00
Jared Quick
0f42b31016
Allow end of options during Thor array parsing (#3547)
Signed-off-by: Jared Quick <jquick@chef.io>
2018-10-30 06:32:16 -04:00
Jared Quick
2bbcdbde9b
Inspec 3.0 (#3512)
* Remove deprecated yumrepo. (#3435)

* Remove deprecations for cli `--format` and metadata.rb (#3452)

* Remove deprecated database_helpers stderr/stdout methods.
Update deprecation text for processes/apache.

* Remove deprecations for `--format` and metadata.rb
Remove deprecated `format` code.
Remove deprecated code test and change json-config format test to use
reporter.
Remove deprecated metadata.rb code
Remove deprecation notice for old supports syntax.
Deprecate metadata.rb from source_reader
Remove rubocop disables as they are no longer required for this code block.
Remove deprecated legacy metadata.rb mock profiles.
Remove deprecated metadata.rb profile tests.
Remove deprecated yumrepo test.

* Allow inspec-3.0 branch to be tested.
* Allow appveyor to test inspec-3.0 branch
* Change runner tests to use reporter rather than format.
Remove deprecated `supports: linux` tests.

* Remove skip from inherited profiles from showing up in reporting (breaking change) (#3332)

* Skip loading dependency profiles if they are unsupported on the current
platform.

Skip loading dependencies if they are unsupported on the current
platform.

Wrap our log and next in a conditional checking if the platform is
supported.

Change a `if !` into a `unless`

Check if the backend is a Train Mock Connection and if so say that the
profile does support the platform.

While iterating through tests being loaded skip when the platform is
unsupported.

We now log a WARN when a profile is skipped due to unsupported platform,
so lets check that.

Modified existing test to log that there are 0 skipped tests, instead of
2.

Add functional test that loads profile-support-skip with a json reporter
to check that our controls are not loaded and that stderr contains our
warning.

* Rather than iterating through each test return before recursion if the platform is
unsupported.

* Resolve tests using a supported platform different from testing platform

Add a control to `test/unit/mock/profiles/complete-profile` that would
work on any OS with a Internet connection. This allows the profile
to execute on any OS with success. `filesystem_spec.rb` was a control
that would only work on Linux and some BSD's.

We want profile tests to consistently work across development and testing
platforms, and not get 'skipped' in some cases.  Travis-CI tests on Linux,
Inspec Dev team uses Linux and MacOS, Appveyor tests on Windows

Also Updated `file_provider_test.rb` for `complete-profile` content changes.

If you `MockLoader.load_profile` on a unsupported platform you might not
hit the usual skip. Lets handle situations where the tests array in
Profile#load_checks_params could be nil.

* Use safe navigation rather than checking if tests is nil.
Update tests to point to unsupported_inspec and account for WARN changes.
Make unsupported_inspec profile support os-family 'unsupported_inspec'

* Fix skip bug when using include/require controls. (#3487)

* Fix skip bug when using include/require controls.
* fix test and feedback.

* Remove need for UUID detection for Automate report (#3507)
* Add json metadata for skipped profiles (#3495)

* Add skip metadata to json reports
* Unify skip messages.
* Update with status field.
* Add testing.
* Fix tests.
* lint
* Add skip exit codes for profile skips.
* Update website for 3.0 launch

Add `plugins` to sidebar.
Change 2.0 -> 3.0 in slim files.
Update 3.0 features list.
* Fix comments
* Update float to numeric.
* Change Float to numeric.
* updated feature list and impact doc
* Change "What's new in InSpec 3.0" -> "Announcing InSpec 3.0"
* Bump VERSION to 3.0.0 (#3511)

* Remove 3.0 testing checks.

* Fix azure link.
2018-10-15 18:25:27 -04:00
Jerry Aldrich
0b0a0a4d48 Change Inspec to InSpec where appropriate (#3494)
Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-10-15 12:09:46 -04:00
Jerry Aldrich
20776b363d Add support for multiple descriptions for controls (#3424)
* Add support for multiple descriptions for controls

This adds the ability to specify multiple descriptions in controls.

Example:

```ruby
control 'my-control' do
  impact 1.0
  title 'My control'
  desc 'A default description'
  desc 'rational', 'I need an example'
  describe file('/tmp') do
    it { should be_directory }
  end
end
```

Many thanks to @jquick for helping me with the unit tests.
* Remove unused `descriptions` method
* Remove unused profile from test mocks
* Respond to feedback

Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-09-26 13:28:57 -04:00
Jared Quick
1c12a544c7
Add string impact options for controls (#3359)
* Add impact class which contains all cvss scores.
* Add testing for impact changes.
* Change symbols to strings for impact.
* Update error messages to be more clear.
* Fix test with new sha

Signed-off-by: Jared Quick <jquick@chef.io>
2018-09-13 14:14:05 -04:00
Jared Quick
6445e2c6d5
Add windows functional tests (#3385)
* Add windows functional tests.
* Fix tests for 2012 server.
* Fix windows build script
* Add more functional tests for windows.
* Update comment with TODO.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-09-12 18:04:16 -04:00
Jared Quick
76b453eee9
Allow target-id passthrough (#3320)
* Allow uuid passthrough
* Update flag to be target-id.
* Updated to use proper formatting for header.
* Fix empty line after cli banner.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-09-05 15:07:34 -04:00
Jared Quick
6120497db1
Convert legacy supports to their platform counterparts (#3333)
* Convert legacy supports to their platform counterparts.
* Fix rubocop lint.
* Update json schema for platform supports.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-08-30 09:11:55 -04:00
Jerry Aldrich
4731d95abd docs: Update example resource (#2904)
* Change `skip_resource` to use raise
* Add `supports` lines to example resource
* Change to rescue `StandardError` vs `Exception`
* Change raise to use `e.message` vs `$!`
* Remove redundant returns
* Change `File.exists?` to `File.exist?`
* Update shasum in tests

Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-04-05 08:50:49 -04:00
Jared Quick
59fd0e8775
Update reporter with breaking inspec 2.0 changes. (#2487)
* Update reporter with breaking inspec 2.0 changes.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-02-14 11:54:20 -05:00
Jared Quick
9930e40a76 Add new "reporter" system (replacement for "formatters"), support multiple reporters per run (#2464)
* Formatter and reporter refactor.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add exception and backtrace to json-min report.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add sha to json-min and include generator version for json profile.

Signed-off-by: Jared Quick <jquick@chef.io>

* Fix deprecated typo and add fallback for cli resource title.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update to build json report and clean up cli logic.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add tests for json reporter.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add cli suppress_log_output? and a fallback for invalid reporter type.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update suppress_log_output? to check if we are outputting to stdout.

Signed-off-by: Jared Quick <jquick@chef.io>

* Update reporter cli optoins to work with json_config.

Signed-off-by: Jared Quick <jquick@chef.io>

* Refactor some safe-navigation and variable names.

Signed-off-by: Jared Quick <jquick@chef.io>

* Add thor banner to show reporter file output syntax.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-02-08 10:06:58 +01:00
Adam Leff
a6582bea9b Remove any "All Rights Reserved" references (#1969)
* Remove any "All Rights Reserved" references

InSpec is licensed and released under the Apache 2.0 license. This
change removes all reference to legacy code files that still had
any Copyright or License lines referring to "All Rights Reserved".

Signed-off-by: Adam Leff <adam@leff.co>

* fix functional tests

Signed-off-by: Christoph Hartmann <chris@lollyrock.com>
2017-06-28 04:14:19 -07:00
Christoph Hartmann
687f1a5827 update unit tests
Signed-off-by: Christoph Hartmann <chris@lollyrock.com>
2017-05-31 00:21:05 -05:00
Dominik Richter
d44b751603 add sha256 checksum to json
Fixes https://github.com/chef/inspec/issues/1658

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
2017-05-11 09:52:28 +02:00
Dominik Richter
738bae0db8 add inspec json schema validation to functional tests
Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
2017-03-22 09:42:36 +01:00
Steven Danna
d29e8768ca Rename --no-write-lockfile to --no-create-lockfile
Signed-off-by: Steven Danna <steve@chef.io>
2016-09-22 10:08:32 +02:00
Steven Danna
2f3a916080 Always write lockfiles for local top-level profiles
This commit threads through some state related to whether or not a
profile is "local", that is whether it is a directory on disk.  If it
is, we then write out the lockfile to disk.

Signed-off-by: Steven Danna <steve@chef.io>
2016-09-22 10:08:32 +02:00
Dominik Richter
6792550f8c adopt new json formatting 2016-09-19 13:45:03 +02:00
Dominik Richter
38f2680cf4 static keys in all json 2016-09-19 12:00:14 +02:00
Dominik Richter
16bd6a14d5 revert control_summary field in output
(1) The field is not yet optimal, the calculations are great!
(2) Changing this field should go together with all other breaking json changes, especially if https://github.com/chef/inspec/pull/811 results in a change.
2016-08-08 11:54:27 +02:00
Kartik Null Cating-Subramanian
8094add5b3 Test summary JSON schema 2016-08-05 13:27:30 -04:00