Commit graph

1577 commits

Author SHA1 Message Date
Ryan Davis
dd7f9abb20 Revamped test/functional/ui_test.rb to check output before exit codes.
Helps with debugging.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-11 21:32:22 -07:00
Ryan Davis
fae4230a41 Removing inspec/profile_vendor from inspec/base_cli.
This was the next most expensive require in the analysis.

Also rearranged the way that ui handled tables to be lazy.

```
% SLOW=1 time rake test:functional

before: Finished in  681.514579s, 0.5136 runs/s, 2.9919 assertions/s.
after : Finished in  642.655918s, 0.5446 runs/s, 3.1728 assertions/s.
```

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-11 19:16:58 -07:00
Ryan Davis
692d6a9ab8 hand applied one that is now marked as unsafe in rubocop
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-11 18:52:38 -07:00
Miah Johnson
68e540a15a Ignore Rubocop checks
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-11 18:52:38 -07:00
Miah Johnson
c579ed3097 Disable rubocop check on TMP_CACHE
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-11 18:52:38 -07:00
Miah Johnson
729177f3c6 chefstyle -a
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-11 18:52:38 -07:00
Miah Johnson
649ed9aca5 This cache needs to be writable. Its a problem.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-11 18:52:38 -07:00
Miah Johnson
e46018a517 chefstyle -a cleanup
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-11 18:52:38 -07:00
Ryan Davis
a5309ea392 blindly applied chefstyle -a
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-11 18:52:03 -07:00
Clinton Wolfe
ec1baf6237
New plugin type: Input (#4019)
New plugin type: Input
2019-06-10 15:13:57 -04:00
Miah Johnson
390d006311 Fix indent
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-06 13:44:18 -07:00
Miah Johnson
b176012abf Maybe stop coverage from running early.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-06 13:43:19 -07:00
Miah Johnson
6b3d96e8a9 Wrap coverage in environment variable CI_ENABLE_COVERAGE
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-06 13:41:57 -07:00
Clinton Wolfe
1628383c8f Mis-spelled a method name
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:22 -04:00
Clinton Wolfe
9366804335 PR Feedback
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:22 -04:00
Clinton Wolfe
4b1ee722b4 Functional test for event log
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:22 -04:00
Clinton Wolfe
4eb815fdd7 syntaxo
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:22 -04:00
Clinton Wolfe
90da768ef0 Move event creation to input_registry to simplify plugin authorship
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:22 -04:00
Clinton Wolfe
53beaff372 Input plugin types work
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:22 -04:00
Clinton Wolfe
9e2e569d85 Add input plugin type API
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:21 -04:00
Clinton Wolfe
962dfc8d90 Failing tests for input plugin
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:21 -04:00
Clinton Wolfe
82f910e320 Plugin type declaration and unit test
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-06-06 12:51:21 -04:00
Ryan Davis
6bed51d69a Fixed up interactions between minitest & simplecov.
Sometime during the 2.x's, `at_exit` changed its ordering. As a result,
a lot of things that were stacking `at_exit`'s broke. This is one of
those since both simplecov and minitest do their thing via `at_exit`.
This switches to simplecov w/ no defaults on, then replicates their
simplecov/defaults.rb with our own.

I'm going to try to get that entire file back upstream but it can live
here for now.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-05 15:16:42 -07:00
Ryan Davis
5e71a6b787 oops
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-04 00:19:26 -07:00
Ryan Davis
58fa148773 Stabilize windows functional testing.
Skip most everything.

After some digging, in those tests that didn't have 100% failures, of
the ~10+% passing, those tests weren't checking enough. So I skip them
too in the hopes that we improve testing across the board.

At this point, we need appveyor to be green more than we need these
tests to be fixed. If that means we skip them, so be it.

These tests will time-bomb at the end of July.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-03 23:27:26 -07:00
Ryan Davis
5573957c52 Go back to cmd from powershell.
Apparently it is too much of a shift and I tested too small of a
sample.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-03 23:11:26 -07:00
Ryan Davis
22635157b5
Merge pull request #4177 from inspec/zenspider/thread_safety_is_hard
Turn off parallel testing (for now?)
2019-06-03 23:04:03 -07:00
Ryan Davis
c40f541a54
Merge pull request #4172 from inspec/mj/jsa
Replace jsonschema with json-schema
2019-06-03 23:03:23 -07:00
Ryan Davis
11638d36dd Now that we aren't parallelizing, start bringing back skipped tests.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-03 14:12:06 -07:00
Ryan Davis
45ef0d84d0 Remove minitest/hell in favor of stable (but sloooow) tests.
Added InspecTest (which we should use across the board) and
ParallelTest (which we should use selectively and with metrics) to
helper.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-03 14:12:06 -07:00
Ryan Davis
d2edd3ce68 Move minitest/hell to test/helper.
Probably going to remove this next and focus on opting-in to parallelization.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-03 14:12:06 -07:00
Ryan Davis
bba9434600 Change the way we load rspec so it never exposes globally.
Essentially, the way rspec does stuff is pretty rude. By injecting
singleton methods into main (instead of Kernel, where all top-level
methods go) and "cleaning" them up using undef_method instead of
remove_method makes it pretty hard for anything else in the testing
arena to operate. I've already fixed the case where rspec would
overwrite minitest's `describe` in a way that made it very hard to put
back, but my fix didn't prevent rspec from doing it _AGAIN_.

This fixes the case where after some rspec interactions, mocha's
`any_instance` method would just disappear. I never did figure out
where or why this was happening, but I did address the how.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-06-03 14:12:06 -07:00
Ryan Davis
a2f2cd5087
Merge pull request #4109 from inspec/zenspider/helper_cleanup
Minor cleanup on test/helper.rb
2019-06-03 13:39:31 -07:00
Miah Johnson
0aea548dd8 Update tests to use json-schema, and test something.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-06-01 23:17:53 -07:00
Miah Johnson
9a580407e6 brittle tests
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-05-31 11:43:44 -07:00
Miah Johnson
a5c8021227 don't remove this leading newline =)
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-05-31 11:43:44 -07:00
Miah Johnson
2a1b1d8e88 Remove all leading newlines.
Signed-off-by: Miah Johnson <miah@chia-pet.org>
2019-05-31 11:43:44 -07:00
Ryan Davis
d10591dc19 Some of our license tests fail if we have CHEF_LICENSE set, so wrap them up.
This still doesn't fix our parallelization issues. I think

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-30 11:17:19 -07:00
Ryan Davis
4790e288a0
Merge pull request #4142 from inspec/zenspider/clean_unnecessary_requires
Clean unnecessary requires
2019-05-29 18:33:06 -07:00
Ryan Davis
dfadc393b9 Even more removals and moves!
Turns out we don't even use CMD in any units... so why make it?

Lots of other stuff got ripped out. Tests basically shouldn't need to
require anything but 1 implementation file and a test framework.
Anything more than that is a smell.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
be125598c8 Additional whitespace cleanup.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
ec248217a7 Extra fix for utils move
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
a8bcba780f Got the rest of the functionals to pass.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
82c9aa6ed6 Got the rest of the tests to pass.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
9ec91484fc Get all non-resource tests fixed again.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
de8258c9f7 Get yaml_test.rb working
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
761f48e63c Get user_test.rb working
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
69f2927983 Stripping test/helper.rb of all the big inspec requires.
Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
61585de0f0 Moved out all of the requires for inspec/resources/* from inspec/resource.rb
All requires moved to inspec/resources.rb

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00
Ryan Davis
adaf2bc364 Removed aws resource requiring from test/helper and inspec/resource.
This speeds up parallel unit test runs from a very consistent 2:49 to
a very consistent 1:53, or a 33% reduction.

Signed-off-by: Ryan Davis <zenspider@chef.io>
2019-05-29 17:58:02 -07:00