Commit graph

25 commits

Author SHA1 Message Date
Clinton Wolfe
502cf1d7b9 Add test helper to detect all-passing exec run
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-04-26 13:14:32 -04:00
Clinton Wolfe
23fdecd9c3 Make sure trailing newline is preserved when ignoring deprecations in functional tests
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-04-25 14:48:35 -04:00
Clinton Wolfe
91c0f1e83b Rewrite fltertable functional tests to DRY up and avoid --json-config
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-04-24 11:37:43 -04:00
Clinton Wolfe
aa19b6e99d Add helper to filter out deprecations on stderr
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-04-23 11:46:13 -04:00
devoptimist
2d49f39adc moved example tests into their own sub dir, and reused examples_path variable
Signed-off-by: devoptimist <sbrown@chef.io>
2019-03-18 15:15:32 +00:00
devoptimist
94d80f6c8f Decoupling test profiles from example profiles
Signed-off-by: devoptimist <sbrown@chef.io>
2019-03-14 00:32:13 +00:00
Clinton Wolfe
066fde854d Allow empty opts to functional test helper run_inspec_process
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-01-11 16:42:46 -08:00
Clinton Wolfe
ad84987357 Fix spurious ampersands in windows invocation
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-01-09 17:16:44 -08:00
Clinton Wolfe
d6b5bdef09 Add diagnostics to the functional test helper
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2019-01-09 17:14:42 -08:00
Clinton Wolfe
c0a46cf841
UI Object for for CLI work (#3618)
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2018-12-12 11:44:16 -05:00
Clinton Wolfe
fefa6c2ecd
Plugin Type: DSLs (#3557)
This PR adds 5 closely related plugin types, which allow a plugin to implement new DSL methods / keywords. The mechanism to activate the plugins are all very similar - basically, in a particular location in the code, `method_missing` is implemented, and is used to activate the particular type of DSL being requested.

4 of the DSL plugin types relate to code that could appear in a profile control file.

* outer_profile_dsl plugins allow you to extend the code in profile Ruby files that appear outside `control` or `describe` blocks.
* control_dsl plugins allow you to extend the code within `control` blocks.
* describe_dsl plugins allow you to extend the code within `describe` blocks.
* test_dsl plugins allow you to extend the code within `it`/`its` blocks.

Finally, the `resource_dsl` plugin allows you to extend the code used within custom resources.

Basic unit tests are provided to prove that the plugin types are properly defined.

A simple plugin fixture defining DSL hooks (based on favorite foods) is included, and is exercised through a set of functional tests.

The plugin developer docs are updated to describe the 5 DSLs.

*Note*: Implementing a plugin using any of the DSL plugin types is experimental.  The contexts that are exposed to the DSL methods are private and poorly documented. The InSpec project does not claim the APIs used by these plugin types are covered by SemVer.  Plugin authors are encouraged to pin tightly to the `inspec` gem in their gemspecs.

Motivation for this plugin comes from the desire to allow passionate community members to implement things like "2 out of 3" tests, example groups, improved serverspec compatibility, "they/their" and other "fluency" changes, as well as make it possible for future work by the InSpec team to be implemented as a core plugin, rather than a direct change to the main codebase.
2018-11-29 14:14:06 -05:00
Clinton Wolfe
35ad5d5528 Improvements to the functional helper run_inspec_process (#3603)
* Add run_inspec_process, with JSON handling, to functional helper
* Add ability to run a plugin to functional helper
* CWD fix for run_inspec_process
* Add support for running a plugin within a functional test
* Move plugin functional helper code to main func helper file
* FuncHelper: Do not send reporter or lock options if command is not exec

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2018-11-16 17:03:40 -05:00
Jared Quick
d33d189d93
Create appvayor workers for Windows functional tests. (#3397)
* Add more windows functional tests and create some appvayor workers.
* Fix uuid test for automate

Signed-off-by: Jared Quick <jquick@chef.io>
2018-11-08 12:00:14 -05:00
Jerry Aldrich
bbc4002b53 Fix profile vendoring on Windows (#3378)
* Fix profile vendoring on Windows

This fixes vendoring on Windows by doing the following:
  - Expanding relative paths (handles `\\`)
  - Ensuring archives after closed after reading (prevents locking)

This also does the following:
  - Removes extra file from testing tar archive
  - Ensures fetching dirs/archives in the local fetcher behaves the same

* Fix profile vendoring on Windows

This fixes vendoring on Windows by doing the following:
  - Expanding relative paths (handles `\\`)
  - Ensuring archives after closed after reading (prevents locking)

This also does the following:
  - Removes extra file from testing tar archive
  - Ensures fetching dirs/archives in the local fetcher behaves the same

* Add vendoring to fuctional testing and fix the tmp path for windows

* Add tests for relative paths and backslashes

* Remove backslashes support in filenames on Linux

Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-09-13 20:19:02 -04:00
Jared Quick
6445e2c6d5
Add windows functional tests (#3385)
* Add windows functional tests.
* Fix tests for 2012 server.
* Fix windows build script
* Add more functional tests for windows.
* Update comment with TODO.

Signed-off-by: Jared Quick <jquick@chef.io>
2018-09-12 18:04:16 -04:00
Jerry Aldrich
9d031053ea Various improvements to vendor command (#3286)
Signed-off-by: Jerry Aldrich <jerryaldrichiii@gmail.com>
2018-09-06 23:28:08 -04:00
Clinton Wolfe
811318f2f8 Plugins API v2: Loader, Base API, and Test Harness (#3278)
* Functional tests for userdir option
* Accepts --config-dir CLI option
* Actually loads a config file from the config dir, more cases to test
* Able to load config and verify contents from config-dir
* Functional tests to ensure precedence for config options
* Enable setting config dir via env var
* .inspec, not .inspec.d
* Begin converting PluginCtl to PluginLoader/Registry
* Able to load and partially validate the plugins.json file
* More work on the plugin loader
* Break the world, move next gen stuff to plugin/
* Be sure to require base cli in bundled plugins
* Move test file
* Revert changes to v1 plugin, so we can have a separate one
* Checkpoint commit
* Move v2 plugin work to v2 area
* Move plugins v1 code into an isolated directory
* rubocop fixes
* Rip out the stuff about a user-dir config file, just use a plugin file
* Two psuedocode test file
* Working base API, moock plugin type, and loader.
* Adjust load path to be more welcoming
* Silence circular depencency warning, which was breaking a unit test
* Linting
* Fix plugin type registry, add tests to cover
* Feedback from Jerry

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
2018-08-16 18:16:32 -04:00
Kevin Formsma
94c2e8181c Add sensitive flag to resources to restrict logging output (#2017)
* Filter check output based on sensitive flag
-Updated check in formatters to filter check output during failures based on
sensitive metadata flag
-Added functional test of output filtering
-Updated documentation with blerb on usage
* Update output format for sensitive resources

Signed-off-by: Kevin Formsma <kevin.formsma@gmail.com>

* Update color output on new test

Update the color output to match the newly-expected non-color format if there are no tests that match.

Signed-off-by: Adam Leff <adam@leff.co>
2017-08-25 16:21:49 -04:00
Christoph Hartmann
47eabbb221 add functional tests for inspec check (#2077)
Signed-off-by: Christoph Hartmann <chris@lollyrock.com>
2017-08-15 20:41:24 +02:00
Dominik Richter
54444e8878 fix intermitten functional vendor test failures (#1919)
* fix intermitten functional vendor test failures

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>

* isolate artifact functional tests to temporary directory

Signed-off-by: Dominik Richter <dominik.richter@gmail.com>
2017-06-12 14:01:26 +02:00
Christoph Hartmann
66449cae32 add functional tests
Signed-off-by: Christoph Hartmann <chris@lollyrock.com>
2016-11-09 09:12:10 -06:00
Victoria Jeffrey
99ce09c4ac fix inherited profile cli report 2016-09-04 18:28:01 +02:00
Victoria Jeffrey
5d5aa6354d fix and add test 2016-09-01 20:39:52 -04:00
Dominik Richter
b14495051a prevent duplicate profile-loading
this happens when the profile is run (exec) and also interpreted (via profile.params). It will load 2 profile context calls (both via Runner) which in turn gets 2 rounds of interpreter+runner executions. This is an issue with auto-generated IDs, due to their random component, which changes in this case
2016-05-06 13:14:40 +02:00
Dominik Richter
17840e0299 split up functional tests 2016-03-25 01:32:03 +01:00