Previous, require_controls was including all controls from the named
profile, despite the documented behavior being that it only includes
controls explicitly pulled in by the user. The cause was two-fold:
1) A previous refactor meant that we weren't removing the rule from the
correct context, and
2) We weren't descending down the dependency tree when filtering rules.
This commit fixes the require_controls DSL method and adds a test to
help prevent future regressions.
Signed-off-by: Steven Danna <steve@chef.io>
Before, a URL based source might be downloaded multiple times during the
dependency fetching and lockfile creation. This commit tries to avoid
this by:
1) Memoizing data about the archive to avoid re-fetching the archive
2) Adding a CachedFetcher wrapper around the fetcher class to help
ensure that callers always consult the cache before fetching.
Signed-off-by: Steven Danna <steve@chef.io>
This commit threads through some state related to whether or not a
profile is "local", that is whether it is a directory on disk. If it
is, we then write out the lockfile to disk.
Signed-off-by: Steven Danna <steve@chef.io>
This allows users to run:
inspec exec ./ --cache PATH
which will use `PATH` as the dir to retrieve and store remote
dependencies. The hope is that this can eventually be used with
`inspec vendor PATH` to package up a profile for offline use.
Signed-off-by: Steven Danna <steve@chef.io>
If a URL based source does not match the shasum recorded in the
lockfile, it likely means a new version has been pushed to the remote
source. In this case, we fail to help ensure that when using a lockfile
we always run the same code as when the lockfile was created.
Signed-off-by: Steven Danna <steve@chef.io>
All resources from deps are added into the control_eval_context used by
the current profile. However, if there is a name conflict, the last
loaded resource wins. The new `require_resource` dsl method allows the
user to do the following:
require_resource(profile: 'profile_name',
resource: 'other',
as: 'renamed')
describe renamed do
...
end
Signed-off-by: Steven Danna <steve@chef.io>
This is a regression introduced by the changes from string to symbol
keys in v0.34.0. It seems that our test cookbook that had a nested
dependency example wasn't actually wired up to run.
This adds a basic functional test and corrects the typo.
Signed-off-by: Steven Danna <steve@chef.io>
The recent changes to provide isolated views of the available resources
was not extended to Rspec::ExampleGroups. This ensures that
ExampleGroups have access to the same resources as the enclosing
Inspec::Rule.
Signed-off-by: Steven Danna <steve@chef.io>
This adds a new git fetcher. In doing so, it also refactors how the
fetchers work a bit to better support fetchers that need to resolve
user-provided sources to fully specified sources appropriate for a
lockfile.
Signed-off-by: Steven Danna <steve@chef.io>
We already monkeypatch require so that it is redirected through the
require_loader. All of the tests pass with this removal. We might
cause some breakage with this removal that we aren't testing, but given
that we are mucking with `require` it seems preferable to have one
mechanism by which we do that and solve any bugs with that single path.
Signed-off-by: Steven Danna <steve@chef.io>
Previously, libraries were loaded by instance_eval'ing them against
the same execution context used for control files. All resources were
registered against a single global registry when the `name` dsl method
was invoked. To obtain seperation of resources, we would mutate the
instance variable holding the globale registry and then change it back
at the end.
Now, we instance_eval library files inside an anonymous class. This
class has its own version of `Inspec.resource` that returns another
class with the resource DSL method and the profile-specific resource
registry.
The goal of these changes is to ensure that the libraries from
dependencies are loaded even if their controls are never included. To
facilitate this, we break up the loading into seperate steps, and move
the loading code into the Profile which has acceess to the dependency
information.
Signed-off-by: Steven Danna <steve@chef.io>
Previously, all resources were loaded into a single resource registry.
Now, each profile context has a resource registry, when a profile's
library is loaded into the profile context, we update the
profile-context-specific resource registry. This local registry is
then used to populate the execution context that the rules are
evaluated in.
Signed-off-by: Steven Danna <steve@chef.io>
This is a minor refactor that I did while studying our loading code in
preparation for some deeper changes to how content loading works. The
overall goal of the refactor is to remove a few places where we were
passing a generic options hash and then only accessing a single item.
The comment hopefully clarifies to new developers in the code base how
content loading works at a high level.
Signed-off-by: Steven Danna <steve@chef.io>
The goal of this change is to provide an isolated view of the available
profiles when the user calls the include_controls or require_controls
APIs. Namely,
- A profile should only be able to reference profiles that are part of
its transitive dependency tree. That is, if the dependency tree for a
profile looks like the following:
A
|- B --> C
|
|- D --> E
Then profile B should only be able to see profile C and fail if it
tries to reference A, D, or E.
- The same profile should be include-able at different versions from
different parts of the tree without conflict. That is, if the
dependency tree for a profile looks like the following:
A
|- B --> C@1.0
|
|- D --> C@2.0
Then profile B should see the 1.0 version of C and profile D should
see the 2.0 profile C with respect to the included controls.
To achieve these goals we:
- Ensure that we construct ProfileContext objects with respect to the
correct dependencies in Inspec::DSL.
- Provide a method of accessing all transitively defined rules on a
ProfileContext without pushing all of the rules onto the same global
namespace.
This does not yet handle attributes or libraries.
Also: Log to STDERR by default
NB: This will result in absolute paths being rendered to lock files. We
think that is OK for now since we are going to build some UX around
path-based dependencies and lock files. Namely, we are going to tell
people it is a bad idea.
Signed-off-by: Steven Danna <steve@chef.io>
Resolved an issue checking ports on windows
The previous version wasn't really checking if a port was accessible as we were only validating if the ping succeeded. Using TcpTestSucceeded to determine if the connection worked or not.
The Molinillo library is a good library for systems that need a
constraint solver that will solve dependency problems requiring a single
version of each named dependency.
In our case, the eventual goal is to allow libraries to have conflicting
transitive dependencies at runtime. Isolation will be provided by
restricting all calls within a given profile to scope which can only see
that profile's dependencies.
To facilitate working on the isolation feature, I've replaced the
Molinillo-based resolver with a minimal resolver which will allow us to
load multiple versions of the same library.
Since we will likely want a good amount of logging around this feature
in the future, I've added a Inspec::Log singleton-style class, replacing
the previous Inpsec::Log which appeared unused in the code base.
Signed-off-by: Steven Danna <steve@chef.io>
This adds a basic prototype of inspec.lock. When the lockfile exists on
disk, the dependencies tree is constructed using the information in the
lock file rather than using the resolver.
Signed-off-by: Steven Danna <steve@chef.io>
This extends the dependency feature to include support for url-based
dependencies. It takes some deviations from the current support for
URLs that we'll likely want to make more consistent.
By default, we store downloaded archives in the cache rather than the
unpacked archive. However, to facilitate debugging, we will prefer the
unpacked archive if we find it in the cache.
Signed-off-by: Steven Danna <steve@chef.io>
This commit is the foundation of the dependency resolution as described in https://github.com/chef/inspec/issues/888 .
It currently only works with local dependencies, as seen in the example inheritance profile.
Tests and full resolution are coming next on the path to an MVP implementation.
Redhat conf_dir detection was regressed in 57d7275 which inadvertently
removed the setting of @conf_dir. Any attempt to use the postgres
resource on RHEL would rain an exception:
inspec> postgres.data_dir
TypeError: no implicit conversion of nil into String
Further, the redhat detection code appears to assume that RHEL always
uses versioned data directories. This however, does not appear to be the
case:
$ cat /etc/redhat-release
CentOS release 6.7 (Final)
$ sudo ls /var/lib/pgsql/
backups data pgstartup.log
The code now can handle both versioned and un-versioned directory
formats on RHEL. Further, it provides diagnostic warnings about
uncertainty in the discovered data directories and configuration
directories.
Signed-off-by: Steven Danna <steve@chef.io>
Previously, if you typed more than 20 characters at the prompt and
attempted pressed Ctrl+a (readline's "Move to start of line" command),
your prompt would appear at the ~11th character from the start of the
line, unable to go further back.
This was a result readline counting the terminal escape sequences we use
for color output as part of the line.
Wrapping these sequences in \001 and \002 instructs readline to ignore
them when doing calculations regarding line-length, resolving the
problem.
This adds a new subcommand:
inspec env [SHELL]
which outputs a shell-appropriate completion script that the user can
source into their shell:
eval "$(inspec env SHELL)"
Currently, we provide completions for ZSH and Bash. The completion
scripts are generated from the data Thor collects.
If the user doesn't provide SHELL we attempt to detect what the user's
shell may be using a number of methods.
Signed-off-by: Steven Danna <steve@chef.io>
(1) The field is not yet optimal, the calculations are great!
(2) Changing this field should go together with all other breaking json changes, especially if https://github.com/chef/inspec/pull/811 results in a change.
This allows the user to write:
describe port(22) do
it { should be_listening }
end
as well as
describe port('22') do
it { should be_listening }
end
without hitting an error.
Fixes#867
Signed-off-by: Steven Danna <steve@chef.io>
The output of `systemctl show SERVICENAME` can be misleading in the
case of non-native services (i.e. services configured via an init script
and integrated with systemd via a shim) or for more sophisticated unit
types.
For example, the UnitFileState of ntp is "bad":
> systemctl show ntp | grep UnitFileState
UnitFileState=bad
despite systemd reporting it as enabled:
> systemctl is-enabled ntp
ntp.service is not a native service, redirecting to
systemd-sysv-install
Executing /lib/systemd/systemd-sysv-install is-enabled ntp
enabled
Further, the old parsing code would have missed unit files in the
following states that are technically enabled:
enabled-runtime, indirect, generated, and transient
Using the `is-enabled` commands ensures that we report the same enabled
status that systemd reports, without having to update our own parsing in
the event that new unit states are added. Additionally, as shown above,
it handles the sysv compatibility helper.
Similarly, the is-active helper command ensures that we always report
the same active/not-active status as systemd would natively. For
instance, a quick reading of `src/systemctl/systemctl.c` in the systemd
source shows that systemctl reports units as active if they are in the
state `UNIT_ACTIVE` or `UNIT_RELOADING`.
Fixes#749
Signed-off-by: Steven Danna <steve@chef.io>
Mixing types in an array without specifying what these fields point to is not just confusing, but also causes issues with endpoints that may consume this data and dont process mixed types. We strive to have a stable api for 1.0 and this is a sin that was left after the major overhaul. Time to fix it.
reporters didnt stick to the formatters that were configured but looked for an old json one. this MR ensures that the formatter that is configured is pulled out to generate the report
this happens when the profile is run (exec) and also interpreted (via profile.params). It will load 2 profile context calls (both via Runner) which in turn gets 2 rounds of interpreter+runner executions. This is an issue with auto-generated IDs, due to their random component, which changes in this case
Full rewrite of all formatters. Create a minimal JSON, a full JSON, and a fallback RSpec formatter. The latter is only needed for corner cases and should not really be used. The former 2 are for (1) running `inspec json` followed by `inspec exec` (`--format json`) and (2) running just `inspec exec --format fulljson`.
When used in combination: `shadow[.filter(...)].entries.each { |entry| ... }`, these entries would not be very descriptive at all. You would basically only retrieve the full filter chain e.g. 20 times, without any information about what entry you are currently looking at. This fixes it, by providing the entry identified by the user name
Instead of just removing all tests because of OS support, supports now acts by adding all tests to the execution context, but doesnt actually execute them. Instead tests are set to skip before they get to the actual execution context
instead of keeping them as flat variables, prefix all internals with `__` to create consistency. Also add accessors on the class-level to expose these values in all rules. This way we keep all variable-names in one location and get some safety on access.
In some instances, when running inspec shell, you dont get any resources inside of it. i.e. `inspec shell` and then `os` will lead to
```ruby
NameError: undefined local variable or method `os' for
from (pry):1:in `add_content'
```
This is because of instance_eval loading withing the given source/line
information and not attaching to the profile context which actually has
all the resources. Fix it by making sure that inspec shell always
attaches to the profile context with resources by providing nil for
source and line information.
This is just for simplicity. I expect other users to make the same mistake when using it, so I would rather our tests crash if we have this type of conflict again and prevent it in the first place. Renaming File to FileResource should take care of all important places
Many of the resources are named as a top-level class with a fairly generic class name, such as "OS". This causes an issue specifically with kitchen-google which depends on a gem which depends on the "os" gem which itself defines an OS class with a different superclass. This prevents users from using TK, Google Compute, and Inspec without this fix.
Some mocked commands had their digest changed as well due to the new indentation, specifically in the User and RegistryKey classes.
I strongly recommend viewing this diff with `git diff --ignore-space-change`
to see the *real* changes. :)
```
describe.one do
describe command("uname -r").stdout do
it { should_not match /x86_64/ }
end
describe test_sth_for_x64_processors do
...
end
end
```
i.e. make sure it doesnt crash just because no block was given due to source/line detection.
also return the result of the rule's delegated describe call and not the rule itself to the outer method. this is for consistency (and the following commits)
This helps reduce any folder structures, weather on disk or in archives, to their relative root paths; i.e. ignore all file-prefixes that are given and go directly to the underlying files, relative to the common folders that contain it
