Add an object model for run_data

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
This commit is contained in:
Clinton Wolfe 2020-05-07 00:42:51 -04:00
parent 8ec249e0cc
commit 3184d5ca9e
8 changed files with 460 additions and 10 deletions

View file

@ -42,14 +42,14 @@ You can tell Chef InSpec to use a different config directory using the INSPEC_CO
Top-level entries in the JSON file:
* `plugins_config_version` - must have the value "1.0.0". Reserved for future format changes.
* `plugins` - an Array of Hashes, each containing information about plugins that are expected to be installed
* `plugins_config_version`| must have the value "1.0.0". Reserved for future format changes.
* `plugins`| an Array of Hashes, each containing information about plugins that are expected to be installed
Each plugin entry may have the following keys:
* `name` - Required. String name of the plugin. Internal machine name of the plugin. Must match `plugin_name` DSL call (see Plugin class below).
* `installation_type` - Optional, default "gem". Selects a loading mechanism, may be either "path" or "gem"
* `installation_path` - Required if installation_type is "path". A `require` will be attempted against this path. It may be absolute or relative; Chef InSpec adds both the process current working directory as well as the Chef InSpec installation root to the load path.
* `name`| Required. String name of the plugin. Internal machine name of the plugin. Must match `plugin_name` DSL call (see Plugin class below).
* `installation_type`| Optional, default "gem". Selects a loading mechanism, may be either "path" or "gem"
* `installation_path`| Required if installation_type is "path". A `require` will be attempted against this path. It may be absolute or relative; Chef InSpec adds both the process current working directory as well as the Chef InSpec installation root to the load path.
TODO: keys for gem installations
@ -301,7 +301,7 @@ The minimum needed for a command is a call to `desc` to set the help message, an
```ruby
desc 'Reports on teaspoons in your beverage, always bad news'
def count
# Someone has executed `inspec sweeten count` - do whatever that entails
# Someone has executed `inspec sweeten count`| do whatever that entails
case beverage_type
when :soda
puts 12
@ -370,6 +370,9 @@ module InspecPlugins::Sweeten
def render
# examine run_data and call output()
end
def self.run_data_schema_constraints
"~> 0.0" # Accept any non-breaking change
end
end
end
```
@ -378,11 +381,105 @@ end
#### Implement render()
The primary responsibility you must fulfill is to implement render. Typically, you will examine the `run_data` Hash, which is provided as an accessor. Call `output(String, newline_wanted = true)` to send output.
The primary responsibility you must fulfill is to implement render. Typically, you will examine the `run_data` structure (documented below), which is provided as an accessor. Call `output(String, newline_wanted = true)` to send output.
#### Implement self.run_data_schema_constraints
The run_data API is versioned. Your plugin should declare which version(s) it is compatible with by implementing a simple class method, which should return a GemSpec-like constraint. Currently, a simple warning is issued if a version mismatch occurs.
The run_data version scheme follows a data-oriented SemVer approach:
* bump patch: fixing a bug in the provenance or description of a data element, no key changes
* bump minor: adding new data elements
* bump major: deleting or renaming data elements
Prior to version 1.0.0, the API is considered unstable, as per SemVer. The current plan is to bump the major version to 1.0.0 when all of the existing core reporters have been migrated to plugins. It is probable that new data elements and new Hash compatibility behavior will be added during the core reporter plugin conversion process.
#### The run_data structure
The `run_data` object contains all data from the Chef InSpec run. This object is a Hash, but includes many fields and is often large. No specific documentation exists for the `run_data` object. See [the legacy JSON reporter](https://github.com/inspec/inspec/blob/2e887a94afcca819da781d4774aa2a5a0b56785e/lib/inspec/reporters/json.rb#L10) for one example of how to iterate over the object.
The `run_data` object contains all data from the Chef InSpec run. Here is an overview of this object:
| Field | Description |
|-------|-------------|
|`run_data.controls`| Array of Control records. Flattened copy of those found in |`run_data.profiles[*].controls[*]`. See there for details.|
|`run_data.platform.name`| String, name of the OS/API that the run targeted.|
|`run_data.platform.release`| String, version number of the OS/API that the run targeted.|
|`run_data.platform.target`| String, target URI|
|`run_data.profiles`| Array of Profile records|
|`run_data.profiles[0].controls`| Array of Control records|
|`run_data.profiles[0].controls[0].code`| String, source code of the control|
|`run_data.profiles[0].controls[0].desc`| String, concatenated copy of all desc tags|
|`run_data.profiles[0].controls[0].descriptions`| Hash, Symbol => String, collection of all the `desc` tags.|
|`run_data.profiles[0].controls[0].id`| String, ID (name) of the control|
|`run_data.profiles[0].controls[0].impact`| Float, severity of the control range 0.0 .. 1.0|
|`run_data.profiles[0].controls[0].refs`| Array of Ref records for `ref` tags|
|`run_data.profiles[0].controls[0].refs[0].ref`| String, human meaningful reference|
|`run_data.profiles[0].controls[0].refs[0].url`| String, URL for reference|
|`run_data.profiles[0].controls[0].results`| Array of Result records, which represent Describe blocks|
|`run_data.profiles[0].controls[0].results[0].backtrace`| Array of Strings, stacktrace if an error occurred.|
|`run_data.profiles[0].controls[0].results[0].code_desc`| String, generated complete description of the test|
|`run_data.profiles[0].controls[0].results[0].exception`| String, name of an exception thrown (if any)|
|`run_data.profiles[0].controls[0].results[0].expectation_message`| String, generated phrase like "is expected to equal 2"|
|`run_data.profiles[0].controls[0].results[0].message`| String, human-friendly text present when test has failed"|
|`run_data.profiles[0].controls[0].results[0].resource`| Undocumented and usually unpopulated; try exploring resource_title |
|`run_data.profiles[0].controls[0].results[0].resource_name`| String, name of the resource used in the test|
|`run_data.profiles[0].controls[0].results[0].resource_title`| Anonymous Class, the actual instance of the Resource. Responds to to_s with the name of the resource.|
|`run_data.profiles[0].controls[0].results[0].run_time`| Float, execution time in seconds for the test|
|`run_data.profiles[0].controls[0].results[0].skip_message`| String, if the test was skipped, explains why (user provided)|
|`run_data.profiles[0].controls[0].results[0].start_time`| DateTime, time the test started executing|
|`run_data.profiles[0].controls[0].results[0].status`| String, one of "passed", "failed", or "skipped".|
|`run_data.profiles[0].controls[0].source_location.line`| Integer, line number of source code where control begins|
|`run_data.profiles[0].controls[0].source_location.ref`| String, relative path to file in which source code resides|
|`run_data.profiles[0].controls[0].tags`| Hash, String => String, collection of `tag`s.|
|`run_data.profiles[0].controls[0].title`| String, optional title of control.|
|`run_data.profiles[0].controls[0].waiver_data.expiration_date`| DateTime, time at which the waiver expires if any|
|`run_data.profiles[0].controls[0].waiver_data.justification`| String, user-provided reason for applying the waiver|
|`run_data.profiles[0].controls[0].waiver_data.run`| Boolean, whether the control should run even if waivered|
|`run_data.profiles[0].controls[0].waiver_data.skipped_due_to_waiver`| Boolean, whether the control was marked `skipped` due to the |waiver system
|`run_data.profiles[0].controls[0].waiver_data.message`| Reserved|
|`run_data.profiles[0].copyright`| String, Copyright text from inspec.yml|
|`run_data.profiles[0].copyright_email`| String, Copyright email from inspec.yml|
|`run_data.profiles[0].depends`| Array of Dependency records|
|`run_data.profiles[0].depends[0].branch`| branch name if it was a git reference|
|`run_data.profiles[0].depends[0].commit`| commit ref if it was a git reference|
|`run_data.profiles[0].depends[0].compliance`| String, "user/profilename" on Automate server if it was an Automate reference|
|`run_data.profiles[0].depends[0].git`| location of dependent profile if it was a git reference|
|`run_data.profiles[0].depends[0].name`| name (assigned alias) of dependent profile|
|`run_data.profiles[0].depends[0].path`| location of dependent profile if it was a local reference|
|`run_data.profiles[0].depends[0].relative_path`| relative path within clone if it was a git reference|
|`run_data.profiles[0].depends[0].skip_message`| Reason if status is "skipped"|
|`run_data.profiles[0].depends[0].supermarket`| String, "user/profilename" on Supermarket server if it was a Supermarket reference|
|`run_data.profiles[0].depends[0].status`| String, one of "loaded" or "skipped"|
|`run_data.profiles[0].depends[0].tag`| tag ref if it was a git reference|
|`run_data.profiles[0].depends[0].version`| semver tag if it was a git reference|
|`run_data.profiles[0].depends[0].url`| location of dependent profile if it was a URL reference|
|`run_data.profiles[0].groups`| Array, of Group records, describing the files that contained controls|
|`run_data.profiles[0].groups[0].controls`| Array of Strings, the IDs of the controls in the file|
|`run_data.profiles[0].groups[0].id`| String, typically a relative path like `controls/myfile.rb`|
|`run_data.profiles[0].groups[0].title`| String, value of a `title` DSL command in the file|
|`run_data.profiles[0].inputs`| Array of Input records describing inputs present in profile|
|`run_data.profiles[0].inputs[0].name`| String name of Input|
|`run_data.profiles[0].inputs[0].options.required`| Boolean, whether input was required.|
|`run_data.profiles[0].inputs[0].options.type`| String, type constraint on input, if any|
|`run_data.profiles[0].inputs[0].options.value`| Value of Input|
|`run_data.profiles[0].license`| String, name of license for the profile|
|`run_data.profiles[0].maintainer`| String, name of the maintainer|
|`run_data.profiles[0].name`| String, machine name of the profile|
|`run_data.profiles[0].parent_profile`| String, name of the parent profile if this is a dependency|
|`run_data.profiles[0].sha256`| String, checksum of the profile|
|`run_data.profiles[0].skip_message`| String, message indicating why the profile was not loaded if status is "skipped"|
|`run_data.profiles[0].summary`| String, A one-line summary from the inspec.yml|
|`run_data.profiles[0].supports`| Array of Support records indicating platform support|
|`run_data.profiles[0].supports[0].platform_family`| Platform restriction by family|
|`run_data.profiles[0].supports[0].platform_name`| Platform restriction by name|
|`run_data.profiles[0].supports[0].platform`| Platform restriction by name|
|`run_data.profiles[0].supports[0].release`| Platform restriction by release|
|`run_data.profiles[0].status`| String, one of "loaded" or "skipped"|
|`run_data.statistics.controls.failed.total`| Integer, total count of failing controls|
|`run_data.statistics.controls.passed.total`| Integer, total count of passing controls|
|`run_data.statistics.controls.skipped.total`| Integer, total count of passing controls|
|`run_data.statistics.controls.total`| Integer, total count of controls|
|`run_data.statistics.duration`| Float, time in seconds for the execution of Resources.|
|`run_data.version`| A String, such as "4.18.108" representing the Chef InSpec version.|
## Implementing Input Plugins

View file

@ -1,3 +1,5 @@
require_relative "../../../run_data"
module Inspec::Plugin::V2::PluginType
class Reporter < Inspec::Plugin::V2::PluginBase
register_plugin_type(:reporter)
@ -6,8 +8,19 @@ module Inspec::Plugin::V2::PluginType
def initialize(config)
@config = config
@run_data = config[:run_data]
apply_report_resize_options unless @run_data.nil?
# Trim the run_data while still a Hash; if it is huge, this
# saves on conversion time
@run_data = config[:run_data] || {}
apply_report_resize_options
unless Inspec::RunData.compatible_schema?(self.class.run_data_schema_constraints)
# Best we can do is warn here, the InSpec run has finished
# TODO: one day, perhaps switch RunData implementations to try to satisfy constraints?
Inspec::Log.warn "Reporter does not support RunData API (#{Inspec::RunData::SCHEMA_VERSION}), Reporter constraints: '#{self.class.run_data_schema_constraints}'"
end
# Convert to RunData object for consumption by Reporter
@run_data = Inspec::RunData.new(@run_data)
@output = ""
end
@ -47,5 +60,9 @@ module Inspec::Plugin::V2::PluginType
def render
raise NotImplementedError, "#{self.class} must implement a `#render` method to format its output."
end
def self.run_data_schema_constraints
raise NotImplementedError, "#{self.class} must implement a `run_data_schema_constraints` class method to declare its compatibiltity with the RunData API."
end
end
end

64
lib/inspec/run_data.rb Normal file
View file

@ -0,0 +1,64 @@
module Inspec
module HashLikeStruct
def keys
members
end
def key?(item)
members.include?(item)
end
end
RunData = Struct.new(
:controls, # Array of Inspec::RunData::Control (flattened)
:other_checks,
:profiles, # Array of Inspec::RunData::Profile
:platform, # Inspec::RunData::Platform
:statistics, # Inspec::RunData::Statistics
:version # String
) do
include HashLikeStruct
def initialize(raw_run_data)
self.controls = raw_run_data[:controls].map { |c| Inspec::RunData::Control.new(c) }
self.profiles = raw_run_data[:profiles].map { |p| Inspec::RunData::Profile.new(p) }
self.statistics = Inspec::RunData::Statistics.new(raw_run_data[:statistics])
self.platform = Inspec::RunData::Platform.new(raw_run_data[:platform])
self.version = raw_run_data[:version]
end
end
class RunData
# This is the data layout version of RunData.
# We plan to follow a data-oriented version of semver:
# patch: fixing a bug in the provenance or description of a data element, no key changes
# minor: adding new data elements
# major: deleting or renaming data elements
# Less than major version 1.0.0, the API is considered unstable.
# The current plan is to bump the major version to 1.0.0 when all of the existing
# core reporters have been migrated to plugins. It is probable that new data elements
# and new Hash compatibility behavior will be added during the core reporter plugin
# conversion process.
SCHEMA_VERSION = "0.1.0".freeze
def self.compatible_schema?(constraints)
reqs = Gem::Requirement.create(constraints)
reqs.satisfied_by?(Gem::Version.new(SCHEMA_VERSION))
end
Platform = Struct.new(
:name, :release, :target
) do
include HashLikeStruct
def initialize(raw_plat_data)
%i{name release target}.each { |f| self[f] = raw_plat_data[f] || "" }
end
end
end
end
require_relative "run_data/result"
require_relative "run_data/control"
require_relative "run_data/profile"
require_relative "run_data/statistics"

View file

@ -0,0 +1,83 @@
module Inspec
class RunData
Control = Struct.new(
:code, # String
:desc, # String
:descriptions, # Hash with custom keys
:id, # String
:impact, # Float
:refs, # Complex local
:results, # complex standalone
:source_location, # Complex local
:tags, # Hash with custom keys
:title, # String
:waiver_data # Complex local
) do
include HashLikeStruct
def initialize(raw_ctl_data)
self.refs = (raw_ctl_data[:refs] || []).map { |r| Inspec::RunData::Control::Ref.new(r) }
self.results = (raw_ctl_data[:results] || []).map { |r| Inspec::RunData::Result.new(r) }
self.source_location = Inspec::RunData::Control::SourceLocation.new(raw_ctl_data[:source_location] || {})
self.waiver_data = Inspec::RunData::Control::WaiverData.new(raw_ctl_data[:waiver_data] || {})
[
:code, # String
:desc, # String
:descriptions, # Hash with custom keys
:id, # String
:impact, # Float
:tags, # Hash with custom keys
:title, # String
].each do |field|
self[field] = raw_ctl_data[field]
end
end
end
class Control
Ref = Struct.new(
:url, :ref
) do
include HashLikeStruct
def initialize(raw_ref_data)
%i{url ref}.each { |f| self[f] = raw_ref_data[f] }
end
end
SourceLocation = Struct.new(
:line, :ref
) do
include HashLikeStruct
def initialize(raw_sl_data)
%i{line ref}.each { |f| self[f] = raw_sl_data[f] }
end
end
# {
# "expiration_date"=>#<Date: 2077-06-01 ((2479821j,0s,0n),+0s,2299161j)>,
# "justification"=>"Lack of imagination",
# "run"=>false,
# "skipped_due_to_waiver"=>true,
# "message"=>""}
WaiverData = Struct.new(
:expiration_date,
:justification,
:run,
:skipped_due_to_waiver,
:message
) do
include HashLikeStruct
def initialize(raw_wv_data)
# These have string keys in the raw data!
%i{
expiration_date
justification
run
skipped_due_to_waiver
message
}.each { |f| self[f] = raw_wv_data[f.to_s] }
end
end
end
end
end

View file

@ -0,0 +1,109 @@
module Inspec
class RunData
Profile = Struct.new(
:controls, # complex standalone
:copyright,
:copyright_email,
:depends, # complex local
:groups, # complex local
:inputs, # complex local
:license,
:maintainer,
:name,
:sha256,
:status,
:summary,
:supports, # complex local
:parent_profile,
:skip_message,
:waiver_data, # Undocumented but used in JSON reporter - should not be?
:title,
:version
) do
include HashLikeStruct
def initialize(raw_prof_data)
self.controls = (raw_prof_data[:controls] || []).map { |c| Inspec::RunData::Control.new(c) }
self.depends = (raw_prof_data[:depends] || []).map { |d| Inspec::RunData::Profile::Dependency.new(d) }
self.groups = (raw_prof_data[:groups] || []).map { |g| Inspec::RunData::Profile::Group.new(g) }
self.inputs = (raw_prof_data[:inputs] || []).map { |i| Inspec::RunData::Profile::Input.new(i) }
self.supports = (raw_prof_data[:supports] || []).map { |s| Inspec::RunData::Profile::Support.new(s) }
%i{
copyright
copyright_email
license
maintainer
name
sha256
status
summary
title
version
parent_profile
skip_message
waiver_data
}.each do |field|
self[field] = raw_prof_data[field]
end
end
end
class Profile
# Good candidate for keyword_init, but that is not in 2.4
Dependency = Struct.new(
:name, :path, :status, :skip_message, :git, :url, :compliance, :supermarket, :branch, :tag, :commit, :version, :relative_path
) do
include HashLikeStruct
def initialize(raw_dep_data)
%i{name path status skip_message git url supermarket compliance branch tag commit version relative_path}.each { |f| self[f] = raw_dep_data[f] }
end
end
Support = Struct.new(
# snake case
:platform_family, :platform_name, :release, :platform
) do
include HashLikeStruct
def initialize(raw_sup_data)
%i{release platform}.each { |f| self[f] = raw_sup_data[f] }
self.platform_family = raw_sup_data[:"platform-family"]
self.platform_name = raw_sup_data[:"platform-name"]
end
end
# Good candidate for keyword_init, but that is not in 2.4
Group = Struct.new(
:title, :controls, :id
) do
include HashLikeStruct
def initialize(raw_grp_data)
%i{title id}.each { |f| self[f] = raw_grp_data[f] }
[:controls].each { |f| self[f] = raw_grp_data[f] || [] }
end
end
Input = Struct.new(
:name, :options
) do
include HashLikeStruct
def initialize(raw_input_data)
self.name = raw_input_data[:name]
self.options = Inspec::RunData::Profile::Input::Options.new(raw_input_data[:options])
end
end
class Input
Options = Struct.new(
# There are probably others
:value,
:type,
:required
) do
include HashLikeStruct
def initialize(raw_opts_data)
%i{value type required}.each { |f| self[f] = raw_opts_data[f] }
end
end
end
end
end
end

View file

@ -0,0 +1,40 @@
module Inspec
class RunData
Result = Struct.new(
:message, # Human-friendly test failure message
:code_desc, # Generated test description
:expectation_message, # a substring of code_desc
:resource_name, # We try to determine this
:run_time, # Float seconds execution time
:skip_message, # String
:start_time, # DateTime
:status, # String
:resource_title, # Ugly internals
# :waiver_data, # Undocumented tramp data / not exposed in this API
:resource, # Undocumented, what is this
:exception,
:backtrace
) do
include HashLikeStruct
def initialize(raw_res_data)
[
:status, # String
:code_desc, # Generated test description
:expectation_message, # a substring of code_desc
:skip_message, # String
:run_time,
:start_time,
:resource_title,
:resource,
:exception,
:backtrace,
:message,
].each do |field|
self[field] = raw_res_data[field]
end
self.resource_name = raw_res_data[:resource_title].instance_variable_get(:@__resource_name__)&.to_s
end
end
end
end

View file

@ -0,0 +1,36 @@
module Inspec
class RunData
# {:duration=>0.018407, :controls=>{:total=>3, :passed=>{:total=>3}, :skipped=>{:total=>0}, :failed=>{:total=>0}}}
Statistics = Struct.new(
:duration,
:controls
) do
include HashLikeStruct
def initialize(raw_stat_data)
self.controls = Inspec::RunData::Statistics::Controls.new(raw_stat_data[:controls])
self.duration = raw_stat_data[:duration]
end
end
class Statistics
Controls = Struct.new(
:total,
:passed,
:skipped,
:failed
) do
include HashLikeStruct
def initialize(raw_stat_ctl_data)
self.total = raw_stat_ctl_data[:total]
self.passed = Inspec::RunData::Statistics::Controls::Total.new(raw_stat_ctl_data[:passed][:total])
self.skipped = Inspec::RunData::Statistics::Controls::Total.new(raw_stat_ctl_data[:skipped][:total])
self.failed = Inspec::RunData::Statistics::Controls::Total.new(raw_stat_ctl_data[:failed][:total])
end
end
class Controls
Total = Struct.new(:total) do
include HashLikeStruct
end
end
end
end
end

View file

@ -21,5 +21,9 @@ module InspecPlugins::ReporterTestFixture
output("p#{profile_count}c#{control_count}t#{test_count}", true)
end
def self.run_data_schema_constraints
">= 0.0"
end
end
end