Usage Telemetry v3 (#6012)

* Remove unused telemetry v1 code

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Sketch out basics of telemetry, with start/stop of invocation telemetry

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Data structure for run telemetry - job capture

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Add per-control and per-run feature detection

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* CHEF-4017 Telemetry job api updations (#6965)

* Added initial changes to jobs api

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Feature flag changes for telemetry

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* move base, debug and null to its own file structure

Signed-off-by: Sathish <sbabu@progress.com>

* make HTTP client post requests

Signed-off-by: Sathish <sbabu@progress.com>

* remove old logic

Signed-off-by: Sathish <sbabu@progress.com>

* make backend class as `HTTP`

Signed-off-by: Sathish <sbabu@progress.com>

* CHEF-7258 Fetch and use licensing information for telemetry (#6964)

* Added method to fetch license ids for inspec

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Added free license check for performing telemetry api call

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* move base, debug and null to its own file structure

Signed-off-by: Sathish <sbabu@progress.com>

* make HTTP client post requests

Signed-off-by: Sathish <sbabu@progress.com>

* remove old logic

Signed-off-by: Sathish <sbabu@progress.com>

* make backend class as `HTTP`

Signed-off-by: Sathish <sbabu@progress.com>

---------

Signed-off-by: Nik08 <nikita.mathur@progress.com>
Signed-off-by: Sathish <sbabu@progress.com>
Co-authored-by: Sathish <sbabu@progress.com>

* Updated control tags and desc value to be used in jobs api

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Added checks for automate run context and free license check

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* capture target mode and id

Signed-off-by: Sathish <sbabu@progress.com>

* profile doesn't need ID

Signed-off-by: Sathish <sbabu@progress.com>

* use run context to set environment data

Signed-off-by: Sathish <sbabu@progress.com>

* refactor `create_wrapper` to be localized

Signed-off-by: Sathish <sbabu@progress.com>

* change all timestamps to be UTC

Signed-off-by: Sathish <sbabu@progress.com>

* Null checks for response and corrected job api endpoint

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Fixed tag values to be sent as string in api call

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* make version as float

Signed-off-by: Sathish <sbabu@progress.com>

* add platform name

Signed-off-by: Sathish <sbabu@progress.com>

* Added control result data in jobs api payload

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Debug logs added for telemetry call

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Removed unwanted telemetry debug class

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Payload fix to pass features data only on per control basis

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Added class function to list all invoked features by feature sub system

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Using feature system to get all invoked features list to be used in jobs api

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Unit tests cases updated and fixed

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* License type check downcased

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Lint fix

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* CHEF-7265 Telemetry opt-in for CINC users (#6966)

* Enabled telemtry opt-in

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Removed old comments

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Unit test case added to validate the disabling telemetry behaviour for inspec user

Signed-off-by: Nik08 <nikita.mathur@progress.com>

---------

Signed-off-by: Nik08 <nikita.mathur@progress.com>

---------

Signed-off-by: Nik08 <nikita.mathur@progress.com>
Signed-off-by: Sathish <sbabu@progress.com>
Co-authored-by: Sathish <sbabu@progress.com>

* Product team review changes - only disable telemetry for commercial license users

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Connection failure handling for telemetry http call

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Testing fix - Remove usage of deleted library

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Telemetry test case fix - Issue caused because unit test are run without feature flag env set

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Fixed and replaced tightly coupled semver versioning regex matching test for telemetry data

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Telemery test fix to use license key from env or a dummy value if not set in env

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Added error logs in case the http call is not successful for telemetry

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Error handling for telemetry start and run calls

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Telemetry opt-in changes (#7055)

* Removed usage of feature system to enable telemetry - making it opt-in by default

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Telemetry disable check fix when no option is passed in args

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Fix in test to use license specs defined for testing

Signed-off-by: Nik08 <nikita.mathur@progress.com>

---------

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* (Restoring) CHEF-10392 load default telemetry url conditionally (#7059)

* load default telemetry url conditionally

Signed-off-by: Sathish <sbabu@progress.com>

* remove version base path
version base path is defined in jobs path already

Signed-off-by: Sathish <sbabu@progress.com>

* use `CHEF_` prefix for the ENV

Signed-off-by: Sathish <sbabu@progress.com>

---------

Signed-off-by: Sathish <sbabu@progress.com>
Co-authored-by: Sathish <sbabu@progress.com>

* Typo fix in features list

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Stub added for CI license key

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* License usage telemetry correction - not track control results (#7060)

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Changes to disable telemetry for other InSpec distros (#7065)

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Lint issue fix

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Removing disable telemetry test - breaks on CI because of commercial license usage

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* CHEF-13228 Chef licensing telemetry documentation (#7056)

* WIP chef telemetry env variable usage updated

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* WIP intro added for chef telemetry - requires edit

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Correction in opt in behaviour of telemetry

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Doc update after default opt in changes

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Doc edit from product

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Doc edit

Signed-off-by: Nik08 <nikita.mathur@progress.com>

* Edits

Signed-off-by: Ian Maddaus <ian.maddaus@progress.com>

---------

Signed-off-by: Nik08 <nikita.mathur@progress.com>
Signed-off-by: Ian Maddaus <ian.maddaus@progress.com>
Co-authored-by: Ian Maddaus <ian.maddaus@progress.com>

* Updated version pinning of chef licensing to version 1 for chef telemetry

Signed-off-by: Nik08 <nikita.mathur@progress.com>

---------

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Signed-off-by: Nik08 <nikita.mathur@progress.com>
Signed-off-by: Sathish <sbabu@progress.com>
Signed-off-by: Ian Maddaus <ian.maddaus@progress.com>
Co-authored-by: Nikita Mathur <Nik08@users.noreply.github.com>
Co-authored-by: Sathish <sbabu@progress.com>
Co-authored-by: Nik08 <nikita.mathur@progress.com>
Co-authored-by: Ian Maddaus <ian.maddaus@progress.com>
This commit is contained in:
Clinton Wolfe 2024-07-22 09:56:24 -04:00 committed by GitHub
parent 6f4425d06e
commit 29242deb7c
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 491 additions and 313 deletions

View file

@ -211,3 +211,11 @@ inspec exec <PROFILE_NAME>
``` ```
This capability is basic and you must synchronize the license servers, otherwise you may get inconsistent results. This capability is basic and you must synchronize the license servers, otherwise you may get inconsistent results.
## Licensing Telemetry service
The Chef Licensing Telemetry service gathers product activation, product usage trends and statistics, environment information, bugs, and other data related to the use of Chef InSpec.
This feature is enabled for free and trial tiers only and isn't enabled for commercial users.
For more information on the data gathered by the Licensing Telemetry service, see the [Progress Privacy Policy](https://www.progress.com/legal/privacy-policy).

View file

@ -1,6 +1,6 @@
LoJePRrMIqFz6d1uu5n3QBqQAPD8wLuLM8PfvdDerFjuX/TFJDFdwdcNZ8b8 nr7EKXZMiAwYI0Kon1ctCMkDulEkovRbT/FRezvP04yx8wVhJaSi7dMhL/mP
KBxFjR5qUTMZizjIUp5Jd6FFI4gSm0RIMKa4UeJCQQAWKJGo/tIbSKLPLWlV NvTzMOuT9G4R/QsP6VV7QKs4eBmAOPGrvgZgyfXDvfe1TPYcvpsVncSXm5rx
m1X1Z869AkvQSJxyaXvS2oKPck/znCbRKEDhuk2kqSyDJlC2BILTVa0sx3nd TO+g7i0XGz9s/FtvdzOpl2urhgOsQ35wk7IsNu9Ktij2HqZw7UmxMvtT954s
4W2J2CwFBlqmYWI1FARkZCMGlfzkjcUqrVrCb3RcZ7bcEYOT5ebIm9zZlbuV aQuW6eVvvM9n+bobEBVSErkhgvOvJ7jZyz5r0cv/uuhrayIC6V1qegod9QHa
n2Di29KFZhl8paEoGq3EYJvxEC7rVtLccei8UteNQcSOWihG61dtPGhHnpS+ uCdasmmEqglyNQYXIM7V7iNrnfuYB80or44Ewi640edHarSw8YU/Tul2Y2l/
/7RNGjrS8s4i/dQHjZlZgV6guki6EqB+DIirVek9PQ== DWeXRHsXxmuEL1wXA9ZIV6wqK0RsxaufwY6M7bqWSQ==

View file

@ -92,3 +92,6 @@
inspec-audit-logging: inspec-audit-logging:
description: Use audit logging. description: Use audit logging.
env_preview: true env_preview: true
inspec-telemetry-client:
description: Perform license usage telemetry.
env_preview: true

View file

@ -62,5 +62,6 @@ Source code obtained from the Chef GitHub repository is made available under Apa
spec.add_dependency "cookstyle" spec.add_dependency "cookstyle"
spec.add_dependency "train-core", ">= 3.11.0" spec.add_dependency "train-core", ">= 3.11.0"
spec.add_dependency "chef-licensing", ">= 0.7.5" # Minimum major version 1 is required for Chef licensing telemetry
spec.add_dependency "chef-licensing", ">= 1.0.0"
end end

View file

@ -19,7 +19,6 @@ require "inspec/rspec_extensions"
require "inspec/globals" require "inspec/globals"
require "inspec/impact" require "inspec/impact"
require "inspec/utils/telemetry" require "inspec/utils/telemetry"
require "inspec/utils/telemetry/global_methods"
require "inspec/plugin/v2" require "inspec/plugin/v2"
require "inspec/plugin/v1" require "inspec/plugin/v1"

View file

@ -57,7 +57,7 @@ class Inspec::InspecCLI < Inspec::BaseCLI
desc: "Disable loading all plugins that the user installed." desc: "Disable loading all plugins that the user installed."
class_option :enable_telemetry, type: :boolean, class_option :enable_telemetry, type: :boolean,
desc: "Allow or disable telemetry", default: false desc: "Allow or disable telemetry", default: true
require "license_acceptance/cli_flags/thor" require "license_acceptance/cli_flags/thor"
include LicenseAcceptance::CLIFlags::Thor include LicenseAcceptance::CLIFlags::Thor

View file

@ -12,6 +12,7 @@ require "inspec/dist"
require "inspec/reporters" require "inspec/reporters"
require "inspec/runner_rspec" require "inspec/runner_rspec"
require "chef-licensing" require "chef-licensing"
require "inspec/utils/telemetry"
# spec requirements # spec requirements
module Inspec module Inspec
@ -179,6 +180,7 @@ module Inspec
} }
Inspec::Log.debug "Starting run with targets: #{@target_profiles.map(&:to_s)}" Inspec::Log.debug "Starting run with targets: #{@target_profiles.map(&:to_s)}"
Inspec::Telemetry.run_starting(runner: self, conf: @conf)
load load
run_tests(with) run_tests(with)
rescue ChefLicensing::SoftwareNotEntitled rescue ChefLicensing::SoftwareNotEntitled
@ -227,6 +229,7 @@ module Inspec
@run_data = @test_collector.run(with) @run_data = @test_collector.run(with)
# dont output anything if we want a report # dont output anything if we want a report
render_output(@run_data) unless @conf["report"] render_output(@run_data) unless @conf["report"]
Inspec::Telemetry.run_ending(runner: self, run_data: @run_data, conf: @conf)
@test_collector.exit_code @test_collector.exit_code
end end

View file

@ -1,3 +1,75 @@
require "inspec/utils/telemetry/collector" require "time" unless defined?(Time.zone_offset)
require "inspec/utils/telemetry/data_series" require "chef-licensing"
require "inspec/utils/telemetry/global_methods" require_relative "telemetry/null"
require_relative "telemetry/http"
require_relative "telemetry/run_context_probe"
module Inspec
class Telemetry
@@instance = nil
@@config = nil
def self.instance
@@instance ||= determine_backend_class.new
end
def self.determine_backend_class
# Don't perform telemetry action for other InSpec distros
# Don't perform telemetry action if running under Automate - Automate does LDC tracking for us
# Don't perform telemetry action if license is a commercial license
if Inspec::Dist::EXEC_NAME != "inspec" ||
Inspec::Telemetry::RunContextProbe.under_automate? ||
license&.license_type&.downcase == "commercial"
return Inspec::Telemetry::Null
end
if Inspec::Dist::EXEC_NAME == "inspec" && telemetry_disabled?
# Issue a warning if an InSpec user is explicitly trying to opt out of telemetry using cli option
Inspec::Log.warn "Telemetry opt-out is not permissible."
end
Inspec::Log.debug "Determined HTTP instance for telemetry"
Inspec::Telemetry::HTTP
end
def self.license
Inspec::Log.debug "Fetching license context for telemetry"
@license = ChefLicensing.license_context
end
######
# These class methods make it convenient to call from anywhere within the InSpec codebase.
######
def self.run_starting(opts)
Inspec::Log.debug "Initiating telemetry for InSpec"
@@config ||= opts[:conf]
instance.run_starting(opts)
rescue StandardError => e
Inspec::Log.debug "Encountered error in Telemetry start run call -> #{e.message}"
end
def self.run_ending(opts)
@@config ||= opts[:conf]
instance.run_ending(opts)
Inspec::Log.debug "Finishing telemetry for InSpec"
rescue StandardError => e
Inspec::Log.debug "Encountered error in Telemetry end run call -> #{e.message}"
end
def self.note_feature_usage(feature_name)
instance.note_feature_usage(feature_name)
end
def self.config
@@config
end
def self.telemetry_disabled?
config.telemetry_options["enable_telemetry"].nil? ? false : !config.telemetry_options["enable_telemetry"]
end
end
end

View file

@ -0,0 +1,147 @@
# frozen_string_literal: true
require "chef-licensing"
require "securerandom" unless defined?(SecureRandom)
require "digest" unless defined?(Digest)
require_relative "../../dist"
module Inspec
class Telemetry
class Base
VERSION = 2.0
TYPE = "job"
JOB_TYPE = "InSpec"
attr_accessor :scratch
def fetch_license_ids
Inspec::Log.debug "Fetching license IDs for telemetry"
@license_keys ||= ChefLicensing.license_keys
end
def create_wrapper
Inspec::Log.debug "Initialising wrapper for telemetry"
{
version: VERSION,
createdTimeUTC: Time.now.getutc.iso8601,
environment: Inspec::Telemetry::RunContextProbe.guess_run_context,
licenseIds: fetch_license_ids,
source: "#{Inspec::Dist::EXEC_NAME}:#{Inspec::VERSION}",
type: TYPE,
}
end
def note_feature_usage(feature_name)
@scratch ||= {}
@scratch[:features] ||= []
@scratch[:features] << feature_name
end
def run_starting(_opts = {})
@scratch ||= {}
@scratch[:features] ||= []
@scratch[:run_start_time] = Time.now.getutc.iso8601
end
def run_ending(opts)
note_per_run_features(opts)
payload = create_wrapper
train_platform = opts[:runner].backend.backend.platform
payload[:platform] = train_platform.name
payload[:jobs] = [{
type: JOB_TYPE,
# Target platform info
environment: {
host: obscure(URI(opts[:runner].backend.backend.uri).host) || "unknown",
os: train_platform.name,
version: train_platform.release,
architecture: train_platform.arch || "",
id: train_platform.uuid,
},
runtime: Inspec::VERSION,
content: [], # one content == one profile
steps: [], # one step == one control
}]
opts[:run_data][:profiles].each do |profile|
payload[:jobs][0][:content] << {
name: obscure(profile[:name]),
version: profile[:version],
sha256: profile[:sha256],
maintainer: profile[:maintainer] || "",
type: "profile",
}
profile[:controls].each do |control|
payload[:jobs][0][:steps] << {
id: obscure(control[:id]),
name: "inspec-control",
description: control[:desc] || "",
target: {
mode: opts[:runner].backend.backend.backend_type,
id: opts[:runner].backend.backend.platform.uuid,
},
resources: [],
features: [],
tags: format_control_tags(control[:tags]),
}
control[:results]&.each do |resource_block|
payload[:jobs][0][:steps].last[:resources] << {
type: "inspec-resource",
name: resource_block[:resource_class],
id: obscure(resource_block[:resource_title].respond_to?(:resource_id) ? resource_block[:resource_title].resource_id : nil) || "unknown",
}
end
# Per-control features.
payload[:jobs][0][:steps].last[:features] = scratch[:features].dup
end
end
Inspec::Log.debug "Final data for telemetry upload -> #{payload}"
# Return payload object for testing
payload
end
def format_control_tags(tags)
tags_list = []
tags.each do |key, value|
tags_list << { name: key.to_s, value: (value || "").to_s }
end
tags_list
end
# Hash text if non-nil
def obscure(cleartext)
return nil if cleartext.nil?
return nil if cleartext.empty?
Digest::SHA2.new(256).hexdigest(cleartext)
end
def note_per_run_features(opts)
note_all_invoked_features
note_gem_dependency_usage(opts)
end
def note_all_invoked_features
Inspec::Feature.list_all_invoked_features.each do |feature|
Inspec::Telemetry.note_feature_usage(feature.to_s)
end
end
def note_gem_dependency_usage(opts)
unless opts[:runner].target_profiles.map do |tp|
tp.metadata.gem_dependencies + \
tp.locked_dependencies.list.map { |_k, v| v.profile.metadata.gem_dependencies }.flatten
end.flatten.empty?
Inspec::Telemetry.note_feature_usage("inspec-gem-deps-in-profiles")
end
end
end
end
end

View file

@ -1,81 +0,0 @@
require "inspec/config"
require "inspec/utils/telemetry/data_series"
require "singleton" unless defined?(Singleton)
module Inspec::Telemetry
# A Singleton collection of data series objects.
class Collector
include Singleton
attr_reader :config
def initialize
@data_series = []
@telemetry_toggled_off = false
load_config
end
# Allow loading a configuration, useful when testing.
def load_config(config = Inspec::Config.cached)
@config = config
end
# Add a data series to the collection.
# @return [True]
def add_data_series(data_series)
@data_series << data_series
end
# The loaded configuration should have a option to configure
# telemetry, if not default to false.
# @return [True, False]
def telemetry_enabled?
if @telemetry_toggled_off
false
else
config_telemetry_options.fetch("enable_telemetry", false)
end
end
# A way to disable the telemetry system.
def disable_telemetry
@telemetry_toggled_off = true
end
# The entire data series collection.
# @return [Array]
def list_data_series
@data_series
end
# Finds the data series object with the specified name and returns it.
# If it does not exist then creates a new data series with that name
# and returns it.
# @return [Inspec::Telemetry::DataSeries]
def find_or_create_data_series(name)
ds = @data_series.select { |data_series| data_series.name.eql?(name) }
if ds.empty?
new_data_series = Inspec::Telemetry::DataSeries.new(name)
@data_series << new_data_series
new_data_series
else
ds.first
end
end
# Blanks the contents of the data series collection.
# Reset telemetry toggle
# @return [True]
def reset!
@data_series = []
@telemetry_toggled_off = false
end
private
# Minimize exposure of Inspec::Config interface
def config_telemetry_options
config.telemetry_options
end
end
end

View file

@ -1,44 +0,0 @@
require "json" unless defined?(JSON)
module Inspec; end
# A minimal Dataseries Object
# Stores the name of the data series and an array of data.
# Stored data should be a object that supports #to_s
module Inspec::Telemetry
class DataSeries
def initialize(name)
@name = name
@enabled = true
@data ||= []
end
attr_reader :data, :name
# This needs to also be set by configuration.
def enabled?
@enabled
end
def disable
@enabled = false
end
def <<(appending_data)
data << appending_data
end
alias push <<
def to_h
{
name: @name,
data: @data,
}
end
def to_json
to_h.to_json
end
end
end

View file

@ -1,22 +0,0 @@
require "inspec/utils/telemetry/collector"
module Inspec
# A Global method to add a data series object to the Telemetry Collection.
# `data_series_name`s are unique, so `:dependency_group` will always return
# the same object.
# `data_point` is optional, you may also supply a block with several data points.
# All data points should allow #to_s
def self.record_telemetry_data(data_series_name, data_point = nil)
coll = Inspec::Telemetry::Collector.instance
return unless coll.telemetry_enabled?
ds = coll.find_or_create_data_series(data_series_name)
return unless ds.enabled?
if block_given?
ds << yield
else
ds << data_point
end
end
end

View file

@ -0,0 +1,40 @@
# frozen_string_literal: true
require_relative "base"
require "faraday" unless defined?(Faraday)
require "inspec/utils/licensing_config"
module Inspec
class Telemetry
class HTTP < Base
TELEMETRY_JOBS_PATH = "v1/job"
TELEMETRY_URL = if ChefLicensing::Config.license_server_url&.match?("acceptance")
ENV["CHEF_TELEMETRY_URL"]
else
"https://services.chef.io/telemetry/"
end
def run_ending(opts)
payload = super
response = connection.post(TELEMETRY_JOBS_PATH) do |req|
req.body = payload.to_json
end
if response.success?
Inspec::Log.debug "HTTP connection with Telemetry Client successful."
Inspec::Log.debug "HTTP response from Telemetry Client -> #{response.to_hash}"
true
else
Inspec::Log.debug "HTTP connection with Telemetry Client faced an error."
Inspec::Log.debug "HTTP error -> #{response.to_hash[:body]["error"]}" if response.to_hash[:body] && response.to_hash[:body]["error"]
false
end
rescue Faraday::ConnectionFailed
Inspec::Log.debug "HTTP connection failure with telemetry url -> #{TELEMETRY_URL}"
end
def connection
Faraday.new(url: TELEMETRY_URL) do |config|
config.request :json
config.response :json
end
end
end
end
end

View file

@ -0,0 +1,11 @@
# frozen_string_literal: true
require_relative "base"
module Inspec
class Telemetry
class Null < Base
def run_starting(_opts); end
def run_ending(_opts); end
end
end
end

View file

@ -1,9 +1,21 @@
module Inspec module Inspec
module Telemetry class Telemetry
# Guesses the run context of InSpec - how were we invoked? # Guesses the run context of InSpec - how were we invoked?
# All stack values here are determined experimentally # All stack values here are determined experimentally
class RunContextProbe class RunContextProbe
# Guess if we are running under Automate
def self.under_automate?
# Currently assume we are under automate if we have an automate-based reporter
Inspec::Config.cached[:reporter]
.keys
.map(&:to_s)
.any? { |n| n =~ /automate/ }
end
# Guess, using stack introspection, if we were called under
# test-kitchen, cli, audit-cookbook, or otherwise.
# TODO add compliance-phase of chef-infra
def self.guess_run_context(stack = nil) def self.guess_run_context(stack = nil)
stack ||= caller_locations stack ||= caller_locations
return "test-kitchen" if kitchen?(stack) return "test-kitchen" if kitchen?(stack)

File diff suppressed because one or more lines are too long

View file

@ -0,0 +1,57 @@
{
"data":{
"cache": {
"lastModified": "2023-01-16T12:05:40Z",
"evaluatedOn": "2023-01-16T12:07:20.114370692Z",
"expires": "2023-01-17T12:07:20.114370783Z",
"cacheControl": "private,max-age:42460"
},
"client": {
"license": "Free",
"status": "Active",
"changesTo": "Grace",
"changesOn": "2024-11-01",
"changesIn": "2 days",
"usage": "Active",
"used": 2,
"limit": 2,
"measure": 2
},
"assets": [
{
"id": "assetguid1",
"name": "Test Asset 1"
},
{
"id": "assetguid2",
"name": "Test Asset 2"
}
],
"features": [
{
"id": "featureguid1",
"name": "Test Feature 1"
},
{
"id": "featureguid2",
"name": "Test Feature 2"
}
],
"entitlement": {
"id": "3ff52c37-e41f-4f6c-ad4d-365192205968",
"name": "Inspec",
"start": "2022-11-01",
"end": "2024-11-01",
"licenses": 2,
"limits": [
{
"measure": "nodes",
"amount": 2
}
],
"entitled": false
}
},
"message": "",
"status_code": 200
}

View file

@ -1,60 +0,0 @@
require "inspec/utils/telemetry"
require "helper"
class TestTelemetryCollector < Minitest::Test
def setup
@collector = Inspec::Telemetry::Collector.instance
@collector.reset!
end
def test_collector_singleton
assert_equal Inspec::Telemetry::Collector.instance, @collector
end
def test_add_data_series
assert_empty @collector.list_data_series
assert @collector.add_data_series(Inspec::Telemetry::DataSeries.new("/resource/File"))
refute_empty @collector.list_data_series
end
def test_list_data_series
assert_empty @collector.list_data_series
@collector.add_data_series(Inspec::Telemetry::DataSeries.new("/resource/File"))
@collector.add_data_series(Inspec::Telemetry::DataSeries.new(:deprecation_group))
assert_equal 2, @collector.list_data_series.count
assert_equal 1, @collector.list_data_series.select { |d| d.name.eql?(:deprecation_group) }.count
assert_kind_of Array, @collector.list_data_series
assert_kind_of Inspec::Telemetry::DataSeries, @collector.list_data_series.first
end
def test_find_or_create_data_series
dg = @collector.find_or_create_data_series(:deprecation_group)
assert_kind_of Inspec::Telemetry::DataSeries, dg
assert_equal :deprecation_group, dg.name
assert_equal @collector.find_or_create_data_series(:deprecation_group), dg
end
def test_reset_singleton
data_series = Inspec::Telemetry::DataSeries.new("/resource/File")
@collector.add_data_series(data_series)
@collector.reset!
assert_equal 0, @collector.list_data_series.count
end
def test_telemetry_enabled
@collector.load_config(Inspec::Config.mock("enable_telemetry" => true))
assert @collector.telemetry_enabled?
end
def test_telemetry_disabled
@collector.load_config(Inspec::Config.mock("enable_telemetry" => false))
refute @collector.telemetry_enabled?
end
def test_disable_telemetry
@collector.load_config(Inspec::Config.mock("enable_telemetry" => true))
assert @collector.telemetry_enabled?
@collector.disable_telemetry
refute @collector.telemetry_enabled?
end
end

View file

@ -1,60 +0,0 @@
require "inspec/utils/telemetry"
require "json"
require "helper"
class TestTelemetryDataSeries < Minitest::Test
def test_name
ds = Inspec::Telemetry::DataSeries.new("fizz")
refute_nil ds
assert_equal "fizz", ds.name
end
def test_data
ds = Inspec::Telemetry::DataSeries.new("fizz")
refute_nil ds.data
assert_kind_of Array, ds.data
assert_empty ds.data
end
def test_data_append
ds = Inspec::Telemetry::DataSeries.new("fizz")
assert_empty ds.data
assert ds << "foo"
assert_equal ["foo"], ds.data
end
def test_data_push_alias
ds = Inspec::Telemetry::DataSeries.new("fizz")
assert_empty ds.data
assert ds.push "bar"
assert_equal ["bar"], ds.data
end
def test_to_h
ds = Inspec::Telemetry::DataSeries.new("fizz")
ds << "foo"
assert_kind_of Hash, ds.to_h
assert_equal "fizz", ds.to_h[:name]
assert_equal ["foo"], ds.to_h[:data]
end
def test_to_json
ds = Inspec::Telemetry::DataSeries.new("fizz")
ds << "foo"
assert_kind_of String, ds.to_json
assert_equal '{"name":"fizz","data":["foo"]}', ds.to_json
assert JSON.parse(ds.to_json)
end
def test_enabled
ds = Inspec::Telemetry::DataSeries.new("fizz")
assert ds.enabled?
end
def test_disable
ds = Inspec::Telemetry::DataSeries.new("fizz")
assert ds.enabled?
ds.disable
refute ds.enabled?
end
end

View file

@ -1,33 +0,0 @@
require "inspec/utils/telemetry"
require "helper"
class TestTelemetryGlobalMethods < Minitest::Test
def setup
@collector = Inspec::Telemetry::Collector.instance
@collector.load_config(Inspec::Config.mock("enable_telemetry" => true))
@collector.reset!
end
def test_record_telemetry_data
assert Inspec.record_telemetry_data(:deprecation_group, "serverspec_compat")
depgrp = @collector.find_or_create_data_series(:deprecation_group)
assert_equal ["serverspec_compat"], depgrp.data
assert_equal :deprecation_group, depgrp.name
end
def test_record_telemetry_data_with_block
Inspec.record_telemetry_data(:deprecation_group) do
"serverspec_compat"
end
depgrp = @collector.find_or_create_data_series(:deprecation_group)
assert_equal ["serverspec_compat"], depgrp.data
assert_equal :deprecation_group, depgrp.name
end
def test_telemetry_disabled
@collector.load_config(Inspec::Config.mock(telemetry: false))
refute Inspec.record_telemetry_data(:deprecation_group, "serverspec_compat")
end
end

View file

@ -0,0 +1,124 @@
require_relative "../../helper"
require_relative "../../../lib/inspec/utils/telemetry"
require_relative "../../../lib/inspec/runner"
module Inspec
class Telemetry::Mock < Telemetry::Base
attr_reader :run_ending_payload
def run_ending(opts)
@run_ending_payload = super(opts)
end
end
end
REGEX = {
version: /^(\d+|\d+\.\d+|\d+\.\d+\.\d+)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$/,
datetime: /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2}(?:\.\d*)?)((-(\d{2}):(\d{2})|Z)?)$/,
uuid: /^[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}$/,
transport: /^[a-z0-9\-\_]+$/,
sha256: /^[0-9a-fA-F]{64}|unknown$/,
}.freeze
describe "Telemetry" do
let(:conf) { Inspec::Config.new({ "enable_telemetry" => false }) }
let(:runner) { Inspec::Runner.new({ command_runner: :generic, reporter: [], conf: conf }) }
let(:run_data) { JSON.parse(File.read("test/fixtures/reporters/run_data_test_profile_a.json"), symbolize_names: true) }
let(:repo_path) { File.expand_path("../../..", __dir__) }
let(:mock_path) { File.join(repo_path, "test", "fixtures") }
let(:valid_client_api_data) { File.read("#{repo_path}/test/fixtures/valid_client_api_data.json") }
let(:profile_path) { File.join(mock_path, "profiles") }
let(:profile) { File.join(profile_path, "dependencies", "profile_a") }
let(:tm) { Inspec::Telemetry::Mock.new }
let(:chef_license_key) { "free-42727540-ddc8-4d4b-0000-80662e03cd73-0000" }
before do
stub_request(:get, "#{ChefLicensing::Config.license_server_url}/v1/listLicenses")
.to_return(
body: {
"data": [chef_license_key],
"message": "",
"status_code": 200,
}.to_json,
headers: { content_type: "application/json" }
)
stub_request(:get, "#{ChefLicensing::Config.license_server_url}/v1/client")
.with(query: { licenseId: chef_license_key, entitlementId: ChefLicensing::Config.chef_entitlement_id })
.to_return(
body: valid_client_api_data ,
headers: { content_type: "application/json" }
)
stub_request(:get, "#{ChefLicensing::Config.license_server_url}/v1/client")
.with(query: { licenseId: [chef_license_key, ENV["CHEF_LICENSE_KEY"]].join(","), entitlementId: ChefLicensing::Config.chef_entitlement_id })
.to_return(
body: valid_client_api_data ,
headers: { content_type: "application/json" }
)
stub_request(:get, "#{ChefLicensing::Config.license_server_url}/v1/client")
.with(query: { licenseId: [ENV["CHEF_LICENSE_KEY"], chef_license_key].join(","), entitlementId: ChefLicensing::Config.chef_entitlement_id })
.to_return(
body: valid_client_api_data ,
headers: { content_type: "application/json" }
)
end
describe "when it runs with a nested profile" do
it "sets the wrapper fields" do
ChefLicensing::Context.license = ChefLicensing.client(license_keys: [chef_license_key])
Inspec::Telemetry.expects(:instance).returns(tm).at_least_once
Inspec::Telemetry.run_ending(runner: runner, run_data: run_data, conf: conf)
runner.add_target(profile)
runner.run
_(tm.run_ending_payload).wont_be_empty
_(tm.run_ending_payload).must_be_kind_of Hash
_(tm.run_ending_payload[:source]).must_match(/^inspec:\d+\.\d+\.\d+$/)
_(tm.run_ending_payload[:licenseIds]).wont_be_empty
_(tm.run_ending_payload[:createdTimeUTC]).must_match(REGEX[:datetime])
_(tm.run_ending_payload[:type]).must_match(/^job$/)
end
it "sets the job fields" do
ChefLicensing::Context.license = ChefLicensing.client(license_keys: [chef_license_key])
Inspec::Telemetry.expects(:instance).returns(tm).at_least_once
Inspec::Telemetry.run_ending(runner: runner, run_data: run_data, conf: conf)
runner.add_target(profile)
runner.run
j = tm.run_ending_payload[:jobs][0]
_(j).wont_be_empty
_(j).must_be_kind_of Hash
_(j[:type]).must_equal("InSpec")
_(j[:environment][:host]).must_match(/^\S+$/)
_(j[:environment][:os]).must_match(/^\S+$/)
_(j[:environment][:version]).must_match(REGEX[:version]) # looser version matching
_(j[:environment][:architecture]).wont_be_empty
_(j[:environment][:id]).must_match(REGEX[:uuid])
_(j[:content]).must_be_kind_of Array
_(j[:content].count).must_equal 2
j[:content].each do |c|
_(c[:name]).wont_be_empty
_(c[:version]).must_match(REGEX[:version])
_(c[:sha256]).must_match(REGEX[:sha256])
_(c[:maintainer]).wont_be_empty
end
_(j[:steps]).must_be_kind_of Array
_(j[:steps].count).must_equal 4
j[:steps].each do |s|
_(s[:name]).must_equal "inspec-control"
_(s[:id]).must_match(REGEX[:sha256])
_(s[:resources]).must_be_kind_of Array
_(s[:features]).wont_be_empty
_(s[:tags]).wont_be_empty
s[:resources].each do |r|
_(r[:type]).must_equal "inspec-resource"
_(r[:name]).wont_be_empty
_(r[:id]).must_match(REGEX[:sha256])
end
end
end
end
end