CFINSPEC-240 Extended file format support for waivers (#6193)

* Added separate waiver file reader and support for csv

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Added support for json format waivers

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Added support for xls and xlsx

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Build issues and updated description of gems

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Doc changes for waivers about supports

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Fix added to check final options presense in config

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Renamed variables from inputs to waivers

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Validation changes with other small changes

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Headers validation added for json and yaml

* Linter issues resolved

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Some refactoring and message change

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* exit code check removed from test cases since not req

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

* Doc change for waiver support for excel by showing example

Signed-off-by: Nikita Mathur <nikita.mathur@chef.io>

Co-authored-by: Clinton Wolfe <clintoncwolfe@gmail.com>

docs-chef-io/content/inspec/waivers.md

@@ -12,8 +12,7 @@ gh_repo = "inspec"
 +++
 
 Waivers is a mechanism to mark controls as "waived" for various reasons, and to
-control the running and/or reporting of those controls. It uses a YAML input file
-that identifies:
+control the running and/or reporting of those controls. A waiver file identifies:
 
 1. which controls are waived
 1. a description of why it is waived
@@ -31,7 +30,7 @@ inspec exec path/to/profile --waiver-file waivers.yaml
 
 ## File Format
 
-Waiver files are [input files](/inspec/inputs/) with a specific format:
+Waiver files support YAML, JSON, CSV, XLSX & XLS format.
 
 ```yaml
 control_id:
@@ -40,6 +39,18 @@ control_id:
   justification: "reason for waiving this control"
 ```
 
+OR
+
+```json
+{
+  "control_id": {
+    "expiration_date": "YYYY-MM-DD",
+    "run": false,
+    "justification": "reason for waiving this control"
+  }
+}
+```
+
 - `expiration_date` sets the day that the waiver file will expire in YYYY-MM-DD format. Waiver files expire at 00:00 at the local time of the system on the specified date. Waiver files without an expiration date are permanent. `expiration_date` is optional.
 - `run` is optional. If absent or true, the control will run and be
   reported, but failures in it won't make the overall run fail. If present and false, the control will not be run. You may use any of yes, no, true or false. To avoid confusion, it is good practice to explicitly specify whether the control should run.
@@ -48,6 +59,8 @@ control_id:
 
 ### Examples:
 
+Example in YAML:
+
 ```yaml
 waiver_control_1_2_3:
   expiration_date: 2019-10-15
@@ -58,3 +71,34 @@ xccdf_org.cisecurity.benchmarks_rule_1.1.1.4_Ensure_mounting_of_hfs_filesystems_
   justification: "This might be a bug in the test. @qateam"
   run: false
 ```
+
+Example in JSON:
+
+```json
+{
+  "waiver_control_1_2_3": {
+    "expiration_date": "2019-10-15T00:00:00.000Z",
+    "justification": "Not needed until Q3. @secteam"
+  },
+  "xccdf_org.cisecurity.benchmarks_rule_1.1.1.4_Ensure_mounting_of_hfs_filesystems_is_disabled": {
+    "expiration_date": "2020-03-01T00:00:00.000Z",
+    "justification": "This might be a bug in the test. @qateam",
+    "run": false
+  }
+}
+```
+
+Example in CSV/XLSX/XLS:
+
+These file formats support the following fields in a file:
+
+* `control_id`
+  _Required_.
+* `justification`
+  _Required_.
+* `run`
+  _Optional_.
+* `expiration_date`
+  _Optional_.
+
+![Waiver File Excel Example](/images/inspec/waivers_file_excel.png)

inspec.gemspec

@@ -34,6 +34,10 @@ Gem::Specification.new do |spec|
   # progress bar streaming reporter plugin support
   spec.add_dependency "progress_bar", "~> 1.3.3"
 
+  # roo support for reading excel waiver files
+  spec.add_dependency "roo", "~> 2.9.0"
+  spec.add_dependency "roo-xls" # extension for roo to read xls files
+
   # Used for Azure profile until integrated into train
   spec.add_dependency "faraday_middleware", ">= 0.12.2", "< 1.1"
 

lib/inspec/exceptions.rb

@@ -10,5 +10,7 @@ module Inspec
     class SecretsBackendNotFound < ArgumentError; end
     class ProfileValidationKeyNotFound < ArgumentError; end
     class ProfileSigningKeyNotFound < ArgumentError; end
+    class WaiversFileNotReadable < ArgumentError; end
+    class WaiversFileDoesNotExist < ArgumentError; end
   end
 end

lib/inspec/rule.rb

@@ -8,6 +8,8 @@ require "inspec/impact"
 require "inspec/resource"
 require "inspec/resources/os"
 require "inspec/input_registry"
+require "inspec/waiver_file_reader"
+require "inspec/utils/convert"
 
 module Inspec
   class Rule
@@ -338,17 +340,20 @@ module Inspec
     # only_if mechanism)
     # Double underscore: not intended to be called as part of the DSL
     def __apply_waivers
-      input_name = @__rule_id # TODO: control ID slugging
-      registry = Inspec::InputRegistry.instance
-      input = registry.inputs_by_profile.dig(__profile_id, input_name)
-      return unless input && input.has_value? && input.value.is_a?(Hash)
+      control_id = @__rule_id # TODO: control ID slugging
+      waiver_files = Inspec::Config.cached.final_options["waiver_file"] if Inspec::Config.cached.respond_to?(:final_options)
+
+      waiver_data_by_profile = Inspec::WaiverFileReader.fetch_waivers_by_profile(__profile_id, waiver_files) unless waiver_files.nil?
+
+      return unless waiver_data_by_profile && waiver_data_by_profile[control_id] && waiver_data_by_profile[control_id].is_a?(Hash)
 
       # An InSpec Input is a datastructure that tracks a profile parameter
       # over time. Its value can be set by many sources, and it keeps a
       # log of each "set" event so that when it is collapsed to a value,
       # it can determine the correct (highest priority) value.
       # Store in an instance variable for.. later reading???
-      @__waiver_data = input.value
+      @__waiver_data = waiver_data_by_profile[control_id]
+
       __waiver_data["skipped_due_to_waiver"] = false
       __waiver_data["message"] = ""
 
@@ -377,6 +382,7 @@ module Inspec
       # expiration_date. We only care here if it has a "run" key and it
       # is false-like, since all non-skipped waiver operations are handled
       # during reporting phase.
+      __waiver_data["run"] = Converter.to_boolean(__waiver_data["run"]) if __waiver_data.key?("run")
       return unless __waiver_data.key?("run") && !__waiver_data["run"]
 
       # OK, apply a skip.

lib/inspec/runner.rb

@@ -60,9 +60,11 @@ module Inspec
       end
 
       if @conf[:waiver_file]
-        waivers = @conf.delete(:waiver_file)
-        @conf[:input_file] ||= []
-        @conf[:input_file].concat waivers
+        @conf[:waiver_file].each do |file|
+          unless File.file?(file)
+            raise Inspec::Exceptions::WaiversFileDoesNotExist, "Waiver file #{file} does not exist."
+          end
+        end
       end
 
       # About reading inputs:

lib/inspec/utils/convert.rb

@@ -5,4 +5,12 @@ module Converter
     val = val.to_i if val =~ /^\d+$/
     val
   end
+
+  def self.to_boolean(value)
+    if ["true", "True", "TRUE", true, "yes", "y", "YES", "Y"].include? value
+      true
+    elsif ["false", "False", "FALSE", false, "no", "n", "NO", "N"].include? value
+      false
+    end
+  end
 end

lib/inspec/utils/waivers/csv_file_reader.rb

@@ -0,0 +1,34 @@
+require "csv" unless defined?(CSV)
+
+module Waivers
+  class CSVFileReader
+    def self.resolve(path)
+      return nil unless File.file?(path)
+
+      @headers ||= []
+      fetch_data(path)
+    end
+
+    def self.fetch_data(path)
+      waiver_data_hash = {}
+      CSV.foreach(path, headers: true) do |row|
+        row_hash = row.to_hash
+        @headers = row_hash.keys if @headers.empty?
+        control_id = row_hash["control_id"]
+        # delete keys and values not required in final hash
+        row_hash.delete("control_id")
+        row_hash.delete_if { |k, v| k.nil? || v.nil? }
+
+        waiver_data_hash[control_id] = row_hash if control_id && !row_hash.blank?
+      end
+
+      waiver_data_hash
+    rescue CSV::MalformedCSVError => e
+      raise "Error reading InSpec waivers in CSV: #{e}"
+    end
+
+    def self.headers
+      @headers
+    end
+  end
+end

lib/inspec/utils/waivers/excel_file_reader.rb

@@ -0,0 +1,39 @@
+require "roo"
+require "roo-xls"
+
+module Waivers
+  class ExcelFileReader
+    def self.resolve(path)
+      return nil unless File.file?(path)
+
+      @headers ||= []
+      fetch_data(path)
+    end
+
+    def self.fetch_data(path)
+      waiver_data_hash = {}
+      file_extension = File.extname(path) == ".xlsx" ? :xlsx : :xls
+      excel_file = Roo::Spreadsheet.open(path, extension: file_extension)
+      excel_file.sheet(0).parse(headers: true).each_with_index do |row, index|
+        if index == 0
+          @headers = row.keys
+        else
+          row_hash = row
+          control_id = row_hash["control_id"]
+          # delete keys and values not required in final hash
+          row_hash.delete("control_id")
+          row_hash.delete_if { |k, v| k.nil? || v.nil? }
+        end
+
+        waiver_data_hash[control_id] = row_hash if control_id && !row_hash.blank?
+      end
+      waiver_data_hash
+    rescue Exception => e
+      raise "Error reading InSpec waivers in Excel: #{e}"
+    end
+
+    def self.headers
+      @headers
+    end
+  end
+end

lib/inspec/utils/waivers/json_file_reader.rb

@@ -0,0 +1,15 @@
+module Waivers
+  class JSONFileReader
+    def self.resolve(path)
+      return nil unless File.file?(path)
+
+      fetch_data(path)
+    end
+
+    def self.fetch_data(path)
+      JSON.parse(File.read(path))
+    rescue JSON::ParserError => e
+      raise "Error reading InSpec waivers in JSON: #{e}"
+    end
+  end
+end

lib/inspec/waiver_file_reader.rb

@@ -0,0 +1,66 @@
+require "inspec/secrets/yaml"
+require "inspec/utils/waivers/csv_file_reader"
+require "inspec/utils/waivers/json_file_reader"
+require "inspec/utils/waivers/excel_file_reader"
+
+module Inspec
+  class WaiverFileReader
+
+    def self.fetch_waivers_by_profile(profile_id, files)
+      read_waivers_from_file(profile_id, files) if @waivers_data.nil? || @waivers_data[profile_id].nil?
+      @waivers_data[profile_id]
+    end
+
+    def self.read_waivers_from_file(profile_id, files)
+      @waivers_data ||= {}
+      output = {}
+
+      files.each do |file_path|
+        file_extension = File.extname(file_path)
+        data = nil
+        if [".yaml", ".yml"].include? file_extension
+          data = Secrets::YAML.resolve(file_path)
+          data = data.inputs unless data.nil?
+          validate_json_yaml(data)
+        elsif file_extension == ".csv"
+          data = Waivers::CSVFileReader.resolve(file_path)
+          headers = Waivers::CSVFileReader.headers
+          validate_headers(headers)
+        elsif file_extension == ".json"
+          data = Waivers::JSONFileReader.resolve(file_path)
+          validate_json_yaml(data)
+        elsif [".xls", ".xlsx"].include? file_extension
+          data = Waivers::ExcelFileReader.resolve(file_path)
+          headers = Waivers::ExcelFileReader.headers
+          validate_headers(headers)
+        end
+        output.merge!(data) if !data.nil? && data.is_a?(Hash)
+
+        if data.nil?
+          raise Inspec::Exceptions::WaiversFileNotReadable,
+              "Cannot find parser for waivers file '#{file_path}'. " \
+              "Check to make sure file has the appropriate extension."
+        end
+      end
+
+      @waivers_data[profile_id] = output
+    end
+
+    def self.validate_headers(headers, json_yaml = false)
+      required_fields = json_yaml ? %w{justification} : %w{control_id justification}
+      all_fields = %w{control_id justification expiration_date run}
+
+      Inspec::Log.warn "Missing column headers: #{(required_fields - headers)}" unless (required_fields - headers).empty?
+      Inspec::Log.warn "Invalid column header: Column can't be nil" if headers.include? nil
+      Inspec::Log.warn "Extra column headers: #{(headers - all_fields)}" unless (headers - all_fields).empty?
+    end
+
+    def self.validate_json_yaml(data)
+      headers = []
+      data.each_value do |value|
+        headers.push value.keys
+      end
+      validate_headers(headers.flatten.uniq, true)
+    end
+  end
+end

test/fixtures/profiles/waivers/basic/files/waivers.csv

@@ -0,0 +1,8 @@
+control_id,justification,run,expiration_date,,,
+03_waivered_no_expiry_ran_passes,Sound reasoning,TRUE,,,,
+04_waivered_no_expiry_ran_fails,Unassailable thinking,TRUE,2077-11-10T00:00:00Z,,,
+,,,,,,
+05_waivered_no_expiry_not_ran,Sheer cleverness,FALSE,,,,
+06_waivered_expiry_in_past_ran_passes,Necessity,TRUE,,,,
+14_waivered_expiry_in_future_z_not_ran,Lack of imagination,FALSE,2077-11-10T00:00:00Z,,,
+random contorl id with no data,,,,,,random data in csv!

test/fixtures/profiles/waivers/basic/files/waivers.json

@@ -0,0 +1,77 @@
+{
+  "03_waivered_no_expiry_ran_passes": {
+    "justification": "Sound reasoning",
+    "run": true
+  },
+  "04_waivered_no_expiry_ran_fails": {
+    "justification": "Unassailable thinking",
+    "run": true
+  },
+  "05_waivered_no_expiry_not_ran": {
+    "justification": "Sheer cleverness",
+    "run": false
+  },
+  "06_waivered_expiry_in_past_ran_passes": {
+    "expiration_date": "1977-06-01T00:00:00.000Z",
+    "justification": "Necessity",
+    "run": true
+  },
+  "07_waivered_expiry_in_past_ran_fails": {
+    "expiration_date": "1977-06-01T00:00:00.000Z",
+    "justification": "Whimsy",
+    "run": true
+  },
+  "08_waivered_expiry_in_past_not_ran": {
+    "expiration_date": "1977-06-01T00:00:00.000Z",
+    "justification": "Contrariness",
+    "run": false
+  },
+  "09_waivered_expiry_in_future_ran_passes": {
+    "expiration_date": "2077-06-01T00:00:00.000Z",
+    "justification": "Handwaving",
+    "run": true
+  },
+  "10_waivered_expiry_in_future_ran_fails": {
+    "expiration_date": "2077-06-01T00:00:00.000Z",
+    "justification": "Didn't feel like it",
+    "run": true
+  },
+  "11_waivered_expiry_in_future_not_ran": {
+    "expiration_date": "2077-06-01T00:00:00.000Z",
+    "justification": "Lack of imagination",
+    "run": false
+  },
+  "12_waivered_expiry_in_future_z_ran_passes": {
+    "expiration_date": "2077-11-10T00:00:00.000Z",
+    "justification": "Handwaving",
+    "run": true
+  },
+  "13_waivered_expiry_in_future_z_ran_fails": {
+    "expiration_date": "2077-11-10T00:00:00.000Z",
+    "justification": "Didn't feel like it",
+    "run": true
+  },
+  "14_waivered_expiry_in_future_z_not_ran": {
+    "expiration_date": "2077-11-10T00:00:00.000Z",
+    "justification": "Lack of imagination",
+    "run": false
+  },
+  "15_waivered_expiry_in_future_string_ran_passes": {
+    "expiration_date": "2077-06-01",
+    "justification": "Handwaving",
+    "run": true
+  },
+  "16_waivered_expiry_in_future_string_ran_fails": {
+    "expiration_date": "2077-06-01",
+    "justification": "Didn't feel like it",
+    "run": true
+  },
+  "17_waivered_expiry_in_future_string_not_ran": {
+    "expiration_date": "2077-06-01",
+    "justification": "Lack of imagination",
+    "run": false
+  },
+  "18_waivered_no_expiry_default_run": {
+    "justification": "Too lazy to specify run, which defaults to true"
+  }
+}

test/fixtures/profiles/waivers/basic/files/wrong-headers.csv

@@ -0,0 +1,8 @@
+control_id_random,justification_random,run_random,expiration_date_random,,,
+03_waivered_no_expiry_ran_passes,Sound reasoning,TRUE,,,,
+04_waivered_no_expiry_ran_fails,Unassailable thinking,TRUE,2077-11-10T00:00:00Z,,,
+,,,,,,
+05_waivered_no_expiry_not_ran,Sheer cleverness,FALSE,,,,
+06_waivered_expiry_in_past_ran_passes,Necessity,TRUE,,,,
+14_waivered_expiry_in_future_z_not_ran,Lack of imagination,FALSE,2077-11-10T00:00:00Z,,,
+random contorl id with no data,,,,,,random data in csv!

test/fixtures/profiles/waivers/basic/files/wrong-headers.json

@@ -0,0 +1,9 @@
+{
+  "03_waivered_no_expiry_ran_passes": {
+    "justification_random": "Sound reasoning",
+    "run_random": true,
+    "expiration_date_random": "1977-06-01T00:00:00.000Z"
+  },
+  "04_waivered_no_expiry_ran_fails": {
+  }
+}

test/fixtures/profiles/waivers/basic/files/wrong-headers.yaml

@@ -0,0 +1,4 @@
+03_waivered_no_expiry_ran_passes:
+  justification_random: Sound reasoning
+  run_random: true
+  expiration_date_random: 2077-11-10T00:00:00Z

test/functional/waivers_test.rb

@@ -8,7 +8,7 @@ describe "waivers" do
   let(:waivers_profiles_path) { "#{profile_path}/waivers" }
   let(:run_result)            { run_inspec_process(cmd, json: true) }
   let(:controls_by_id)        { run_result; @json.dig("profiles", 0, "controls").map { |c| [c["id"], c] }.to_h }
-  let(:cmd)                   { "exec #{waivers_profiles_path}/#{profile_name} --input-file #{waivers_profiles_path}/#{profile_name}/files/#{waiver_file}" }
+  let(:cmd)                   { "exec #{waivers_profiles_path}/#{profile_name} --waiver-file #{waivers_profiles_path}/#{profile_name}/files/#{waiver_file}" }
 
   attr_accessor :out
 
@@ -115,6 +115,110 @@ describe "waivers" do
     end
   end
 
+  describe "a fully pre-slugged control file with csv format waiver file" do
+    let(:profile_name) { "basic" }
+    let(:waiver_file) { "waivers.csv" }
+
+    # rubocop:disable Layout/AlignHash
+    {
+      "01_not_waivered_passes"                          => "passed",
+      "02_not_waivered_fails"                           => "failed",
+      "03_waivered_no_expiry_ran_passes"                => "passed",
+      "04_waivered_no_expiry_ran_fails"                 => "failed",
+      "05_waivered_no_expiry_not_ran"                   => "skipped",
+      "06_waivered_expiry_in_past_ran_passes"           => "passed",
+      "14_waivered_expiry_in_future_z_not_ran"          => "skipped",
+    }.each do |control_id, expected|
+      it "has all of the expected outcomes #{control_id}" do
+        assert_test_outcome expected, control_id
+
+        if control_id !~ /not_waivered/
+          assert_waiver_annotation control_id
+        else
+          refute_waiver_annotation control_id
+        end
+      end
+    end
+  end
+
+  describe "a fully pre-slugged control file with json format waiver file" do
+    let(:profile_name) { "basic" }
+    let(:waiver_file) { "waivers.json" }
+
+    # rubocop:disable Layout/AlignHash
+    {
+      "01_not_waivered_passes"                          => "passed",
+      "02_not_waivered_fails"                           => "failed",
+      "03_waivered_no_expiry_ran_passes"                => "passed",
+      "04_waivered_no_expiry_ran_fails"                 => "failed",
+      "05_waivered_no_expiry_not_ran"                   => "skipped",
+      "06_waivered_expiry_in_past_ran_passes"           => "passed",
+      "14_waivered_expiry_in_future_z_not_ran"          => "skipped",
+    }.each do |control_id, expected|
+      it "has all of the expected outcomes #{control_id}" do
+        assert_test_outcome expected, control_id
+
+        if control_id !~ /not_waivered/
+          assert_waiver_annotation control_id
+        else
+          refute_waiver_annotation control_id
+        end
+      end
+    end
+  end
+
+  describe "a fully pre-slugged control file with XLSX format waiver file" do
+    let(:profile_name) { "basic" }
+    let(:waiver_file) { "waivers.xlsx" }
+
+    # rubocop:disable Layout/AlignHash
+    {
+      "01_not_waivered_passes"                          => "passed",
+      "02_not_waivered_fails"                           => "failed",
+      "03_waivered_no_expiry_ran_passes"                => "passed",
+      "04_waivered_no_expiry_ran_fails"                 => "failed",
+      "05_waivered_no_expiry_not_ran"                   => "skipped",
+      "06_waivered_expiry_in_past_ran_passes"           => "passed",
+      "14_waivered_expiry_in_future_z_not_ran"          => "skipped",
+    }.each do |control_id, expected|
+      it "has all of the expected outcomes #{control_id}" do
+        assert_test_outcome expected, control_id
+
+        if control_id !~ /not_waivered/
+          assert_waiver_annotation control_id
+        else
+          refute_waiver_annotation control_id
+        end
+      end
+    end
+  end
+
+  describe "a fully pre-slugged control file with XLS format waiver file" do
+    let(:profile_name) { "basic" }
+    let(:waiver_file) { "waivers.xls" }
+
+    # rubocop:disable Layout/AlignHash
+    {
+      "01_not_waivered_passes"                          => "passed",
+      "02_not_waivered_fails"                           => "failed",
+      "03_waivered_no_expiry_ran_passes"                => "passed",
+      "04_waivered_no_expiry_ran_fails"                 => "failed",
+      "05_waivered_no_expiry_not_ran"                   => "skipped",
+      "06_waivered_expiry_in_past_ran_passes"           => "passed",
+      "14_waivered_expiry_in_future_z_not_ran"          => "skipped",
+    }.each do |control_id, expected|
+      it "has all of the expected outcomes #{control_id}" do
+        assert_test_outcome expected, control_id
+
+        if control_id !~ /not_waivered/
+          assert_waiver_annotation control_id
+        else
+          refute_waiver_annotation control_id
+        end
+      end
+    end
+  end
+
   describe "with --filter-waived-controls flag" do
     it "can execute and not hit failures" do
       inspec("exec " + "#{waivers_profiles_path}/purely-broken-controls" + " --filter-waived-controls --waiver-file #{waivers_profiles_path}/purely-broken-controls/files/waivers.yml" + " --no-create-lockfile" + " --no-color")
@@ -203,4 +307,64 @@ describe "waivers" do
       end
     end
   end
+
+  describe "with a waiver file that does not exist" do
+    let(:profile_name) { "basic" }
+    let(:waiver_file) { "no_file.yaml" }
+    it "raise file does not exist standard error" do
+      result = run_result
+      assert_includes result.stderr, "no_file.yaml does not exist"
+      assert_equal 1, result.exit_status
+    end
+  end
+
+  describe "with a waiver file with wrong headers" do
+    let(:profile_name) { "basic" }
+
+    describe "using csv file" do
+      let(:waiver_file) { "wrong-headers.csv" }
+      it "raise warnings" do
+        result = run_result
+        assert_includes result.stderr, "Missing column headers: [\"control_id\", \"justification\"]"
+        assert_includes result.stderr, "Invalid column header: Column can't be nil"
+        assert_includes result.stderr, "Extra column headers: [\"control_id_random\", \"justification_random\", \"run_random\", \"expiration_date_random\", nil]"
+      end
+    end
+
+    describe "using xlsx file" do
+      let(:waiver_file) { "wrong-headers.xlsx" }
+      it "raise warnings" do
+        result = run_result
+        assert_includes result.stderr, "Missing column headers: [\"control_id\", \"justification\"]"
+        assert_includes result.stderr, "Extra column headers: [\"control_id_random\", \"justification_random\", \"run_random\", \"expiration_date_random\"]"
+      end
+    end
+
+    describe "using xls file" do
+      let(:waiver_file) { "wrong-headers.xls" }
+      it "raise warnings" do
+        result = run_result
+        assert_includes result.stderr, "Missing column headers: [\"control_id\", \"justification\"]"
+        assert_includes result.stderr, "Extra column headers: [\"control_id_random\", \"justification_random\", \"run_random\", \"expiration_date_random\"]"
+      end
+    end
+
+    describe "using json file" do
+      let(:waiver_file) { "wrong-headers.json" }
+      it "raise warnings" do
+        result = run_result
+        assert_includes result.stderr, "Missing column headers: [\"justification\"]"
+        assert_includes result.stderr, "Extra column headers: [\"justification_random\", \"run_random\", \"expiration_date_random\"]"
+      end
+    end
+
+    describe "using yaml file" do
+      let(:waiver_file) { "wrong-headers.yaml" }
+      it "raise warnings" do
+        result = run_result
+        assert_includes result.stderr, "Missing column headers: [\"justification\"]"
+        assert_includes result.stderr, "Extra column headers: [\"justification_random\", \"run_random\", \"expiration_date_random\"]"
+      end
+    end
+  end
 end