inspec/docs-chef-io
Sonu Saha a4cd3553eb
Backport: CHEF-6437: Implement different version of inspec export (#6816) (Inspec 5) (#6849)
* CHEF-6437: Implement different version of `inspec export` (#6816)

* Failing test for export - should not evaluate

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Sketch out a info_from_parse method

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Temporary commit to checkpoint experimental work

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Basic control ids extraction

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Modify to capture entire block

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Ability to parse desc, impact and title of a control (#6662)

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Rework per-control metadata collectors to be class-based

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* REFACTOR: make a common base class for collectors

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* memoise `info_from_parse`

Signed-off-by: Sathish <sbabu@progress.com>

* Add --legacy-export option to inspec export (#6661)

* support legacy export option

Signed-off-by: Sathish <sbabu@progress.com>

* ability to run legacy export option

Signed-off-by: Sathish <sbabu@progress.com>

---------

Signed-off-by: Sathish <sbabu@progress.com>

* Improve ControlIDCollector and other fields of export data (#6686)

* Parse tags & refs from the ast nodes

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ENHANCE: Improve Desc collector to collect description

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ENHANCE: Only loop through the child node of begin block

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Fix bug/todo to handle duplicacy of control ids

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST - a profile which fails to properly be exported but is likely to be used by MITRE

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Revert "FIX: Fix bug/todo to handle duplicacy of control ids"

This reverts commit 46d66e0026.

* Revert "ENHANCE: Only loop through the child node of begin block"

This reverts commit 47c92d8746.

* ADD: Add code key in control data

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ADD: Add source_location key in controls data

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* HACK: Update the location ref for the controls

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Update variable name as latest changes

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Fix source location ref for all controls in a file

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Improve tagcollector to handle other data types

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Improve tagcollector to handle different types of tags

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ENHANCE & TEST: Improve tag collector to collector different tag styles and add test for it

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* update groups

Signed-off-by: Sathish <sbabu@progress.com>

* Add yml data to export info_from_parse

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add inputs to export data info_from_parse

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add status and status_messages

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Initialize all control fields

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* WIP: Filter controls using --controls

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add inputs collector class - rules remaining

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Parse inputs from dsl - 1

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST: Uncomment tests to verify export

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST: Include test for different desc

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST: Include test for different title

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST: Include test for different ref

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Default impact to 0.5 and add test

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Avoid duplicate inputs

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add test for inputs

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* REFACTOR: Minor refactoring of tests

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Uncomment test for refs

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

---------

Signed-off-by: Sonu Saha <sonu.saha@progress.com>
Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Signed-off-by: Sathish <sbabu@progress.com>
Co-authored-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Co-authored-by: Sathish <sbabu@progress.com>

* Update option to match inspec's coding standard

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Handle inputs within control block

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST & ENHANCE: Enhance parser and add more tests

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Fix broken test for profile_test

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Update groups after filtering control

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add --legacy-export support to inspec json

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST: Fix broken test & fix group filters

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* DOCS: Manually update cli.md to include export cmd

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add tag filtering support to export

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST: Add test for tag and control based filtering

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* LINT: Fix lint offense

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* CHORE: Remove addressed todo and update comments

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* CHEF-6493: Support `--legacy-export` option in `inspec archive` (#6829)

* Introduce --legacy-export flag to archive command

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add more test to verify --legacy-export with archive

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Update logic to fetch info based on --legacy-export flag

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

---------

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Enhance InputCollector to match pattern instead of to indexing children type to avoid nil errors

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Improve RefCollector to handle ref   ({:ref=>'Some ref', :url=>'https://'\}\) syntax

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Improve RefCollector and TagCollector to handle variables values from inputs/attributes

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Run inspec check using output info_from_parse (#6673)

* Add test fixture profile that emits evaluation markers on stderr

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Failing test for export - should not evaluate

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Sketch out a info_from_parse method

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Temporary commit to checkpoint experimental work

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Basic control ids extraction

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Modify to capture entire block

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Ability to parse desc, impact and title of a control (#6662)

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Rework per-control metadata collectors to be class-based

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* REFACTOR: make a common base class for collectors

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* memoise `info_from_parse`

Signed-off-by: Sathish <sbabu@progress.com>

* Add --legacy-export option to inspec export (#6661)

* support legacy export option

Signed-off-by: Sathish <sbabu@progress.com>

* ability to run legacy export option

Signed-off-by: Sathish <sbabu@progress.com>

---------

Signed-off-by: Sathish <sbabu@progress.com>

* Parse tags & refs from the ast nodes

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ENHANCE: Improve Desc collector to collect description

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ENHANCE: Only loop through the child node of begin block

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Fix bug/todo to handle duplicacy of control ids

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* TEST - a profile which fails to properly be exported but is likely to be used by MITRE

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

* Revert "FIX: Fix bug/todo to handle duplicacy of control ids"

This reverts commit 46d66e0026.

* Revert "ENHANCE: Only loop through the child node of begin block"

This reverts commit 47c92d8746.

* ADD: Add code key in control data

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ADD: Add source_location key in controls data

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* HACK: Update the location ref for the controls

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Update variable name as latest changes

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Fix source location ref for all controls in a file

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Improve tagcollector to handle other data types

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* FIX: Improve tagcollector to handle different types of tags

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* ENHANCE & TEST: Improve tag collector to collector different tag styles and add test for it

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* update groups

Signed-off-by: Sathish <sbabu@progress.com>

* Add yml data to export info_from_parse

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add inputs to export data info_from_parse

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Add status and status_messages

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* Initialize all control fields

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* make description `default` as a symbol

Signed-off-by: Sathish Babu <sbabu@progress.com>

* define `checks` as Set

Signed-off-by: Sathish Babu <sbabu@progress.com>

* Collect tests as part of collector
and store it in `checks`

Signed-off-by: Sathish Babu <sbabu@progress.com>

* refactor to read `ID` from controls which is an Array now unlike an Hash in `params.controls`

Signed-off-by: Sathish Babu <sbabu@progress.com>

* read yaml params from metadata

Signed-off-by: Sathish Babu <sbabu@progress.com>

* use to Array to simply DS as the o/p ie being converted to JSON

Signed-off-by: Sathish Babu <sbabu@progress.com>

* move old check as legacy check

Signed-off-by: Sathish Babu <sbabu@progress.com>

* support `legacy_check` as an option to run checks in legacy mode

Signed-off-by: Sathish Babu <sbabu@progress.com>

* fix tests to support `legacy_checks`

Signed-off-by: Sathish Babu <sbabu@progress.com>

* update document for check

Signed-off-by: Sathish Babu <sbabu@progress.com>

* Update usage doc for --legaccy-check

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>

---------

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Signed-off-by: Sonu Saha <sonu.saha@progress.com>
Signed-off-by: Sathish <sbabu@progress.com>
Signed-off-by: Sathish Babu <sbabu@progress.com>
Co-authored-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Co-authored-by: Sonu Saha <98935583+ahasunos@users.noreply.github.com>
Co-authored-by: Sonu Saha <sonu.saha@progress.com>

* LINT: Fix lint offense

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

* do not include tests to controls by default

Signed-off-by: Sathish Babu <sbabu@progress.com>

* generate info with tests for check

Signed-off-by: Sathish Babu <sbabu@progress.com>

---------

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Signed-off-by: Sonu Saha <sonu.saha@progress.com>
Signed-off-by: Sathish <sbabu@progress.com>
Signed-off-by: Sathish Babu <sbabu@progress.com>
Co-authored-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Co-authored-by: Sathish <sbabu@progress.com>
Co-authored-by: Sathish Babu <80091550+sathish-progress@users.noreply.github.com>

* LINT: Fix lint offense

Signed-off-by: Sonu Saha <sonu.saha@progress.com>

---------

Signed-off-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Signed-off-by: Sonu Saha <sonu.saha@progress.com>
Signed-off-by: Sathish <sbabu@progress.com>
Signed-off-by: Sathish Babu <sbabu@progress.com>
Co-authored-by: Clinton Wolfe <clintoncwolfe@gmail.com>
Co-authored-by: Sathish <sbabu@progress.com>
Co-authored-by: Sathish Babu <80091550+sathish-progress@users.noreply.github.com>
2023-11-08 13:33:08 -05:00
..
archetypes Use shortcodes in text, move shortcodes into inspec subdir 2022-06-10 16:47:46 -04:00
content/inspec Backport: CHEF-6437: Implement different version of inspec export (#6816) (Inspec 5) (#6849) 2023-11-08 13:33:08 -05:00
layouts/shortcodes Use shortcodes in text, move shortcodes into inspec subdir 2022-06-10 16:47:46 -04:00
netlify_production CFINSPEC-44 : Add missing <title> tag to index.html 2022-02-15 20:03:26 +05:30
static/images/inspec CFINSPEC-240 Extended file format support for waivers: JSON & CSV (#6369) 2023-01-17 07:27:47 -05:00
tools/vale Add vale config to inspec repository 2022-05-17 10:44:59 -04:00
.gitignore Clean up docs previews 2020-12-10 18:53:19 -08:00
.vale.ini Add vale config to inspec repository 2022-05-17 10:44:59 -04:00
config.toml replaced /main/ from /master/ 2021-09-27 17:24:28 +05:30
go.mod Move docs from www to docs-chef-io 2020-09-14 13:53:40 -07:00
main.go Move docs from www to docs-chef-io 2020-09-14 13:53:40 -07:00
Makefile Move shortcodes from chef-web-docs to inspec repo 2022-04-04 15:44:46 -04:00
netlify.toml Remove Hugo version from Netlify config 2022-05-23 12:14:56 -04:00
README.md Update docs readme 2022-06-10 16:47:46 -04:00

Chef InSpec Documentation

This is the home of the InSpec Documentation that is deployed on https://docs.chef.io/inspec/ using Hugo modules.

Updating InSpec Docs Content

There are two steps to updating the Chef InSpec documentation on docs.chef.io:

  1. Update the documentation in the inspec/inspec repository.
  2. Update the InSpec repository module in chef-web-docs.

Documentation Content

Please keep all of the InSpec documentation in the content/inspec directory.

The diagram below shows how documentation content should be organized:

.
├── docs-chef-io
│   ├── archetypes                 # page templates
│   ├── content
│   │   ├── inspec                 # InSpec documentation
│   │   │   ├── resources          # InSpec resource documentation
│   ├── static
│   |   ├── images
│   │   |   ├── inspec             # images
│   ├── layouts
│   |   ├── shortcodes             # reusable text files

To add a new Markdown page, run the following command from the docs-chef-io directory:

hugo new content/inspec/FILENAME.md

This will create a draft page with enough front matter to get you going.

Hugo uses Goldmark which is a superset of Markdown that includes GitHub styled tables, task lists, and definition lists.

See our Style Guide for more information about formatting documentation using Markdown.

Page Front Matter

At the top of each page is a block of front matter that Hugo uses to add a title to each page and locate each page in the left navigation menu in <docs.chef.io>. Below is an example of a page's front matter:

+++
title = "PAGE TITLE"
draft = false
gh_repo = "inspec"

[menu]
  [menu.inspec]
    title = "PAGE TITLE"
    identifier = "inspec/PAGE TITLE"
    parent = "inspec"
    weight = WEIGHT
+++

title is the page title.

gh_repo = "inspec" adds an "[edit on GitHub]" link to the top of each page that links to the Markdown page in the inspec/inspec repository.

[menu.inspec] places the page in the Chef InSpec section of the left navigation menu in <docs.chef.io>. All the parameters following this configure the link in the left navigation menu to the page.

title is the title of page in the Chef InSpec menu.

identifier is the identifier for a page in the menu. It should formatted like this: inspec/RESOURCE NAME. Change PLATFORM to the same value as the platform parameter above. Change RESOURCE NAME to the name of the resource.

parent is the identifier for the section of the menu that the page will be found in. This value is set in the chef-web-docs menu config. Set this value to the section of the InSpec menu that you want the page to located in.

weight sorts the page in the menu relative to other pages. Higher weights are lower in the menu. Pages without a weight are sorted alphabetically by page title.

Resource Pages

All resource pages are located in docs-chef-io/content/inspec/resources/.

InSpec Resources Index Page

The InSpec resources index page is located in docs-chef-io/content/inspec/resources/_index.md and can be found on https://docs.chef.io/inspec/resources/.

The resources on this page are organized by platform. All resources in a platform are added to this page using the inspec_resources shortcode. To add a new group of resources of a specific platform, add the shortcode and specify the platform parameter: {{< inspec_resources platform="PLATFORM" >}}

The inspec_resources shortcode searches for the platform parameter in all pages in the content/inspec/resources directory and then creates a list of those pages that have the specified parameter.

New Resource Pages

Create a new resource page by running hugo new -k resource inspec/resources/RESOURCE_NAME.md from the docs-chef-io directory.

This will create a new resource page with enough content preformatted to get you started.

Resource Page Front Matter

At the top of each resource is a block of front matter that Hugo uses to add a title to each page and locate each page in the left navigation menu in <docs.chef.io>. Below is an example of a page's front matter:

+++
title = "RESOURCE NAME resource"
platform = "PLATFORM"
gh_repo = "inspec"

[menu.inspec]
title = "RESOURCE NAME"
identifier = "inspec/resources/PLATFORM/RESOURCE NAME"
parent = "inspec/resources/PLATFORM"
+++

title is the page title.

platform = PLATFORM sorts each page into the correct category in the Chef InSpec resources list page. Specify the platform that this page should be listed under in the resources list page.

gh_repo = "inspec" adds an "[edit on GitHub]" link to the top of each page that links to the Markdown page in the inspec/inspec repository.

[menu.inspec] places the page in the Chef InSpec section of the left navigation menu in <docs.chef.io>. All the parameters following this configure the link in the left navigation menu to the page.

title is the title of page in the Chef InSpec menu.

identifier is the identifier for a page in the menu. It should formatted like this: inspec/resources/PLATFORM/RESOURCE NAME. Change PLATFORM to the same value as the platform parameter above. Change RESOURCE NAME to the name of the resource.

parent = "inspec/resources/PLATFORM" is the identifier for the section of the menu that the page will be found in. This value is set in the chef-web-docs menu config. Change PLATFORM to the same value as the platform parameter.

Shortcodes

A shortcode is a file with a block of text that can be added in multiple places in our documentation by referencing the shortcode file name.

There are two types of shortcodes: Markdown and HTML.

Add a Markdown shortcode to a page by wrapping the filename, without the .md file suffix, in double curly braces and percent symbols. For example: {{% inspec_filter_table %}}.

Add an HTML shortcode to a page by wrapping the filename, without the .html file suffix, in double curly braces and greater than and lesser than symbols. For example: {{< shortcode_name >}}.

Some shortcodes take parameters; for example, the inspec_resources shortcode requires a platform parameter: {{< inspec_resources platform="PLATFORM" >}}

Local Development Environment

We use Hugo, Go, andNPM to build the Chef Documentation website. You will need Hugo 0.83.1 or higher installed and running to build and view our documentation properly.

To install Hugo, NPM, and Go on Windows and macOS:

  • On macOS run: brew install hugo node go
  • On Windows run: choco install hugo-extended nodejs golang

To install Hugo on Linux, run:

  • apt install -y build-essential
  • snap install node --classic --channel=12
  • snap install hugo --channel=extended

Preview InSpec Documentation

There are two ways to preview the documentation in inspec:

  • submit a PR
  • make serve

Submit a PR

When you submit a PR to inspec/inspec, Netlify will build the documentation and add a notification to the GitHub pull request page. You can review your documentation changes as they would appear on docs.chef.io.

make serve

Running make serve will clone a copy of chef/chef-web-docs into docs-chef-io. That copy will be configured to build the InSpec documentation from docs-chef-io and live reload if any changes are made while the Hugo server is running.

Clean Your Local Environment

If you have a local copy of chef-web-docs cloned into docs-chef-io, running make clean_all will delete the cloned copy of chef-web-docs. Hugo will rebuild everything from scratch the next time you run make serve.

Update the InSpec Docs Content on doc.chef.io

We use Hugo modules to build Chef's documentation from multiple repositories. Expeditor will submit a pull request that updates the documentation in chef/chef-web-docs the next time InSpec is promoted.

The Docs Team can also update the InSpec documentation if changes need to be made before the next promotion.

You can also submit a pull request to chef-web-docs to update the InSpec docs content on docs.chef.io. See the chef-web-docs readme for information on updating a Hugo module in chef-web-docs.

Documentation Feedback

We love getting feedback, questions, or comments.

Email

Send an email to Chef-Docs@progress.com for documentation bugs, ideas, thoughts, and suggestions. This email address is not a support email address. If you need support, contact Chef Support.

GitHub issues

Submit an issue to the InSpec repo for "important" documentation bugs that may need visibility among a larger group, especially in situations where a doc bug may also surface a product bug.

Submit an issue to chef-web-docs for doc feature requests and minor documentation issues.