Generate configuration documentation from code

README.md

@@ -4,6 +4,10 @@
 
 Nix support for building [cargo] crates.
 
+* [Install](#install)
+* [Configuration](#configuration)
+* [Comparison](#install)
+
 ## Install
 
 Use [niv]:
@@ -24,8 +28,39 @@ in naersk.buildPackage ./path/to/rust
 
 _NOTE_: `./path/to/rust/` should contain a `Cargo.lock`.
 
-The `buildPackage` function also accepts an attribute set. For more
+## Configuration
-information, consult [`config.nix`](./config.nix).
+
+The `buildPackage` function also accepts an attribute set. The attributes are
+described below. When the argument passed in _not_ an attribute set, e.g.
+
+``` nix
+naersk.buildPackage theArg
+```
+
+it is converted to an attribute set equivalent to `{ root = theArg; }`.
+
+| Attribute | Description |
+| - | - |
+| `name` | The name of the derivation. |
+| `version` | The version of the derivation. |
+| `src` | Used by `naersk` as source input to the derivation. When `root` is not set, `src` is also used to discover the `Cargo.toml` and `Cargo.lock`. |
+| `root` | Used by `naersk` to read the `Cargo.toml` and `Cargo.lock` files. May be different from `src`. When `src` is not set, `root` is (indirectly) used as `src`. |
+| `cargoBuild` | The command to use for the build. Default: `''cargo build "''${cargo_release}" -j $NIX_BUILD_CORES -Z unstable-options --out-dir out''` |
+| `doCheck` | When true, `checkPhase` is run. Default: `true` |
+| `cargoTestCommands` | The commands to run in the `checkPhase`. Default: `[ ''cargo test "''${cargo_release}" -j $NIX_BUILD_CORES'' ]` |
+| `buildInputs` | Extra `buildInputs` to all derivations. Default: `[]` |
+| `doDoc` | When true, `cargo doc` is run and a new output `doc` is generated. Default: `true` |
+| `release` | When true, all cargo builds are run with `--release`. Default: `true` |
+| `override` | An override for all derivations involved in the build. Default: `(x: x)` |
+| `singleStep` | When true, no intermediary (dependency-only) build is run. Enabling `singleStep` greatly reduces the incrementality of the builds. Default: `false` |
+| `targets` | The targets to build if the `Cargo.toml` is a virtual manifest. |
+| `copyBins` | When true, the resulting binaries are copied to `$out/bin`. Default: `true` |
+| `copyDocsToSeparateOutput` | When true, the documentation is generated in a different output, `doc`. Default: `true` |
+| `doDocFail` | When true, the build fails if the documentation step fails; otherwise the failure is ignored. Default: `false` |
+| `removeReferencesToSrcFromDocs` | When true, references to the nix store are removed from the generated documentation. Default: `true` |
+| `compressTarget` | When true, the build output of intermediary builds is compressed with [`Zstandard`](https://facebook.github.io/zstd/). This reduces the size of closures. Default: `true` |
+| `copyTarget` | When true, the `target/` directory is copied to `$out`. Default: `false` |
+| `usePureFromTOML` | Whether to use the `fromTOML` built-in or not. When set to `false` the python package `remarshal` is used instead (in a derivation) and the JSON output is read with `builtins.fromJSON`. This is a workaround for old versions of Nix. May be used safely from Nix 2.3 onwards where all bugs in `builtins.fromTOML` seem to have been fixed. Default: `true` |
 
 ## Comparison
 
@@ -45,6 +80,5 @@ Finally `naersk` supports incremental builds by first performing a
 dependency-only build, and _then_ a build that depends on the top-level crate's
 code and configuration.
 
-
 [cargo]: https://crates.io/
 [niv]: https://github.com/nmattia/niv

README.tpl.md

@@ -0,0 +1,63 @@
+# Naersk
+
+[![CircleCI](https://circleci.com/gh/nmattia/naersk.svg?style=svg)](https://circleci.com/gh/nmattia/naersk)
+
+Nix support for building [cargo] crates.
+
+* [Install](#install)
+* [Configuration](#configuration)
+* [Comparison](#install)
+
+## Install
+
+Use [niv]:
+
+``` shell
+$ niv add nmattia/naersk
+```
+
+And then
+
+``` nix
+let
+    pkgs = import <nixpkgs> {};
+    sources = import ./nix/sources.nix;
+    naersk = pkgs.callPackage sources.naersk {};
+in naersk.buildPackage ./path/to/rust
+```
+
+_NOTE_: `./path/to/rust/` should contain a `Cargo.lock`.
+
+## Configuration
+
+The `buildPackage` function also accepts an attribute set. The attributes are
+described below. When the argument passed in _not_ an attribute set, e.g.
+
+``` nix
+naersk.buildPackage theArg
+```
+
+it is converted to an attribute set equivalent to `{ root = theArg; }`.
+
+GEN_CONFIGURATION
+
+## Comparison
+
+There are two other notable Rust frameworks in Nix: `rustPlatform` and
+`carnix`.
+
+`naersk` uses `cargo` directly, as opposed to `carnix` which emulates `cargo`'s
+build logic. Moreover `naersk` sources build information directly from the
+project's `Cargo.lock` which makes any code generation unnecessary.
+
+For the same reason, `naersk` does not need anything like `rustPlatform`'s
+`cargoSha256`. All crates are downloaded using the `sha256` checksums provided
+in the project's `Cargo.lock`. Because `cargo` doesn't register checksums for
+`git` dependencies, **`naersk` does not support `git` dependencies**.
+
+Finally `naersk` supports incremental builds by first performing a
+dependency-only build, and _then_ a build that depends on the top-level crate's
+code and configuration.
+
+[cargo]: https://crates.io/
+[niv]: https://github.com/nmattia/niv

config.nix

@@ -1,54 +1,61 @@
 { lib, libb, builtinz, arg }:
 let
   mkAttrs = attrs0:
-    { # foo bar< baz
+  {   # The name of the derivation.
-      # baz? foooo
+      name = attrs0.name or null;
-      # one
+      # The version of the derivation.
-      # two
+      version = attrs0.version or null;
-      # three
+      # Used by `naersk` as source input to the derivation. When `root` is not
-      # four
+      # set, `src` is also used to discover the `Cargo.toml` and `Cargo.lock`.
-      root = attrs0.root or null;
-      # hello src
-      # hello world
       src = attrs0.src or null;
-      # hello src
+      # Used by `naersk` to read the `Cargo.toml` and `Cargo.lock` files. May
-      usePureFromTOML = attrs0.usePureFromTOML or true;
+      # be different from `src`. When `src` is not set, `root` is (indirectly)
-      # hello src
+      # used as `src`.
-      compressTarget = attrs0.compressTarget or true;
+      root = attrs0.root or null;
-      # hello src
+      # The command to use for the build.
-      doCheck = attrs0.doCheck or true;
-      # hello src
-      buildInputs = attrs0.buildInputs or [];
-      # hello src
-      removeReferencesToSrcFromDocs = attrs0.removeReferencesToSrcFromDocs or true;
-      # hello src
-      doDoc = attrs0.doDoc or true;
-      # hello src
-      doDocFail = attrs0.doDocFail or false;
-      # hello src
-      release = attrs0.release or true;
-      # hello src
-      override = attrs0.override or (x: x);
-      # hello src
       cargoBuild = attrs0.cargoBuild or
           ''cargo build "''${cargo_release}" -j $NIX_BUILD_CORES -Z unstable-options --out-dir out'';
-      # hello src
+      # When true, `checkPhase` is run.
-      singleStep = attrs0.singleStep or false;
+      doCheck = attrs0.doCheck or true;
-      # hello src
+      # The commands to run in the `checkPhase`.
-      targets =  attrs0.targets or null;
-      # hello src
-      name = attrs0.name or null;
-      # hello src
-      version = attrs0.version or null;
-      # hello src
       cargoTestCommands = attrs0.cargoTestCommands or
           [ ''cargo test "''${cargo_release}" -j $NIX_BUILD_CORES'' ];
-      # hello src
+      # Extra `buildInputs` to all derivations.
-      copyTarget = attrs0.copyTarget or false;
+      buildInputs = attrs0.buildInputs or [];
-      # hello src
+      # When true, `cargo doc` is run and a new output `doc` is generated.
+      doDoc = attrs0.doDoc or true;
+      # When true, all cargo builds are run with `--release`.
+      release = attrs0.release or true;
+      # An override for all derivations involved in the build.
+      override = attrs0.override or (x: x);
+      # When true, no intermediary (dependency-only) build is run. Enabling
+      # `singleStep` greatly reduces the incrementality of the builds.
+      singleStep = attrs0.singleStep or false;
+      # The targets to build if the `Cargo.toml` is a virtual manifest.
+      targets =  attrs0.targets or null;
+      # When true, the resulting binaries are copied to `$out/bin`.
       copyBins = attrs0.copyBins or true;
-      # hello src
+      # When true, the documentation is generated in a different output, `doc`.
       copyDocsToSeparateOutput =  attrs0.copyDocsToSeparateOutput or true;
+      # When true, the build fails if the documentation step fails; otherwise
+      # the failure is ignored.
+      doDocFail = attrs0.doDocFail or false;
+      # When true, references to the nix store are removed from the generated
+      # documentation.
+      removeReferencesToSrcFromDocs = attrs0.removeReferencesToSrcFromDocs or true;
+      # When true, the build output of intermediary builds is compressed with
+      # [`Zstandard`](https://facebook.github.io/zstd/). This reduces the size
+      # of closures.
+      compressTarget = attrs0.compressTarget or true;
+      # When true, the `target/` directory is copied to `$out`.
+      copyTarget = attrs0.copyTarget or false;
+      # Whether to use the `fromTOML` built-in or not. When set to `false` the
+      # python package `remarshal` is used instead (in a derivation) and the
+      # JSON output is read with `builtins.fromJSON`.
+      # This is a workaround for old versions of Nix. May be used safely from
+      # Nix 2.3 onwards where all bugs in `builtins.fromTOML` seem to have been
+      # fixed.
+      usePureFromTOML = attrs0.usePureFromTOML or true;
     };
 
   argIsAttrs =

docparse/src/main.rs

@@ -37,33 +37,39 @@ fn print_mk_attrs(mk_attrs: SyntaxNode) {
         .body()
         .and_then(AttrSet::cast)
         .expect("Not a pattern");
+
+    println!("| Attribute | Description |");
+    println!("| - | - |");
     for e in body.entries() {
         let k = e.key().expect("No key").path().next().unwrap();
         let mshown = e
             .value()
             .and_then(OrDefault::cast)
             .expect("Is not OrDefault")
-            .default().and_then(|def|
+            .default()
-        {
+            .and_then(|def| {
-            let shown = format!("{}", def);
+                let shown = format!("{}", def);
-            if shown != "null" {
+                if shown != "null" {
-                Some(shown)
+                    Some(shown)
-            } else {
+                } else {
-                None
+                    None
-            }
+                }
-        });
+            });
         let e = e.node().clone();
         let c = find_comment(e).expect("No comment");
-        println!("### {} \n", k);
+        let mut lines = vec![];
-        println!("{}", c);
+        for l in c.lines() {
-        if let Some(shown) = mshown {
+            lines.push(l);
-            println!("");
-            println!("_default value:_");
-            println!("``` nix");
-            println!("{}", shown);
-            println!("```");
         }
-        println!("");
+
+        let sss;
+        if let Some(shown) = mshown {
+            lines.push("Default:");
+            sss = format!("`{}`", shown).to_string();
+            lines.push(&sss);
+        }
+        let descr = lines.join(" ");
+        println!("| `{}` | {} |", k, descr);
     }
 }
 
@@ -89,10 +95,11 @@ fn find_comment(node: SyntaxNode) -> Option<String> {
             _ => break,
         }
     }
+    comments.reverse();
     let doc = comments
         .iter()
         .map(|it| it.trim_start_matches('#').trim())
         .collect::<Vec<_>>()
-        .join("\n        ");
+        .join("\n");
     return Some(doc).filter(|it| !it.is_empty());
 }

script/gen

@@ -0,0 +1,12 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash
+#!nix-shell -I nixpkgs=./nix
+#!nix-shell -p nix
+
+set -euo pipefail
+
+echo "Updating README"
+
+cat $(nix-build ./test.nix -A readme) > README.md
+
+echo done

script/test

@@ -2,9 +2,6 @@
 #!nix-shell -i bash
 #!nix-shell -I nixpkgs=./nix
 #!nix-shell -p nix
-#!nix-shell --pure
-#!nix-shell --keep SSL_CERT_FILE
-#!nix-shell --keep NIX_SSL_CERT_FILE
 
 set -euo pipefail
 

test.nix

@@ -8,6 +8,33 @@ let
 in
 rec
 {
+  readme = pkgs.runCommand "readme-gen" {}
+    ''
+    cat ${./README.tpl.md} > $out
+    ${docparse}/bin/docparse ${./config.nix} >> gen
+    sed -e '/GEN_CONFIGURATION/{r gen' -e 'd}' -i $out
+    '';
+
+  docparse = naersk.buildPackage {
+    root = ./docparse;
+    src = builtins.filterSource (
+      p: t:
+        let
+          p' = pkgs.lib.removePrefix (toString ./docparse + "/") p;
+        in
+        p' == "Cargo.lock" ||
+        p' == "Cargo.toml" ||
+        p' == "src" ||
+        p' == "src/main.rs"
+      ) ./docparse;
+    };
+
+  readme_test = pkgs.runCommand "readme-test" {}
+    ''
+    diff ${./README.md} ${readme}
+    touch $out
+    '';
+
   # error[E0554]: `#![feature]` may not be used on the stable release channel
   # rustfmt = naersk.buildPackage sources.rustfmt { doDocFail = false; };
   # rustfmt_test = pkgs.runCommand "rustfmt-test"