dev,docs: Improve and document DX with Python

CONTRIBUTING.md

@@ -12,6 +12,9 @@ way to help us fix issues quickly. Check out
 For more information on how to run and debug tests, check out
 [Running and debugging tests](./docs/testing.md).
 
+Also refer to [Setting up your development environment](./dev-setup.md) to get
+the best possible development experience.
+
 ## How to find issues to work on
 
 If you're looking for a low-hanging fruit, check out

docs/INDEX.md

@@ -19,4 +19,5 @@
 
 ### For contributors
 
+- [Setting up your development environment](./dev-setup.md)
 - [Running and debugging tests](./testing.md)

docs/dev-setup.md

@@ -0,0 +1,75 @@
+# Setting up your development environment
+
+**This guide assumes you have flakes enabled.**
+
+disko uses Nix flake's `devShells` output and [direnv](https://direnv.net/) to
+set up the development environment in two ways.
+
+The quickest way to get started is to run:
+
+```
+nix develop
+```
+
+However, if you use a shell other than bash, working inside `nix develop` might
+get annoying quickly. An alternative is to use direnv, which sets up the
+environment in your current shell session:
+
+```console
+# nix shell nixpkgs#direnv
+direnv: error /home/felix/repos-new/temp/disko/.envrc is blocked. Run `direnv allow` to approve its content
+# direnv allow
+direnv: loading ~/repos-new/temp/disko/.envrc
+direnv: using flake
+direnv: export +AR +AS +CC +CONFIG_SHELL +CXX +HOST_PATH +IN_NIX_SHELL +LD +NIX_BINTOOLS +NIX_BINTOOLS_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_BUILD_CORES +NIX_BUILD_TOP +NIX_CC +NIX_CC_WRAPPER_TARGET_HOST_x86_64_unknown_linux_gnu +NIX_CFLAGS_COMPILE +NIX_ENFORCE_NO_NATIVE +NIX_HARDENING_ENABLE +NIX_LDFLAGS +NIX_STORE +NM +OBJCOPY +OBJDUMP +RANLIB +READELF +SIZE +SOURCE_DATE_EPOCH +STRINGS +STRIP +TEMP +TEMPDIR +TMP +TMPDIR +__structuredAttrs +buildInputs +buildPhase +builder +cmakeFlags +configureFlags +depsBuildBuild +depsBuildBuildPropagated +depsBuildTarget +depsBuildTargetPropagated +depsHostHost +depsHostHostPropagated +depsTargetTarget +depsTargetTargetPropagated +doCheck +doInstallCheck +dontAddDisableDepTrack +mesonFlags +name +nativeBuildInputs +out +outputs +patches +phases +preferLocalBuild +propagatedBuildInputs +propagatedNativeBuildInputs +shell +shellHook +stdenv +strictDeps +system ~PATH ~XDG_DATA_DIRS
+```
+
+You can now run `./disko2 dev --help` or `pytest` to confirm everything is working.
+
+If you're working exclusively in the terminal, you're all set.
+
+## IDE Integration
+
+### VSCode
+
+If you're using VSCode or any of its forks, you should install the recommended
+extensions. You can find them in .vscode/extensions.json, searching for
+`@recommended` in the Extensions Panel, or by opening the
+command pallette and searching "Show Recommended Extensions".
+
+You can then install all extensions in one click:
+
+![VSCode button "Install workspace recommended extensions](./img/vscode-recommended-ext.png)
+
+When you do this (and every time you open the repository again), the
+[direnv extension](https://marketplace.visualstudio.com/items?itemName=mkhl.direnv)
+will prompt you in the bottom right corner to restart all extensions once it
+has loaded the development environment:
+
+![Direnv extension asking "Environment updated. Restart extensions?"](./img/vscode-direnv-prompt.png)
+
+Click "Restart" to make all dependencies available to VSCode.
+
+Afterwards, open the command pallette, search "Python: Select Interpreter" and
+press Enter. The "Select Interpreter" dialog will open:
+
+![VSCode Python "Select Interpreter" dialog](./img/vscode-select-python.png)
+
+Do not select the interpreters tagged "Recommended" or "Global"! These will be
+the ones installed on your system or currently selected in the extension.
+Instead, pick one of the ones below them, this will be the latest available
+package that includes the python packages specified in the `devShell`!
+
+Now you're all set! You might want to also enable the "Format on Save" settings
+and run the "Format Document" command once to select a formatter for each file
+type.
+
+Remember to go through these steps again whenever updating the flake.
+
+### Other IDEs
+
+These are just notes. Feel free to submit a PR that adds support and
+documentation for other IDEs!
+
+[Jetbrains Marketplace has two direnv extensions available](https://plugins.jetbrains.com/search?excludeTags=internal&search=direnv)
+

docs/testing.md

@@ -1,5 +1,20 @@
 # Running and debugging tests
 
+This assumes you've already set up your development environment!
+
+See [Setting up your development environment](./dev-setup.md)
+
+## Python tests
+
+Big parts of disko are written in Python. You can test all Python
+functionality by simply running
+
+```
+pytest
+```
+
+## VM tests
+
 Disko makes extensive use of VM tests. All examples you can find in
 [the example directory](../example) have a respective test suite that verifies
 the example is working in [the tests directory](../tests/). They utilize the
@@ -14,7 +29,7 @@ However, you don't need to know about all of the inner workings to interact with
 the tests effectively. For some of the most common operations, see the sections
 below.
 
-## Run just one of the tests
+### Run just one of the tests
 
 ```sh
 nix build --no-link .#checks.x86_64-linux.simple-efi
@@ -27,7 +42,7 @@ virtual devices, run disko to format them, reboot, verify the VM boots properly,
 and then run the code specified in `extraTestScript` to validate that the
 partitions have been created and were mounted as expected.
 
-### How `extraTestScript` works
+#### How `extraTestScript` works
 
 This is written in Python. The most common lines you'll see look something like
 this:
@@ -45,7 +60,7 @@ Disko currently (as of 2024-10-16) doesn't have any tests that utilize multiple
 VMs at once, so the only machine available in these scripts is always just the
 default `machine`.
 
-## Debugging tests
+### Debugging tests
 
 If you make changes to disko, you might break a test, or you may want to modify
 a test to prevent regressions. In these cases, running the full test with
@@ -131,7 +146,7 @@ vdb    253:16   0    4G  0 disk
 You can find some additional details in
 [the NixOS manual's section on interactive testing](https://nixos.org/manual/nixos/stable/#sec-running-nixos-tests-interactively).
 
-## Running all tests at once
+### Running all tests at once
 
 If you have a bit of experience, you might be inclined to run `nix flake check`
 to run all tests at once. However, we instead recommend using

flake.nix

@@ -60,6 +60,14 @@
             diskoVersion = version;
           };
 
+          # TODO: Add a CI pipeline instead that runs nix run .#pytest inside nix develop
+          pytest-ci-only = pkgs.runCommand "pytest" { nativeBuildInputs = [ pkgs.python3Packages.pytest ]; } ''
+            cd ${./.}
+            # eval_config runs nix, which is forbidden inside of nix derivations by default
+            pytest -vv --doctest-modules -p no:cacheprovider --ignore=tests/disko_lib/eval_config
+            touch $out
+          '';
+
           shellcheck = pkgs.runCommand "shellcheck" { nativeBuildInputs = [ pkgs.shellcheck ]; } ''
             cd ${./.}
             shellcheck src/disk-deactivate/disk-deactivate disko disko2
@@ -69,7 +77,7 @@
         # FIXME: aarch64-linux seems to hang on boot
         lib.optionalAttrs pkgs.hostPlatform.isx86_64 (nixosTests // { inherit disko-install; }) //
         pkgs.lib.optionalAttrs (!pkgs.buildPlatform.isRiscV64 && !pkgs.hostPlatform.isx86_32) {
-          inherit shellcheck;
+          inherit pytest-ci-only shellcheck;
           inherit (self.packages.${system}) disko-doc;
         });
 
@@ -81,7 +89,9 @@
           default = pkgs.mkShell {
             name = "disko-dev";
             packages = (with pkgs; [
-              ruff # Formatter and linter
+              nixpkgs-fmt # Formatter for Nix code
+              shellcheck # Linter for shell scripts
+              ruff # Formatter and linter for Python
               (python3.withPackages (ps: [
                 ps.mypy # Static type checker
                 ps.pytest # Test runner

pyproject.toml

@@ -21,6 +21,7 @@ mypy_path = "src"
 
 [tool.pytest.ini_options]
 pythonpath = ["src"]
+addopts = ["--doctest-modules"]
 
 [tool.autoflake]
 remove_all_unused_imports = true