Mirrors/nixos-and-flakes-book
2
0
Fork 0
mirror of https://github.com/ryan4yin/nixos-and-flakes-book synced 2024-11-14 00:17:08 +00:00
Code Issues Projects Releases Packages Wiki Activity

feat: add new chapter - development

Browse source
This commit is contained in:
Ryan Yin 2023-06-27 10:53:11 +08:00
parent 1b5937ec65
commit 5d0a512393
15 changed files with 1491 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

26
docs/.vitepress/config.ts
View file

@ -265,6 +265,32 @@ export default defineConfig({
}, },
], ],
}, },
{
text: "在 NixOS 上进行开发工作",
items: [
{ text: "介绍", link: "/zh/development/intro.md" },
{
text: "各语言的开发环境",
link: "/zh/development/dev-environments.md",
},
{
text: "软件打包",
link: "/zh/development/packaging-101.md",
},
{
text: "跨平台编译",
link: "/zh/development/cross-platform-compilation.md",
},
{
text: "分布式构建",
link: "/zh/development/distributed-building.md",
},
{
text: "内核开发",
link: "/zh/development/kernel-development.md",
},
],
},
{ {
text: "其他进阶话题", text: "其他进阶话题",
items: [{ text: "介绍", link: "/zh/advanced-topics/index.md" }], items: [{ text: "介绍", link: "/zh/advanced-topics/index.md" }],

6
docs/development/cross-platform-compilation.md Normal file
View file

@ -0,0 +1,6 @@
## Cross-platform compilation
First of all, on any Linux platform, there are two ways to do cross-platform compilation.
Taking the building of an`aarch64-linux` program on a `x86_64-linux` host as an example, the two methods are described as follows:
TODO

11
docs/development/dev-environments.md Normal file
View file

@ -0,0 +1,11 @@
## Dev Environments
We have learned how to build development environments, but it's a bit tedious to write `flake.nix` for each project.
Luckily, some people in the community have done this for us. The following repository contains development environment templates for most programming languages. Just copy and paste them:
- [dev-templates](https://github.com/the-nix-way/dev-templates)
If you think the structure of `flake.nix` is still too complicated and want a simpler way, you can also consider using the following project, which encapsulates Nix more thoroughly and provides users with a simpler definition:
- [cachix/devenv](https://github.com/cachix/devenv)

150
docs/development/distributed-building.md Normal file
View file

@ -0,0 +1,150 @@
## Distributed Building
Distributed building can speed up the build process by make use of multiple machines.
NixOS's official cache.nixos.org provides the vast majority of caches for the X86_64 architecture, so distributed builds are generally not very useful for ordinary NixOS users.
Distributed building is of great application value only in scenarios where there is no cache available:
1. Users of RISC-V or ARM64 architectures (especially RISC-V), because there are very few caches for these two architectures in the official cache repository, which often requires a lot of local compilation.
2. Users who customize the system a lot, because the packages in the official cache repository are all default configurations, if you change the build parameters, then the official cache is not applicable, then you need to compile locally.
2. For instance, in the embedded scenario, there is often a need for customization of the underlying kernel, drivers, etc., which leads to the need for local compilation.
### Configure distributed building
Currently, there is no official documentation for this, and I have listed some recommended reference documents at the end of this chapter, along with my distributed build configuration (a NixOS Module):
```nix
{ ... }: {
####################################################################
#
# NixOS's Configuration for Remote Building / Distributed Building
#
####################################################################
# set local's max-job to 0 to force remote building(disable local building)
# nix.settings.max-jobs = 0;
nix.distributedBuilds = true;
nix.buildMachines =
let
sshUser = "ryan";
# ssh key's path on local machine
sshKey = "/home/ryan/.ssh/ai-idols";
systems = [
# native arch
"x86_64-linux"
# emulated arch using binfmt_misc and qemu-user
"aarch64-linux"
"riscv64-linux"
];
# all available system features are poorly documentd here:
# https://github.com/NixOS/nix/blob/e503ead/src/libstore/globals.hh#L673-L687
supportedFeatures = [
"benchmark"
"big-parallel"
"kvm"
];
in
[
# Nix seems always give priority to trying to build remotely
# to make use of the local machine's high-performance CPU, do not set remote builder's maxJobs too high.
{
# some of my remote builders are running NixOS
# and has the same sshUser, sshKey, systems, etc.
inherit sshUser sshKey systems supportedFeatures;
# the hostName should be:
# 1. a hostname that can be resolved by DNS
# 2. the ip address of the remote builder
# 3. a host alias defined globally in /etc/ssh/ssh_config
hostName = "aquamarine";
# remote builder's max-job
maxJobs = 3;
# speedFactor's a signed integer
# but it seems that it's not used by Nix, takes no effect
speedFactor = 1;
}
{
inherit sshUser sshKey systems supportedFeatures;
hostName = "ruby";
maxJobs = 2;
speedFactor = 1;
}
{
inherit sshUser sshKey systems supportedFeatures;
hostName = "kana";
maxJobs = 2;
speedFactor = 1;
}
];
# optional, useful when the builder has a faster internet connection than yours
nix.extraOptions = ''
builders-use-substitutes = true
'';
# define the host alias for remote builders
# this config will be written to /etc/ssh/ssh_config
programs.ssh.extraConfig = ''
Host ai
HostName 192.168.5.100
Port 22
Host aquamarine
HostName 192.168.5.101
Port 22
Host ruby
HostName 192.168.5.102
Port 22
Host kana
HostName 192.168.5.103
Port 22
'';
# define the host key for remote builders so that nix can verify all the remote builders
# this config will be written to /etc/ssh/ssh_known_hosts
programs.ssh.knownHosts = {
# 星野 愛久愛海, Hoshino Aquamarine
aquamarine = {
hostNames = [ "aquamarine" "192.168.5.101" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDnCQXlllHoLX5EvU+t6yP/npsmuxKt0skHVeJashizE";
};
# 星野 瑠美衣, Hoshino Rubii
ruby = {
hostNames = [ "ruby" "192.168.5.102" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIE7n11XxB8B3HjdyAsL3PuLVDZxWCzEOUTJAY8+goQmW";
};
# 有馬 かな, Arima Kana
kana = {
hostNames = [ "kana" "192.168.5.103" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ3dDLOZERP1nZfRz3zIeVDm1q2Trer+fWFVvVXrgXM1";
};
};
}
```
## Defects
The problems I have observed so far are:
1. You cannot specify which hosts to use at build time, you can only specify a list of hosts in the configuration file, and nix automatically selects available hosts.
two。
2. When choosing a host, I found that Nix always preferred the remote host, while my local host had the best performance, which caused the local host's CPU to be underutilized.
3. The smallest unit of distributed building is Derivation, so when building some big packages, other machines may be idle for a long time, waiting for the big package to be built, which leads to a waste of resources.
## References
- [Distributed build - NixOS Wiki](https://nixos.wiki/wiki/Distributed_build)
- [Document available system features - nix#7380](https://github.com/NixOS/nix/issues/7380)
- [Distributed builds seem to disable local builds nix#2589](https://github.com/NixOS/nix/issues/2589)
- [Offloading NixOS builds to a faster machine](https://sgt.hootr.club/molten-matter/nix-distributed-builds/)
- [tests/nixos/remote-builds.nix - Nix Source Code](https://github.com/NixOS/nix/blob/713836112/tests/nixos/remote-builds.nix#L46)

291
docs/development/intro.md Normal file
View file

@ -0,0 +1,291 @@
## Development on NixOS
Due to the reproducibility of NixOS itself, it is very suitable for building development environments.
But if you want to migrate the experience on other distros directly to NixOS, you may encounter problems, because NixOS has its own set of logic, we will explain this briefly below.
On NixOS, it is recommended to install only common tools in the global environment, such as `git`, `vim`, `emacs`, `tmux`, `zsh`, etc., while the development environment of each language should be an independent environment for each project.
You should NOT install the development environment of each language in the global environment, and the project environment should be completely isolated from each other, and will not affect each other.
We will introduce how the development environment works in NixOS in the following sections.
### Create a development environment
We can create a development environment via `pkgs.mkShell { ... }`, and open an interactive Bash Shell of this development environment via `nix develop`.
Let's take a look at [the source code of `pkgs.mkShell`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/mkshell/default.nix) to see how it works:
```nix
{ lib, stdenv, buildEnv }:
# A special kind of derivation that is only meant to be consumed by the
# nix-shell.
{ name ? "nix-shell"
, # a list of packages to add to the shell environment
packages ? [ ]
, # propagate all the inputs from the given derivations
inputsFrom ? [ ]
, buildInputs ? [ ]
, nativeBuildInputs ? [ ]
, propagatedBuildInputs ? [ ]
, propagatedNativeBuildInputs ? [ ]
, ...
}@attrs:
let
mergeInputs = name:
(attrs.${name} or [ ]) ++
(lib.subtractLists inputsFrom (lib.flatten (lib.catAttrs name inputsFrom)));
rest = builtins.removeAttrs attrs [
"name"
"packages"
"inputsFrom"
"buildInputs"
"nativeBuildInputs"
"propagatedBuildInputs"
"propagatedNativeBuildInputs"
"shellHook"
];
in
stdenv.mkDerivation ({
inherit name;
buildInputs = mergeInputs "buildInputs";
nativeBuildInputs = packages ++ (mergeInputs "nativeBuildInputs");
propagatedBuildInputs = mergeInputs "propagatedBuildInputs";
propagatedNativeBuildInputs = mergeInputs "propagatedNativeBuildInputs";
shellHook = lib.concatStringsSep "\n" (lib.catAttrs "shellHook"
(lib.reverseList inputsFrom ++ [ attrs ]));
phases = [ "buildPhase" ];
# ......
# when distributed building is enabled, prefer to build locally
preferLocalBuild = true;
} // rest)
```
`pkgs.mkShell { ... }` is a special Derivation (Nix package). Its `name` `buildInputs` and other parameters are customizable, and `shellHook` is a special parameter that will be executed when `nix develop` enters the environment.
Here is a `flake.nix` that defined a development environment with nodejs 18 installed:
```nix
{
description = "A Nix-flake-based Node.js development environment";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05";
};
outputs = { self , nixpkgs ,... }: let
# system should match the system you are running on
# system = "x86_64-linux";
system = "x86_64-darwin";
in {
devShells."${system}".default = let
pkgs = import nixpkgs {
inherit system;
overlays = [
(self: super: rec {
nodejs = super.nodejs-18_x;
pnpm = super.nodePackages.pnpm;
yarn = (super.yarn.override { inherit nodejs; });
})
];
};
in pkgs.mkShell {
# create an environment with nodejs-18_x, pnpm, and yarn
packages = with pkgs; [
node2nix
nodejs
pnpm
yarn
];
shellHook = ''
echo "node `${pkgs.nodejs}/bin/node --version`"
'';
};
};
}
```
Create an empty folder, save the above configuration as `flake.nix`, and then execute `nix develop` (or more precisely, you can use `nix develop .#default`), you will find that you have entered a nodejs 18 development environment, you can use `node` `npm` `pnpm` `yarn` and other commands. And when you just entered, `shellHook` was also executed, outputting the current version of nodejs.
### Enter the build environment of any Nix package
Now let's take a look at `nix develop`, first read the help document output by `nix develop --help`:
```
Name
nix develop - run a bash shell that provides the build environment of a derivation
Synopsis
nix develop [option...] installable
# ......
```
It tells us that `nix develop` accepts a parameter `installable`, which means that we can enter the development environment of any installable Nix package through it, not just the environment created by `pkgs.mkShell`.
By default, `nix develop` will try to use the following attributes in the flake outputs:
- `devShells.<system>.default`
- `packages.<system>.default`
If we use `nix develop /path/to/flake#<name>` to specify the flake package address and flake output name, then `nix develop` will try the following attributes in the flake outputs:
- `devShells.<system>.<name>`
- `packages.<system>.<name>`
- `legacyPackages.<system>.<name>`
Now let's try it out. First, test it to confirm that We don't have `c++` `g++` and other compilation-related commands in the current environment:
```shell
ryan in 🌐 aquamarine in ~
› c++
c++: command not found
ryan in 🌐 aquamarine in ~
› g++
g++: command not found
```
Then use `nix develop` to enter the build environment of the `hello` package in `nixpkgs`:
```shell
# login to the build environment of the package `hello`
ryan in 🌐 aquamarine in ~
› nix develop nixpkgs#hello
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› env | grep CXX
CXX=g++
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› c++ --version
g++ (GCC) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› g++ --version
g++ (GCC) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```
We can see that the `CXX` environment variable have been set, and the `c++` `g++` and other commands can be used normally now.
In addition, we can also call every build phase of the `hello` package normally:
> The default execution order of all build phases of a Nix package is: `$prePhases unpackPhase patchPhase $preConfigurePhases configurePhase $preBuildPhases buildPhase checkPhase $preInstallPhases installPhase fixupPhase installCheckPhase $preDistPhases distPhase $postPhases`
```shell
# unpack source code
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› unpackPhase
unpacking source archive /nix/store/pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
source root is hello-2.12.1
setting SOURCE_DATE_EPOCH to timestamp 1653865426 of file hello-2.12.1/ChangeLog
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› ls
hello-2.12.1
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› cd hello-2.12.1/
# generate Makefile
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via ❄️ impure (hello-2.12.1-env)
› configurePhase
configure flags: --prefix=/tmp/xxx/outputs/out --prefix=/tmp/xxx/outputs/out
checking for a BSD-compatible install... /nix/store/02dr9ymdqpkb75vf0v1z2l91z2q3izy9-coreutils-9.3/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /nix/store/02dr9ymdqpkb75vf0v1z2l91z2q3izy9-coreutils-9.3/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
# ......
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating po/Makefile.in
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
# build the package
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via C v12.3.0-gcc via ❄️ impure (hello-2.12.1-env) took 2s
› buildPhase
build flags: SHELL=/run/current-system/sw/bin/bash
make all-recursive
make[1]: Entering directory '/tmp/xxx/hello-2.12.1'
# ......
ranlib lib/libhello.a
gcc -g -O2 -o hello src/hello.o ./lib/libhello.a
make[2]: Leaving directory '/tmp/xxx/hello-2.12.1'
make[1]: Leaving directory '/tmp/xxx/hello-2.12.1'
# run the built program
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via C v12.3.0-gcc via ❄️ impure (hello-2.12.1-env)
› ./hello
Hello, world!
```
This usage is mainly used to debug the build process of a Nix package, or to execute some commands in the build environment of a Nix package.
### `nix shell` & `nix run`
Compare to `nix develop`, these two commands are much simpler and easier to understand.
`nix shell` is used to enter an environment containing the specified Nix package and open an interactive shell for it:
```shell
# hello not exists
› hello
hello: command not found
# enter an environment containing hello
› nix shell nixpkgs#hello
# now hello exists
› hello
Hello, world!
```
`nix run` is used to create an environment containing the specified installable and run the installable in it:
```shell
# hello not exists
› hello
hello: command not found
# enter an environment containing hello and run it
› nix run nixpkgs#hello
Hello, world!
```
Because `nix run` will directly run the Nix package as an installable, the Nix package used as its parameter must be able to generate an executable program.
According to the description of `nix run --help`, `nix run` will execute `<out>/bin/<name>`, where `<out>` is the root directory of a Derivation, and `<name>` is selected in the following order:
- The meta.mainProgram attribute of the derivation.
- The pname attribute of the derivation.
- The name part of the value of the name attribute of the derivation.
For instance, if name is set to `hello-1.10`, nix run will run $out/bin/hello.
## References
- [pkgs.mkShell - nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-mkShell)
- [A minimal nix-shell](https://fzakaria.com/2021/08/02/a-minimal-nix-shell.html)
- [One too many shell, Clearing up with nix' shells nix shell and nix-shell - Yannik Sander](https://blog.ysndr.de/posts/guides/2021-12-01-nix-shells/)

115
docs/development/kernel-development.md Normal file
View file

@ -0,0 +1,115 @@
# Kernel Development
> WIP work in progress
An example of kernel development with `flake.nix`.
```nix
{
description = "NixOS running on LicheePi 4A";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05-small";
# custom kernel's source
thead-kernel = {
url = "github:revyos/thead-kernel/lpi4a";
flake = false;
};
};
outputs = inputs@{
self
,nixpkgs
,thead-kernel
,... }:
let
pkgsKernel = import nixpkgs {
localSystem = "x86_64-linux";
crossSystem = {
config = "riscv64-unknown-linux-gnu";
};
overlays = [
(self: super: {
# use gcc 13 to compile this custom kernel
linuxPackages_thead = super.linuxPackagesFor (super.callPackage ./pkgs/kernel {
src = thead-kernel;
stdenv = super.gcc13Stdenv;
kernelPatches = with super.kernelPatches; [
bridge_stp_helper
request_key_helper
];
});
})
];
};
in
{
nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
specialArgs = {
inherit nixpkgs pkgsKernel;
};
modules = [
{
# cross-compile this flake.
nixpkgs.crossSystem = {
system = "riscv64-linux";
};
}
./modules/licheepi4a.nix
./modules/sd-image-lp4a.nix
];
};
# use `nix develop .#kernel` to enter the environment with the custom kernel build environment available.
# and then use `unpackPhase` to unpack the kernel source code and cd into it.
# then you can use `make menuconfig` to configure the kernel.
#
# problem
# - using `make menuconfig` - Unable to find the ncurses package.
devShells.x86_64-linux.kernel = pkgsKernel.linuxPackages_thead.kernel.dev;
# use `nix develop .#fhs` to enter the fhs test environment defined here.
devShells.x86_64-linux.fhs = let
pkgs = import nixpkgs {
system = "x86_64-linux";
};
in
# the code here is mainly copied from:
# https://nixos.wiki/wiki/Linux_kernel#Embedded_Linux_Cross-compile_xconfig_and_menuconfig
(pkgs.buildFHSUserEnv {
name = "kernel-build-env";
targetPkgs = pkgs_: (with pkgs_;
[
# we need theses packages to run `make menuconfig` successfully.
pkgconfig
ncurses
pkgsKernel.gcc13Stdenv.cc
gcc
]
++ pkgs.linux.nativeBuildInputs);
runScript = pkgs.writeScript "init.sh" ''
# set the cross-compilation environment variables.
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
export ARCH=riscv
export PKG_CONFIG_PATH="${pkgs.ncurses.dev}/lib/pkgconfig:"
exec bash
'';
}).env;
};
}
```
With the above `flake.nix`, I can enter the kernel build environment with `nix develop .#kernel`, and then use `unpackPhase` to unpack the kernel source code and cd into it.
But I can't use `make menuconfig` to configure the kernel, because the `ncurses` package is missing in this environment.
To solve this problem, I add a `fhs` environment to install the `ncurses` package and other necessary packages, and then I can use `nix develop .#fhs` to enter this environment and use `make menuconfig` to configure the kernel.
## References
- https://github.com/jordanisaacs/kernel-module-flake

22
docs/development/packaging-101.md Normal file
View file

@ -0,0 +1,22 @@
# Packging 101
> WIP work in progress
TODO
### 1. stdenv
TODO
### 2. language specific frameworks
TODO
## References
- [NixOS Series 3: Software Packaging 101](https://lantian.pub/en/article/modify-computer/nixos-packaging.lantian/)
- [stdenv - Nixpkgs Manual](https://github.com/NixOS/nixpkgs/tree/nixos-unstable/doc/languages-frameworks)
- [languages-frameworks - Nixpkgs Manual](https://github.com/NixOS/nixpkgs/tree/nixos-unstable/doc/stdenv)

3
docs/nixos-with-flakes/introduction-to-flakes.md
View file

@ -25,7 +25,8 @@ Here are the classic Nix commands and related concepts that are no longer needed
2. `nix-env`: `nix-env` is a core command-line tool for classic Nix used to manage software packages in the user environment. It installs packages from the data sources added by `nix-channel`, so the installed package's version are influenced by the channel. Packages installed with `nix-env` are not automatically recorded in Nix's declarative configuration and are entirely outside of its control, making them difficult to reproduce on other machines. Therefore, it is not recommended to use this tool. 2. `nix-env`: `nix-env` is a core command-line tool for classic Nix used to manage software packages in the user environment. It installs packages from the data sources added by `nix-channel`, so the installed package's version are influenced by the channel. Packages installed with `nix-env` are not automatically recorded in Nix's declarative configuration and are entirely outside of its control, making them difficult to reproduce on other machines. Therefore, it is not recommended to use this tool.
1. The corresponding command in Flakes is `nix profile`, it's not recommended to use it either. 1. The corresponding command in Flakes is `nix profile`, it's not recommended to use it either.
3. `nix-shell`: `nix-shell` is used to create a temporary shell environment, which is useful for development and testing. 3. `nix-shell`: `nix-shell` is used to create a temporary shell environment, which is useful for development and testing.
1. In Flakes, it is replaced by `nix develop` and `nix shell`. 1. This tool is a bit complicated, so it is split into three sub-commands in Flakes: `nix develop`, `nix shell` and `nix run`.
2. We will introduce these three commands in detail in the "Development" chapter.
4. `nix-build`: `nix-build` is used to build Nix packages, and it places the build results in `/nix/store`, but it does not record them in Nix's declarative configuration. 4. `nix-build`: `nix-build` is used to build Nix packages, and it places the build results in `/nix/store`, but it does not record them in Nix's declarative configuration.
1. In Flakes, `nix-build` is replaced by `nix build`. 1. In Flakes, `nix-build` is replaced by `nix build`.
5. ... 5. ...

278
docs/zh/development/cross-platform-compilation.md Normal file
View file

@ -0,0 +1,278 @@
## 跨平台编译
首先在任何 Linux 平台上，都有两种方法进行跨平台构建。
以在 x86_64 架构上构建 aarch64 架构程序为例，两种构建方法说明如下：
1. 使用 QEMU 模拟 aarch64 架构，然后在模拟器中编译程序
1. 缺点是指令集模拟，性能低下
2. 优点是能利用上 NixOS 的 binary cache，不需要自己编译所有内容
2. 使用交叉编译器编译 aarch64 架构的程序
1. 缺点是无法利用 NixOS 的 binary cache，需要自己编译所有内容（交叉编译也有 cache，但是里面基本没啥东西）
2. 优点是不需要指令集模拟，性能高
如果使用方法一，则需要在构建机的 NixOS 配置中启用 aarch64 架构的 binfmt_misc
如果使用方法二，就不需要启用 binfmt_misc 了，但是需要通过交叉编译工具链来执行编译。
### 交叉编译
nixpkgs 包含了一系列预定义好的交叉编译工具链，其名为 `pkgsCross`，我们先通过 `nix repl` 来看看有哪些工具链：
```shell
› nix repl '<nixpkgs>'
warning: future versions of Nix will require using `--file` to load a file
Welcome to Nix 2.13.3. Type :? for help.
Loading installable ''...
Added 19273 variables.
nix-repl> pkgsCross.<TAB>
pkgsCross.aarch64-android pkgsCross.msp430
pkgsCross.aarch64-android-prebuilt pkgsCross.musl-power
pkgsCross.aarch64-darwin pkgsCross.musl32
pkgsCross.aarch64-embedded pkgsCross.musl64
pkgsCross.aarch64-multiplatform pkgsCross.muslpi
pkgsCross.aarch64-multiplatform-musl pkgsCross.or1k
pkgsCross.aarch64be-embedded pkgsCross.pogoplug4
pkgsCross.arm-embedded pkgsCross.powernv
pkgsCross.armhf-embedded pkgsCross.ppc-embedded
pkgsCross.armv7a-android-prebuilt pkgsCross.ppc64
pkgsCross.armv7l-hf-multiplatform pkgsCross.ppc64-musl
pkgsCross.avr pkgsCross.ppcle-embedded
pkgsCross.ben-nanonote pkgsCross.raspberryPi
pkgsCross.fuloongminipc pkgsCross.remarkable1
pkgsCross.ghcjs pkgsCross.remarkable2
pkgsCross.gnu32 pkgsCross.riscv32
pkgsCross.gnu64 pkgsCross.riscv32-embedded
pkgsCross.i686-embedded pkgsCross.riscv64
pkgsCross.iphone32 pkgsCross.riscv64-embedded
pkgsCross.iphone32-simulator pkgsCross.rx-embedded
pkgsCross.iphone64 pkgsCross.s390
pkgsCross.iphone64-simulator pkgsCross.s390x
pkgsCross.loongarch64-linux pkgsCross.sheevaplug
pkgsCross.m68k pkgsCross.vc4
pkgsCross.mingw32 pkgsCross.wasi32
pkgsCross.mingwW64 pkgsCross.x86_64-darwin
pkgsCross.mips-linux-gnu pkgsCross.x86_64-embedded
pkgsCross.mips64-linux-gnuabi64 pkgsCross.x86_64-freebsd
pkgsCross.mips64-linux-gnuabin32 pkgsCross.x86_64-netbsd
pkgsCross.mips64el-linux-gnuabi64 pkgsCross.x86_64-netbsd-llvm
pkgsCross.mips64el-linux-gnuabin32 pkgsCross.x86_64-unknown-redox
pkgsCross.mipsel-linux-gnu
pkgsCross.mmix
```
如果想将一个 flake 全局的 `pkgs` 设置为交叉编译工具链，只需要在 `flake.nix` 中添加一个 Module，示例如下：
```nix
{
description = "NixOS running on LicheePi 4A";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05";
};
outputs = inputs@{ self, nixpkgs, ... }: {
nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem {
# native platform
system = "x86_64-linux";
modules = [
# add this module, to enable cross-compilation.
{
nixpkgs.crossSystem = {
# target platform
system = "riscv64-linux";
};
}
# ...... other modules
];
};
};
}
```
模块中的 `nixpkgs.crossSystem` 参数用于将 `pkgs` 设置为交叉编译工具链，这样构建出的内容全都会是 `riscv64-linux` 架构的。
### 通过模拟系统进行跨平台编译
第二种方法是通过模拟系统进行跨平台编译，这种方法的好处是不需要交叉编译工具链，但是需要往 binfmt_misc 模块中注册 qemu 的二进制文件。
要使用这种方法，首先你的构建机需要在配置中启用 binfmt_misc 模块，如果你的构建机是 NixOS，将如下配置添加到你的 NixOS Module 即可启用 `aarch64-linux` 与 `riscv64-linux` 两种架构的模拟构建系统：
```nix
{ ... }:
{
# ......
# Enable binfmt emulation.
boot.binfmt.emulatedSystems = [ "aarch64-linux" "riscv64-linux" ];
# ......
}
```
至于 `flake.nix`，它的设置方法非常简单，比前面交叉编译的设置还要简单，示例如下：
```nix
{
description = "NixOS running on LicheePi 4A";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05";
};
outputs = inputs@{ self, nixpkgs, ... }: {
nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem {
# native platform
system = "riscv64-linux";
modules = [
# ...... other modules
];
};
};
}
```
可以看到我们未添加任何额外的模块，仅仅是指定了 `system` 为 `riscv64-linux`.
Nix 在构建时会自动检测当前系统是否为 `riscv64-linux`，如果不是，它会自动通过 QEMU 模拟系统进行构建，对用户而言这些底层操作完全是透明的。
#### Linux binfmt_misc
前面只说了怎么用，如果你想了解更底层的细节，这里也简单介绍一下。
binfmt_misc 是 Linux 内核的一项功能，全称是混杂二进制格式的内核支持（Kernel Support for miscellaneous Binary Formats），它能够使 Linux 支持运行几乎任何 CPU 架构的程序，包括 X86_64、ARM64、RISCV64 等。
为了能够让 binfmt_misc 运行任意格式的程序，至少需要做到两点：特定格式二进制程序的识别方式，以及其对应的解释器位置。虽然 binfmt_misc 听上去很强大，其实现的方式却意外地很容易理解，类似于 bash 解释器通过脚本文件的第一行（如#!/usr/bin/python3）得知该文件需要通过什么解释器运行，binfmt_misc 也预设了一系列的规则，如读取二进制文件头部特定位置的魔数，或者根据文件扩展名（如.exe、.py）以判断可执行文件的格式，随后调用对应的解释器去运行该程序。Linux 默认的可执行文件格式是 elf，而 binfmt_misc 的出现拓宽了 Linux 的执行限制，将一点展开成一个面，使得各种各样的二进制文件都能选择它们对应的解释器执行。
注册一种格式的二进制程序需要将一行有 `:name:type:offset:magic:mask:interpreter:flags` 格式的字符串写入 `/proc/sys/fs/binfmt_misc/register` 中，格式的详细解释这里就略过了。
由于人工写入上述 binfmt_misc 的注册信息比较麻烦，社区提供了一个容器来帮助我们自动注册，这个容器就是 binfmt，运行一下该容器就能安装各种格式的 binfmt_misc 模拟器了，举个例子：
```shell
# 注册所有架构
podman run --privileged --rm tonistiigi/binfmt:latest --install all
# 仅注册常见的 arm/riscv 架构
docker run --privileged --rm tonistiigi/binfmt --install arm64,riscv64,arm
```
binfmt_misc 模块自 Linux 2.6.12-rc2 版本中引入，先后经历了几次功能上的略微改动。
Linux 4.8 中新增“F”（fix binary，固定二进制）标志位，使 mount 命名空间变更和 chroot 后的环境中依然能够正常调用解释器执行二进制程序。由于我们需要构建多架构容器，必须使用“F”标志位才能 binfmt_misc 在容器中正常工作，因此内核版本需要在 4.8 以上才可以。
总的来说，比起一般情况显式调用解释器去执行非原生架构程序，binfmt_misc 产生的一个重要意义在于透明性。有了 binfmt_misc 后，用户在执行程序时不需要再关心要用什么解释器去执行，好像任何架构的程序都能够直接执行一样，而可配置的“F”标志位更是锦上添花，使解释器程序在安装时立即就被加载进内存，后续的环境改变也不会影响执行过程。
### 自定义构建工具链
有时候我们会需要使用自定义的工具链进行构建，比如使用自己编译的 gcc，或者使用自己编译的 musl libc 等等，这种修改可以通过 overlays 来实现。
举个例子，我们来尝试下使用使用不同的 gcc 版本，通过 `nix repl` 来测试下：
```shell
› nix repl -f '<nixpkgs>'
Welcome to Nix 2.13.3. Type :? for help.
Loading installable ''...
Added 17755 variables.
# 通过 overlays 替换掉 gcc
nix-repl> a = import <nixpkgs> { crossSystem = { config = "riscv64-unknown-linux-gnu"; }; overlays = [ (self: super: { gcc = self.gcc12; }) ]; }
# 查看下 gcc 版本，确实改成 12.2 了
nix-repl> a.pkgsCross.riscv64.stdenv.cc
«derivation /nix/store/jjvvwnf3hzk71p65x1n8bah3hrs08bpf-riscv64-unknown-linux-gnu-stage-final-gcc-wrapper-12.2.0.drv»
# 再看下未修改的 gcc 版本，还是 11.3
nix-repl> pkgs.pkgsCross.riscv64.stdenv.cc
«derivation /nix/store/pq3g0wq3yfc4hqrikr03ixmhqxbh35q7-riscv64-unknown-linux-gnu-stage-final-gcc-wrapper-11.3.0.drv»
```
那么如何在 Flakes 中使用这种方法呢？示例 `flake.nix` 内容如下:
```nix
{
description = "NixOS running on LicheePi 4A";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05-small";
};
outputs = { self, nixpkgs, ... }:
{
nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
modules = [
{
nigpkgs.crossSystem = {
config = "riscv64-unknown-linux-gnu";
};
# 改用 gcc12
nixpkgs.overlays = [ (self: super: { gcc = self.gcc12; }) ];
}
# other moduels ......
];
};
};
}
```
上面的方法会替换掉全局的 `pkgs.gcc`，很可能会导致大量的缓存失效，从而需要在本地本地构建非常多的 Nix 包。
为了避免这个问题，更好的办法是创建一个新的 `pkgs` 实例，仅在构建我们想修改的包时才使用这个实例，`flake.nix` 示例如下：
```nix
{
description = "NixOS running on LicheePi 4A";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05-small";
};
outputs = { self, nixpkgs, ... }: let
# 自定义一个新的 pkgs 实例，使用 gcc12
pkgs-gcc12 = import nixpkgs {
localSystem = "x86_64-linux";
crossSystem = {
config = "riscv64-unknown-linux-gnu";
};
overlays = [
(self: super: { gcc = self.gcc12; })
];
};
in {
nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
specialArgs = {
# pass the new pkgs instance to the module
inherit pkgs-gcc12;
};
modules = [
{
nigpkgs.crossSystem = {
config = "riscv64-unknown-linux-gnu";
};
}
({pkgs-gcc12, ...}: {
# 使用 pkgs-gcc12 实例
environment.systemPackages = [ pkgs-gcc12.hello ];
})
# other moduels ......
];
};
};
}
```
通过上述方法，我们可以很方便地自定义部分软件包的构建工具链，而不影响其他软件包的构建。
## References
- [Cross compilation - nix.dev](https://nix.dev/tutorials/cross-compilation)
- [容器镜像多架构支持介绍](https://www.cnblogs.com/frankming/p/16870285.html)

11
docs/zh/development/dev-environments.md Normal file
View file

@ -0,0 +1,11 @@
## Dev Environments
前面我们已经学习了构建开发环境的实现原理，但是每次都要自己写一堆重复性较高的 `flake.nix`，略显繁琐。
幸运的是，社区已经有人为我们做好了这件事，如下这个仓库中包含了绝大多数编程语言的开发环境模板，直接复制粘贴下来就能用：
- [dev-templates](https://github.com/the-nix-way/dev-templates)
如果你觉得 `flake.nix` 的结构还是太复杂了，希望能有更简单的方法，也可以考虑使用下面这个项目，它对 Nix 做了更彻底的封装，对用户提供了更简单的定义：
- [cachix/devenv](https://github.com/cachix/devenv)

147
docs/zh/development/distributed-building.md Normal file
View file

@ -0,0 +1,147 @@
## 分布式构建
分布式构建可以通过多台机器来分担本地的编译压力，加快构建速度。
NixOS 官方的 cache.nixos.org 中提供了绝大多数 X86_64 架构的缓存，因此对于普通 X86_64 的用户，一般不需要分布式构建。
分布式构建只在没有缓存可用的场景下才有较大应用价值，主要有这几种应用场景：
1. RISC-V 或 ARM64 架构的用户（尤其是 RISC-V），因为官方缓存仓库中这两个架构的缓存很少，导致经常需要大量本地编译。
2. 对系统进行大量定制的用户，因为官方缓存仓库中的 packages 都是默认配置，如果你改了构建参数，那么官方缓存就不适用了，这时候就需要本地编译。
1. 比如嵌入式场景下往往对底层内核、驱动等有定制需求，导致需要本地编译。
### 配置分布式构建
官方没有详细文档讲这个，我在文末列出了一些建议阅读的参考文档，同时如下是我的分布式构建配置（一个 NixOS Module）：
```nix
{ ... }: {
####################################################################
#
# NixOS's Configuration for Remote Building / Distributed Building
#
####################################################################
# set local's max-job to 0 to force remote building(disable local building)
# nix.settings.max-jobs = 0;
nix.distributedBuilds = true;
nix.buildMachines =
let
sshUser = "ryan";
# ssh key's path on local machine
sshKey = "/home/ryan/.ssh/ai-idols";
systems = [
# native arch
"x86_64-linux"
# emulated arch using binfmt_misc and qemu-user
"aarch64-linux"
"riscv64-linux"
];
# all available system features are poorly documentd here:
# https://github.com/NixOS/nix/blob/e503ead/src/libstore/globals.hh#L673-L687
supportedFeatures = [
"benchmark"
"big-parallel"
"kvm"
];
in
[
# Nix seems always give priority to trying to build remotely
# to make use of the local machine's high-performance CPU, do not set remote builder's maxJobs too high.
{
# some of my remote builders are running NixOS
# and has the same sshUser, sshKey, systems, etc.
inherit sshUser sshKey systems supportedFeatures;
# the hostName should be:
# 1. a hostname that can be resolved by DNS
# 2. the ip address of the remote builder
# 3. a host alias defined globally in /etc/ssh/ssh_config
hostName = "aquamarine";
# remote builder's max-job
maxJobs = 3;
# speedFactor's a signed integer
# but it seems that it's not used by Nix, takes no effect
speedFactor = 1;
}
{
inherit sshUser sshKey systems supportedFeatures;
hostName = "ruby";
maxJobs = 2;
speedFactor = 1;
}
{
inherit sshUser sshKey systems supportedFeatures;
hostName = "kana";
maxJobs = 2;
speedFactor = 1;
}
];
# optional, useful when the builder has a faster internet connection than yours
nix.extraOptions = ''
builders-use-substitutes = true
'';
# define the host alias for remote builders
# this config will be written to /etc/ssh/ssh_config
programs.ssh.extraConfig = ''
Host ai
HostName 192.168.5.100
Port 22
Host aquamarine
HostName 192.168.5.101
Port 22
Host ruby
HostName 192.168.5.102
Port 22
Host kana
HostName 192.168.5.103
Port 22
'';
# define the host key for remote builders so that nix can verify all the remote builders
# this config will be written to /etc/ssh/ssh_known_hosts
programs.ssh.knownHosts = {
# 星野 愛久愛海, Hoshino Aquamarine
aquamarine = {
hostNames = [ "aquamarine" "192.168.5.101" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDnCQXlllHoLX5EvU+t6yP/npsmuxKt0skHVeJashizE";
};
# 星野 瑠美衣, Hoshino Rubii
ruby = {
hostNames = [ "ruby" "192.168.5.102" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIE7n11XxB8B3HjdyAsL3PuLVDZxWCzEOUTJAY8+goQmW";
};
# 有馬 かな, Arima Kana
kana = {
hostNames = [ "kana" "192.168.5.103" ];
publicKey = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJ3dDLOZERP1nZfRz3zIeVDm1q2Trer+fWFVvVXrgXM1";
};
};
}
```
## 缺陷
目前我观察到的问题有：
1. 无法在构建时指定使用哪些主机，只能在配置文件中指定一个主机列表，然后 nix 会自动选择可用的主机。
2. 在选择主机时，我发现 Nix 总是优先选择远程主机，而我本地主机的性能最强，这导致本地主机的 CPU 无法充分利用。
3. 多机远程构建是以 Derivation 为单位的，因此在构建一些比较大的包时，其他机器可能会空闲很久，一直等这个大包构建完毕，这导致了资源的浪费。
1. 在构建的 packages 较多并且可以并行执行时，可以轻松将所有主机的 CPU 都用上，这确实非常爽。
## References
- [Distributed build - NixOS Wiki](https://nixos.wiki/wiki/Distributed_build)
- [Document available system features - nix#7380](https://github.com/NixOS/nix/issues/7380)
- [Distributed builds seem to disable local builds nix#2589](https://github.com/NixOS/nix/issues/2589)
- [Offloading NixOS builds to a faster machine](https://sgt.hootr.club/molten-matter/nix-distributed-builds/)
- [tests/nixos/remote-builds.nix - Nix Source Code](https://github.com/NixOS/nix/blob/713836112/tests/nixos/remote-builds.nix#L46)

295
docs/zh/development/intro.md Normal file
View file

@ -0,0 +1,295 @@
## 在 NixOS 上进行开发工作
由于 NixOS 自身可复现的特性，它非常适合用于搭建开发环境。
但是如果你想直接将在其他发行版上的环境搭建经验用在 NixOS 上，可能会遇到许多问题，因为 NixOS 有自己的一套逻辑在，下面我们先对此稍作说明。
在 NixOS 上，全局环境中只建议安装一些通用的工具，比如 `git`、`vim`、`emacs`、`tmux`、`zsh` 等等，而各语言的开发环境，最好是每个项目都有一个独立的环境。
为了简便，你也可以考虑提前为常用语言创建一些通用的开发环境，在需要时切换进去。
总而言之，NixOS 上的开发环境不应该装在全局，应该是一个个独立的项目环境，或许还带有一些通用的语言环境，但是它们都是完全隔离的，不会相互影响。
在本章中我们先学习一下 Nix Flakes 开发环境的实现原理，后面的章节再按使用场景介绍一些更具体的内容。
### 创建与使用开发环境
在 Nix Flakes 中，我们可以通过 `pkgs.mkShell { ... }` 来定义一个项目环境，通过 `nix develop` 来打开一个该开发环境的交互式 Bash Shell.
为了更好的使用上述两个功能，我们先来看看它们的原理。
[`pkgs.mkShell` 的源码](https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/mkshell/default.nix)如下：
```nix
{ lib, stdenv, buildEnv }:
# A special kind of derivation that is only meant to be consumed by the
# nix-shell.
{ name ? "nix-shell"
, # a list of packages to add to the shell environment
packages ? [ ]
, # propagate all the inputs from the given derivations
inputsFrom ? [ ]
, buildInputs ? [ ]
, nativeBuildInputs ? [ ]
, propagatedBuildInputs ? [ ]
, propagatedNativeBuildInputs ? [ ]
, ...
}@attrs:
let
mergeInputs = name:
(attrs.${name} or [ ]) ++
(lib.subtractLists inputsFrom (lib.flatten (lib.catAttrs name inputsFrom)));
rest = builtins.removeAttrs attrs [
"name"
"packages"
"inputsFrom"
"buildInputs"
"nativeBuildInputs"
"propagatedBuildInputs"
"propagatedNativeBuildInputs"
"shellHook"
];
in
stdenv.mkDerivation ({
inherit name;
buildInputs = mergeInputs "buildInputs";
nativeBuildInputs = packages ++ (mergeInputs "nativeBuildInputs");
propagatedBuildInputs = mergeInputs "propagatedBuildInputs";
propagatedNativeBuildInputs = mergeInputs "propagatedNativeBuildInputs";
shellHook = lib.concatStringsSep "\n" (lib.catAttrs "shellHook"
(lib.reverseList inputsFrom ++ [ attrs ]));
phases = [ "buildPhase" ];
# ......
# when distributed building is enabled, prefer to build locally
preferLocalBuild = true;
} // rest)
```
可以看到 `pkgs.mkShell { ... }` 本质上就是一个特殊的 Derivation（Nix 包），它的 `name` `buildInputs` 等参数都是可自定义的，而 `shellHook` 则是一个特殊的参数，它会在 `nix develop` 进入该环境时被执行。
如下是一份 `flake.nix` 文件，它定义了一个 nodejs 18 的开发环境：
```nix
{
description = "A Nix-flake-based Node.js development environment";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05";
};
outputs = { self , nixpkgs ,... }: let
# system should match the system you are running on
# system = "x86_64-linux";
system = "x86_64-darwin";
in {
devShells."${system}".default = let
pkgs = import nixpkgs {
inherit system;
overlays = [
(self: super: rec {
nodejs = super.nodejs-18_x;
pnpm = super.nodePackages.pnpm;
yarn = (super.yarn.override { inherit nodejs; });
})
];
};
in pkgs.mkShell {
# create an environment with nodejs-18_x, pnpm, and yarn
packages = with pkgs; [
node2nix
nodejs
pnpm
yarn
];
shellHook = ''
echo "node `${pkgs.nodejs}/bin/node --version`"
'';
};
};
}
```
建个空文件夹，将上面的配置保存为 `flake.nix`，然后执行 `nix develop`（或者更精确点，可以用 `nix develop .#default`），你会发现你已经进入了一个 nodejs 18 的开发环境，可以使用 `node` `npm` `pnpm` `yarn` 等命令了。而且刚进入时，`shellHook` 也被执行了，输出了当前 nodejs 的版本。
### 进入任何 Nix 包的构建环境
现在再来看看 `nix develop`，先读下 `nix develop --help` 输出的帮助文档：
```
Name
nix develop - run a bash shell that provides the build environment of a derivation
Synopsis
nix develop [option...] installable
# ......
```
可以看到 `nix develop` 接受的参数是 `installable`，这说明我们可以通过它进入任何一个 installable 的 Nix 包的开发环境，而不仅仅是 `pkgs.mkShell` 创建的环境。
默认情况下，`nix develop` 命令会尝试 flake outputs 中的如下属性：
- `devShells.<system>.default`
- `packages.<system>.default`
而如果我们通过 `nix develop /path/to/flake#<name>` 来指定了 flake 包地址以及 flake output name，那么 `nix develop` 命令会尝试 flake outputs 中的如下属性：
- `devShells.<system>.<name>`
- `packages.<system>.<name>`
- `legacyPackages.<system>.<name>`
现在来尝试一下，首先测试下，确认我当前环境中没有 `c++` `g++` 这这些编译相关的命令：
```shell
ryan in 🌐 aquamarine in ~
› c++
c++: command not found
ryan in 🌐 aquamarine in ~
› g++
g++: command not found
```
现在通过 `nix develop` 进入到 `hello` 的构建环境，然后再次测试下：
```shell
# login to the build environment of the package `hello`
ryan in 🌐 aquamarine in ~
› nix develop nixpkgs#hello
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› env | grep CXX
CXX=g++
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› c++ --version
g++ (GCC) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
ryan in 🌐 aquamarine in ~ via ❄️ impure (hello-2.12.1-env)
› g++ --version
g++ (GCC) 12.3.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```
可以看到 `CXX` `CXXCPP` 环境变量已经被设置好了，而且 `c++` `g++` 等命令也可以正常使用了。
此外我们还可以正常调用 `hello` 这个 Nix 包的各构建阶段命令：
> 提前说明下，一个 Nix 包的所有构建阶段及其默认的执行顺序为：`$prePhases unpackPhase patchPhase $preConfigurePhases configurePhase $preBuildPhases buildPhase checkPhase $preInstallPhases installPhase fixupPhase installCheckPhase $preDistPhases distPhase $postPhases`
```shell
# 解压源码包
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› unpackPhase
unpacking source archive /nix/store/pa10z4ngm0g83kx9mssrqzz30s84vq7k-hello-2.12.1.tar.gz
source root is hello-2.12.1
setting SOURCE_DATE_EPOCH to timestamp 1653865426 of file hello-2.12.1/ChangeLog
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› ls
hello-2.12.1
ryan in 🌐 aquamarine in /tmp/xxx via ❄️ impure (hello-2.12.1-env)
› cd hello-2.12.1/
# generate Makefile
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via ❄️ impure (hello-2.12.1-env)
› configurePhase
configure flags: --prefix=/tmp/xxx/outputs/out --prefix=/tmp/xxx/outputs/out
checking for a BSD-compatible install... /nix/store/02dr9ymdqpkb75vf0v1z2l91z2q3izy9-coreutils-9.3/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /nix/store/02dr9ymdqpkb75vf0v1z2l91z2q3izy9-coreutils-9.3/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
# ......
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating po/Makefile.in
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
config.status: executing po-directories commands
config.status: creating po/POTFILES
config.status: creating po/Makefile
# build the package
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via C v12.3.0-gcc via ❄️ impure (hello-2.12.1-env) took 2s
› buildPhase
build flags: SHELL=/run/current-system/sw/bin/bash
make all-recursive
make[1]: Entering directory '/tmp/xxx/hello-2.12.1'
# ......
ranlib lib/libhello.a
gcc -g -O2 -o hello src/hello.o ./lib/libhello.a
make[2]: Leaving directory '/tmp/xxx/hello-2.12.1'
make[1]: Leaving directory '/tmp/xxx/hello-2.12.1'
# run the built program
ryan in 🌐 aquamarine in /tmp/xxx/hello-2.12.1 via C v12.3.0-gcc via ❄️ impure (hello-2.12.1-env)
› ./hello
Hello, world!
```
这种用法的主要应用场景是调试某个 Nix 包的构建过程，或者在某个 Nix 包的构建环境中执行一些命令。
### `nix shell` 与 `nix run`
与 `nix develop` 相比，这两个命令就简单且好理解很多了。
`nix shell` 用于进入到一个含有指定 Nix 包的环境并为它打开一个交互式 shell：
```shell
# hello 不存在
› hello
hello: command not found
# 进入到一个含有 hello 的 shell 环境
› nix shell nixpkgs#hello
# hello 可以用了
› hello
Hello, world!
```
`nix run` 则是创建一个含有指定 Nix 包的环境，并在该环境中直接运行该 Nix 包：
```shell
# hello 不存在
› hello
hello: command not found
# 创建一个含有 hello 的环境并运行它
› nix run nixpkgs#hello
Hello, world!
```
因为 `nix run` 会直接将 Nix 包运行起来，所以作为其参数的 Nix 包必须能生成一个可执行程序。
根据 `nix run --help` 的说明，`nix run` 会执行 `<out>/bin/<name>` 这个命令，其中 `<out>` 是一个 Derivation 的根目录，`<name>` 则按如下顺序进行选择尝试：
- Derivation 的 `meta.mainProgram` 属性
- Derivation 的 `pname` 属性
- Derivation 的 `name` 属性中去掉版本号后的内容
比如说我们上面测试的包 hello，`nix run` 实际会执行 `$out/bin/hello` 这个程序。
## References
- [pkgs.mkShell - nixpkgs manual](https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-mkShell)
- [A minimal nix-shell](https://fzakaria.com/2021/08/02/a-minimal-nix-shell.html)
- [One too many shell, Clearing up with nix' shells nix shell and nix-shell - Yannik Sander](https://blog.ysndr.de/posts/guides/2021-12-01-nix-shells/)

116
docs/zh/development/kernel-development.md Normal file
View file

@ -0,0 +1,116 @@
# Kernel Development
> WIP 本文还有待完善
一个我 licheepi4a 官方内核开发调试环境的 `flake.nix` 如下
```nix
{
description = "NixOS running on LicheePi 4A";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-23.05-small";
# custom kernel's source
thead-kernel = {
url = "github:revyos/thead-kernel/lpi4a";
flake = false;
};
};
outputs = inputs@{
self
,nixpkgs
,thead-kernel
,... }:
let
pkgsKernel = import nixpkgs {
localSystem = "x86_64-linux";
crossSystem = {
config = "riscv64-unknown-linux-gnu";
};
overlays = [
(self: super: {
# use gcc 13 to compile this custom kernel
linuxPackages_thead = super.linuxPackagesFor (super.callPackage ./pkgs/kernel {
src = thead-kernel;
stdenv = super.gcc13Stdenv;
kernelPatches = with super.kernelPatches; [
bridge_stp_helper
request_key_helper
];
});
})
];
};
in
{
nixosConfigurations.lp4a = nixpkgs.lib.nixosSystem {
system = "x86_64-linux";
specialArgs = {
inherit nixpkgs pkgsKernel;
};
modules = [
{
# cross-compile this flake.
nixpkgs.crossSystem = {
system = "riscv64-linux";
};
}
./modules/licheepi4a.nix
./modules/sd-image-lp4a.nix
];
};
# use `nix develop .#kernel` to enter the environment with the custom kernel build environment available.
# and then use `unpackPhase` to unpack the kernel source code and cd into it.
# then you can use `make menuconfig` to configure the kernel.
#
# problem
# - using `make menuconfig` - Unable to find the ncurses package.
devShells.x86_64-linux.kernel = pkgsKernel.linuxPackages_thead.kernel.dev;
# use `nix develop .#fhs` to enter the fhs test environment defined here.
devShells.x86_64-linux.fhs = let
pkgs = import nixpkgs {
system = "x86_64-linux";
};
in
# the code here is mainly copied from:
# https://nixos.wiki/wiki/Linux_kernel#Embedded_Linux_Cross-compile_xconfig_and_menuconfig
(pkgs.buildFHSUserEnv {
name = "kernel-build-env";
targetPkgs = pkgs_: (with pkgs_;
[
# we need theses packages to run `make menuconfig` successfully.
pkgconfig
ncurses
pkgsKernel.gcc13Stdenv.cc
gcc
]
++ pkgs.linux.nativeBuildInputs);
runScript = pkgs.writeScript "init.sh" ''
# set the cross-compilation environment variables.
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
export ARCH=riscv
export PKG_CONFIG_PATH="${pkgs.ncurses.dev}/lib/pkgconfig:"
exec bash
'';
}).env;
};
}
```
通过上面的 `flake.nix`，我可以通过 `nix develop .#kernel` 进入到内核的构建环境中，执行 `unpackPhase` 解压出内核源码。
但是不能执行 `make menuconfig` 进行内核的配置，因为该环境中缺少 `ncurses` 等包。
所以我第二步是退出再通过 `nix develop .#fhs` 进入到另一个添加了必需包的 FHS 环境中，再执行 `make menuconfig` 进行内核的配置，以及后续的构建调试。
## References
- https://github.com/jordanisaacs/kernel-module-flake

20
docs/zh/development/packaging-101.md Normal file
View file

@ -0,0 +1,20 @@
# Packging
TODO
### 1. stdenv 构建介绍
TODO
### 2. language specific frameworks
TODO
## References
- [NixOS Series 3: Software Packaging 101](https://lantian.pub/en/article/modify-computer/nixos-packaging.lantian/)
- [stdenv - Nixpkgs Manual](https://github.com/NixOS/nixpkgs/tree/nixos-unstable/doc/languages-frameworks)
- [languages-frameworks - Nixpkgs Manual](https://github.com/NixOS/nixpkgs/tree/nixos-unstable/doc/stdenv)

2
docs/zh/nixos-with-flakes/introduction-to-flakes.md
View file

@ -28,7 +28,7 @@ Nix 于 2020 年推出了 `nix-command` & `flakes` 两个新特性，它们提
1. 通过 `nix-env` 安装的包不会被自动记录到 Nix 的声明式配置中，是完全脱离掌控的，无法在其他主机上复现，因此不推荐使用。 1. 通过 `nix-env` 安装的包不会被自动记录到 Nix 的声明式配置中，是完全脱离掌控的，无法在其他主机上复现，因此不推荐使用。
2. 在 Nix Flakes 中对应的命令为 `nix profile`，此命令也同样不推荐使用。 2. 在 Nix Flakes 中对应的命令为 `nix profile`，此命令也同样不推荐使用。
3. `nix-shell`: nix-shell 用于创建一个临时的 shell 环境 3. `nix-shell`: nix-shell 用于创建一个临时的 shell 环境
1. 在 Nix Flakes 中它被 `nix develop` 与 `nix shell` 取代了。 1. 这玩意儿可能有点复杂了，因此在 Nix Flakes 中它被拆分成了三个子命令 `nix develop`, `nix shell` 以及 `nix run`，我们会在「构建开发环境」一章详细介绍这三个命令。
4. `nix-build`: 用于构建 Nix 包，它会将构建结果放到 `/nix/store` 路径下，但是不会记录到 Nix 的声明式配置中。 4. `nix-build`: 用于构建 Nix 包，它会将构建结果放到 `/nix/store` 路径下，但是不会记录到 Nix 的声明式配置中。
1. 在 Nix Flakes 中对应的命令为 `nix build` 1. 在 Nix Flakes 中对应的命令为 `nix build`
5. ... 5. ...