Skip to content

Instantly share code, notes, and snippets.

@MatrixManAtYrService

MatrixManAtYrService/0_intro.md

Last active March 19, 2025 09:17
Show Gist options
  • Save MatrixManAtYrService/95459c761449dafd8a24da10f2d5a5be to your computer and use it in GitHub Desktop.
Save MatrixManAtYrService/95459c761449dafd8a24da10f2d5a5be to your computer and use it in GitHub Desktop.
Download ZIP
Raw
0_intro.md

This gist contains notes that I took while installing NixOS onto a nanopi r5s. Many thanks to the kind folks in this issue who helped me along the way.

Raw
1_nixos_on_nanopi_r5s.md

Prepare the image builder

Follow the readme at https://github.com/bdew/nanopi-image.

This involves modifying my builder machine's configuration.nix so that it was prepared to emulate aarch64-linux. After making the change and running running nixos-rebuild I rebooted that machine since the modifications were under boot.*, not sure if that was necessary.

Build and decompress the image

git clone [email protected]:bdew/nanopi-image.git
cd nanopi-image
nix build
nix-shell -p zstd
zstd -d result/nanopi-r5s-nixos.img.zst -o nanopi-r5s-nixos.img
exit

Determine sd card path

I knew I had an 8GB card, so it was easy to pick out

❯  lsblk
   NAME                                          MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
   sda                                             8:0    1   7.4G  0 disk
   └─sda1                                          8:1    1   7.4G  0 part

For me it's /dev/sda, yours may differ

Write image to SD card

Careful, this will overwrite the partition table, and much data, for whichever device you point it at.

sudo sh -c 'cat nanopi-r5s-nixos.img > /dev/sdX && sync'

Boot nanopi-r5s to newly written SD card

user: nix password: nix

Figure out ip address

ifconfig

For me is was 192.168.90.220

Copy image to SD card

Back on the builder machine:

scp nanopi-r5s-nixos.img [email protected]:~

SSH to nanopi

You can proceed directly on the nanopi-r5s if you prefer, but I like my terminal emulator better.

ssh [email protected]

Determine device for eMMC

$ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
mmcblk1      179:0    0  28.9G  0 disk
└─mmcblk1p1  179:1    0  28.9G  0 part
mmcblk1boot0 179:32   0     4M  1 disk
mmcblk1boot1 179:64   0     4M  1 disk
mmcblk0      179:96   0   7.4G  0 disk
└─mmcblk0p1  179:97   0   7.4G  0 part /nix/store

Mine was mmcblk1, identified based on the size.

Write image to eMMC

Consider backing up the contents of your eMMC first. I didn't because I live dangerously... and pay for it often.

sudo sh -c 'cat nanopi-r5s-nixos.img > /dev/mmcblk1 && sync'

Two NIXOS's

$ lsblk -f
    NAME         FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
    mmcblk1
    └─mmcblk1p1  ext4   1.0   NIXOS      44444444-4444-4444-8888-888888888888                /nix/store
    mmcblk1boot0
    mmcblk1boot1
    mmcblk0
    └─mmcblk0p1  ext4   1.0   NIXOS      44444444-4444-4444-8888-888888888888    1.6G    73% /nix/store

What to do with two NixOS's?

Notice that the configuration.nix in the image has this:

  config = {
    fileSystems = {
      "/" = {
        device = "/dev/disk/by-label/NIXOS";
        fsType = "ext4";
      };
      "/var/log" = {
        fsType = "tmpfs";
      };
    };

My plan is to use the eMMC for /boot and my nvme disk for everything else.

Relabel Partitions

Let's re-label the eMMC partition so that it's clear that it's for booting.

While we're at it, we'll generate random UUIDs for both partitions involved, because UUIDs are not supposed to overlap (not strictly necessary but who knows if it will create confusion later on).

lsblk -f
	NAME         FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
	mmcblk1
	└─mmcblk1p1  ext4   1.0   NIXOS      44444444-4444-4444-8888-888888888888                /nix/store
	mmcblk1boot0
	mmcblk1boot1
	mmcblk0
	└─mmcblk0p1  ext4   1.0   NIXOS      44444444-4444-4444-8888-888888888888    1.6G    73% /nix/store
   nvme0n1

sudo e2label /dev/mmcblk1p1 NIXOS_BOOT
sudo tune2fs -U random /dev/mmcblk1p1
sudo tune2fs -U random /dev/mmcblk0p1

lsblk -f
	NAME         FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
	mmcblk1
	└─mmcblk1p1  ext4   1.0   NIXOS_BOOT 9447ed30-51f7-4257-b165-fcfdb63d0442
	mmcblk1boot0
	mmcblk1boot1
	mmcblk0
	└─mmcblk0p1  ext4   1.0   NIXOS      dbdd68df-78df-4823-81d9-00fd2e07d8a7    1.6G    73% /nix/store
   nvme0n1

Prepare NVMe partition

Get a nix shell with parted. (If you prefer vi to nano, consider adding vim to the nix-shell also, we'll use it later. You can ignore warnings about Nix search path)

Initialize the partition table. (Only do this if you're ok with losing whatever is on that drive).

nix-shell -p parted
sudo parted /dev/nvme0n1 mklabel gpt

Create some swap space (optional).

sudo parted /dev/nvme0n1 mkpart primary linux-swap 1MiB 8GiB
sudo mkswap -L NIXOS_SWAP /dev/nvme0n1p1

Create the root partition.

sudo parted /dev/nvme0n1 mkpart primary ext4 8GiB 100%
sudo mkfs.ext4 -L NIXOS_ROOT /dev/nvme0n1p2

Resize the eMMC partition and filesystem

Later on, the NixOS installer will need a bit more space than it currently has. This is because the image on the eMMC has not been booted to, so it hasn't had the chance to expand to fill the whole device.

If you've removed the SD card and booted to the eMMC at any point, this step is not necessary, since it happens on boot.

sudo parted /dev/mmcblk1 ---pretend-input-tty resizepart 1 100%
sudo resize2fs /dev/mmcblk1p1

Check it

$ lsblk -f
	NAME         FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
	mmcblk1
	└─mmcblk1p1  ext4   1.0   NIXOS_BOOT 9447ed30-51f7-4257-b165-fcfdb63d0442   24.7G     9%
	mmcblk1boot0
	mmcblk1boot1
	mmcblk0
	└─mmcblk0p1  ext4   1.0   NIXOS      dbdd68df-78df-4823-81d9-00fd2e07d8a7    1.6G    73% /nix/store
	                                                                                         /
	nvme0n1
	├─nvme0n1p2  ext4   1.0   NIXOS_ROOT 0f630180-9411-4da2-9ad7-07caf8d0626f    424G     1% 
	└─nvme0n1p1  swap   1     NIXOS_SWAP f0bbd20d-2ae1-4b8f-b9f7-f62967309fe8

  • NIXOS is the label that the image built with, that goes with our SD card. If we need to tinker with the other parts, we can insert the SD card and the device will boot here, that way nothing else will be mounted and we'll be able to make changes.

  • NIXOS_BOOT refers to the eMMC, the only part of it that we care about is its /boot folder, which U-boot will find, but for now it is identical to the SD card except for its label

  • NIXOS_SWAP is swap space (not yet used)

  • NIXOS_ROOT is empty space where we're about to put NixOS

Prepare /mnt

Mount the partitions we'll be installing to (this is covered in the NixOS manual)

sudo mkdir /mnt
sudo mount /dev/disk/by-label/NIXOS_ROOT /mnt

sudo mkdir /mnt/boot
sudo mount /dev/disk/by-label/NIXOS_BOOT /mnt/boot

sudo swapon /dev/disk/by-label/NIXOS_SWAP

Check it again.

$ lsblk -f
	NAME         FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
	mmcblk1
	└─mmcblk1p1  ext4   1.0   NIXOS_BOOT 9447ed30-51f7-4257-b165-fcfdb63d0442   13.3M    94% /mnt/boot
	mmcblk1boot0
	mmcblk1boot1
	mmcblk0
	└─mmcblk0p1  ext4   1.0   NIXOS      dbdd68df-78df-4823-81d9-00fd2e07d8a7    1.6G    73% /nix/store
	                                                                                         /
	nvme0n1
	├─nvme0n1p2  ext4   1.0   NIXOS_ROOT 0f630180-9411-4da2-9ad7-07caf8d0626f  426.6G     0% /mnt
	└─nvme0n1p1  swap   1     NIXOS_SWAP f0bbd20d-2ae1-4b8f-b9f7-f62967309fe8                [SWAP]

Generate nixos configs

sudo nixos-generate-config --root /mnt
    writing /mnt/etc/nixos/hardware-configuration.nix...
    writing /mnt/etc/nixos/configuration.nix...

Update configuration.nix to use our lables

The generated configuration.nix is less appropriate than the ones that came bunded with the image, overwrite it from the unchanged copy on the SD card (recall that /mnt is where we're installing TO, and / is where we're installing from)

sudo cp /etc/nixos/configuration.nix /mnt/etc/nixos/configuration.nix

We've modified some partition labels, so let's edit our new configuration.nix to use them.

sudo nano /mnt/etc/nixos/configuration.nix

Change this:

  config = {
    fileSystems = {
      "/" = {
        device = "/dev/disk/by-label/NIXOS";
        fsType = "ext4";
      };
      "/var/log" = {
        fsType = "tmpfs";
      };
    };

To this:

  config = {
    fileSystems = {
      "/" = {
        device = "/dev/disk/by-label/NIXOS_ROOT";
        fsType = "ext4";
      };
      "/boot" = {
        device = "/dev/disk/by-label/NIXOS_BOOT";
        fsType = "ext4";
      };
      "/var/log" = {
        fsType = "tmpfs";
      };
    };

Install NixOS

Now, when the installer (or nixos-rebuild) runs, it will update the partitions with those labels. Previously, we mapped those labels such that /boot goes on the eMMC, and / goes on the nvme drive.

We're ready, let's install.

sudo nixos-install --root /mnt

Done?

After this completes, you should be able to power off the device, remove the SD card, and when it boots, / will be on the nvme drive, as desired.

Before we do that though, let's recap and maybe clean up.

Compare the SD card boot config to the eMMC boot config:

$ cat /mnt/boot/extlinux/extlinux.conf
	# Generated file, all changes will be lost on nixos-rebuild!

	# Change this to e.g. nixos-42 to temporarily boot to an older configuration.
	DEFAULT nixos-default

	MENU TITLE ------------------------------------------------------------
	TIMEOUT 10

	LABEL nixos-default
	  MENU LABEL NixOS - Default
	  LINUX ../nixos/hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-Image
	  INITRD ../nixos/7wb7ls1b5yzbw31dd0hm7nm6880hpkzg-initrd-linux-6.13.2-initrd
	  APPEND init=/nix/store/xl6ngc9lpd6nsvjxgvx50g7dsxr9whnf-nixos-system-nixos-24.11pre-git/init console=tty0 earlycon=uart8250,mmio32,0xfe660000 loglevel=4
	  FDT ../nixos/hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-dtbs/rockchip/rk3568-nanopi-r5s.dtb

	LABEL nixos-1-default
	  MENU LABEL NixOS - Configuration 1-default (2025-02-14 03:46 - 24.11pre-git)
	  LINUX ../nixos/hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-Image
	  INITRD ../nixos/7wb7ls1b5yzbw31dd0hm7nm6880hpkzg-initrd-linux-6.13.2-initrd
	  APPEND init=/nix/store/xl6ngc9lpd6nsvjxgvx50g7dsxr9whnf-nixos-system-nixos-24.11pre-git/init console=tty0 earlycon=uart8250,mmio32,0xfe660000 loglevel=4
	  FDT ../nixos/hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-dtbs/rockchip/rk3568-nanopi-r5s.dtb

$ cat /boot/extlinux/extlinux.conf
	# Generated file, all changes will be lost on nixos-rebuild!

	# Change this to e.g. nixos-42 to temporarily boot to an older configuration.
	DEFAULT nixos-default

	MENU TITLE ------------------------------------------------------------
	TIMEOUT 10

	LABEL nixos-default
	  MENU LABEL NixOS - Default
	  LINUX ../nixos/hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-Image
	  INITRD ../nixos/ysblmbd10vv532dxa9a6kvrrcpvrdnyc-initrd-linux-6.13.2-initrd
	  APPEND init=/nix/store/7clw9zzwzzv9iwwfa2b2m9c3d1xjcnc9-nixos-system-nixos-24.11.20250208.a45fa36/init console=tty0 earlycon=uart8250,mmio32,0xfe660000 loglevel=4
	  FDT ../nixos/hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-dtbs/rockchip/rk3568-nanopi-r5s.dtb

We can see that the installer added a new entry. This might look familliar if you're used to booting with GRUB.

I was suprised to see this, because I did't notice any output along these lines while the board was booting. This is because it happens before HDMI kicks in. If you want to see it, you need to connect over TTL serial. More on that in a moment, you might never have to bother with it.

At this time we have three complete nixos installs. One is on the SD card, the other is on the eMMC (mostly unused, except for /boot), and the last is on the nvme drive.

I am worried that I will one day become confused about which one I have booted to. Removing the SD card will remove the confusion for one of them.

In order to remove confusion for the other, I will delete /mnt/boot/etc (so there's only one configuration.nix) and /mnt/boot/nix (so there's only one /nix/store).

Think first, these could hurt if you miss:

sudo rm -r /mnt/boot/etc
sudo rm -r /mnt/boot/nix

Now there's just one /nix and one /etc, no room for ambiguity.

A Mystery Remains

There are multiple structures present within /mnt/boot...

$ ls -l /mnt/boot
drwxr-xr-x 4 root root   4096 Jan  1  1970 boot
drwxr-xr-x 2 root root   4096 Feb 14 04:12 extlinux
drwx------ 2 root root  16384 Jan  1  1980 lost+found
drwxr-xr-x 3 root root   4096 Feb 14 04:12 nixos
-r--r--r-- 1 root root 220868 Jan  1  1970 nix-path-registration

$ tree /mnt/boot -L 2
/mnt/boot
├── boot
│   ├── extlinux
│   └── nixos
├── extlinux
│   ├── extlinux.conf
│   └── extlinux.conf.tmp.53
└─── nixos
    ├── 7wb7ls1b5yzbw31dd0hm7nm6880hpkzg-initrd-linux-6.13.2-initrd
    ├── hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-dtbs
    └── hahk5dlcl232fcv267lk1cwfxvp4ccrg-linux-6.13.2-Image

...two separate extlinux directories , and two separate nixos folders as well.

Probably I could get away with deleting one of these (whichever U-Boot doesn't need). I found this post which indicates that /extlinux/extlinux.conf will be preferred over the other, but I'm not feeling so adventurous as to delete the others just to find out.

Reboot into NixOS on the NVMe

Now it's time to reboot and 🤞 ...

...It worked! After booting without an SD card and logging in, I saw found that everything was where I wanted it:

$ lsblk -f
	NAME         FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
	mmcblk1
	└─mmcblk1p1  ext4   1.0   NIXOS_BOOT 9447ed30-51f7-4257-b165-fcfdb63d0442   26.9G     1% /boot
	mmcblk1boot0
	mmcblk1boot1
	nvme0n1
	├─nvme0n1p1  swap   1     NIXOS_SWAP f0bbd20d-2ae1-4b8f-b9f7-f62967309fe8                [SWAP]
	└─nvme0n1p2  ext4   1.0   NIXOS_ROOT 0f630180-9411-4da2-9ad7-07caf8d0626f    424G     1% /nix/store
Raw
2_debug-UART.md

Previous attempts to do this failed in a way that just gave me a blank screen over HDMI. I realized later that in order to see what was going on in this case, I needed to connect over TTL serial.

For that I used a USB-to-serial adapter. I had one of these lying around. From the nanopi wiki I have learned that this connection happens at 1500000 baud. As it turns out, that's too slow for my adapter. I had to buy a new one.

The one that worked had a FTDI FT232RL chip.

IMG_20250213_220016_095

In order to make this work I had to solder a three-pin header onto the board near the USB-C port. Then I wired the USB adapter to the board.

nanopi-r5s        usb adapter
    tx       <-->     rx
    rx       <-->     tx
    gnd      <-->     gnd

(note how the white/grey wires cross to match transmit with receive)

IMG_20250210_151457_199

After this I discovered the device name by diffing the output of ls /dev before and after plugging it in. It appeared as /dev/ttyUSB0.

To use it I ran:

nix-shell -p minicom
minicom -b 1500000 -D -d /dev/ttyUSB0

As soon as I got it up and running, I discovered that I didn't need it after all. Funny how that works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment