path: root/docs/research-sandreas-zarch.org

#+TITLE: Research: sandreas/zarch ZFSBootMenu Installation
#+DATE: 2026-01-22
#+AUTHOR: Research Notes

* Overview

This document summarizes research on the [[https://github.com/sandreas/zarch][sandreas/zarch]] GitHub repository for
Arch Linux ZFS installation. The project uses ZFSBootMenu, native encryption,
and automatic snapshots via zrepl.

* Project Philosophy

sandreas/zarch is described as a "single, non-modular file with some minor
config profiles" - the author explicitly avoids a "modular multi-script beast."
This contrasts with our more modular approach but offers useful patterns.

** Key Features
- ZFSBootMenu as bootloader (not GRUB)
- Native ZFS encryption (AES-256-GCM)
- Automatic snapshots via zrepl
- EFI-only (no BIOS support)
- Profile-based configuration

* ZFSBootMenu Installation

** Download and Install
#+begin_src bash
# Create EFI directory
mkdir -p /efi/EFI/ZBM

# Download latest ZFSBootMenu EFI binary
wget -c https://get.zfsbootmenu.org/latest.EFI -O /efi/EFI/ZBM/ZFSBOOTMENU.EFI

# Or use curl variant
curl -o /boot/efi/EFI/ZBM/VMLINUZ.EFI -L https://get.zfsbootmenu.org/efi
#+end_src

** EFI Boot Entry Registration
#+begin_src bash
efibootmgr --disk $DISK --part 1 \
    --create \
    --label "ZFSBootMenu" \
    --loader '\EFI\ZBM\ZFSBOOTMENU.EFI' \
    --unicode "spl_hostid=$(hostid) zbm.timeout=3 zbm.prefer=zroot zbm.import_policy=hostid" \
    --verbose
#+end_src

** Key ZFSBootMenu Parameters
| Parameter              | Purpose                                        |
|------------------------+------------------------------------------------|
| zbm.timeout=N          | Seconds to wait before auto-booting default    |
| zbm.prefer=POOL        | Preferred pool for default boot environment    |
| zbm.import_policy      | Pool import strategy (hostid recommended)      |
| zbm.skip               | Skip menu and boot default immediately         |
| zbm.show               | Force menu display                             |
| spl_hostid=0xXXXXXXXX  | Host ID for pool import validation             |

** Kernel Command Line for Boot Environments
#+begin_src bash
# Set inherited command line on ROOT dataset
zfs set org.zfsbootmenu:commandline="quiet loglevel=0" zroot/ROOT

# Set pool bootfs property
zpool set bootfs=zroot/ROOT/arch zroot
#+end_src

* Dataset Layout

** zarch Dataset Structure
#+begin_example
$POOL                    mountpoint=none
$POOL/ROOT               mountpoint=none (container for boot environments)
$POOL/ROOT/arch          mountpoint=/, canmount=noauto (active root)
$POOL/home               mountpoint=/home (shared across boot environments)
#+end_example

** Comparison: Our archzfs Dataset Structure
#+begin_example
zroot                    mountpoint=none, canmount=off
zroot/ROOT               mountpoint=none, canmount=off
zroot/ROOT/default       mountpoint=/, canmount=noauto, reservation=5-20G
zroot/home               mountpoint=/home
zroot/home/root          mountpoint=/root
zroot/media              mountpoint=/media, compression=off
zroot/vms                mountpoint=/vms, recordsize=64K
zroot/var                mountpoint=/var, canmount=off
zroot/var/log            mountpoint=/var/log
zroot/var/cache          mountpoint=/var/cache
zroot/var/lib            mountpoint=/var/lib, canmount=off
zroot/var/lib/pacman     mountpoint=/var/lib/pacman
zroot/var/lib/docker     mountpoint=/var/lib/docker
zroot/var/tmp            mountpoint=/var/tmp, auto-snapshot=false
zroot/tmp                mountpoint=/tmp, auto-snapshot=false
#+end_example

** Key Differences
- zarch: Minimal dataset layout (ROOT, home)
- archzfs: Fine-grained datasets with workload-specific tuning
- archzfs: Separate /var/log, /var/cache, /var/lib/docker
- archzfs: recordsize=64K for VM storage
- archzfs: compression=off for media (already compressed)

* ZFS Pool Creation

** zarch Pool Creation (with encryption)
#+begin_src bash
zpool create -f \
    -o ashift=12 \
    -O compression=lz4 \
    -O acltype=posixacl \
    -O xattr=sa \
    -O relatime=off \
    -O atime=off \
    -O encryption=aes-256-gcm \
    -O keylocation=prompt \
    -O keyformat=passphrase \
    -o autotrim=on \
    -m none \
    $POOL ${DISK}-part2
#+end_src

** Our archzfs Pool Creation (with encryption)
#+begin_src bash
zpool create -f \
    -o ashift="$ASHIFT" \
    -o autotrim=on \
    -O acltype=posixacl \
    -O atime=off \
    -O canmount=off \
    -O compression="$COMPRESSION" \
    -O dnodesize=auto \
    -O normalization=formD \
    -O relatime=on \
    -O xattr=sa \
    -O encryption=aes-256-gcm \
    -O keyformat=passphrase \
    -O keylocation=prompt \
    -O mountpoint=none \
    -R /mnt \
    "$POOL_NAME" $pool_config
#+end_src

** Key Differences
| Option          | zarch             | archzfs               | Notes                           |
|-----------------+-------------------+-----------------------+---------------------------------|
| compression     | lz4               | zstd (configurable)   | zstd better ratio, more CPU     |
| atime           | off               | off                   | Same                            |
| relatime        | off               | on                    | archzfs uses relatime instead   |
| dnodesize       | (default)         | auto                  | Better extended attribute perf  |
| normalization   | (default)         | formD                 | Unicode consistency             |

* Snapshot Automation

** zarch: zrepl Configuration

zarch uses zrepl for automated snapshots with this retention grid:

#+begin_example
1x1h(keep=4) | 24x1h(keep=1) | 7x1d(keep=1) | 4x1w(keep=1) | 12x4w(keep=1) | 1x53w(keep=1)
#+end_example

This means:
- Keep 4 snapshots within the last hour
- Keep 1 snapshot per hour for 24 hours
- Keep 1 snapshot per day for 7 days
- Keep 1 snapshot per week for 4 weeks
- Keep 1 snapshot per 4 weeks for 12 periods (48 weeks)
- Keep 1 snapshot per year

#+begin_src yaml
# Example zrepl.yml structure
jobs:
  - name: snapjob
    type: snap
    filesystems:
      "zroot<": true
    snapshotting:
      type: periodic
      interval: 15m
      prefix: zrepl_
    pruning:
      keep:
        - type: grid
          grid: 1x1h(keep=all) | 24x1h | 14x1d
          regex: "^zrepl_.*"
        - type: regex
          negate: true
          regex: "^zrepl_.*"
#+end_src

** archzfs: Pacman Hook Approach

Our approach uses pre-transaction snapshots:
#+begin_src bash
# /etc/pacman.d/hooks/zfs-snapshot.hook
[Trigger]
Operation = Upgrade
Operation = Install
Operation = Remove
Type = Package
Target = *

[Action]
Description = Creating ZFS snapshot before pacman transaction...
When = PreTransaction
Exec = /usr/local/bin/zfs-pre-snapshot
#+end_src

** Comparison: Snapshot Approaches
| Feature           | zrepl (zarch)            | Pacman Hook (archzfs)        |
|-------------------+--------------------------+------------------------------|
| Trigger           | Time-based (15 min)      | Event-based (pacman)         |
| Retention         | Complex grid policy      | Manual or sanoid             |
| Granularity       | High (frequent)          | Package transaction focused  |
| Recovery Point    | ~15 minutes              | Last package operation       |
| Storage overhead  | Higher (more snapshots)  | Lower (fewer snapshots)      |

** Alternative: sanoid (mentioned in archzfs)
Sanoid provides similar functionality to zrepl with simpler configuration:
#+begin_src ini
# /etc/sanoid/sanoid.conf
[zroot/ROOT/default]
use_template = production
recursive = yes

[template_production]
frequently = 0
hourly = 24
daily = 7
weekly = 4
monthly = 12
yearly = 1
autosnap = yes
autoprune = yes
#+end_src

* EFI and Boot Partition Strategy

** zarch: 512MB EFI, ZFSBootMenu
- Single 512MB EFI partition (type EF00)
- ZFSBootMenu EFI binary downloaded from upstream
- No GRUB, no separate boot partition on ZFS
- Kernel/initramfs stored on ZFS root (ZFSBootMenu reads them)

** archzfs: 1GB EFI, GRUB with ZFS Support
- 1GB EFI partition per disk
- GRUB with ZFS module for pool access
- Redundant EFI partitions synced via rsync
- Boot files in EFI partition (not ZFS)

** Trade-offs

| Aspect              | ZFSBootMenu                    | GRUB + ZFS                   |
|---------------------+--------------------------------+------------------------------|
| Boot environment    | Native (designed for ZFS)      | Requires ZFS module          |
| Snapshot booting    | Built-in, interactive          | Custom GRUB menu entries     |
| Encryption          | Prompts for key automatically  | More complex setup           |
| EFI space needed    | Minimal (~512MB)               | Larger (kernel/initramfs)    |
| Complexity          | Simpler (single binary)        | More moving parts            |
| Recovery            | Can browse/rollback at boot    | Requires grub.cfg regen      |

* Pacman Hooks and Systemd Services

** zarch Services
#+begin_example
zfs-import-cache
zfs-import.target
zfs-mount
zfs-zed
zfs.target
set-locale-once.service  (custom first-boot locale config)
#+end_example

** archzfs Services
#+begin_example
zfs.target
zfs-import-scan.service  (instead of cache-based)
zfs-mount.service
zfs-import.target
NetworkManager
avahi-daemon
sshd
#+end_example

** Key Difference: Import Method
- zarch: Uses zfs-import-cache (requires cachefile)
- archzfs: Uses zfs-import-scan (scans with blkid, no cachefile needed)

The scan method is simpler and more portable (works if moving disks between
systems).

* mkinitcpio Configuration

** zarch Approach
#+begin_src bash
sed -i '/^HOOKS=/s/block filesystems/block zfs filesystems/g' /etc/mkinitcpio.conf
#+end_src

** archzfs Approach
#+begin_src bash
HOOKS=(base udev microcode modconf kms keyboard keymap consolefont block zfs filesystems)
#+end_src

** Important Notes
- Both use busybox-based udev (not systemd hook)
- archzfs explicitly removes autodetect to ensure all storage drivers included
- archzfs removes fsck (ZFS doesn't use it)
- archzfs includes microcode early loading

* Useful Patterns to Consider

** 1. Profile-Based Configuration
zarch uses a profile directory system:
#+begin_example
default/
  archpkg.txt      # Official packages
  aurpkg.txt       # AUR packages
  services.txt     # Services to enable
  zarch.conf       # Core configuration
  custom-chroot.sh # Custom post-install
#+end_example

This allows maintaining multiple configurations (desktop, server, VM) cleanly.

** 2. ZFSBootMenu for Simpler Boot
For future consideration:
- Native ZFS boot environment support
- Interactive snapshot selection at boot
- Simpler encryption key handling
- Smaller EFI partition needs

** 3. zrepl for Time-Based Snapshots
For systems needing frequent snapshots beyond pacman transactions:
- 15-minute intervals for development machines
- Complex retention policies
- Replication to remote systems

** 4. AUR Helper Installation Pattern
#+begin_src bash
# Build yay as regular user, install as root
su -c "git clone https://aur.archlinux.org/yay-bin.git" "$USER_NAME"
arch-chroot -u "$USER_NAME" /mnt makepkg -D /home/$USER_NAME/yay-bin -s
pacman -U --noconfirm yay-bin-*.pkg.tar.*
#+end_src

* References

- [[https://github.com/sandreas/zarch][sandreas/zarch GitHub Repository]]
- [[https://zfsbootmenu.org/][ZFSBootMenu Official Site]]
- [[https://docs.zfsbootmenu.org/en/latest/][ZFSBootMenu Documentation]]
- [[https://zrepl.github.io/][zrepl Documentation]]
- [[https://wiki.archlinux.org/title/ZFS][Arch Wiki: ZFS]]
- [[https://github.com/acrion/zfs-autosnap][zfs-autosnap - Pre-upgrade Snapshots]]
- [[https://aur.archlinux.org/packages/pacman-zfs-hook][pacman-zfs-hook AUR Package]]
- [[https://florianesser.ch/posts/20220714-arch-install-zbm/][Guide: Install Arch Linux on encrypted zpool with ZFSBootMenu]]

* Action Items for archzfs

Based on this research, potential improvements:

1. [ ] Consider adding ZFSBootMenu as alternative bootloader option
2. [ ] Evaluate zrepl for systems needing frequent time-based snapshots
3. [ ] Document the grub-zfs-snap vs ZFSBootMenu trade-offs
4. [ ] Consider profile-based configuration for different use cases
5. [ ] Add sanoid configuration to archsetup for automated snapshot retention