path: root/docs/research-btrfs-expansion.org

#+TITLE: Research: Expanding Project to Support Btrfs
#+DATE: 2026-01-23
#+AUTHOR: Craig Jennings & Claude

* Executive Summary

This document covers research and architecture planning for expanding the archzfs
project to support both ZFS and Btrfs filesystems, providing equivalent functionality
for both.

* Project Renaming

Since "archzfs" implies ZFS-only, the project needs a new name.

** Naming Options

| Name | Pros | Cons |
|------+------+------|
| archsnap | Short, implies snapshots | Doesn't mention filesystems |
| archfs | Generic filesystem installer | Might be too generic |
| archroot | Implies root filesystem setup | Already used by other tools |
| arch-snap-install | Descriptive | Long |
| snaparch | Short, memorable | Sounds like "snap" package manager |
| archraid | Implies multi-disk | Doesn't mention snapshots |
| arch-resilient | Implies reliability | Long, vague |

** Recommendation
*archsnap* - Short, memorable, emphasizes the snapshot capability that's the core
value proposition of both ZFS and Btrfs configurations.

* Btrfs Equivalent Features

** Feature Parity Matrix

| Feature (ZFS) | Btrfs Equivalent | Tool/Method |
|---------------+------------------+-------------|
| ZFS native encryption | LUKS2 + dm-crypt | cryptsetup |
| zpool mirror | btrfs raid1 | mkfs.btrfs -d raid1 -m raid1 |
| zpool stripe | btrfs raid0 | mkfs.btrfs -d raid0 |
| zpool raidz1 | btrfs raid5 | NOT RECOMMENDED (unstable) |
| zpool raidz2 | btrfs raid6 | NOT RECOMMENDED (unstable) |
| ZFS datasets | Btrfs subvolumes | btrfs subvolume create |
| zfs snapshot | btrfs snapshot | btrfs subvolume snapshot |
| ZFSBootMenu | GRUB + grub-btrfs | grub-btrfs package |
| sanoid/syncoid | btrbk or snapper | snapper + snap-pac |
| zfs rollback | snapper rollback | snapper undochange |
| Pre-pacman snapshots | snap-pac | snap-pac + snapper |

** Important Btrfs Limitations

*** RAID 5/6 is Unstable
#+BEGIN_QUOTE
"Parity RAID (RAID 5/6) code has multiple serious data-loss bugs in it."
-- Btrfs Wiki
#+END_QUOTE

For multi-disk Btrfs, only offer:
- *raid1* (mirror) - Stable, recommended
- *raid10* (striped mirrors) - Requires 4+ disks
- *raid0* (stripe) - No redundancy, not recommended for root

*** No Native Encryption
Btrfs doesn't have native encryption. Use LUKS2 as layer beneath:
#+BEGIN_SRC bash
cryptsetup luksFormat /dev/sdX
cryptsetup open /dev/sdX cryptroot
mkfs.btrfs /dev/mapper/cryptroot
#+END_SRC

*** Bootloader Considerations
- *GRUB* is required for booting into snapshots (via grub-btrfs)
- *systemd-boot* can't boot from btrfs snapshots (kernel on FAT32 ESP)
- ZFSBootMenu equivalent: GRUB + grub-btrfs + grub-btrfsd daemon

* Recommended Btrfs Subvolume Layout

Based on research from multiple sources, this layout provides optimal snapshot
management and rollback capability:

#+BEGIN_SRC
btrfs volume (LABEL=archsnap)
│
├── @ (subvol) → mounted at /
├── @home (subvol) → mounted at /home
├── @snapshots (subvol) → mounted at /.snapshots
├── @var_log (subvol) → mounted at /var/log
├── @var_cache (subvol) → mounted at /var/cache
└── @var_tmp (subvol) → mounted at /var/tmp
#+END_SRC

** Why This Layout?

1. *@ for root* - Main system, snapshotted for rollbacks
2. *@home separate* - User data not affected by system rollbacks
3. *@snapshots separate* - Snapper requirement for snapshot storage
4. *@var_log separate* - Logs persist across rollbacks, required for read-only snapshots
5. *@var_cache separate* - Package cache excluded from snapshots (saves space)
6. *@var_tmp separate* - Temp files excluded from snapshots

** Mount Options

#+BEGIN_SRC bash
# Recommended mount options
BTRFS_OPTS="noatime,compress=zstd,space_cache=v2,discard=async"

# Example fstab entries (NO subvolid - allows rollback)
UUID=xxx  /           btrfs  $BTRFS_OPTS,subvol=@           0 0
UUID=xxx  /home       btrfs  $BTRFS_OPTS,subvol=@home       0 0
UUID=xxx  /.snapshots btrfs  $BTRFS_OPTS,subvol=@snapshots  0 0
UUID=xxx  /var/log    btrfs  $BTRFS_OPTS,subvol=@var_log    0 0
UUID=xxx  /var/cache  btrfs  $BTRFS_OPTS,subvol=@var_cache  0 0
UUID=xxx  /var/tmp    btrfs  $BTRFS_OPTS,subvol=@var_tmp    0 0
#+END_SRC

*CRITICAL*: Do NOT use subvolid= in fstab - it breaks rollbacks!

* Snapshot Management Stack

** Recommended: Snapper + snap-pac + grub-btrfs + btrfs-assistant

| Component | Purpose | Package |
|-----------+---------+---------|
| snapper | Snapshot creation, retention, rollback | snapper |
| snap-pac | Automatic pre/post pacman snapshots | snap-pac |
| grub-btrfs | Bootable snapshot menu in GRUB | grub-btrfs |
| grub-btrfsd | Auto-update GRUB on snapshot changes | grub-btrfs (service) |
| btrfs-assistant | GUI for snapshot management | btrfs-assistant (AUR) |
| snapper-gui | Alternative lightweight GUI | snapper-gui (AUR) |

** Alternative: btrbk

btrbk is simpler than snapper and handles both snapshots and remote backups:
#+BEGIN_SRC conf
# /etc/btrbk.conf
snapshot_preserve_min   2d
snapshot_preserve       24h 7d 4w 12m

volume /
  snapshot_dir /.snapshots
  subvolume @
#+END_SRC

** Retention Policy (Snapper)

Default snapper timeline for root:
#+BEGIN_SRC conf
# /etc/snapper/configs/root
TIMELINE_CREATE="yes"
TIMELINE_CLEANUP="yes"
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="5"
TIMELINE_LIMIT_DAILY="7"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="0"
TIMELINE_LIMIT_YEARLY="0"
#+END_SRC

Keeps: 5 hourly + 7 daily = ~12 snapshots at any time.

* Multi-Disk Btrfs Setup

** Mirror (2+ disks) - RECOMMENDED

#+BEGIN_SRC bash
# Create mirrored filesystem
mkfs.btrfs -L archsnap -d raid1 -m raid1 /dev/sda2 /dev/sdb2

# Mount using ANY device (btrfs auto-discovers others)
mount /dev/sda2 /mnt

# Create subvolumes
btrfs subvolume create /mnt/@
btrfs subvolume create /mnt/@home
# ... etc
#+END_SRC

** Stripe (2+ disks) - NOT RECOMMENDED for root

#+BEGIN_SRC bash
# Striped data, mirrored metadata (default)
mkfs.btrfs -L archsnap -d raid0 -m raid1 /dev/sda2 /dev/sdb2
#+END_SRC

Metadata is mirrored by default even in raid0 mode for safety.

** Converting Single to Mirror

#+BEGIN_SRC bash
# Add second device to existing filesystem
mount /dev/sda2 /mnt
btrfs device add /dev/sdb2 /mnt
btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt
#+END_SRC

* Reference Repositories Cloned

Located in =reference-repos/=:

| Repository | Purpose |
|------------+---------|
| grub-btrfs | GRUB snapshot menu integration |
| easy-arch | Automated Arch+Btrfs installer |
| arch-btrfs-installation | Manual install guide |
| btrfs-assistant | GUI snapshot manager |
| snap-pac | Pacman hook for snapper |
| btrbk | Alternative snapshot/backup tool |
| buttermanager | GUI btrfs management |
| alis | Arch Linux Install Script (multi-fs) |

* Installer Architecture Changes

** Current Structure (ZFS-only)

#+BEGIN_SRC
install-archzfs
├── Phase 1: Configuration gathering (fzf prompts)
├── Phase 2: Disk partitioning (EFI + ZFS)
├── Phase 3: ZFS pool/dataset creation
├── Phase 4: Pacstrap base system
├── Phase 5: System configuration
├── Phase 6: ZFSBootMenu installation
├── Phase 7: Genesis snapshot
└── Phase 8: Cleanup
#+END_SRC

** Proposed Structure (Multi-filesystem)

#+BEGIN_SRC
install-archsnap
├── Phase 1: Configuration gathering
│   ├── Hostname, timezone, locale, keymap
│   ├── Disk selection (multi-select)
│   ├── RAID level (if multi-disk)
│   ├── NEW: Filesystem selection (ZFS / Btrfs)
│   ├── Encryption passphrase
│   ├── Root password
│   ├── NEW: Initial user password
│   └── SSH configuration
│
├── Phase 2: Disk partitioning
│   ├── EFI partition (same for both)
│   └── Root partition (type depends on FS choice)
│
├── Phase 3: Filesystem creation
│   ├── IF ZFS: create pool + datasets
│   └── IF Btrfs: create volume + subvolumes
│
├── Phase 4: Pacstrap base system
│   ├── Common packages
│   ├── IF ZFS: zfs-dkms, zfs-utils
│   └── IF Btrfs: btrfs-progs, snapper, snap-pac, grub-btrfs
│
├── Phase 5: System configuration
│   ├── Common config (locale, timezone, hostname)
│   ├── IF ZFS: ZFS-specific configs
│   └── IF Btrfs: snapper configs, grub-btrfs configs
│
├── Phase 6: Bootloader installation
│   ├── IF ZFS: ZFSBootMenu
│   └── IF Btrfs: GRUB + grub-btrfs
│
├── Phase 7: Genesis snapshot
│   ├── IF ZFS: zfs snapshot -r zroot@genesis
│   └── IF Btrfs: snapper create -c root -d "genesis"
│
└── Phase 8: Cleanup
#+END_SRC

** Code Organization

#+BEGIN_SRC bash
install-archsnap
├── lib/
│   ├── common.sh      # Shared functions (colors, prompts, fzf)
│   ├── disk.sh        # Partitioning (shared)
│   ├── zfs.sh         # ZFS-specific functions
│   ├── btrfs.sh       # Btrfs-specific functions
│   └── config.sh      # Config file handling
├── configs/
│   ├── snapper-root.conf
│   └── btrbk.conf.example
└── install-archsnap   # Main script
#+END_SRC

* Testing Strategy

** VM Test Matrix

- Single ZFS (1 disk): Install, boot, reboot, rollback+reboot
- Single Btrfs (1 disk): Install, boot, reboot, rollback+reboot
- Mirror ZFS (2 disk): Install, fail disk, recover
- Mirror Btrfs (2 disk, raid1): Install, fail disk, recover
- RAIDZ1 ZFS (3 disk): Install, fail disk, recover

** Validation Checks

*** Fresh Install (all configs)
- Partitions created correctly (EFI + root)
- Filesystem created (pool/subvols)
- All mount points accessible
- Packages installed (pacman -Q zfs-dkms or btrfs-progs)
- Services enabled (zfs.target or snapper-timeline.timer)
- Bootloader installed (ZFSBootMenu or GRUB + grub-btrfs)
- fstab correct (no subvolid for btrfs)
- Can boot without ISO

*** Reboot Survival
Critical: Verify system survives reboot cleanly. Catches:
- Misconfigured services slamming CPU/network
- Services that fail to start properly
- ZFS import issues
- fstab mount failures

Checks after reboot:
- System reaches login prompt
- All filesystems mounted (findmnt)
- No failed services (systemctl --failed)
- CPU/memory/network normal (no runaway processes)
- Can SSH in (if network configured)

*** Snapshot Operations
- Manual snapshot creates successfully
- Snapshot appears in list (zfs list -t snap / snapper list)
- Pre-pacman snapshot created automatically (install package, verify)
- Snapshot visible in boot menu (ZFSBootMenu / GRUB)
- Can boot into snapshot

*** Rollback + Reboot
Important: Rollback MUST include reboot to validate fully.
Do NOT use ZFSBootMenu's built-in rollback (known bug, needs filing).
Use zfsrollback script or snapper rollback instead.

Checks:
- Rollback command completes without error
- Reboot completes successfully
- System state matches snapshot after reboot
- Previously installed test package is gone
- No orphaned/broken state

*** Failure Recovery (multi-disk only)
- Pool/volume shows correct redundancy (zpool status / btrfs fi show)
- Survives single disk removal (zpool offline / btrfs device delete)
- Boots with degraded array
- Warnings displayed about degraded state
- Can resilver/rebuild after disk replaced (zpool replace / btrfs replace)
- Pool returns to healthy state

*** Encryption
- Passphrase prompt appears at boot
- Correct unlock with right passphrase
- Fails gracefully with wrong passphrase (retry prompt, not panic)
- Data unreadable when mounted elsewhere without key
- Pool/volume not auto-imported without passphrase

** Known Issues

*** ZFSBootMenu Rollback Bug
ZFSBootMenu's built-in rollback feature has issues (to be filed).
Workaround: Use zfsrollback script from installed system or live ISO.

** Test Scripts

#+BEGIN_SRC
scripts/
├── test-zfs-single.sh
├── test-zfs-mirror.sh
├── test-zfs-raidz.sh
├── test-btrfs-single.sh
├── test-btrfs-mirror.sh
└── test-configs/
    ├── zfs-single.conf
    ├── zfs-mirror.conf
    ├── btrfs-single.conf
    └── btrfs-mirror.conf
#+END_SRC

* CLI and GUI Tools

** CLI Tools to Install (Both Filesystems)

| Tool | Purpose |
|------+---------|
| archsnap-snapshot | Create manual snapshot (wrapper) |
| archsnap-rollback | Interactive rollback (fzf-based) |
| archsnap-list | List snapshots |
| archsnap-prune | Manual pruning |

These would be thin wrappers that detect the filesystem and call the appropriate
underlying tool (zfs/snapper/btrbk).

** GUI Options

*** For Btrfs
- *btrfs-assistant* (recommended) - Full-featured, actively maintained
- *snapper-gui* - Lighter weight alternative

*** For ZFS
- No equivalent mature GUI exists
- Could build a simple zenity/yad wrapper

* Implementation Phases

** Phase 1: Refactor Current Installer
1. Extract common functions to lib/common.sh
2. Extract ZFS-specific code to lib/zfs.sh
3. Add filesystem selection prompt
4. Rename project to archsnap

** Phase 2: Implement Btrfs Support
1. Create lib/btrfs.sh with btrfs-specific functions
2. Implement subvolume creation
3. Implement snapper configuration
4. Implement GRUB + grub-btrfs installation
5. Create btrfs genesis snapshot

** Phase 3: Testing Infrastructure
1. Create test configs for all scenarios
2. Update test-vm.sh for btrfs testing
3. Create automated test suite
4. Test multi-disk configurations

** Phase 4: CLI Tools
1. Create archsnap-snapshot wrapper
2. Create archsnap-rollback wrapper
3. Update documentation

** Phase 5: Documentation & Polish
1. Update README for dual-filesystem support
2. Create BTRFS-specific documentation
3. Update troubleshooting guides

* Sources

** Btrfs Installation
- [[https://wiki.archlinux.org/title/Btrfs][Arch Wiki - Btrfs]]
- [[https://github.com/egara/arch-btrfs-installation][arch-btrfs-installation]]
- [[https://gist.github.com/mjkstra/96ce7a5689d753e7a6bdd92cdc169bae][Modern Arch+Btrfs Guide]]

** Snapshot Management
- [[https://github.com/Antynea/grub-btrfs][grub-btrfs]]
- [[https://wiki.archlinux.org/title/Snapper][Arch Wiki - Snapper]]
- [[https://github.com/wesbarnett/snap-pac][snap-pac]]
- [[https://github.com/digint/btrbk][btrbk]]

** GUI Tools
- [[https://gitlab.com/btrfs-assistant/btrfs-assistant][btrfs-assistant]]
- [[https://github.com/egara/buttermanager][buttermanager]]

** Multi-Disk
- [[https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices.html][Btrfs Wiki - Multiple Devices]]
- [[https://thelinuxcode.com/set-up-btrfs-raid/][How to Set Up Btrfs RAID]]