diff options
| author | Craig Jennings <c@cjennings.net> | 2026-01-23 13:08:38 -0600 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-01-23 13:08:38 -0600 |
| commit | 0ff02dda8a2bfb21bb223e42054066626e429ba1 (patch) | |
| tree | ed5bb5748264912f1a0003da2dcf6ae9555ae85b /docs/research-btrfs-expansion.org | |
| parent | 3b5363234599a72d4dd04847c1124d6381ca21c7 (diff) | |
| download | archangel-0ff02dda8a2bfb21bb223e42054066626e429ba1.tar.gz archangel-0ff02dda8a2bfb21bb223e42054066626e429ba1.zip | |
Research: Btrfs expansion and multi-filesystem support
Comprehensive research for expanding project to support both ZFS and Btrfs:
- Feature parity analysis (ZFS vs Btrfs equivalents)
- Btrfs limitations: RAID 5/6 unstable, no native encryption (use LUKS)
- Recommended subvolume layout: @, @home, @snapshots, @var_log, @var_cache
- Snapshot stack: snapper + snap-pac + grub-btrfs + btrfs-assistant
- Multi-disk support: raid1 (mirror), raid10 only
- Installer architecture proposal with modular lib/ structure
- Testing strategy and VM test matrix
- Project rename recommendation: archsnap
Cloned 8 reference repos: grub-btrfs, easy-arch, arch-btrfs-installation,
btrfs-assistant, snap-pac, btrbk, buttermanager, alis
Diffstat (limited to 'docs/research-btrfs-expansion.org')
| -rw-r--r-- | docs/research-btrfs-expansion.org | 408 |
1 files changed, 408 insertions, 0 deletions
diff --git a/docs/research-btrfs-expansion.org b/docs/research-btrfs-expansion.org new file mode 100644 index 0000000..300f5cd --- /dev/null +++ b/docs/research-btrfs-expansion.org @@ -0,0 +1,408 @@ +#+TITLE: Research: Expanding Project to Support Btrfs +#+DATE: 2026-01-23 +#+AUTHOR: Craig Jennings & Claude + +* Executive Summary + +This document covers research and architecture planning for expanding the archzfs +project to support both ZFS and Btrfs filesystems, providing equivalent functionality +for both. + +* Project Renaming + +Since "archzfs" implies ZFS-only, the project needs a new name. + +** Naming Options + +| Name | Pros | Cons | +|------+------+------| +| archsnap | Short, implies snapshots | Doesn't mention filesystems | +| archfs | Generic filesystem installer | Might be too generic | +| archroot | Implies root filesystem setup | Already used by other tools | +| arch-snap-install | Descriptive | Long | +| snaparch | Short, memorable | Sounds like "snap" package manager | +| archraid | Implies multi-disk | Doesn't mention snapshots | +| arch-resilient | Implies reliability | Long, vague | + +** Recommendation +*archsnap* - Short, memorable, emphasizes the snapshot capability that's the core +value proposition of both ZFS and Btrfs configurations. + +* Btrfs Equivalent Features + +** Feature Parity Matrix + +| Feature (ZFS) | Btrfs Equivalent | Tool/Method | +|---------------+------------------+-------------| +| ZFS native encryption | LUKS2 + dm-crypt | cryptsetup | +| zpool mirror | btrfs raid1 | mkfs.btrfs -d raid1 -m raid1 | +| zpool stripe | btrfs raid0 | mkfs.btrfs -d raid0 | +| zpool raidz1 | btrfs raid5 | NOT RECOMMENDED (unstable) | +| zpool raidz2 | btrfs raid6 | NOT RECOMMENDED (unstable) | +| ZFS datasets | Btrfs subvolumes | btrfs subvolume create | +| zfs snapshot | btrfs snapshot | btrfs subvolume snapshot | +| ZFSBootMenu | GRUB + grub-btrfs | grub-btrfs package | +| sanoid/syncoid | btrbk or snapper | snapper + snap-pac | +| zfs rollback | snapper rollback | snapper undochange | +| Pre-pacman snapshots | snap-pac | snap-pac + snapper | + +** Important Btrfs Limitations + +*** RAID 5/6 is Unstable +#+BEGIN_QUOTE +"Parity RAID (RAID 5/6) code has multiple serious data-loss bugs in it." +-- Btrfs Wiki +#+END_QUOTE + +For multi-disk Btrfs, only offer: +- *raid1* (mirror) - Stable, recommended +- *raid10* (striped mirrors) - Requires 4+ disks +- *raid0* (stripe) - No redundancy, not recommended for root + +*** No Native Encryption +Btrfs doesn't have native encryption. Use LUKS2 as layer beneath: +#+BEGIN_SRC bash +cryptsetup luksFormat /dev/sdX +cryptsetup open /dev/sdX cryptroot +mkfs.btrfs /dev/mapper/cryptroot +#+END_SRC + +*** Bootloader Considerations +- *GRUB* is required for booting into snapshots (via grub-btrfs) +- *systemd-boot* can't boot from btrfs snapshots (kernel on FAT32 ESP) +- ZFSBootMenu equivalent: GRUB + grub-btrfs + grub-btrfsd daemon + +* Recommended Btrfs Subvolume Layout + +Based on research from multiple sources, this layout provides optimal snapshot +management and rollback capability: + +#+BEGIN_SRC +btrfs volume (LABEL=archsnap) +│ +├── @ (subvol) → mounted at / +├── @home (subvol) → mounted at /home +├── @snapshots (subvol) → mounted at /.snapshots +├── @var_log (subvol) → mounted at /var/log +├── @var_cache (subvol) → mounted at /var/cache +└── @var_tmp (subvol) → mounted at /var/tmp +#+END_SRC + +** Why This Layout? + +1. *@ for root* - Main system, snapshotted for rollbacks +2. *@home separate* - User data not affected by system rollbacks +3. *@snapshots separate* - Snapper requirement for snapshot storage +4. *@var_log separate* - Logs persist across rollbacks, required for read-only snapshots +5. *@var_cache separate* - Package cache excluded from snapshots (saves space) +6. *@var_tmp separate* - Temp files excluded from snapshots + +** Mount Options + +#+BEGIN_SRC bash +# Recommended mount options +BTRFS_OPTS="noatime,compress=zstd,space_cache=v2,discard=async" + +# Example fstab entries (NO subvolid - allows rollback) +UUID=xxx / btrfs $BTRFS_OPTS,subvol=@ 0 0 +UUID=xxx /home btrfs $BTRFS_OPTS,subvol=@home 0 0 +UUID=xxx /.snapshots btrfs $BTRFS_OPTS,subvol=@snapshots 0 0 +UUID=xxx /var/log btrfs $BTRFS_OPTS,subvol=@var_log 0 0 +UUID=xxx /var/cache btrfs $BTRFS_OPTS,subvol=@var_cache 0 0 +UUID=xxx /var/tmp btrfs $BTRFS_OPTS,subvol=@var_tmp 0 0 +#+END_SRC + +*CRITICAL*: Do NOT use subvolid= in fstab - it breaks rollbacks! + +* Snapshot Management Stack + +** Recommended: Snapper + snap-pac + grub-btrfs + btrfs-assistant + +| Component | Purpose | Package | +|-----------+---------+---------| +| snapper | Snapshot creation, retention, rollback | snapper | +| snap-pac | Automatic pre/post pacman snapshots | snap-pac | +| grub-btrfs | Bootable snapshot menu in GRUB | grub-btrfs | +| grub-btrfsd | Auto-update GRUB on snapshot changes | grub-btrfs (service) | +| btrfs-assistant | GUI for snapshot management | btrfs-assistant (AUR) | +| snapper-gui | Alternative lightweight GUI | snapper-gui (AUR) | + +** Alternative: btrbk + +btrbk is simpler than snapper and handles both snapshots and remote backups: +#+BEGIN_SRC conf +# /etc/btrbk.conf +snapshot_preserve_min 2d +snapshot_preserve 24h 7d 4w 12m + +volume / + snapshot_dir /.snapshots + subvolume @ +#+END_SRC + +** Retention Policy (Snapper) + +Default snapper timeline for root: +#+BEGIN_SRC conf +# /etc/snapper/configs/root +TIMELINE_CREATE="yes" +TIMELINE_CLEANUP="yes" +TIMELINE_MIN_AGE="1800" +TIMELINE_LIMIT_HOURLY="5" +TIMELINE_LIMIT_DAILY="7" +TIMELINE_LIMIT_WEEKLY="0" +TIMELINE_LIMIT_MONTHLY="0" +TIMELINE_LIMIT_YEARLY="0" +#+END_SRC + +Keeps: 5 hourly + 7 daily = ~12 snapshots at any time. + +* Multi-Disk Btrfs Setup + +** Mirror (2+ disks) - RECOMMENDED + +#+BEGIN_SRC bash +# Create mirrored filesystem +mkfs.btrfs -L archsnap -d raid1 -m raid1 /dev/sda2 /dev/sdb2 + +# Mount using ANY device (btrfs auto-discovers others) +mount /dev/sda2 /mnt + +# Create subvolumes +btrfs subvolume create /mnt/@ +btrfs subvolume create /mnt/@home +# ... etc +#+END_SRC + +** Stripe (2+ disks) - NOT RECOMMENDED for root + +#+BEGIN_SRC bash +# Striped data, mirrored metadata (default) +mkfs.btrfs -L archsnap -d raid0 -m raid1 /dev/sda2 /dev/sdb2 +#+END_SRC + +Metadata is mirrored by default even in raid0 mode for safety. + +** Converting Single to Mirror + +#+BEGIN_SRC bash +# Add second device to existing filesystem +mount /dev/sda2 /mnt +btrfs device add /dev/sdb2 /mnt +btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt +#+END_SRC + +* Reference Repositories Cloned + +Located in =reference-repos/=: + +| Repository | Purpose | +|------------+---------| +| grub-btrfs | GRUB snapshot menu integration | +| easy-arch | Automated Arch+Btrfs installer | +| arch-btrfs-installation | Manual install guide | +| btrfs-assistant | GUI snapshot manager | +| snap-pac | Pacman hook for snapper | +| btrbk | Alternative snapshot/backup tool | +| buttermanager | GUI btrfs management | +| alis | Arch Linux Install Script (multi-fs) | + +* Installer Architecture Changes + +** Current Structure (ZFS-only) + +#+BEGIN_SRC +install-archzfs +├── Phase 1: Configuration gathering (fzf prompts) +├── Phase 2: Disk partitioning (EFI + ZFS) +├── Phase 3: ZFS pool/dataset creation +├── Phase 4: Pacstrap base system +├── Phase 5: System configuration +├── Phase 6: ZFSBootMenu installation +├── Phase 7: Genesis snapshot +└── Phase 8: Cleanup +#+END_SRC + +** Proposed Structure (Multi-filesystem) + +#+BEGIN_SRC +install-archsnap +├── Phase 1: Configuration gathering +│ ├── Hostname, timezone, locale, keymap +│ ├── Disk selection (multi-select) +│ ├── RAID level (if multi-disk) +│ ├── NEW: Filesystem selection (ZFS / Btrfs) +│ ├── Encryption passphrase +│ ├── Root password +│ ├── NEW: Initial user password +│ └── SSH configuration +│ +├── Phase 2: Disk partitioning +│ ├── EFI partition (same for both) +│ └── Root partition (type depends on FS choice) +│ +├── Phase 3: Filesystem creation +│ ├── IF ZFS: create pool + datasets +│ └── IF Btrfs: create volume + subvolumes +│ +├── Phase 4: Pacstrap base system +│ ├── Common packages +│ ├── IF ZFS: zfs-dkms, zfs-utils +│ └── IF Btrfs: btrfs-progs, snapper, snap-pac, grub-btrfs +│ +├── Phase 5: System configuration +│ ├── Common config (locale, timezone, hostname) +│ ├── IF ZFS: ZFS-specific configs +│ └── IF Btrfs: snapper configs, grub-btrfs configs +│ +├── Phase 6: Bootloader installation +│ ├── IF ZFS: ZFSBootMenu +│ └── IF Btrfs: GRUB + grub-btrfs +│ +├── Phase 7: Genesis snapshot +│ ├── IF ZFS: zfs snapshot -r zroot@genesis +│ └── IF Btrfs: snapper create -c root -d "genesis" +│ +└── Phase 8: Cleanup +#+END_SRC + +** Code Organization + +#+BEGIN_SRC bash +install-archsnap +├── lib/ +│ ├── common.sh # Shared functions (colors, prompts, fzf) +│ ├── disk.sh # Partitioning (shared) +│ ├── zfs.sh # ZFS-specific functions +│ ├── btrfs.sh # Btrfs-specific functions +│ └── config.sh # Config file handling +├── configs/ +│ ├── snapper-root.conf +│ └── btrbk.conf.example +└── install-archsnap # Main script +#+END_SRC + +* Testing Strategy + +** VM Test Matrix + +| Config | Disks | Filesystem | RAID | Test | +|--------+-------+------------+------+------| +| Single ZFS | 1 | ZFS | - | Install, boot, rollback | +| Single Btrfs | 1 | Btrfs | - | Install, boot, rollback | +| Mirror ZFS | 2 | ZFS | mirror | Install, fail disk, recover | +| Mirror Btrfs | 2 | Btrfs | raid1 | Install, fail disk, recover | +| RAIDZ1 ZFS | 3 | ZFS | raidz1 | Install, fail disk, recover | + +** Test Scenarios + +1. *Fresh Install* + - Single disk, multi-disk + - Interactive and config-file modes + +2. *Snapshot Operations* + - Manual snapshot creation + - Automatic pre-pacman snapshots + - Boot into snapshot + - Rollback to snapshot + +3. *Failure Recovery* + - Single disk failure (mirror/raidz) + - Corrupt boot + - Kernel update breaks system + +4. *Encryption* + - ZFS native encryption + - LUKS + Btrfs encryption + +** Test Scripts + +#+BEGIN_SRC +scripts/ +├── test-zfs-single.sh +├── test-zfs-mirror.sh +├── test-zfs-raidz.sh +├── test-btrfs-single.sh +├── test-btrfs-mirror.sh +└── test-configs/ + ├── zfs-single.conf + ├── zfs-mirror.conf + ├── btrfs-single.conf + └── btrfs-mirror.conf +#+END_SRC + +* CLI and GUI Tools + +** CLI Tools to Install (Both Filesystems) + +| Tool | Purpose | +|------+---------| +| archsnap-snapshot | Create manual snapshot (wrapper) | +| archsnap-rollback | Interactive rollback (fzf-based) | +| archsnap-list | List snapshots | +| archsnap-prune | Manual pruning | + +These would be thin wrappers that detect the filesystem and call the appropriate +underlying tool (zfs/snapper/btrbk). + +** GUI Options + +*** For Btrfs +- *btrfs-assistant* (recommended) - Full-featured, actively maintained +- *snapper-gui* - Lighter weight alternative + +*** For ZFS +- No equivalent mature GUI exists +- Could build a simple zenity/yad wrapper + +* Implementation Phases + +** Phase 1: Refactor Current Installer +1. Extract common functions to lib/common.sh +2. Extract ZFS-specific code to lib/zfs.sh +3. Add filesystem selection prompt +4. Rename project to archsnap + +** Phase 2: Implement Btrfs Support +1. Create lib/btrfs.sh with btrfs-specific functions +2. Implement subvolume creation +3. Implement snapper configuration +4. Implement GRUB + grub-btrfs installation +5. Create btrfs genesis snapshot + +** Phase 3: Testing Infrastructure +1. Create test configs for all scenarios +2. Update test-vm.sh for btrfs testing +3. Create automated test suite +4. Test multi-disk configurations + +** Phase 4: CLI Tools +1. Create archsnap-snapshot wrapper +2. Create archsnap-rollback wrapper +3. Update documentation + +** Phase 5: Documentation & Polish +1. Update README for dual-filesystem support +2. Create BTRFS-specific documentation +3. Update troubleshooting guides + +* Sources + +** Btrfs Installation +- [[https://wiki.archlinux.org/title/Btrfs][Arch Wiki - Btrfs]] +- [[https://github.com/egara/arch-btrfs-installation][arch-btrfs-installation]] +- [[https://gist.github.com/mjkstra/96ce7a5689d753e7a6bdd92cdc169bae][Modern Arch+Btrfs Guide]] + +** Snapshot Management +- [[https://github.com/Antynea/grub-btrfs][grub-btrfs]] +- [[https://wiki.archlinux.org/title/Snapper][Arch Wiki - Snapper]] +- [[https://github.com/wesbarnett/snap-pac][snap-pac]] +- [[https://github.com/digint/btrbk][btrbk]] + +** GUI Tools +- [[https://gitlab.com/btrfs-assistant/btrfs-assistant][btrfs-assistant]] +- [[https://github.com/egara/buttermanager][buttermanager]] + +** Multi-Disk +- [[https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices.html][Btrfs Wiki - Multiple Devices]] +- [[https://thelinuxcode.com/set-up-btrfs-raid/][How to Set Up Btrfs RAID]] |
