diff options
Diffstat (limited to 'docs/research-btrfs-expansion.org')
| -rw-r--r-- | docs/research-btrfs-expansion.org | 451 |
1 files changed, 0 insertions, 451 deletions
diff --git a/docs/research-btrfs-expansion.org b/docs/research-btrfs-expansion.org deleted file mode 100644 index 8a6c1a6..0000000 --- a/docs/research-btrfs-expansion.org +++ /dev/null @@ -1,451 +0,0 @@ -#+TITLE: Research: Expanding Project to Support Btrfs -#+DATE: 2026-01-23 -#+AUTHOR: Craig Jennings & Claude - -* Executive Summary - -This document covers research and architecture planning for expanding the archzfs -project to support both ZFS and Btrfs filesystems, providing equivalent functionality -for both. - -* Project Renaming - -Since "archzfs" implies ZFS-only, the project needs a new name. - -** Naming Options - -| Name | Pros | Cons | -|------+------+------| -| archsnap | Short, implies snapshots | Doesn't mention filesystems | -| archfs | Generic filesystem installer | Might be too generic | -| archroot | Implies root filesystem setup | Already used by other tools | -| arch-snap-install | Descriptive | Long | -| snaparch | Short, memorable | Sounds like "snap" package manager | -| archraid | Implies multi-disk | Doesn't mention snapshots | -| arch-resilient | Implies reliability | Long, vague | - -** Recommendation -*archsnap* - Short, memorable, emphasizes the snapshot capability that's the core -value proposition of both ZFS and Btrfs configurations. - -* Btrfs Equivalent Features - -** Feature Parity Matrix - -| Feature (ZFS) | Btrfs Equivalent | Tool/Method | -|---------------+------------------+-------------| -| ZFS native encryption | LUKS2 + dm-crypt | cryptsetup | -| zpool mirror | btrfs raid1 | mkfs.btrfs -d raid1 -m raid1 | -| zpool stripe | btrfs raid0 | mkfs.btrfs -d raid0 | -| zpool raidz1 | btrfs raid5 | NOT RECOMMENDED (unstable) | -| zpool raidz2 | btrfs raid6 | NOT RECOMMENDED (unstable) | -| ZFS datasets | Btrfs subvolumes | btrfs subvolume create | -| zfs snapshot | btrfs snapshot | btrfs subvolume snapshot | -| ZFSBootMenu | GRUB + grub-btrfs | grub-btrfs package | -| sanoid/syncoid | btrbk or snapper | snapper + snap-pac | -| zfs rollback | snapper rollback | snapper undochange | -| Pre-pacman snapshots | snap-pac | snap-pac + snapper | - -** Important Btrfs Limitations - -*** RAID 5/6 is Unstable -#+BEGIN_QUOTE -"Parity RAID (RAID 5/6) code has multiple serious data-loss bugs in it." --- Btrfs Wiki -#+END_QUOTE - -For multi-disk Btrfs, only offer: -- *raid1* (mirror) - Stable, recommended -- *raid10* (striped mirrors) - Requires 4+ disks -- *raid0* (stripe) - No redundancy, not recommended for root - -*** No Native Encryption -Btrfs doesn't have native encryption. Use LUKS2 as layer beneath: -#+BEGIN_SRC bash -cryptsetup luksFormat /dev/sdX -cryptsetup open /dev/sdX cryptroot -mkfs.btrfs /dev/mapper/cryptroot -#+END_SRC - -*** Bootloader Considerations -- *GRUB* is required for booting into snapshots (via grub-btrfs) -- *systemd-boot* can't boot from btrfs snapshots (kernel on FAT32 ESP) -- ZFSBootMenu equivalent: GRUB + grub-btrfs + grub-btrfsd daemon - -* Recommended Btrfs Subvolume Layout - -Based on research from multiple sources, this layout provides optimal snapshot -management and rollback capability: - -#+BEGIN_SRC -btrfs volume (LABEL=archsnap) -│ -├── @ (subvol) → mounted at / -├── @home (subvol) → mounted at /home -├── @snapshots (subvol) → mounted at /.snapshots -├── @var_log (subvol) → mounted at /var/log -├── @var_cache (subvol) → mounted at /var/cache -└── @var_tmp (subvol) → mounted at /var/tmp -#+END_SRC - -** Why This Layout? - -1. *@ for root* - Main system, snapshotted for rollbacks -2. *@home separate* - User data not affected by system rollbacks -3. *@snapshots separate* - Snapper requirement for snapshot storage -4. *@var_log separate* - Logs persist across rollbacks, required for read-only snapshots -5. *@var_cache separate* - Package cache excluded from snapshots (saves space) -6. *@var_tmp separate* - Temp files excluded from snapshots - -** Mount Options - -#+BEGIN_SRC bash -# Recommended mount options -BTRFS_OPTS="noatime,compress=zstd,space_cache=v2,discard=async" - -# Example fstab entries (NO subvolid - allows rollback) -UUID=xxx / btrfs $BTRFS_OPTS,subvol=@ 0 0 -UUID=xxx /home btrfs $BTRFS_OPTS,subvol=@home 0 0 -UUID=xxx /.snapshots btrfs $BTRFS_OPTS,subvol=@snapshots 0 0 -UUID=xxx /var/log btrfs $BTRFS_OPTS,subvol=@var_log 0 0 -UUID=xxx /var/cache btrfs $BTRFS_OPTS,subvol=@var_cache 0 0 -UUID=xxx /var/tmp btrfs $BTRFS_OPTS,subvol=@var_tmp 0 0 -#+END_SRC - -*CRITICAL*: Do NOT use subvolid= in fstab - it breaks rollbacks! - -* Snapshot Management Stack - -** Recommended: Snapper + snap-pac + grub-btrfs + btrfs-assistant - -| Component | Purpose | Package | -|-----------+---------+---------| -| snapper | Snapshot creation, retention, rollback | snapper | -| snap-pac | Automatic pre/post pacman snapshots | snap-pac | -| grub-btrfs | Bootable snapshot menu in GRUB | grub-btrfs | -| grub-btrfsd | Auto-update GRUB on snapshot changes | grub-btrfs (service) | -| btrfs-assistant | GUI for snapshot management | btrfs-assistant (AUR) | -| snapper-gui | Alternative lightweight GUI | snapper-gui (AUR) | - -** Alternative: btrbk - -btrbk is simpler than snapper and handles both snapshots and remote backups: -#+BEGIN_SRC conf -# /etc/btrbk.conf -snapshot_preserve_min 2d -snapshot_preserve 24h 7d 4w 12m - -volume / - snapshot_dir /.snapshots - subvolume @ -#+END_SRC - -** Retention Policy (Snapper) - -Default snapper timeline for root: -#+BEGIN_SRC conf -# /etc/snapper/configs/root -TIMELINE_CREATE="yes" -TIMELINE_CLEANUP="yes" -TIMELINE_MIN_AGE="1800" -TIMELINE_LIMIT_HOURLY="5" -TIMELINE_LIMIT_DAILY="7" -TIMELINE_LIMIT_WEEKLY="0" -TIMELINE_LIMIT_MONTHLY="0" -TIMELINE_LIMIT_YEARLY="0" -#+END_SRC - -Keeps: 5 hourly + 7 daily = ~12 snapshots at any time. - -* Multi-Disk Btrfs Setup - -** Mirror (2+ disks) - RECOMMENDED - -#+BEGIN_SRC bash -# Create mirrored filesystem -mkfs.btrfs -L archsnap -d raid1 -m raid1 /dev/sda2 /dev/sdb2 - -# Mount using ANY device (btrfs auto-discovers others) -mount /dev/sda2 /mnt - -# Create subvolumes -btrfs subvolume create /mnt/@ -btrfs subvolume create /mnt/@home -# ... etc -#+END_SRC - -** Stripe (2+ disks) - NOT RECOMMENDED for root - -#+BEGIN_SRC bash -# Striped data, mirrored metadata (default) -mkfs.btrfs -L archsnap -d raid0 -m raid1 /dev/sda2 /dev/sdb2 -#+END_SRC - -Metadata is mirrored by default even in raid0 mode for safety. - -** Converting Single to Mirror - -#+BEGIN_SRC bash -# Add second device to existing filesystem -mount /dev/sda2 /mnt -btrfs device add /dev/sdb2 /mnt -btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt -#+END_SRC - -* Reference Repositories Cloned - -Located in =reference-repos/=: - -| Repository | Purpose | -|------------+---------| -| grub-btrfs | GRUB snapshot menu integration | -| easy-arch | Automated Arch+Btrfs installer | -| arch-btrfs-installation | Manual install guide | -| btrfs-assistant | GUI snapshot manager | -| snap-pac | Pacman hook for snapper | -| btrbk | Alternative snapshot/backup tool | -| buttermanager | GUI btrfs management | -| alis | Arch Linux Install Script (multi-fs) | - -* Installer Architecture Changes - -** Current Structure (ZFS-only) - -#+BEGIN_SRC -install-archzfs -├── Phase 1: Configuration gathering (fzf prompts) -├── Phase 2: Disk partitioning (EFI + ZFS) -├── Phase 3: ZFS pool/dataset creation -├── Phase 4: Pacstrap base system -├── Phase 5: System configuration -├── Phase 6: ZFSBootMenu installation -├── Phase 7: Genesis snapshot -└── Phase 8: Cleanup -#+END_SRC - -** Proposed Structure (Multi-filesystem) - -#+BEGIN_SRC -install-archsnap -├── Phase 1: Configuration gathering -│ ├── Hostname, timezone, locale, keymap -│ ├── Disk selection (multi-select) -│ ├── RAID level (if multi-disk) -│ ├── NEW: Filesystem selection (ZFS / Btrfs) -│ ├── Encryption passphrase -│ ├── Root password -│ ├── NEW: Initial user password -│ └── SSH configuration -│ -├── Phase 2: Disk partitioning -│ ├── EFI partition (same for both) -│ └── Root partition (type depends on FS choice) -│ -├── Phase 3: Filesystem creation -│ ├── IF ZFS: create pool + datasets -│ └── IF Btrfs: create volume + subvolumes -│ -├── Phase 4: Pacstrap base system -│ ├── Common packages -│ ├── IF ZFS: zfs-dkms, zfs-utils -│ └── IF Btrfs: btrfs-progs, snapper, snap-pac, grub-btrfs -│ -├── Phase 5: System configuration -│ ├── Common config (locale, timezone, hostname) -│ ├── IF ZFS: ZFS-specific configs -│ └── IF Btrfs: snapper configs, grub-btrfs configs -│ -├── Phase 6: Bootloader installation -│ ├── IF ZFS: ZFSBootMenu -│ └── IF Btrfs: GRUB + grub-btrfs -│ -├── Phase 7: Genesis snapshot -│ ├── IF ZFS: zfs snapshot -r zroot@genesis -│ └── IF Btrfs: snapper create -c root -d "genesis" -│ -└── Phase 8: Cleanup -#+END_SRC - -** Code Organization - -#+BEGIN_SRC bash -install-archsnap -├── lib/ -│ ├── common.sh # Shared functions (colors, prompts, fzf) -│ ├── disk.sh # Partitioning (shared) -│ ├── zfs.sh # ZFS-specific functions -│ ├── btrfs.sh # Btrfs-specific functions -│ └── config.sh # Config file handling -├── configs/ -│ ├── snapper-root.conf -│ └── btrbk.conf.example -└── install-archsnap # Main script -#+END_SRC - -* Testing Strategy - -** VM Test Matrix - -- Single ZFS (1 disk): Install, boot, reboot, rollback+reboot -- Single Btrfs (1 disk): Install, boot, reboot, rollback+reboot -- Mirror ZFS (2 disk): Install, fail disk, recover -- Mirror Btrfs (2 disk, raid1): Install, fail disk, recover -- RAIDZ1 ZFS (3 disk): Install, fail disk, recover - -** Validation Checks - -*** Fresh Install (all configs) -- Partitions created correctly (EFI + root) -- Filesystem created (pool/subvols) -- All mount points accessible -- Packages installed (pacman -Q zfs-dkms or btrfs-progs) -- Services enabled (zfs.target or snapper-timeline.timer) -- Bootloader installed (ZFSBootMenu or GRUB + grub-btrfs) -- fstab correct (no subvolid for btrfs) -- Can boot without ISO - -*** Reboot Survival -Critical: Verify system survives reboot cleanly. Catches: -- Misconfigured services slamming CPU/network -- Services that fail to start properly -- ZFS import issues -- fstab mount failures - -Checks after reboot: -- System reaches login prompt -- All filesystems mounted (findmnt) -- No failed services (systemctl --failed) -- CPU/memory/network normal (no runaway processes) -- Can SSH in (if network configured) - -*** Snapshot Operations -- Manual snapshot creates successfully -- Snapshot appears in list (zfs list -t snap / snapper list) -- Pre-pacman snapshot created automatically (install package, verify) -- Snapshot visible in boot menu (ZFSBootMenu / GRUB) -- Can boot into snapshot - -*** Rollback + Reboot -Important: Rollback MUST include reboot to validate fully. -Do NOT use ZFSBootMenu's built-in rollback (known bug, needs filing). -Use zfsrollback script or snapper rollback instead. - -Checks: -- Rollback command completes without error -- Reboot completes successfully -- System state matches snapshot after reboot -- Previously installed test package is gone -- No orphaned/broken state - -*** Failure Recovery (multi-disk only) -- Pool/volume shows correct redundancy (zpool status / btrfs fi show) -- Survives single disk removal (zpool offline / btrfs device delete) -- Boots with degraded array -- Warnings displayed about degraded state -- Can resilver/rebuild after disk replaced (zpool replace / btrfs replace) -- Pool returns to healthy state - -*** Encryption -- Passphrase prompt appears at boot -- Correct unlock with right passphrase -- Fails gracefully with wrong passphrase (retry prompt, not panic) -- Data unreadable when mounted elsewhere without key -- Pool/volume not auto-imported without passphrase - -** Known Issues - -*** ZFSBootMenu Rollback Bug -ZFSBootMenu's built-in rollback feature has issues (to be filed). -Workaround: Use zfsrollback script from installed system or live ISO. - -** Test Scripts - -#+BEGIN_SRC -scripts/ -├── test-zfs-single.sh -├── test-zfs-mirror.sh -├── test-zfs-raidz.sh -├── test-btrfs-single.sh -├── test-btrfs-mirror.sh -└── test-configs/ - ├── zfs-single.conf - ├── zfs-mirror.conf - ├── btrfs-single.conf - └── btrfs-mirror.conf -#+END_SRC - -* CLI and GUI Tools - -** CLI Tools to Install (Both Filesystems) - -| Tool | Purpose | -|------+---------| -| archsnap-snapshot | Create manual snapshot (wrapper) | -| archsnap-rollback | Interactive rollback (fzf-based) | -| archsnap-list | List snapshots | -| archsnap-prune | Manual pruning | - -These would be thin wrappers that detect the filesystem and call the appropriate -underlying tool (zfs/snapper/btrbk). - -** GUI Options - -*** For Btrfs -- *btrfs-assistant* (recommended) - Full-featured, actively maintained -- *snapper-gui* - Lighter weight alternative - -*** For ZFS -- No equivalent mature GUI exists -- Could build a simple zenity/yad wrapper - -* Implementation Phases - -** Phase 1: Refactor Current Installer -1. Extract common functions to lib/common.sh -2. Extract ZFS-specific code to lib/zfs.sh -3. Add filesystem selection prompt -4. Rename project to archsnap - -** Phase 2: Implement Btrfs Support -1. Create lib/btrfs.sh with btrfs-specific functions -2. Implement subvolume creation -3. Implement snapper configuration -4. Implement GRUB + grub-btrfs installation -5. Create btrfs genesis snapshot - -** Phase 3: Testing Infrastructure -1. Create test configs for all scenarios -2. Update test-vm.sh for btrfs testing -3. Create automated test suite -4. Test multi-disk configurations - -** Phase 4: CLI Tools -1. Create archsnap-snapshot wrapper -2. Create archsnap-rollback wrapper -3. Update documentation - -** Phase 5: Documentation & Polish -1. Update README for dual-filesystem support -2. Create BTRFS-specific documentation -3. Update troubleshooting guides - -* Sources - -** Btrfs Installation -- [[https://wiki.archlinux.org/title/Btrfs][Arch Wiki - Btrfs]] -- [[https://github.com/egara/arch-btrfs-installation][arch-btrfs-installation]] -- [[https://gist.github.com/mjkstra/96ce7a5689d753e7a6bdd92cdc169bae][Modern Arch+Btrfs Guide]] - -** Snapshot Management -- [[https://github.com/Antynea/grub-btrfs][grub-btrfs]] -- [[https://wiki.archlinux.org/title/Snapper][Arch Wiki - Snapper]] -- [[https://github.com/wesbarnett/snap-pac][snap-pac]] -- [[https://github.com/digint/btrbk][btrbk]] - -** GUI Tools -- [[https://gitlab.com/btrfs-assistant/btrfs-assistant][btrfs-assistant]] -- [[https://github.com/egara/buttermanager][buttermanager]] - -** Multi-Disk -- [[https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices.html][Btrfs Wiki - Multiple Devices]] -- [[https://thelinuxcode.com/set-up-btrfs-raid/][How to Set Up Btrfs RAID]] |