diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/research-btrfs-expansion.org | 408 | ||||
| -rw-r--r-- | docs/session-context.org | 96 |
2 files changed, 467 insertions, 37 deletions
diff --git a/docs/research-btrfs-expansion.org b/docs/research-btrfs-expansion.org new file mode 100644 index 0000000..300f5cd --- /dev/null +++ b/docs/research-btrfs-expansion.org @@ -0,0 +1,408 @@ +#+TITLE: Research: Expanding Project to Support Btrfs +#+DATE: 2026-01-23 +#+AUTHOR: Craig Jennings & Claude + +* Executive Summary + +This document covers research and architecture planning for expanding the archzfs +project to support both ZFS and Btrfs filesystems, providing equivalent functionality +for both. + +* Project Renaming + +Since "archzfs" implies ZFS-only, the project needs a new name. + +** Naming Options + +| Name | Pros | Cons | +|------+------+------| +| archsnap | Short, implies snapshots | Doesn't mention filesystems | +| archfs | Generic filesystem installer | Might be too generic | +| archroot | Implies root filesystem setup | Already used by other tools | +| arch-snap-install | Descriptive | Long | +| snaparch | Short, memorable | Sounds like "snap" package manager | +| archraid | Implies multi-disk | Doesn't mention snapshots | +| arch-resilient | Implies reliability | Long, vague | + +** Recommendation +*archsnap* - Short, memorable, emphasizes the snapshot capability that's the core +value proposition of both ZFS and Btrfs configurations. + +* Btrfs Equivalent Features + +** Feature Parity Matrix + +| Feature (ZFS) | Btrfs Equivalent | Tool/Method | +|---------------+------------------+-------------| +| ZFS native encryption | LUKS2 + dm-crypt | cryptsetup | +| zpool mirror | btrfs raid1 | mkfs.btrfs -d raid1 -m raid1 | +| zpool stripe | btrfs raid0 | mkfs.btrfs -d raid0 | +| zpool raidz1 | btrfs raid5 | NOT RECOMMENDED (unstable) | +| zpool raidz2 | btrfs raid6 | NOT RECOMMENDED (unstable) | +| ZFS datasets | Btrfs subvolumes | btrfs subvolume create | +| zfs snapshot | btrfs snapshot | btrfs subvolume snapshot | +| ZFSBootMenu | GRUB + grub-btrfs | grub-btrfs package | +| sanoid/syncoid | btrbk or snapper | snapper + snap-pac | +| zfs rollback | snapper rollback | snapper undochange | +| Pre-pacman snapshots | snap-pac | snap-pac + snapper | + +** Important Btrfs Limitations + +*** RAID 5/6 is Unstable +#+BEGIN_QUOTE +"Parity RAID (RAID 5/6) code has multiple serious data-loss bugs in it." +-- Btrfs Wiki +#+END_QUOTE + +For multi-disk Btrfs, only offer: +- *raid1* (mirror) - Stable, recommended +- *raid10* (striped mirrors) - Requires 4+ disks +- *raid0* (stripe) - No redundancy, not recommended for root + +*** No Native Encryption +Btrfs doesn't have native encryption. Use LUKS2 as layer beneath: +#+BEGIN_SRC bash +cryptsetup luksFormat /dev/sdX +cryptsetup open /dev/sdX cryptroot +mkfs.btrfs /dev/mapper/cryptroot +#+END_SRC + +*** Bootloader Considerations +- *GRUB* is required for booting into snapshots (via grub-btrfs) +- *systemd-boot* can't boot from btrfs snapshots (kernel on FAT32 ESP) +- ZFSBootMenu equivalent: GRUB + grub-btrfs + grub-btrfsd daemon + +* Recommended Btrfs Subvolume Layout + +Based on research from multiple sources, this layout provides optimal snapshot +management and rollback capability: + +#+BEGIN_SRC +btrfs volume (LABEL=archsnap) +│ +├── @ (subvol) → mounted at / +├── @home (subvol) → mounted at /home +├── @snapshots (subvol) → mounted at /.snapshots +├── @var_log (subvol) → mounted at /var/log +├── @var_cache (subvol) → mounted at /var/cache +└── @var_tmp (subvol) → mounted at /var/tmp +#+END_SRC + +** Why This Layout? + +1. *@ for root* - Main system, snapshotted for rollbacks +2. *@home separate* - User data not affected by system rollbacks +3. *@snapshots separate* - Snapper requirement for snapshot storage +4. *@var_log separate* - Logs persist across rollbacks, required for read-only snapshots +5. *@var_cache separate* - Package cache excluded from snapshots (saves space) +6. *@var_tmp separate* - Temp files excluded from snapshots + +** Mount Options + +#+BEGIN_SRC bash +# Recommended mount options +BTRFS_OPTS="noatime,compress=zstd,space_cache=v2,discard=async" + +# Example fstab entries (NO subvolid - allows rollback) +UUID=xxx / btrfs $BTRFS_OPTS,subvol=@ 0 0 +UUID=xxx /home btrfs $BTRFS_OPTS,subvol=@home 0 0 +UUID=xxx /.snapshots btrfs $BTRFS_OPTS,subvol=@snapshots 0 0 +UUID=xxx /var/log btrfs $BTRFS_OPTS,subvol=@var_log 0 0 +UUID=xxx /var/cache btrfs $BTRFS_OPTS,subvol=@var_cache 0 0 +UUID=xxx /var/tmp btrfs $BTRFS_OPTS,subvol=@var_tmp 0 0 +#+END_SRC + +*CRITICAL*: Do NOT use subvolid= in fstab - it breaks rollbacks! + +* Snapshot Management Stack + +** Recommended: Snapper + snap-pac + grub-btrfs + btrfs-assistant + +| Component | Purpose | Package | +|-----------+---------+---------| +| snapper | Snapshot creation, retention, rollback | snapper | +| snap-pac | Automatic pre/post pacman snapshots | snap-pac | +| grub-btrfs | Bootable snapshot menu in GRUB | grub-btrfs | +| grub-btrfsd | Auto-update GRUB on snapshot changes | grub-btrfs (service) | +| btrfs-assistant | GUI for snapshot management | btrfs-assistant (AUR) | +| snapper-gui | Alternative lightweight GUI | snapper-gui (AUR) | + +** Alternative: btrbk + +btrbk is simpler than snapper and handles both snapshots and remote backups: +#+BEGIN_SRC conf +# /etc/btrbk.conf +snapshot_preserve_min 2d +snapshot_preserve 24h 7d 4w 12m + +volume / + snapshot_dir /.snapshots + subvolume @ +#+END_SRC + +** Retention Policy (Snapper) + +Default snapper timeline for root: +#+BEGIN_SRC conf +# /etc/snapper/configs/root +TIMELINE_CREATE="yes" +TIMELINE_CLEANUP="yes" +TIMELINE_MIN_AGE="1800" +TIMELINE_LIMIT_HOURLY="5" +TIMELINE_LIMIT_DAILY="7" +TIMELINE_LIMIT_WEEKLY="0" +TIMELINE_LIMIT_MONTHLY="0" +TIMELINE_LIMIT_YEARLY="0" +#+END_SRC + +Keeps: 5 hourly + 7 daily = ~12 snapshots at any time. + +* Multi-Disk Btrfs Setup + +** Mirror (2+ disks) - RECOMMENDED + +#+BEGIN_SRC bash +# Create mirrored filesystem +mkfs.btrfs -L archsnap -d raid1 -m raid1 /dev/sda2 /dev/sdb2 + +# Mount using ANY device (btrfs auto-discovers others) +mount /dev/sda2 /mnt + +# Create subvolumes +btrfs subvolume create /mnt/@ +btrfs subvolume create /mnt/@home +# ... etc +#+END_SRC + +** Stripe (2+ disks) - NOT RECOMMENDED for root + +#+BEGIN_SRC bash +# Striped data, mirrored metadata (default) +mkfs.btrfs -L archsnap -d raid0 -m raid1 /dev/sda2 /dev/sdb2 +#+END_SRC + +Metadata is mirrored by default even in raid0 mode for safety. + +** Converting Single to Mirror + +#+BEGIN_SRC bash +# Add second device to existing filesystem +mount /dev/sda2 /mnt +btrfs device add /dev/sdb2 /mnt +btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt +#+END_SRC + +* Reference Repositories Cloned + +Located in =reference-repos/=: + +| Repository | Purpose | +|------------+---------| +| grub-btrfs | GRUB snapshot menu integration | +| easy-arch | Automated Arch+Btrfs installer | +| arch-btrfs-installation | Manual install guide | +| btrfs-assistant | GUI snapshot manager | +| snap-pac | Pacman hook for snapper | +| btrbk | Alternative snapshot/backup tool | +| buttermanager | GUI btrfs management | +| alis | Arch Linux Install Script (multi-fs) | + +* Installer Architecture Changes + +** Current Structure (ZFS-only) + +#+BEGIN_SRC +install-archzfs +├── Phase 1: Configuration gathering (fzf prompts) +├── Phase 2: Disk partitioning (EFI + ZFS) +├── Phase 3: ZFS pool/dataset creation +├── Phase 4: Pacstrap base system +├── Phase 5: System configuration +├── Phase 6: ZFSBootMenu installation +├── Phase 7: Genesis snapshot +└── Phase 8: Cleanup +#+END_SRC + +** Proposed Structure (Multi-filesystem) + +#+BEGIN_SRC +install-archsnap +├── Phase 1: Configuration gathering +│ ├── Hostname, timezone, locale, keymap +│ ├── Disk selection (multi-select) +│ ├── RAID level (if multi-disk) +│ ├── NEW: Filesystem selection (ZFS / Btrfs) +│ ├── Encryption passphrase +│ ├── Root password +│ ├── NEW: Initial user password +│ └── SSH configuration +│ +├── Phase 2: Disk partitioning +│ ├── EFI partition (same for both) +│ └── Root partition (type depends on FS choice) +│ +├── Phase 3: Filesystem creation +│ ├── IF ZFS: create pool + datasets +│ └── IF Btrfs: create volume + subvolumes +│ +├── Phase 4: Pacstrap base system +│ ├── Common packages +│ ├── IF ZFS: zfs-dkms, zfs-utils +│ └── IF Btrfs: btrfs-progs, snapper, snap-pac, grub-btrfs +│ +├── Phase 5: System configuration +│ ├── Common config (locale, timezone, hostname) +│ ├── IF ZFS: ZFS-specific configs +│ └── IF Btrfs: snapper configs, grub-btrfs configs +│ +├── Phase 6: Bootloader installation +│ ├── IF ZFS: ZFSBootMenu +│ └── IF Btrfs: GRUB + grub-btrfs +│ +├── Phase 7: Genesis snapshot +│ ├── IF ZFS: zfs snapshot -r zroot@genesis +│ └── IF Btrfs: snapper create -c root -d "genesis" +│ +└── Phase 8: Cleanup +#+END_SRC + +** Code Organization + +#+BEGIN_SRC bash +install-archsnap +├── lib/ +│ ├── common.sh # Shared functions (colors, prompts, fzf) +│ ├── disk.sh # Partitioning (shared) +│ ├── zfs.sh # ZFS-specific functions +│ ├── btrfs.sh # Btrfs-specific functions +│ └── config.sh # Config file handling +├── configs/ +│ ├── snapper-root.conf +│ └── btrbk.conf.example +└── install-archsnap # Main script +#+END_SRC + +* Testing Strategy + +** VM Test Matrix + +| Config | Disks | Filesystem | RAID | Test | +|--------+-------+------------+------+------| +| Single ZFS | 1 | ZFS | - | Install, boot, rollback | +| Single Btrfs | 1 | Btrfs | - | Install, boot, rollback | +| Mirror ZFS | 2 | ZFS | mirror | Install, fail disk, recover | +| Mirror Btrfs | 2 | Btrfs | raid1 | Install, fail disk, recover | +| RAIDZ1 ZFS | 3 | ZFS | raidz1 | Install, fail disk, recover | + +** Test Scenarios + +1. *Fresh Install* + - Single disk, multi-disk + - Interactive and config-file modes + +2. *Snapshot Operations* + - Manual snapshot creation + - Automatic pre-pacman snapshots + - Boot into snapshot + - Rollback to snapshot + +3. *Failure Recovery* + - Single disk failure (mirror/raidz) + - Corrupt boot + - Kernel update breaks system + +4. *Encryption* + - ZFS native encryption + - LUKS + Btrfs encryption + +** Test Scripts + +#+BEGIN_SRC +scripts/ +├── test-zfs-single.sh +├── test-zfs-mirror.sh +├── test-zfs-raidz.sh +├── test-btrfs-single.sh +├── test-btrfs-mirror.sh +└── test-configs/ + ├── zfs-single.conf + ├── zfs-mirror.conf + ├── btrfs-single.conf + └── btrfs-mirror.conf +#+END_SRC + +* CLI and GUI Tools + +** CLI Tools to Install (Both Filesystems) + +| Tool | Purpose | +|------+---------| +| archsnap-snapshot | Create manual snapshot (wrapper) | +| archsnap-rollback | Interactive rollback (fzf-based) | +| archsnap-list | List snapshots | +| archsnap-prune | Manual pruning | + +These would be thin wrappers that detect the filesystem and call the appropriate +underlying tool (zfs/snapper/btrbk). + +** GUI Options + +*** For Btrfs +- *btrfs-assistant* (recommended) - Full-featured, actively maintained +- *snapper-gui* - Lighter weight alternative + +*** For ZFS +- No equivalent mature GUI exists +- Could build a simple zenity/yad wrapper + +* Implementation Phases + +** Phase 1: Refactor Current Installer +1. Extract common functions to lib/common.sh +2. Extract ZFS-specific code to lib/zfs.sh +3. Add filesystem selection prompt +4. Rename project to archsnap + +** Phase 2: Implement Btrfs Support +1. Create lib/btrfs.sh with btrfs-specific functions +2. Implement subvolume creation +3. Implement snapper configuration +4. Implement GRUB + grub-btrfs installation +5. Create btrfs genesis snapshot + +** Phase 3: Testing Infrastructure +1. Create test configs for all scenarios +2. Update test-vm.sh for btrfs testing +3. Create automated test suite +4. Test multi-disk configurations + +** Phase 4: CLI Tools +1. Create archsnap-snapshot wrapper +2. Create archsnap-rollback wrapper +3. Update documentation + +** Phase 5: Documentation & Polish +1. Update README for dual-filesystem support +2. Create BTRFS-specific documentation +3. Update troubleshooting guides + +* Sources + +** Btrfs Installation +- [[https://wiki.archlinux.org/title/Btrfs][Arch Wiki - Btrfs]] +- [[https://github.com/egara/arch-btrfs-installation][arch-btrfs-installation]] +- [[https://gist.github.com/mjkstra/96ce7a5689d753e7a6bdd92cdc169bae][Modern Arch+Btrfs Guide]] + +** Snapshot Management +- [[https://github.com/Antynea/grub-btrfs][grub-btrfs]] +- [[https://wiki.archlinux.org/title/Snapper][Arch Wiki - Snapper]] +- [[https://github.com/wesbarnett/snap-pac][snap-pac]] +- [[https://github.com/digint/btrbk][btrbk]] + +** GUI Tools +- [[https://gitlab.com/btrfs-assistant/btrfs-assistant][btrfs-assistant]] +- [[https://github.com/egara/buttermanager][buttermanager]] + +** Multi-Disk +- [[https://archive.kernel.org/oldwiki/btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices.html][Btrfs Wiki - Multiple Devices]] +- [[https://thelinuxcode.com/set-up-btrfs-raid/][How to Set Up Btrfs RAID]] diff --git a/docs/session-context.org b/docs/session-context.org index 580be99..2f390fd 100644 --- a/docs/session-context.org +++ b/docs/session-context.org @@ -4,48 +4,70 @@ * Current Session: Friday 2026-01-23 @ 11:28 CST ** Current Task -Code review of archzfs project complete. Issues documented in todo.org. +Btrfs expansion research complete. Architecture documented. ** Completed This Session -- Merged zfsbootmenu branch to main (fast-forward, pushed to origin) +- Merged zfsbootmenu branch to main - ratio booted successfully with ZFSBootMenu -- Researched code review best practices (10+ web searches) -- Created docs/project-workflows/code-review.org - - Core Checklist: ~27 items across pyramid levels - - Deep Dives: Security, Performance, Concurrency, Error Handling, API Compatibility, Dependencies -- Conducted senior developer code review of entire archzfs codebase -- Added 14 new tasks to todo.org based on code review findings - -** Code Review Findings Summary -*** Critical (Priority A) -- README.org has outdated GRUB references (now uses ZFSBootMenu) -- Missing LICENSE file (referenced but doesn't exist) -- custom/archsetup-zfs is a non-functional skeleton -- Initial password should be configurable, not hardcoded - -*** Important (Priority B) -- Stale SESSION-CONTEXT.md should be deleted -- PLAN-zfsbootmenu-implementation.org should move to docs/ -- Makefile lint target swallows errors -- Unclear directories need documentation or gitignore -- Empty docs/someday-maybe.org - -*** Cleanup (Priority C) -- Inconsistent shebangs across scripts -- Inconsistent email addresses -- No .editorconfig for formatting -- Test scripts need documentation +- Created code review workflow (docs/project-workflows/code-review.org) +- Conducted senior developer code review, added 14 cleanup tasks +- Committed and pushed changes -** Files Created This Session -- docs/project-workflows/code-review.org - Code review workflow +*** Btrfs Expansion Research +- Researched btrfs best practices for Arch Linux +- Researched snapshot management: snapper, btrbk, snap-pac +- Researched GRUB integration: grub-btrfs, grub-btrfsd +- Researched multi-disk configurations: raid1, raid10 (raid5/6 unstable) +- Researched GUI tools: btrfs-assistant, snapper-gui, buttermanager +- Cloned 8 reference repositories to reference-repos/ +- Created comprehensive research document: docs/research-btrfs-expansion.org + +** Key Research Findings + +*** Project Rename Recommendation +*archsnap* - Short, memorable, emphasizes snapshot capability + +*** Btrfs Feature Parity +| ZFS Feature | Btrfs Equivalent | +|-------------+------------------| +| Native encryption | LUKS2 + dm-crypt | +| Mirror | btrfs raid1 | +| ZFSBootMenu | GRUB + grub-btrfs | +| sanoid | snapper + snap-pac | +| Datasets | Subvolumes | -** Files Modified This Session -- todo.org - Added Makefile targets, code review tasks (14 new items) +*** Btrfs Limitations +- RAID 5/6 is UNSTABLE - do not offer +- No native encryption - must use LUKS +- Need GRUB for bootable snapshots (not systemd-boot) -** Pending User Review -- docs/project-workflows/code-review.org (task in todo.org) +*** Recommended Subvolume Layout +- @ → / +- @home → /home +- @snapshots → /.snapshots +- @var_log → /var/log +- @var_cache → /var/cache + +*** Snapshot Stack +snapper + snap-pac + grub-btrfs + btrfs-assistant + +** Reference Repos Cloned +- grub-btrfs (GRUB snapshot menu) +- easy-arch (automated installer) +- arch-btrfs-installation (manual guide) +- btrfs-assistant (GUI) +- snap-pac (pacman hooks) +- btrbk (snapshot/backup) +- buttermanager (GUI) +- alis (multi-fs installer) + +** Files Created This Session +- docs/project-workflows/code-review.org +- docs/research-btrfs-expansion.org ** Next Steps -- User to review code-review.org document -- Address Priority A issues from code review -- Delete SESSION-CONTEXT.md (root) after confirming this file is canonical +1. Review research document with Craig +2. Decide on project name +3. Decide on implementation approach +4. Create implementation tasks in todo.org +5. Begin Phase 1: Refactor current installer |