From d9bf5d2ffc471fc22c1bcc6ec80a881319ea1678 Mon Sep 17 00:00:00 2001 From: Craig Jennings Date: Mon, 19 Jan 2026 11:34:32 -0600 Subject: Update session context with full workflow details Documents automated sanity test, build-release workflow, and all fixes made during this session. --- SESSION-CONTEXT.md | 164 +++++++++++++++++++++++++++++++++++------------------ 1 file changed, 110 insertions(+), 54 deletions(-) diff --git a/SESSION-CONTEXT.md b/SESSION-CONTEXT.md index 3fc6c05..ee2d12f 100644 --- a/SESSION-CONTEXT.md +++ b/SESSION-CONTEXT.md @@ -1,36 +1,67 @@ # Session Context - 2026-01-19 ## Summary -Added ZFS snapshot management scripts, fixed grub-zfs-snap syntax error, and created build-release workflow. +Major session: Added ZFS snapshot scripts, fixed grub-zfs-snap, created fully automated build-release workflow with sanity testing. -## Key Changes This Session +## Key Accomplishments -### 1. Added zfssnapshot and zfsrollback to ISO -- Copied from archsetup to `custom/zfssnapshot` and `custom/zfsrollback` -- Now installed to `/usr/local/bin/` on live ISO and target systems -- Added fzf to `install-archzfs` pacstrap list (required dependency) -- Increased fzf height in zfsrollback from 40% to 70% for better usability -- Created task in archsetup inbox to remove duplicate scripts there +### 1. ZFS Snapshot Management Scripts +- Copied `zfssnapshot` and `zfsrollback` from archsetup to archzfs ISO +- Scripts now installed to `/usr/local/bin/` on live ISO and target systems +- Added `fzf` to `install-archzfs` pacstrap (required by zfsrollback) +- Fixed zfsrollback fzf height (40% → 70%) +- Fixed zfsrollback to sort datasets by depth (children before parents) for proper rollback order +- Created task in archsetup inbox to remove duplicate scripts ### 2. Fixed grub-zfs-snap Syntax Error -- **Bug**: `\$(grub-probe ...)` was writing literal string to grub.cfg -- **Fix**: Removed backslash so command executes at config generation time -- **File**: `custom/grub-zfs-snap` line 106 +- **Bug:** `\$(grub-probe ...)` wrote literal string to grub.cfg +- **Fix:** Removed backslash so command executes at config generation time +- **File:** `custom/grub-zfs-snap` line 106 - GRUB doesn't support bash-style `$()` command substitution -### 3. Created build-release Workflow -- **Script**: `scripts/build-release` -- **Purpose**: Automate full ISO build and distribution -- **Steps**: Build → Sanity test (QEMU) → Distribute -- **Targets**: - - `~/Downloads/isos` (always) - - `truenas.local:/mnt/vault/isos` (if reachable) - - ARCHZFS labeled USB drive (detected via `blkid -L ARCHZFS`) - - Ventoy USB drive (detected by label "Ventoy" or `ventoy/` directory) -- **Options**: `--skip-build`, `--skip-test` - -### 4. Added eBPF Tracing Tools (earlier in session) -- Added to ISO: bpftrace, bcc-tools, perf, w3m +### 3. Automated Sanity Test (`scripts/sanity-test.sh`) +- Boots ISO in headless QEMU (no display) +- Waits for SSH availability (port 2223, timeout 180s) +- Runs 13 automated tests via SSH: + - ZFS module loaded + - ZFS/zpool commands work + - Custom scripts present (zfsrollback, zfssnapshot, grub-zfs-snap, etc.) + - fzf installed + - LTS kernel running + - archsetup directory present +- Reports pass/fail summary +- Fully automated - no human input required + +### 4. Build-Release Workflow (`scripts/build-release`) +Complete automated workflow for ISO build and distribution: + +**Usage:** +```bash +sudo ./scripts/build-release # Full: build, test, distribute +sudo ./scripts/build-release --yes # Full with auto-confirm dd +sudo ./scripts/build-release --skip-build --skip-test --yes # Just distribute +``` + +**Options:** +- `--skip-build` - Skip ISO build, use existing +- `--skip-test` - Skip sanity test +- `--yes, -y` - Auto-confirm dd to ARCHZFS drive + +**Distribution Targets:** +1. `~/Downloads/isos/` (always) +2. `truenas.local:/mnt/vault/isos/` (via cjennings@, checks reachable) +3. ARCHZFS labeled drive (detected via `blkid -L ARCHZFS`, writes via dd) +4. Ventoy drive (detected by label "Ventoy" or ventoy/ directory) + +**Key Fixes:** +- Uses `SUDO_USER` to get real user's home (not /root) +- SSH/SCP runs as real user to use their SSH keys +- Uses `cjennings@truenas.local` (not root) +- No removable check for ARCHZFS (Framework expansion cards show as internal) +- Graceful handling of failures (warns and continues) + +### 5. Added eBPF Tracing Tools +- Added to ISO: `bpftrace`, `bcc-tools`, `perf`, `w3m` - Updated RESCUE-GUIDE.txt with sections 9 (System Tracing) and 10 (Terminal Web Browsing) ## Technical Notes @@ -38,51 +69,76 @@ Added ZFS snapshot management scripts, fixed grub-zfs-snap syntax error, and cre ### os-prober Warning ``` Warning: os-prober will not be executed to detect other bootable partitions. -Systems on them will not be added to the GRUB boot configuration. -Check GRUB_DISABLE_OS_PROBER documentation entry. ``` -- **Meaning**: GRUB won't scan for other OSes (Windows, etc.) to add to boot menu -- **Why**: Disabled by default since GRUB 2.06 for security (avoids probing untrusted partitions) -- **Impact**: None for single-OS ZFS systems; dual-boot would need to enable it -- **To enable**: Add `GRUB_DISABLE_OS_PROBER=false` to `/etc/default/grub`, then regenerate config -- **Decision**: Not needed for archzfs default config +- Disabled by default since GRUB 2.06 for security +- Not needed for single-OS ZFS systems +- To enable: Add `GRUB_DISABLE_OS_PROBER=false` to `/etc/default/grub` ### Genesis Snapshot Best Practice -When rolling back to genesis, the rollback script itself must be part of genesis. We now: -1. Install zfssnapshot, zfsrollback, and fzf via `install-archzfs` -2. These are present before genesis snapshot is taken -3. Future rollbacks preserve the tools needed to perform rollbacks +Scripts must be part of genesis snapshot to survive rollback: +1. `install-archzfs` installs zfssnapshot, zfsrollback, fzf +2. These exist before genesis snapshot is taken +3. Future rollbacks preserve the tools + +### Framework Laptop Note +Expansion card drives show as internal (RM=0) but are hot-swappable. +Don't check for removable flag when detecting ARCHZFS drives. -### Ratio Machine Credentials +### Ratio Machine - Host: ratio (192.168.86.48) -- Login: root -- Password: cmjdase1n - -## Files Modified -- `custom/zfssnapshot` (new) -- `custom/zfsrollback` (new) -- `custom/grub-zfs-snap` (fixed syntax error) -- `custom/install-archzfs` (added fzf to pacstrap) -- `custom/RESCUE-GUIDE.txt` (added sections 9, 10) -- `build.sh` (added bpftrace, bcc-tools, perf, w3m, copy commands for new scripts) -- `scripts/build-release` (new) -- `TODO.org` (marked zfsrollback/zfssnapshot task done) -- `assets/cogito-hardware-specs.txt` (moved from inbox) +- Login: root / cmjdase1n +- Scripts deployed: zfssnapshot, zfsrollback, grub-zfs-snap, fzf + +### Cogito Machine (Build Host) +- Framework Desktop ML with AMD Ryzen AI MAX+ 395 +- Specs in `assets/cogito-hardware-specs.txt` + +## Files Modified/Created This Session + +**New Files:** +- `scripts/sanity-test.sh` - Automated ISO testing +- `scripts/build-release` - Build and distribution workflow +- `custom/zfssnapshot` - ZFS snapshot creation +- `custom/zfsrollback` - Interactive ZFS rollback +- `assets/cogito-hardware-specs.txt` - Build machine specs +- `/home/cjennings/code/archsetup/inbox/remove-zfs-scripts.md` - Task for archsetup + +**Modified Files:** +- `custom/grub-zfs-snap` - Fixed syntax error +- `custom/install-archzfs` - Added fzf to pacstrap +- `custom/RESCUE-GUIDE.txt` - Added tracing/browser sections +- `build.sh` - Added bpftrace, bcc-tools, perf, w3m, zfs script copies +- `TODO.org` - Updated tasks ## Commits This Session ``` +d0e76f0 Add --yes flag for fully automated distribution +2c261bc Update build-release for TrueNAS and Framework drives +c68e550 Fix build-release for running with sudo +d3eaaff Add automated sanity test for ISO verification +5eba633 Fix zfsrollback to process children before parents 4e7e3fe Fix grub-zfs-snap command substitution syntax error ee668df Increase fzf height in zfsrollback to 70% 7c0d655 Add fzf to target system packages 156dde5 Add zfssnapshot and zfsrollback scripts to ISO e924c53 Add TODO for zfsrollback and zfssnapshot scripts d2c2cca Add eBPF tracing tools and w3m terminal browser +66c92a2 Add session context for 2026-01-19 c719d45 Add build-release script for ISO build and distribution ``` -## Pending -- Run `sudo ./scripts/build-release` to build and distribute the ISO -- Verify on ratio after archsetup completes +## Final Distribution (This Session) +``` +ISO: archzfs-vmlinuz-6.12.66-lts-2026-01-19-x86_64.iso (5.2GB) + +Distributed to: + ✓ /home/cjennings/Downloads/isos/ + ✓ truenas.local:/mnt/vault/isos/ + ✓ /dev/sda (ARCHZFS boot drive) + ✓ /dev/sdb1 (Ventoy) +``` -## Related Files in Other Projects -- `/home/cjennings/code/archsetup/inbox/remove-zfs-scripts.md` - Task to remove duplicate scripts from archsetup +## Pending/Future Work +- User should run archsetup on ratio after genesis rollback +- Consider adding `--full-test` option to build-release for comprehensive testing +- archsetup inbox task: remove duplicate zfssnapshot/zfsrollback scripts -- cgit v1.2.3