diff options
| author | Craig Jennings <c@cjennings.net> | 2026-09-13 09:23:39 -0500 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-09-13 09:23:39 -0500 |
| commit | 2f81174b1de13d9148113fb9f3d389ca6c61ac5c (patch) | |
| tree | 3fe538bd1b43364ccfd0a1de2107ac5171423aee /working/velox-reinstall/velox-reinstall-runbook.org | |
| parent | c9f9a9881548114e49752d5bc4feba4a83ed4d84 (diff) | |
| download | archsetup-2f81174b1de13d9148113fb9f3d389ca6c61ac5c.tar.gz archsetup-2f81174b1de13d9148113fb9f3d389ca6c61ac5c.zip | |
chore(tasks): close the velox reinstall drill and the lock-clock bug
The reinstall drill finished on 2026-08-14 and every finding it surfaced already has its own task, so what remained was the record. I filed its working-dir artifacts into permanent homes. The runbook went to docs/ with an Outcome section, since the checklist was never ticked as it ran. The UEFI boot-entry reference went to docs/ too. The three reinstall-gap reports went to docs/design/. The rescued wttrin bundle moved into its own working dir under the task that owns it. Every inbound link is repointed and working/velox-reinstall/ is gone.
The lock-screen clock bug is closed because it no longer happens on velox. I couldn't identify the commit that fixed it from the dotfiles or archsetup logs, and the note says so. Its manual-testing check retires with it.
Diffstat (limited to 'working/velox-reinstall/velox-reinstall-runbook.org')
| -rw-r--r-- | working/velox-reinstall/velox-reinstall-runbook.org | 161 |
1 files changed, 0 insertions, 161 deletions
diff --git a/working/velox-reinstall/velox-reinstall-runbook.org b/working/velox-reinstall/velox-reinstall-runbook.org deleted file mode 100644 index 0c14946..0000000 --- a/working/velox-reinstall/velox-reinstall-runbook.org +++ /dev/null @@ -1,161 +0,0 @@ -#+TITLE: Velox Reinstall Runbook — DR Test of archangel + archsetup -#+AUTHOR: Craig Jennings -#+DATE: 2026-08-13 - -Context: velox's mainboard swapped Intel → AMD (Ryzen AI 9 HX 370, Radeon -890M, 96GB RAM). Old SSD intact but the new board's NVRAM has no boot entry, -and velox is ZFS root + ZFSBootMenu, so a stock Arch USB can't even read the -pool. Decision: full reinstall via archangel + archsetup, run deliberately as -a disaster-recovery test of the ISO and scripts before the Sunday flight. -Recent backup in hand; ratio available as the working machine. - -Fallback ordering if the test finds a real gap: -- Before partitioning starts: the old system is intact — the ZBM repair - route (efibootmgr entry pointing at the ZBM loader on the ESP, then - amd-ucode swap in a chroot) is still available. -- After partitioning: the floor is a manual Arch install; the backup makes - that survivable. - -* Phase 0 — Preflight on ratio (agent-driven, done before you leave the desk) - -- [ ] Rebuild the ISO with archsetup baked in: the 2026-08-02 ISO predates - the microcode vendor-detection fix (archsetup, 2026-08-08) and was built - without ARCHSETUP_DIR at all. - #+begin_src sh - cd ~/code/archangel && sudo ARCHSETUP_DIR="$HOME/code/archsetup" ./build.sh - #+end_src - Two traps in that one line, and either alone silently produces a bare ISO - with archsetup absent (archangel, 2026-08-20). The =VAR=value= form is - required because sudo's =env_reset= discards an exported variable. And - =$HOME= is required because zsh does not expand a tilde on the right-hand - side of an assignment — the earlier =ARCHSETUP_DIR=~/code/archsetup= here - passed the literal string. build.sh now warns and reports baked/not-baked - in its closing summary, so the failure is visible rather than silent. -- [ ] build.sh fixes before the final rebuild (archangel repo): - - rsync exclude for =.ai= (keeps =archsetup/.ai/private-design/= — the - credential audit — off the portable USB stick). - - copy =installer/velox-*.conf= to =airootfs/root/= so the machine profile - is on the ISO at =/root/velox-zfs.conf=. -- [ ] Verify the ISO carries: =/code/archsetup= (with =install_cpu_microcode=), - =/root/velox-zfs.conf=, no =.ai/private-design=. Loop-mount or unsquashfs - spot-check. -- [X] USB ready (done 2026-08-13 15:25): the new ISO was copied to the Ventoy - drive, sha256-verified against the source, and the 2026-04-09 + 2026-06-16 - archangel ISOs removed. Boot the stick and pick - =archangel-2026-08-13-vmlinuz-6.18.43-lts-x86_64.iso= from the Ventoy menu. - -* Phase 1 — UEFI setup on velox (BIOS screen, before any boot) - -- [ ] Disable Secure Boot. Mandatory — the ZFS kernel modules are unsigned; - the new board ships with it enforced by factory default. -- [ ] Set the system clock. The board swap reset the RTC to 2025-01-01; - a wrong clock breaks TLS and pacman signature checks in the live env. - Rough accuracy is fine — NTP tightens it once networked. -- [ ] While you're in setup: check boot-order UI shows the USB. - -* Phase 2 — Salvage pass (live ISO, BEFORE running the installer) — REQUIRED - -NOT optional insurance. Verified 2026-08-13: velox's newest truenas backup is -DAILY.0 = 2026-07-06 — five weeks stale. The backup timer on velox broke -around Jul 6 (truenas itself only went dark Jul 24, and it's back now; ratio -and mybitch backed up today). Everything since Jul 6 exists only on the old -SSD — including =wolf.conf.gpg= (created Jul 29), which is therefore in NO -backup at all. This pass also keeps the repair fallback alive until -partitioning starts. - -- [ ] Network up (=nmtui= or ethernet), then confirm clock: =timedatectl=. -- [ ] Import the old pool read-only and unlock: - #+begin_src sh - zpool import -N -o readonly=on -R /mnt zroot - zfs load-key zroot # passphrase prompt - zfs mount zroot/ROOT/default - zfs mount -a 2>/dev/null # home datasets etc.; ignore failures - #+end_src -- [ ] Push a full fresh backup to truenas over the LAN — mirror the layout - the backup job uses (etc + home), into a clearly-named one-off dir: - #+begin_src sh - rsync -aHAX --info=progress2 /mnt/etc /mnt/home \ - truenas:/mnt/vault/backups/velox/pre-reinstall-2026-08-13/ - #+end_src - (=/usr= is in the regular backups but is all reinstallable — skip unless - paranoid. The 96GB-RAM board will not be the bottleneck; the LAN is.) -- [ ] Spot-check the copy landed: =wolf.conf.gpg=, =.ssh=, =.gnupg=, newest - files in =~/documents= and =~/downloads=. -- [ ] Check for uncommitted repo work and either push or note it: - =~/.emacs.d= (known: the auto-dim-other-buffers.el unresolved merge), - =~/.dotfiles=, anything under =~/code=. -- [ ] Export cleanly: =cd /; zfs unmount -a; zpool export zroot=. - -* Phase 3 — Install (the actual DR test) - -- [ ] Review the profile, then run the installer: - #+begin_src sh - less /root/velox-zfs.conf # FILESYSTEM=zfs, HOSTNAME=velox, single nvme - archangel --config-file /root/velox-zfs.conf - #+end_src - Note: the profile's ZFS_PASSPHRASE / ROOT_PASSWORD are the =welcome= - placeholders — fine for install; both change post-install (=zfs change-key - zroot= for the pool, =passwd= for root). -- [ ] Record every rough edge as a DR-test finding — that's the point of - running it this way. Anything that needs a manual nudge gets a todo entry - in archangel or archsetup afterward. -- [ ] Reboot into ZBM → boot the new environment. - -* Phase 4 — archsetup (first boot of the installed system) - -- [ ] Log in as root, network up, then verify the clock synced. -- [ ] Get archsetup — two paths, test the offline one since this is a DR - drill (the online curl path is the everyday alternative): - #+begin_src sh - # offline: mount the install USB and copy the baked tree - mount /dev/disk/by-label/ARCHANGEL* /mnt 2>/dev/null || mount /dev/sdX1 /mnt - cp -r /mnt/code/archsetup /root/archsetup && cd /root/archsetup - ./archsetup - #+end_src -- [ ] Expected on the new board: =install_cpu_microcode= detects - AuthenticAMD and installs amd-ucode (verified 2026-08-13, 7/7 tests). - Podman socket, camera udev rule, tlp radio state, ZFS /tmp mask are all - in the installer now — none need manual application afterward. -- [ ] archsetup clones + stows dotfiles. The velox host tier has no Intel - assumptions (swept 2026-08-13); maint's capability probe runtime-detects - amd-pstate. - -* Phase 5 — Post-install restore + verification - -- [ ] Restore from backup (credentials, ssh keys, gpg, user data). The - secrets-bundle-in-ISO design is not built yet — manual restore is the - known gap, not a test failure. -- [ ] WireGuard: decrypt + re-place =wolf.conf.gpg= at =~/.config/wireguard/=; - re-import the NM profile (autoconnect off, as before). -- [ ] Change the placeholder passwords: =passwd=, =zfs change-key zroot=. -- [ ] PSR workaround — REQUIRED on this board. The Ryzen AI 300 has a known - idle instability (Panel Self Refresh hangs/reboots the machine; hit during - the live session 2026-08-13). Add =amdgpu.dcdebugmask=0x610= to the - installed system's kernel command line — velox boots via ZBM, so set it on - the pool: =zfs set org.zfsbootmenu:commandline="... amdgpu.dcdebugmask=0x610" zroot/ROOT/default= - (keep the existing args; append). Revisit after a BIOS update ≥3.05 or a - kernel that fixes PSR on Strix Point — track via the Framework issue - tracker (SoftwareFirmwareIssueTracker #110). -- [ ] New-hardware spot-checks: - - =journalctl -k | grep -i microcode= — amd-ucode applied. - - =cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver= — expect - amd-pstate(-epp). - - wifi + bluetooth up (new board radios), touchpad behavior, camera. - - =glxinfo -B= / =vulkaninfo --summary= — Radeon 890M on RADV. -- [ ] Fresh clones automatically carry the post-purge rewritten git history — - closes the clone-reconcile rider from 2026-08-11 without action. -- [ ] Fix and verify the backup timer on the fresh install — it was silently - broken since ~Jul 6. After the first manual run succeeds, confirm a new - DAILY.0 appears under =truenas:/mnt/vault/backups/velox/=. Diagnose why it - broke (timer unit dead? mount failure? credential?) if the old journal - survives in the salvage copy. -- [ ] Update the machine-identity memory: velox is now AMD (amd-pstate), - both daily drivers AMD. Fix the stale =intel_pstate= comment in - =airplane-mode= line 6 while at it (cosmetic). -- [ ] File every DR-test finding in the owning project's todo. - -* Timing - -Today is Thursday; the flight is Sunday. Target: Phases 0–4 tonight or -Friday, leaving Saturday as pure buffer. If the install stalls past Friday -evening, cut losses to the manual-install floor. |
