diff options
Diffstat (limited to 'docs/specs')
| -rw-r--r-- | docs/specs/2026-08-25-topgrade-guarded-upgrade-spec.org | 213 |
1 files changed, 213 insertions, 0 deletions
diff --git a/docs/specs/2026-08-25-topgrade-guarded-upgrade-spec.org b/docs/specs/2026-08-25-topgrade-guarded-upgrade-spec.org new file mode 100644 index 0000000..d9ec8d4 --- /dev/null +++ b/docs/specs/2026-08-25-topgrade-guarded-upgrade-spec.org @@ -0,0 +1,213 @@ +#+TITLE: Guarded-Upgrade Completion — keeping topgrade freshness honest +#+AUTHOR: Craig Jennings +#+DATE: 2026-08-25 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Guarded-upgrade completion +:PROPERTIES: +:ID: 81cdfd72-db96-43d3-aa03-779878c99f3e +:END: +- [2026-08-25 Tue @ 18:45 -0600] decisions closed 7/7. The kernel decision reversed on the velox DKMS failure chain: held on every everyday run, landed only in the dedicated session behind a DKMS/initramfs/snapshot gate. +- [2026-08-25 Tue @ 18:30 -0600] redirected: the everyday path is a live split upgrade (apply everything the guard would not block, defer the rest); the boot-time oneshot becomes the completion step for the deferred set. Decided while running exactly that by hand on ratio. +- [2026-08-25 Tue @ 06:39:42 -0600] drafted. Grounded in a live read of the maint engine, the pacman hooks, and the boot path on velox, not memory. The topgrade-freshness diagnosis that motivates it is in this session's log. + +* Metadata + +| Status | draft | +|----------+-------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+-------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+-------------------------------------------------------------| +| Related | maint =topgrade_age= metric; =hypr-live-update-guard= hook | + +* Summary + +The waybar maintenance module shows topgrade freshness as permanently stale. The cause is a real one: on a machine running Hyprland, a full =topgrade= almost never exits 0, because its system step upgrades GPU/compositor libraries that the =hypr-live-update-guard= pacman hook correctly refuses to swap under a live session. The freshness stamp is gated on topgrade's exit code, so a correct, protective refusal reads as "you never run updates." This spec designs a safe path to actually complete a guarded upgrade, and makes that completion record the freshness stamp, so the metric tracks the true state of the system. + +* Problem / Context + +The metric reads one cache key, =topgrade_run= (=~/.local/state/maint/topgrade_run.json=). Absent, the probe (=maint/src/maint/probes/updates.py:114=) returns WARN, "no topgrade run recorded". Two writers stamp it: the =topgrade= PATH wrapper (=~/.dotfiles/hyprland/.local/bin/topgrade=) on =rc -eq 0=, and the panel's TOPGRADE lever (=doctor.py=), which returns before the stamp on any non-zero exit. The read path is sound (a sandboxed =maint stamp topgrade= writes the file and =maint status= then reads freshness 0); the file is simply never written. + +It is never written because topgrade rarely exits 0 on this machine, and the reason is specific rather than flaky. =/etc/pacman.d/hooks/10-hypr-live-update-guard.hook= is a =PreTransaction=/=AbortOnFail= hook that, when Hyprland is running and an upgrade changes the on-disk version of a GPU/compositor library, prints a BLOCKED banner and exits 1 — aborting the whole transaction before any file is swapped. Its trigger set is =mesa=, =mesa-*=, =wayland=, =libdrm=, =libglvnd=, =hyprland=, =aquamarine=, =hyprutils=, =hyprgraphics=, =vulkan-radeon=, =vulkan-intel=, =vulkan-mesa-layers=, =nvidia-utils=, =lib32-nvidia-utils=, =xorg-xwayland=. The guard exists for a proven failure: replacing those libraries under a live compositor makes the next GPU call hit a now-deleted mapping and SIGABRT, taking every Wayland client down (hit on ratio 2026-06-07). + +So when any of those libraries has an update pending — a frequent event — topgrade's =system= step (it runs =yay=) aborts non-zero, topgrade returns non-zero, and neither writer stamps. The observed case: on 2026-08-24 topgrade ran at 17:49, hit the guard on =mesa= (26.1.7 → 26.2.1), and failed; the upgrade was then finished by hand with the guard's sentinel override, entirely outside the wrapper, so nothing stamped. The metric has read stale ever since. + +Two framings of the fix are in tension, and choosing between them is the spec's central decision. Either the metric means "how recently did you run the sweep" (recency), so the stamp should decouple from topgrade's exit; or it means "is the system up to date" (state), so staying stale while a guarded upgrade is deferred is *correct* and the only real defect is that safely completing that upgrade doesn't stamp. This spec takes the state framing (see Decisions). + +* Goals and Non-Goals + +** Goals +- A safe, low-friction way to apply a guarded (GPU/compositor-library) upgrade, with Hyprland not live at swap time. +- That completion records the =topgrade_run= freshness stamp, so the metric clears when the system is genuinely current. +- A boot-time upgrade path that can never lock the machine out of its session, however it fails. +- The installer owns the durable pieces so a rebuilt machine has them without hand-setup. + +** Non-Goals +- Weakening or bypassing the =hypr-live-update-guard= hook. It stays exactly as strict; this builds *around* it, not through it. +- Making the full topgrade ecosystem sweep (git repos, vim, npm, ...) run at boot. Those never need a stopped compositor and are out of the boot path. +- Changing how the kernel hazard is *guarded*. The hook stays silent on kernels; the split script holds them back on a live run as a second, separately-reasoned list (see Design), which is a deferral policy rather than a guard. +- A general offline-update system for all of pacman. Scope is the guarded-library case. + +** Scope tiers +- v1: the split-upgrade script (live: apply the non-blocked remainder, defer the rest, run the ecosystem sweep with the system step off, report the deferred set); maint's UPDATE/TOPGRADE levers route through it; an "apply on reboot" affordance that installs the held kernel live and arms the boot-time oneshot for the GPU/compositor set. +- Out of scope: full-sweep-at-boot; touching the guard's policy. +- vNext: none open — the kernel deferral that was vNext is now part of v1's held set. + +* Design + +The shape follows one principle: the only part of topgrade that needs a stopped compositor is its =system= step when a guarded library is pending. Everything else runs fine live and rarely fails. So the safe path is small and targeted — apply the guarded system upgrade with Hyprland down, once, and stamp it — while the ordinary full sweep stays a normal live =topgrade= run. + +Three pieces, at two altitudes — but the everyday gesture is not a reboot. It is a normal live update that simply leaves the dangerous few behind. + +*The split script.* A pacman =PreTransaction= hook can only abort or allow the transaction it is handed; it cannot drop targets from it. So "upgrade everything except the guarded set" cannot live in the hook — it lives one layer up, in a script the panel calls. On a live run the script: refreshes the sync db and reads the pending set (=checkupdates=); computes the *blocked set* = the guard's own trigger list (read from the installed hook's =Target= lines, so there is one source of truth, and version-aware the way the guard is — a same-version reinstall is not a swap) plus the *kernel set* (every installed kernel with its =-headers=, always as a set; held on every everyday run because a failed DKMS rebuild on velox's ZFS root leaves the machine unbootable — see the kernel decision); clears the news hook (=informant read=) where installed; runs =pacman -Syu --noconfirm --ignore=<blocked set>=; runs the AUR-only remainder (=yay -Sua --noconfirm=, AUR packages pinning a guarded version hold themselves back); then runs =topgrade --disable system,git_repos -y= so the other ecosystems still get their sweep and topgrade can actually exit 0. It writes the deferred set to a state file the panel reads, and exits 0 when the live part succeeded, whatever was deferred. The guard hook stays installed as the backstop for a bare =pacman -Syu= typed at a shell; on the driven path it never fires. Proof of concept: this exact sequence, run by hand on ratio on 2026-08-25 while Hyprland was live, resolved 724 of 730 pending packages (Emacs 31.1 among them) with the six guard hits deferred — after one unrelated fix, an orphaned =qemu-block-gluster= that had been dropped from the repo. + +*For the user.* UPDATE and TOPGRADE on the panel run the split script; they succeed, and the panel shows "N deferred" when the script held anything back. Landing the deferred set is a dedicated session, chosen on purpose, run in the foreground from the panel's action or =guarded-upgrade --complete= in a terminal: first the kernel set, live, with the desktop still up; then the gate — every DKMS module built for the new kernel, a fresh initramfs, and on a ZFS root a pre-pacman snapshot to fall back on. If the gate fails the script stops there, names what failed, and does not reboot; the machine keeps running on the old kernel and the desktop is available for the fix. If it passes, the script arms a persistent flag for the GPU/compositor set and offers to reboot (or, from a TTY with no compositor, applies that set directly). On the next boot, before the autologin shell starts Hyprland, the deferred guarded upgrade runs in the console — the guard passes freely because nothing is live — the stamp is written, the flag is cleared, and boot continues into the session. No second reboot: the libraries are already current before anything maps them. If anything goes wrong, the machine still boots into Hyprland and the panel still shows the pending upgrade, so you are never worse off than before arming. + +*For the implementer.* A persistent arm flag (a file on a non-tmpfs path, e.g. =/var/lib/archsetup/apply-upgrade-on-boot=, so it survives the reboot the =/run= guard sentinel cannot). A system oneshot, =archsetup-boot-upgrade.service=, =ConditionPathExists= on the flag, ordered =Before=getty@tty1.service= so it completes before autologin execs Hyprland — this ordering is mandatory, because a parallel run would let Hyprland start mid-swap and reintroduce the exact crash the guard prevents. The unit is bounded (=TimeoutStartSec=) and best-effort: its failure or timeout must not fail any target the session needs, so boot proceeds past it regardless. Its =ExecStart= runs, as the user: =informant read= (clear the news hook that would otherwise abort the transaction), then =topgrade --only system= (or the equivalent =yay -Syu=), then =maint stamp topgrade= on success, then removes the flag unconditionally (a one-shot arm — a failed attempt disarms rather than retrying every boot). =sudo= works unattended (=%cjennings NOPASSWD: ALL=), so no password prompt wedges it. + +The stamp also needs to happen when the upgrade is completed by other safe means — the by-hand sentinel-override path, or a =maint= command that does the same thing. The cleanest single home for the stamp is a small =maint apply-upgrade= (or a flag in the existing lever) that performs the guarded system upgrade and stamps on success, which both the boot unit and an interactive TTY run call. That keeps one code path that "completes a guarded upgrade and records it," rather than three writers that can drift. + +* Alternatives Considered + +** A. Decouple the stamp from topgrade's exit code (stamp on any real run) +- Good, because it is a one-line change to the wrapper and needs no boot machinery. +- Bad, because it throws away honest signal: a topgrade that was blocked from applying a real upgrade would read as "fresh," so the metric stops meaning "up to date." On this machine the blocked case is the common case, so the metric would be fresh precisely when an upgrade is outstanding. +- Neutral, because the failed steps still surface elsewhere (pending-updates count), so freshness would become redundant rather than wrong. + +** B. Run the full topgrade live with the guard overridden, then reboot +- Good, because it needs no new unit — arm the sentinel, run, reboot. +- Bad, because the dangerous window is the whole rest of the run: mesa swaps early, then topgrade spends minutes on other ecosystems while the live compositor is one new GL context (a new window, the wallpaper daemon) away from SIGABRT. topgrade's own reboot-at-end is that window, not a fix for it. +- Neutral, because it would stamp naturally on success — if it survived. + +** C. Manual TTY ritual only (log out, run topgrade at the console, reboot), plus stamp +- Good, because it is the safest path and needs almost no code — just make the completion stamp. +- Bad, because it is all manual, every guarded-upgrade day; the friction is why it won't happen consistently, which is how the metric got stale in the first place. +- Neutral, because it is exactly what the boot unit automates, so it is really "v1 minus the automation." + +** D. Boot-time armed oneshot, arch-only (this spec) +- Good, because the risky swap happens with nothing live, the run is one bounded transaction with a tiny prompt surface, it stamps on success, and a failure degrades to "boots normally, try again." +- Bad, because it puts a unit on the boot critical path, which must be bounded and non-fatal with care, and it is the most to build. +- Neutral, because it composes with C: the same =maint apply-upgrade= path serves both an interactive TTY run and the boot unit. + +** E. Split the live run: apply the non-blocked remainder now, defer the rest (this spec's everyday path) +- Good, because it is what a careful operator does by hand anyway — and did, on ratio, the day this was decided. The live run succeeds on the common day, topgrade exits 0, the AUR and every other ecosystem stay current, and the guard's abort becomes the rare path rather than the default. +- Bad, because Arch calls any =--ignore= run a partial upgrade. In practice pacman still enforces declared dependencies, so anything needing the newer mesa fails resolution instead of installing broken; the residual exposure is a package with an *unversioned* dependency built against a new ABI, which for mesa/wayland/libdrm is rare. Named, accepted. +- Bad, because a deferred set nobody surfaces is a set that silently never lands — the same trap as the freshness stamp, one layer down. So the script must record the deferred set durably and the panel must show it; this is why D stays in the design as the completion step rather than being replaced. +- Neutral, because it does not change the guard at all; it changes who decides the transaction's contents. + +* Decisions [7/7] + +** DONE Metric means state, not recency +CLOSED: [2026-08-25 Tue 18:45] +- Owner / by-when: Craig / 2026-08-25 +- Context: the stamp gate can mean "ran the sweep" or "system is current." The whole fix differs by which. +- Decision: We will keep the state meaning. Freshness stays stale while a guarded upgrade is genuinely un-applied, and the fix is to make *safe completion* stamp — not to loosen the gate. +- Consequences: easier — the metric stays trustworthy as an is-current signal, and Alternative A is off the table. Harder — completion now needs a real safe path (the rest of this spec) rather than a one-line wrapper change. + +** DONE Everyday mechanism is the split live run (Alternative E); the boot oneshot (D) completes the deferred set +CLOSED: [2026-08-25 Tue 18:30] +- Owner / by-when: Craig / 2026-08-25 +- Context: the first draft made D the primary gesture, which means every guarded-library day is a reboot day. Craig's read while watching the ratio run: when the guard would trip, the rational move is to upgrade everything *except* the guarded and kernel items, then run the rest of topgrade without the yay piece — and that logic should be a script we can keep editing, not something baked into the panel. +- Decision: I will build E as the path UPDATE and TOPGRADE always take on a live session, and keep D as the way the deferred set lands (arm + reboot). C remains the manual fallback through the same script from a TTY (no compositor → nothing blocked → a full run). B stays rejected on the live-swap risk. +- Consequences: easier — the common day is one live run that succeeds; reboots are reserved for the days the deferred set is non-empty, and even then the machine keeps working until the reboot is convenient. Harder — two lists to maintain (the guard's, read from the hook; the kernel list, owned by the script), a state file the panel must render, and the partial-upgrade caveat above to keep an eye on. + +** DONE The script lives in archsetup beside the guard, and maint calls it +CLOSED: [2026-08-25 Tue 18:45] +- Owner / by-when: Craig / 2026-08-25 +- Context: the panel (dotfiles =maint=) and the guard (archsetup =scripts/hypr-live-update-guard=, installed to =/usr/local/bin=) live in different repos, and maint already carries its own copy of the trigger list as =[updates] guard_patterns= in the thresholds TOML. +- Decision: ship the split script in archsetup next to the guard, installed by the same installer step, reading the blocked list from the installed hook so the guard and the script can never disagree. maint's UPDATE/TOPGRADE levers change their =argv= to the script; the TOML patterns stay as the panel's *display-side* mirror (the badge that says a run will defer) and gain a test asserting they match the hook. +- Consequences: easier — one owner for "which libraries are dangerous," and a rebuilt machine gets the script with the guard. Harder — a cross-repo change (archsetup ships it, dotfiles wires it), so the rollout is two commits, archsetup first. + +** DONE What a split run stamps +CLOSED: [2026-08-25 Tue 18:45] +- Owner / by-when: Craig / 2026-08-25 +- Context: under the state framing, a run that deferred six packages left the system *not* current, yet the sweep ran and every other ecosystem is fresh. +- Decision: the script stamps =topgrade_run= only when the deferred set is empty. When it is non-empty it writes the deferred set to its own cache key, and the panel renders that as its own state ("6 deferred — apply on reboot") rather than as stale freshness. The boot oneshot stamps when it completes the deferred set. Freshness keeps meaning "current"; the deferred badge carries the other half. +- Consequences: easier — no signal is thrown away, and the reboot nag has a precise count behind it. Harder — one more cache key and one more probe in maint. + +** DONE Kernel set is held on every everyday run and lands only in the dedicated session, gated on the DKMS result +CLOSED: [2026-08-25 Tue 18:45] +- Owner / by-when: Craig / 2026-08-25 +- Context: I first wrote this as "install the kernel live at apply-on-reboot," on the reasoning that a kernel swap crashes nothing and the modules-vanish window ends with the reboot. Craig asked what happens on velox when the DKMS rebuild fails, and the answer changed the decision. Velox is an encrypted ZFS root with =/boot= inside the root dataset, one kernel (=linux-lts=), and =zfs-dkms=. On a kernel upgrade the DKMS build runs PostTransaction, after the kernel is swapped and the old modules are deleted, so nothing can abort; a failed build leaves a new kernel beside an initramfs built for the old one, whose =zfs.ko= won't load, and the next boot can't import the pool. It is survivable — ZFSBootMenu can boot the pre-pacman snapshot, which holds the old kernel, initramfs, and modules — but it is a recovery session, not an update. Ratio (btrfs root, two kernels, zfs only for a data pool) is exposed only at the pool. The realistic triggers are a kernel major outrunning OpenZFS's supported range, a kernel upgraded without its headers, a toolchain regression, or a full disk. +- Decision: the script holds the kernel set — every installed kernel with its =-headers=, moved as a set, never one without the other — on every everyday run, on both machines, so there is one rule rather than a per-host exception. The kernel set lands only in the dedicated session, live, while a working desktop exists for diagnosing, and the script gates what follows on the result: =dkms status= reports every DKMS module installed for the new kernel version, the initramfs is newer than the kernel image, and on a ZFS root a pre-pacman snapshot exists. A failed gate stops with the failure named and never reboots. The GPU/compositor set follows only after the gate passes — armed for the boot oneshot, or applied from a TTY. Kernels stay off the guard's list (the hook would block a TTY kernel upgrade for no reason). "Install the kernel live at apply-on-reboot" is withdrawn. +- Consequences: easier — an everyday UPDATE can never put velox into the unbootable state, and the day the kernel moves is one Craig chose, sitting at the machine, expecting to handle issues. Harder — the kernel deferral is now standing, so the dedicated session has to happen on a cadence (security fixes ride the kernel), and the panel's deferred count carries a kernel most days; the gate is one more script to test, with fakes for =dkms status= and the image timestamps. + +** DONE Boot run applies exactly the deferred GPU/compositor set, nothing else +CLOSED: [2026-08-25 Tue 18:45] +- Owner / by-when: Craig / 2026-08-25 +- Context: the only packages that need a stopped compositor are the guard's trigger set; the rest run fine live and are what usually fail. The first draft phrased this as =topgrade --only system=; with the split script that wording is stale, and with the kernel decision above the kernel is not part of what boot applies either. +- Decision: the boot oneshot runs the script's =--complete= form scoped to the deferred GPU/compositor set: one pacman transaction, no ecosystem sweep, no kernel. The full topgrade sweep stays a normal live run through the everyday path. +- Consequences: easier — the boot path is fast, has a tiny interactive-prompt surface, and rarely fails. Harder — freshness after a boot run reflects the guarded set specifically, which is what the stamp decision above already accounts for. + +** DONE Arm flag lives on a persistent path and is one-shot +CLOSED: [2026-08-25 Tue 18:45] +- Owner / by-when: Craig / 2026-08-25 +- Context: the guard's =/run= sentinel is tmpfs and cleared on reboot, so it cannot carry an intent across the reboot. A boot that retries forever on failure is its own outage. +- Decision: We will use a persistent flag (=/var/lib/archsetup/=) that the boot unit removes unconditionally at the end of its attempt — success or failure disarms. +- Consequences: easier — the intent survives exactly one reboot and a failed attempt never wedges subsequent boots. Harder — a failed attempt needs re-arming, which is correct (a human decides to try again) but is a manual step. + +* Implementation phases + +** Phase 1 — The split-upgrade script (archsetup) +=scripts/guarded-upgrade= (name open), installed to =/usr/local/bin= by the step that installs the guard. Behaviour as in Design: pending set → blocked set (hook =Target= lines, version-aware) ∪ held-kernel set when a compositor is live → =informant read= if present → =pacman -Syu --noconfirm --ignore=…= → =yay -Sua --noconfirm= → =topgrade --disable system,git_repos -y= → deferred set written to a state file → stamp only when nothing was deferred → exit 0 on a successful live part. The kernel set is derived from what is installed (every =linux*= kernel package and its =-headers=), never a hardcoded pair, and is always held or applied whole. Flags: =--dry-run= (print the plan and the deferred set, change nothing), =--no-topgrade=, =--no-aur=, =--complete= (the dedicated-session form: apply the kernel set live, run the gate, then arm the GPU/compositor set or, with no compositor live, apply it directly; stamp when the deferred set is empty). The gate is its own small script, =kernel-modules-check=: for each kernel under =/usr/lib/modules=, =dkms status= reports every registered module =installed= for it, and its initramfs is newer than its =vmlinuz=; on a ZFS root, a =pre-pacman_= snapshot of the root dataset exists. It exits non-zero with the failing item named, and =--complete= refuses to arm or reboot on that exit. Usable from a TTY at once. Tests (pytest beside the guard's): blocked-set computation against a fixture hook and version map; the kernel set is derived from the installed kernels and held whole on every everyday run; the =--ignore= list is exactly blocked ∪ kernel set; the state file round-trips; stamps only on an empty deferred set; =--dry-run= is IO-free; the gate passes and fails on fake =dkms status= output, image timestamps, and snapshot listings, and =--complete= never reaches the arm step on a failed gate. + +** Phase 2 — Wire maint to it (dotfiles) +UPDATE and TOPGRADE levers change their =argv= to the script; the press-again-to-force sentinel wrap goes away (the driven path never trips the guard). A new probe reads the deferred-set state file and the panel renders "N deferred — apply on reboot" as its own row. A test asserts the TOML =guard_patterns= equal the installed hook's =Target= list. Tests under the maint fake harness. + +** Phase 3 — "Apply on reboot" and the boot-time unit (archsetup + maint) +The panel action installs the held-kernel set live, writes the persistent arm flag, and offers to reboot. =archsetup-boot-upgrade.service=, installed by the installer: =ConditionPathExists= the flag, =Before=getty@tty1.service=, =TimeoutStartSec= bounded, non-fatal to every session target, =ExecStart= runs the script's =--complete= form as the user and removes the flag unconditionally. Installer step + unit file + the =/var/lib/archsetup/= flag directory. Tests in =tests/installer-steps/= for the install step; the arm action's tests in maint; a documented manual boot test (defer, arm, reboot, observe) in =todo.org= under Manual testing and validation. + +** Phase 4 — Docs, rollout, and both daily drivers +Document the flow (arm → reboot → console upgrade → session). Roll the unit to velox and ratio (installer already covers a rebuild; existing machines need the one-time install). Confirm the ratio path matches. + +* Acceptance criteria +- [ ] With a guarded library pending and Hyprland live, UPDATE applies everything else, exits 0, and the panel shows the exact deferred set; the guard hook does not fire. +- [ ] The same run with no compositor live (a TTY) applies everything but the kernel set and stamps only if nothing was deferred. +- [ ] A =--complete= run whose DKMS build fails stops before arming or rebooting, names the failure, and leaves the machine running on the old kernel; on velox the pre-pacman snapshot it required is bootable from ZFSBootMenu. +- [ ] With a guarded library pending, arming and rebooting applies it in the console before Hyprland starts, and =maint status= then reads a fresh =topgrade_age=. +- [ ] A boot-upgrade failure (a failed step, a timeout, an aborted transaction) never blocks the session: the machine boots into Hyprland, the flag is cleared, and the panel still shows the pending work. +- [ ] Unread Arch news does not wedge the boot run (=informant read= precedes the transaction). +- [ ] A guarded upgrade completed from a TTY via the Phase-1 path stamps freshness identically to the boot unit. +- [ ] The =hypr-live-update-guard= hook is unchanged and still blocks a live guarded swap. + +* Readiness dimensions +Answer each, or write "N/A because…". +- Data model & ownership: the arm flag (=/var/lib/archsetup/=, installer-owned) and the =topgrade_run= cache key (maint-owned). No user-authored data. +- Errors, empty states & failure: the boot unit is best-effort and self-disarming; every failure path lands in "boot normally, metric stays stale, re-arm to retry." Named, non-silent. +- Security & privacy: relies on the existing =%cjennings NOPASSWD: ALL=; the unit runs the upgrade as the user via sudo, adds no new privilege. Note the NOPASSWD breadth as a pre-existing fact, not introduced here. +- Observability: the boot run's output is on the console; its systemd unit status and journal record success/failure; the panel reflects the cleared or still-pending state after boot. +- Performance & scale: one pacman/yay transaction at boot; bounded by =TimeoutStartSec=. Negligible boot-time cost when the flag is absent (=ConditionPathExists= skips the unit). +- Reuse & lost opportunities: reuses the guard's trigger list by reading the installed hook (single source of truth for "which libs are dangerous"), =informant=, =maint stamp=, =checkupdates=, and topgrade's own step switches. The one duplicate that exists today — maint's TOML =guard_patterns= — is kept as a display mirror and pinned to the hook by a test rather than removed. +- Architecture fit & weak points: integration points are the pacman hook set, getty autologin ordering, and the maint cache. Weak point: the =Before=getty@tty1= ordering is load-bearing for safety; a parallel run reintroduces the live-swap crash. Mitigated by making the ordering explicit and tested-by-inspection. +- Config surface: the arm flag path and the timeout. Defaults safe (absent flag = no-op). +- Documentation plan: a short "reboot to apply guarded upgrades" note in the maint docs; the installer step self-documents in-comment. +- Dev tooling: installer-step pytest for Phase 2; maint unit tests for Phases 1 and 3; a manual boot test in =todo.org=. +- Rollout, compatibility & rollback: additive; removing the unit and flag reverts fully. Existing machines need a one-time install; a rebuild gets it from the installer. Rollback leaves the guard and manual TTY path intact. +- External APIs & deps: topgrade =--only system=, =informant read=, =yay=, =maint stamp= — all verified present on velox this session. No external service. + +* Risks, Rabbit Holes, and Drawbacks +- Boot critical path: the unit sits ahead of autologin, so a hang would delay boot. Mitigated by =TimeoutStartSec= and non-fatal wiring; worst case is a bounded delay, then a normal session. +- Interactive prompts under no stdin: =yay=/pacman can still prompt (provider choice, replace, AUR review) even with =assume_yes=. The =--only system= scope and =--noconfirm=-style flags shrink this to near zero, but a prompt with no stdin fails the run (benign) — needs a genuinely non-interactive invocation, verified in Phase 2. +- Partial ecosystem state: N/A for the GPU hazard — each pacman run is one atomic transaction, so there is no half-swapped library. The =--ignore= run is a partial upgrade in Arch's sense; pacman's dependency resolution is the safety net, and the residual unversioned-ABI exposure is accepted in Alternative E. +- Orphans that block resolution: a package dropped from the repo but still pinning an old version (ratio's =qemu-block-gluster= on 2026-08-25) fails the whole transaction. The script should detect the "could not satisfy dependencies" case, name the foreign package, and stop with the remedy — never =-Rdd= on its own. +- The kernel on a DKMS ZFS root: a failed =zfs-dkms= build after the kernel swap cannot be aborted (the DKMS hooks are PostTransaction) and leaves velox unbootable on the new kernel. Mitigated by holding the kernel set on every everyday run, landing it only in the dedicated session behind the gate, and by the standing fallback: =/boot= lives in the root dataset, the =05-zfs-snapshot= hook snapshots it before every transaction, and ZFSBootMenu can boot that snapshot. The pacman cache also keeps the previous kernel and =zfs-dkms= for a downgrade. Ratio's exposure is its data pool only (btrfs root, two kernels). +- Standing kernel deferral: because the everyday run never moves the kernel, the dedicated session has to happen on a cadence or kernel security fixes sit unapplied. The panel's deferred row is the reminder; a stale-kernel age in maint is a possible follow-up. + +* Testing / Verification / Rollout +Phase-1 and Phase-3 logic under the maint fake harness; Phase-2 install under =tests/installer-steps/=. The one thing no unit test can cover — that an armed reboot actually applies the upgrade pre-session and stamps — is a scripted manual test in =todo.org= (arm with a guarded lib pending, reboot, confirm the console run, the fresh metric, and a normal session). Roll to velox first, then ratio. + +* Review and iteration history +** 2026-08-25 Tue @ 18:45 -0600 — Craig Jennings — author +- What: closed all seven decisions. Reversed the kernel decision (hold on every everyday run; land only in the dedicated session, gated on DKMS built, initramfs fresh, snapshot present; withdrew "install live at apply-on-reboot"), reworded the boot-scope decision for the split design, added the =kernel-modules-check= gate to Phase 1 and the acceptance criteria, and wrote the velox failure chain and the ZFSBootMenu fallback into Risks. +- Why: on velox a failed =zfs-dkms= rebuild after a kernel swap is unabortable and unbootable; that belongs in a session I chose, not in an update I expected to touch applications. +- Artifacts: this session's log (velox boot layout verified live: ZBM on the ESP, =/boot= in =zroot/ROOT/default=, one kernel, =zfs-dkms 2.4.4=). +** 2026-08-25 Tue @ 18:30 -0600 — Craig Jennings — author +- What: made the split live run (E) the everyday path and the boot oneshot (D) the completion step; added the script-ownership, stamp-semantics, and kernel-hold decisions; rewrote the phases around the script; added the orphan-blocks-resolution risk. +- Why: watching a 724-of-730 guarded run succeed by hand on ratio made it obvious the guard's abort should be the rare path, and that the logic belongs in an editable script the panel calls rather than in the panel. +- Artifacts: this session's log; the ratio run (=ratio-upgrade.service=, =/var/log/ratio-upgrade.log=). +** 2026-08-25 Tue @ 06:39:42 -0600 — Craig Jennings — author +- What: initial draft. +- Why: the topgrade-freshness metric reads permanently stale because the guard blocks the arch step; designing a safe completion path rather than loosening the gate. +- Artifacts: this session's log; =hypr-live-update-guard= hook; maint =topgrade_age= probe. |