docs: add maintenance console design + interactive prototypes

The design doc replaces the install-health framing in system-monitor-design-ideas.org with a maintenance console: dense metric subpanels behind a persistent selector, a doctor that streams into a results wall, determinate remedies only (guard-armed updates, curation lifecycles for noise, listeners, containers, and orphans), rotary band selectors on the tall subpanels, and a CLI-first build with a four-layer test strategy. Every decision is dated in the doc.

Prototypes A-E5 keep the exploration path: five divergent layouts converging into E5, a fully interactive simulation whose GOOD/BAD snapshots become the build's test fixtures.

todo.org gains the spec-create task, the panel keybinding-family task, and scope notes on the net/bt doctor retrofit.

1 files changed, 527 insertions, 0 deletions

diff --git a/docs/design/maintenance-console-design-ideas.org b/docs/design/maintenance-console-design-ideas.org
new file mode 100644
index 0000000..066c25d
--- /dev/null
+++ b/docs/design/maintenance-console-design-ideas.org
@@ -0,0 +1,527 @@
+#+TITLE: Maintenance Console — Design Ideas
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-07-06
+
+* Direction
+
+A single-host maintenance console (GUI, waybar-launched) that surfaces every
+health metric for this Arch workstation and, where the remediation is safe,
+runs it. It reshapes the earlier install-health/workstation-contract framing
+(=system-monitor-design-ideas.org=) into a maintenance surface with a doctor
+action.
+
+The console is the glanceable, single-host version of the home project's
+=system-health-check.org= workflow (~1000 lines, capability-dispatched across
+ratio/velox/mybitch/truenas). The console owns the routine-maintenance and
+at-a-glance-status slice; the workflow stays as the escalation path for
+forensic investigation and as the only sanctioned way to run updates.
+
+* Scope: hosts (Q2 resolved 2026-07-06)
+
+v1 targets *both* daily drivers — ratio and velox — as first-class hosts, not
+ratio-first-velox-later. Consequences for v1:
+
+- *Capability dispatch is mandatory from day one*, not deferred. The console
+  probes the live host and runs only applicable checks (btrfs RAID1 on ratio vs
+  ZFS-primary on velox; AMD amd-pstate on ratio vs Intel intel_pstate on velox).
+  This mirrors the home workflow's Phase 0 probe.
+- *velox-only metrics ship in v1*: battery health (capacity / cycles) and the
+  unclean-shutdown / suspend-failure rate — both currently GAPs, both driven by
+  velox being the machine that travels and suspends.
+- The ZFS-primary remediation paths (scrub, snapshot retention, pool health) are
+  built and tested, not stubbed.
+
+* Thresholds source of truth (Q3 resolved 2026-07-06)
+
+One machine-readable file, =maintenance-thresholds.toml=, holds every severity
+value (cache-size trigger, disk %, scrub-age warn/crit, snapshot retention
+limits, temp bands, backup-staleness windows, etc.). archsetup owns and ships
+it; both consumers read the *installed* path, so neither reaches into the
+other's repo:
+
+- the maintenance console (dotfiles code) reads it at runtime,
+- the system-health-check workflow reads it instead of hardcoding severity
+  rules in prose.
+
+The workflow's hard-won values migrate into the TOML as the seed content (the
+snapshot MONTHLY limit that bit /home, the 10 GB cache trigger, the scrub-age
+bands). Install path is an implementation sub-question — a stable location both
+consumers reach (e.g. =~/.config/archsetup/maintenance-thresholds.toml=).
+
+* Workflow ownership — move system-health-check into archsetup (proposed 2026-07-06)
+
+Craig's call: the =system-health-check.org= workflow should live in archsetup,
+not the home project. Rationale — the home project is scoped to finances,
+health, and personal matters; system design, execution, and maintenance are
+archsetup's domain. Home only owns the workflow by inherited accident, not by
+fit. Moving it here also collapses the Q3 coupling: the TOML source-of-truth and
+its workflow consumer end up in the same project.
+
+Move scope (home → archsetup), to sequence when Craig gives the go:
+- =system-health-check.org= (the workflow itself)
+- =homelab-inventory/*.org= (ratio/velox/mybitch/truenas capability inventories
+  it cross-references)
+- any home-project references to the workflow (startup, project-workflows index)
+
+Cross-project mechanics: the archsetup side (receiving the files, wiring the
+TOML) is in-scope here; the home side (removing the originals, updating home's
+references) is the home project's scope — handled by a handoff note to home's
+inbox or a home session, not edited blind from here.
+
+* Panel shape
+
+Three regions, driven by the Automation column of the metrics table below:
+
+- *Actionable* (left) — metrics with a lever. Automation = Auto or Confirm.
+  Two doctor actions serve this column: "Clean up" fires every Auto metric
+  unattended; "Review & fix" opens a preview for the Confirm metrics and acts
+  only on approval.
+- *Diagnostic* (right) — read-only telemetry. Automation = None or Human. A
+  state color (green/amber/red) and the value; no button. Red here is the
+  signal to run the workflow.
+- *Updates* (quarantined strip) — Automation = Workflow. Shown as a count with
+  notable packages named; the only affordance is "run the workflow." The panel
+  never applies updates in place.
+
+*Bar glyph* tracks the worst *Diagnostic* state only — not the actionable
+count. A big package cache is boring; a SMART failure is a fire. Actionable
+clutter must not turn the bar red or it trains you to ignore it.
+
+** Doctor = live output wall (Q1 resolved 2026-07-06)
+
+"Clean up" and "Review & fix" are not fire-and-forget buttons. Running a doctor
+opens an *output wall* — one lamp per action, streaming in realtime as each
+runs:
+
+- *amber* while the action is running,
+- *green* on success (with the reclaimed amount / result inline),
+- *red* on failure.
+
+Feedback is always shown (not just when something was reclaimed) — you watch it
+happen. This is the same live-results shape that should back the other doctors
+(net, bluetooth), so every doctor in the system reads the same way. See the
+todo task to retrofit the net + bt doctors to realtime lamp output.
+
+* Automation legend
+
+| Value    | Meaning                                                                                                   |
+|----------+-----------------------------------------------------------------------------------------------------------|
+| Auto     | Doctor "Clean up" button — fire unattended, reversible or harmless                                        |
+|----------+-----------------------------------------------------------------------------------------------------------|
+| Confirm  | Doctor "Review & fix" — preview the change, act only on one click                                         |
+|----------+-----------------------------------------------------------------------------------------------------------|
+| Human    | Panel nudges; a human decides and acts (no button, not the agent)                                         |
+|----------+-----------------------------------------------------------------------------------------------------------|
+| Workflow | vLater — metric shown read-only; no in-panel button. Agent-workflow assistance deferred (decided          |
+|          | 2026-07-07, see below)                                                                                    |
+|----------+-----------------------------------------------------------------------------------------------------------|
+| None     | Diagnostic only — no software remediation exists (hardware, telemetry)                                    |
+|----------+-----------------------------------------------------------------------------------------------------------|
+
+*Workflow buttons removed (decided 2026-07-07, prototyping E5).* The panel
+carries no "run workflow" affordances — not on failed units, unclean boots,
+updates, or CVEs. The metrics stay on the board as read-only telemetry; AI
+assistance via the system-health-check workflow is a vLater feature. v1 ships
+only determinate remedies (fixed, scriptable actions with predictable
+outcomes).
+
+*Determinate remedies adopted (decided 2026-07-07).* Every remedy below is
+Confirm-tier; contextual levers appear only when the metric is off-nominal:
+
+- /Service restarts & enables/: failed units (systemctl restart + reset-failed),
+  fail2ban, cronie, chronyd (+ makestep), tailscaled, snapper timers, zram
+  config re-apply. DNS/NetworkManager delegates to the net panel's doctor
+  (deep-link, not a duplicate repair chain).
+- /Deterministic maintenance ops/: btrfs balance -dusage=50 (unallocated low),
+  reinstall owning packages on pacman -Qkk failures, rsyncshot RUN NOW (result
+  streams to the output well), smartctl -t short self-test, snapshot-retention
+  repair (write sane TIMELINE limits + cleanup — the /home lesson as a
+  one-press fix). btrfs device-error counter reset stays manual: resetting
+  without diagnosis masks a dying drive.
+- /Composite macro/: RECLAIM SPACE on the disk-usage cell — runs every reclaim
+  lever (cache, journal, coredumps, app logs, docker tier-1, snapper cleanup)
+  as one output-wall stream.
+- /Disruptive but determinate/: REBOOT behind arm-to-fire, offered when
+  running kernel != installed.
+- /Still read-only/: temps, throttling, battery, memory/OOM, taint, journal
+  error content, kernel/hw events, listeners, unclean-shutdown rate — physical
+  or investigative; vLater AI territory.
+
+*Updates join the Confirm layer (decided 2026-07-07).* Two levers on the
+updates strip, both behind the live-update guard (mesa/hyprland/wayland
+runtime in the pending set):
+
+- UPDATE — repo + AUR system update.
+- TOPGRADE — full ecosystem run. The panel's wrapper always passes
+  --disable git (topgrade's git step rebase-autostashes ~/code/*/ — never
+  under a live session).
+
+*MEM·PWR: evidence + two levers + expectation tags (decided 2026-07-07).*
+Mostly physics, so the category leans watch-only — with these additions:
+
+- /CPU mode selector/: a free segmented control (PERF · BAL · POWER) writing
+  the EPP hint — set the active mode to anything, not drift-repair against a
+  declared default. amd-pstate on ratio, intel_pstate on velox.
+- /Battery charge limit/ (velox-only, capability-gated): SET 80% writes
+  charge_control_end_threshold — the standard longevity cap. Battery *health*
+  (capacity vs design, cycles) stays watch-only hardware telemetry.
+- /Evidence drill-downs/ (digest idiom): top-5 RAM consumers under memory,
+  recent boots listed clean/unclean under the unclean-rate, and throttle/OOM
+  events with timestamps. Evidence makes the numbers actionable even where no
+  button exists.
+- /KILL on top-memory items/ (revised 2026-07-07 — Craig): arm-to-fire, four
+  guards: the arm shows the exact victim (name + size); SIGTERM not SIGKILL,
+  with the outcome reported to the wall; PID + process name revalidated at
+  fire time so a recycled PID can't be hit; session-critical names (systemd,
+  the compositor, the panel itself) render a disabled key — protected. A
+  SIGKILL escalation for TERM-survivors is vLater.
+- /Expectation-setting, panel-wide/: every leverless cell's sub-line carries an
+  explicit tag ("hardware — watch only", "evidence below"), and each subpanel
+  header shows the split — "N fixable · M watch" — so the user knows their
+  agency before reading a single cell.
+
+*Refresh cadence (decided 2026-07-07).* Four tiers, matching probe cost:
+
+- /Live group, panel open/: temps, memory free + top consumers, throttle
+  state — re-read every ~3 s while their subpanel is visible, gated on
+  panel-open exactly like the audio panel's meters. Stop when hidden.
+- /Fast local tier, panel open/: re-probed every ~30 s while the panel is up;
+  additionally, any metric re-probes immediately after an action that touches
+  it (fire CLEAN → cache re-measured, not assumed).
+- /On open/: the hydration tiers re-run (fast reads first, process probes
+  behind them — sub-second perceived).
+- /Network tier/: checkupdates / arch-audit / AUR / firmware stay on the
+  hourly systemd-timer cache with age shown; refreshed on demand only.
+- /Panel closed/: the waybar glyph is fed by a light background scan every
+  ~30 min (systemd timer writing the state file the glyph reads) — the bar
+  stays honest without the panel running.
+
+*Journal errors get a digest, not a fix (decided 2026-07-07).* No generic
+remedy exists — an error-priority line is a symptom of an arbitrary subsystem —
+so the panel ships four determinate assists instead:
+
+- /Digest/: the cell expands to errors grouped by syslog identifier — count,
+  message snippet, first/last seen, and the exact next command (journalctl -u
+  <unit> -b) when the identifier maps to a unit. Top-10 groups, read-only.
+- /MARK KNOWN with a full lifecycle/: arm-to-fire showing the exact pattern
+  before it stores; marked groups move to a dim KNOWN section (never vanish)
+  with per-row UNMARK; every mark/unmark logs to the results wall; marks carry
+  date + example. Patterns bind to identifier + message, never a whole unit —
+  a muted service's *new* errors still surface.
+- /Two noise layers/: shipped defaults (bluetoothd HFP, pixman, xkbcomp) in
+  the packaged TOML, user marks in a separate user file merged over it (a
+  template sync never eats curation). CLEAR MARKS (arm-to-fire) empties the
+  user layer and re-enables shipped defaults; unmarking a shipped default
+  records a disable flag in the user layer.
+- /OPEN JOURNAL/: launches a terminal running journalctl -p err -b — the same
+  delegation pattern as NET DOCTOR.
+
+Ruled out: auto-restarting units that log errors (error ≠ failed) and
+keyword-driven fix suggestions (vLater AI territory).
+
+*Full-sweep findings — all committed to v1 (decided 2026-07-07).* Every metric
+was audited against the converged checklist (honest label, evidence digest
+where a count hides detail, curation lifecycle where "expected" is config
+knowledge, guarded per-item remedies, cross-links, watch-only tags). Adopted,
+all prototyped:
+
+- /Storage/: disk top-consumers digest (evidence only — no file deletion
+  keys); per-device error rows on RAID1 when counters are nonzero,
+  cross-checked against SMART; SMART sub-line carries the last self-test
+  result. Spec note: a real scrub runs hours — the ring needs a running-%
+  state, not an instant reset.
+- /Packages/: orphan digest (name + size) with per-package REMOVE (armed) and
+  KEEP — the curation lifecycle encoding "intentional, not orphaned" (the
+  rust lesson); batch REMOVE ALL skips kept packages. Per-file pacnew rows
+  tagged safe-delete (reflector-managed) vs needs-merge, MERGE delegating to
+  a terminal diff. CVEs named: package · CVE id · severity. AUR and firmware
+  names spelled out.
+- /systemd/: failed units upgraded from a count-lever to a per-unit roster
+  (name · since · exit code · journalctl hint) with per-row RESTART + RESET;
+  is-system-running names its cause ("degraded — N failed units below");
+  taint letters decoded.
+- /Logs/: coredumps grouped by binary (count · last · coredumpctl hint),
+  cleared with the CLEAR action; kernel/hw events listed when not clean
+  (hardware — watch only).
+- /Services/: docker system df breakdown (images / containers / volumes /
+  build cache with per-type reclaimable); stopped containers upgraded to the
+  full signal/expected curation lifecycle (MARK EXPECTED / UNMARK, shipped
+  default: winvm) with per-container START; cron expected-entries drift
+  roster.
+- /Snapshots/: count split by type — timeline (auto-pruned) · single
+  (manual — escapes timeline cleanup, the pile-up risk) · pre/post — with
+  oldest-single named and DELETE STALE (armed, keeps newest 2) when singles
+  accumulate.
+- /Network minors/: fail2ban shows recent-ban count; NTP shows offset.
+
+Rationale for prototyping everything (Craig): real estate and complexity have
+bitten before — surface those limits in the disposable prototype, not after
+functionality exists behind the UI.
+
+*Vertical compression → rotary band selector (2026-07-07).* First attempt —
+the MEM·PWR three-column layout on Packages/Logs/Services — lost too much row
+detail to third-width truncation (Craig's verdict after use). Replaced by a
+*rotary band selector*: the amplifier input-selector idiom. A machined knob
+(click to cycle) whose needle swings to engraved band labels, one per evidence
+section (ORPHANS · PACNEW · ADVISORIES; SIGNAL · KNOWN NOISE · COREDUMPS ·
+KERNEL/HW; CONTAINERS · DOCKER DISK · CRON & BACKUPS). Each band carries its
+own status lamp (section health at a glance without switching) plus a count;
+the selected band gets a gold underline and the needle. One section renders at
+a time at full width, restoring complete row detail. Deliberately distinct
+from the category tiles, console keys, and the CPU-mode segmented control —
+each selection idiom in the panel now has its own visual voice. MEM·PWR keeps
+its three-column evidence strip (short rows fit fine at third-width).
+
+*Listeners get the same treatment (decided 2026-07-07).* The count becomes
+"unexpected listeners" — evidence digest (process · port · bind address from
+ss -tlnp), expected-list curation with the full MARK EXPECTED / UNMARK /
+CLEAR lifecycle (shipped defaults: sshd, mpd, tailscaled; user marks in the
+user layer), and guarded per-socket remedies: STOP (systemctl stop, armed)
+when a unit owns the socket, KILL (SIGTERM, armed) otherwise. Severity keys
+on unexpected AND public-bind (0.0.0.0/::) — a loopback listener warns, an
+exposed one fails — and when ufw is down the signal header names the exposure
+("ufw down — N public binds exposed"). Stopped containers likewise gained a
+contextual START lever (allowlist: winvm), and the firewall its ENABLE.
+
+*Updates strip border is state-tiered (decided 2026-07-07).* Green when
+nothing pending, amber for ordinary pending/AUR/firmware counts, red when
+CVEs exist or pending exceeds the "a lot" threshold (or the update cache has
+gone stale — staleness window in the TOML). The CVE badge renders only when
+the count is nonzero.
+
+*Guard arms instead of blocking (revised 2026-07-07).* When the guard trips,
+the key arms (red, "press again to run anyway — or apply from a TTY") rather
+than hard-refusing. The user decides; the footgun is acknowledged and
+deliberately handed over. After a system update lands, the panel offers a
+reboot: a REBOOT key (arm-to-fire) appears on the updates strip and the
+reboot-required metric flips.
+
+*No per-ecosystem update metrics.* Topgrade's step set (yay, rustup, cargo,
+pipx, npm/pnpm, gem, go, flatpak, fwupd, tmux/zsh/nvim plugins, git repos) has
+no cheap offline "updates available?" probe — mirroring it means a network
+round-trip per registry at panel-open. Instead: one *topgrade freshness*
+metric (wrapper stamps last-run time; threshold in the TOML) whose remedy is
+the TOPGRADE lever, plus a *firmware updates* count in the updates strip
+(fwupd refreshes metadata on its own timer; the panel reads the cache).
+
+Rationale for the hard lines: system updates are Workflow, never Auto — the
+2026-06-07 Hyprland crash was a live -Syu swapping mesa+hyprland under the
+running session, and the standing rule is never -Syu live under Hyprland when
+the mesa/hyprland/wayland runtime is in the set. Hardware findings (SMART, MCE,
+thermal) are None — the fix is replacing a drive or clearing a fan, not
+software.
+
+* Metrics — Storage & filesystem integrity
+
+| Metric                           | Fix / lever                      | Automation | Notes                             |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| Package cache size               | paccache -r / -ruk0              | Auto       | Reclaim, all re-downloadable      |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| paccache -rk1 (keep 1 version)   | paccache -rk1                    | Confirm    | Frees most; kills downgrade       |
+|                                  |                                  |            | headroom                          |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| Disk usage (df)                  | —                                | None       | Fix is cache/snapshot/prune       |
+|                                  |                                  |            | levers                            |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| Btrfs unallocated space          | —                                | None       | Chunk headroom; diagnostic        |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| Btrfs scrub age                  | btrfs scrub start                | Confirm    | GAP; ZFS has this, btrfs doesn't. |
+|                                  |                                  |            | IO-heavy, on-demand               |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| Btrfs device stats (per-drive    | btrfs device stats --reset       | None       | GAP; RAID1 early-warning ahead of |
+| error counters)                  |                                  |            | SMART; reset after review         |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| ZFS pool health / errors         | —                                | None       | CRITICAL if state != ONLINE or    |
+|                                  |                                  |            | errors > 0                        |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| ZFS scrub age                    | zpool scrub                      | Confirm    | Covered                           |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| ZFS capacity                     | —                                | None       | Perf degrades > 80%; the ZFS      |
+|                                  |                                  |            | headroom metric (no unallocated   |
+|                                  |                                  |            | concept)                          |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| ZFS fragmentation (FRAG)         | —                                | None       | GAP; no defrag exists — the       |
+|                                  |                                  |            | remedy is snapshot pruning +      |
+|                                  |                                  |            | staying under 80%                 |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| ZFS autotrim (SSD TRIM)          | zpool set autotrim=on / zpool    | Confirm    | GAP; velox; the fstrim.timer      |
+|                                  | trim                             |            | counterpart on ZFS                |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| fstrim.timer enabled + firing    | systemctl enable --now           | Confirm    | GAP; standard SSD hygiene         |
+|                                  | fstrim.timer                     |            |                                   |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| pacman file integrity (Qkk)      | reinstall package                | Workflow   | GAP; modified/missing files need  |
+|                                  |                                  |            | judgment                          |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| SMART health                     | —                                | None       | Hardware; replace drive           |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+| SMART wear / temperature         | —                                | None       | Diagnostic                        |
+|----------------------------------+----------------------------------+------------+-----------------------------------|
+
+* Metrics — Snapshots
+
+| Metric                         | Fix / lever              | Automation | Notes                                       |
+|--------------------------------+--------------------------+------------+---------------------------------------------|
+| Snapper count / retention      | snapper cleanup / delete | Confirm    | Manual (single) snapshots need explicit     |
+|                                |                          |            | choice                                      |
+|--------------------------------+--------------------------+------------+---------------------------------------------|
+| ZFS snapshot count / retention | zfs destroy              | Confirm    | Runaway retention                           |
+|--------------------------------+--------------------------+------------+---------------------------------------------|
+| Snapshot auto-timer running    | systemctl enable timer   | Confirm    | Is the auto-snapshot service firing         |
+|--------------------------------+--------------------------+------------+---------------------------------------------|
+
+* Metrics — Packages & security
+
+| Metric                            | Fix / lever                  | Automation | Notes                                |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+| Orphaned packages                 | pacman -Rns (named args)     | Confirm    | Review first (rust looked orphaned,  |
+|                                   |                              |            | was intentional)                     |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+| Pending updates count             | —                            | Workflow   | Never auto; workflow-only            |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+| .pacnew files                     | diff + merge / delete        | Confirm    | Allowlist auto-deletes mirrorlist /  |
+|                                   |                              |            | locale.gen                           |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+| arch-audit CVEs                   | reviewed update              | Workflow   | GAP; top-priority add. Fix is an     |
+|                                   |                              |            | update                               |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+| Keyring freshness                 | pacman -Sy archlinux-keyring | Confirm    | GAP; stale keyring breaks update     |
+|                                   |                              |            | signatures                           |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+| AUR / foreign staleness (Qm)      | paru -Sua                    | Workflow   | GAP; AUR updates reviewed, not auto  |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+| Reboot required (kernel mismatch) | reboot                       | Human      | GAP; uname -r vs /usr/lib/modules.   |
+|                                   |                              |            | You pick when                        |
+|-----------------------------------+------------------------------+------------+--------------------------------------|
+
+* Metrics — systemd & boot
+
+| Metric                         | Fix / lever            | Automation | Notes                                         |
+|--------------------------------+------------------------+------------+-----------------------------------------------|
+| is-system-running (degraded?)  | —                      | None       | GAP; one-token whole-system verdict;          |
+|                                |                        |            | candidate for the bar glyph                   |
+|--------------------------------+------------------------+------------+-----------------------------------------------|
+| Failed systemd units           | restart / investigate  | Workflow   | Depends why it failed                         |
+|--------------------------------+------------------------+------------+-----------------------------------------------|
+| Maintenance timers enabled +   | systemctl enable timer | Confirm    | GAP; meta-metric — are                        |
+| firing                         |                        |            | paccache/btrfs-scrub/fstrim/reflector/snapper |
+|                                |                        |            | firing                                        |
+|--------------------------------+------------------------+------------+-----------------------------------------------|
+| Kernel taint flag              | —                      | None       | GAP; tainted != 0 = proprietary module or     |
+|                                |                        |            | prior crash                                   |
+|--------------------------------+------------------------+------------+-----------------------------------------------|
+
+* Metrics — Logs & coredumps
+
+| Metric                             | Fix / lever                     | Automation | Notes                            |
+|------------------------------------+---------------------------------+------------+----------------------------------|
+| Coredump count                     | coredumpctl clean (keep recent) | Auto       | Keep last few days for forensics |
+|------------------------------------+---------------------------------+------------+----------------------------------|
+| Journald disk usage                | journalctl --vacuum-size/time   | Auto       | Bounded journal                  |
+|------------------------------------+---------------------------------+------------+----------------------------------|
+| App-log cleanup (>7d)              | log-cleanup cron trigger        | Auto       | Already a cron; manual trigger   |
+|------------------------------------+---------------------------------+------------+----------------------------------|
+| Journal error count (real vs       | —                               | Workflow   | Forensic; noise-filtered count   |
+| noise)                             |                                 |            |                                  |
+|------------------------------------+---------------------------------+------------+----------------------------------|
+| Kernel/hardware events             | —                               | None       | Forensic, hardware               |
+| (MCE/USB/thermal/GPU)              |                                 |            |                                  |
+|------------------------------------+---------------------------------+------------+----------------------------------|
+
+* Metrics — Memory, thermal, power
+
+| Metric                                  | Fix / lever | Automation | Notes                                      |
+|-----------------------------------------+-------------+------------+--------------------------------------------|
+| Memory free / OOM kills                 | —           | Workflow   | OOM = investigate                          |
+|-----------------------------------------+-------------+------------+--------------------------------------------|
+| Swap / zram present + healthy           | —           | None       | Diagnostic                                 |
+|-----------------------------------------+-------------+------------+--------------------------------------------|
+| CPU / GPU temperatures                  | —           | None       | Hardware                                   |
+|-----------------------------------------+-------------+------------+--------------------------------------------|
+| Thermal throttling active               | —           | None       | Cooling issue                              |
+|-----------------------------------------+-------------+------------+--------------------------------------------|
+| Battery health (capacity / cycles)      | —           | None       | GAP; laptop; ties to open suspend todo     |
+|-----------------------------------------+-------------+------------+--------------------------------------------|
+| Unclean-shutdown / suspend-failure rate | —           | Workflow   | GAP; ratio flagged ~75% unclean 2026-06-08 |
+|-----------------------------------------+-------------+------------+--------------------------------------------|
+
+* Metrics — Network & security posture
+
+| Metric                            | Fix / lever               | Automation | Notes                 |
+|-----------------------------------+---------------------------+------------+-----------------------|
+| DNS / NetworkManager reachability | restart NM                | Workflow   | CRITICAL if down      |
+|-----------------------------------+---------------------------+------------+-----------------------|
+| Firewall active (ufw / nftables)  | ufw enable                | Confirm    | GAP; security posture |
+|-----------------------------------+---------------------------+------------+-----------------------|
+| Unexpected listeners (ss -tlnp)   | —                         | None       | GAP; security review  |
+|-----------------------------------+---------------------------+------------+-----------------------|
+| Tailscale peers                   | tailscale up              | Confirm    | Covered               |
+|-----------------------------------+---------------------------+------------+-----------------------|
+| fail2ban running + bans           | systemctl start           | Confirm    | Covered               |
+|-----------------------------------+---------------------------+------------+-----------------------|
+| NTP sync (chrony)                 | systemctl restart chronyd | Confirm    | Covered               |
+|-----------------------------------+---------------------------+------------+-----------------------|
+
+* Metrics — Services, backups, virt
+
+| Metric                          | Fix / lever              | Automation | Notes                                      |
+|---------------------------------+--------------------------+------------+--------------------------------------------|
+| rsyncshot backup freshness      | —                        | Workflow   | CRITICAL if daily > 48h; investigate       |
+|                                 |                          |            | failure                                    |
+|---------------------------------+--------------------------+------------+--------------------------------------------|
+| Docker/podman reclaimable       | prune tier 1 / tiers 2-3 | Confirm    | Tier 1 nearly Auto; 2-3 destructive        |
+|---------------------------------+--------------------------+------------+--------------------------------------------|
+| Docker stopped containers       | —                        | None       | Mostly expected (WinVM on-demand)          |
+|---------------------------------+--------------------------+------------+--------------------------------------------|
+| libvirt VM state                | —                        | None       | Expected off                               |
+|---------------------------------+--------------------------+------------+--------------------------------------------|
+| Cron running + expected entries | systemctl enable cronie  | Confirm    | rsyncshot + log-cleanup entries            |
+|---------------------------------+--------------------------+------------+--------------------------------------------|
+
+* Architecture & testing (decided 2026-07-07)
+
+*CLI-first, GUI as a face.* The console ships as the fourth panel sibling: a
+=maint= Python package in dotfiles (like =net/=, =bt/=, =audio/=) with probe
+modules (read-only collectors), a remedies module, =cli.py=, and =gui.py=
+driving the same code. =maint status --json= is the contract; =maint fix
+<thing>= is every lever. The GUI never does anything the CLI can't.
+
+Safety mechanics baked into the CLI:
+- global =--dry-run= prints the exact command instead of executing — free test
+  surface, and the GUI's arm-press can display it ("this will run: …").
+- hard read/write split: collectors never elevate; every remedy is an
+  allowlisted exact argv in one small auditable module.
+
+*Four test layers (safest → scariest):*
+
+1. /Unit, fake binaries — no VM, no root (~90% of surface)./ Probes are
+   parsers over command output: feed canned smartctl/btrfs/journalctl/pacman/
+   ss/docker output via fakes on PATH (the net suite's fake-curl and audio's
+   fake-parec pattern). Remedies tested as command construction (assert the
+   argv, don't run it). The live-update guard is a pure function over a
+   package list.
+2. /Read-only integration on the live machine./ All collectors are read-only
+   by design — =maint status --json= runs safely against real hosts.
+3. /Remedies in a VM — archsetup's existing harness./
+   =scripts/testing/run-test.sh= boots the installer VM; a maint scenario
+   breaks things deliberately over ssh (stop cronie, mask fstrim, orphan
+   packages, fill the cache), runs =maint fix …=, asserts post-state. No GUI
+   in the VM. Add qcow2 snapshot/restore between remedy tests so each starts
+   pristine and destructive remedies can't contaminate each other. Pure
+   pacman-level tests may use a throwaway systemd-nspawn container instead
+   (lighter); the VM stays for systemd/btrfs/reboot territory.
+4. /GUI on the host, never in the VM./ AT-SPI smoke like the sibling panels,
+   driven by fixture data. The prototype's GOOD/BAD snapshots become those
+   fixtures — =MAINT_PANEL_FIXTURE=bad= renders the degraded board without a
+   degraded machine, conforming to the =maint status --json= schema.
+
+* Open questions
+
+- RESOLVED 2026-07-06 — live output wall (amber running / green done / red fail),
+  realtime, always shown. See "Doctor = live output wall" above.
+- RESOLVED 2026-07-06 — velox is a first-class v1 target alongside ratio. See
+  "Scope: hosts" below.
+- RESOLVED 2026-07-06 — single machine-readable thresholds file
+  (=maintenance-thresholds.toml=) is the source of truth, *owned by archsetup*.
+  Both the console and the system-health-check workflow read it, so they can
+  never drift. See "Thresholds source of truth" and "Workflow ownership" below.