docs: catalogue net + bt failure modes and draft two doctor expansion specs

I ran the audio doctor's design arc for the net and bluetooth doctors: a blind by-layer research catalogue of failure modes, a symptom-cluster triage, then a spec per doctor.

The taxonomy holds ~74 network and ~55 bluetooth distinct root causes, each sourced to a forum or issue-tracker report, sorted into eight network and five bluetooth symptom clusters. The clusters are keyed to each doctor's existing probe ladder, so the fix-versus-guide boundary falls along the tiers the doctor already walks. Every entry carries a remedy class (auto, privileged, reboot-tail, or guide), reusing the audio doctor's four-class run-time privilege model.

The two specs are expansions, not rewrites. The net doctor is already the most mature of the three: its classifier reaches six of the eight clusters today, so its spec adds only the control-plane cluster (a rival network manager, a masked NetworkManager, a bad keyfile) and sharper auth naming. The bt doctor's chain is structurally right but blind at both ends, so its spec adds a probe that names the firmware blob the kernel already logged and a probe that catches an adapter that power-on won't keep on across reboots. Both stay DRAFT with three open decisions each, headed for spec-review.

1 files changed, 197 insertions, 0 deletions

diff --git a/docs/specs/2026-07-11-net-doctor-expansion-spec.org b/docs/specs/2026-07-11-net-doctor-expansion-spec.org
new file mode 100644
index 0000000..aed22b8
--- /dev/null
+++ b/docs/specs/2026-07-11-net-doctor-expansion-spec.org
@@ -0,0 +1,197 @@
+#+TITLE: Net Doctor Expansion — the clusters the ladder doesn't yet name
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-07-11
+#+TODO: TODO | DONE
+#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED
+
+* DRAFT Net Doctor Expansion
+:PROPERTIES:
+:ID:       ce29b103-ed9d-4f56-bf8c-9ed8fe680ff3
+:END:
+- [2026-07-11 Sat @ 00:08:41 -0500] DRAFT — drafted. Extends the existing net doctor (=~/.dotfiles/net/=, shipped) using the network half of the failure taxonomy ([[file:../design/2026-07-10-net-bt-failure-taxonomy.org][2026-07-10-net-bt-failure-taxonomy.org]]). Grounded in a read of the live engine, not memory.
+
+* Metadata
+
+| Field    | Value                                                                             |
+|----------+-----------------------------------------------------------------------------------|
+| Status   | draft                                                                             |
+|----------+-----------------------------------------------------------------------------------|
+| Owner    | Craig Jennings                                                                    |
+|----------+-----------------------------------------------------------------------------------|
+| Reviewer | Craig Jennings                                                                    |
+|----------+-----------------------------------------------------------------------------------|
+| Related  | [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][net/bt failure taxonomy]] ; the cross-panel run-time-privilege and copy+close tasks |
+|----------+-----------------------------------------------------------------------------------|
+
+* Summary
+
+The net doctor is the most mature of the three panel doctors: its probe ladder already walks link → IP → gateway → route → DNS → egress, and its classifier already names most of the taxonomy's failure clusters with the lightest fix. This spec closes the two clusters it does *not* reach — control-plane conflicts (two network managers fighting over one interface, a masked NetworkManager, a keyfile the daemon silently refuses) and the sharper naming of the terminal auth cluster — and adopts the cross-panel run-time privilege model. It is an expansion of a working doctor, not a rewrite.
+
+* Problem / Context
+
+The failure taxonomy sorted ~74 real network failure modes into eight symptom clusters. Read against the live classifier (=~/.dotfiles/net/src/net/classify.py=), the net doctor already reaches six of them: cluster 1 (=rfkill=, =manage-device=, no-hardware), cluster 2 (DHCP-failed), cluster 3 (=tunnel-down=, =vpn-policy=), cluster 5 (=resolved-restart=, DNS-not-resolving, =dns-test=/=dns-override=), and cluster 6 (=portal=, =clock-sync=, =proxy=, and the =upstream-not-local= terminal STOP). The classifier's terminal-first ordering already refuses to loop repairs against a wrong password, a held portal, or a VPN-owned route. That is a lot of the taxonomy, already built.
+
+Two clusters fall through.
+
+** The control plane can be broken while the radio looks fine (cluster 7)
+
+The taxonomy's largest untouched cluster is NetworkManager itself. When =dhcpcd.service= runs beside NM's internal DHCP client, or =systemd-networkd= and NM both claim one link, or =iwd= and =wpa_supplicant= are both active, the interface flaps or never leases — and every existing probe reads a plausible-looking radio with no verdict that names the fight. NM masked (=systemctl start= returns "Unit is masked") reads as "NetworkManager isn't running" today, which points =nm-restart= at a service that cannot start. A =.nmconnection= keyfile that isn't =600= root-owned is silently skipped by the daemon, so a saved network "just won't connect" with no signal the doctor surfaces. These are distinct root causes with distinct fixes, and the doctor currently has one verdict (=nm-restart=) covering the whole control plane.
+
+** The auth cluster is named too coarsely (cluster 4)
+
+=classify= detects an auth failure from NM state 120 / GENERAL.REASON and returns =needs-user-action= — correct, and correctly terminal. But the taxonomy shows the auth cluster is not one failure: a pure-WPA3/SAE association failure, a hidden SSID never probed, an enterprise cert mismatch, and a wrong regulatory domain are different faults with different next actions, and some (SAE key-mgmt, =wifi.hidden yes=) are one-line profile fixes rather than "re-enter the password." The doctor collapses them all to one message.
+
+** The flaky/drops cluster needs signal the one-shot doctor doesn't collect (cluster 8)
+
+Powersave disconnects, roaming stalls, USB autosuspend, no-reconnect-after-resume, firmware crashloops — these are intermittent, and a single-shot "why am I offline right now" probe cannot see them. Naming them needs event-log correlation the doctor doesn't do. This cluster is real but out of v1; it is named here so the boundary is explicit.
+
+* Goals and Non-Goals
+
+** Goals
+
+- Add a control-plane cluster: a probe that detects a second network/DHCP manager active alongside NetworkManager, a masked/failed NM distinct from a merely-stopped one, and a keyfile-permission fault, each with its own verdict and the lightest fix.
+- Sharpen the auth verdict: extract the specific auth cluster cause (SAE, hidden SSID, enterprise cert, regdom) so =needs-user-action= names the real next step, and apply the one-line profile fix where one exists.
+- Adopt the cross-panel run-time privilege model (Auto / Privileged / Reboot-tail / Guide, resolved from =sudo -n= + tty + GUI), so control-plane repairs that need root (=systemctl disable dhcpcd=, =chmod= a keyfile, =systemctl unmask=) run under the same Confirm/Arm-default floor the audio doctor defined.
+
+** Non-Goals
+
+- Rebuilding the existing ladder. Clusters 1/2/3/5/6 stay exactly as they classify today; this spec only adds where the taxonomy shows a gap.
+- The flaky/drops cluster (cluster 8). Naming intermittent faults needs event-log correlation that a one-shot doctor doesn't do. Logged to =todo.org=, not built here.
+- Turning the doctor into a NetworkManager profile editor. The auth-cluster fixes are limited to the one-line profile settings that get a stuck association online (SAE key-mgmt, hidden flag); it will not manage certificates, enterprise identities, or credential entry — those stay Guide.
+- Silent privileged action. Every Privileged/Reboot-tail control-plane repair defaults to Confirm/Arm, never silent Auto — same stance as the audio spec.
+
+** Scope tiers
+
+- *v1:* the control-plane conflict probe and its verdicts (rival-manager, NM-masked, keyfile-perms); sharper auth-cluster reason extraction plus the SAE/hidden one-line fixes; adoption of the run-time privilege model for the new root-needing repairs.
+- *Out of scope:* the flaky/drops cluster; certificate/enterprise credential management; any change to the six clusters already classified.
+- *vNext:* event-log correlation for cluster 8 (powersave/roam/suspend/autosuspend drop signatures); a DoT/DNSSEC-specific verdict distinguishing "the venue resolver mangles DNSSEC" from the generic DNS-not-resolving; per-profile autoconnect/duplicate-profile hygiene. All logged to =todo.org=.
+
+* Design
+
+** For the user
+
+Nothing changes for the six clusters that already work. What changes is that two failure shapes that today produce a wrong or vague verdict start naming themselves.
+
+When a second manager is fighting NetworkManager for the link, the wall says so by name — "dhcpcd is running alongside NetworkManager and they're fighting over the interface" — and offers the fix (stop the rival) rather than bouncing a connection that will flap again the moment the other daemon re-grabs it. When NM is masked, the verdict distinguishes "masked — it can't start until unmasked" from "stopped," so FIX unmasks rather than uselessly restarting. When a saved network silently won't load because its keyfile is world-readable, the doctor names the permission fault instead of leaving the user to wonder why a known-good network never activates.
+
+When association fails on auth, the wall stops saying only "authentication failed" and names which auth: a WPA3-only network the profile isn't set for, a hidden SSID that needs the hidden flag, an enterprise network missing its CA cert. The first two carry a one-press fix; the rest tell the user the specific thing to supply.
+
+** For the implementer
+
+*** The control-plane probe
+
+A new read-only probe tier, beside the existing ones, answering three questions the current ladder never asks:
+
+1. *Is a rival manager active?* Check whether =dhcpcd.service=, =systemd-networkd.service=, or a standalone =iwd.service= is active while NetworkManager is also active and owns (or wants to own) the link. This is a =systemctl is-active= read plus NM's backend setting — bounded, no hang. A rival that is active is the verdict; the fix is =systemctl disable --now <rival>= (Privileged), never a connection bounce.
+2. *Is NM masked or failed, as distinct from stopped?* =systemctl is-enabled NetworkManager= returns =masked=; the unit's =ActiveState=/=Result= distinguishes a crash-loop from a clean stop. A masked NM gets an =unmask= verdict (Privileged), not the existing =nm-restart=.
+3. *Does the active profile's keyfile have the wrong permissions?* For the selected connection, stat its =/etc/NetworkManager/system-connections/*.nmconnection=; a non-=600= or non-root-owned file is the silent-skip fault. Fix: =chmod 600= + =chown root= (Privileged).
+
+The probe runs before the existing "NetworkManager isn't running" rule, because a masked NM and a rival-manager fight are both more specific than "not running" and would otherwise be mis-verdicted by it.
+
+*** The classifier gains control-plane verdicts
+
+=classify= adds, in terminal-and-specificity order ahead of the generic =nm-restart= rule: =rival-manager= (needs-user-action → Privileged fix), =nm-masked= (fixable → unmask), =keyfile-perms= (fixable → chmod/chown). Each carries evidence naming the specific rival/unit/file. These are additive; the existing rules below them are untouched.
+
+*** The auth verdict takes a reason
+
+=gather_context='s existing =auth_failed_reason= (already extracted from GENERAL.REASON and the journal) grows a small classifier: SAE/PMF, hidden-SSID, enterprise-cert, regdom, or generic-PSK. The =needs-user-action= message is keyed off it. For SAE and hidden-SSID — the two with a deterministic one-line profile fix — the verdict becomes =fixable= with a Privileged/Auto profile-modify action (=key-mgmt sae= + PMF, or =wifi.hidden yes=) rather than terminal. The rest stay =needs-user-action= with a sharpened message.
+
+*** The privilege model
+
+The new repairs (=systemctl disable <rival>=, =unmask=, =chmod=/=chown= a keyfile, profile-modify) are the net doctor's first root-needing doctor repairs beyond the ones already in =priv.py=. They adopt the cross-panel run-time resolution the audio spec defined: Privileged remedies run silently where passwordless sudo exists (every archsetup install), prompt on a CLI with a tty, and default to Confirm/Arm — never silent Auto. This is the same standard, not a net-specific one; the shared implementation is the tracked cross-panel task.
+
+* Alternatives Considered
+
+** Fold the control-plane faults into the existing =nm-restart= verdict
+
+- Good, because it is zero new classifier surface.
+- Bad, because =nm-restart= is the wrong fix for all three: restarting a masked NM fails, restarting NM does not stop a rival =dhcpcd=, and it does nothing for a bad keyfile. A shared verdict would send the doctor's one fix at three faults it can't fix.
+- Rejected. Different root causes with different fixes are different verdicts — the same principle the audio spec used for =pulse-hung= vs =pulse-down=.
+
+** Make the doctor a full profile editor for the auth cluster
+
+- Good, because it could fix more auth failures automatically.
+- Bad, because credential and certificate entry is a genuine user decision, not a repair — the doctor cannot invent an enterprise CA or a password. Auto-editing profiles beyond the two deterministic one-liners risks writing a wrong setting the user then has to unwind.
+- Rejected in favor of fixing only SAE key-mgmt and the hidden flag, and guiding the rest.
+
+** Build the flaky/drops cluster now with a synthetic re-probe
+
+- Good, because it would catch powersave/autosuspend "works then dies" cases.
+- Bad, because a one-shot doctor invoked when the user is already offline has no drop history to read; catching intermittent faults needs the panel's event log correlated over time, which is a separate probe surface.
+- Rejected for v1; logged as vNext.
+
+* Decisions [0/3]
+
+** TODO The control-plane probe and its three verdicts
+Context: cluster 7 is the taxonomy's largest untouched cluster, and the current single =nm-restart= verdict mis-serves a masked NM, a rival manager, and a bad keyfile.
+Decision (proposed): we will add a read-only control-plane probe (rival-manager active-check, NM masked-vs-failed-vs-stopped, active-profile keyfile permissions) and three verdicts ahead of the generic =nm-restart= rule, each with the lightest specific fix.
+Consequences: the doctor names the fight instead of bouncing a link that will re-flap; harder — three new verdicts and a probe that reads =systemctl= state plus a stat, and the ordering has to sit ahead of the existing not-running rule without disturbing it.
+Owner: Craig. By: before Phase 1 lands.
+
+** TODO How far the auth-cluster fix goes
+Context: the auth cluster is terminal today; some members (SAE, hidden SSID) have deterministic one-line profile fixes, others (enterprise cert, credential) do not.
+Decision (proposed): we will extract the specific auth reason and make only SAE-key-mgmt and hidden-flag =fixable=; everything else stays =needs-user-action= with a sharpened, cause-named message.
+Consequences: two more auth failures self-heal; harder — the doctor now writes to a connection profile, which is a heavier action than a bounce, and the boundary between "fix" and "guide" inside one cluster has to be defended so it doesn't creep into credential management.
+Owner: Craig. By: before Phase 2 lands.
+
+** TODO Adopt the run-time privilege model as the cross-panel standard
+Context: the new control-plane repairs need root; the audio spec already defined the four-class run-time model and made it a cross-panel standard.
+Decision (proposed): we will adopt it verbatim — Privileged repairs run where passwordless sudo exists, prompt on a tty, default to Confirm/Arm, never silent Auto — sharing the implementation with the other panels rather than reimplementing it.
+Consequences: the net doctor's root repairs are consistent with audio/bt/maint; harder — it couples this spec to the shared privilege-model task, so the ordering across panels has to be settled (which panel lands the shared code).
+Owner: Craig. By: before Phase 1 lands.
+
+* Implementation phases
+
+Each phase leaves the tree green and independently useful, as the existing net phases did.
+
+** TODO Phase 0 — the control-plane probe (read-only)
+Pure engine, no classifier changes. A probe module that reports rival-manager state, NM masked/failed/stopped, and active-profile keyfile permissions into the diag context. =net diag --json= shows the new signals. Fakes: injected =systemctl is-active/is-enabled= results and a temp system-connections tree.
+
+** TODO Phase 1 — control-plane verdicts + the privilege model
+=classify= gains =rival-manager=, =nm-masked=, =keyfile-perms=, ordered ahead of the generic not-running rule. The run-time privilege resolution lands (shared with the cross-panel task) and the three new fixes register as Privileged/Confirm. =net doctor= names them; =net doctor --fix= applies them under the Confirm floor.
+
+** TODO Phase 2 — the sharpened auth verdict
+The auth-reason classifier; SAE and hidden-SSID become =fixable= profile-modify repairs; the rest get cause-named =needs-user-action= messages. Pairwise over (reason × profile-state).
+
+** TODO Phase 3 — flip this spec to IMPLEMENTED
+And log the vNext items (flaky/drops cluster, DoT/DNSSEC verdict, profile hygiene) to =todo.org=.
+
+* Acceptance criteria
+
+- [ ] With =dhcpcd.service= active alongside NetworkManager, the doctor reports =rival-manager= naming dhcpcd, and FIX stops it rather than bouncing the connection.
+- [ ] A masked NetworkManager reports =nm-masked= (distinct from stopped), and FIX unmasks it.
+- [ ] A non-600 keyfile for the active profile reports =keyfile-perms=, and FIX corrects the permissions.
+- [ ] A WPA3-only association failure reports the SAE cause and (with --fix) sets the profile's key-mgmt, rather than saying only "authentication failed."
+- [ ] An enterprise-cert auth failure stays =needs-user-action= but names the missing CA cert.
+- [ ] Every new root-needing repair defaults to Confirm/Arm and never runs silently as Auto.
+- [ ] The six existing clusters classify exactly as they do today (regression).
+
+* Readiness dimensions
+
+- *Data model & ownership* — the diag context gains control-plane signals (generated per-probe) and a richer =auth_failed_reason= (generated from GENERAL.REASON + journal, as today). The doctor never writes NM config except the two auth profile-modifies and the keyfile-perms fix, all under the Confirm floor.
+- *Errors, empty states & failure* — an unreadable =systemctl=/=stat= yields "unknown," never a false rival/masked/perms verdict (the safe direction). Partial reads degrade to the existing behavior.
+- *Security & privacy* — SSIDs and connection names are already redacted by =redact.py=; the keyfile probe reads permissions, not secrets. The auth-reason extraction must not leak the PSK (it reads NM's reason string, not the key).
+- *Observability* — the wall names the specific rival/unit/file. =--json= carries the new context.
+- *Performance & scale* — three =systemctl= reads and a stat; negligible beside the existing probe cost.
+- *Reuse & lost opportunities* — reuses =gather_context='s existing auth-reason extraction, =priv.py=, and the shared cross-panel privilege model rather than a net-local one. =classify= stays the single verdict authority.
+- *Architecture fit* — the new probe is additive beside the existing tiers; the classifier additions sit ahead of the generic not-running rule and leave the rest untouched. Weak point: ordering — the new verdicts must precede =nm-restart= or a masked/rival case mis-classifies.
+- *Config surface* — none new. N/A.
+- *Documentation plan* — module docstrings, as the package does today. The wall is the user documentation.
+- *Dev tooling* — =make test= and the net panel smoke cover it; the control-plane probe needs injected =systemctl= state, a fixture shape the package already uses elsewhere.
+- *Rollout, compatibility & rollback* — additive; =net doctor= with no new fault behaves as today. The auth profile-modify and keyfile-perms fixes change persisted NM state, so both are Confirm-tier and reversible by the user.
+- *External APIs & deps* — =systemctl is-active/is-enabled=, =nmcli=, and =/etc/NetworkManager/system-connections= layout are all already used by the engine; no new packages. The exact rival-manager set (dhcpcd/networkd/iwd) is verified against the live system before Phase 0.
+
+* Risks, rabbit holes, and drawbacks
+
+The rival-manager check can false-positive if a rival service is active but not actually contending for the same interface (e.g. =systemd-networkd= managing a container bridge while NM owns wifi). The probe must scope the conflict to the link the doctor is diagnosing, not merely "is networkd running" — otherwise it cries wolf on a legitimate split. This is the main correctness rabbit hole and wants a test with a bridge-only networkd.
+
+The auth-cluster boundary between "fix" and "guide" is a slope. SAE and the hidden flag are safe because they are deterministic and reversible; the temptation is to add "just one more" auto-fix until the doctor is editing enterprise profiles it shouldn't. The Non-Goal is the guardrail; hold it.
+
+Coupling to the shared privilege-model task means this spec can't fully land until that model exists somewhere. Sequence it: whichever panel lands the shared code first, the others depend on it.
+
+* Review and iteration history
+
+** 2026-07-11 Sat @ 00:08:41 -0500 — Craig Jennings — Author
+- What: drafted the net doctor expansion from the network half of the failure taxonomy. v1 adds the control-plane cluster (rival-manager, NM-masked, keyfile-perms) and sharpens the auth verdict; the flaky/drops cluster is staged to vNext.
+- Why: the taxonomy showed the net doctor already reaches six of eight clusters, and the two it misses (control plane, auth naming) are where its verdicts are wrong or vague today.
+- Artifacts: [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][the net/bt failure taxonomy]]; code read across =~/.dotfiles/net/src/net/= (=classify.py=, =diag.py=, =doctor.py=, =repair.py=, =priv.py=).