1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
|
#+TITLE: Net Doctor Expansion — the clusters the ladder doesn't yet name
#+AUTHOR: Craig Jennings
#+DATE: 2026-07-11
#+TODO: TODO | DONE
#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED
* DRAFT Net Doctor Expansion
:PROPERTIES:
:ID: ce29b103-ed9d-4f56-bf8c-9ed8fe680ff3
:END:
- [2026-07-11 Sat @ 00:08:41 -0500] DRAFT — drafted. Extends the existing net doctor (=~/.dotfiles/net/=, shipped) using the network half of the failure taxonomy ([[file:../design/2026-07-10-net-bt-failure-taxonomy.org][2026-07-10-net-bt-failure-taxonomy.org]]). Grounded in a read of the live engine, not memory.
* Metadata
| Field | Value |
|----------+-----------------------------------------------------------------------------------|
| Status | draft |
|----------+-----------------------------------------------------------------------------------|
| Owner | Craig Jennings |
|----------+-----------------------------------------------------------------------------------|
| Reviewer | Craig Jennings |
|----------+-----------------------------------------------------------------------------------|
| Related | [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][net/bt failure taxonomy]] ; the cross-panel run-time-privilege and copy+close tasks |
|----------+-----------------------------------------------------------------------------------|
* Summary
The net doctor is the most mature of the three panel doctors: its probe ladder already walks link → IP → gateway → route → DNS → egress, and its classifier already names most of the taxonomy's failure clusters with the lightest fix. This spec closes the two clusters it does *not* reach — control-plane conflicts (two network managers fighting over one interface, a masked NetworkManager, a keyfile the daemon silently refuses) and the sharper naming of the terminal auth cluster — and adopts the cross-panel run-time privilege model. It is an expansion of a working doctor, not a rewrite.
* Problem / Context
The failure taxonomy sorted ~74 real network failure modes into eight symptom clusters. Read against the live classifier (=~/.dotfiles/net/src/net/classify.py=), the net doctor already reaches six of them: cluster 1 (=rfkill=, =manage-device=, no-hardware), cluster 2 (DHCP-failed), cluster 3 (=tunnel-down=, =vpn-policy=), cluster 5 (=resolved-restart=, DNS-not-resolving, =dns-test=/=dns-override=), and cluster 6 (=portal=, =clock-sync=, =proxy=, and the =upstream-not-local= terminal STOP). The classifier's terminal-first ordering already refuses to loop repairs against a wrong password, a held portal, or a VPN-owned route. That is a lot of the taxonomy, already built.
Two clusters fall through.
** The control plane can be broken while the radio looks fine (cluster 7)
The taxonomy's largest untouched cluster is NetworkManager itself. When =dhcpcd.service= runs beside NM's internal DHCP client, or =systemd-networkd= and NM both claim one link, or =iwd= and =wpa_supplicant= are both active, the interface flaps or never leases — and every existing probe reads a plausible-looking radio with no verdict that names the fight. NM masked (=systemctl start= returns "Unit is masked") reads as "NetworkManager isn't running" today, which points =nm-restart= at a service that cannot start. A =.nmconnection= keyfile that isn't =600= root-owned is silently skipped by the daemon, so a saved network "just won't connect" with no signal the doctor surfaces. These are distinct root causes with distinct fixes, and the doctor currently has one verdict (=nm-restart=) covering the whole control plane.
** The auth cluster is named too coarsely (cluster 4)
=classify= detects an auth failure from NM state 120 / GENERAL.REASON and returns =needs-user-action= — correct, and correctly terminal. But the taxonomy shows the auth cluster is not one failure: a pure-WPA3/SAE association failure, a hidden SSID never probed, an enterprise cert mismatch, and a wrong regulatory domain are different faults with different next actions, and some (SAE key-mgmt, =wifi.hidden yes=) are one-line profile fixes rather than "re-enter the password." The doctor collapses them all to one message.
** The flaky/drops cluster needs signal the one-shot doctor doesn't collect (cluster 8)
Powersave disconnects, roaming stalls, USB autosuspend, no-reconnect-after-resume, firmware crashloops — these are intermittent, and a single-shot "why am I offline right now" probe cannot see them. Naming them needs event-log correlation the doctor doesn't do. This cluster is real but out of v1; it is named here so the boundary is explicit.
* Goals and Non-Goals
** Goals
- Add a control-plane cluster: a probe that detects a second network/DHCP manager active alongside NetworkManager, a masked/failed NM distinct from a merely-stopped one, and a keyfile-permission fault, each with its own verdict and the lightest fix.
- Sharpen the auth verdict: extract the specific auth cluster cause (SAE, hidden SSID, enterprise cert, regdom) so =needs-user-action= names the real next step, and apply the one-line profile fix where one exists.
- Adopt the cross-panel run-time privilege model (Auto / Privileged / Reboot-tail / Guide, resolved from =sudo -n= + tty + GUI), so control-plane repairs that need root (=systemctl disable dhcpcd=, =chmod= a keyfile, =systemctl unmask=) run under the same Confirm/Arm-default floor the audio doctor defined.
** Non-Goals
- Rebuilding the existing ladder. Clusters 1/2/3/5/6 stay exactly as they classify today; this spec only adds where the taxonomy shows a gap.
- The flaky/drops cluster (cluster 8). Naming intermittent faults needs event-log correlation that a one-shot doctor doesn't do. Logged to =todo.org=, not built here.
- Turning the doctor into a NetworkManager profile editor. The auth-cluster fixes are limited to the one-line profile settings that get a stuck association online (SAE key-mgmt, hidden flag); it will not manage certificates, enterprise identities, or credential entry — those stay Guide.
- Silent privileged action. Every Privileged/Reboot-tail control-plane repair defaults to Confirm/Arm, never silent Auto — same stance as the audio spec.
** Scope tiers
- *v1:* the control-plane conflict probe and its verdicts (rival-manager, NM-masked, keyfile-perms); sharper auth-cluster reason extraction plus the SAE/hidden one-line fixes; adoption of the run-time privilege model for the new root-needing repairs.
- *Out of scope:* the flaky/drops cluster; certificate/enterprise credential management; any change to the six clusters already classified.
- *vNext:* event-log correlation for cluster 8 (powersave/roam/suspend/autosuspend drop signatures); a DoT/DNSSEC-specific verdict distinguishing "the venue resolver mangles DNSSEC" from the generic DNS-not-resolving; per-profile autoconnect/duplicate-profile hygiene. All logged to =todo.org=.
* Design
** For the user
Nothing changes for the six clusters that already work. What changes is that two failure shapes that today produce a wrong or vague verdict start naming themselves.
When a second manager is fighting NetworkManager for the link, the wall says so by name — "dhcpcd is running alongside NetworkManager and they're fighting over the interface" — and offers the fix (stop the rival) rather than bouncing a connection that will flap again the moment the other daemon re-grabs it. When NM is masked, the verdict distinguishes "masked — it can't start until unmasked" from "stopped," so FIX unmasks rather than uselessly restarting. When a saved network silently won't load because its keyfile is world-readable, the doctor names the permission fault instead of leaving the user to wonder why a known-good network never activates.
When association fails on auth, the wall stops saying only "authentication failed" and names which auth: a WPA3-only network the profile isn't set for, a hidden SSID that needs the hidden flag, an enterprise network missing its CA cert. The first two carry a one-press fix; the rest tell the user the specific thing to supply.
** For the implementer
*** The control-plane probe
A new read-only probe tier, beside the existing ones, answering three questions the current ladder never asks:
1. *Is a rival manager active?* Check whether =dhcpcd.service=, =systemd-networkd.service=, or a standalone =iwd.service= is active while NetworkManager is also active and owns (or wants to own) the link. This is a =systemctl is-active= read plus NM's backend setting — bounded, no hang. A rival that is active is the verdict; the fix is =systemctl disable --now <rival>= (Privileged), never a connection bounce.
2. *Is NM masked or failed, as distinct from stopped?* =systemctl is-enabled NetworkManager= returns =masked=; the unit's =ActiveState=/=Result= distinguishes a crash-loop from a clean stop. A masked NM gets an =unmask= verdict (Privileged), not the existing =nm-restart=.
3. *Does the active profile's keyfile have the wrong permissions?* For the selected connection, stat its =/etc/NetworkManager/system-connections/*.nmconnection=; a non-=600= or non-root-owned file is the silent-skip fault. Fix: =chmod 600= + =chown root= (Privileged).
The probe runs before the existing "NetworkManager isn't running" rule, because a masked NM and a rival-manager fight are both more specific than "not running" and would otherwise be mis-verdicted by it.
*** The classifier gains control-plane verdicts
=classify= adds, in terminal-and-specificity order ahead of the generic =nm-restart= rule: =rival-manager= (needs-user-action → Privileged fix), =nm-masked= (fixable → unmask), =keyfile-perms= (fixable → chmod/chown). Each carries evidence naming the specific rival/unit/file. These are additive; the existing rules below them are untouched.
*** The auth verdict takes a reason
=gather_context='s existing =auth_failed_reason= (already extracted from GENERAL.REASON and the journal) grows a small classifier: SAE/PMF, hidden-SSID, enterprise-cert, regdom, or generic-PSK. The =needs-user-action= message is keyed off it. For SAE and hidden-SSID — the two with a deterministic one-line profile fix — the verdict becomes =fixable= with a Privileged/Auto profile-modify action (=key-mgmt sae= + PMF, or =wifi.hidden yes=) rather than terminal. The rest stay =needs-user-action= with a sharpened message.
*** The privilege model
The new repairs (=systemctl disable <rival>=, =unmask=, =chmod=/=chown= a keyfile, profile-modify) are the net doctor's first root-needing doctor repairs beyond the ones already in =priv.py=. They adopt the cross-panel run-time resolution the audio spec defined: Privileged remedies run silently where passwordless sudo exists (every archsetup install), prompt on a CLI with a tty, and default to Confirm/Arm — never silent Auto. This is the same standard, not a net-specific one; the shared implementation is the tracked cross-panel task.
* Alternatives Considered
** Fold the control-plane faults into the existing =nm-restart= verdict
- Good, because it is zero new classifier surface.
- Bad, because =nm-restart= is the wrong fix for all three: restarting a masked NM fails, restarting NM does not stop a rival =dhcpcd=, and it does nothing for a bad keyfile. A shared verdict would send the doctor's one fix at three faults it can't fix.
- Rejected. Different root causes with different fixes are different verdicts — the same principle the audio spec used for =pulse-hung= vs =pulse-down=.
** Make the doctor a full profile editor for the auth cluster
- Good, because it could fix more auth failures automatically.
- Bad, because credential and certificate entry is a genuine user decision, not a repair — the doctor cannot invent an enterprise CA or a password. Auto-editing profiles beyond the two deterministic one-liners risks writing a wrong setting the user then has to unwind.
- Rejected in favor of fixing only SAE key-mgmt and the hidden flag, and guiding the rest.
** Build the flaky/drops cluster now with a synthetic re-probe
- Good, because it would catch powersave/autosuspend "works then dies" cases.
- Bad, because a one-shot doctor invoked when the user is already offline has no drop history to read; catching intermittent faults needs the panel's event log correlated over time, which is a separate probe surface.
- Rejected for v1; logged as vNext.
* Decisions [0/3]
** TODO The control-plane probe and its three verdicts
Context: cluster 7 is the taxonomy's largest untouched cluster, and the current single =nm-restart= verdict mis-serves a masked NM, a rival manager, and a bad keyfile.
Decision (proposed): we will add a read-only control-plane probe (rival-manager active-check, NM masked-vs-failed-vs-stopped, active-profile keyfile permissions) and three verdicts ahead of the generic =nm-restart= rule, each with the lightest specific fix.
Consequences: the doctor names the fight instead of bouncing a link that will re-flap; harder — three new verdicts and a probe that reads =systemctl= state plus a stat, and the ordering has to sit ahead of the existing not-running rule without disturbing it.
Owner: Craig. By: before Phase 1 lands.
** TODO How far the auth-cluster fix goes
Context: the auth cluster is terminal today; some members (SAE, hidden SSID) have deterministic one-line profile fixes, others (enterprise cert, credential) do not.
Decision (proposed): we will extract the specific auth reason and make only SAE-key-mgmt and hidden-flag =fixable=; everything else stays =needs-user-action= with a sharpened, cause-named message.
Consequences: two more auth failures self-heal; harder — the doctor now writes to a connection profile, which is a heavier action than a bounce, and the boundary between "fix" and "guide" inside one cluster has to be defended so it doesn't creep into credential management.
Owner: Craig. By: before Phase 2 lands.
** TODO Adopt the run-time privilege model as the cross-panel standard
Context: the new control-plane repairs need root; the audio spec already defined the four-class run-time model and made it a cross-panel standard.
Decision (proposed): we will adopt it verbatim — Privileged repairs run where passwordless sudo exists, prompt on a tty, default to Confirm/Arm, never silent Auto — sharing the implementation with the other panels rather than reimplementing it.
Consequences: the net doctor's root repairs are consistent with audio/bt/maint; harder — it couples this spec to the shared privilege-model task, so the ordering across panels has to be settled (which panel lands the shared code).
Owner: Craig. By: before Phase 1 lands.
* Implementation phases
Each phase leaves the tree green and independently useful, as the existing net phases did.
** TODO Phase 0 — the control-plane probe (read-only)
Pure engine, no classifier changes. A probe module that reports rival-manager state, NM masked/failed/stopped, and active-profile keyfile permissions into the diag context. =net diag --json= shows the new signals. Fakes: injected =systemctl is-active/is-enabled= results and a temp system-connections tree.
** TODO Phase 1 — control-plane verdicts + the privilege model
=classify= gains =rival-manager=, =nm-masked=, =keyfile-perms=, ordered ahead of the generic not-running rule. The run-time privilege resolution lands (shared with the cross-panel task) and the three new fixes register as Privileged/Confirm. =net doctor= names them; =net doctor --fix= applies them under the Confirm floor.
** TODO Phase 2 — the sharpened auth verdict
The auth-reason classifier; SAE and hidden-SSID become =fixable= profile-modify repairs; the rest get cause-named =needs-user-action= messages. Pairwise over (reason × profile-state).
** TODO Phase 3 — flip this spec to IMPLEMENTED
And log the vNext items (flaky/drops cluster, DoT/DNSSEC verdict, profile hygiene) to =todo.org=.
* Acceptance criteria
- [ ] With =dhcpcd.service= active alongside NetworkManager, the doctor reports =rival-manager= naming dhcpcd, and FIX stops it rather than bouncing the connection.
- [ ] A masked NetworkManager reports =nm-masked= (distinct from stopped), and FIX unmasks it.
- [ ] A non-600 keyfile for the active profile reports =keyfile-perms=, and FIX corrects the permissions.
- [ ] A WPA3-only association failure reports the SAE cause and (with --fix) sets the profile's key-mgmt, rather than saying only "authentication failed."
- [ ] An enterprise-cert auth failure stays =needs-user-action= but names the missing CA cert.
- [ ] Every new root-needing repair defaults to Confirm/Arm and never runs silently as Auto.
- [ ] The six existing clusters classify exactly as they do today (regression).
* Readiness dimensions
- *Data model & ownership* — the diag context gains control-plane signals (generated per-probe) and a richer =auth_failed_reason= (generated from GENERAL.REASON + journal, as today). The doctor never writes NM config except the two auth profile-modifies and the keyfile-perms fix, all under the Confirm floor.
- *Errors, empty states & failure* — an unreadable =systemctl=/=stat= yields "unknown," never a false rival/masked/perms verdict (the safe direction). Partial reads degrade to the existing behavior.
- *Security & privacy* — SSIDs and connection names are already redacted by =redact.py=; the keyfile probe reads permissions, not secrets. The auth-reason extraction must not leak the PSK (it reads NM's reason string, not the key).
- *Observability* — the wall names the specific rival/unit/file. =--json= carries the new context.
- *Performance & scale* — three =systemctl= reads and a stat; negligible beside the existing probe cost.
- *Reuse & lost opportunities* — reuses =gather_context='s existing auth-reason extraction, =priv.py=, and the shared cross-panel privilege model rather than a net-local one. =classify= stays the single verdict authority.
- *Architecture fit* — the new probe is additive beside the existing tiers; the classifier additions sit ahead of the generic not-running rule and leave the rest untouched. Weak point: ordering — the new verdicts must precede =nm-restart= or a masked/rival case mis-classifies.
- *Config surface* — none new. N/A.
- *Documentation plan* — module docstrings, as the package does today. The wall is the user documentation.
- *Dev tooling* — =make test= and the net panel smoke cover it; the control-plane probe needs injected =systemctl= state, a fixture shape the package already uses elsewhere.
- *Rollout, compatibility & rollback* — additive; =net doctor= with no new fault behaves as today. The auth profile-modify and keyfile-perms fixes change persisted NM state, so both are Confirm-tier and reversible by the user.
- *External APIs & deps* — =systemctl is-active/is-enabled=, =nmcli=, and =/etc/NetworkManager/system-connections= layout are all already used by the engine; no new packages. The exact rival-manager set (dhcpcd/networkd/iwd) is verified against the live system before Phase 0.
* Risks, rabbit holes, and drawbacks
The rival-manager check can false-positive if a rival service is active but not actually contending for the same interface (e.g. =systemd-networkd= managing a container bridge while NM owns wifi). The probe must scope the conflict to the link the doctor is diagnosing, not merely "is networkd running" — otherwise it cries wolf on a legitimate split. This is the main correctness rabbit hole and wants a test with a bridge-only networkd.
The auth-cluster boundary between "fix" and "guide" is a slope. SAE and the hidden flag are safe because they are deterministic and reversible; the temptation is to add "just one more" auto-fix until the doctor is editing enterprise profiles it shouldn't. The Non-Goal is the guardrail; hold it.
Coupling to the shared privilege-model task means this spec can't fully land until that model exists somewhere. Sequence it: whichever panel lands the shared code first, the others depend on it.
* Review and iteration history
** 2026-07-11 Sat @ 00:08:41 -0500 — Craig Jennings — Author
- What: drafted the net doctor expansion from the network half of the failure taxonomy. v1 adds the control-plane cluster (rival-manager, NM-masked, keyfile-perms) and sharpens the auth verdict; the flaky/drops cluster is staged to vNext.
- Why: the taxonomy showed the net doctor already reaches six of eight clusters, and the two it misses (control plane, auth naming) are where its verdicts are wrong or vague today.
- Artifacts: [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][the net/bt failure taxonomy]]; code read across =~/.dotfiles/net/src/net/= (=classify.py=, =diag.py=, =doctor.py=, =repair.py=, =priv.py=).
|
