diff options
| -rw-r--r-- | docs/design/2026-07-10-audio-failure-taxonomy.org | 444 | ||||
| -rw-r--r-- | docs/design/2026-07-10-net-bt-failure-taxonomy.org | 434 | ||||
| -rw-r--r-- | docs/specs/2026-07-09-audio-doctor-spec.org | 1 | ||||
| -rw-r--r-- | docs/specs/2026-07-10-audio-doctor-input-side-spec.org | 494 | ||||
| -rw-r--r-- | docs/specs/2026-07-11-bt-doctor-expansion-spec.org | 250 | ||||
| -rw-r--r-- | docs/specs/2026-07-11-net-doctor-expansion-spec.org | 266 | ||||
| -rw-r--r-- | scripts/testing/net-scenarios/10-nm-masked.sh | 29 | ||||
| -rw-r--r-- | scripts/testing/net-scenarios/20-rival-manager.sh | 27 | ||||
| -rw-r--r-- | scripts/testing/net-scenarios/30-keyfile-perms.sh | 30 | ||||
| -rwxr-xr-x | scripts/testing/run-net-scenarios.sh | 115 | ||||
| -rw-r--r-- | todo.org | 96 |
11 files changed, 2186 insertions, 0 deletions
diff --git a/docs/design/2026-07-10-audio-failure-taxonomy.org b/docs/design/2026-07-10-audio-failure-taxonomy.org new file mode 100644 index 0000000..efb6e0a --- /dev/null +++ b/docs/design/2026-07-10-audio-failure-taxonomy.org @@ -0,0 +1,444 @@ +#+TITLE: Linux Audio Failure Taxonomy — for the audio doctor +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-10 + +* Purpose + +A catalogue of real, user-reported ways Linux audio input (microphone) and output (speakers/headphones) fail, grounded in forum and issue-tracker reports. Built to feed the audio doctor: each entry will later be triaged into build (an auto/confirm remedy the doctor applies), guide (a diagnosis plus the exact fix the user runs), or out (beyond the doctor's reach). Triage is a later pass — for now the goal is breadth. + +Target: 50 input failure modes and 50 output failure modes, or exhaustion. Each entry names the layer, the symptom as a user reports it, the root cause, the known resolution, and a source. + +** Saturation test (2026-07-10) + +To check whether the nine-cluster-per-direction taxonomy was representative or an artifact of how we first sampled, a blind resample: ten agents drew 108 fresh failure reports through five different doors per direction (specific hardware lines, external devices, non-Arch distros, broad community Q&A, and call/gaming apps), weighted toward 2024-2026 and non-Arch sources, none of them shown the clusters or the existing entries. Every result was then classified against the existing 18 clusters. + +Result: zero of 108 needed a new cluster. Roughly 70 re-found entries already in the catalogue (many hitting the exact same bug and source URL we had cited independently), which is entry-level saturation. The other ~34 were genuinely new distinct root causes, and all of them slotted into existing clusters. The new results spread across every cluster and concentrated where the original catalogue did (cluster 1 firmware/driver, and the app cluster). The app cluster grew densest, which confirms the Chrome incident that started this was representative, not a fluke. + +Conclusion: the clusters — what the doctor's probe ladder and verdict structure rest on — are complete and stable. Entries will keep growing (hardware × kernel × app is unbounded), but the design can rest on the taxonomy. The ~34 new entries the sweep surfaced are folded in below, under a "Saturation resample additions" heading in each direction, each carrying its cluster and remedy class inline. + +* Input (microphone) failure modes + +Built 2026-07-10 from a parallel research sweep across eight layers (kernel/driver/firmware, ALSA, PipeWire/WirePlumber, Bluetooth, USB, app/portal, hardware, config/misc). 64 raw entries deduped to 58 distinct root causes. Each names the layer, the user-reported symptom, the cause, the known fix (with the concrete command where one exists), whether the fix needs root or a reboot, and a source. The =sudo/reboot= field is the first triage signal: a no-sudo, no-reboot fix is a candidate for an auto or guide remedy; a both/physical fix can only ever be a guide. + +** Kernel / driver / firmware + +- Missing SOF firmware for a DSP-attached mic (both). Internal mic not recognised on a modern Intel laptop; speakers fine. Digital mic hangs off the Intel DSP and needs Sound Open Firmware. Fix: =pacman -S sof-firmware alsa-ucm-conf=, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=287361][arch 287361]] +- Stale =dmic_detect=0= / =dsp_driver=1= modprobe override (both). Mic dead after an old "fix"; no DMIC input. A leftover =/etc/modprobe.d= option forces the legacy HDA path and bypasses the SOF driver that owns the digital mic. Fix: delete the override, install sof-firmware, rebuild initramfs, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=260827][arch 260827]] +- Missing codec model / ADC quirk, ALC236/ALC256 (both). Capture controls visible and unmuted, but =arecord= records silence. =snd_hda_intel= lacks the board-specific pin/ADC-gain quirk, so the ADC is never routed. Fix: =options snd-hda-intel model=...= workaround; real fix is a kernel patch_realtek entry. [[https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/2142994][launchpad 2142994]] +- Kernel-version regression broke a working mic (both). Internal mic recorded fine until a kernel update, then stopped. A regression between builds broke ALC256 capture routing. Fix: boot the last-good kernel or install linux-lts until a fixed build lands. [[https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2117316][launchpad 2117316]] +- AMD ACP6x DMIC needs a DMI quirk, not firmware (both). Ryzen laptop internal array mic absent or silent; sof-firmware changes nothing. The acp6x mic array needs the laptop's DMI entry in the =SND_SOC_AMD_YC_MACH= quirk table. Fix: newer kernel with the board's DMI quirk, or add the entry and rebuild. [[https://bbs.archlinux.org/viewtopic.php?id=311093][arch 311093]] +- Missing NHLT ACPI table in firmware (both). No DMIC; dmesg shows "NHLT table not found". The platform firmware omits the NHLT table, so SOF can't discover the digital-mic count. Fix: flash firmware that supplies NHLT, or override the dmic count via SOF modprobe params. [[https://github.com/MrChromebox/firmware/issues/920][mrchromebox 920]] +- Brand-new AMD ACP7.0 (Strix Halo) has no mic driver/UCM yet (both). Internal DMIC non-functional on very recent AMD hardware. The kernel ACP7.0 SOF driver and UCM profile weren't upstreamed yet. Fix: wait for the kernel + alsa-ucm-conf that add ACP7.0 support. [[https://github.com/alsa-project/alsa-ucm-conf/issues/745][ucm-conf 745]] + +** ALSA (profiles, UCM, mixer, pins, state) + +- Wrong card profile, no input route (no). Speakers work, no capture device appears, only a ".monitor". The card sits on an output-only profile so ALSA never exposes the capture PCM. Fix: =pactl set-card-profile <card> output:analog-stereo+input:analog-stereo=. [[https://bbs.archlinux.org/viewtopic.php?id=269173][arch 269173]] +- ACP vs UCM detection mismatch (no). alsa-card-profiles fixes the mic but PipeWire breaks it again. WirePlumber's ACP probing (or an auto-selected UCM) mis-detects the capture ports. Fix: set =api.alsa.use-acp = false= (or =use-ucm = true=) in WirePlumber, restart wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=291807][arch 291807]] +- Capture / Internal Mic switch muted in mixer (no). Mic recognized but records silence. The "Capture" or "Internal Mic" mixer switch is toggled off in ALSA state. Fix: =amixer set Capture cap= (or Space on the item in alsamixer). [[https://wiki.archlinux.org/title/Advanced_Linux_Sound_Architecture/Troubleshooting][archwiki alsa]] +- Input Source enum on the wrong pin (no). Plugged-in mic silent; the wrong jack is live. The "Input Source" control selects a different pin (Rear vs Front Mic). Fix: =amixer set 'Input Source' 'Front Mic'=. [[https://bbs.archlinux.org/viewtopic.php?id=145531][arch 145531]] +- alsactl restores a muted capture at boot (yes). Mic works after manual unmute, muted again every reboot. alsa-restore replays a saved state with capture muted. Fix: unmute, then =sudo alsactl store=. [[https://bbs.archlinux.org/viewtopic.php?id=183211][arch 183211]] +- Internal Mic Boost at zero (no). Mic works but far too quiet to use. The "Mic Boost" gain sits at 0. Fix: =amixer set 'Internal Mic Boost' 100%=. [[https://forum.manjaro.org/t/microphone-array-input-too-quiet-alsamixer-mic-boost-not-changing-anything/90904][manjaro 90904]] +- Auto-Mute Mode disabling capture (no). Internal mic cuts out, often tied to headphone-jack state. The codec's "Auto-Mute Mode" mutes internal capture when a jack is sensed. Fix: =amixer set 'Auto-Mute Mode' Disabled=. [[https://wiki.archlinux.org/title/Advanced_Linux_Sound_Architecture/Troubleshooting][archwiki alsa]] +- Wrong default capture in .asoundrc (no). Apps with no picker record from the wrong card. =pcm.!default= points at the wrong capture card. Fix: set =pcm.!default= to the right =plughw= in =~/.asoundrc=; find it with =arecord -l=. [[https://bbs.archlinux.org/viewtopic.php?id=274238][arch 274238]] +- Missing UCM capture profile for the codec (no). No input profile offered at all for the built-in mic. alsa-ucm-conf lacks a UCM definition for the card. Fix: fall back to ACP (=use-ucm = false=) or install/patch the card's UCM config. [[https://github.com/alsa-project/alsa-ucm-conf/issues/594][ucm-conf 594]] + +** PipeWire / WirePlumber + +- WirePlumber fails to start on old/empty config (no; sudo only if under /etc). No audio devices at all after an update; wpctl lists nothing. WirePlumber won't parse a leftover 0.4-era config or empty drop-in and dies. Fix: remove the offending config, =systemctl --user restart wireplumber=. [[https://bbs.archlinux.org/viewtopic.php?id=294649][arch 294649]] +- Capture source muted on startup, stale WP state (no). Mic dead every boot until you mute then unmute it. Stale state in =~/.local/state/wireplumber= restores the source muted. Fix: stop wireplumber, =rm -r ~/.local/state/wireplumber=, start; or =wpctl set-mute <id> 0=. [[https://bbs.archlinux.org/viewtopic.php?id=297727][arch 297727]] +- Default source is a monitor / wrong device, and not persisted (no). Apps pick up silence or loopback; the "mic" is a sink Monitor, or the default resets to the wrong card each reboot. WirePlumber's auto-selection picks the wrong node and doesn't persist a manual default. Fix: =wpctl set-default <mic-id>=, and pin it with a node.name rule to survive reboot. [[https://bbs.archlinux.org/viewtopic.php?id=285076][arch 285076]] +- pipewire-pulse dead or wrong socket (no). pactl apps report "connection refused"; pw-cat may still work. pipewire-pulse isn't running, or XDG_RUNTIME_DIR is wrong so =/run/user/1000/pulse/native= is missing. Fix: correct XDG_RUNTIME_DIR, =systemctl --user restart pipewire pipewire-pulse wireplumber=. [[https://bbs.archlinux.org/viewtopic.php?id=281062][arch 281062]] +- Node suspended, never resumes (no). Mic works once, then silent when idle; drop when an app reopens it. =session.suspend-timeout-seconds= parks the source and it fails to wake on some 0.5.x builds. Fix: drop-in setting =session.suspend-timeout-seconds = 0=, restart wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=309630][arch 309630]] +- A filter/virtual source shadows the real mic (no). Only a virtual "echo-cancel" or EasyEffects source shows; apps capture nothing, or apps get the raw unfiltered mic instead of the processed one. module-echo-cancel / filter-chain / EasyEffects creates a virtual source with the wrong node.target or default. Fix: fix node.target, or =wpctl set-default <right-id>=. [[https://docs.pipewire.org/page_module_echo_cancel.html][pw echo-cancel]] +- Broken ~/.config/wireplumber override disables the node (no). Internal mic stops defaulting after a config edit; system configs seem ignored. A malformed or disabling user drop-in shadows the system defaults. Fix: remove/repair the drop-in, restart wireplumber. [[https://discussion.fedoraproject.org/t/wireplumber-not-connecting-microphone/63354][fedora 63354]] +- Forced or mismatched sample rate locks the source (no). Mic stuck at 48 kHz, or silent, or apps fail to open it; also seen on USB interfaces that only do 48k. A global =default.clock.rate= / =node.force-rate= pins the graph to a rate the device or app can't negotiate, with no allowed-rates fallback. Fix: set =default.clock.allowed-rates= to a list rather than forcing one, restart pipewire; runtime =pw-metadata -n settings 0 clock.force-rate 0=. [[https://bbs.archlinux.org/viewtopic.php?id=288932][arch 288932]] + +** Bluetooth + +- Headset stuck in A2DP, no mic input (no). Headphones play great but the mic never appears. A2DP is sink-only, no microphone source. Fix: =pactl set-card-profile <bt-card> headset-head-unit=. [[https://bbs.archlinux.org/viewtopic.php?id=280718][arch 280718]] +- oFono installed but unconfigured, HFP profile absent (yes, to remove pkg). No HFP/HSP profile listed after moving to PipeWire. oFono grabs the HFP backend but is never configured. Fix: remove oFono (=pacman -R ofono=) or set =bluez5.hfphsp-backend = "native"=, restart pipewire. [[https://bbs.archlinux.org/viewtopic.php?id=269195][arch 269195]] +- mSBC codec not enabled / unavailable (no). Headset won't offer mSBC; mic is muffled CVSD or missing. mSBC role/codec not enabled in WirePlumber. Fix: set =bluez5.codecs = [ sbc sbc_xq msbc ]= and enable =hfp_hf=, restart wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=266252][arch 266252]] +- Auto-switch to headset profile never triggers (no). Open a call app and the mic stays dead; nothing flips to HFP. WirePlumber's autoswitch isn't firing. Fix: confirm autoswitch is on, or switch manually with set-card-profile. [[https://forum.endeavouros.com/t/my-bluetooth-headphone-mic-not-reconisied-after-switching-to-pipewire/17586][endeavour 17586]] +- Codec negotiation silently drops to baseline (no). Mic works but quality is terrible; preferred codec never applies. Negotiation fails and PipeWire quietly falls back. Fix: pin codec order in a WirePlumber =bluez5.codecs= rule, restart wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=300230][arch 300230]] +- Adapter/headset rejects hsp_ag and hfp_ag together (no). Enabling both roles breaks the profile on some headsets (Sony WH-1000XM3). Fix: restrict roles to =hfp_hf=/=hfp_ag= (drop =hsp_ag=), restart wireplumber. [[https://pipewire.pages.freedesktop.org/wireplumber/daemon/configuration/bluetooth.html][wp bluetooth]] +- BT mic invisible to arecord / /proc/asound (no). ALSA tools show no Bluetooth mic, so the user thinks it's undetected. The bluez5 mic is a PipeWire node, not an ALSA card, by design. Fix: list it with =wpctl status=, record with =pw-record=. [[https://wiki.archlinux.org/title/PipeWire][archwiki pipewire]] +- Switching to the mic profile tanks playback quality (no; expected). Music turns to tinny phone-call audio the moment the mic works. HFP shares narrow bandwidth for duplex. Fix: expected tradeoff; switch back to A2DP off-call, or enable mSBC to soften it. [[https://webcodr.io/2024/10/fixing-no-a2dp-with-bluetooth-headsets-on-linux/][webcodr]] + +** USB microphones + +- USB autosuspend powers the mic down while idle (both). Mic works at first, then silent or vanishes after sitting unused. Kernel autosuspend puts the idle device to sleep and it never wakes cleanly. Fix: a udev rule pinning =power/control=on= for the device, or =usbcore.autosuspend=-1=. [[https://bbs.archlinux.org/viewtopic.php?id=199504][arch 199504]] +- Focusrite Scarlett needs a quirk / device_setup (both). XLR interface enumerates but capture is broken or wrong-channel. The device mis-reports config; snd-usb-audio needs a quirk (or the scarlett2 driver, built-in from 6.7). Fix: =options snd_usb_audio ... device_setup=1= on older kernels, or run kernel >= 6.7. [[https://blog.rtrace.io/posts/fedora-support-focusrite-scarlett/][rtrace scarlett]] +- Intermittent disconnect with URB resets (no). Mic drops mid-use; dmesg shows disconnect and "cannot submit urb". A flaky bus/hub/cable link causing URB failures. Fix: use a direct motherboard port or powered hub, swap the cable. [[https://linux-usb.vger.kernel.narkive.com/MdlWpDZG/cannot-submit-urb-0-error-22-internal-error-followed-by-usb-hung-tasks][linux-usb list]] +- Bus/hub can't supply enough power (no). Interface fails to power up or drops out, often through an unpowered hub. A bus-powered interface exceeds what a USB2 port or passive hub delivers. Fix: powered hub or the interface's PSU; a direct rear port. [[https://bbs.archlinux.org/viewtopic.php?id=267926][arch 267926]] +- Mic dead after boot until physically replugged (both). USB mic gives no input after cold boot; replug fixes it every time. Ports stay powered through shutdown, so the device never power-cycles to enumerate at boot. Fix: enable ErP / "power off USB on shutdown" in BIOS. [[https://bbs.archlinux.org/viewtopic.php?id=280206][arch 280206]] +- WirePlumber cached a no-input card profile for the USB device (no). Card present but no capture source appears. WirePlumber remembers a profile with input off, or a corrupt state cache. Fix: =pactl set-card-profile <card> input:...=; if it won't stick, clear =~/.local/state/wireplumber=. [[https://bbs.archlinux.org/viewtopic.php?id=294444][arch 294444]] + +** Application / browser / portal / sandbox + +- Wayland browser returns an empty mic list to WebRTC (no). Google Meet shows no microphone, no permission prompt (Firefox/Chrome/Zen on Wayland). The WebRTC PipeWire capturer isn't active so enumerateDevices() yields nothing. Fix: launch with =--enable-features=WebRTCPipeWireCapturer --ozone-platform=wayland= (Chromium) or the portal prefs (Firefox). [[https://github.com/zen-browser/desktop/issues/9745][zen 9745]] +- Flatpak app lacks device / pulse permission (no). Discord (Flatpak) can't hear the mic. The sandbox withholds the mic; no =--device= or pulse access. Fix: =flatpak override --user --device=all <app>= (or Flatseal). [[https://lionir.ca/flathub-discord-permissions-explained/][flathub discord]] +- Flatpak app has no audio socket exposed (no). Zoom/Viber Flatpak sees no mic at all. The sandbox lacks the audio socket (=--socket=pulseaudio= / =xdg-run/pipewire-0=). Fix: =flatpak override --user --socket=pulseaudio <app>=. [[https://forums.linuxmint.com/viewtopic.php?t=456895][mint 456895]] +- Snap audio-record interface not connected, or AppArmor mediation (yes). Mumble/OBS snap gets no mic; or pipewire-pulse breaks on a confined client's connect. The =audio-record= interface is off by default and doesn't auto-connect; the AppArmor/LSM path can block or crash. Fix: =sudo snap connect <snap>:audio-record=, or the pipewire-pulse SystemCallFilter drop-in. [[https://forum.snapcraft.io/t/i-am-unable-to-record-audio-from-several-snaps/18703][snapcraft 18703]] +- xdg-desktop-portal not started under Hyprland (no; relogin). Screen/mic capture silently fails, no portal prompt on Hyprland. The portal needs =graphical-session.target=, which a bare Hyprland exec never raises. Fix: import the environment and raise the session target in the Hyprland startup. [[https://github.com/hyprwm/Hyprland/discussions/10841][hyprland 10841]] +- Permission-store caches a stale "deny" (no). An app was denied once and never re-prompts. The portal permission store persists the earlier "no". Fix: =flatpak permission-reset <app-id>= (or =permission-remove devices microphone=). [[https://man.archlinux.org/man/flatpak-permission-reset.1.en][flatpak permission-reset]] +- Browser per-site mic permission set to Block (no). A site says mic blocked though the OS mic works. Chrome stores a per-origin "Not allowed" rule. Fix: Site settings, reset Microphone to Ask/Allow. [[https://support.google.com/chrome/answer/2693767][chrome mic]] +- An app grabs the mic exclusively, starving others (no). While OBS/Discord runs, other apps get no mic; or arecord reports the device busy. The app opens the node (or raw ALSA hw) in exclusive mode. Fix: point the app at shared pipewire capture instead of the raw device source. [[https://obsproject.com/forum/threads/mic-echos-into-all-outputs-arch-pipewire.150207/][obs 150207]] + +** Hardware / physical / BIOS + +- Hardware mic-mute / privacy kill-switch (physical). All ALSA levels up and unmuted, still no signal. A physical slider or EC-level mute holds capture off below software. Fix: toggle the physical switch; some models need a BIOS re-enable. [[https://bbs.archlinux.org/viewtopic.php?id=182671][arch 182671]] +- Laptop mic-mute key not honored, e.g. ThinkPad Fn+F4 (no). The LED toggles but the key never mutes/unmutes, and pactl is out of sync. The key maps to F20 and the EC mute isn't wired to the audio stack. Fix: bind keycode 190 (F20) to a pactl source-mute toggle. [[https://bbs.archlinux.org/viewtopic.php?id=268269][arch 268269]] +- Combo jack keeps the internal mic selected (yes, if model option). Headset plugged in but the internal mic stays active. The shared combo jack doesn't expose "Headset Microphone"; an auto-mute/model quirk. Fix: pick "Headset Microphone" in pavucontrol, or =model=headset-mode= / disable Auto-Mute. [[https://bbs.archlinux.org/viewtopic.php?id=252594][arch 252594]] +- Jack retasked to the wrong function (both). A mic port never registers a device, or is seen as line-out. A buggy BIOS pin-config assigns the jack the wrong default function. Fix: =hdajackretask=, override the pin to Microphone, install boot override, reboot. [[https://fossies.org/linux/alsa-tools/hdajackretask/README][hdajackretask]] +- TRRS wiring mismatch, CTIA vs OMTP (physical). Audio plays but the headset mic is dead. The headset is wired to the opposite standard (mic/ground swapped). Fix: an OMTP-to-CTIA adapter, or a USB dongle. [[https://forums.linuxmint.com/viewtopic.php?t=336989][mint 336989]] +- Broken / loose 3.5mm jack (physical). Mic cuts in/out when the cable is flexed, or nothing detected. Bent contacts or a cracked solder joint. Fix: confirm on another device; use a USB dongle if the jack is dead. [[https://forums.linuxmint.com/viewtopic.php?t=452191][mint 452191]] +- A webcam's built-in mic is a separate USB device (yes, for udev rule). The system switches to low-quality audio from the webcam the user didn't know had a mic. The webcam enumerates as its own USB Audio Class interface. Fix: select the intended mic as default, or blacklist the webcam's audio interface with a udev rule. [[https://www.mjt.me.uk/posts/blacklisting-certain-microphones-linux/][mjt blacklist]] +- BIOS/UEFI disables the internal mic or audio controller (both). No capture device present at all in Linux. A firmware setting turns off the mic or the audio controller. Fix: re-enable it in BIOS setup. [[https://www.dell.com/community/en/conversations/inspiron/bios-setting-to-disable-onboard-microphone/7331179][dell inspiron]] + +** Config / permissions / misc + +- PulseAudio and pipewire-pulse both installed (yes). Mic worked on PulseAudio, took no input after switching; both seem to run. Leftover pulseaudio auto-spawns and grabs the device, fighting pipewire-pulse. Fix: remove pulseaudio, set =autospawn = no=, enable =pipewire-pulse.socket=. [[https://bbs.archlinux.org/viewtopic.php?id=283345][arch 283345]] +- User lacks realtime privileges / RTKit denied (both). Mic crackles or drops; logs show "could not make thread realtime using RTKit". No polkit session and user not in the realtime group. Fix: =pacman -S realtime-privileges=, add user to =realtime=, re-login. [[https://forum.manjaro.org/t/pulseeffects-cant-get-realtime-priority/42584][manjaro 42584]] +- XDG_RUNTIME_DIR unset in ssh/cron/non-login session (no). Recording works in the desktop terminal but fails from ssh or cron. Non-login sessions skip pam_systemd, so the socket in =/run/user/<uid>= isn't found. Fix: run as the seat user with =XDG_RUNTIME_DIR= set. [[https://bbs.archlinux.org/viewtopic.php?id=286176][arch 286176]] +- Auto-gain (AGC) pulling the mic level toward zero (no). The mic gets progressively quieter; something keeps lowering the input volume. An app or WirePlumber is allowed to auto-adjust the source gain and only ratchets down. Fix: a WirePlumber access rule blocking the client from changing gain. [[https://www.lumeh.org/wiki/audio/stop-adjusting-my-microphone/][lumeh]] + +** Saturation resample additions (2026-07-10) + +Genuinely-new input root causes from the blind resample, each tagged with the cluster it fits and its remedy class (so this doubles as their triage). Zero forced a new cluster. + +- Mic boost too high, records static/distortion (cluster 3 · AUTO). The inverse of boost-at-zero. Fix: lower Mic Boost. [[https://bbs.archlinux.org/viewtopic.php?id=211318][arch 211318]] +- App auto-gain zeroes the mic, Zoom/Discord (cluster 7 · GUIDE). The app's own auto-input-adjust drives the level to 0. Fix: disable "automatically adjust microphone volume" / input-sensitivity. [[https://community.zoom.com/t5/Meetings/Zoom-for-Linux-not-working-well-with-newer-distributions/m-p/22805][zoom community]] +- App selects the wrong input, "monitor of output", Teams (cluster 7 · GUIDE). Fix: set the device before launch, avoid the in-call chooser. [[https://learn.microsoft.com/en-us/answers/questions/183216/in-ms-teams-for-linux-microphone-stopped-working][ms teams]] +- App requires PulseAudio, fails on bare ALSA, Signal/Webex (cluster 7 · GUIDE). WebRTC won't enumerate devices without the Pulse layer. Fix: install pipewire-pulse. [[https://github.com/signalapp/Signal-Desktop/issues/5099][signal 5099]] +- Browser WebRTC captures one channel only, Jitsi/Firefox (cluster 7 · GUIDE). Firefox-specific channel handling. Fix: Chrome as workaround. [[https://github.com/jitsi/jitsi-meet/issues/16828][jitsi 16828]] +- Mic classified as an output device, Pop COSMIC (cluster 2 · GUIDE). A desktop-environment device-role bug. Fix: none confirmed; tracked upstream. [[https://github.com/pop-os/cosmic-epoch/issues/1368][cosmic 1368]] +- SOF topology-file mismatch, DMIC visible but silent (cluster 1 · PRIV, REBOOT-TAIL). The firmware is present but the wrong .tplg loads. Fix: swap the topology file in firmware-sof. [[https://bugzilla.redhat.com/show_bug.cgi?id=1772498][rh 1772498]] +- Webcam mic needs a USB init-delay quirk, Logitech C922 (cluster 1 · REBOOT-TAIL). Fix: kernel with USB_QUIRK_DELAY_INIT for the device. [[https://bugs.launchpad.net/bugs/1885259][lp 1885259]] +- USB interface implicit-feedback / clock-source stutter, Behringer UMC (cluster 4 · REBOOT-TAIL). Fix: =implicit_fb= modprobe, or kernel 5.19+. [[https://nandakumar.org/blog/2023/01/upgrade-linux-for-umc202hd.html][nandakumar]] +- USB dual-channel device won't do input and output at once until a profile toggle, Elgato Wave (cluster 4 · AUTO). Fix: toggle the Pro Audio vs Duplex profile. [[https://forum.level1techs.com/t/elgato-wave-3-fails-to-work-as-microphone/221807][level1techs]] +- USB mic not reliably enumerated, a port/controller issue or a PipeWire add-race (cluster 4 · AUTO). Fix: restart PipeWire, or a different port. [[https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/1534][pw 1534]] +- PipeWire buffer/scheduling failure, source active but no samples (cluster 5 · GUIDE). "port_use_buffers" / "scheduling stopped node" in the log. Fix: unresolved upstream. [[https://github.com/NixOS/nixpkgs/issues/271847][nixpkgs 271847]] +- Sudo-run audio tools corrupt the WirePlumber state (cluster 9 · AUTO). Fix: =rm -r ~/.local/state/wireplumber=; never run audio tools under sudo. [[https://www.kubuntuforums.net/forum/currently-supported-releases/kubuntu-24-04-nitpick-noble-lts/hardware-support-bg/681510-kubuntu-24-04-microphone-volume-at-100-per-cent-on-every-reboot][kubuntu]] +- Audio-group membership blocks per-session access, older udev/ConsoleKit distros (cluster 9 · PRIV). Fix: remove the user from the =audio= group. [[https://wiki.ubuntu.com/Audio/TheAudioGroup][ubuntu audio group]] +- Windows fast-startup wedges audio on a dual-boot (cluster 9 · GUIDE). Fix: disable Windows fast boot, or reload the sof module. [[https://bbs.archlinux.org/viewtopic.php?id=279824][arch 279824]] +- Dead mic transducer / faulty mic module (cluster 8 · GUIDE). Fix: hardware defect, replaced under warranty. [[https://community.frame.work/t/fedora-41-13th-gen-intel-core-microphone-not-working/64010][framework 64010]] + +* Input triage + +The 58 input failure modes sorted into nine symptom clusters, each owned by a probe the doctor runs (or would run). Every entry carries a remedy class: + +- *AUTO* — user-scope, reversible, runs anywhere and on any machine. The existing Auto/Confirm tiers apply. +- *PRIV* — needs sudo. Runs silently where passwordless sudo exists (every archsetup install), prompts on a CLI with a tty, degrades to Guide only in a GUI with neither. Defaults to Confirm/Arm, never silent Auto. +- *REBOOT-TAIL* — the applicable part runs (often PRIV), then the doctor instructs the reboot it cannot complete or verify past. +- *GUIDE* — nothing to run: a physical action, a BIOS setting, an app-permission the doctor can't grant, or a wait-for-upstream fix. Diagnose and print the instruction. + +Rough distribution: ~26 AUTO, ~6 PRIV, ~6 REBOOT-TAIL, ~20 GUIDE (a handful are AUTO now plus PRIV to persist). The buildable share (AUTO plus the runnable part of PRIV/REBOOT-TAIL) is a little over half. Triage is a proposal — correct the class on any line. + +** Cluster 1 — no capture device at all +Probe: kernel =/proc/asound= plus the unit tier. New work: dmesg-pattern hints would let the doctor name the specific cause (firmware-load-fail, NHLT-missing) rather than a generic "no hardware". + +- Missing SOF firmware :: REBOOT-TAIL — install =sof-firmware= (PRIV), reboot to finish, verify after. +- Stale dmic_detect / dsp_driver modprobe override :: REBOOT-TAIL — remove the override, rebuild initramfs (PRIV), reboot. +- Missing codec model / ADC quirk :: REBOOT-TAIL — the =model=...= modprobe workaround (PRIV) plus reboot; GUIDE the real kernel-quirk fix. +- Kernel regression broke the mic :: GUIDE — boot the previous kernel or install linux-lts; the doctor can't choose your boot kernel. +- AMD ACP6x DMIC DMI quirk :: GUIDE — needs a newer kernel carrying the board quirk; nothing to run. +- Missing NHLT ACPI table :: GUIDE — flash firmware that supplies NHLT; out of reach. +- New AMD ACP7.0 unsupported :: GUIDE — wait for the kernel plus UCM support. +- Focusrite Scarlett needs a quirk :: REBOOT-TAIL — =device_setup= modprobe (PRIV) plus reboot on old kernels; GUIDE "run kernel >= 6.7" otherwise. + +** Cluster 2 — device present but records silence +Probe: semantic tier (source mute, volume, pin, default). This is the input mirror of the output side's silenced and stale-default rules. + +- Wrong card profile, no input route :: AUTO — =set-card-profile= to a duplex/input profile. +- Capture / Internal Mic switch muted :: AUTO — unmute via wpctl/pactl (and the ALSA capture switch via amixer). +- Input Source enum on the wrong pin :: AUTO — set Input Source to the connected pin. +- alsactl restores a muted capture at boot :: AUTO + PRIV — unmute now (AUTO); persist with =sudo alsactl store= (PRIV). +- Auto-Mute Mode disabling capture :: AUTO — set Auto-Mute Disabled. +- Wrong default capture in .asoundrc :: AUTO — set the right default source (the PipeWire-era fix, not the legacy file). +- Default source is a monitor / wrong device :: AUTO — =set-default= to the real mic, pin it. +- Capture source muted on startup (stale WP state) :: AUTO — unmute; clear the stale WirePlumber state. +- Filter / virtual source shadows the real mic :: AUTO — =set-default= to the real source, or fix node.target. + +** Cluster 3 — works but too quiet +Probe: semantic volume/gain. + +- Internal Mic Boost at zero :: AUTO — raise Mic Boost. +- Auto-gain (AGC) pulling the level to zero :: AUTO — a WirePlumber rule blocking the client from changing gain. + +** Cluster 4 — works then dies or drops out +Probe: node state over time / suspend. New work: a re-probe-after-idle check to catch the suspend cases. + +- Node suspended, never resumes :: AUTO — =session.suspend-timeout-seconds = 0= drop-in plus restart. +- Forced or mismatched sample rate :: AUTO — =allowed-rates= config plus restart. +- USB autosuspend powers the mic down :: PRIV — a udev =power/control=on= rule (or =usbcore.autosuspend=-1=, REBOOT-TAIL). +- Intermittent USB disconnect (URB resets) :: GUIDE — swap the cable or use a direct port; physical. +- USB bus/hub can't supply power :: GUIDE — a powered hub or the interface PSU; physical. +- Mic dead after boot until replug :: GUIDE — enable ErP in BIOS so the port power-cycles; physical/BIOS. + +** Cluster 5 — the sound server lost a device the hardware has +Probe: kernel-vs-graph (the mic-unrecognized rule) plus WirePlumber liveness. This is where the spec's coarse set-emptiness rule lives. + +- ACP vs UCM detection mismatch :: AUTO — set =use-acp=/=use-ucm= plus restart wireplumber. +- Missing UCM capture profile :: AUTO — fall back to ACP; GUIDE the UCM patch/install. +- WirePlumber fails to start on old config :: AUTO — remove the user drop-in plus restart (PRIV if the file is under =/etc=). +- pipewire-pulse dead or wrong socket :: AUTO — restart the user services; check XDG_RUNTIME_DIR. +- Broken ~/.config/wireplumber override :: AUTO — remove/repair the drop-in plus restart. +- WirePlumber cached a no-input profile (USB) :: AUTO — =set-card-profile input:...=; clear stale state if it won't stick. + +** Cluster 6 — Bluetooth mic +Probe: bluez card profile plus codec. Needs the graph-first gate from the review so a BT mic never reads as no-mic-hardware. + +- Headset stuck in A2DP, no mic :: AUTO — =set-card-profile headset-head-unit=. +- oFono installed but unconfigured :: PRIV — remove oFono (or set the native backend in config, AUTO). +- mSBC codec not enabled :: AUTO — codecs config plus restart wireplumber. +- Auto-switch to headset profile never triggers :: AUTO — set-card-profile / autoswitch config. +- Codec negotiation drops to baseline :: AUTO — pin codec order. +- Adapter rejects hsp_ag plus hfp_ag together :: AUTO — restrict roles in config. +- BT mic invisible to arecord :: GUIDE — not a fault; it's a PipeWire node. The graph-first gate prevents a false no-mic-hardware here. +- Switching to the mic profile tanks playback :: GUIDE — expected A2DP/HFP tradeoff. + +** Cluster 7 — stack healthy, an app can't hear me +Probe: none in-stack. This is the tail on a healthy verdict: name the likely app/portal cause and print the fix. The Chrome incident lives here. + +- Wayland browser returns an empty mic list :: GUIDE — the WebRTC/PipeWire launch flags or browser prefs. +- Flatpak app lacks device permission :: GUIDE — print =flatpak override --user --device=all <app>=. +- Flatpak app has no audio socket :: GUIDE — print =flatpak override --user --socket=pulseaudio <app>=. +- Snap audio-record interface not connected :: GUIDE — print =sudo snap connect <snap>:audio-record=. +- xdg-desktop-portal not started under Hyprland :: GUIDE — fix the Hyprland startup env / session target. +- Permission-store cached a deny :: GUIDE — print =flatpak permission-reset <app>=. +- Browser per-site mic block :: GUIDE — the browser's site settings. +- An app grabs the mic exclusively :: GUIDE — change the app to shared capture. + +** Cluster 8 — hardware / physical +Probe: kernel tier plus a signature test (issue an unmute, re-probe; a mute that doesn't clear is a hardware switch). + +- Hardware mic-mute / privacy kill-switch :: GUIDE — flip the physical switch; the doctor detects the unmute-doesn't-stick signature. +- Laptop mic-mute key not honored (F20) :: GUIDE — bind the key to a pactl toggle (a config change). +- Combo jack keeps the internal mic selected :: AUTO + PRIV — select Headset Mic (AUTO); the =model== option to persist (PRIV, reboot). +- Jack retasked to the wrong function :: REBOOT-TAIL — hdajackretask override plus install plus reboot. +- TRRS wiring mismatch (CTIA/OMTP) :: GUIDE — a wiring adapter; physical. +- Broken / loose 3.5mm jack :: GUIDE — physical/hardware. +- Webcam's built-in mic is a separate device :: AUTO + PRIV — =set-default= away from the webcam (AUTO); a udev blacklist to persist (PRIV). +- BIOS/UEFI disables the internal mic :: GUIDE — re-enable it in BIOS; physical/firmware. + +** Cluster 9 — environment / config +Probe: process plus environment checks (a leftover pulseaudio process, XDG_RUNTIME_DIR, group membership). + +- PulseAudio and pipewire-pulse both installed :: PRIV — remove pulseaudio, set =autospawn = no=. +- User lacks realtime privileges / RTKit :: PRIV — =limits.d= plus the realtime group (relogin/reboot). +- XDG_RUNTIME_DIR unset in ssh/cron :: GUIDE — set it in the non-login context; environment-specific. + +* Output (speakers / headphones) failure modes + +Built 2026-07-10 from a parallel sweep across eight output-specific layers (kernel/driver/firmware, ALSA mixer/pin, PipeWire routing, Bluetooth, HDMI/DisplayPort, USB DAC + hardware/BIOS, app/browser, and audio quality: crackle/xrun/latency). 64 raw entries deduped to 59 distinct root causes. Same fields and =sudo/reboot= triage signal as the input section. + +** Kernel / driver / firmware + +- SOF DSP firmware not installed (both). Fresh install shows only "Dummy Output"; nothing in =/proc/asound/cards=; dmesg "failed to load DSP firmware". The Intel DSP needs Sound Open Firmware to boot. Fix: =pacman -S sof-firmware=, reboot. [[https://github.com/basecamp/omarchy/issues/6110][omarchy 6110]] +- Cirrus CS35L41 smart amp, missing ACPI _DSD (both). Card present, speakers dead on 2023 ASUS/HP; dmesg "ACPI _DSD Properties are missing for HID CSC3551". BIOS omits the props the =cs35l41-hda= driver needs, and the amp firmware is required. Fix: install =firmware-cirrus=, apply the _DSD override, or kernel >= 6.9 / BIOS update. [[https://asus-linux.org/guides/cirrus-amps/][asus-linux cirrus]] +- TI TAS2781 smart amp, driver/firmware absent (both). Lenovo Legion speakers silent; "Direct firmware load for TIAS2781RCA2.bin failed". The =SND_HDA_SCODEC_TAS2781= driver isn't built and the RCA firmware is undistributable. Fix: kernel with the option enabled + TI firmware in =/lib/firmware=. [[https://bbs.archlinux.org/viewtopic.php?id=289379][arch 289379]] +- Wrong HDA model autodetection, speaker dead (both). ALSA works but the internal speaker is silent; only headphone/HDMI plays. =snd_hda_intel= picked the wrong codec model so the speaker pin/amp isn't enabled. Fix: =options snd-hda-intel model=<quirk>= in =/etc/modprobe.d/=. [[https://bbs.archlinux.org/viewtopic.php?pid=1109929][arch 1109929]] +- snd_soc_skl grabs the card, only Dummy Output (both). After a kernel update, sound gone; only "Dummy Output". The SST driver claims the controller and the DMIC probe fails, leaving no playback path. Fix: =options snd-hda-intel dmic_detect=0= + =blacklist snd_soc_skl=. [[https://www.linuxuprising.com/2018/06/fix-no-sound-dummy-output-issue-in.html][linuxuprising]] +- AMD Ryzen HDA codec probe timeout (both). No internal speakers; =/proc/asound= shows only the HDMI codec. The analog codec probe fails ("no codecs initialized") on Renoir/Cezanne. Fix: newer kernel/firmware; =probe_mask=1= / =dmic_detect=0= workaround. [[https://forums.linuxmint.com/viewtopic.php?t=448906][mint 448906]] +- Kernel regression killed HDA output (both). Speakers worked, went silent after a kernel bump. An HDA/codec regression in the new kernel. Fix: boot the previous kernel or linux-lts until a fix lands. [[https://bbs.archlinux.org/viewtopic.php?id=278370][arch 278370]] + +** ALSA (mixer, pins, jack, state) + +- Master / PCM channel muted, MM in alsamixer (no). No sound from any output. The mixer channel is left muted. Fix: =amixer sset Master unmute; amixer sset PCM unmute=. [[https://bbs.archlinux.org/viewtopic.php?id=200955][arch 200955]] +- Auto-Mute Mode silences speakers (no). Speakers muted when headphones are (un)plugged, or stay muted with nothing plugged. The codec's Auto-Mute ties speaker output to jack-sense. Fix: =amixer sset 'Auto-Mute Mode' Disabled=. [[https://bbs.archlinux.org/viewtopic.php?id=185098][arch 185098]] +- alsactl restores a muted master at boot (yes). Sound works after manual unmute, muted every reboot. alsa-restore replays a muted state, or runs before the card enumerates. Fix: unmute, =sudo alsactl store=; add a delay to alsa-restore if it races. [[https://bugs.archlinux.org/task/35438][arch bug 35438]] +- Front/Surround/Center/LFE channel muted (no). Only some speakers play. Per-channel mixer controls muted by default on multi-channel codecs. Fix: =amixer set Surround/Center/LFE 100% unmute=; verify =speaker-test -c 6=. [[https://bbs.archlinux.org/viewtopic.php?id=172793][arch 172793]] +- Dead headphone pin needs hdajackretask (both). Speakers work, headphone jack silent and undetected. A BIOS pin-config leaves the headphone pin unconnected. Fix: =hdajackretask=, override the pin to Headphone, install boot override, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=229296][arch 229296]] +- Phantom jack-sense keeps speakers muted (yes). Speakers stay muted because the codec reports headphones plugged when nothing is. A spurious/stuck jack-detect state. Fix: force re-detect via hdajackretask reconfigure, or disable jack-detect for the speaker path. [[https://bbs.archlinux.org/viewtopic.php?id=265363][arch 265363]] +- Wrong codec model quirk, no headphone or speaker (both). The auto-parser misdetects the board and picks the wrong pin routing. Fix: =options snd-hda-intel model=<quirk>= in =/etc/modprobe.d/=, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=252295][arch 252295]] +- Headphone path maps volume to the wrong control (yes). Headphone slider does nothing / headphones silent. The card-profile mixer path sets the Speaker element =volume = off=, breaking routing. Fix: edit =analog-output-headphones.conf=, change =volume = off= to =merge=, restart audio. [[https://bbs.archlinux.org/viewtopic.php?id=237456][arch 237456]] + +** PipeWire / WirePlumber routing + +- Default sink lands on HDMI instead of speakers (no). Boot with silence; playback goes to HDMI while speakers idle. WirePlumber auto-selects the HDMI node as default. Fix: =wpctl set-default <speaker-id>=. [[https://bbs.archlinux.org/viewtopic.php?id=309588][arch 309588]] +- Default keeps reverting to the wrong sink (no). set-default holds until reboot, then flips back every session. Each sink's =priority.session= re-picks the wrong default. Fix: set =priority.session= per node in a WirePlumber rule, restart wireplumber. [[https://gist.github.com/DasBen/df9c2f6c18fff54a0ad6d19304915429][dasben gist]] +- Sink suspended on idle, doesn't resume, and pops on start/stop (no). After a few seconds of silence the sink stays suspended; a loud pop/click and a delay when it wakes. =session.suspend-timeout-seconds= parks the node and the codec power-cycles. Fix: drop-in =session.suspend-timeout-seconds = 0=, restart wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=264337][arch 264337]] +- WirePlumber fails to start or a stale config jams routing (no; sudo if under /etc). Total silence; wpctl reports no sinks. The service crashes on a bad/incompatible config (user drop-in or a leftover =/etc/wireplumber/wireplumber.conf=). Fix: =journalctl --user -u wireplumber=, remove/move the offending config, restart. [[https://bbs.archlinux.org/viewtopic.php?id=294649][arch 294649]] +- Corrupted WirePlumber restore state (no). No output after an update; sink present but never usable, or wrong routing persists. A corrupt =~/.local/state/wireplumber/= wedges default-node selection. Fix: =rm -rf ~/.local/state/wireplumber/=, restart pipewire pipewire-pulse wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=290825][arch 290825]] +- pipewire-pulse dead, Pulse apps silent (no). Some apps play, but pactl/Pulse clients fail "Connection refused". The pipewire-pulse socket/service isn't running. Fix: =systemctl --user enable --now pipewire-pulse.socket pipewire-pulse.service=. [[https://bbs.archlinux.org/viewtopic.php?id=281062][arch 281062]] +- Sink present but silent until a mute/unmute toggle (no). Correct default, un-suspended, but no sound until volume is touched or mute toggled. WirePlumber restores a muted / 0-volume state, or the node never opens until a mute change forces activation. Fix: =wpctl set-mute @DEFAULT_AUDIO_SINK@ 0= and =set-volume ... 1.0=. [[https://bbs.archlinux.org/viewtopic.php?id=300427][arch 300427]] + +** Bluetooth output + +- A2DP Sink profile deselects itself / unavailable (yes). A2DP switches back off when picked; only HSP/HFP shows. Stale pairing state in =/var/lib/bluetooth= from a PulseAudio-era config confuses bluez negotiation. Fix: stop bluetooth, remove/rename =/var/lib/bluetooth=, restart, re-pair. [[https://bbs.archlinux.org/viewtopic.php?id=280717][arch 280717]] +- LDAC/aptX never negotiates, falls to SBC (no). Headphones connect but sound bad, stuck on SBC. Decoder libs absent, or =bluez5.codecs= order drops to baseline. Fix: install the codec packages, set =bluez5.codecs = [ ldac aptx_hd aptx aac sbc ]=, re-pair. [[https://bbs.archlinux.org/viewtopic.php?id=290413][arch 290413]] +- Playback drops to HSP/HFP mid-song (no). Music goes mono/tinny when an app opens the mic. WirePlumber autoswitch-to-headset-profile fires on any input stream. Fix: =wpctl settings --save bluetooth.autoswitch-to-headset-profile false=; force =a2dp-sink=. [[https://wiki.archlinux.org/title/Bluetooth_headset][archwiki bt]] +- BT sink not made default on connect (no). Headphones connect but audio keeps playing from laptop speakers. No switch-on-connect; the new sink's priority doesn't beat the built-in. Fix: =wpctl set-default <bt-sink-id>=, or a priority.session rule, or module-switch-on-connect. [[https://bbs.archlinux.org/viewtopic.php?id=304269][arch 304269]] +- Silent sink after suspend/resume despite reconnect (no). After waking, headphones reconnect but running apps play no sound. The stream isn't re-initialized on the resumed sink; a seat-monitoring race. Fix: =monitor.bluez.seat-monitoring = disabled= drop-in; workaround =systemctl --user restart wireplumber pipewire=. [[https://forums.linuxmint.com/viewtopic.php?p=2781321][mint 2781321]] +- Volume slider does nothing, absolute-volume conflict (no). OS volume has no effect on the headphones. PipeWire's absolute-volume passthrough hands volume to the device. Fix: =bluez5.enable-hw-volume = false= in a WirePlumber rule to restore software volume. [[https://wiki.debian.org/BluetoothUser/a2dp][debian a2dp]] +- Adapter/dongle never exposes an audio sink (both). Dongle pairs but no A2DP sink; dmesg shows a firmware load failure. Controller firmware missing/outdated (e.g. MediaTek MT7921). Fix: install/update linux-firmware, reload btusb or reboot. [[https://bbs.archlinux.org/viewtopic.php?id=296026][arch 296026]] + +** HDMI / DisplayPort + +- GPU HDMI codec never initializes (both). No HDMI sink appears; aplay shows no HDMI device. The DSP/legacy driver-path selection is wrong so the HDMI codec is skipped. Fix: =options snd_intel_dspcfg dsp_driver=3=, rebuild initramfs. [[https://bbs.archlinux.org/viewtopic.php?id=290824][arch 290824]] +- NVIDIA GPU audio removed by a runtime-PM udev rule (both). The NVIDIA HDMI card is missing or "inactive," only profile "off". An old NVIDIA runtime-PM udev rule powers down the audio function. Fix: comment out the audio-removal lines in the udev rule, reload/reboot. [[https://forums.developer.nvidia.com/t/no-audio-over-hdmi-linux-only/236341][nvidia 236341]] +- NVIDIA HDMI audio needs DRM modesetting (both). Video works over HDMI, no audio sink. Without nvidia-drm modeset the HDMI audio function isn't exposed. Fix: =options nvidia_drm modeset=1=, rebuild initramfs. [[https://bbs.archlinux.org/viewtopic.php?id=264504][arch 264504]] +- HDMI audio dies after suspend/resume, codec state (yes). Sound before suspend, silent after; dmesg "spurious response". The HD-Audio codec state isn't restored across resume. Fix: =sudo alsa force-reload= after resume, or =alsactl store=/=restore=. [[https://forums.developer.nvidia.com/t/nvidia-hdmi-audio-errors-after-resume-snd-hda-intel-spurious-response-last-cmd/149003][nvidia 149003]] +- Empty ELD after resume marks the port disconnected (yes). HDMI port shows unavailable post-resume; WirePlumber logs "ELD info empty". A kernel regression fails to re-read the ELD on resume. Fix: boot linux-lts until fixed; replug the cable to force an ELD re-read. [[https://bbs.archlinux.org/viewtopic.php?id=306343][arch 306343]] +- Wrong monitor is the audio sink, multi-monitor (no). Two displays on one GPU; audio goes to the wrong one. Only one HDMI/DP profile is active and the default picks the wrong port. Fix: =pactl set-card-profile <card> output:hdmi-stereo-extraN= for the right port. [[https://bbs.archlinux.org/viewtopic.php?id=295404][arch 295404]] +- HDMI sink present but silent until daemon restart (no). HDMI profile selectable but no sound until PipeWire restarts. WirePlumber didn't pick up the sink on hotplug. Fix: =systemctl --user restart pipewire pipewire-pulse wireplumber=. [[https://bbs.archlinux.org/viewtopic.php?id=280352][arch 280352]] +- HDMI sample-rate mismatch, display rejects the stream (no). HDMI audio crackles or drops on some content; the TV/monitor rejects non-48k rates. PipeWire negotiates a rate the display's ELD doesn't accept. Fix: pin the rate, =pw-metadata -n settings 0 clock.force-rate 48000= or set allowed-rates. [[https://bbs.archlinux.org/viewtopic.php?id=288932][arch 288932]] + +** USB DAC and hardware / BIOS + +- USB autosuspend powers the DAC down when idle (both). DAC plays, then silent after inactivity; sometimes needs a replug. The kernel autosuspends the port and severs the stream. Fix: =usbcore.autosuspend=-1=, or a udev rule setting =power/control=on=, or a silent keep-alive stream. [[https://destinmoulton.com/notes/howto/linux-usb-audio-keep-alive-service/][destinmoulton]] +- USB DAC not auto-selected as default output (no). Sound keeps coming from onboard after the DAC is plugged in. WirePlumber gives the DAC an equal/lower node priority. Fix: a WirePlumber rule raising =priority.session=/=priority.driver= for the device, restart wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=290341][arch 290341]] +- Kernel USB-audio regression, DAC has only an "Off" profile (both). After an upgrade the DAC (MOTU M4) enumerates but the only profile is "Off". A snd-usb-audio regression breaks stream setup for some devices. Fix: boot linux-lts or wait for the fixed kernel; downgrade as a stopgap. [[https://bbs.archlinux.org/viewtopic.php?id=279889][arch 279889]] +- DAC needs an implicit-feedback quirk to sync (both). Playback stutters or drops to silence and won't recover; the device has no explicit feedback endpoint. snd-usb-audio doesn't apply generic implicit-feedback sync. Fix: =options snd_usb_audio implicit_fb=1= (or the per-device quirk_flags bit) in =/etc/modprobe.d/=. [[https://lkml.iu.edu/hypermail/linux/kernel/2206.0/08494.html][lkml usb-audio]] +- Headphone jack not detected, speakers keep playing (both). Plugging headphones doesn't mute the speakers; no auto-switch on insert. The HDA auto-parser picks the wrong model so jack events aren't wired to the switch. Fix: =options snd-hda-intel model=<board>= in =/etc/modprobe.d/=, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=265363][arch 265363]] +- Front-panel headphone jack not wired by BIOS pin config (both). The front jack produces nothing and never appears; rear jacks work. The BIOS marks that codec pin "not connected". Fix: =hdajackretask= override the pin to Headphone/Line-out, install the boot override, reboot. [[https://github.com/alsa-project/alsa-tools/blob/master/hdajackretask/README][hdajackretask]] +- BIOS-disabled onboard audio, or a GPU HDA controller hides it (both). No card detected (empty =aplay -l=), or only the GPU/HDMI device appears. Onboard audio is disabled in BIOS, or the GPU's HDA controller takes over ordering. Fix: re-enable onboard audio in BIOS; =options snd-hda-intel enable=1,0= to disable the GPU's audio function. [[https://bbs.archlinux.org/viewtopic.php?id=221296][arch 221296]] + +** Application / browser / sandbox + +- Flatpak app has no PulseAudio socket (no). A Flatpak app is silent while native apps play. The manifest was built without =--socket=pulseaudio=. Fix: =flatpak override --user --socket=pulseaudio <app>=. [[https://github.com/flatpak/flatpak/issues/878][flatpak 878]] +- Flatpak silent on a PipeWire-only system (no). App with pulse permission still has no sound on pure PipeWire. Flatpak has no native PipeWire socket permission. Fix: =flatpak override --user --filesystem=xdg-run/pipewire-0 <app>=. [[https://github.com/flathub/com.valvesoftware.Steam/issues/1451][flathub steam 1451]] +- Snap app can't play, audio-playback interface unconnected (no). A snap (often =--dangerous=) produces no sound. Locally-installed snaps skip the auto-connect assertion. Fix: =snap connect <snap>:audio-playback=. [[https://bugzilla.mozilla.org/show_bug.cgi?id=1685284][mozilla 1685284]] +- Browser per-tab mute zeroes the app stream (no). Firefox/Chrome plays nothing; the app stream shows muted, device fine. A tab was muted, muting the per-stream sink-input. Fix: Unmute Tab, or un-mute the app's stream in pavucontrol. [[https://bbs.archlinux.org/viewtopic.php?id=275694][arch 275694]] +- Wine/Proton game silent, wrong audio driver (no; sudo for 32-bit libs). A Wine/Proton game has no output. The Wine driver is set wrong, or 32-bit audio libs are missing. Fix: winecfg output device to System default; install =lib32-libpulse= / =lib32-pipewire=. [[https://bbs.archlinux.org/viewtopic.php?id=135032][arch 135032]] +- App hard-coded to raw hw:0, device busy (no). MPD/shairport errors "Device or resource busy" and stays silent, blocking others. The app points at raw =hw:0=, grabbing the card exclusively. Fix: change the app's device to =default= or =plughw=; =fuser -v /dev/snd/*= to find the holder. [[https://community.volumio.com/t/fix-mpd-failed-to-open-alsa-device-hw-0-0-device-busy/686][volumio mpd]] +- Chromium doesn't follow the default-sink change (no). After switching the default output, Chrome keeps playing to the old sink. Chromium latches its stream to the start-time sink. Fix: reassign the Chromium stream in pavucontrol; PipeWire remembers it. [[https://bbs.archlinux.org/viewtopic.php?id=116888][arch 116888]] +- Electron app routed to the wrong sink (no). Discord is silent while YouTube works; its stream went elsewhere. Electron apps share a generic stream name so a prior routing rule misdirects it. Fix: move the app's stream to the right sink in pavucontrol while it plays. [[https://bbs.archlinux.org/viewtopic.php?id=277447][arch 277447]] + +** Audio quality (crackle, xruns, latency) + +- Quantum drops too low under load (no). Crackling/popping that worsens under CPU load. PipeWire's dynamic quantum negotiates a buffer that underruns the sink. Fix: =default.clock.min-quantum = 1024= (raise as needed) in a pipewire.conf.d drop-in, restart pipewire. [[https://bbs.archlinux.org/viewtopic.php?id=303024][arch 303024]] +- Realtime scheduling denied, RTKit (yes). Xruns/dropouts uncorrelated with CPU load; the audio thread misses deadlines. PipeWire can't get SCHED_FIFO (RTKit unavailable or capped). Fix: rtkit-daemon running, user in =realtime=, =/etc/security/limits.d/= granting rtprio + memlock; re-login. [[https://bbs.archlinux.org/viewtopic.php?id=283334][arch 283334]] +- Timer-based scheduling on a fussy USB DAC (no). Periodic crackle on a USB interface. tsched mis-estimates the batch USB device's buffer position. Fix: a WirePlumber ALSA rule with =api.alsa.disable-tsched = true= and a modest headroom, or the Pro Audio profile. [[https://bbs.archlinux.org/viewtopic.php?id=284888][arch 284888]] +- Forced global sample rate causes resample artifacts (no). Distortion on 44.1k content, or a stream stuck upsampling; also seen on USB DACs that only accept a limited rate set. A hard =default.clock.rate= forces everything through the resampler. Fix: set =default.clock.allowed-rates= to a list so the device follows the source, restart pipewire. [[https://bbs.archlinux.org/viewtopic.php?id=288932][arch 288932]] +- Small quantum fine for music, too tight for browser/games (no). Audio crackles only in Chrome/Firefox or games, clean elsewhere. The heavy client can't keep the small global quantum fed. Fix: launch with =PIPEWIRE_LATENCY=2048/48000=, or a WirePlumber stream rule setting =node.latency= for that app. [[https://forum.endeavouros.com/t/pipewire-guide-audio-crackling-popping-and-latency/69602][endeavour 69602]] +- HDA codec power-save clicks and BT-latency hits (yes). Loud click/pop on first play after idle on internal HDA; also wrecks Bluetooth latency. =snd_hda_intel power_save= cuts codec power and clicks on wake. Fix: =options snd_hda_intel power_save=0 power_save_controller=N= in =/etc/modprobe.d/=. [[https://forums.linuxmint.com/viewtopic.php?t=186531][mint 186531]] +- Insufficient ALSA headroom on an emulated/quirky device (no). Constant fine crackle from a sink whose hardware pointer PipeWire can't track (VM audio, cheap onboard). The default small =api.alsa.headroom= leaves no slack. Fix: a WirePlumber ALSA rule adding =api.alsa.headroom = 2048=. [[https://pipewire.pages.freedesktop.org/wireplumber/daemon/configuration/alsa.html][wp alsa config]] + +* Output triage + +The 59 output failure modes sorted into nine symptom clusters, same remedy classes as the input triage (AUTO / PRIV / REBOOT-TAIL / GUIDE — see that section's legend). The clusters shift for output: no mic-specific groups, but HDMI/DisplayPort and degraded-audio (crackle/xrun) each earn a dedicated cluster, and speaker-vs-headphone jack routing replaces the input pin cluster. + +Rough distribution: ~24 AUTO, ~8 PRIV, ~11 REBOOT-TAIL, ~16 GUIDE. Output leans more REBOOT-TAIL than input, because its hardware tail (smart-amp firmware, HDA model quirks, hdajackretask, GPU modeset) is heavier. Triage is a proposal — correct the class on any line. + +** Cluster 1 — no sound at all / dummy output +Probe: units + kernel =/proc/asound= + =dsp_driver=. The output mirror of input cluster 1. + +- SOF DSP firmware not installed :: REBOOT-TAIL — install =sof-firmware= (PRIV), reboot. +- Cirrus CS35L41 smart amp, missing _DSD :: REBOOT-TAIL — =firmware-cirrus= + the _DSD override (PRIV), reboot; GUIDE the BIOS-update path. +- TI TAS2781 smart amp absent :: GUIDE — needs a kernel built with the driver + TI firmware; nothing to run on a stock stack. +- Wrong HDA model autodetection :: REBOOT-TAIL — =model== modprobe (PRIV) + reboot. +- snd_soc_skl grabs the card, only Dummy Output :: REBOOT-TAIL — =dmic_detect=0= + blacklist =snd_soc_skl= (PRIV) + reboot. +- AMD Ryzen HDA codec probe timeout :: REBOOT-TAIL — the =probe_mask=/=dmic_detect= workaround (PRIV) + reboot; GUIDE a newer kernel. +- Kernel regression killed HDA output :: GUIDE — boot the previous kernel / linux-lts. +- Kernel USB-audio regression, DAC "Off" profile :: GUIDE — boot linux-lts / wait for the fix. +- BIOS-disabled onboard, or a GPU HDA controller steals ordering :: GUIDE + PRIV — re-enable in BIOS (GUIDE); =enable=1,0= modprobe to demote the GPU's audio (PRIV, reboot). + +** Cluster 2 — sound plays but from the wrong device +Probe: default sink + priority. The output side of routing. + +- Default sink lands on HDMI instead of speakers :: AUTO — =set-default= to the speakers. +- Default keeps reverting to the wrong sink :: AUTO — a =priority.session= rule. +- WirePlumber fails to start / stale config jams routing :: AUTO — remove the drop-in + restart (PRIV if under =/etc=). +- Corrupted WirePlumber restore state :: AUTO — =rm= the state + restart. +- pipewire-pulse dead, Pulse apps silent :: AUTO — restart the user services. +- USB DAC not auto-selected as default :: AUTO — a =priority.session= rule. + +** Cluster 3 — right default but silent (muted or zero) +Probe: semantic sink mute/volume. The output mirror of input cluster 2. + +- Master / PCM channel muted :: AUTO — unmute. +- Auto-Mute Mode silences speakers :: AUTO — disable Auto-Mute. +- alsactl restores a muted master at boot :: AUTO + PRIV — unmute now (AUTO); =sudo alsactl store= to persist (PRIV). +- Front/Surround/Center/LFE channel muted :: AUTO — unmute the channels. +- Sink present but silent until a mute/unmute toggle :: AUTO — =set-mute 0= / =set-volume=. + +** Cluster 4 — speakers work but headphones don't (or the reverse) +Probe: jack-sense + pin config. Mostly hdajackretask/model-quirk territory, so this cluster leans REBOOT-TAIL. + +- Dead headphone pin needs hdajackretask :: REBOOT-TAIL — hdajackretask override + install + reboot. +- Phantom jack-sense keeps speakers muted :: REBOOT-TAIL — hdajackretask reconfigure; GUIDE if it needs a codec quirk. +- Wrong codec model quirk, no headphone or speaker :: REBOOT-TAIL — =model== modprobe + reboot. +- Headphone path maps volume to the wrong control :: PRIV — edit the alsa-card-profile mixer path (a system file), restart audio. +- Headphone jack not detected, speakers keep playing :: REBOOT-TAIL — =model== modprobe + reboot. +- Front-panel headphone jack not wired by BIOS :: REBOOT-TAIL — hdajackretask override + install + reboot. + +** Cluster 5 — works then dies or pops +Probe: node state / suspend + re-probe-after-idle. + +- Sink suspended on idle, pops on start/stop :: AUTO — =session.suspend-timeout-seconds = 0= drop-in + restart. +- USB autosuspend powers the DAC down :: PRIV — a udev =power/control=on= rule (or =usbcore.autosuspend=-1=, REBOOT-TAIL). +- DAC needs an implicit-feedback quirk :: REBOOT-TAIL — =implicit_fb= modprobe (PRIV) + reboot. + +** Cluster 6 — Bluetooth output +Probe: bluez profile + codec. + +- A2DP Sink profile deselects / unavailable :: PRIV — clear =/var/lib/bluetooth= + re-pair (or a config fix, AUTO). +- LDAC/aptX never negotiates, falls to SBC :: PRIV — install the codec packages; AUTO for the codecs config. +- Playback drops to HSP/HFP mid-song :: AUTO — autoswitch off; force =a2dp-sink=. +- BT sink not made default on connect :: AUTO — =set-default=; a switch-on-connect rule. +- Silent sink after suspend/resume :: AUTO — a seat-monitoring drop-in; restart as the workaround. +- Volume slider does nothing (absolute volume) :: AUTO — =enable-hw-volume = false=. +- Adapter never exposes a sink (firmware) :: REBOOT-TAIL — update linux-firmware, reload/reboot. + +** Cluster 7 — HDMI / DisplayPort +Probe: GPU audio codec + ELD. Output-only cluster. + +- GPU HDMI codec never initializes :: REBOOT-TAIL — =dsp_driver= modprobe + initramfs + reboot. +- NVIDIA GPU audio removed by a PM udev rule :: PRIV — edit the udev rule (reload/reboot). +- NVIDIA HDMI audio needs DRM modesetting :: REBOOT-TAIL — =modeset=1= modprobe + initramfs + reboot. +- HDMI audio dies after suspend/resume :: PRIV — =sudo alsa force-reload= (or alsactl store/restore). +- Empty ELD after resume (kernel regression) :: GUIDE — boot linux-lts; replug to force an ELD re-read. +- Wrong monitor is the audio sink :: AUTO — =set-card-profile= to the right hdmi-extra port. +- HDMI sink present but silent until daemon restart :: AUTO — restart the user services. +- HDMI sample-rate mismatch, display rejects :: AUTO — pin the rate. + +** Cluster 8 — degraded audio (crackle, xrun, latency) +Probe: xrun counters (pw-top) + config. Output-only cluster; sound plays but is degraded. + +- Quantum drops too low under load :: AUTO — a =min-quantum= drop-in + restart. +- Realtime scheduling denied (RTKit) :: PRIV — =limits.d= + the realtime group (relogin). +- Timer-based scheduling on a fussy USB DAC :: AUTO — a =disable-tsched= rule. +- Forced global sample rate causes resample artifacts :: AUTO — =allowed-rates= config. +- Small quantum too tight for browser/games :: AUTO — per-app =PIPEWIRE_LATENCY= / a stream rule. +- HDA power-save clicks (and BT-latency hits) :: PRIV — =power_save=0= modprobe. +- Insufficient ALSA headroom :: AUTO — a =headroom= rule. + +** Cluster 9 — stack healthy, an app is silent +Probe: none in-stack; the tail on a healthy verdict. The output mirror of input cluster 7. + +- Flatpak app has no PulseAudio socket :: GUIDE — print =flatpak override --user --socket=pulseaudio <app>=. +- Flatpak silent on a PipeWire-only system :: GUIDE — print =flatpak override --user --filesystem=xdg-run/pipewire-0 <app>=. +- Snap audio-playback interface unconnected :: GUIDE — print =snap connect <snap>:audio-playback=. +- Browser per-tab mute zeroes the stream :: GUIDE — unmute the tab, or the app's stream. +- Wine/Proton game silent, wrong audio driver :: GUIDE — winecfg output device; install the 32-bit audio libs. +- App hard-coded to raw hw:0, device busy :: GUIDE — change the app's device to =default= / =plughw=. +- Chromium doesn't follow the default-sink change :: GUIDE — reassign the stream in pavucontrol. +- Electron app routed to the wrong sink :: GUIDE — move the stream in pavucontrol. + +** Saturation resample additions (2026-07-10) + +Genuinely-new output root causes from the blind resample, each tagged with the cluster it fits and its remedy class. Zero forced a new cluster. + +- App picks the wrong audio backend, Spotify (cluster 9 · GUIDE). The app's built-in backend mis-routes. Fix: force the pulseaudio backend in the launch flags (=--audio-api=pulseaudio=). [[https://bbs.archlinux.org/viewtopic.php?id=285730][arch 285730]] +- A PipeWire regression breaks game audio, Proton XAudio2 titles (cluster 9 · GUIDE). Fix: downgrade PipeWire, or update to the fixed release. [[https://github.com/ValveSoftware/Proton/issues/8730][proton 8730]] +- App muted in the desktop per-app mixer, KDE Chrome (cluster 9 · AUTO). The Flatpak permission is fine; the app's stream is muted in the DE volume applet. Fix: unmute the app there. [[https://github.com/flathub/com.google.Chrome/issues/297][chrome 297]] +- App connection stale after an audio-stack update, Firefox (cluster 9 · AUTO). Fix: restart the app or the audio daemon. [[https://forum.endeavouros.com/t/solved-no-audio-in-firefox-from-pipewire-or-pulseaudio-update/26032][endeavour 26032]] +- Output defaults to the wrong digital out, needs alsactl init, Framework 16 (cluster 2 · AUTO). Fix: =alsactl init=, then select the analog output. [[https://community.frame.work/t/solved-framework-16-ai-300-series-ubuntu-24-04-speakers-work-but-not-headphones/81363][framework 81363]] +- WirePlumber disabled in config, no enumeration, NixOS (cluster 2 · AUTO). Fix: enable wireplumber in the config, rebuild. [[https://github.com/NixOS/nixpkgs/issues/402999][nixpkgs 402999]] +- A distro update drops an audio package, openSUSE =wireplumber-audio= (cluster 1 · PRIV). Fix: reinstall and lock the package. [[https://forums.opensuse.org/t/no-output-or-input-devices-after-zypper-dup-only-dummy-output-in-pipewire-on-tumbleweed/185154][opensuse 185154]] +- USB audio module not auto-loaded after a kernel update (cluster 1 · PRIV). Fix: add =snd_usb_audio= to =modules-load.d=. [[https://forums.linuxmint.com/viewtopic.php?t=443590][mint 443590]] +- Stray =model=generic= modprobe forces the wrong codec (cluster 1 · REBOOT-TAIL). Left by OEM/installer tooling. Fix: delete the leftover =/etc/modprobe.d= line. [[https://bugs.launchpad.net/bugs/2092728][lp 2092728]] +- SOF driver aborts when the codec is hda_generic, Acer Swift (cluster 1 · REBOOT-TAIL). Fix: kernel 5.8+; =dmic_detect=0= workaround. [[https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1877757][lp 1877757]] +- Cirrus CS35L56 smart-amp firmware not applied, ROG G14 2024 (cluster 1 · REBOOT-TAIL). A newer amp than the CS35L41 already catalogued. Fix: newer kernel + firmware. [[https://www.mail-archive.com/debian-kernel@lists.debian.org/msg139811.html][debian-kernel]] +- Creative CA0132 card needs a kernel quirk, AE-5 Plus (cluster 1 · GUIDE). Reuses an existing PCI id with a different config. Fix: no userspace fix; kernel patch. [[https://forum.manjaro.org/t/soundblaster-creative-ae-5-plus-no-sound/60516][manjaro 60516]] +- ALC4080 enumerates as USB not HDA, front-panel node not exposed (cluster 4 · AUTO). Fix: an explicit ALSA sink in =pipewire.conf= pointing at the FP device. [[https://discussion.fedoraproject.org/t/headphone-audio-not-working-asus-rog-strix-z590-f-alc4080/76380][fedora 76380]] +- GPU HDMI audio KAE regression, Intel Arc DG2 (cluster 7 · GUIDE). Fix: kernel fix disables KAE for DG2; update kernel. [[https://www.phoronix.com/news/Intel-DG2-Audio-Fix-Linux-KAE][phoronix]] +- USB DAC silent at a specific sample rate, FiiO 44.1k (cluster 8 · AUTO). Fix: set =allowed-rates=. [[https://bbs.archlinux.org/viewtopic.php?id=180561][arch 180561]] +- BIOS firmware bug, crackle or dead speakers fixed by a BIOS update, MSI GF63 (cluster 1 · GUIDE). Broke on Windows too. Fix: flash the vendor BIOS update. [[https://forums.linuxmint.com/viewtopic.php?t=424474][mint 424474]] +- USB sound device not reliably enumerated after boot, an init race, Creative/Schiit (cluster 5 · GUIDE). Fix: replug; pin the profile. [[https://forum.zorin.com/t/usb-soundcard-creative-sound-blaster-hd-not-always-recognized-after-boot/66177][zorin 66177]] +- Audio-group membership blocks access, older udev/ConsoleKit distros (cluster 9 · PRIV). The same cross-cutting trap the input side hit. Fix: remove the user from the =audio= group. [[https://wiki.ubuntu.com/Audio/TheAudioGroup][ubuntu audio group]] + +* Sources + +Every entry carries its source inline as an org link. Reports span the Arch/Manjaro/EndeavourOS/Fedora/Ubuntu/Mint/openSUSE forums, askubuntu, the PipeWire and WirePlumber docs, ALSA and Flatpak issue trackers, kernel bug trackers, and vendor forums (NVIDIA, Dell). Collected 2026-07-10. diff --git a/docs/design/2026-07-10-net-bt-failure-taxonomy.org b/docs/design/2026-07-10-net-bt-failure-taxonomy.org new file mode 100644 index 0000000..74790c6 --- /dev/null +++ b/docs/design/2026-07-10-net-bt-failure-taxonomy.org @@ -0,0 +1,434 @@ +#+TITLE: Linux Network and Bluetooth Failure Taxonomy — for the net and bt doctors +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-10 + +* Purpose + +A catalogue of real, user-reported ways Linux networking (WiFi/ethernet/DNS/egress) and Bluetooth fail, grounded in forum and issue-tracker reports. Built to feed the net-panel and bt-panel doctors the same way the audio taxonomy fed the audio doctor: each entry is triaged into a remedy class — AUTO (the doctor applies a user-scope reversible fix), PRIV (needs root; runs where passwordless sudo exists), REBOOT-TAIL (a privileged fix that only takes hold after a reboot), or GUIDE (nothing to run — a physical action, a config decision, or an upstream/venue problem). This mirrors [[file:2026-07-10-audio-failure-taxonomy.org][the audio taxonomy]] and reuses its four-class privilege model. + +The net and bt doctors already exist (=~/.dotfiles= =net/= and =bluetooth/=). This taxonomy grounds their *expansion* — widening the diagnosis space and hardening the verdict/remedy structure — the way the audio input-side spec grew from the audio taxonomy. It fed two DRAFT specs: [[file:../specs/2026-07-11-net-doctor-expansion-spec.org][the net doctor expansion]] and [[file:../specs/2026-07-11-bt-doctor-expansion-spec.org][the bt doctor expansion]]. + +** Method + +Built 2026-07-10 from a blind, by-layer parallel research fan-out — the same first-sweep method as the audio taxonomy. Eleven agents (six network, five bluetooth), each scoped to one layer and blind to any cluster structure, drew real failure reports from Arch BBS/wiki, kernel bugzilla/launchpad/gitlab, distro forums, and upstream issue trackers, weighted toward 2023-2026 and the Arch / NetworkManager / systemd-resolved / bluez stack. Each entry names the layer, the user-reported symptom, the root cause, the known fix (with the concrete command where one exists), the privilege/reboot signal, and a source. + +The =(privilege signal)= after each title is the first triage cue: =no= = user-scope, no reboot; =yes= = needs root; =both= = root AND a reboot; =physical= = a hardware/physical/BIOS action. A =no= is an AUTO candidate; =both= or =physical= can only ever be REBOOT-TAIL or GUIDE. + +A saturation resample (the audio taxonomy's blind-different-axis check) is a later pass, noted at the end. + +* Network failure modes + +Six layers, mirroring the net doctor's probe ladder (link → IP/DHCP → gateway/route → DNS → egress) plus the NetworkManager control plane that sits beside all of them. ~98 raw entries from the sweep; the obvious cross-layer duplicates (VPN route capture, PMTUD blackhole, broken-IPv6/happy-eyeballs, unmanaged device, dhcpcd-vs-NM, wifi powersave, connectivity-check false negative, MAC randomization, iwd/wpa_supplicant backend) are merged inline and cross-noted, leaving ~74 distinct root causes. + +** Link / radio / driver / firmware / rfkill + +- Missing iwlwifi firmware for a very recent Intel card (both). No wifi interface in =ip link=; dmesg "Direct firmware load for iwlwifi-… failed with error -2". The installed linux-firmware predates the card's blob. Fix: update/install linux-firmware (or linux-firmware-git), reboot. [[https://bbs.archlinux.org/viewtopic.php?id=193570][arch 193570]] +- iwlwifi firmware-load regression after a kernel update (both). Wifi vanished right after an upgrade; dmesg shows iwlwifi failing to load firmware it loaded before. A kernel-side change broke firmware negotiation for that module. Fix: boot linux-lts or update to the fixed kernel, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=304848][arch 304848]] +- Realtek rtw89 in-tree module shadowed by an AUR DKMS driver (both). RTL8852BE wifi disappeared after an update; two conflicting modules present. A stale 8852be/rtw89-dkms-git overrides the in-tree rtw89_8852be. Fix: remove the AUR DKMS package, ensure linux-firmware is current, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=286109][arch 286109]] +- MediaTek mt7921e broken by a linux-firmware update (both). Wifi worked, then a pacman upgrade killed it; firmware load timeout / "hardware init failed" for WIFI_RAM_CODE_MT7961. A regression in the packaged blob fails to init the chip. Fix: downgrade/pin linux-firmware-mediatek to the last-good version, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=308360][arch 308360]] +- Broadcom BCM4360 needs the proprietary wl driver (both). "No wifi adapter"; lspci shows BCM4360 but no wlan interface. The open brcm/b43/bcma drivers claim it but can't drive it. Fix: install broadcom-wl-dkms, blacklist b43/bcma/brcmsmac/brcmfmac/ssb, rebuild initramfs, reboot. [[https://wiki.archlinux.org/title/Broadcom_wireless][archwiki broadcom]] +- broadcom-wl (non-DKMS) breaks after a kernel upgrade (both). Wifi gone after a kernel bump; wl won't load against the new kernel. The prebuilt module isn't rebuilt automatically. Fix: switch to broadcom-wl-dkms, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=305252][arch 305252]] +- USB wifi dongle needs an out-of-tree DKMS driver (both). An RTL8811AU/8812AU/8821AU dongle enumerates in lsusb but no wlan interface appears. No in-tree driver binds it. Fix: install the matching DKMS package with kernel headers, reboot. [[https://aur.archlinux.org/packages/rtl8812au-dkms-git][aur rtl8812au]] +- Wifi soft-blocked by rfkill (no). =rfkill list= reports "Soft blocked: yes"; the radio is off. Software (a prior toggle, a hotkey, NM state) set the soft block. Fix: =rfkill unblock wifi= (or =nmcli radio wifi on=). [[https://bbs.archlinux.org/viewtopic.php?id=236715][arch 236715]] +- Wifi hard-blocked by a physical switch / airplane / BIOS (physical). "Hard blocked: yes"; no software clears it. A kill-switch, Fn toggle, or BIOS setting physically disables the radio. Fix: flip the physical/Fn switch or enable the radio in BIOS. [[https://bbs.archlinux.org/viewtopic.php?id=207376][arch 207376]] +- rfkill stuck hard-blocked after suspend from competing WMI modules (both). Wifi comes back hard-blocked after resume and unblock won't clear it. Two vendor WMI modules (acer_wmi + hp_wmi) both claim the switch and wedge it. Fix: blacklist the wrong WMI module, rebuild initramfs, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=236657][arch 236657]] +- iwlwifi crashloops on resume, needs a module reload (yes). Wifi dead after suspend; dmesg shows microcode errors / firmware crashloop until the driver reloads. Firmware state isn't restored across resume. Fix: =modprobe -r iwlmvm iwlwifi && modprobe iwlwifi= (or a system-sleep hook). [[https://bbs.archlinux.org/viewtopic.php?id=293404][arch 293404]] +- ath11k (QCA6390/QCNFA765) firmware crash, dead until reload (yes). Wifi drops after resume or at random; dmesg "ath11k_pci … firmware crashed: MHI_CB_SYS_ERROR". The card's firmware faults and doesn't recover. Fix: =modprobe -r ath11k_pci && modprobe ath11k_pci=. [[https://bbs.archlinux.org/viewtopic.php?id=310363][arch 310363]] +- WiFi powersave drops the connection (yes; merges the auth-layer and NM-layer duplicates). Wifi connects then stalls/disconnects, worse on battery. Aggressive power management (driver =power_save= and/or NM =wifi.powersave=3=) breaks association with some APs. Fix: =iw dev wlan0 set power_save off= now; persist via NM =[connection] wifi.powersave=2= drop-in and/or =options iwlwifi power_save=0= in modprobe.d. [[https://bbs.archlinux.org/viewtopic.php?id=171342][arch 171342]] +- USB wifi dongle killed by USB autosuspend (yes). A working dongle drops out after idle and returns on replug; =power/control= reads "auto". Autosuspend powers the device down mid-use. Fix: a udev rule pinning =power/control=on= (or disable USB autosuspend in TLP). [[https://bbs.archlinux.org/viewtopic.php?id=184823][arch 184823]] +- RTL8125 ethernet flips from r8125 to in-tree r8169 after a kernel update (both). Wired network dies after an upgrade; the NIC binds r8169 and shows NO-CARRIER. A kernel update overrode the AUR r8125 module. Fix: blacklist r8169, install/force r8125-dkms, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=262120][arch 262120]] +- RTL8125 link downshifts / won't negotiate full speed (yes). The 2.5GbE NIC links but only at 1Gbps/100Mbps; ethtool shows the reduced speed. EEE / Giga-Lite negotiation downshifts the PHY. Fix: =ethtool --set-eee <if> eee off= (or force speed via =ethtool -s=), or swap cable/port. [[https://bbs.archlinux.org/viewtopic.php?id=264742][arch 264742]] +- Ethernet NO-CARRIER from an unplugged/faulty cable or dead switch port (physical). Interface UP but NO-CARRIER, no IP ever assigned. No link partner: cable unseated/broken or switch port down. Fix: reseat/replace the cable, try another port — no software repair until carrier returns. [[https://bbs.archlinux.org/viewtopic.php?id=268745][arch 268745]] +- Adapter present but left "unmanaged" by NetworkManager (no/yes; merges the NM-layer duplicate). =nmcli device= lists the device "unmanaged". Another daemon (iwd, systemd-networkd) owns it, or an =unmanaged-devices= rule / =NM_MANAGED=no= excludes it. Fix: =nmcli device set <dev> managed yes= (AUTO); remove the config rule or disable the rival manager to persist (PRIV). [[https://bbs.archlinux.org/viewtopic.php?id=291445][arch 291445]] + +** WiFi association / authentication + +- Wrong PSK, NM state 120 / secrets required (no; TERMINAL). "Secrets were required, but not provided"; journal "4-Way Handshake failed — pre-shared key may be incorrect". NM stored the wrong key — no repair fixes it; the user must re-enter. Fix: =nmcli con modify <name> wifi-sec.psk <key>= or re-enter in the picker. [[https://bbs.archlinux.org/viewtopic.php?id=306757][arch 306757]] +- Pure WPA3-SAE won't associate (no). "No PSK available for association" / "SME: Failed to set WPA key management" on a WPA3-only AP. The profile defaults to WPA-PSK key-mgmt with no PMF, which a pure-SAE AP rejects. Fix: =nmcli con modify <name> wifi-sec.key-mgmt sae 802-11-wireless-security.pmf 3=. [[https://bbs.archlinux.org/viewtopic.php?id=256573][arch 256573]] +- PMF mismatch on a WPA2/WPA3 transition AP (no). SAE assoc rejected on mixed-mode; connects to WPA2 but not the SAE path. Transition mode requires PMF-capable and rejects SAE when 802.11w isn't negotiated. Fix: match the AP's PMF — =nmcli con modify <name> 802-11-wireless-security.pmf 2=. [[https://hostap.shmoo.narkive.com/38Iowl2x/ieee-802-11w-support-in-wpa-supplicant-management-frame-protection-required-but-client-did-not][hostap 802.11w]] +- SAE H2E-only AP vs hunt-and-peck client (yes). WPA3 assoc fails "unspecified failure" on a Hash-to-Element-only AP. wpa_supplicant defaults to hunt-and-peck; the PWE method mismatches. Fix: =sae_pwe=2= in wpa_supplicant global config. [[https://wiki.archlinux.org/title/Wpa_supplicant][archwiki wpa_supplicant]] +- eduroam PEAP cert validation failed (no). "Certificate verification failed"; EAP drops. Missing/wrong CA cert, or NM ignoring system CA certs. Fix: set the correct CA cert / server-name in 802-1x (temporarily =system-ca-certs=false= to test). [[https://debianforum.de/forum/viewtopic.php?t=169560][debianforum 169560]] +- iwd eduroam bad_certificate (no). iwd refuses eduroam "bad_certificate" though it works elsewhere. iwd's EAP needs the server CA cert and a matching domain. Fix: add =EAP-PEAP-CACert=<ca.pem>= (and Domain/ServerDomainMask) to the iwd profile. [[https://bbs.archlinux.org/viewtopic.php?id=291921][arch 291921]] +- 802.1X won't save/verify the CA cert (no). Enterprise wifi only connects with "No CA certificate required" ticked. NM wants a PEM (not DER) bundle and mishandles the ca-cert setting. Fix: =nmcli con modify <name> 802-1x.ca-cert /etc/ssl/certs/ca-certificates.crt=. [[https://bugzilla.redhat.com/show_bug.cgi?id=1455833][redhat 1455833]] +- Wrong eduroam anonymous/outer identity (no). Auth rejected at the RADIUS server. The outer identity has the wrong realm, or the anonymous ID is reused as the inner identity. Fix: set =anonymous_identity="anonymous@realm"= and =identity="user@realm"= to the home realm. [[https://community.jisc.ac.uk/library/network-and-technology-service-docs/2021-04-advisory-android-11-configuration-issues][jisc eduroam]] +- Hidden SSID never connects (no). A saved non-broadcast SSID doesn't auto-connect; "No network with SSID found". NM won't send a directed probe unless the profile is flagged hidden. Fix: =nmcli con modify <name> wifi.hidden yes=. [[https://bbs.archlinux.org/viewtopic.php?id=39839][arch 39839]] +- Randomized MAC breaks captive portal / MAC allow-list (no; merges the NM-layer duplicate). Connects but the portal re-prompts every time, or a MAC-filtered router refuses the device. Per-connection random MAC changes each connect. Fix: =nmcli con modify <name> wifi.cloned-mac-address stable= (or =permanent=). [[https://fedoramagazine.org/randomize-mac-address-nm/][fedora NM MAC]] +- scan-rand-mac-address breaks scan/assoc on some drivers (yes). After an NM upgrade no networks appear, or assoc fails "Authentication timeout" (ath9k/ath9k_htc, some Realtek/Atheros). The driver can't handle the randomized scan MAC. Fix: =[device] wifi.scan-rand-mac-address=no= drop-in. [[https://github.com/qca/open-ath9k-htc-firmware/issues/132][qca ath9k-htc 132]] +- Wrong regulatory domain hides/blocks channels (yes). A 5GHz AP is invisible / "no suitable AP"; the card won't use the AP's channels. Regulatory domain unset or wrong, so the kernel forbids those frequencies. Fix: =iw reg set <CC>=, persist via =WIRELESS_REGDOM= (install wireless-regdb). [[https://bbs.archlinux.org/viewtopic.php?id=249979][arch 249979]] +- Won't roam, stuck on a weak AP (no). The client clings to a weak BSSID even with a stronger same-SSID AP in range. NM+wpa_supplicant roaming is conservative. Fix: bounce the connection, tune bgscan / lock a bssid, or try the iwd backend. [[https://bbs.archlinux.org/viewtopic.php?id=291466][arch 291466]] +- 4-way-handshake timeout / deauth loop, reason 15 (both). "4way_handshake_timeout" or repeated deauth (Reason 15) on iwlwifi/some drivers. An 802.11n handshake bug or AP port-security kicks the client mid-handshake. Fix: =options iwlwifi 11n_disable=1= (reboot), or try the other backend. [[https://bbs.archlinux.org/viewtopic.php?id=264378][arch 264378]] +- iwd associates but NM(wpa_supplicant) doesn't (yes; see also the NM backend entries). Standalone iwd connects but NM's wpa_supplicant path fails to associate on the same AP. Backend-specific supplicant behavior differs. Fix: =[device] wifi.backend=iwd= drop-in, disable standalone iwd.service, restart NM. [[https://bbs.archlinux.org/viewtopic.php?id=292603][arch 292603]] + +** IP / DHCP / gateway / routing / VPN policy + +- dhcpcd competing with NM's internal DHCP client (yes; merges the two DHCP-conflict duplicates). Interface never gets an IPv4 address; NM stuck "getting IP configuration". A standalone =dhcpcd.service= and NM's internal client fight over the interface. Fix: =systemctl disable --now dhcpcd= and let NM own DHCP. [[https://bbs.archlinux.org/viewtopic.php?id=165793][arch 165793]] +- dhcpcd 9 backend breaks NM IPv4 (yes). No IPv4 after a dhcpcd upgrade; IPv6 may still work. NM's dhcpcd backend was incompatible with dhcpcd 9's changed behavior. Fix: =dhcp=internal= in NetworkManager.conf (or NM ≥1.30.4-3), restart NM. [[https://bugs.archlinux.org/task/66231][arch fs66231]] +- APIPA / link-local only, no DHCP answer (no). The interface self-assigns 169.254.x.x; only local-link hosts reachable. No DHCP server answered in the timeout. Fix: re-trigger DHCP — =nmcli device reapply <dev>=; a recurring case is a server-side fault. [[https://oneuptime.com/blog/post/2026-03-20-fix-apipa-169-254-address/view][apipa 169.254]] +- Default route never installed by the connection (yes). Has an IP and pings the LAN, but no default route. The profile has =ipv4.never-default yes= ("use only for resources on its network"). Fix: =nmcli con modify <name> ipv4.never-default no=, up. [[https://bbs.archlinux.org/viewtopic.php?id=84453][arch 84453]] +- DHCP gateway in a different subnet than the lease (yes). IP assigned but no default route; "SIOCADDRT: No such process". The DHCP-offered gateway is off the assigned prefix, so the on-link route is refused. Fix: add the on-link route then default (=ip route add <gw> dev <iface>; ip route add default via <gw>=), or fix the DHCP scope. [[https://bbs.archlinux.org/viewtopic.php?id=72685][arch 72685]] +- Duplicate IPv4 on the LAN, address skipped by ACD (yes). Connection comes up but no usable address; journal logs the conflicting MAC. NM's RFC 5227 conflict detection found the IP already in use. Fix: change the offending host / DHCP reservation, or set a distinct static IP. [[https://bbs.archlinux.org/viewtopic.php?id=287424][arch 287424]] +- IPv6 RA not received, no SLAAC address or route (yes). No IPv6 address/route; interface hangs "configuring". =accept_ra=0=, forwarding enabled without =accept_ra=2=, or an upstream RA-guard drops ICMPv6 type 134. Fix: =sysctl net.ipv6.conf.<iface>.accept_ra=2= (or unblock RA at the router). [[https://oneuptime.com/blog/post/2026-03-20-ipv6-ra-not-received/view][ipv6 ra]] +- DHCPv6-only network, address but no route (yes). An IPv6 address via stateful DHCPv6, but no default route. Routing lives in the RA, not DHCPv6. Fix: ensure the router sends an RA with the M flag and let NM process both (=ipv6.method auto=). [[https://bbs.archlinux.org/viewtopic.php?id=273336][arch 273336]] +- Happy-eyeballs stall on broken IPv6 (no; the routing angle — see also the DNS AAAA and egress v6 entries). Pages load with a visible hang; IPv6 is present but black-holed. Dual-stack apps race IPv6 first and wait to fall back. Fix: =nmcli con modify <name> ipv6.method disabled= (or fix the upstream v6 path). [[https://oneuptime.com/blog/post/2026-03-20-ipv6-happy-eyeballs-issues/view][happy-eyeballs]] +- WireGuard AllowedIPs=0.0.0.0/0 strands the host when the tunnel drops (no). Everything offline after the VPN drops; ping and DNS fail. Full-tunnel installed a default route through wg0 with no live peer. Fix: bring the tunnel down (=wg-quick down <iface>= / =nmcli con down <vpn>=) to restore the physical route. [[https://discourse.coreelec.org/t/wireguard-client-not-connected-when-allowedips-0-0-0-0-0/18187][wg allowedips]] +- WireGuard split-tunnel misconfig captures the LAN (yes). VPN connects but LAN devices/printers become unreachable. =AllowedIPs=0.0.0.0/0= used where a split tunnel was intended. Fix: set AllowedIPs to only the intended subnets, re-up. [[https://oneuptime.com/blog/post/2026-03-20-wireguard-allowedips-ipv4-split-tunneling/view][wg split-tunnel]] +- OpenVPN redirect-gateway route left behind after an abnormal disconnect (yes). No internet/DNS after the tunnel dies; can't even reconnect. redirect-gateway rewrote the default route and a hard link-down never restored it. Fix: =ip route del default dev tun0; systemctl restart openvpn-client@<name>= (use =def1= going forward). [[https://forums.openvpn.net/viewtopic.php?t=22572][openvpn redirect-gw]] +- Tailscale exit node captures the default route (no). Internet dies after selecting an exit node; tailnet peers still reachable. The exit-node route conflicts with the working one, or policy lacks =autogroup:internet=. Fix: =tailscale set --exit-node== to drop back to the local default route. [[https://github.com/tailscale/tailscale/issues/2654][tailscale exitnode]] +- Docker bridge subnet collides with a real external subnet (yes). A specific external host (172.17.x.x) is unreachable while the rest of the internet works. docker0 claimed 172.17.0.0/16. Fix: set ="bip": "172.26.0.1/16"= in daemon.json, prune, restart docker. [[https://github.com/moby/moby/issues/36619][moby 36619]] +- libvirt virbr0 subnet collides with the real network (yes). Host/VMs can only reach the bridge/host, not the WAN. virbr0 came up on a range overlapping the LAN. Fix: re-address the virtual net via =virsh net-edit default=, destroy/start it. [[https://bugzilla.redhat.com/show_bug.cgi?id=811967][rhbz 811967]] +- Wifi + ethernet up with conflicting metrics, traffic exits the dead link (yes). Intermittent/no connectivity with two links up; egress uses the wrong interface. Both installed a default route and the lower metric wins even when it can't reach the internet. Fix: raise the bad link's metric — =nmcli con modify <name> ipv4.route-metric <higher>=. [[https://www.baeldung.com/linux/change-network-routing-metric][route-metric]] + +** DNS / resolver / DoT / DNSSEC + +- Dangling resolv.conf symlink after resolved is disabled/not started (yes). Can ping 1.1.1.1 but nothing resolves; =/etc/resolv.conf= symlinks to a stub file that doesn't exist. The stub only exists while systemd-resolved runs. Fix: =systemctl enable --now systemd-resolved= (or repoint the link at a real file). [[https://github.com/systemd/systemd/issues/5395][systemd 5395]] +- resolv.conf not pointing at the stub while resolved runs (yes). Names fail though resolved is up; resolv.conf is a stale static file. An installer/upgrade replaced it. Fix: =ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf=, verify with =resolvectl status=. [[https://wiki.archlinux.org/title/Systemd-resolved][archwiki resolved]] +- systemd-resolved stub 127.0.0.53 unreachable / port 53 conflict (yes). resolv.conf says 127.0.0.53 but every query is connection-refused. A second resolver (dnsmasq, bind) holds port 53, so the stub never binds. Fix: stop the conflicting resolver or set DNSStubListener, restart resolved. [[https://github.com/systemd/systemd/issues/10298][systemd 10298]] +- NetworkManager not integrated with resolved (yes). resolved is up but knows no upstream servers; NM writes its own resolv.conf. =dns=systemd-resolved= not set, so DHCP servers never reach resolved. Fix: add =dns=systemd-resolved= under [main] in NetworkManager.conf, restart NM. [[https://wiki.archlinux.org/title/Systemd-resolved][archwiki resolved]] +- DNSSEC=yes rejects a non-compliant venue resolver (yes). Pages won't load on hotel/ISP wifi though ping works; "DNSSEC validation failed". resolved rejects the venue resolver that mangles DNSSEC records. Fix: =DNSSEC=allow-downgrade= (or no) in resolved.conf, restart resolved. [[https://bbs.archlinux.org/viewtopic.php?id=240427][arch 240427]] +- DNSSEC "no-signature" verdict sticks after a network change (yes). Resolution dies sporadically after reconnecting; resolved reuses a cached bogus/downgrade verdict from the previous network. Fix: =resolvectl reset-server-features= (and flush), or DNSSEC=allow-downgrade. [[https://github.com/systemd/systemd/issues/12388][systemd 12388]] +- DoT strict mode with port 853 blocked (yes). Everything stalls; DNSOverTLS=yes forbids fallback and the network drops 853. Fix: =DNSOverTLS=opportunistic= (or no) in resolved.conf, restart resolved. [[https://wiki.archlinux.org/title/Systemd-resolved][archwiki resolved]] +- DoT strict to an unreachable / cert-mismatched server (yes). "Failed to invoke gnutls_handshake: Error in the certificate verification"; the pinned DoT server is down or MITM'd. Strict mode plus IP#hostname pinning rejects the cert and won't fall back. Fix: correct the server/#hostname, switch provider, or relax to opportunistic, restart resolved. [[https://github.com/systemd/systemd/issues/16531][systemd 16531]] +- DHCP handed a broken resolver (no). Nothing resolves on one network though egress works; the DHCP DNS server is dead/filtering. Fix: =nmcli con mod <con> ipv4.ignore-auto-dns yes ipv4.dns "1.1.1.1 9.9.9.9"=, up. [[http://adam.younglogic.com/2019/05/using-nmcli-to-set-nameservers/][nmcli dns]] +- Manually-pinned 8.8.8.8 blocked by the venue (no). A hard-set public DNS resolves nothing though 1.1.1.1 pings; the network blocks external port 53. Fix: drop the manual pin and use DHCP DNS (=ipv4.ignore-auto-dns no=, clear =ipv4.dns=), reconnect. [[https://forum.netgate.com/topic/187060/port-53-dns][netgate port 53]] +- Stale poisoned cache after captive-portal login (yes). First lookups after the portal keep returning the portal/blocked address. Pre-login hijacked answers were cached. Fix: =resolvectl flush-caches=. [[https://oneuptime.com/blog/post/2026-03-20-flush-dns-cache-linux-macos-windows/view][flush-caches]] +- VPN split-DNS not applied / leaks to the ISP resolver (no). Internal/corp names fail (or a censored ISP resolver still answers) after connecting the VPN. NM/resolved didn't install the VPN link's domain as a routing domain. Fix: =resolvectl domain <vpn-link> '~corp.example'= and =resolvectl default-route <vpn-link> false=. [[https://blogs.gnome.org/mcatanzaro/2020/12/17/understanding-systemd-resolved-split-dns-and-vpn-configuration/][catanzaro split-dns]] +- IPv6 AAAA lookups stall resolution (yes; the DNS angle of the broken-v6 family). Every name takes ~5s then works; A and AAAA go in parallel and AAAA is never answered. Broken/half-configured IPv6 swallows the AAAA query. Fix: fix or disable IPv6 on the link, or =options single-request timeout:1=. [[https://bbs.archlinux.org/viewtopic.php?id=262926][arch 262926]] +- Another daemon overwrites resolv.conf (yes). DNS works then breaks (or breaks after VPN up/down) as dhcpcd/openvpn/openresolv rewrites resolv.conf. Multiple tools claim it with no coordination. Fix: pick one manager (openresolv =resolvconf=NO=, dhcpcd =nohook resolv.conf=), point resolv.conf at the stub, restart resolved. [[https://github.com/adrienverge/openfortivpn/issues/674][openfortivpn 674]] +- nsswitch.conf hosts line broken (yes). All resolution fails, or LAN/mDNS names never resolve; the hosts line lacks =resolve=/=dns= in the right order or references an uninstalled nss module. Fix: set =hosts: mymachines resolve [!UNAVAIL=return] files myhostname dns=. [[https://man.archlinux.org/man/nss-resolve.8.en][nss-resolve]] +- Avahi/.local mDNS not resolving (yes). *.local names don't resolve though unicast DNS works. nss-mdns not wired in, or resolved's built-in mDNS collides with avahi. Fix: install nss-mdns, add =mdns_minimal [NOTFOUND=return]= before =resolve=, enable avahi-daemon, disable resolved MulticastDNS if both run. [[https://wiki.archlinux.org/title/Avahi][archwiki avahi]] + +** Egress / captive portal / MTU / proxy / clock / upstream + +- Captive portal held state, no auto-popup (no). Wifi associates and gets a lease but every site fails; no login window appears. Without a DE NetworkManager integration or a dispatcher script, NM detects the PORTAL state but nothing opens the login page. Fix: open a plain-HTTP probe (=xdg-open http://neverssl.com=) or the URL NM reports; add an nm-dispatcher script to auto-launch. [[https://wiki.archlinux.org/title/NetworkManager#Captive_portals][archwiki captive portals]] +- Connectivity check over HTTPS never triggers the portal redirect (no). Behind a portal the login popup never fires. NM's connectivity URI was probed over HTTPS, which the portal can't transparently redirect. Fix: point NM's check at an HTTP URI (=20-connectivity.conf= =uri=http://…=) and open a plain-HTTP page. [[https://bugzilla.redhat.com/show_bug.cgi?id=1252839][redhat 1252839]] +- Custom DNS / DoT / DNSSEC hides the captive portal (yes). No portal page ever resolves and the machine stays cut off. resolved pinned to 1.1.1.1 with DoT/DNSSEC bypasses the portal's DNS hijack, so lookups just fail. Fix: drop DoT/DNSSEC for the captive network (=resolvectl dnsovertls <iface> no; resolvectl dnssec <iface> no=), log in, restore. [[https://github.com/systemd/systemd/issues/27485][systemd 27485]] +- HSTS blocks the portal redirect in the browser (no). The login page won't load; the browser shows a cert/HSTS error instead of the form. The first site was HSTS-pinned, so the browser refuses the portal's HTTP redirect. Fix: browse to a non-HSTS plain-HTTP address (http://neverssl.com, http://captive.apple.com). [[https://community.ui.com/questions/Solution-for-HSTS-issues-with-captive-portal/17b033e7-3dfe-4830-af8f-bf6ead23d8b0][ui hsts portal]] +- PMTUD blackhole from dropped ICMP (no; merges the IP-layer duplicate). Small requests and DNS work, pages start then stall, large downloads/SSH hang. A middlebox drops ICMP "fragmentation needed", so PMTUD never learns the real MTU and oversized packets vanish. Fix: lower the interface MTU (=nmcli con modify <con> 802-11-wireless.mtu 1400=), or MSS-clamp. [[https://ipfyi.com/scenario/path-mtu-blackhole/][pmtud blackhole]] +- PPPoE / VPN link with a lower MTU not clamped (no). Browsing works but big transfers / some HTTPS hang. A PPPoE (1492) or VPN path has a smaller MTU and the too-large segments get dropped. Fix: set the tunnel/link MTU down (=.mtu 1420= for VPN, 1492 for PPPoE) or MSS-clamp on the gateway. [[https://thelineman.ca/articles/article-8-mtu-vpn-mss][vpn mtu/mss]] +- Stale http_proxy env var points at a dead proxy (no). Every curl/wget/pacman fails though the network is fine; browsers may work. A leftover =http_proxy= points at an offline/off-network proxy. Fix: unset the vars, remove the export from =~/.profile= / =/etc/environment=. [[https://everything.curl.dev/usingcurl/proxies/env.html][curl proxy env]] +- Unreachable PAC file off the corporate network hangs everything (no). Away from the office the browser stalls with no error. A system proxy set to "automatic" with a PAC URL that only resolves on the corporate LAN blocks waiting instead of falling back to DIRECT. Fix: switch system proxy to None (=gsettings … org.gnome.system.proxy mode 'none'=) or clear the PAC URL. [[https://bugzilla.mozilla.org/show_bug.cgi?id=1121800][ff pac hang]] +- Clock skew breaks every TLS handshake (yes). "Your connection is not private" on every HTTPS site though ping/DNS work; the clock is hours/years off. A dual-boot Windows RTC-localtime, unsynced NTP, or a dead CMOS battery leaves the clock wrong. Fix: =timedatectl set-ntp true= (=set-local-rtc 0= on dual-boot), replace the CMOS battery if it recurs. [[https://wiki.archlinux.org/title/System_time][archwiki system time]] +- Firewall default-deny drops all egress (yes). No traffic leaves right after enabling a firewall, or after both ufw and firewalld are on; even DNS fails. A default outgoing-deny policy, or two firewalls fighting over nftables. Fix: allow egress (=ufw default allow outgoing=) and run only one firewall. [[https://wiki.archlinux.org/title/Uncomplicated_Firewall][archwiki ufw]] +- VPN kill-switch / leftover iptables rule strangles egress after VPN drops (yes; distinct from the route-capture case). Internet dies the moment the VPN disconnects and never returns until reboot. A kill-switch rule pinned traffic to tun0 and the leftover rule keeps dropping everything on the real interface. Fix: flush the stale rules (=iptables -F; iptables -P OUTPUT ACCEPT=, or restart the firewall), reconnect. [[https://bbs.archlinux.org/viewtopic.php?id=300104][arch ufw killswitch]] +- IPv6 egress broken while IPv4 works (no; the egress angle of the broken-v6 family). Pages load slowly/intermittently; IPv4-only hosts are fine. The network advertises IPv6 with no working route and Happy Eyeballs keeps trying the dead AAAA path. Fix: =nmcli con modify <con> ipv6.method disabled= until the network's IPv6 is fixed. [[https://help.ubuntu.com/community/WebBrowsingSlowIPv6IPv4][ubuntu slow ipv6]] +- False "limited connectivity" from a local DNS/VPN service (no; merges the NM-layer connectivity-check duplicate). The applet shows "limited/no internet" and apps that gate on NM refuse to work, yet browsing functions. NM's check to its connectivity URI fails — a local Pi-hole/VPN blocks the probe host, a false negative. Fix: point =20-connectivity.conf= at a reachable 204 endpoint, or =[connectivity] enabled=false=, reload NM. [[https://bbs.archlinux.org/viewtopic.php?id=272292][arch nm limited]] +- ISP outage or modem stuck in a bad state — not locally fixable (physical). Everything downstream is "connected, no internet"; the router's WAN IP is 0.0.0.0. The upstream link is down or the modem latched into a bad state. The doctor must recognize this and STOP. Fix: power-cycle modem then router; check ISP status / contact the provider. [[https://whizz-tech.com/support/wifi-router-fixes/wifi-stops-working-after-power-outage/][modem bad state]] +- Specific outbound port blocked while web works (no). Browsing is fine but email won't send or a service times out. The network/ISP blocks that port (SMTP 25, some VPN ports). A partial block, not a full outage. Fix: use an allowed alternate (submission 587, VPN over 443/TCP), or switch networks — the policy isn't host-fixable. [[https://help.dreamhost.com/hc/en-us/articles/217071167-Port-25-Blocking][port 25 blocking]] + +** NetworkManager control plane / backends / config + +- NetworkManager service masked or failed to start (yes). No networking at all; "Unit NetworkManager.service is masked" or the unit dies immediately. The unit was masked, or an invalid unit file / malformed NetworkManager.conf blocks startup. Fix: =systemctl unmask NetworkManager && systemctl start NetworkManager=; reinstall if the unit is corrupt, validate the conf. [[https://bbs.archlinux.org/viewtopic.php?id=193036][arch 193036]] +- NetworkManager segfaults and restart-loops (yes). Connectivity drops intermittently; journal "Failed with result 'core-dump'" and repeated respawns. A bug in the running NM build crashes the daemon. Fix: =coredumpctl info NetworkManager=, then update/downgrade the package, restart NM. [[https://bbs.archlinux.org/viewtopic.php?id=236562][arch 236562]] +- iwd and wpa_supplicant both active, backends fight (both). Wifi scans but never associates, or associates then drops; both daemons drive one radio. Fix: pick one backend, =systemctl disable --now= the other, set =[device] wifi.backend==, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=292603][arch 292603]] +- wifi.backend=iwd configured but iwd not installed/running (both). No wifi device appears at all; NM acts as if there's no radio. NM is told to use iwd but the package isn't installed or its service can't start. Fix: =pacman -S iwd= (or remove the drop-in to fall back), reboot. [[https://bbs.archlinux.org/viewtopic.php?id=305442][arch 305442]] +- systemd-networkd and NetworkManager both managing the link (both). The interface gets an IP then loses it, or flaps; two managers own one NIC. Fix: =systemctl disable --now systemd-networkd systemd-networkd.socket= (mask if dbus-activated), reboot. [[https://bbs.archlinux.org/viewtopic.php?id=184164][arch 184164]] +- Keyfile permissions void a saved connection (yes). A saved network shows in nmcli but never activates; journal marks it insecure and skips it. The =.nmconnection= isn't 600 root-owned. Fix: =chmod 600= and =chown root:root= the file in system-connections, =nmcli connection reload=. [[https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/8/html/configuring_and_managing_networking/assembly_networkmanager-connection-profiles-in-keyfile-format_configuring-and-managing-networking][rhel keyfile]] +- Autoconnect disabled on the profile (no). The network works when activated by hand but never comes up on boot/resume. =connection.autoconnect=no=. Fix: =nmcli connection modify <name> connection.autoconnect yes= (user-scope). [[https://bbs.archlinux.org/viewtopic.php?id=281611][arch 281611]] +- Duplicate/conflicting profiles for one SSID (no). Two entries for the same network; the wrong one activates. Leftover =.nmconnection= files created a second profile with conflicting settings. Fix: =nmcli connection delete <uuid>= the stale one, or raise the good one's =autoconnect-priority=. [[https://bbs.archlinux.org/viewtopic.php?id=274552][arch 274552]] +- A dispatcher script errors and blocks connectivity (yes). Network events don't complete, or a connectivity hook silently doesn't run. A script in dispatcher.d fails on a permission/ownership problem or loops. Fix: make it root-owned and non-group-writable or remove it; debug via =journalctl -u NetworkManager-dispatcher=. [[https://man.archlinux.org/man/NetworkManager-dispatcher.8.en][arch dispatcher]] +- Polkit won't authorize a non-root user to control networking (no). =nmcli= up/modify fails "Not authorized to control networking". No polkit agent, or the login has no active seat session. Fix: start a polkit agent and ensure a proper loginctl session (relogin via the display manager), or add a polkit rule. [[https://bbs.archlinux.org/viewtopic.php?id=141465][arch 141465]] +- No reconnect after suspend/resume (yes). After waking, wifi/wired stays down until the service is kicked. On resume NM doesn't re-establish the link. Fix: =systemctl restart NetworkManager= (the doctor's nm-restart); durable fix is a system-sleep hook. [[https://bbs.archlinux.org/viewtopic.php?id=234725][arch 234725]] +- NetworkManager-wait-online hangs boot (both). Boot stalls ~90s on NetworkManager-wait-online; the desktop is late. The service waits for connectivity that never arrives (unplugged NIC, non-autoconnect port). Fix: =systemctl disable NetworkManager-wait-online.service= (or drop the unused profile); effective next boot. [[https://bbs.archlinux.org/viewtopic.php?id=298817][arch 298817]] + +* Bluetooth failure modes + +Five layers, mirroring the bt doctor's chain (adapter present → rfkill → bluetooth.service → adapter powered → per-device → audio profile). ~70 raw entries from the sweep; hardware-block duplicates (a Bluetooth hard-block appears in every one of the first three layers) and the pairing/connection overlaps are merged inline, leaving ~55 distinct root causes. + +** Adapter / hardware / driver / firmware + +- btusb module not auto-loaded (yes). lsusb shows the adapter but no hci0, =/sys/class/bluetooth= empty, "No default controller available". btusb didn't load at boot (missing modules-load entry or a blacklist). Fix: =modprobe btusb=, persist via =/etc/modules-load.d/bluetooth.conf=. [[https://bbs.archlinux.org/viewtopic.php?id=267428][arch 267428]] +- Intel ibt-*.sfi firmware missing/mismatched (both). No controller; dmesg "Direct firmware load for intel/ibt-NN-N.sfi failed with error -2". linux-firmware lacks the exact .sfi/.ddc the controller requests. Fix: update linux-firmware (or symlink the requested name), reboot. [[https://bbs.archlinux.org/viewtopic.php?id=302314][arch 302314]] +- Intel "Failed to send firmware data" during load (both). Adapter present then dies; "hci0: Failed to send firmware data (-38)/(-110)". Firmware transfer aborts (USB/init timing, often kernel-specific). Fix: =options btusb reset=1= and reload, or downgrade the kernel, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=248784][arch 248784]] +- Intel AX200/AX210 controller lost after a kernel update (both). WiFi fine, BT gone, "No default controller available" right after an upgrade. A kernel/firmware regression stops the BT half enumerating. Fix: boot linux-lts or downgrade until fixed, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=282611][arch 282611]] +- Intel broken-initial-NCMD init hang (both). "hci0: command tx timeout" during setup; the controller never comes up. Some Intel controllers need the BTUSB_INTEL_BROKEN_INITIAL_NCMD quirk absent on older kernels. Fix: run a kernel new enough to carry the quirk, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=226312][arch 226312]] +- MediaTek MT7921/MT7961 firmware filename mismatch (both). No adapter; "Direct firmware load for mediatek/BT_RAM_CODE_MT7961_1a_2_hdr.bin failed with error -2". linux-firmware ships the blob compressed or under a different name. Fix: decompress/rename to the requested file, reload btusb, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=296026][arch 296026]] +- MediaTek MT7922 btmtk regression after a kernel bump (both). BT stopped initializing after a specific kernel; WiFi unaffected. A regression in the mt7921e/btmtk path. Fix: downgrade to the last working kernel (or linux-lts), reboot. [[https://bbs.archlinux.org/viewtopic.php?id=313561][arch 313561]] +- MediaTek MT7922 stuck in a residual power state (physical). "No Bluetooth adapters found" persists across warm reboots; only a full power-off clears it. The controller latched in a bad state a soft reboot doesn't reset. Fix: shut down, pull power ~10s, cold boot. [[https://bbs.archlinux.org/viewtopic.php?id=308169][arch 308169]] +- MediaTek MT7920/MT7922 bogus firmware in linux-firmware (both). WiFi works, BT half fails "Failed to set up firmware" after a linux-firmware update. A shipped BT blob is broken in the current package. Fix: downgrade linux-firmware-mediatek to the prior version, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=311544][arch 311544]] +- MediaTek MT7925 killed by aggressive USB autosuspend (both). BT flaky/absent on a recent kernel; "Opcode 0x0c03 failed: -16". Kernel power-saving suspends the controller. Fix: =usbcore.autosuspend=-1 btusb.enable_autosuspend=n= on the kernel cmdline, reboot. [[https://bbs.archlinux.org/viewtopic.php?id=306366][arch 306366]] +- Realtek RTL8761B(U) firmware name mismatch (yes). USB dongle dead; driver requests rtl_bt/rtl8761bu_fw.bin but firmware ships rtl8761b_fw.bin. Fix: symlink to the bu names in /lib/firmware/rtl_bt (or update linux-firmware), replug — no reboot. [[https://bbs.archlinux.org/viewtopic.php?id=309671][arch 309671]] +- Realtek RTL8852BU/newer dongle needs an out-of-tree driver (both). A recent combo dongle isn't recognized at all; no in-kernel support. Fix: build the vendor DKMS driver (lwfinger/rtl8852bu), reboot. [[https://github.com/lwfinger/rtl8852bu][rtl8852bu]] +- Fake/clone CSR dongle resets and times out (yes). A cheap 0a12:0001 "CSR" dongle works briefly then "command tx timeout"; "Unbranded CSR clone detected". A counterfeit controller mishandles CSR reset. Fix: rely on the kernel clone quirks, or =options btusb reset=1 enable_autosuspend=0=, reload btusb. [[https://bbs.archlinux.org/viewtopic.php?id=272230][arch 272230]] +- Qualcomm WCN6855/QCA6390 version read fails (both). No BT; "Reading QCA version information failed (-110)" via ath11k. Missing/mismatched QCA firmware or a btqca init-timing bug. Fix: update linux-firmware and run a newer kernel (or reload ath11k_pci/btusb), reboot. [[https://bbs.archlinux.org/viewtopic.php?id=284506][arch 284506]] +- Broadcom BCM .hcd patch file missing (both). Adapter enumerates then stalls; "hci0: BCM: Patch brcm/BCMxxxxx-….hcd not found". The .hcd firmware isn't in linux-firmware. Fix: install broadcom-bt-firmware (AUR) or drop the correct .hcd into /lib/firmware/brcm, reboot. [[https://github.com/winterheart/broadcom-bt-firmware/issues/30][broadcom-bt-fw 30]] +- USB autosuspend powers the adapter off after suspend/idle (both; the hardware angle — see also the resume entry in the rfkill layer). BT works at boot, then hci0 vanishes after a suspend/resume or idle; only a full shutdown restores it. btusb autosuspend cuts power and the controller doesn't recover. Fix: =btusb.enable_autosuspend=0= on the kernel cmdline (or exclude the device in TLP), reboot. [[https://forum.manjaro.org/t/bluetooth-adapter-disappears-after-suspend-only-shutdown-can-fix/157919][manjaro 157919]] +- Bluetooth hard-blocked by a BIOS/hardware switch (physical; the canonical hard-block — appears identically across the rfkill layer). No adapter and =rfkill list= shows Bluetooth "Hard blocked: yes" (or hci0 absent) after a wireless switch/Fn toggle or a BIOS setting. Fix: flip the physical switch / Fn key, or enable Bluetooth in BIOS/UEFI — no software can clear a hard block. [[https://github.com/systemd/systemd/issues/4117][systemd 4117]] + +** rfkill / power / service + +- Soft-blocked adapter needs manual unblock (no). =rfkill list= shows Bluetooth "Soft blocked: yes"; bluetoothctl "Failed to set power on: org.bluez.Error.Failed". A prior software toggle left the soft-block set and bluetoothd won't power a blocked adapter. Fix: =rfkill unblock bluetooth= (then =bluetoothctl power on=). [[https://bbs.archlinux.org/viewtopic.php?id=169530][arch 169530]] +- AutoEnable off leaves BT powered down every boot (yes). Adapter present and service running, but "Powered: no" at every login. bluez main.conf has AutoEnable=false (also the static default when main.conf is absent), so bluetoothd never powers the controller. Fix: =AutoEnable=true= under [Policy] in main.conf, restart bluetooth. [[https://github.com/bluez/bluez/issues/886][bluez 886]] +- bluetooth.service masked (yes). =systemctl start bluetooth= returns "Unit bluetooth.service is masked". The unit was masked by an earlier disable or a tuning guide. Fix: =systemctl unmask bluetooth.service && systemctl enable --now bluetooth=. [[https://bbs.archlinux.org/viewtopic.php?id=220081][arch 220081]] +- bluetooth.service not enabled at boot (yes). BT works after a manual start but is "inactive (dead)" on every fresh boot. On Arch the service ships disabled, so nothing starts bluetoothd at boot. Fix: =systemctl enable --now bluetooth.service=. [[https://bbs.archlinux.org/viewtopic.php?id=287428][arch 287428]] +- D-Bus policy blocks bluetoothd from owning org.bluez (yes). "D-Bus setup failed: Connection is not allowed to own the service 'org.bluez'"; bluetoothctl can't reach bluez. A missing/broken =/usr/share/dbus-1/system.d/bluetooth.conf= policy (partial install/upgrade). Fix: reinstall bluez to restore the policy, restart dbus and bluetooth. [[https://bbs.archlinux.org/viewtopic.php?id=283337][arch 283337]] +- bluez upgrade leaves a bad unit-file setting (yes). After a bluez update "Unit bluetooth.service has a bad unit file setting". A changed ExecStart path or a removed option in the shipped unit breaks parsing until daemon-reload. Fix: reconcile the .pacnew, correct ExecStart to =/usr/lib/bluetooth/bluetoothd=, =daemon-reload && restart bluetooth=. [[https://github.com/bluez/bluez/issues/736][bluez 736]] +- Airplane mode / physical switch hard-blocks BT (physical; the rfkill-layer view of the shared hard-block). =rfkill list= "Hard blocked: yes"; software toggles and unblock do nothing. A physical/firmware kill switch, Fn key, or BIOS airplane state holds the block. Fix: turn airplane mode off / flip the switch, or enable BT in BIOS. [[https://bugzilla.redhat.com/show_bug.cgi?id=713006][rh 713006]] +- WiFi rfkill toggle also kills BT on a combo chip (no). Turning WiFi off drops Bluetooth too, and both come up soft-blocked at boot. A shared WiFi/BT combo exposes a coupled rfkill so the wlan block cascades. Fix: =rfkill unblock bluetooth= after the toggle; unblock all at boot. [[https://forum.artixlinux.org/index.php/topic,5062.0.html][artix 5062]] +- TLP disables Bluetooth on startup (yes). BT is soft-blocked at every boot on a TLP laptop though the service is fine. TLP's =DEVICES_TO_DISABLE_ON_STARTUP= includes bluetooth (or RESTORE_DEVICE_STATE restores off). Fix: remove bluetooth from that list in =/etc/tlp.conf=; =rfkill unblock bluetooth= now. [[https://dev.to/sharafat/disable-bluetooth-at-startup-in-tlp-2k8d][tlp startup]] +- systemd-rfkill restores a stale soft-block at boot (yes). BT was on before shutdown but boots soft-blocked, or airplane-off never sticks. systemd-rfkill saved a blocked state to =/var/lib/systemd/rfkill= and reapplies it (a hotplug race can overwrite the good state). Fix: =rfkill unblock bluetooth=; clear the saved state (or mask systemd-rfkill if TLP owns radio state). [[https://github.com/systemd/systemd/issues/42148][systemd 42148]] +- rfkill block puts bluetoothd into a 100% CPU loop (yes). After =rfkill block bluetooth= (an airplane toggle) bluetoothd spins at 100% CPU and the adapter is unresponsive even after unblock. A bluez rfkill-handling regression busy-loops. Fix: =systemctl restart bluetooth= after unblocking; upgrade/downgrade bluez past the affected version. [[https://github.com/bluez/bluez/issues/785][bluez 785]] +- Adapter powers back off after a desktop-driven power-off (no). =bluetoothctl power on= fails specifically after BT was switched off in GNOME/desktop settings; only the desktop toggle restores it. The desktop's saved off-state re-blocks the adapter under the CLI. Fix: re-enable from the desktop toggle, or =rfkill unblock bluetooth= and power on. [[https://discourse.gnome.org/t/cannot-power-on-bluetooth-from-bluetoothctl-if-it-was-powered-off-from-the-gnome-settings/24782][gnome 24782]] +- BT soft-blocked/unresponsive on resume, needs a module reload (yes; distinct from the hardware-autosuspend case — here power state is stuck, not cut). After suspend/resume the adapter is lost or soft-blocked; toggling "shows" enabled but does nothing and restarting the service doesn't help. btusb fails to re-init on resume, leaving a stuck rfkill/power state. Fix: =rmmod btusb && modprobe btusb= (then restart bluetooth); automate via a post-resume sleep hook. [[https://forums.opensuse.org/t/btusb-broken-after-resuming-from-sleep/133499][opensuse 133499]] +- Wrong/missing default controller with multiple adapters (no; see also the multi-controller pairing case). "No default controller available" or commands hit the wrong hciN when a dongle and an onboard radio are both present. bluez picked a different default than the user wants. Fix: =bluetoothctl select <MAC>= then power on; pin it if needed. [[https://bbs.archlinux.org/viewtopic.php?id=286766][arch 286766]] + +** Pairing / bonding + +- Generic AuthenticationFailed on pair (no). "Failed to pair: org.bluez.Error.AuthenticationFailed" on an audio device. A half-written or contested bond from a prior attempt makes the device reject the new authentication. Fix: bluetoothctl disconnect, untrust, remove <mac>, restart bluetooth, pair from a clean scan; factory-reset the device if it persists. [[https://bbs.archlinux.org/viewtopic.php?id=267381][arch 267381]] +- Paired but not trusted, no auto-reconnect (no). Device pairs but never reconnects on its own after a reboot/power cycle; =info= shows "Trusted: no". bluez won't auto-accept a connection from an untrusted bond. Fix: =bluetoothctl trust <mac>=. [[https://techwiser.com/fix-bluetooth-device-doesnt-auto-connect-in-linux/][techwiser autoconnect]] +- Stale link key after a dual-boot re-pair (yes). A device paired in Windows no longer connects in Linux with AuthenticationFailed. A device stores one link key per host; re-pairing under the other OS overwrote it. Fix: =bluetoothctl remove <mac>= and pair again, or sync the keys between OSes (bt-dualboot / edit the LinkKey under /var/lib/bluetooth). [[https://armujahid.me/blog/dual-boot-bluetooth-pairing/][armujahid dualboot]] +- No default pairing agent registered (no). Pairing a PIN/keyboard device silently fails or the passkey prompt never appears (esp. headless). bluetoothctl's built-in agent is unregistered once it exits, so nothing answers the request. Fix: =agent on= then =default-agent= (or a persistent bt-agent / NoInputNoOutput agent for unattended Just Works). [[https://technotes.kynetics.com/2018/pairing-agents-bluez/][kynetics agents]] +- BLE device never appears in scan (no). A BLE peripheral shows in other OSes but never in =bluetoothctl scan=, while BR/EDR devices scan fine. The scan defaults to BR/EDR-only or LE isn't enabled. Fix: =menu scan / transport le= (or =btmgmt le on=); last resort =ControllerMode = le= in main.conf. [[https://bbs.archlinux.org/viewtopic.php?id=281064][arch 281064]] +- Corrupt device cache survives remove (yes). Repeated pairing failures for one device; =remove= reports success but the cache stays and the bad state returns. bluez doesn't fully purge =/var/lib/bluetooth/<adapter>/<mac>=. Fix: stop bluetooth, =rm -rf= the stale device dir, start bluetooth, pair fresh. [[https://github.com/bluez/bluez/issues/191][bluez 191]] +- ConnectionAttemptFailed / br-connection-page-timeout (yes). Pairing or first connect fails "Page Timeout" / "br-connection-page-timeout", often after a bluez upgrade. The daemon's baseband page times out; a version regression triggered it. Fix: =systemctl restart bluetooth= and retry; downgrade bluez if it recurs after an update. [[https://github.com/bluez/bluez/issues/230][bluez 230]] +- Re-pair loop: pairs then instantly disconnects (yes). Device pairs, drops within seconds, and re-pairing only works until the next attempt. A corrupt in-memory bond/device DB wedges the daemon. Fix: =rm -rf /var/lib/bluetooth/*= (or the device dir) and =systemctl restart bluetooth=, then pair once cleanly. [[https://bbs.archlinux.org/viewtopic.php?id=267233][arch 267233]] +- Headset un-pairs itself on disconnect (yes). A headset loses its bond every time it disconnects. The device sends an HID virtual_cable_unplug on disconnect and bluez treats it as an un-pair. Fix: pair again; no clean user setting — avoid fully powering the device off; restart bluetooth if the bond is gone. [[https://github.com/bluez/bluez/issues/2048][bluez 2048]] +- BLE address rotates every session (physical). A BLE device pairs once but can't reconnect; its address differs each power cycle. It uses a random/resolvable-private address, so a bond keyed to the old address won't match unless IRK resolution works. Fix: pair so the IRK is exchanged (bond, not just connect), or disable address privacy on the device; re-pair to the current identity. [[https://github.com/hbldh/bleak/issues/363][bleak 363]] +- Legacy keyboard needs the PIN typed on the device (physical). A legacy BT keyboard fails with an authentication timeout during pairing. It uses legacy PIN/passkey-entry: bluez shows a passkey that must be typed on the keyboard itself. Fix: register a KeyboardDisplay agent, type the displayed passkey on the keyboard, press Enter within the timeout. [[https://bbs.archlinux.org/viewtopic.php?id=174239][arch 174239]] +- Too many bonded devices, keys lost (yes). With many devices paired, after a reboot or dongle re-insert none connect. Past ~14 bonds bluez fails to restore the LTKs. Fix: =bluetoothctl remove= enough devices to drop below the limit, restart bluetooth. [[https://github.com/bluez/bluez/issues/201][bluez 201]] +- Dual-mode device pairs on the wrong transport (yes). A dual-mode device (AirPods) refuses to pair/connect because bluez tries BR/EDR when the device needs LE, with no per-device override. Fix: =ControllerMode = bredr= (or le) in main.conf [General], restart, pair, then restore =dual=. [[https://github.com/bluez/bluez/issues/514][bluez 514]] +- Adapter Pairable/Discoverable off (no). Scan finds nothing or the controller rejects pairing. The adapter has =Pairable: no= / =Discoverable: no=. Fix: =power on=, =pairable on=, =scan on= (or set the main.conf defaults). [[https://wiki.archlinux.org/title/Bluetooth][archwiki bt]] +- Target device not in pairing mode (physical). A peripheral never appears in scan though the adapter works. The device is powered but not advertising. Fix: hold the pairing button until the LED signals, re-run =scan on=. [[https://bbs.archlinux.org/viewtopic.php?id=261540][arch 261540]] +- Paired without a bond via blueman (no). A gamepad/mouse paired through blueman connects but is unusable, while bluetoothctl works. blueman completed pairing without establishing bonding, so no usable link key was stored. Fix: remove the device and =bluetoothctl pair <mac>= (which bonds), then trust and connect. [[https://github.com/blueman-project/blueman/issues/2219][blueman 2219]] + +** Connection stability + +- Xbox/PS controller drops seconds after connecting unless ERTM is off (both). A controller pairs and connects, then disconnects within seconds, repeatedly. bluez's Enhanced Retransmission Mode is incompatible with these controllers' L2CAP handling. Fix: =echo 1 > /sys/module/bluetooth/parameters/disable_ertm=, persist =options bluetooth disable_ertm=1= in modprobe.d, reboot. [[https://github.com/atar-axis/xpadneo/issues/295][xpadneo 295]] +- Paired+trusted device never reconnects on boot/resume (no). A trusted HID/audio device stays disconnected after boot or resume and needs a manual connect each time. AutoEnable only powers the adapter; it doesn't initiate outbound connections to peripherals that don't self-reconnect. Fix: install and enable bluetooth-autoconnect (a systemd service that connects trusted devices when the adapter powers on, including post-resume). [[https://github.com/jrouleau/bluetooth-autoconnect][bt-autoconnect]] +- ReconnectIntervals/AutoEnable policy leaves the link dead after a drop (yes). After a device drops out of range and returns, bluetoothd doesn't re-establish the link. The [Policy] AutoEnable and [GATT] ReconnectIntervals settings are unset or too sparse. Fix: set =AutoEnable=true= and =ReconnectIntervals=1,2,4,8,16,32,64= in main.conf, restart bluetooth. [[https://wiki.archlinux.org/title/Bluetooth][archwiki bt]] +- USB 3.0 port radio noise degrades BT on the same side of the chassis (physical). A BT mouse/headset stutters or drops, worst under USB load or with an external SSD in an adjacent USB3 port; SuperSpeed signaling adds ~20 dB of broadband noise across 2.4-2.5 GHz. Fix: move the dongle/device off the USB3 port (USB2 port or an extension cable to distance the receiver). [[https://www.usb.org/sites/default/files/327216.pdf][usb-if 327216]] +- WiFi/BT coexistence on a 2.4GHz combo chip causes stutter and drops (yes). On an AX200 (and similar combos) BT audio/HID stutters and 2.4GHz WiFi collapses when both radios are active, sharing one antenna path. Fix: move WiFi to 5GHz, or tune coexistence via =/etc/modprobe.d/iwlwifi.conf=, reload iwlwifi. [[https://community.intel.com/t5/Wireless/Wi-Fi-6-AX200-Bluetooth-causing-WiFi-to-disconnect-randomly/td-p/1437797][intel ax200]] +- btusb USB autosuspend causes input lag/dropout after idle (both; the connection-health view of the adapter-layer autosuspend cause). A BT mouse/keyboard lags or briefly disconnects after no input, then wakes sluggishly; the kernel autosuspends the btusb controller. Fix: =btusb.enable_autosuspend=0= on the kernel cmdline (or a udev =power/control=on= rule), reboot. [[https://community.frame.work/t/solved-bluetooth-mouse-lag-linux-autosuspend/26763][framework autosuspend]] +- Adapter dead / devices won't reconnect after suspend-resume (yes; the same resume-init failure the rfkill layer's module-reload entry names, from the device-connection side). After resume the adapter is gone or devices refuse to reconnect (dmesg shows btusb resume errors). The controller doesn't survive suspend/resume cleanly on some kernels. Fix: a systemd sleep hook that runs =rmmod btusb; modprobe btusb= (or restarts bluetooth) post-resume. [[https://bugzilla.redhat.com/show_bug.cgi?id=2100761][rh 2100761]] +- BLE device disconnects with reason 0x08 from a too-aggressive supervision timeout (yes). A BLE peripheral connects then drops after a few seconds with reason 0x08 (Timeout); the negotiated supervision timeout is too short for the interval and latency. Fix: raise it via =/sys/kernel/debug/bluetooth/hci0/supervision_timeout=, persist with a startup service (debugfs resets on reboot). [[https://bbs.archlinux.org/viewtopic.php?id=279872][arch 279872]] +- Realtek RTL8761B wrong-firmware reconnect-loop (yes; same root cause as the adapter-layer name-mismatch entry, manifesting as a fast reconnect loop). An RTL8761B/BU dongle disconnects and reconnects several times a second; autodetect requests the wrong firmware name (or a bad blob). Fix: symlink the correct blob in /lib/firmware/rtl_bt, reload btusb. [[https://bbs.archlinux.org/viewtopic.php?id=309671][arch 309671]] +- Disconnected/loose internal antenna gives a few-feet range then drops (physical). BT works only within a couple of feet and cuts out beyond; a U.FL antenna lead popped off the card or none is attached. Fix: reseat the U.FL coax, or use an external USB adapter; keep the device closer as a stopgap. [[https://community.intel.com/t5/Wireless/How-can-I-deal-with-the-extremely-limited-range-of-Bluetooth-in/td-p/692325][intel range]] +- "Software caused connection abort" on connect-after-scan (no). A device with a fine bond aborts with "Software caused connection abort" (err 103) when a scan was stopped just before connecting; stopping discovery races the connection attempt in bluez. Fix: keep discovery running across the connect, or retry without a preceding scan-stop. [[https://github.com/hbldh/bleak/issues/631][bleak 631]] +- A2DP audio stutters whenever a BT mouse is also connected (no). Headset audio skips only while a BT mouse/HID is connected on the same adapter; the two profiles contend for the single controller's scheduling. Fix: switch the audio codec (SBC → SBC-XQ/LDAC) to lower airtime, or move the mouse to a separate dongle; disconnecting the mouse clears it. [[https://bbs.archlinux.org/viewtopic.php?id=279749][arch 279749]] +- BT mouse stutters at ~20 Hz unless a scan is active (no). Mouse motion is choppy but becomes smooth the instant a scan starts; without background inquiry the controller's scheduling starves the HID link. Fix: keep discovery on as a workaround, or apply the connection-parameter fix below. [[https://bbs.archlinux.org/viewtopic.php?id=261632][arch 261632]] +- BLE mouse lag from a too-high negotiated connection interval (yes). A BLE mouse feels laggy because it negotiated a power-saving (long) connection interval and high slave latency. Fix: set =[ConnectionParameters]= (MinInterval=6, MaxInterval=9, Latency=44, Timeout=216) in =/var/lib/bluetooth/<adapter>/<device>/info=, restart bluetooth. [[https://wiki.archlinux.org/title/Bluetooth_mouse][archwiki bt mouse]] + +** Bluetooth audio profiles / device classes + +Bluetooth-specific audio failures the bt doctor names at its last chain link. The generic PipeWire/WirePlumber internals are owned by [[file:2026-07-10-audio-failure-taxonomy.org][the audio taxonomy]]; these are the ones a bt-panel probe would surface for a connected BT audio device. + +- Stale bluez pairing state hides A2DP (yes). Headset connects but only HSP/HFP exist, no A2DP sink. Corrupt/partial pairing records in /var/lib/bluetooth leave A2DP unadvertised. Fix: =rm -rf /var/lib/bluetooth/<adapter>/<dev>=, restart bluetooth, re-pair. [[https://bbs.archlinux.org/viewtopic.php?id=280717][arch 280717]] +- Leftover pulseaudio-bluetooth module steals the endpoint (yes). After moving to PipeWire the device connects but no sink shows. pulseaudio-bluetooth still installed grabs the bluez D-Bus endpoint so pipewire-pulse never creates the card. Fix: =pacman -Rns pulseaudio-bluetooth pulseaudio=, install pipewire-pulse, restart the session. [[https://bbs.archlinux.org/viewtopic.php?id=267690][arch 267690]] +- bluez SPA plugin not installed, no card at all (yes). Device connects at the protocol level but pw-dump shows no bluez node; "api.bluez5.enum.dbus could not be loaded". The libspa bluetooth plugin is missing, so WirePlumber's bluez monitor can't enumerate it. Fix: install the PipeWire bluetooth SPA plugin (libspa-0.2-bluetooth on split distros), restart wireplumber. [[https://bbs.archlinux.org/viewtopic.php?id=304688][arch 304688]] +- WirePlumber seat-monitoring drops the node (no). A connected BT audio device intermittently has no card/node on a non-logind or nested seat. WirePlumber gates node creation on an active logind seat and misfires. Fix: =monitor.bluez.seat-monitoring = disabled= drop-in, restart wireplumber. [[https://pipewire.pages.freedesktop.org/wireplumber/daemon/configuration/bluetooth.html][wp bt config]] +- Autoswitch-to-headset fires on any input stream (no). Music drops to mono phone-call quality whenever any app opens the mic. WirePlumber's =bluetooth.autoswitch-to-headset-profile= flips to HSP/HFP on any capture stream and doesn't switch back. Fix: =wpctl settings --save bluetooth.autoswitch-to-headset-profile false=, or force =a2dp-sink= via set-card-profile. [[https://bbs.archlinux.org/viewtopic.php?pid=1973004][arch 1973004]] +- Codec falls back to SBC, wanted codec never negotiates (no). aptX/LDAC-capable headphones connect but only SBC is active. Default codec order drops to the SBC baseline, or the wanted codec isn't in the allowed set. Fix: =bluez5.codecs = [ ldac aptx_hd aptx aac sbc ]= in a wireplumber drop-in, reconnect. [[https://bbs.archlinux.org/viewtopic.php?id=300230][arch 300230]] +- AAC unavailable because PipeWire lacks fdk-aac (yes). An AAC-only headset never offers AAC and lands on SBC. PipeWire was built without the fdk-aac module (non-free, omitted on some builds). Fix: install libfdk-aac and a PipeWire build with =bluez5-codec-aac= enabled. [[https://bbs.archlinux.org/viewtopic.php?id=289177][arch 289177]] +- LDAC negotiates but adaptive bitrate stays low (no). LDAC is active yet sounds no better than SBC; bitrate never reaches 990k. The default =ldac.quality = auto= collapses to the low tier on a busy 2.4GHz link. Fix: pin =bluez5.a2dp.ldac.quality = "hq"= in a monitor.bluez.rules drop-in, reconnect. [[https://www.guyrutenberg.com/2025/01/09/configuring-ldac-quality-in-pipewire/][guyrutenberg ldac]] +- BT sink not made default, audio stays on laptop speakers (no). Headphones connect but sound keeps coming from the built-in speakers until switched by hand. WirePlumber doesn't promote a freshly-connected BT sink, and the default isn't restored across reconnect. Fix: =wpctl set-default <bt-sink-id>= now; persist with a priority.session bump or default-target restore. [[https://bbs.archlinux.org/viewtopic.php?id=300751][arch 300751]] +- Absolute-volume passthrough makes the slider a no-op (both). The OS volume slider does nothing or only toggles mute/max. AVRCP absolute-volume hands volume to device hardware that only honors 0/100. Fix: =bluez5.enable-hw-volume = false= in a wireplumber drop-in, restart pipewire+wireplumber. [[https://wiki.archlinux.org/title/Bluetooth_headset][archwiki bt headset]] +- Silent sink after suspend/resume despite reconnect (no; the BT-transport view of the audio taxonomy's resume-silence family). After wake the device reconnects but the playing stream is silent; audio returns only on a new track. The resumed transport isn't re-initialized for the existing stream. Fix: =wpctl set-default= to bounce the stream, or disable suspend-on-idle for the node; scripted disconnect/reconnect on resume as a workaround. [[https://forums.linuxmint.com/viewtopic.php?p=2781321][mint 2781321]] +- oFono present blocks native HFP / mSBC (yes; overlaps the audio taxonomy's BT-mic cluster — kept here for the bt-doctor probe). No HFP profile or only CVSD narrowband; mSBC never appears. An installed-but-unconfigured oFono claims the HFP backend so the native backend (which offers mSBC) stays inactive. Fix: remove ofono (or set the native HFP backend), restart pipewire, reconnect. [[https://bbs.archlinux.org/viewtopic.php?id=266252][arch 266252]] +- mSBC wideband unsupported by adapter/kernel (physical). HFP mic works only in muffled narrowband; mSBC never negotiates even on the native backend. mSBC needs adapter+kernel support for USB ALT6 / the transparent SCO path, which some controllers lack. Fix: none in software — use a different adapter, or accept CVSD narrowband. [[https://www.collabora.com/news-and-blog/news-and-events/pipewire-bluetooth-support-status-update.html][collabora bt]] +- HFP profile output dead, mic works (no). In HSP/HFP the mic captures fine but there's no playback through the headset. The native HFP backend's SCO output path fails to route on certain PipeWire versions/devices. Fix: force a2dp-sink for playback and only enter HFP for the call; try switching the HFP backend or a PipeWire version without the regression. [[https://discussion.fedoraproject.org/t/no-audio-output-with-bluetooth-headset-in-hsp-hfp-profile-mic-works-output-doesn-t/155135][fedora hfp-output]] +- Device seen as LE/GATT-only, no audio class (yes). A headset connects but is treated as a data/LE device with no A2DP sink. bluez brings it up over LE (GATT only) instead of Classic/BR-EDR. Fix: =DisableLE= (or =ControllerMode = bredr=) in main.conf, restart bluetooth, remove and re-pair. [[https://bbs.archlinux.org/viewtopic.php?id=303288][arch 303288]] +- Battery percent never reported (yes). =bluetoothctl info= shows no Battery line and the tray shows no BT battery. bluez's battery provider is gated behind experimental mode. Fix: =Experimental = true= under [General] in main.conf, restart bluetooth, reconnect (the device must expose the Battery Service). [[https://github.com/bluez/bluez/issues/362][bluez 362]] +- Multi-device: first-connected BT audio wins, wrong default (no). Two BT audio devices paired; audio always goes to whichever connected first and the other can't be made default cleanly. WirePlumber keeps the earlier sink as default. Fix: =wpctl set-default <desired-sink-id>=; give the preferred device a higher priority.session via a monitor.bluez.rules match. [[https://0110.be/posts/Connecting_two_Bluetooth_headsets_to_your_Linux_system:_audio_routing_in_PipeWire][0110.be dual bt]] +- HFP narrowband mic during calls is an expected tradeoff (no; GUIDE). Opening the mic (calls/meetings) tanks playback to tinny mono. Inherent to HFP/HSP sharing one low-bandwidth SCO link for bidirectional audio; A2DP has no mic path. Fix: none to repair — keep A2DP for listening, accept HFP for calls, or use a duplex codec (FastStream / aptX-LL) that carries the mic without leaving A2DP. [[https://webcodr.io/2024/10/fixing-no-a2dp-with-bluetooth-headsets-on-linux/][webcodr a2dp]] + +* Network triage + +The ~74 network failure modes sorted into eight symptom clusters, each owned by a tier the net doctor already probes (link → IP → route → DNS → egress), plus a control-plane cluster for NetworkManager itself and a drops cluster for intermittent faults. Every entry carries a remedy class, using the audio taxonomy's four: + +- *AUTO* — user-scope, reversible, no root. For net this is the nmcli surface the doctor already drives (bounce/reset, =con modify= for dns/mtu/ipv6/mac/autoconnect, tunnel-down, portal-open) plus user-service restarts. Polkit on an active session (or passwordless sudo on an archsetup install) covers these. +- *PRIV* — needs root: =systemctl restart NetworkManager/systemd-resolved=, editing =/etc/=, installing a firmware/driver package, a modprobe option. Runs silently where passwordless sudo exists, else Confirm/Arm. +- *REBOOT-TAIL* — the runnable part (often PRIV: firmware/driver install, initramfs rebuild, kernel change, a modprobe option) then the reboot the doctor instructs but can't verify past. +- *GUIDE* — nothing to run: the user must supply a secret/cert, flip a physical switch, change a BIOS setting, make a config trade-off, or wait out a venue/ISP outage. + +Rough distribution: ~30 AUTO, ~14 PRIV, ~12 REBOOT-TAIL, ~24 GUIDE (several are AUTO-now-plus-PRIV-to-persist, or AUTO-plus-GUIDE where the fix is a decision). The buildable share (AUTO plus the runnable part of PRIV/REBOOT-TAIL) is a little over half — the same shape as audio. Triage is a proposal; correct any class. + +Two structural notes carried from the audio design. First, the clusters map onto the probe ladder, so the fix/guide boundary falls along tiers the doctor already walks — cluster N is "the fault the Nth probe step catches". Second, the AUTO core is a handful of primitives (bounce/reset, an nmcli-setter, tunnel-down, a user-service restart, portal-open); the breadth is in the *diagnosis*, not the remedies — exactly as the audio doctor's ~50 diagnoses collapsed to five remedy primitives. + +** Cluster 1 — no link (radio/carrier down) +Probe: the link step (=_step_link=: rfkill, has-wifi-device, carrier). New work: a dmesg firmware-load hint would let the doctor name the specific blob (iwlwifi/rtw89/mt7921) rather than a generic "no adapter". + +- Wifi soft-blocked by rfkill :: AUTO — =rfkill unblock wifi= / =nmcli radio wifi on= (the doctor's rfkill repair). +- Wifi hard-blocked by a physical switch / BIOS :: GUIDE — flip the switch or enable in BIOS. +- rfkill stuck hard-blocked after suspend (WMI conflict) :: REBOOT-TAIL — blacklist the wrong WMI module, rebuild initramfs, reboot. +- Missing iwlwifi/mt7921/rtw89 firmware for a new card :: REBOOT-TAIL — install/update linux-firmware (PRIV), reboot. +- Firmware-load regression after a kernel/firmware update :: REBOOT-TAIL — boot linux-lts or pin the last-good firmware, reboot. +- Realtek rtw89 shadowed by an AUR DKMS driver :: REBOOT-TAIL — remove the DKMS package, reboot. +- Broadcom BCM4360 needs the wl driver / wl breaks after a kernel upgrade :: REBOOT-TAIL — install broadcom-wl-dkms, blacklist the open drivers, reboot. +- USB wifi dongle needs an out-of-tree DKMS driver :: REBOOT-TAIL — install the DKMS package, reboot. +- Ethernet NO-CARRIER (cable/switch) :: GUIDE — reseat/replace the cable, try another port. +- RTL8125 flips to r8169 after a kernel update :: REBOOT-TAIL — blacklist r8169, install r8125-dkms, reboot. +- RTL8125 downshifts / won't negotiate speed :: PRIV — =ethtool --set-eee eee off= (GUIDE the cable/switch swap). +- Adapter left "unmanaged" by NetworkManager :: AUTO + PRIV — =nmcli device set managed yes= (AUTO); remove the config rule / disable the rival manager to persist (PRIV). Also lives in cluster 7. + +** Cluster 2 — associated but no IP +Probe: the DHCP step (=_step_dhcp=). New work: distinguish "no DHCP answer" (server-side) from "conflict/ACD refused" (a duplicate on the LAN) so the message points the right way. + +- dhcpcd competing with NM's internal client :: PRIV — =systemctl disable --now dhcpcd=. Also cluster 7. +- dhcpcd 9 backend breaks NM IPv4 :: PRIV — =dhcp=internal= in NetworkManager.conf. +- APIPA / no DHCP answer :: AUTO — re-trigger DHCP (=nmcli device reapply=); a recurrence is server-side (GUIDE). +- Duplicate IPv4, address skipped by ACD :: GUIDE — change the conflicting host/reservation (a network decision). +- IPv6 RA not received, no SLAAC :: PRIV — =sysctl accept_ra=2= (or GUIDE the router RA-guard). +- DHCPv6-only network, address but no route :: GUIDE — the router must send an RA with the M flag; =ipv6.method auto= is AUTO but the fix is router-side. + +** Cluster 3 — have IP, no route to the internet +Probe: gateway + route + VPN-policy steps (=_step_gateway=, =_step_route=, =_vpn_policy_signal=). This is where the doctor's tunnel-down repair and the "VPN owns the route" terminal case live. + +- Default route never installed (=never-default=) :: AUTO — =nmcli con modify never-default no=, up. +- DHCP gateway off the lease's subnet :: PRIV — add the on-link route then default (GUIDE the DHCP-scope fix). +- Wifi+ethernet conflicting metrics, exits the dead link :: AUTO — raise the bad link's =ipv4.route-metric=. +- WireGuard AllowedIPs=0.0.0.0/0 strands the host :: AUTO — bring the tunnel down (the doctor's tunnel-down repair). +- WireGuard split-tunnel captures the LAN :: GUIDE — fix AllowedIPs (a config decision). +- OpenVPN redirect-gateway route left behind :: PRIV — flush the stale route, restart the client. +- Tailscale exit node captures the default route :: AUTO — =tailscale set --exit-node==. +- Docker bridge / libvirt virbr0 subnet collision :: PRIV — re-address the bridge (a config decision, GUIDE-adjacent). + +** Cluster 4 — authentication rejected (mostly terminal) +Probe: NM state 120 + GENERAL.REASON + the journal tail (the doctor's =_recent_auth_failure=). This is the cluster the classifier already marks TERMINAL and refuses to loop repairs on — the fix is almost always the user supplying a correct secret or cert. + +- Wrong PSK, NM state 120 :: GUIDE (TERMINAL) — the user re-enters the key; no repair fixes it. +- Pure WPA3-SAE won't associate :: AUTO — set =key-mgmt sae= + PMF on the profile. +- PMF mismatch on a transition AP :: AUTO — match the AP's =pmf=. +- SAE H2E-only vs hunt-and-peck :: PRIV — =sae_pwe=2= in wpa_supplicant config. +- eduroam/enterprise cert validation failed (PEAP/iwd/802.1X/anon-identity) :: GUIDE — supply the correct CA cert / server-name / realm; the doctor can't invent institutional certs. +- Hidden SSID never connects :: AUTO — =wifi.hidden yes= on the profile. +- Randomized MAC breaks a portal / allow-list :: AUTO — =cloned-mac-address stable= (or =permanent=). +- scan-rand-mac-address breaks scan/assoc :: PRIV — a =wifi.scan-rand-mac-address=no= drop-in. +- Wrong regulatory domain hides channels :: PRIV — =iw reg set=, persist via wireless-regdom. +- Won't roam, stuck on a weak AP :: AUTO — bounce the connection (GUIDE the bgscan/backend tuning). +- 4-way-handshake timeout / deauth loop :: REBOOT-TAIL — =11n_disable=1= modprobe option, reboot. + +** Cluster 5 — route works, names don't +Probe: dns-config + resolver-health + dns-resolve + the doctor's dns-test (which proves venue-resolver-broken vs egress-dead) and its dns-override/revert repair. + +- Dangling resolv.conf symlink / resolv.conf not at the stub :: PRIV — enable resolved or repoint the symlink. +- resolved stub unreachable / port-53 conflict :: PRIV — stop the conflicting resolver, restart resolved. +- NM not integrated with resolved :: PRIV — =dns=systemd-resolved= in NetworkManager.conf. +- DNSSEC rejects a venue resolver / sticky no-signature verdict :: PRIV — =DNSSEC=allow-downgrade= (=resolvectl reset-server-features= is AUTO for the sticky case). GUIDE the trade-off decision. +- DoT strict with 853 blocked / unreachable server :: PRIV — =DNSOverTLS=opportunistic=. GUIDE the trade-off. +- DHCP handed a broken resolver / pinned public DNS blocked :: AUTO — the doctor's dns-override (switch to working DNS) / drop the pin. +- Stale poisoned cache after a portal login :: AUTO — =resolvectl flush-caches=. +- VPN split-DNS not applied :: AUTO — =resolvectl domain/default-route= on the VPN link. +- IPv6 AAAA lookups stall :: AUTO — disable IPv6 on the link (or the single-request option). Also cluster 8. +- Another daemon overwrites resolv.conf :: PRIV — pick one manager, point resolv.conf at the stub. +- nsswitch.conf hosts line / avahi mDNS broken :: PRIV — fix the hosts line, install nss-mdns. + +** Cluster 6 — names resolve, egress blocked +Probe: the http/egress-edges steps + the terminal upstream outcome. This is the cluster where the doctor most often must STOP and say "not locally fixable" rather than loop repairs. + +- Captive portal held state / no auto-popup / HTTPS-check miss / HSTS block :: AUTO — the doctor's portal-open (load a plain-HTTP probe); PRIV to fix NM's connectivity URI to http. +- Custom DNS/DoT hides the captive portal :: AUTO + PRIV — drop DoT/DNSSEC for the network, log in, restore. +- PMTUD blackhole / PPPoE-VPN MTU not clamped :: AUTO — the doctor's mtu-test then mtu-override (lower the interface MTU). +- Stale http_proxy env / unreachable PAC :: AUTO — clear the proxy env / set system proxy to None. +- Clock skew breaks TLS :: PRIV — the doctor's clock-sync (=timedatectl set-ntp true=); GUIDE the CMOS-battery replacement. +- Firewall default-deny / VPN kill-switch leftover rule :: PRIV — allow egress / flush the stale rule. +- IPv6 egress broken while IPv4 works :: AUTO — disable IPv6 on the connection. Also cluster 8. +- False "limited connectivity" from a local DNS/VPN :: PRIV — fix NM's connectivity URI (AUTO to disable the check). Also cluster 7. +- ISP outage / modem in a bad state :: GUIDE (TERMINAL) — power-cycle the modem, contact the ISP; the doctor recognizes and stops. +- Specific outbound port blocked :: GUIDE — use an allowed alternate or switch networks; not host-fixable. + +** Cluster 7 — control plane broken (NetworkManager itself) +Probe: process/service state + config checks. New work: a "two managers fighting" detector (dhcpcd / systemd-networkd / iwd co-active with NM) and a keyfile-permission check, since these fail silently with a healthy-looking radio. + +- NM masked / failed to start :: PRIV — =systemctl unmask && start=; reinstall if corrupt. +- NM segfaults / restart-loops :: PRIV — coredump triage, update/downgrade the package. +- iwd + wpa_supplicant both active / backend mismatch / iwd configured-not-installed :: REBOOT-TAIL — pick one backend, disable the other, reboot. +- systemd-networkd + NM both managing the link :: REBOOT-TAIL — disable networkd, reboot. +- Keyfile permissions void a saved connection :: PRIV — =chmod 600=, =chown root=, reload. +- Autoconnect disabled :: AUTO — =connection.autoconnect yes=. +- Duplicate/conflicting profiles for one SSID :: AUTO — delete the stale profile / raise the good one's priority. +- Dispatcher script errors and blocks connectivity :: PRIV — fix ownership or remove the script. +- Polkit won't authorize a non-root user :: GUIDE — start a polkit agent / relogin for an active seat session. +- NetworkManager-wait-online hangs boot :: PRIV — disable the wait-online service. + +** Cluster 8 — flaky / drops / powersave +Probe: over-time / event-log signals (the panel's eventlog) plus a re-probe-after-idle check — the least-developed tier, the net analogue of the audio "works then dies" cluster. New work: correlate drop events with powersave/roam/suspend transitions. + +- WiFi powersave drops the connection :: AUTO — =iw set power_save off= now; PRIV to persist the NM/modprobe setting. +- USB wifi dongle killed by autosuspend :: PRIV — a udev =power/control=on= rule. +- iwlwifi/ath11k firmware crashloop on resume :: PRIV — reload the module (a sleep hook to persist). +- No reconnect after suspend/resume :: AUTO — the doctor's nm-restart; PRIV for a durable sleep hook. +- IPv6 happy-eyeballs stalls :: AUTO — disable IPv6 on the link. Shared with clusters 5/6 — same broken-v6 family, three probe points (route absent, egress black-holed, AAAA unanswered). + +* Bluetooth triage + +The ~55 bluetooth failure modes sorted into five symptom clusters, each owned by a link in the bt doctor's existing chain (adapter → rfkill/service/powered → per-device → audio). Same four remedy classes. The bt doctor's design already encodes a key constraint that shapes this triage: *connecting and pairing are user intents, not health repairs* — so the doctor never auto-connects or auto-pairs, and the destructive re-pair is always user-confirmed. That pushes clusters 3 and 4 heavily toward GUIDE and user-initiated actions, with far fewer silent AUTO fixes than the net side. + +Rough distribution: ~12 AUTO, ~16 PRIV, ~12 REBOOT-TAIL, ~15 GUIDE. The buildable share is lower than net's because the adapter/firmware cluster is almost entirely PRIV/REBOOT-TAIL/GUIDE and the pairing cluster is mostly user-driven. The doctor's existing safe-auto tiers (unblock, power-on, service-restart, a2dp) all live in clusters 2 and 5 — which is exactly where the buildable share concentrates. + +** Cluster 1 — no adapter +Probe: the adapter step (=btctl.show=). New work: a dmesg firmware-load hint would name the vendor blob (ibt-*.sfi, BT_RAM_CODE_MT7961, rtl8761, QCA, BCM .hcd) instead of a generic "no adapter found" — the direct parallel to the audio cluster-1 dmesg-hint proposal. + +- btusb not auto-loaded :: PRIV — =modprobe btusb=, persist via modules-load.d. +- Intel/MediaTek/Realtek/Qualcomm/Broadcom firmware missing or mismatched :: REBOOT-TAIL — update linux-firmware or symlink/decompress the requested blob (PRIV), reboot. +- Intel "failed to send firmware data" / broken-initial-NCMD :: REBOOT-TAIL — =btusb reset=1= or a newer kernel, reboot. +- Controller lost after a kernel update (Intel AX / MT7922 / QCA) :: REBOOT-TAIL — boot linux-lts or downgrade, reboot. +- MediaTek MT7922 stuck in a residual power state :: GUIDE — full power-off, pull power ~10s, cold boot. +- Realtek/Broadcom dongle needs an out-of-tree driver or .hcd :: REBOOT-TAIL — build the DKMS driver / install the firmware, reboot. +- Fake CSR clone resets/times out :: PRIV — the kernel clone quirks or =btusb reset=1=, reload. +- MT7925 killed by USB autosuspend :: REBOOT-TAIL — =btusb.enable_autosuspend=n= on the cmdline, reboot. Also cluster 4. +- Bluetooth hard-blocked by BIOS/switch :: GUIDE — flip the switch / enable in BIOS. + +** Cluster 2 — adapter blocked, unpowered, or the daemon is down +Probe: the rfkill + service + powered steps. This is where all four of the doctor's safe auto-repairs (unblock, service-restart, power-on) live. + +- Soft-blocked adapter :: AUTO — the doctor's unblock (=rfkill unblock bluetooth=). +- Airplane / physical hard-block :: GUIDE — turn airplane mode off / flip the switch (the doctor deliberately won't touch this radio). +- AutoEnable off, powered down every boot :: PRIV — =AutoEnable=true= in main.conf. +- bluetooth.service masked / not enabled :: PRIV — =systemctl unmask / enable --now= (service-restart is the AUTO tier for a stopped-but-enabled service). +- D-Bus policy blocks org.bluez / bad unit-file after a bluez upgrade :: PRIV — reinstall bluez / reconcile the .pacnew, daemon-reload. +- WiFi rfkill toggle also killed BT (combo chip) :: AUTO — the doctor's unblock. +- TLP disables BT on startup / systemd-rfkill restores a stale block :: PRIV — remove BT from the TLP list / clear the saved rfkill state (unblock is AUTO now). +- rfkill block spins bluetoothd at 100% CPU :: AUTO — service-restart after unblock. +- Adapter powers back off after a desktop toggle :: AUTO — unblock + power-on (or re-enable from the desktop). +- BT stuck/soft-blocked on resume, needs a module reload :: PRIV — reload btusb (a sleep hook to persist). Also cluster 4. +- Wrong/missing default controller (multiple adapters) :: AUTO — =bluetoothctl select <MAC>= then power-on. + +** Cluster 3 — won't pair / bond +Probe: none in-chain — pairing is a user intent the panel offers, not something the doctor health-repairs. The doctor's one destructive tier here (re-pair = remove + pair) is always user-confirmed, never chained. New work: a "stale bond / corrupt cache" signature (a device that fails auth repeatedly with a bond present) would let the doctor *offer* the re-pair with confidence rather than guess. + +- Generic AuthenticationFailed / re-pair loop / corrupt cache survives remove :: PRIV (user-confirmed) — stop bluetooth, =rm -rf= the stale =/var/lib/bluetooth/<adapter>/<mac>=, restart, re-pair. +- Stale link key after a dual-boot re-pair :: GUIDE — remove and re-pair (or sync keys between OSes); a user decision. +- Paired but not trusted, no auto-reconnect :: AUTO — =bluetoothctl trust <mac>= (a safe, non-connecting fix). Bridges to cluster 4. +- No default pairing agent :: GUIDE — register an agent (=agent on=); mostly a headless-setup concern. +- BLE device never appears in scan / not in pairing mode / adapter not pairable :: GUIDE — enable LE transport / put the device in pairing mode / =pairable on= — user actions. +- ConnectionAttemptFailed / page-timeout after a bluez upgrade :: PRIV — restart bluetooth; downgrade bluez if it recurs. +- Headset un-pairs itself on disconnect / BLE address rotates :: GUIDE — re-pair; a device-behavior limitation. +- Legacy keyboard needs the PIN typed on-device :: GUIDE — type the passkey on the keyboard. +- Too many bonded devices, keys lost :: PRIV (user-confirmed) — remove devices below the ~14 limit, restart. +- Dual-mode device pairs on the wrong transport / seen as LE-only :: PRIV — =ControllerMode = bredr= in main.conf, restart, re-pair. Shared with cluster 5. +- Paired without a bond via blueman :: GUIDE — re-pair with =bluetoothctl pair= (which bonds). + +** Cluster 4 — paired but the link won't hold +Probe: the per-device connected-state step. A disconnected paired device is information, not a failure; reconnecting is user-initiated. New work: connection-parameter / coexistence hints (name USB3 noise, 2.4GHz coexistence, or a too-aggressive supervision timeout) so a "keeps dropping" verdict points somewhere. + +- Paired+trusted device never reconnects on boot/resume :: PRIV — install bluetooth-autoconnect, or set =ReconnectIntervals= in main.conf. +- Xbox/PS controller drops unless ERTM is off :: REBOOT-TAIL — =disable_ertm=1= modprobe option, reboot. +- USB3 port radio noise / loose antenna / short range :: GUIDE — move the device off the USB3 port, reseat the antenna, close the distance. +- WiFi/BT 2.4GHz coexistence stutter :: PRIV — move WiFi to 5GHz / tune the coexistence modprobe option (GUIDE the band move). +- btusb autosuspend input lag / adapter dead after resume :: REBOOT-TAIL — =btusb.enable_autosuspend=0= on the cmdline (PRIV sleep hook for the resume case). Shared with clusters 1/2. +- BLE supervision-timeout / connection-interval too aggressive :: PRIV — raise the timeout / set =[ConnectionParameters]= (a startup service to persist debugfs). +- RTL8761B wrong-firmware reconnect loop :: PRIV — symlink the correct blob, reload. Same cause as cluster 1's name-mismatch. +- "Software caused connection abort" on connect-after-scan :: AUTO — retry the connect without a preceding scan-stop. +- A2DP stutters with a BT mouse also connected / mouse stutters without a scan :: GUIDE — lower audio-codec airtime or move the mouse to a dongle; a contention trade-off. + +** Cluster 5 — connected but audio is wrong +Probe: the audio-profile step (=audio.pw_dump=, card/sink for the device). The doctor's a2dp repair (force the A2DP profile) is the AUTO tier here. The generic sound-server internals belong to the audio taxonomy; these are the BT-specific ones the bt panel names. + +- No A2DP sink / stale bluez state hides A2DP :: PRIV (user-confirmed) — clear the stale =/var/lib/bluetooth= record, re-pair (the a2dp repair is AUTO when the card is merely on the wrong profile). +- Stuck in HSP/HFP, autoswitch-to-headset fired :: AUTO — the doctor's a2dp repair / disable autoswitch. +- Leftover pulseaudio-bluetooth steals the endpoint / bluez SPA plugin missing :: PRIV — remove pulseaudio-bluetooth / install the SPA plugin. +- WirePlumber seat-monitoring drops the node :: PRIV — a =seat-monitoring = disabled= drop-in. +- Codec falls back to SBC / LDAC stuck low / AAC missing :: AUTO — a =bluez5.codecs= / ldac-quality drop-in (PRIV/REBOOT for the fdk-aac rebuild). +- BT sink not made default :: AUTO — =wpctl set-default= (the audio taxonomy's set-default primitive). +- Absolute-volume slider no-op :: PRIV — =enable-hw-volume = false= drop-in. +- Silent sink after resume :: AUTO — bounce the stream via set-default. +- oFono blocks native HFP / mSBC :: PRIV — remove ofono / set the native backend. +- mSBC unsupported by adapter / HFP output dead :: GUIDE — a hardware or PipeWire-version limitation. +- Device seen as LE/GATT-only, no audio class :: PRIV — =ControllerMode = bredr=, re-pair. Shared with cluster 3. +- Battery percent never reported :: PRIV — =Experimental = true= in main.conf. +- Multi-device, wrong audio default :: AUTO — =wpctl set-default= / a priority.session rule. +- HFP narrowband mic during calls :: GUIDE — an inherent A2DP/HFP trade-off. + +* Saturation test + +Deferred to a later pass, mirroring the audio taxonomy's blind-different-axis resample. When run, it must sample on a *different* axis than this by-layer first sweep — by hardware vendor, by distro, by device class (headset vs mouse vs controller vs enterprise-wifi vs VPN), weighted toward non-Arch 2024-2026 sources — and run the agents blind to these clusters, so it tests whether the eight net / five bt clusters are representative rather than re-deriving them. The audio resample forced zero new clusters from 108 fresh reports; the expectation here is the same, but it is unproven until run. + diff --git a/docs/specs/2026-07-09-audio-doctor-spec.org b/docs/specs/2026-07-09-audio-doctor-spec.org index 7860dac..5851332 100644 --- a/docs/specs/2026-07-09-audio-doctor-spec.org +++ b/docs/specs/2026-07-09-audio-doctor-spec.org @@ -8,6 +8,7 @@ :PROPERTIES: :ID: 6ce3f2ef-052e-40cb-9067-42f8e665f813 :END: +- [2026-07-10 Fri] Superseded in part, later the same day: the DOCTOR section-header key and its DIAGNOSE key are replaced by a doctor key per direction, and the classifier grows an input side. See [[file:2026-07-10-audio-doctor-input-side-spec.org][2026-07-10-audio-doctor-input-side-spec.org]] (DRAFT). This spec's other decisions stand. The gap: nothing here ever examined the microphone, and the Non-Goals never said it would not. - [2026-07-10 Fri] IMPLEMENTED — Phases 0 through 4 shipped (dotfiles =4d42eb3=, =01a1d80=, =76857e3=, =7223c51=, =1550ac8=, =4320255=, =bd33440=). Xruns stayed out of v1 as decided. Per-application stream routing remains a Non-Goal. The wall carries the net panel's copy + close controls, added past the spec at Craig's request; standardizing them across the other panels is tracked as its own task. - [2026-07-09 Thu] READY — all eight Decisions closed. Written from the roam-inbox ask "look into a doctor button for the audio panel: what happens if pulseaudio or its equivalent is borked? would we know? how could we test for it? what remedies would we have?" Probe set, classifier, and remedy tiers are grounded in a live survey of ratio's stack. Phase 0 shipped ahead of the rest. diff --git a/docs/specs/2026-07-10-audio-doctor-input-side-spec.org b/docs/specs/2026-07-10-audio-doctor-input-side-spec.org new file mode 100644 index 0000000..0e7e529 --- /dev/null +++ b/docs/specs/2026-07-10-audio-doctor-input-side-spec.org @@ -0,0 +1,494 @@ +#+TITLE: Audio Doctor, Input Side — a doctor per direction +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-10 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Audio Doctor, Input Side +:PROPERTIES: +:ID: 45efdd66-6d22-45c0-a633-ec41079e38ec +:END: +- [2026-07-10 Fri] DRAFT — drafted. Extends the audio doctor ([[file:2026-07-09-audio-doctor-spec.org][2026-07-09-audio-doctor-spec.org]], IMPLEMENTED) to the microphone, and replaces its single DOCTOR header key with one doctor per direction. Grounded in a live survey of ratio, not memory. + +* Metadata + +| Field | Value | +|----------+----------------------------------------------------------------------------| +| Status | draft | +|----------+----------------------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+----------------------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+----------------------------------------------------------------------------| +| Related | [[file:../../todo.org][todo.org]] : "The audio doctor never checks the microphone" | +|----------+----------------------------------------------------------------------------| +| Parent | [[file:2026-07-09-audio-doctor-spec.org][2026-07-09-audio-doctor-spec.org]] — this supersedes its DOCTOR-key decision | +|----------+----------------------------------------------------------------------------| + +* Summary + +The audio doctor is output-only. It never examines the microphone, so a muted mic, a default source naming an unplugged device, and a mic the sound server cannot see all classify as =healthy=. + +This adds the input side, and in doing so answers a question no probe can: whether a machine with no microphone is broken or merely mic-less. The answer comes from the user's finger. A doctor key on the INPUTS section header means "there should be a mic here", so an empty source list under that press is a failure rather than a shrug. + +* Problem / Context + +Three separate failures, all currently invisible. + +** The classifier does not look at the source + +=diag.probe_semantic= already collects =default_source= and =default_source_present=. =classify.py= reads neither. The word "source" appears once in that module, in the graph row that counts how many exist. Every semantic rule the classifier applies to the sink (is a default set, is it present, is it muted, is it at zero volume) has no source counterpart. + +So the =healthy= verdict prints "Units are up, the graph answers, the Pulse layer answers, and the default output is present and audible." Output. Only. A user reading "the sound stack is healthy" concludes the mic is fine, and the doctor never checked. + +** Nobody can tell an absent microphone from a broken one + +A desktop with no microphone is a normal machine. So an empty source list cannot be a fault the way =no-output-devices= is, and the classifier has no way to know which machine it is on. This is not a probe problem. No amount of reading the graph settles it, because the missing fact lives in the user's head. + +** The doctor cannot see below PipeWire + +When Chrome stopped recognizing the microphone on 2026-07-10, the stack was genuinely healthy: the Shure MV7+ was the default source, unmuted, at 82%, with push-to-talk off. The doctor would have said =healthy= and been right about the stack and useless to the user. + +The layer that would have helped is the one the doctor never reads. The kernel's own capture list distinguishes "the sound server lost a microphone the kernel can see" from "nothing is plugged in", and those two faults have nothing in common. + +** Live survey (ratio, 2026-07-10) + +Facts this spec is built on, verified rather than remembered: + +- =/proc/asound/card*/pcm*c= lists capture-capable ALSA devices. On ratio: =Generic_1= (ALC623 analog), =MV7= (Shure), =BRIO= (Logitech). A =pcm*c= node means the card has a capture PCM; =pcm*p= would be playback. +- That path is a kernel filesystem read. It needs no package, spawns no process, and *cannot hang* — unlike every existing probe tier. =arecord -l= reports the same set but is owned by =alsa-utils=, which is a dependency the engine does not otherwise carry. +- =diag= already tags each stream =input= or =output= (=_STREAM_CLASSES=, mapping =Stream/Input/Audio= and =Stream/Output/Audio=). =active_streams()= returns both without distinction. +- Push-to-talk mutes the default source and leaves it muted between holds (=ptt.toggle_plan=). =ptt.read_state()= persists =armed=, so the deliberate mute is already knowable. +- A monitor source is a legitimate default source. =probe_semantic= passes =include_monitors=True= on purpose. + +* Goals and Non-Goals + +** Goals + +- Classify the input side with the same rigor as the output side: default set, default present, muted, at zero volume. +- Never report an armed push-to-talk mic as a fault. +- Distinguish "the sound server does not see a microphone the kernel does" from "no capture hardware is attached", and say which. +- Let the user assert that a microphone should exist, without a dialog. +- One doctor at a time. + +** Non-Goals + +- Rebuilding push-to-talk. See the decision below; the mute is the safety property, not the bug. +- Per-application capture routing. If Chrome cannot see a source the graph plainly has, that is Chrome's fault, and Helvum and qpwgraph exist. This spec makes the doctor able to *say* the stack is fine, which is the sentence that was missing. It does not chase the client. +- Diagnosing hardware below the kernel. If =/proc/asound= shows no capture device, this spec reports that fact. It does not test cables, USB ports, or drivers. +- Sample-rate, latency, or xrun concerns. Still out, as in the parent spec. +- Silent privileged action. The doctor may use =sudo= for the input and output expansion (see the decision "The doctor may use sudo, resolved by context at run time"), but never silently: every privileged or reboot-tail remedy defaults to Confirm or Arm tier. Auto stays reserved for user-scope, reversible remedies. + +** Scope tiers + +- *v1:* the kernel capture tier; PTT-aware source classification; input verdicts and their remedies; the two section-header doctor keys with mutual exclusion; explicit =audio doctor --output= / =--input= flags, output aliased as the default. +- *Out of scope:* PTT rebuild; per-application routing; hardware diagnosis. +- *vNext:* per-device correlation between the kernel capture set and graph sources (v1 uses the coarse set-emptiness rule instead); and a third sighting of "the graph is fine and one client cannot use it", which turns into its own design question about whether the doctor should name the application layer. Both logged to =todo.org=, not built here. + +* Design + +** For the user + +The DOCTOR section header we shipped on 2026-07-10 goes away. In its place, the OUTPUTS and INPUTS engraved section headers each grow a doctor key, beside the device counts they already carry. The keys diagnose the direction they sit above, and both write to the same results wall below. + +The keys do not repeat the direction words. The CONTROLS section already carries INPUT and OUTPUT keys — the mute toggles that read LIVE / MUTE — so a doctor key labelled OUTPUT a few rows up would read as a fourth mute control. The doctor keys are named for what they examine instead: SPEAKERS on the OUTPUTS header, MICROPHONE on the INPUTS. Each carries a distinct diagnostic treatment so a user reads it as a different kind of key, not a device control. See the decision below; that treatment is the same on every panel's doctor. + +Pressing MICROPHONE says something a probe cannot: that this machine is supposed to have a microphone. That single bit resolves the ambiguity the classifier cannot. An empty source list is normal on a desktop; an empty source list *under a press of the MICROPHONE doctor* is a failure, and the doctor now has license to say so. + +The wall names which doctor ran, and only one runs at a time. Pressing SPEAKERS while a MICROPHONE run is in flight does nothing, because the wall holds one verdict and a second run would overwrite the first mid-read. + +Diagnosis remains free and read-only. The escalation the user asked for — "we tried reloading all the devices and there is still no microphone" — is a *repair*, not a diagnosis, because reloading devices means restarting WirePlumber. So the diagnose verdict names what is wrong, and FIX is what tries the reload and re-probes. The wall then carries a claim the doctor earned rather than one it assumed: + +#+begin_example +DOCTOR · MICROPHONE diagnose · read-only + ok pipewire.service active + ok wireplumber.service active + ok pipewire-pulse.service active + ok graph 4 sinks, 3 sources (via pw-dump) + ok pulse compat answering + FAIL capture hardware kernel sees MV7, BRIO, Generic_1 — the graph has none + -- default input not checked (no input devices) + + verdict: The sound server does not see your microphone. + The kernel has 3 capture devices and PipeWire has none, so the + hardware is attached and the software lost it. FIX reloads the + device profiles. +#+end_example + +And when the kernel sees nothing either, the doctor stops rather than offering a remedy for a fault that is not there: + +#+begin_example + FAIL capture hardware no capture device is attached + + verdict: No microphone is attached. + The kernel sees no capture hardware at all, so this is a cable, a + port, or a device that is off. Nothing in the sound stack can fix it. +#+end_example + +** For the implementer + +*** A tier below the graph + +The probe ladder gains a tier at the bottom, and its position is the point. Today's order is =systemctl= (cannot hang), =pw-dump= (alive when Pulse is dead), =pactl= (may hang). The kernel tier reads =/proc/asound=, which is a filesystem read that cannot hang, cannot fail for want of a package, and answers when every other layer is dead. It is the only probe in the doctor that carries no timeout, because it cannot block. + +It answers one question: which cards expose a capture PCM. A card directory containing a =pcm*c= entry has one. That set is what separates a lost device from an absent one. + +The comparison is deliberately coarse in v1, because the two layers speak different namespaces. The kernel tier names cards by ALSA id (=MV7=, =BRIO=); the graph names sources by PipeWire =node.name=. Mapping one to the other device-by-device is a rabbit hole, and the doctor does not need it. The v1 rule is a set-emptiness test, not a per-device match: =mic-unrecognized= fires when the kernel capture set is non-empty *and* the graph has zero non-monitor hardware sources. A monitor source does not count as a microphone (it is desktop-audio capture), so a graph carrying only monitors still reads as "no real input the kernel's hardware could be feeding." That coarse rule catches the whole-mic-lost case this feature exists for without ever having to prove that card X is the same device as source Y. Per-device correlation is a vNext refinement, logged, not built. + +*** The classifier takes a direction + +=classify(ctx)= becomes =classify(ctx, side)= where side is =output= or =input=. The terminal-fault ladder is shared and unchanged: PipeWire down, WirePlumber down, Pulse hung, Pulse dead, tooling missing. All of them break both directions, and the first one that fires wins regardless of side. + +Below that the ladder forks. The output side keeps its rules exactly as they are. The input side gets, in order: no capture hardware; hardware present but no source in the graph; a default source that names an absent device; a mic muted or at zero. Every input rule is gated on the user having pressed the input doctor, which is what makes an empty source list a fault instead of a fact. + +Push-to-talk sits inside that last rule. =diag.gather()= reads =ptt.read_state()= into the context, and a muted default source with PTT armed is not a finding. It is the mic doing exactly what the user asked. The wall reports it as an informational row rather than saying nothing, because a user who has forgotten PTT is armed deserves to be told why the mic is silent. + +*** The guard learns direction + +=active_streams()= returns running streams of both directions, and =repair.guard()= refuses the audible remedies whenever any of them is running. That is already slightly wrong: a playback remedy is refused today when the only thing running is a recording. Filtering by direction fixes that and is what the mic remedy needs. + +Unmuting a microphone is not audible. It is a privacy event. If something is capturing when the remedy lands, the user's voice goes out. So the mic-unmute remedy is guarded against *input* streams with the same press-again override the audible remedies use, and the playback remedies are guarded against *output* streams. + +* Failure modes and remedies + +The whole point of a doctor is that every fault it can name has a stated fix, and the reader can see the two together. This is that map: what goes wrong at each probe tier, the verdict the classifier reaches, and the remedy FIX runs. The tiers are the probe ladder, top to bottom, so a fault lower in the table is one the doctor reaches only after the tiers above it answered clean. + +The first four tiers are shared: a broken unit, a dead graph, or a dead Pulse layer takes out both directions at once, and the same verdict serves whichever doctor the user pressed. The kernel tier and the source-side semantic rules are what this spec adds. + +| Tier | What goes wrong | Verdict | Remedy | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 1 units (=systemctl=) | =pipewire= not active, or | =graph-down= | 5: restart =pipewire= (Arm) | +| | the graph will not answer | | | +| | though the unit claims up | | | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 1 units | =wireplumber= not active | =no-session-manager= | 1: restart =wireplumber= | +| | | | (Auto) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 1 units | =pipewire-pulse= not active | =pulse-down= | 4: restart =pipewire-pulse= | +| | | | (Confirm) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 2 graph (=pw-dump=) | the graph is up but has no | =no-output-devices= | 1: restart =wireplumber=, | +| | sink | | which re-creates devices | +| | | | (Auto) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 3 pulse (=pactl=) | the socket accepts and never | =pulse-hung= | 4: restart =pipewire-pulse= | +| | answers | | (Confirm) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 3 pulse | the socket refuses | =pulse-down= | 4: restart =pipewire-pulse= | +| | | | (Confirm) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 3 pulse | a probe's own tool is | =tooling-missing= | none: the stack is | +| | missing | | unobserved, not broken | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 4 semantic, output | the default sink names a | =stale-default= | 2: set a present default | +| | device that is absent | | sink (Auto) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 4 semantic, output | the default sink is muted or | =silenced= | 3: unmute and restore volume | +| | at zero volume | | (Confirm) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 4 semantic, input | the default source names a | =mic-stale-default= | 7: set a present default | +| | device that is absent | | source (Auto) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 4 semantic, input | the default source is muted | =mic-silenced= | 8: unmute and restore volume | +| | or at zero, PTT off | | (Confirm) | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 4 semantic, input | the default source is muted, | healthy, informational row | none | +| | PTT armed | | | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 5 kernel (=/proc/asound=) | kernel capture set non- | =mic-unrecognized= | 6: reload profiles, restart | +| | empty, graph has no non- | | =wireplumber= (Confirm) | +| | monitor hardware source | | | +|---------------------------+------------------------------+----------------------------+------------------------------| +| 5 kernel | no capture device is | =no-mic-hardware= | none: a cable, a port, or a | +| | attached at all | | device that is off | +|---------------------------+------------------------------+----------------------------+------------------------------| +| all clean | nothing is wrong | =healthy= | none | +|---------------------------+------------------------------+----------------------------+------------------------------| + +Three entries carry no remedy on purpose, and they are not the same kind of no. =tooling-missing= means the doctor could not observe the stack, =no-mic-hardware= means there is nothing in the stack to fix, and =healthy= means there is nothing wrong. A doctor that offered a button for any of the three would teach the user to distrust the wall. + +Remedies 1 and 6 both restart WirePlumber, and that is deliberate: =no-output-devices= and =mic-unrecognized= are different diagnoses that happen to share a fix. Collapsing them would couple two faults that should read apart, the same call the parent spec made for =pulse-hung= and =pulse-down=. + +* Verdict clusters and remedy classes + +The verdicts and remedies above are the v1 core. Their full shape comes from a failure taxonomy built and saturation-tested after the first draft: [[file:../design/2026-07-10-audio-failure-taxonomy.org][docs/design/2026-07-10-audio-failure-taxonomy.org]], a catalogue of 74 input and 77 output real, user-reported failure modes. A blind resample (108 fresh reports through different sources, none shown the structure) forced zero new clusters, so the taxonomy is representative, not an artifact of sampling. It sorts every failure into nine symptom clusters per direction, and those clusters are what the doctor's verdicts name. + +** The clusters are the verdict structure, and they map to the probe ladder + +The nine input clusters are: (1) no capture device at all, (2) device present but records silence, (3) works but too quiet, (4) works then dies or drops, (5) the server lost a device the hardware has, (6) Bluetooth mic, (7) stack healthy but an app can't hear, (8) hardware/physical, (9) environment/config. Output mirrors this, swapping the mic-specific clusters for speaker-vs-headphone jack routing, HDMI/DisplayPort, and a degraded-audio (crackle/xrun) cluster. + +The clusters are not arbitrary. Seven of the nine are the tiers the doctor's probe ladder already separates (units, graph, pulse, semantic, kernel), and the two it cannot fully see (cluster 7 app-layer, and part of cluster 8 hardware) are exactly the ones that resolve to Guide. So the boundary between what the doctor fixes and what it can only guide falls along the tiers it already probes. The v1 verdicts in the table above are the subset of these clusters the current ladder reaches; the rest are staged below. + +** Remedy classes come from the privilege decision + +Every remedy carries one of four classes, resolved at run time (see the decision "The doctor may use sudo, resolved by context at run time"): Auto, Privileged, Reboot-tail, Guide. The taxonomy tags every entry, so the buildable share is known — a little over half of each direction. The Auto core reduces to five primitives (=set-default=, =set-card-profile=, unmute, restart-services, config-drop-in-plus-restart) applied across roughly fifty diagnoses, so the breadth is in the diagnosis, not the remedies. + +** Three new probes the taxonomy implies + +Read-only and bounded, in the spirit of the kernel tier: + +- *dmesg-pattern hints* (cluster 1). So "no capture hardware" can name the specific cause — firmware-load-fail, NHLT-missing, codec-probe-timeout — instead of a generic verdict, and the Guide can print the right command. +- *the unmute-doesn't-stick signature* (cluster 8). Issue an unmute, re-probe; a mute that will not clear is a hardware switch, not software. This is what tells an Auto (software mute) from a Guide (flip the physical switch). +- *re-probe-after-idle* (cluster 4). The current single-shot probe misses the suspend and autosuspend "works then dies" cases; a second read after an idle window catches them. + +* Alternatives Considered + +** Rebuild push-to-talk so it does not mute the mic + +The idea: keep the source unmuted, and instead unlink its nodes from capturing applications (=pw-link=), or stand up a virtual source that applications connect to and link the real mic into it only while the key is held. + +- Good, because the doctor would see an unmuted mic and need no special case, and because an application's own level meter would keep working. +- Bad, because the microphone stays live at the server and silence becomes a property of graph topology. A crash mid-hold, or a link that fails to tear down, is a hot mic while every indicator reads muted. Source-mute is the one state where the server itself guarantees nothing is captured. +- Bad, because the virtual-source variant requires every application to select the virtual device, which breaks per-application device choice — the thing the panel exists to give. +- Neutral, because the doctor does not need it: =ptt.read_state()= already persists the fact that the mute is deliberate. + +Rejected. The mute is the safety property. + +** One doctor that classifies both directions at once + +- Good, because one press and one verdict. +- Bad, because =classify()= returns exactly one verdict, so a machine with a stale sink default and a muted mic forces the classifier to rank two unrelated faults. Any ranking is a guess about what the user came to fix. +- Bad, because it cannot resolve the absent-microphone ambiguity at all. No press means no assertion, and the doctor is back to shrugging. + +Rejected in favor of one doctor per direction, which supplies the missing intent as a side effect of where the user pressed. + +** Use =arecord -l= for the kernel tier + +- Good, because it is the obvious tool and its output is human-readable. +- Bad, because it is owned by =alsa-utils=, a dependency the audio engine does not otherwise carry, so its absence would have to be modelled as a fourth =tooling-missing= case. +- Bad, because it spawns a process, which means a timeout, which means another way for the bottom tier to fail. +- Neutral, because it reads the same kernel data =/proc/asound= exposes directly. + +Rejected. The bottom tier should be the one thing in the doctor that cannot fail. + +* Review findings [1/10] + +Second review pass, 2026-07-10 (four critical lenses: buildability, technical correctness, adversarial failure-modes, internal consistency). The technical foundations checked out — every load-bearing code and =/proc/asound= claim verified against the real tree. These findings are what the passes surfaced beyond that. The mechanical inconsistencies they also found (a "two new remedies" miscount, two stray OUTPUT/INPUT doctor-key labels) are already fixed in place. + +** TODO Mic-unmute must fail closed on an unreadable graph :blocking: +The stream-active guard abstains and proceeds when =pw-dump= cannot read the graph. That is right for the *output* remedies, whose whole job is repairing a dark graph. It is wrong for the mic-unmute remedy (8), because unmuting is a privacy event, not an availability one: with the graph unreadable the guard cannot see a running capture stream, so FIX would unmute a live mic mid-recording with no second press and no warning. The mic-unmute remedy must fail *closed* on an unreadable graph — refuse, or fall back to =pactl list source-outputs= — never abstain-and-proceed. Every other finding here produces a wrong verdict; this one produces a wrong action. Folds into the open capture-guard decision. (blocking) + +** TODO A non-ALSA microphone classifies as no-mic-hardware :blocking: +A Bluetooth or USB-non-ALSA mic never appears under =/proc/asound/card*=, because bluez runs through PipeWire, not ALSA. The =no-mic-hardware= verdict is gated only on "kernel capture set empty", with no graph clause, so a working BT-headset mic (a real non-monitor source in the graph) reads as "No microphone is attached." The input ladder must consult the graph first: a working non-monitor source in the graph is =healthy= (or routes to the semantic rules) and skips the kernel tier entirely. =no-mic-hardware= fires only when the kernel set is empty *and* the graph has no non-monitor source; =mic-unrecognized= only when the kernel set is non-empty *and* the graph has none. This also resolves the ladder-order contradiction below. Folds into the open input-verdicts decision. (blocking) + +** TODO The input ladder's evaluation order is stated two ways :important: +The "For the implementer" prose lists the kernel-tier rules first (no-mic-hardware, mic-unrecognized), then the semantic rules. The failure table orders the semantic-input rows (tier 4) above the kernel rows (tier 5) and says lower rows are reached only after upper tiers answer clean. Those are opposite orders for the same four rules. The finding above settles it: graph-source presence is the top gate, so define one order and make both the prose and the table match it. + +** TODO The probe does not yet surface source mute, volume, or a monitor flag :important: +=probe_semantic= produces =default_sink_muted= / =default_sink_volume= but no source equivalents, and neither the graph nor the semantic source dicts carry a monitor flag (=pactl._device= strips =device.class=). The =mic-silenced= rule reads source mute/volume the context never provides, and the "non-monitor hardware source" test cannot be computed. Phase 0 must extend =probe_semantic= to surface the source fields and tag sources monitor-vs-hardware, and the =mic-unrecognized= rule must read the =pw-dump= source population (=Audio/Source=, monitor-excluding), not the =pactl include_monitors=True= one — reading the monitor-inclusive list would mean a monitor is always present and the rule never fires. Neither change is scoped by an open decision; both would stall Phase 0/1. + +** TODO The direction must thread through findings() and doctor(), not only classify() :important: +The spec says =classify(ctx)= becomes =classify(ctx, side)= but never that =findings(ctx)= gains =side= too, though the wall shows side-specific rows. =doctor.doctor()= is the sole caller and invokes both =classify= and =findings= twice each (initial and re-probe), plus =final_findings= / =final_verdict=; it must take =side= and thread it to all of them, and =cli.cmd_doctor= must pass the resolved side. Name this in the phase text so it is not left for the implementer to rediscover. + +** TODO Remedy 6 restarts WirePlumber unguarded, mid-call :important: +=mic-unrecognized='s remedy (6, restart WirePlumber) is in neither guard set, so a user on a call can press FIX and reload the device profiles with no guard and no second-press consent. A profile reload can re-select the default sink and drop the call's routing. Decide whether remedy 6 is guarded — against streams of *both* directions, since a WirePlumber restart touches the whole graph — as part of the open capture-guard decision. + +** TODO A stale armed=true PTT state can mask a genuinely muted mic :important: +The safe-degrade argument covers only an unreadable or missing PTT state file (=read_state= returns disarmed). It does not cover =armed=true= left stale — Hyprland restarted and dropped the binds, or the mic was muted by the CONTROLS toggle for an unrelated reason while PTT happened to be armed. =read_state= cannot detect staleness, so the doctor reports "healthy, PTT armed" on a genuinely dead mic, the exact failure this feature exists to fix. At minimum, name the limitation; better, have the informational row say the mic is muted *and* that PTT is armed, so the user is not told the mic is simply fine. + +** TODO Minor build-notes to fold in before Phase 3 :minor: +Three smaller items the passes raised. The =healthy= verdict text is output-worded ("the default output is present and audible") and needs side-aware wording for a clean =MICROPHONE= run. The two keys must clear =doctor_running= on the worker's error path so a raised worker cannot wedge both keys insensitive for the session — the shipped =bg= / =doctor_abort= pattern already does this, so it is a spec-explicitness note, not a code gap. And a USB mic pressed during its ~1s enumeration window (kernel sees the card before WirePlumber creates the source) fires a transient =mic-unrecognized=; self-correcting, worth a line acknowledging the race. + +** TODO Open Decisions still gate implementation :blocking: +The workflow treats unresolved decisions as implementation blockers, because a builder would be accepting product and safety tradeoffs mid-build. Partly addressed as of 2026-07-10: Craig closed the push-to-talk and kernel-tier decisions, and the CLI-flag decision closed with his direction, so the cookie is now =[6/9]=. Three decisions remain open — the direction-aware capture guard, the input verdict and remedy table, and the doctor-key naming and cross-panel style. Close or explicitly risk-accept each before implementation starts. (blocking) + +** DONE Kernel capture devices are not mapped to graph sources :blocking: +CLOSED: [2026-07-10 Fri] +The finding: the kernel names cards by ALSA id and the graph names sources by PipeWire =node.name=, so "compare the kernel capture set against the graph's source list" was undefined, and monitor sources muddied it further. Resolved 2026-07-10 by adopting the reviewer's coarse rule as the v1 definition: =mic-unrecognized= fires when the kernel capture set is non-empty and the graph has zero non-monitor hardware sources. No per-device correlation, no namespace mapping — that whole-mic-lost case is the one this feature exists for, and per-device matching is logged as vNext. The rule now appears in the "For the implementer" kernel-tier note, the failure table, and the input-verdicts decision. + +* Decisions [7/10] + +** DONE A doctor key per direction, on the section headers +CLOSED: [2026-07-10 Fri] +Context: an empty source list is normal on a mic-less desktop and a failure on a machine that should have one. No probe can tell them apart. + +Decision: we will put a doctor key on each of the OUTPUTS and INPUTS engraved section headers, and retire the standalone DOCTOR header and its DIAGNOSE key. Pressing the input doctor asserts that a microphone should exist. + +Consequences: the ambiguity is resolved by the user without a dialog, and the check sits beside the devices it judges. Harder: this supersedes a decision in the parent spec that shipped hours earlier, so the parent's history must say so. The panel also grows a third and fourth console key, and the 400px width has to absorb them. + +** DONE One doctor at a time +CLOSED: [2026-07-10 Fri] +Context: the wall holds one verdict, and a run clears the wall when it begins. + +Decision: we will allow only one doctor run in flight. SPEAKERS must finish before MICROPHONE can be pressed, and both keys go insensitive for the duration. + +Consequences: the wall is never a mix of two runs, and "which verdict is this" never needs asking. Harder: nothing, since the model already clears on begin and locks on =doctor_running=. The keys simply join the lock. + +** DONE An absent microphone is not a fault unless the user says so +CLOSED: [2026-07-10 Fri] +Context: =no-output-devices= is a fault because a machine with no way to play sound is broken. The mirror is false: a machine with no way to record is ordinary. + +Decision: we will not add a source-side =no-input-devices= fault. The input rules fire only under a press of the input doctor, which supplies the assertion that a mic should exist. + +Consequences: the doctor never cries wolf on a mic-less desktop. Harder: the classifier now takes a side parameter, so it is no longer a pure function of the context alone. It stays pure; the side is an input. + +** DONE Does push-to-talk get rebuilt? +CLOSED: [2026-07-10 Fri] +Context: PTT mutes the default source and leaves it muted between holds, so a naive mirror of the sink rules would classify an armed mic as "silenced, FIX unmutes it". Pressing FIX would break PTT and leave a hot mic the user believes is muted. Craig asked whether PTT could avoid muting. + +Decision: we will not rebuild PTT. =diag.gather()= will read =ptt.read_state()= into the context, and an armed-PTT muted mic will be reported as an informational row rather than a finding. + +Consequences: a few lines rather than a rewrite, and the safety property is preserved — source-mute is the only state where the server guarantees no capture. Harder: the doctor now depends on a runtime state file, so a stale or unreadable file must degrade to "not armed" (which =read_state()= already does) and the mic then reads as a genuine finding. That is the safe direction: it over-reports a muted mic rather than under-reporting a hot one. + +Craig ratified 2026-07-10: not rebuilding is fine. + +*** Discussion +The alternatives are recorded above under "Rebuild push-to-talk so it does not mute the mic". The load-bearing argument is that link-based silence fails open. + +** DONE The kernel tier reads =/proc/asound=, not =arecord= +CLOSED: [2026-07-10 Fri] +Context: the doctor needs a layer below PipeWire to tell a lost microphone from an absent one. + +Decision: we will read =/proc/asound/card*/pcm*c= directly. A card exposing a capture PCM has capture hardware. + +Consequences: the bottom tier cannot hang, cannot be absent, and needs no timeout — the only probe in the doctor with that property. Harder: it is Linux-specific and reads a kernel interface rather than a supported API, so a kernel that reorganizes =/proc/asound= would break it silently. The fallback is that an unreadable =/proc/asound= yields "unknown", never "no hardware", so the doctor abstains rather than lying. + +Craig ratified 2026-07-10: all okay. + +** TODO The capture guard, and what it refuses +Context: unmuting a microphone is not audible, but it is a privacy event. =active_streams()= currently ignores stream direction, so a playback remedy is refused today when only a recording is running. + +Decision (proposed): we will make the guard direction-aware. Playback remedies (3, 4, 5) refuse on running *output* streams; the mic-unmute remedy refuses on running *input* streams. Both take the same press-again override, and both abstain on an unreadable graph, as today. + +Consequences: the guard stops over-refusing, and unmuting a mic mid-recording takes deliberate consent. Harder: it changes the behavior of shipped remedies, so the existing guard tests move with it rather than being deleted. + +Owner: Craig. By: before Phase 2 lands. + +** TODO The input verdicts and their remedies +Context: the input side needs its own verdict vocabulary and its own remedy table entries. + +Decision (proposed): four new verdicts, and three new remedies. + +| Verdict | Means | Remedy | Tier | +|---------------------+------------------------------------------+-------------------------------------------+---------| +| =no-mic-hardware= | the kernel sees no capture device | none | — | +|---------------------+------------------------------------------+-------------------------------------------+---------| +| =mic-unrecognized= | kernel capture set non-empty, graph has | 6: reload device profiles (restart | Confirm | +| | no non-monitor hardware source | WirePlumber) | | +|---------------------+------------------------------------------+-------------------------------------------+---------| +| =mic-stale-default= | the default source names a device that | 7: set a present default source | Auto | +| | is absent | | | +|---------------------+------------------------------------------+-------------------------------------------+---------| +| =mic-silenced= | the default source is muted or at zero, | 8: unmute and restore the mic volume | Confirm | +| | and PTT is not armed | | | +|---------------------+------------------------------------------+-------------------------------------------+---------| + +=no-mic-hardware= carries no remedy for the same reason =tooling-missing= carries none: offering a fix for a fault that is not in the stack teaches the user to distrust the wall. + +=mic-unrecognized= uses the coarse set-emptiness rule from the "For the implementer" kernel-tier note, not a per-device match. The kernel names cards by ALSA id and the graph names sources by PipeWire =node.name=; those are different namespaces, and correlating them device-by-device is a vNext refinement. v1 fires =mic-unrecognized= when the kernel capture set is non-empty and the graph has zero non-monitor hardware sources. This resolves the second spec-review finding, which flagged the matching rule as undefined. + +Consequences: the verdict table grows from ten to fourteen, and =classify.findings()= grows a capture-hardware row and a default-input row. Harder: remedy 6 restarts WirePlumber, which remedy 1 already does for a different verdict. Two remedies invoking one action is fine; collapsing them would couple two diagnoses that should stay apart. + +Owner: Craig. By: before Phase 1 lands. + +** DONE The CLI grows explicit =--output= and =--input= flags, output aliased as the default +CLOSED: [2026-07-10 Fri] +Context: =audio doctor= today accepts only =--fix= and =--force=. There is no =--output=, so adding a lone =--input= would read as a special case bolted onto a bare command rather than one of a symmetric pair. + +Decision: we will add both =--output= and =--input= as explicit direction flags, and alias bare =audio doctor= to =audio doctor --output=. Both accept =--fix= and =--force=. The two flags are mutually exclusive; passing both is an error. + +Consequences: the two directions are named symmetrically, and =--input= has a sibling to pair against rather than standing alone. Bare =audio doctor= keeps its current meaning, so nothing that scripts it breaks, and "output is the default" is now a stated alias a reader can see rather than a convention to infer. Harder: =audio doctor --input --fix --force= is a long line, which is the cost of explicitness over =audio doctor input=; the subcommand form was rejected because it would break every existing call. + +Craig's call, 2026-07-10: "alias the command and have =audio doctor --output= be the one that is run by default. That way it's symmetrical, and =audio doctor --input= has a place." + +** TODO Name and style the doctor keys, distinct from the device controls +Context: the CONTROLS section already carries INPUT and OUTPUT keys — the mute toggles that read LIVE / MUTE. A doctor key labelled OUTPUT or INPUT on the section header a few rows up would read as a fourth mute control. Craig flagged the collision and asked for a name and a look that mark the doctor keys as a different function, held consistent across every panel. + +Decision (proposed): the doctor keys are named for what they diagnose — SPEAKERS on the OUTPUTS header, MICROPHONE on the INPUTS. Craig delegated the naming ("I'll let you choose"); these are the choice. Each key carries a distinct diagnostic treatment: a small health glyph (a stethoscope, =nf-md-stethoscope=) ahead of the word, plus a subtly distinct outline, so the key stays inside the dupré aesthetic while reading as diagnostic rather than a device control. The same treatment applies to the doctor keys already shipped in the other panels — net's DOCTOR, maint's CLEAN UP / REVIEW & FIX — so a doctor key looks like a doctor key everywhere. + +Consequences: no word collides with the CONTROLS keys; the two doctor keys carry distinct AT-SPI names the smoke can target; the diagnostic affordance is consistent across the panel family. Harder: it touches every panel's =gui.py= and the shared panel CSS, so the cross-module restyle is its own task rather than part of this build, and it sits beside the already-filed "copy + close on every wall" convergence. The exact glyph and outline are an aesthetic call that wants Craig's eye on a screenshot before it lands — which is why this stays open while the naming is settled. + +Owner: Craig (the visual). By: before Phase 3, where the keys are built. + +** DONE The doctor may use sudo, resolved by context at run time +CLOSED: [2026-07-10 Fri] +Context: the parent spec decided "no sudo anywhere", correct when the feature was purely user-scope PipeWire. The input and output expansion reaches firmware, ALSA saved state, modprobe overrides, and package management, which genuinely need root. Craig also wants the CLI usable as a generic doctor on machines that are not archsetup installs. + +Decision: this supersedes the parent's no-sudo decision. The doctor resolves its privilege at run time from three signals — passwordless sudo available (=sudo -n true=, which succeeds or fails instantly and never hangs), a tty to prompt at, and whether it is the GUI panel. Four remedy classes follow: *Auto* (user-scope, reversible, runs anywhere), *Privileged* (needs sudo — runs silently where passwordless sudo exists, prompts in a CLI with a tty, and degrades to Guide only in a GUI with neither), *Reboot-tail* (the applicable part runs, then the doctor instructs the reboot it cannot complete or verify past), and *Guide* (nothing to run: a physical switch, a BIOS setting, a wait-for-upstream fix). Every archsetup install has passwordless sudo, so on Craig's machines Privileged remedies run; the generic-CLI case prompts. + +Consequences: many failure modes that were guide-only (=alsactl store=, a modprobe override, removing a conflicting package) become applicable remedies on archsetup and on any CLI. Harder, and load-bearing: passwordless sudo is not consequence-free, so every Privileged and Reboot-tail remedy defaults to *Confirm or Arm* tier, never silent Auto — a firmware install or a module reload always takes a deliberate second press even when sudo will not prompt. Auto stays reserved for the user-scope, reversible remedies. This model is not audio-specific: the net, bluetooth, and maint doctors adopt the same run-time privilege resolution and the same Confirm/Arm-default for privileged actions, tracked as its own cross-panel task. + +Craig agreed 2026-07-10 to both the model and the Confirm/Arm-default safety stance, and to making it a cross-panel standard. + +* Implementation phases + +Each phase leaves the tree green and independently useful, as the parent spec's phases did. + +** TODO Phase 0 — the kernel tier, PTT in the context, direction-aware streams +Pure engine, no classifier changes, no UI. =diag.probe_kernel()= reads =/proc/asound=; =diag.gather()= adds =kernel= and =ptt= to the context; =active_streams(graph, direction=None)= filters. =audio diag --json= shows the new context. Fakes: a temp =/proc/asound=-shaped tree via an injectable root, since the real one cannot be faked on PATH. + +** TODO Phase 1 — =classify(ctx, side)= and the input verdicts +Pure, fixture-driven. The shared terminal ladder, then the fork. Pairwise over (kernel capture set × graph sources × default source state × PTT armed). This phase adds the direction flags: =--output= and =--input=, mutually exclusive, with bare =audio doctor= aliased to =--output=. =audio doctor --input= read-only prints findings and a verdict, and =audio doctor= keeps printing exactly what it does today. + +** TODO Phase 2 — the input remedies and the direction-aware guard +Remedies 6, 7, 8, each re-probing its own claim. The guard splits by direction. =audio doctor --input --fix=. The existing guard tests move rather than being deleted. + +** TODO Phase 3 — two section-header doctor keys +Retire the DOCTOR header row and its DIAGNOSE key. OUTPUTS and INPUTS each gain one. Mutual exclusion on =doctor_running=. The wall header names the run. AT-SPI smoke over both keys and the fix path, on fixtures. + +** TODO Phase 4 — flip this spec to IMPLEMENTED +And amend the parent spec's history to record that its DOCTOR-key decision was superseded here. + +** Later phases — the taxonomy expansion (vNext) +v1 (phases 0-4) ships the buildable Auto core and the Guide tail for the clusters the current probe ladder already separates, and flips this spec to IMPLEMENTED. The rest of the taxonomy ([[file:../design/2026-07-10-audio-failure-taxonomy.org][the taxonomy]]) is staged as later phases so v1 is not blocked on them. Each is a small, independent addition; the taxonomy is the backlog and this list is the order: + +- The dmesg-hint probe and the firmware/driver Guide tail (cluster 1). +- The unmute-doesn't-stick signature probe and the hardware-layer Guide (cluster 8). +- Re-probe-after-idle for the suspend and autosuspend cases (cluster 4). +- The Privileged and Reboot-tail remedy machinery: the run-time privilege resolution (=sudo -n= / tty / GUI detection) and the Confirm/Arm-default floor. +- The app-layer Guide tail (clusters 7 and 9): the tail on a healthy verdict that names the likely app or portal cause and prints the fix. + +These grow into their own spec when picked up. + +* Acceptance criteria + +- A machine with no capture hardware, under a press of the input doctor, reports =no-mic-hardware= and offers nothing to press. +- A machine whose kernel lists a capture device the graph has no source for reports =mic-unrecognized=, and FIX reloads the device profiles and re-probes. +- An armed-PTT muted mic classifies as =healthy=, with an informational row saying push-to-talk is armed. +- A muted mic with PTT disarmed classifies as =mic-silenced= and FIX unmutes it. +- FIX on the mic while something is recording is refused, and a second press proceeds. +- Restarting a playback remedy while only a microphone is recording is no longer refused. +- Pressing SPEAKERS while MICROPHONE is running does nothing. +- =audio doctor= with no flag behaves exactly as it does today. + +* Readiness dimensions + +- *Data model & ownership* — the context dict gains =kernel= (generated, per-probe) and =ptt= (read from the runtime state file that =ptt.py= owns; the doctor never writes it). +- *Errors, empty states & failure* — an unreadable =/proc/asound= yields =unknown=, never "no hardware". An unreadable PTT state reads as disarmed, which over-reports a muted mic rather than under-reporting a hot one. Both are the safe direction. +- *Security & privacy* — the mic-unmute remedy is the only privacy-relevant action in the feature, and it is guarded against running capture streams with an explicit second-press override. No credentials, no logs. +- *Observability* — the wall is the observability surface, and it names which doctor ran. The =--json= output carries the full context including the kernel tier. +- *Performance & scale* — the kernel tier is a directory listing. Everything else is unchanged. +- *Reuse & lost opportunities* — =classify.findings()= remains the single row-builder for the CLI and the GUI. =ptt.read_state()= is reused rather than reimplemented. =diag.unit_active= stays the single source of truth for unit state. +- *Architecture fit* — the fork lives in =classify()=, which is pure and already fixture-tested. The GUI change is confined to the section-header rows and the model's key-sensitivity logic. +- *Config surface* — none. N/A. +- *Documentation plan* — the module docstrings carry the design, as in the parent. No user docs; the wall is the documentation. +- *Dev tooling* — =make test= and =make test-panel-audio= cover it. The kernel tier needs an injectable root rather than a PATH fake, which is a new fixture shape for this package. +- *Rollout, compatibility & rollback* — =audio doctor= with no flag is unchanged, so nothing that scripts it breaks. The GUI change is not reversible by config; it is a redesign of a key that shipped the same day. +- *External APIs & deps* — =/proc/asound= layout verified live on ratio, 2026-07-10. No new packages. + +* Risks, rabbit holes, and drawbacks + +The kernel tier reads an interface rather than an API. If a future kernel reorganizes =/proc/asound=, the probe reports =unknown= and the doctor abstains, which is the failure mode we want, but nothing will announce the breakage. A test against a recorded tree pins the shape we expect. + +Remedy 6 restarts WirePlumber to reload device profiles, which is what remedy 1 already does. The temptation is to collapse them. Do not: the two verdicts describe different faults, and a shared remedy is not a shared diagnosis. The parent spec made exactly this call for =pulse-hung= and =pulse-down=. + +The panel is 400px wide and gains two console keys while losing one row. The layout needs eyes on it before Phase 3 closes, and no automated check will catch an ugly one. + +This spec supersedes a decision from a spec that reached IMPLEMENTED the same day. That is a smell worth naming: the DOCTOR-key placement was decided without the input side in view, and one conversation with the user overturned it. The lesson is not that the first decision was careless. It is that the input side should have been in scope from the start, and the parent's Non-Goals never said it was not. + +* Review and iteration history + +** 2026-07-10 Fri @ 18:52:07 -0500 — Craig Jennings — Reviewer +What changed or was recommended: a second review pass across four critical lenses — buildability, technical correctness, adversarial failure-modes, internal consistency. Verified every load-bearing code and =/proc/asound= claim against the real tree; all held. Recorded eight new findings (two blocking: mic-unmute failing open on an unreadable graph, and a non-ALSA mic mis-classifying as no-mic-hardware) and fixed three mechanical inconsistencies in place (a remedy miscount and two stray OUTPUT/INPUT doctor-key labels). +Why: the design was sound at the tier level, but it had two safety and correctness holes that only surface on real hardware (a webcam or Bluetooth mic, an unreadable graph), plus three build gaps that sit outside the open decisions and would stall Phase 0/1. +Artifacts: [[*Review findings][Review findings]]; code read across =~/.dotfiles/audio/src/audio/= (=diag.py=, =classify.py=, =repair.py=, =doctor.py=, =ptt.py=, =pactl.py=); live =/proc/asound= survey on ratio. + +** 2026-07-10 Fri @ 10:55:19 -0500 — Codex — Reviewer +What changed or was recommended: applied the spec-review workflow and recorded two blocking findings: open Decisions still gate implementation, and kernel-capture-to-graph-source matching is undefined. +Why: the current implementation confirms the microphone gap and output-only classifier, but these unresolved choices would force the builder to decide safety and classification behavior during implementation. +Artifacts: [[*Review findings][Review findings]]; current code read in =~/.dotfiles/audio/src/audio/diag.py=, =classify.py=, =repair.py=, =cli.py=; related tracking in [[file:../../todo.org][todo.org]] under "The audio doctor never checks the microphone". + +** 2026-07-10 Fri @ 10:38:12 -0500 — Craig Jennings — Author +What: drafted the spec. Four probe tiers become five, the classifier takes a direction, the panel gets a doctor per direction, and push-to-talk stays as it is. +Why: the doctor calls a stack healthy while the microphone is dead, and the ambiguity of an absent mic cannot be resolved by any probe. +Artifacts: live survey of ratio (=/proc/asound= capture nodes, =_STREAM_CLASSES=, =ptt.read_state()=), and the Chrome incident of the same morning. diff --git a/docs/specs/2026-07-11-bt-doctor-expansion-spec.org b/docs/specs/2026-07-11-bt-doctor-expansion-spec.org new file mode 100644 index 0000000..b255b71 --- /dev/null +++ b/docs/specs/2026-07-11-bt-doctor-expansion-spec.org @@ -0,0 +1,250 @@ +#+TITLE: Bt Doctor Expansion — name the blob, catch the boot-disable +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-11 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Bt Doctor Expansion +:PROPERTIES: +:ID: 3d4d61c4-e5df-44e9-b8e0-40b31452c3f7 +:END: +- [2026-07-11 Sat @ 02:30 -0500] DOING — decomposed into build tasks (spec-response Phase 6); parent task in =todo.org= bound by =:SPEC_ID:=. Phases 0-1 (the two read-only probes + the firmware-hint Guide verdict) are buildable now and =:solo:=; Phase 2 (the persistent fix) is gated on the shared cross-panel privilege model. +- [2026-07-11 Sat @ 02:00 -0500] READY — third skeptical re-review returned Ready with caveats, no blocking findings; all round-2 resolutions verified against the engine. Caveat accepted: Phase 2 (the persistent fix) depends on the shared cross-panel privilege model, which doesn't exist yet; Phase 0 (both read-only probes) and Phase 1 (the firmware-hint Guide) are buildable today. +- [2026-07-11 Sat @ 01:45 -0500] DRAFT — round-2 review + response. A skeptical re-review caught a real blocker: the AutoEnable default is =true=, not =false= (verified, bluez 5.87), so round 1's "dead every boot on absent config" premise was backwards and would fire a false positive on healthy machines. Corrected — the fault fires only on explicit AutoEnable=false / disabled service / TLP. Also folded three non-blocking corrections (the =code= key is additive-not-existing; the INI-setter is real Phase 2 work; the sequencing caveat now actually in Risks). Findings =[8/8]=, decisions =[3/3]=. Awaiting a third re-review. +- [2026-07-11 Sat @ 01:20 -0500] DRAFT — review incorporated (spec-response). All four findings dispositioned (=[4/4]=), all three decisions accepted and closed (=[3/3]=). The verdict-representation blocker resolved by defining a "verdict" as a step outcome in the existing status model. +- [2026-07-11 Sat @ 00:59:30 -0500] DRAFT — reviewed (spec-review). Stays DRAFT: three decisions open plus one =:blocking:= finding (the bt doctor has no named-verdict layer, so the proposed =no-adapter-firmware=/=powered-off-persistent= verdicts need their representation defined first). Design confirmed against the live engine — the two target gaps (kernel-log firmware hint, boot-persistence read) genuinely do not exist yet. Findings in =* Review findings=. +- [2026-07-11 Sat @ 00:08:41 -0500] DRAFT — drafted. Extends the existing bt doctor (=~/.dotfiles/bluetooth/=, shipped) using the bluetooth half of the failure taxonomy ([[file:../design/2026-07-10-net-bt-failure-taxonomy.org][2026-07-10-net-bt-failure-taxonomy.org]]). Grounded in a read of the live engine, not memory. + +* Metadata + +| Field | Value | +|----------+-----------------------------------------------------------------------------------| +| Status | doing | +|----------+-----------------------------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+-----------------------------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+-----------------------------------------------------------------------------------| +| Related | [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][net/bt failure taxonomy]] ; the cross-panel run-time-privilege and copy+close tasks | +|----------+-----------------------------------------------------------------------------------| + +* Summary + +The bt doctor walks a clean chain — adapter → rfkill → bluetooth.service → powered → per-device → audio profile — and applies four safe repairs (unblock, power-on, service-restart, a2dp). The failure taxonomy shows the chain is structurally right but blind in two specific spots the user hits most: when there is no adapter it says "no adapter found" without naming *why* (a missing firmware blob the dmesg log already names), and it never checks whether the adapter is configured to power on at boot, so an =AutoEnable=false= laptop that is dead every login reads as a healthy powered-off adapter. This spec adds those two probes and adopts the cross-panel privilege model. It is an expansion of a working doctor. + +* Problem / Context + +The taxonomy sorted ~55 real bluetooth failure modes into five clusters keyed to the doctor's chain. Read against the live engine (=~/.dotfiles/bluetooth/src/bt/doctor.py=), the doctor already handles cluster 2 well (unblock, service-restart, power-on are its exact auto-fix tiers) and cluster 5 partially (the =a2dp= repair forces the A2DP profile for a card stuck in HSP/HFP or with no sink). Clusters 3 (pairing) and 4 (connection stability) are deliberately light — the doctor treats connecting and pairing as user intents, not health repairs, and that is a correct design choice, not a gap. The two real gaps are at the ends of the chain. + +** "No adapter found" doesn't say why (cluster 1) + +=_adapter_step= reports a missing adapter as "no Bluetooth adapter found (hardware/driver)" and stops. But the taxonomy's cluster 1 is almost entirely *firmware* faults, and the kernel already logged the specific cause: "Direct firmware load for intel/ibt-…sfi failed", "mediatek/BT_RAM_CODE_MT7961… failed", "BCM: Patch …hcd not found", "Reading QCA version information failed". The doctor has that log a =journalctl -k= read away and doesn't consult it. A user who sees "no adapter — hardware/driver" is left to search; a user who sees "the Intel firmware blob ibt-20-1-4.sfi failed to load — update linux-firmware and reboot" has the fix. This is the direct parallel to the audio doctor's cluster-1 dmesg-hint proposal, and the same read-only, bounded shape. + +** A boot-disabled adapter reads as merely powered-off (cluster 2) + +=_powered_step= checks whether the adapter is powered *now* and offers =power-on=. But =power-on= via bluetoothctl doesn't persist: on a machine with =AutoEnable=false= explicitly set (bluez's compiled default is =true= — verified in bluez 5.87's =/etc/bluetooth/main.conf= — so an absent or unset config auto-enables, and the fault is an explicit opt-out, not the missing-config case), or a TLP laptop that disables bluetooth on startup, the adapter is dead again next boot. The doctor fixes the symptom every session and never names the cause. The taxonomy's cluster 2 has a whole sub-family here — AutoEnable explicitly off, service-not-enabled-at-boot, TLP-disables-on-startup, systemd-rfkill-restores-a-stale-block — that all present as "powered off / blocked" and all need a *persistent* fix the doctor doesn't distinguish from a one-shot power-on. + +** Pairing and connection clusters stay light on purpose (clusters 3, 4) + +The doctor doesn't auto-pair or auto-connect, and the destructive re-pair is always user-confirmed. That is right. The one addition worth considering is a *stale-bond signature* — a device that fails to connect with a bond present, repeatedly — so the doctor can *offer* the re-pair with confidence instead of the user guessing. That is a candidate, held as a decision, not a v1 commitment. The rest of clusters 3/4 (physical range, USB3 noise, coexistence, connection parameters) are Guide-only and mostly out of a health doctor's reach. + +* Goals and Non-Goals + +** Goals + +- When there is no adapter, name the specific firmware/driver cause by reading the kernel log, and print the matching fix — instead of a generic "hardware/driver." +- Distinguish a *persistently* disabled adapter (AutoEnable off, service not enabled at boot, TLP-disabled) from a merely powered-off one, and offer the persistent fix rather than a one-shot power-on that dies next boot. +- Adopt the cross-panel run-time privilege model for the new root-needing repairs (editing main.conf's AutoEnable, =systemctl enable=, editing tlp.conf), under the Confirm/Arm-default floor. + +** Non-Goals + +- Auto-pairing or auto-connecting devices. Connecting and pairing are user intents; the doctor's existing stance stands. +- Fixing the firmware itself. The dmesg-hint probe *names* the missing blob and prints the update/reboot instruction; it does not install firmware or rebuild initramfs inside a diagnose run (that is a Reboot-tail Guide the user runs). +- The physical/coexistence connection-stability faults (USB3 noise, range, 2.4GHz coexistence, connection parameters). These are Guide-only and stay out of the health chain. +- Silent privileged action. Editing main.conf, enabling the service, or touching tlp.conf all default to Confirm/Arm, never silent Auto. + +** Scope tiers + +- *v1:* the dmesg firmware-hint probe for the no-adapter case (cluster 1); the boot-enablement probe distinguishing AutoEnable-off / service-not-enabled / TLP-disabled from a plain power-off (cluster 2), with the persistent fix; adoption of the run-time privilege model. +- *Out of scope:* auto-pair/auto-connect; firmware installation inside a run; the physical/coexistence connection faults. +- *vNext:* the stale-bond re-pair-offer signature (cluster 3, pending the decision below); a connection-parameter/coexistence hint tail for a "keeps dropping" verdict (cluster 4); the bluetooth-specific audio-profile expansion beyond the current a2dp repair (codec fallback, default-sink, absolute-volume — several overlap the audio taxonomy and want coordination with the audio doctor). All logged to =todo.org=. + +* Design + +** For the user + +Two verdicts that today under-inform start telling the user the actual cause. + +When there is no adapter, the wall stops at "no adapter" no longer. It names the blob and the fix: "the MediaTek BT firmware (BT_RAM_CODE_MT7961) failed to load — update linux-firmware and reboot," pulled straight from the kernel log the doctor now reads. When the log shows a clean absence (no firmware error, genuinely no controller), it says that instead — the same fail-honest distinction the audio doctor draws between "the server lost a device" and "nothing is attached." + +When the adapter is powered off, the doctor asks a second question: is it *supposed* to be on at boot? If AutoEnable is off, or bluetooth.service isn't enabled, or TLP is disabling it on startup, the verdict names that — "your adapter powers off every boot because AutoEnable is false" — and FIX offers the persistent fix (set AutoEnable, enable the service) rather than a power-on that will be undone by the next reboot. A plain "it's off right now" still gets the quick power-on. + +** For the implementer + +*** How the new "verdicts" are represented + +The bt doctor has no verdict-enum layer like the net doctor's =classify= actions. A step result today carries a fixed key set — =id=, =status= (=pass=/=fail=/=warn=/=info=), =title=, =evidence=, =elapsed_ms=, =safety=, =next_action= (=schema.step=) — and the run rolls the statuses up into an =overall= (=ok=/=warn=/=fail=, =doctor.py:182-183=). So a "verdict" in this spec is not a new =overall= value — it is a distinct step *outcome*: the same =status= with a specific message and evidence, and a =next_action= carrying the fix for the persistent case. The human distinction rides in the existing =evidence= and =next_action= fields — =format_doctor_human= renders =evidence= (=cli.py:94=), so the named blob or persistence cause needs no formatter change. If a stable machine identifier is wanted so =--json= consumers can branch on the outcome without string-matching, that is an *additive* =code= key on =schema.step= — and because the step schema is a locked test contract, adding it ripples into any test asserting exact step shape, so it is called out here rather than assumed. =_adapter_step= and =_powered_step= already own their results; the new probes add branches inside them. Nothing touches the =overall= vocabulary or =AUTO_FIX= tiers — the additions stay inside the existing step-status model rather than inventing a parallel verdict system. + +*** The dmesg firmware-hint probe + +When =_adapter_step= finds no adapter, run a bounded =journalctl -k -b --no-pager= (or =dmesg=) read, scanned for the known firmware-load-failure signatures per vendor (Intel ibt-*.sfi, MediaTek BT_RAM_CODE, Realtek rtl_bt, Broadcom BCM .hcd, Qualcomm QCA version-read). A match sets the =no-adapter-firmware= code on the adapter step, naming the blob and the Reboot-tail Guide (update linux-firmware / symlink the blob / reboot). No match keeps the existing generic no-adapter outcome. The read reuses the engine's existing bounded pattern — =cmd.run(journalctl -u bluetooth -n 1)= already runs for service-log evidence (=doctor.py:84=, 5s timeout) — so the kernel-log read is the same bounded shape, read-only, and runs only in the no-adapter branch, costing nothing on a healthy adapter. + +*** The boot-enablement probe + +When =_powered_step= finds the adapter powered off (or =_rfkill_step= finds a soft-block), consult three persistence signals: bluez =AutoEnable= (parse =/etc/bluetooth/main.conf= [Policy]), =systemctl is-enabled bluetooth=, and whether TLP's =DEVICES_TO_DISABLE_ON_STARTUP= lists bluetooth. The fault fires only on an *explicit* boot-disable: =AutoEnable=false= set in main.conf (bluez's compiled default is =true=, so an absent or unset key means auto-enable-on and is never the fault), a =disabled= bluetooth.service, or a TLP entry. A powered-off adapter with one of those gets a =powered-off-persistent= outcome distinct from the plain =powered-off=; its fix is the persistent one (set AutoEnable / enable the service / edit tlp.conf), Privileged/Confirm. The existing =power-on= stays for the plain case (including the common absent-config machine, which auto-enables by default). + +*** The privilege model + +The new repairs — editing main.conf, =systemctl enable bluetooth=, editing tlp.conf — are root-needing and adopt the cross-panel run-time model: run where passwordless sudo exists, prompt on a tty, default to Confirm/Arm, never silent Auto. Same shared implementation as the net and audio doctors. =priv.py= today exposes exactly one privileged verb, =restart-bluetooth=, via a plain verb→argv dispatch (=priv.py:25-34=); the three new fixes add verbs to it, each a narrowly-scoped operation — set the single =AutoEnable= key, enable one named unit, set the single TLP key — not a general edit-file-as-root. The passwordless-sudo grant widens by three tight verbs, not by a root file-editor, so the Confirm floor is the second guard rather than the only one. One caveat the implementer must not miss: =systemctl enable bluetooth= is a clean argv verb, but the two config edits are not. main.conf is INI — setting =AutoEnable= needs an idempotent setter that creates the [Policy] section when absent and preserves comments (a shipped helper or crudini). tlp.conf is a flat =KEY=VALUE= file — the fix removes =bluetooth= from the =DEVICES_TO_DISABLE_ON_STARTUP= list, a list-edit, not a section-create. Both are real Phase 2 work, not one-line argv verbs. The existing four auto-fix tiers (unblock, power-on, service-restart, a2dp) are unchanged — they are already user-scope or already the doctor's safe tier. + +* Alternatives Considered + +** Print a generic "check firmware" hint without reading the log + +- Good, because it needs no journal read. +- Bad, because "check your firmware" is exactly the uselessly-generic verdict this fixes; the whole value is naming the specific blob the kernel already identified. +- Rejected. The log has the answer; read it. + +** Make power-on always persistent (set AutoEnable on every power-on) + +- Good, because the adapter would then stay on across boots without a separate verdict. +- Bad, because it conflates a deliberate boot-disable (a user or TLP choosing bluetooth-off-by-default for battery) with a fault, and silently overrides a policy the user may have set on purpose. Persistence is a distinct decision that deserves a distinct, confirmed action. +- Rejected in favor of a separate verdict the user confirms. + +** Build the stale-bond re-pair signature into v1 + +- Good, because it would let the doctor offer re-pair with confidence. +- Bad, because re-pair is destructive (removes the bond) and the signature ("fails to connect with a bond present, repeatedly") needs care to not fire on a merely-out-of-range device; getting it wrong offers a destructive fix for a transient condition. +- Held as an open decision, not a v1 commitment. + +* Decisions [3/3] + +** DONE The dmesg firmware-hint probe +Context: cluster 1 is almost all firmware faults the kernel already logs, and the doctor reports a generic "hardware/driver" instead of the named blob. +Decision: we will read the kernel log in the no-adapter branch, match the known per-vendor firmware-load-failure signatures, and set a =no-adapter-firmware= outcome code naming the blob and its Reboot-tail Guide; no match falls back to the generic no-adapter outcome. +Consequences: the most common no-adapter cause becomes self-explaining; harder — a per-vendor signature table to maintain, and the read must stay bounded and only run in the no-adapter branch so it never slows a healthy run. +Resolution: accepted as proposed (the generic "check firmware" alternative was rejected — the whole value is naming the specific blob). Owner: Craig. + +** DONE The boot-enablement probe and the persistent-vs-transient split +Context: =power-on= doesn't persist, so an AutoEnable-off / not-enabled / TLP-disabled adapter is dead every boot and the doctor fixes only the symptom. +Decision: we will add a persistence probe (AutoEnable, service-enabled, TLP) and a =powered-off-persistent= outcome code distinct from =powered-off=, whose fix is the persistent one under the Confirm floor; the plain power-on stays for the transient case. +Consequences: the boot-disable cause is named and fixed once instead of every session; harder — the doctor must not override a *deliberate* boot-disable, so the persistent fix is always confirmed and the outcome names the specific persistence cause rather than blanket-enabling. +Resolution: accepted as proposed (the always-persistent-power-on alternative was rejected — it would silently override a deliberate battery-saving boot-disable). Round-2 correction: the verdict fires only on an *explicit* AutoEnable=false / disabled service / TLP entry — bluez's compiled default is =true= (verified, bluez 5.87), so absent or unset config is auto-enable-on and never the fault. Owner: Craig. + +** DONE Whether the stale-bond re-pair-offer lands in v1 or vNext +Context: a device failing to connect with a bond present, repeatedly, is a re-pair candidate — but re-pair is destructive and the signature can misfire on an out-of-range device. +Decision: defer to vNext. v1 keeps re-pair strictly user-initiated; the signature is designed and validated before the doctor ever *offers* it. +Consequences: no risk of the doctor offering a destructive fix for a transient condition in v1; harder — the "keeps failing to connect" case stays unnamed for now, which is the one cluster-3 gap a user might reasonably expect the doctor to catch. +Resolution: accepted — deferral confirmed. The "keeps failing to connect" gap is logged to vNext. Owner: Craig. + +* Review findings [8/8] + +** DONE The bt doctor has no named-verdict layer, so "emit a verdict named X" is undefined :blocking: +The spec proposes named verdicts throughout — =no-adapter-firmware=, =powered-off-persistent= "distinct from =powered-off=" — mirroring the net doctor's action-identifier vocabulary. But the bt doctor has no such layer. Verified: a step carries a =status= ∈ =pass/fail/warn/info= and the run has an =overall= ∈ =ok/warn/fail= (=doctor.py:182-183=); there are no verdict identifiers as *names*. So "add a =powered-off-persistent= verdict distinct from =powered-off=" had no existing mechanism to attach to. +Disposition: accepted. Added Design "How the new 'verdicts' are represented": a verdict here is a distinct step *outcome* — the same =status= plus a stable machine =code= and a specific message/evidence, set inside =_adapter_step= / =_powered_step=, rendered by =format_doctor_human= and carried in =--json=. Nothing touches the =overall= vocabulary or =AUTO_FIX= tiers. The Decisions and probe descriptions now speak in "outcome code" terms, so the representation is defined before Phase 1. + +** DONE New root repairs expand the NOPASSWD sudoers surface — name it in rollout +=priv.py= today exposes a single privileged verb, =restart-bluetooth= (=priv.py:27-34=), backed by a NOPASSWD sudoers entry. The three new persistent fixes (edit main.conf, =systemctl enable bluetooth=, edit tlp.conf) are each a new root-capable verb. +Disposition: accepted, modified. Rather than just "name the expansion," the resolution constrains each new verb to a narrowly-scoped operation (set the single =AutoEnable= key, enable one named unit, set the single TLP key) — not a general edit-file-as-root — so the passwordless-sudo grant widens by three tight verbs, not a root file-editor. Folded into the privilege-model design and the Rollout dimension. + +** DONE The diagnostic surface is =bt doctor='s JSON, not a =bt diag= subcommand +Phase 0 said "=bt diag --json= (or the equivalent) shows the new signals." Verified: there is no =bt diag= subcommand; =diagnose()= is the report authority (=doctor.py:174=), =doctor()= wraps it (=:207=), JSON is the default output, and =format_doctor_human= (=cli.py:94=) renders the human view. +Disposition: accepted. Phase 0 now points at the real surface (the doctor's JSON report + =format_doctor_human=) and drops the =bt diag= reference. + +** DONE Firmware-hint probe is new but has a bounded precedent — cite it +Phase 0's =journalctl -k= reader is genuinely new (=_adapter_step= reads nothing beyond =btctl.show=, =doctor.py:46-50=), but the engine already runs a bounded =journalctl -u bluetooth -n 1= for service-log evidence (=doctor.py:84=, timeout 5s via =cmd.run=). +Disposition: accepted. The firmware-hint probe design now cites =doctor.py:84= as the bounded precedent the kernel-log read reuses, so the implementer reaches for the same =cmd.run= shape rather than an unbounded call. + +** DONE Round 2 (skeptical review): the AutoEnable default is true, not false :blocking: +Round 1 asserted (twice) that =AutoEnable=false= is "bluez's static default when main.conf is absent," and the whole "dead every boot" premise rested on it. A skeptical re-review read the installed bluez 5.87 =/etc/bluetooth/main.conf= and found the opposite: "Defaults to 'true'." Verified independently (=#AutoEnable=true= is the commented compiled default, bluez 5.87). An implementer trusting the round-1 text would code default=False and report =powered-off-persistent= falsely on every common healthy machine with no/default config — the exact "targets a non-problem" failure. +Disposition: accepted — a real blocker. Corrected the premise everywhere: the fault now fires only on an *explicit* AutoEnable=false / disabled service / TLP entry, never on config absence. Fixed Problem/Context, the boot-enablement probe design, decision 2, and added an acceptance criterion that absent/default config gets the transient power-on. + +** DONE Round 2 (skeptical review): the machine =code= field doesn't exist on the step schema +Round 1's representation subsection spoke of "a stable machine =code=" as if the step result already had one. The skeptical review confirmed =schema.step= returns a fixed key set (=id=, =status=, =title=, =evidence=, =elapsed_ms=, =safety=, =next_action=) with no =code=, and the step schema is a locked test contract. +Disposition: accepted. The subsection now carries the human distinction in the existing =evidence=/=next_action= fields (no formatter change), and names the optional =code= key as an *additive* schema change with the test-contract ripple called out — not assumed to exist. + +** DONE Round 2 (skeptical review): "single-key set" for main.conf/tlp.conf hides real work +Round 1 called the config-edit verbs "narrowly-scoped" without naming the mechanism. The skeptical review noted =systemctl enable= is a clean argv verb but editing an INI key needs an idempotent setter (create the section if absent, preserve comments), which is more than a one-liner. +Disposition: accepted. The privilege-model design now names the INI-setter requirement and marks it Phase 2 work, distinct from the argv-clean service-enable verb. + +** DONE Round 2 (skeptical review): the shared-privilege-code sequencing caveat was claimed but missing +The round-1 review-history entry said the shared-privilege-model sequencing caveat was "already in Risks," but the Risks section did not mention it, and Phase 1 lands "shared cross-panel" code while =priv.py= is bt-local with one verb today — so Phase 1 isn't independently landable until that shared model ships. +Disposition: accepted. Added the sequencing caveat to Risks, naming Phase 0 (the two read-only probes) as the independently-landable slice and noting the AutoEnable correction makes those probes worth landing on their own. + +* Implementation phases + +Each phase leaves the tree green and independently useful, as the existing bt phases did. + +** TODO Phase 0 — the two read-only probes +Pure engine, no repair changes. The dmesg firmware-hint reader (no-adapter branch) and the boot-enablement reader (AutoEnable / service-enabled / TLP), both reporting into the =diagnose()= report dict. That dict feeds both views: the human summary (the default) and =--json= (=cli.py:113-116=); =format_doctor_human= (=cli.py:94=) renders the new evidence. There is no separate =bt diag= subcommand. Fakes: a canned journal buffer with each vendor's signature, and injected main.conf / =systemctl is-enabled= / tlp.conf states. + +** TODO Phase 1 — the firmware-hint verdict (Guide, no privilege dependency) +=diagnose= emits =no-adapter-firmware= (naming the blob, Reboot-tail Guide) when the signature matches. No auto-fix — this verdict is a Guide, so it needs no privilege model and lands independently of the shared cross-panel code. The run-time privilege resolution is a Phase 2 prerequisite, not this phase. + +** TODO Phase 2 — the persistent-power verdict and its fix (gated on the shared privilege model) +=powered-off-persistent= distinct from =powered-off=; the persistent fix (set AutoEnable / enable service / tlp) registers Privileged/Confirm through =priv.py= (which is one verb today), plus the INI/list setters above. =bt doctor --fix= applies it under the Confirm floor; the plain power-on path is unchanged. Hard ordering gate: this phase must not land before the shared cross-panel run-time privilege model exists — shipping a Privileged fix before the Confirm floor would let =--fix= edit root config via passwordless sudo ungated, the exact outcome the model forbids. + +** TODO Phase 3 — flip this spec to IMPLEMENTED +And log the vNext items (stale-bond signature, connection-parameter hints, bt-audio-profile expansion) to =todo.org=. + +* Acceptance criteria + +- [ ] With no adapter and a MediaTek/Intel/Realtek/Broadcom/Qualcomm firmware-load error in the kernel log, the doctor names the specific blob and prints the update-and-reboot Guide. +- [ ] With no adapter and no firmware error in the log, the doctor reports the generic "no controller attached" without inventing a firmware cause. +- [ ] An adapter that is powered off with AutoEnable=false *explicitly set* reports =powered-off-persistent= and FIX offers the persistent fix, not just a one-shot power-on. +- [ ] An adapter powered off with AutoEnable on (a transient off) still gets the quick power-on. +- [ ] An adapter powered off with main.conf absent or AutoEnable unset is treated as auto-enable-on (bluez's compiled default is true) and gets the transient power-on, never =powered-off-persistent=. +- [ ] Every new root-needing repair defaults to Confirm/Arm and never runs silently as Auto. +- [ ] The existing chain (rfkill/service/powered/device/audio) and its four auto-fix tiers classify and repair exactly as today (regression). + +* Readiness dimensions + +- *Data model & ownership* — the diagnose report gains a firmware-hint field (generated from the kernel log) and boot-persistence signals (generated from main.conf / systemctl / tlp.conf). The doctor writes main.conf / tlp.conf / the service enablement only under the Confirm floor. +- *Errors, empty states & failure* — an unreadable journal or main.conf yields "unknown," never a false firmware or persistence verdict. A no-match firmware scan falls back to the existing generic verdict (the safe direction). +- *Security & privacy* — device MACs are already redacted by =redact.py=; the new probes read the kernel log and config files, no secrets. The firmware-signature scan must not surface unrelated journal content. +- *Observability* — the wall names the blob and the persistence cause. The =--json= output carries the new signals. +- *Performance & scale* — the journal read runs only in the no-adapter branch; the persistence read only when powered-off/blocked. Neither touches a healthy run. +- *Reuse & lost opportunities* — reuses the existing chain, =btctl=, =sysio=, =priv.py=, and the shared cross-panel privilege model rather than a bt-local one. =diagnose= stays the single report authority. +- *Architecture fit* — both probes are additive inside existing chain branches (=_adapter_step=, =_powered_step=), not new chain links. Weak point: the firmware-signature table is vendor-specific and will drift as new hardware appears; it degrades to the generic verdict rather than mis-naming, which bounds the risk. +- *Config surface* — none new for the doctor; it *reads* bluez/tlp config and *writes* it only as a confirmed repair. N/A for its own knobs. +- *Documentation plan* — module docstrings, as the package does today. The wall is the user documentation. +- *Dev tooling* — =make test= and the bt panel smoke cover it; the new probes need a canned journal fixture and injected config states, fixture shapes the package can adopt. +- *Rollout, compatibility & rollback* — additive; the existing chain is untouched. The persistent fixes change bluez/tlp config and service enablement, so all are Confirm-tier and reversible by the user. Each new privileged repair adds a narrowly-scoped verb to =priv.py= (single-key sets, one named unit-enable) rather than a general root file-edit, so the passwordless-sudo surface grows by three tight verbs beyond today's single =restart-bluetooth=; the sudoers/priv change ships with the fixes. +- *External APIs & deps* — =journalctl -k=/=dmesg=, =/etc/bluetooth/main.conf=, =systemctl is-enabled=, and =/etc/tlp.conf= layouts are verified against the live system before Phase 0. The per-vendor firmware signatures come from the taxonomy's cluster-1 sources; no new packages. + +* Risks, rabbit holes, and drawbacks + +The firmware-signature table is the maintenance rabbit hole: each vendor logs its failure differently and new hardware adds new strings. The mitigation is that a miss degrades to the existing generic verdict — the doctor is never *worse* than today, only sometimes not-better — so the table can grow incrementally without a correctness cliff. + +The persistent-power fix must not fight a deliberate choice. A user (or TLP on a battery-conscious laptop) may want bluetooth off by default; blanket-enabling AutoEnable would override that silently. The verdict names the specific persistence cause and the fix is always confirmed, so the user sees exactly what would change before it does. + +The bt-audio-profile expansion (vNext) overlaps the audio taxonomy's Bluetooth-mic cluster. When it is picked up, it needs coordination with the audio doctor so the two panels don't both claim the same A2DP/HFP diagnosis with divergent verdicts. Named here so the seam is known. + +The privilege model this spec adopts is shared cross-panel code that does not exist yet — =priv.py= is bt-local today with one verb (=restart-bluetooth=). Phase 1 (which lands the run-time privilege resolution) can only land independently once that shared model ships somewhere; sequence it so whichever panel lands the shared code first, the others depend on it. Until then, Phase 0 (the two read-only probes) is the independently-landable slice, and the AutoEnable-default correction means those probes are worth landing on their own — they add the firmware and boot-persistence *signals* even before any privileged fix exists. + +* Review and iteration history + +** 2026-07-11 Sat @ 00:08:41 -0500 — Craig Jennings — Author +- What: drafted the bt doctor expansion from the bluetooth half of the failure taxonomy. v1 adds the dmesg firmware-hint probe (cluster 1) and the boot-enablement probe (cluster 2); the pairing/connection/audio expansions are staged to vNext. +- Why: the taxonomy showed the doctor's chain is structurally right but blind at its ends — a no-adapter verdict that won't name the firmware blob the kernel already logged, and a power-on that doesn't persist across boots. +- Artifacts: [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][the net/bt failure taxonomy]]; code read across =~/.dotfiles/bluetooth/src/bt/= (=doctor.py=, =repair.py=, =btctl.py=, =sysio.py=, =audio.py=, =priv.py=). + +** 2026-07-11 Sat @ 00:59:30 -0500 — Claude Code (archsetup) — Reviewer +- What: ran spec-review. Rubric =Not ready=. Recorded four findings, one =:blocking:= (no named-verdict layer to attach the proposed verdicts to) and three non-blocking (sudoers-surface expansion, the real diagnostic surface, the bounded firmware-read precedent). The three proposed decisions remain open. +- Why: the design is sound and the code read confirmed both target gaps are real — the engine reads no kernel log for firmware hints and nothing for boot persistence (=main.conf=/=is-enabled=/=tlp.conf= all grep-clean). What holds the rubric is that the spec borrows the net doctor's verdict-naming language, but the bt doctor is step-status-based (=status= pass/fail/warn/info, =overall= ok/warn/fail, =doctor.py:182-183=) with no verdict enum — so how each new "verdict" is represented is undefined. Once that representation is decided and Craig accepts the three decisions, this reaches =Ready with caveats= (shared privilege-model code being the sequencing caveat, already in Risks). +- Artifacts: engine facts verified across =doctor.py= (:25 AUTO_FIX, :46-50 no-adapter branch, :84 bounded journalctl precedent, :100-107 powered step, :174-183 diagnose/overall), =priv.py:27-34= (single =restart-bluetooth= verb), =cli.py:94/173= (formatter + json default), =redact.py:11-20=. Findings in =* Review findings=. + +** 2026-07-11 Sat @ 01:20:00 -0500 — Claude Code (archsetup) — Responder +- What: ran spec-response. Dispositioned all four findings (three accepts, one accept-with-modify: the new privileged verbs are narrowly scoped, not a general root file-editor) and closed all three decisions as accepted. Added Design "How the new 'verdicts' are represented" defining a verdict as a step outcome code in the existing status model; cited the bounded =journalctl= precedent; corrected the Phase 0 diagnostic surface; constrained the sudoers expansion in the privilege-model design and the Rollout dimension. Both =[/]= cookies now read complete. +- Why: convergence toward implementation-ready. The verdict-representation finding was the real blocker — the spec had borrowed net's verdict language onto a step-status engine, so defining "verdict = outcome code" is what makes the two probes implementable without inventing a parallel system. +- Artifacts: findings =[4/4]=, decisions =[3/3]=. Scope expanded only by the tightened privilege-verb obligation, filed in the Rollout dimension, so the readiness rubric holds. Awaiting re-review to flip DRAFT → READY. + +** 2026-07-11 Sat @ 01:45:00 -0500 — Claude Code (archsetup) — Reviewer + Responder (round 2) +- What: a skeptical adversarial re-review returned Not ready with one hard blocker — the AutoEnable default is =true=, not =false= (round 1 had it backwards, verified against bluez 5.87), which would fire =powered-off-persistent= falsely on every healthy default-config machine. Corrected the premise everywhere so the fault fires only on explicit signals. Also fixed three non-blocking issues: the machine =code= is an additive schema change (=schema.step= has no =code= key today), the INI-setter for main.conf/tlp.conf is real Phase 2 work not a one-liner, and the shared-privilege-code sequencing caveat is now actually in Risks (round 1's history claimed it was). Findings now =[8/8]=. +- Why: round 1 rubber-stamped the AutoEnable default without checking the installed bluez. Round 2 verified it directly (=#AutoEnable=true= in =/etc/bluetooth/main.conf=). The correction also strengthens phasing: Phase 0's read-only probes are worth landing on their own, before any privileged fix. +- Artifacts: bluez 5.87 =/etc/bluetooth/main.conf= (AutoEnable default true); =schema.step= (fixed key set, no =code=); =priv.py:25-34= (verb→argv dispatch). Awaiting a third re-review. + +** 2026-07-11 Sat @ 02:00:00 -0500 — Claude Code (archsetup) — Reviewer (round 3) +- What: third skeptical adversarial re-review. Verdict =Ready with caveats=, no blocking findings. Verified all four round-2 resolutions: the AutoEnable premise is now self-consistent (every fault sentence keys on an explicit signal, acceptance criteria cover the absent-config case), the verdict representation is correct against =schema.step= (no =code= key; the distinction rides =evidence=/=next_action=, rendered by =cli.py:98-102= with no formatter change), the INI-setter is concretely named, and Phase 0 is genuinely independently landable. Folded two non-blocking corrections (the human summary is the doctor default not JSON; tlp.conf is a flat list-edit not an INI section) and decoupled Phase 1's Guide-only firmware verdict from the blocked privilege scaffolding. Flipped DRAFT → READY. +- Why: the loop terminates at the rubric. Two skeptical passes plus code re-verification found no remaining blocker. The caveat — Phase 2's persistent fix needs the shared privilege model — is recorded in Risks, Decision 2/the privilege design, and a hard ordering gate on Phase 2; Phases 0 and 1 build today. +- Artifacts: findings =[8/8]=, decisions =[3/3]=. Non-blocking notes folded into wording; no open blocker. diff --git a/docs/specs/2026-07-11-net-doctor-expansion-spec.org b/docs/specs/2026-07-11-net-doctor-expansion-spec.org new file mode 100644 index 0000000..0666dfe --- /dev/null +++ b/docs/specs/2026-07-11-net-doctor-expansion-spec.org @@ -0,0 +1,266 @@ +#+TITLE: Net Doctor Expansion — the clusters the ladder doesn't yet name +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-11 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Net Doctor Expansion +:PROPERTIES: +:ID: ce29b103-ed9d-4f56-bf8c-9ed8fe680ff3 +:END: +- [2026-07-11 Sat @ 02:30 -0500] DOING — decomposed into build tasks (spec-response Phase 6); parent task in =todo.org= bound by =:SPEC_ID:=. Phase 0 (read-only control-plane probe) is buildable now and =:solo:=; Phases 1-2 are gated on the shared cross-panel privilege model. The systemic connection-name redaction gap is a separate filed task. +- [2026-07-11 Sat @ 02:00 -0500] READY — third skeptical re-review returned Ready with caveats, no blocking findings; all round-2 resolutions verified against the engine. Caveat accepted: Phases 1-2 depend on the shared cross-panel run-time privilege model, which doesn't exist yet, so only Phase 0 (read-only detection) is buildable today. The redaction de-scope (parity + separate systemic task) is the one product call for Craig's eye. +- [2026-07-11 Sat @ 01:45 -0500] DRAFT — round-2 review + response. A skeptical re-review caught two blockers the first round missed (one I introduced): the redaction "copy/--json surface" does not exist, and =rival-manager= as =needs-user-action= could never run its own fix. Both corrected — parity redaction + systemic gap filed separately; all three control-plane verdicts are =fixable=. Auth signal repointed at profile key-mgmt + scan security; masked-NM early-return path named. Findings =[9/9]=, decisions =[3/3]=. Awaiting a third re-review. +- [2026-07-11 Sat @ 01:20 -0500] DRAFT — review incorporated (spec-response). All five findings dispositioned (=[5/5]=), all three decisions accepted and closed (=[3/3]=). Both cookies complete; awaiting a re-review to flip DRAFT → READY. Redaction blocker resolved (later found wrong in round 2). +- [2026-07-11 Sat @ 00:59:30 -0500] DRAFT — reviewed (spec-review). Stays DRAFT: three decisions open plus one =:blocking:= finding (connection names are not redacted today, contra the spec's Security dimension). Design confirmed against the live engine — the two target gaps (=is-enabled= reads, =system-connections= reads) genuinely do not exist yet, so the expansion targets real ground. Findings in =* Review findings=. +- [2026-07-11 Sat @ 00:08:41 -0500] DRAFT — drafted. Extends the existing net doctor (=~/.dotfiles/net/=, shipped) using the network half of the failure taxonomy ([[file:../design/2026-07-10-net-bt-failure-taxonomy.org][2026-07-10-net-bt-failure-taxonomy.org]]). Grounded in a read of the live engine, not memory. + +* Metadata + +| Field | Value | +|----------+-----------------------------------------------------------------------------------| +| Status | doing | +|----------+-----------------------------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+-----------------------------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+-----------------------------------------------------------------------------------| +| Related | [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][net/bt failure taxonomy]] ; the cross-panel run-time-privilege and copy+close tasks | +|----------+-----------------------------------------------------------------------------------| + +* Summary + +The net doctor is the most mature of the three panel doctors: its probe ladder already walks link → IP → gateway → route → DNS → egress, and its classifier already names most of the taxonomy's failure clusters with the lightest fix. This spec closes the two clusters it does *not* reach — control-plane conflicts (two network managers fighting over one interface, a masked NetworkManager, a keyfile the daemon silently refuses) and the sharper naming of the terminal auth cluster — and adopts the cross-panel run-time privilege model. It is an expansion of a working doctor, not a rewrite. + +* Problem / Context + +The failure taxonomy sorted ~74 real network failure modes into eight symptom clusters. Read against the live classifier (=~/.dotfiles/net/src/net/classify.py=), the net doctor already reaches six of them: cluster 1 (=rfkill=, =manage-device=), cluster 2 (the DHCP-failed path), cluster 3 (=tunnel-down=, =vpn-policy=), cluster 5 (=resolved-restart=, =dns-test=), and cluster 6 (=portal=, =clock-sync=, =proxy=, and the =upstream-not-local= terminal STOP). Those are =classify='s real action identifiers; a few taxonomy labels (no-hardware, DHCP-failed, DNS-not-resolving) are message text the actions carry, not separate verdicts, and =dns-override= is a doctor =FIX_CHAIN=, not a classifier action. The classifier's terminal-first ordering already refuses to loop repairs against a wrong password, a held portal, or a VPN-owned route. That is a lot of the taxonomy, already built. + +Two clusters fall through. + +** The control plane can be broken while the radio looks fine (cluster 7) + +The taxonomy's largest untouched cluster is NetworkManager itself. When =dhcpcd.service= runs beside NM's internal DHCP client, or =systemd-networkd= and NM both claim one link, or =iwd= and =wpa_supplicant= are both active, the interface flaps or never leases — and every existing probe reads a plausible-looking radio with no verdict that names the fight. NM masked (=systemctl start= returns "Unit is masked") reads as "NetworkManager isn't running" today, which points =nm-restart= at a service that cannot start. A =.nmconnection= keyfile that isn't =600= root-owned is silently skipped by the daemon, so a saved network "just won't connect" with no signal the doctor surfaces. These are distinct root causes with distinct fixes, and no current probe detects any of them. The classifier does carry granular control-plane actions (=reset=, =bounce=, =rfkill=, =nm-restart=), but =nm-restart= is narrowly the dead-service restart, and nothing reads =systemctl is-enabled= or the =system-connections= keyfiles — the reads that would surface a masked NM, a rival manager, or a bad keyfile. The gap is missing detection, not one verdict overloaded across three faults. + +** The auth cluster is named too coarsely (cluster 4) + +=classify= detects an auth failure from NM state 120 / GENERAL.REASON and returns =needs-user-action= — correct, and correctly terminal. But the taxonomy shows the auth cluster is not one failure: a pure-WPA3/SAE association failure, a hidden SSID never probed, an enterprise cert mismatch, and a wrong regulatory domain are different faults with different next actions, and some (SAE key-mgmt, =wifi.hidden yes=) are one-line profile fixes rather than "re-enter the password." The doctor collapses them all to one message. + +** The flaky/drops cluster needs signal the one-shot doctor doesn't collect (cluster 8) + +Powersave disconnects, roaming stalls, USB autosuspend, no-reconnect-after-resume, firmware crashloops — these are intermittent, and a single-shot "why am I offline right now" probe cannot see them. Naming them needs event-log correlation the doctor doesn't do. This cluster is real but out of v1; it is named here so the boundary is explicit. + +* Goals and Non-Goals + +** Goals + +- Add a control-plane cluster: a probe that detects a second network/DHCP manager active alongside NetworkManager, a masked/failed NM distinct from a merely-stopped one, and a keyfile-permission fault, each with its own verdict and the lightest fix. +- Sharpen the auth verdict: extract the specific auth cluster cause (SAE, hidden SSID, enterprise cert, regdom) so =needs-user-action= names the real next step, and apply the one-line profile fix where one exists. +- Adopt the cross-panel run-time privilege model (Auto / Privileged / Reboot-tail / Guide, resolved from =sudo -n= + tty + GUI), so control-plane repairs that need root (=systemctl disable dhcpcd=, =chmod= a keyfile, =systemctl unmask=) run under the same Confirm/Arm-default floor the audio doctor defined. + +** Non-Goals + +- Rebuilding the existing ladder. Clusters 1/2/3/5/6 stay exactly as they classify today; this spec only adds where the taxonomy shows a gap. +- The flaky/drops cluster (cluster 8). Naming intermittent faults needs event-log correlation that a one-shot doctor doesn't do. Logged to =todo.org=, not built here. +- Turning the doctor into a NetworkManager profile editor. The auth-cluster fixes are limited to the one-line profile settings that get a stuck association online (SAE key-mgmt, hidden flag); it will not manage certificates, enterprise identities, or credential entry — those stay Guide. +- Silent privileged action. Every Privileged/Reboot-tail control-plane repair defaults to Confirm/Arm, never silent Auto — same stance as the audio spec. + +** Scope tiers + +- *v1:* the control-plane conflict probe and its verdicts (rival-manager, NM-masked, keyfile-perms); sharper auth-cluster reason extraction plus the SAE/hidden one-line fixes; adoption of the run-time privilege model for the new root-needing repairs. +- *Out of scope:* the flaky/drops cluster; certificate/enterprise credential management; any change to the six clusters already classified. +- *vNext:* event-log correlation for cluster 8 (powersave/roam/suspend/autosuspend drop signatures); a DoT/DNSSEC-specific verdict distinguishing "the venue resolver mangles DNSSEC" from the generic DNS-not-resolving; per-profile autoconnect/duplicate-profile hygiene. All logged to =todo.org=. + +* Design + +** For the user + +Nothing changes for the six clusters that already work. What changes is that two failure shapes that today produce a wrong or vague verdict start naming themselves. + +When a second manager is fighting NetworkManager for the link, the wall says so by name — "dhcpcd is running alongside NetworkManager and they're fighting over the interface" — and offers the fix (stop the rival) rather than bouncing a connection that will flap again the moment the other daemon re-grabs it. When NM is masked, the verdict distinguishes "masked — it can't start until unmasked" from "stopped," so FIX unmasks rather than uselessly restarting. When a saved network silently won't load because its keyfile is world-readable, the doctor names the permission fault instead of leaving the user to wonder why a known-good network never activates. + +When association fails on auth, the wall stops saying only "authentication failed" and names which auth: a WPA3-only network the profile isn't set for, a hidden SSID that needs the hidden flag, an enterprise network missing its CA cert. The first two carry a one-press fix; the rest tell the user the specific thing to supply. + +** For the implementer + +*** The control-plane probe + +A new read-only probe tier, beside the existing ones, answering three questions the current ladder never asks: + +1. *Is a rival manager active?* Check whether =dhcpcd.service=, =systemd-networkd.service=, or a standalone =iwd.service= is active while NetworkManager is also active and owns (or wants to own) the link. This is a =systemctl is-active= read plus NM's backend setting — bounded, no hang. A rival that is active is the verdict; the fix is =systemctl disable --now <rival>= (Privileged), never a connection bounce. +2. *Is NM masked or failed, as distinct from stopped?* =systemctl is-enabled NetworkManager= returns =masked=; the unit's =ActiveState=/=Result= distinguishes a crash-loop from a clean stop. A masked NM gets an =unmask= verdict (Privileged), not the existing =nm-restart=. +3. *Does the active profile's keyfile have the wrong permissions?* For the selected connection, stat its =/etc/NetworkManager/system-connections/*.nmconnection=; a non-=600= or non-root-owned file is the silent-skip fault. Fix: =chmod 600= + =chown root= (Privileged). + +The probe runs before the existing "NetworkManager isn't running" rule, because a masked NM and a rival-manager fight are both more specific than "not running" and would otherwise be mis-verdicted by it. One ordering subtlety verified against the engine: a fully-down NM makes =nmcli= raise and =diagnose= early-returns with only the service and link steps (=diag.py:503-511=), so the masked/failed check has to run inside that early-return path — otherwise a masked NM short-circuits before the new probe and never reaches its verdict. + +*** The classifier gains control-plane verdicts + +=classify= adds, in specificity order ahead of the generic =nm-restart= rule: =rival-manager= (fixable → Privileged disable-rival), =nm-masked= (fixable → unmask), =keyfile-perms= (fixable → chmod/chown). All three are =fixable= — the doctor only applies a repair when the outcome is =fixable= (=doctor.py:181=), so a terminal =needs-user-action= would never run its fix — and each is Privileged, so =net doctor --fix= still gates it on the Confirm floor rather than acting silently. Each carries evidence naming the specific rival/unit/file. These are additive; the existing rules below them are untouched. + +*** The auth verdict takes a reason + +The specific auth cause is not in =auth_failed_reason= — that REASON/journal string (=doctor.py:32-56=) only marks *that* auth failed, not whether it was SAE, a hidden SSID, or an enterprise cert. The distinction is read from signals the engine already has: the active profile's key-mgmt (=manage.py:36-47=, which already detects WPA3/SAE incompatibility) and the scanned network's SECURITY flags (=nmcli.py:164=). From those, =gather_context= derives a small classifier: SAE/PMF, hidden-SSID, enterprise-cert, or generic-PSK. (regdom has no engine signal — grep-clean — so it stays Guide.) The =needs-user-action= message is keyed off it. For SAE and hidden-SSID — the two with a deterministic one-line profile fix — the verdict becomes =fixable= with a Privileged/Auto profile-modify action (=key-mgmt sae= + PMF, or =wifi.hidden yes=) rather than terminal. The rest stay =needs-user-action= with a sharpened message. + +*** The privilege model + +The new repairs (=systemctl disable <rival>=, =unmask=, =chmod=/=chown= a keyfile, profile-modify) are the net doctor's first root-needing doctor repairs beyond the ones already in =priv.py=. They adopt the cross-panel run-time resolution the audio spec defined: Privileged remedies run silently where passwordless sudo exists (every archsetup install), prompt on a CLI with a tty, and default to Confirm/Arm — never silent Auto. This is the same standard, not a net-specific one; the shared implementation is the tracked cross-panel task. Concretely, the three fixes register through =priv.py='s =VERBS= table and =repair.py='s =ACTIONS= registry (=disable-rival=, =unmask-nm=, =chmod-keyfile=) — the existing dispatch path, not a new one — and each verb is a narrowly-scoped command (disable one named unit, unmask NetworkManager, chmod/chown one keyfile), never a general run-as-root, so the passwordless-sudo grant stays tight. + +*** Redaction of the new evidence + +The new verdicts surface a connection name (=rival-manager=, =keyfile-perms=) and a =/etc/NetworkManager/system-connections/*.nmconnection= basename. The redaction reality, verified against the engine: SSID redaction exists only in the event log (=redact_event=, gated on =redact_ssid=, default off); the copyable report scrubs MAC/IP only (=scrub_text=, =redact.py=), and =--json= is a raw =json.dumps(out)= with no redaction at all (=cli.py=). Connection names already appear in the clear in the link-step evidence (=diag.py:103=, e.g. "wlan0 connected (HomeNetwork)") and in =--json= today. So there is no copy-vs-wall redaction surface to piggyback on, and the new verdicts add no new leak class — a connection name already shows for the link step. v1 keeps parity: the new evidence is redacted exactly as the existing link-step evidence is (MAC/IP via =scrub_text=; connection name in the clear). The systemic gap — connection names and SSIDs leaking into the copyable report and =--json= across every step, gated behind a default-off toggle — predates this spec and spans the whole engine, so it is filed as its own task rather than half-solved for two new verdicts. The keyfile probe reads permissions, not secrets; the auth-reason extraction reads NM's reason string, not the PSK. + +* Alternatives Considered + +** Fold the control-plane faults into the existing =nm-restart= verdict + +- Good, because it is zero new classifier surface. +- Bad, because =nm-restart= is the wrong fix for all three: restarting a masked NM fails, restarting NM does not stop a rival =dhcpcd=, and it does nothing for a bad keyfile. A shared verdict would send the doctor's one fix at three faults it can't fix. +- Rejected. Different root causes with different fixes are different verdicts — the same principle the audio spec used for =pulse-hung= vs =pulse-down=. + +** Make the doctor a full profile editor for the auth cluster + +- Good, because it could fix more auth failures automatically. +- Bad, because credential and certificate entry is a genuine user decision, not a repair — the doctor cannot invent an enterprise CA or a password. Auto-editing profiles beyond the two deterministic one-liners risks writing a wrong setting the user then has to unwind. +- Rejected in favor of fixing only SAE key-mgmt and the hidden flag, and guiding the rest. + +** Build the flaky/drops cluster now with a synthetic re-probe + +- Good, because it would catch powersave/autosuspend "works then dies" cases. +- Bad, because a one-shot doctor invoked when the user is already offline has no drop history to read; catching intermittent faults needs the panel's event log correlated over time, which is a separate probe surface. +- Rejected for v1; logged as vNext. + +* Decisions [3/3] + +** DONE The control-plane probe and its three verdicts +Context: cluster 7 is the taxonomy's largest untouched cluster, and no current probe detects a masked NM, a rival manager, or a bad keyfile. +Decision: we will add a read-only control-plane probe (rival-manager active-check, NM masked-vs-failed-vs-stopped, active-profile keyfile permissions) and three verdicts ahead of the generic =nm-restart= rule, each with the lightest specific fix. +Consequences: the doctor names the fight instead of bouncing a link that will re-flap; harder — three new verdicts and a probe that reads =systemctl= state plus a stat, and the ordering has to sit ahead of the existing not-running rule without disturbing it. +Resolution: accepted as proposed (the fold-into-nm-restart alternative was rejected — that fix cannot address any of the three faults). Owner: Craig. + +** DONE How far the auth-cluster fix goes +Context: the auth cluster is terminal today; some members (SAE, hidden SSID) have deterministic one-line profile fixes, others (enterprise cert, credential) do not. +Decision: we will extract the specific auth reason and make only SAE-key-mgmt and hidden-flag =fixable=; everything else stays =needs-user-action= with a sharpened, cause-named message. Both profile-modify fixes are Confirm-tier (persisted state). +Consequences: two more auth failures self-heal; harder — the doctor now writes to a connection profile, which is a heavier action than a bounce, and the boundary between "fix" and "guide" inside one cluster has to be defended so it doesn't creep into credential management (the Non-Goal is the guardrail). +Resolution: accepted as proposed. Owner: Craig. + +** DONE Adopt the run-time privilege model as the cross-panel standard +Context: the new control-plane repairs need root; the audio spec already defined the four-class run-time model and made it a cross-panel standard. +Decision: we will adopt it verbatim — Privileged repairs run where passwordless sudo exists, prompt on a tty, default to Confirm/Arm, never silent Auto — sharing the implementation with the other panels rather than reimplementing it. +Consequences: the net doctor's root repairs are consistent with audio/bt/maint; harder — it couples this spec to the shared privilege-model task, so the ordering across panels has to be settled (which panel lands the shared code). That sequencing caveat is recorded in Risks. +Resolution: accepted — this adopts an already-decided cross-panel standard, not a new choice. Owner: Craig. + +* Review findings [9/9] + +** DONE Connection names are not redacted today, but the spec assumes they are :blocking: +The Security & privacy dimension stated "SSIDs and connection names are already redacted by =redact.py=." Verified against the live engine: =redact.py= redacts SSID, MAC, IP, secret-keys, and =portal_url= only (=redact.py:9-53=) — there is no connection-name redaction. The new =rival-manager= and =keyfile-perms= verdicts surface a connection name and a =/etc/NetworkManager/system-connections/*.nmconnection= path (the file basename is the connection name) into the wall and the =--json= output. +Disposition: accepted, modified in one detail. Rather than redact everywhere (which would blank the network name on the user's own screen), v1 redacts the connection name and keyfile basename at the same copy/=--json= surface the existing SSID redaction covers — the shareable text is scrubbed, the on-screen wall still names the network. Folded into Design "Redaction of the new evidence," the Security & privacy dimension, Phase 1, and a new acceptance criterion. + +** DONE "One =nm-restart= verdict covers the whole control plane" overstates current scope +Problem/Context and the first Alternative rested the motivation on the claim that the doctor "currently has one verdict (=nm-restart=) covering the whole control plane." Verified: =nm-restart= is one narrow action for a dead NetworkManager *service* only (=classify.py:96-97=, =repair.py:705=); the classifier already carries granular actions (=reset=, =bounce=, =rfkill=, =manage-device=, =resolved-restart=…). The real, verified gap is that *no* probe detects a rival manager, a masked NM, or a bad keyfile (=is-enabled= and =system-connections= reads are grep-clean). +Disposition: accepted. Reworded the cluster-7 paragraph to rest on the missing detections; the first Alternative already argues against reusing =nm-restart= as a fix, which stays valid. + +** DONE "=systemctl is-enabled= already used by the engine" is inaccurate +External APIs & deps said =systemctl is-active/is-enabled= are "already used by the engine." Verified: only =is-active= is used (=cmd.py:27=); =is-enabled= has zero hits, and =system-connections= is not read today either. +Disposition: accepted. Corrected the dimension to say =is-active= and =nmcli= are used today, =is-enabled= and the keyfile reads/=stat= are new bounded calls. + +** DONE Problem/Context lists message text as if it were classifier verdicts +The six-cluster evidence listed =no-hardware=, =DHCP-failed=, =DNS-not-resolving=, =dns-override= as classifier verdicts. Verified: the first three are message text, and =dns-override= is a doctor =FIX_CHAIN= (=doctor.py:24=), not a =classify= action. +Disposition: accepted. Rewrote the listing to cite =classify='s real action identifiers and note which taxonomy labels are message text. + +** DONE Name the =priv.py= / =repair.py= integration point for the new privileged repairs +The spec said it reuses =priv.py= but the phase plan did not name the concrete integration: privileged repairs dispatch through =priv.py='s =VERBS= table (=priv.py:150=) and =repair.py='s =ACTIONS= registry (=repair.py:695=), not a new path. +Disposition: accepted, extended. Phase 1 and the privilege-model design now name =disable-rival=/=unmask-nm=/=chmod-keyfile= as =VERBS= + =ACTIONS= entries, and add that each verb is a narrowly-scoped command (not a general run-as-root) so the passwordless-sudo grant stays tight. + +** DONE Round 2 (skeptical review): the redaction "copy/--json surface" does not exist :blocking: +The round-1 disposition claimed v1 would redact at "the same copy/=--json= surface the existing SSID redaction covers." A second skeptical review traced every redaction call site and found no such surface: SSID redaction lives only in =redact_event= (event log, gated on =redact_ssid=, default off), =--json= is a raw =json.dumps(out)= with no redaction, and the copyable report scrubs MAC/IP only (=scrub_text=). Connection names already appear in the clear in the link-step evidence (=diag.py:103=) today. So the round-1 resolution invented a surface that isn't there. +Disposition: accepted — the round-1 fix was wrong and is corrected. v1 keeps parity with existing link-step behavior (the new verdicts leak no more than the link step already does); the systemic connection-name/SSID redaction gap across the report and =--json= is filed as a separate task. Rewrote Design "Redaction of the new evidence," the Security dimension, Phase 1, and the acceptance criterion. + +** DONE Round 2 (skeptical review): rival-manager can't satisfy its own FIX criterion :blocking: +Round 1 marked =rival-manager= =needs-user-action= (terminal) while the acceptance criterion required FIX to stop the rival. But the doctor only applies a repair when the outcome is =fixable= (=doctor.py:181=), so a terminal =rival-manager= would never run =disable-rival=. +Disposition: accepted. =rival-manager= is now =fixable= (Privileged/Confirm), consistent with =nm-masked= and =keyfile-perms=; the Confirm floor still gates it. Corrected the classifier design and added an acceptance criterion that all three control-plane verdicts are =fixable=. + +** DONE Round 2 (skeptical review): auth-cluster classifier pointed at the wrong signal +Round 1 said =auth_failed_reason= would carry the SAE/hidden/enterprise distinction. The skeptical review confirmed that REASON/journal string only marks *that* auth failed; the actual discriminating data lives in the profile key-mgmt (=manage.py:36-47=, already detects SAE incompatibility) and the scanned SECURITY flags (=nmcli.py:164=). +Disposition: accepted. Phase 2 / the auth-verdict design now reads those signals instead of =auth_failed_reason=; regdom stays Guide (no engine signal). No new data collection is forced. + +** DONE Round 2 (skeptical review): masked-NM must be reached in the NM-down early-return +When NM is fully down, =nmcli= raises and =diagnose= early-returns with only the service and link steps (=diag.py:503-511=), so a masked NM would short-circuit before the control-plane probe and never reach the new verdict. +Disposition: accepted. The control-plane probe design now states the masked/failed check must run inside that early-return path. + +* Implementation phases + +Each phase leaves the tree green and independently useful, as the existing net phases did. + +** TODO Phase 0 — the control-plane probe (read-only) +Pure engine, no classifier changes. A probe module that reports rival-manager state, NM masked/failed/stopped, and active-profile keyfile permissions into the diag context. =net diag --json= shows the new signals. Fakes: injected =systemctl is-active/is-enabled= results and a temp system-connections tree. + +** TODO Phase 1 — control-plane verdicts + the privilege model +=classify= gains =rival-manager=, =nm-masked=, =keyfile-perms= (all =fixable=), ordered ahead of the generic not-running rule, with the masked/failed check reachable in the NM-down early-return path. The run-time privilege resolution lands (shared with the cross-panel task) and the three new fixes register as Privileged/Confirm through =priv.py='s =VERBS= table and =repair.py='s =ACTIONS= registry (=disable-rival=, =unmask-nm=, =chmod-keyfile=), each a narrowly-scoped verb. The new evidence keeps parity with existing redaction (MAC/IP via =scrub_text=); the systemic connection-name gap is a separate task, not this phase. =net doctor= names them; =net doctor --fix= applies them under the Confirm floor. Hard ordering gate: this phase must not land before the shared run-time privilege model exists — =priv.py= today is a bare =VERBS=+sudo dispatcher with no Confirm/Arm resolution (=priv.py:150=) and =_attempt= runs repairs ungated, so shipping the Privileged =fixable= verdicts first would let =net doctor --fix= silently disable =dhcpcd= via passwordless sudo, the exact outcome the model forbids. Phase 0 (read-only detection) has no such dependency and lands first; if the fault-naming is wanted before the privilege model, the verdicts can ship detection-only (no =--fix= action) as an interim slice. + +** TODO Phase 2 — the sharpened auth verdict +The auth-reason classifier; SAE and hidden-SSID become =fixable= profile-modify repairs; the rest get cause-named =needs-user-action= messages. Pairwise over (reason × profile-state). + +** TODO Phase 3 — flip this spec to IMPLEMENTED +And log the vNext items (flaky/drops cluster, DoT/DNSSEC verdict, profile hygiene) to =todo.org=. + +* Acceptance criteria + +- [ ] With =dhcpcd.service= active alongside NetworkManager, the doctor reports =rival-manager= naming dhcpcd, and FIX stops it rather than bouncing the connection. +- [ ] A masked NetworkManager reports =nm-masked= (distinct from stopped), and FIX unmasks it. +- [ ] A non-600 keyfile for the active profile reports =keyfile-perms=, and FIX corrects the permissions. +- [ ] A WPA3-only association failure reports the SAE cause and (with --fix) sets the profile's key-mgmt, rather than saying only "authentication failed." +- [ ] An enterprise-cert auth failure stays =needs-user-action= but names the missing CA cert. +- [ ] Every new root-needing repair defaults to Confirm/Arm and never runs silently as Auto. +- [ ] =rival-manager=, =nm-masked=, and =keyfile-perms= are all =fixable= outcomes, so =net doctor --fix= actually runs their repairs (a terminal outcome would be skipped by =doctor.py:181=). +- [ ] The =rival-manager= and =keyfile-perms= verdicts expose the connection name no more than the existing link-step evidence does (MAC/IP scrubbed, connection name in the clear — parity, not a new leak). The systemic connection-name/=--json= redaction gap is tracked as a separate task. +- [ ] The six existing clusters classify exactly as they do today (regression). + +* Readiness dimensions + +- *Data model & ownership* — the diag context gains control-plane signals (generated per-probe) and a richer =auth_failed_reason= (generated from GENERAL.REASON + journal, as today). The doctor never writes NM config except the two auth profile-modifies and the keyfile-perms fix, all under the Confirm floor. +- *Errors, empty states & failure* — an unreadable =systemctl=/=stat= yields "unknown," never a false rival/masked/perms verdict (the safe direction). Partial reads degrade to the existing behavior. +- *Security & privacy* — verified: SSID redaction exists only in the event log (=redact_event=, gated on =redact_ssid=, default off); the copyable report scrubs MAC/IP only (=scrub_text=) and =--json= is raw. Connection names already appear in the clear in the link-step evidence and =--json= today, so the new =rival-manager=/=keyfile-perms= verdicts add no new leak class — they reach parity with existing behavior. The systemic connection-name/SSID redaction gap across the report and =--json= is pre-existing and filed as a separate task. The keyfile probe reads permissions, not secrets; the auth-reason extraction reads NM's reason string, not the PSK. +- *Observability* — the wall names the specific rival/unit/file. =--json= carries the new context. +- *Performance & scale* — three =systemctl= reads and a stat; negligible beside the existing probe cost. +- *Reuse & lost opportunities* — reuses =gather_context='s existing auth-reason extraction, =priv.py=, and the shared cross-panel privilege model rather than a net-local one. =classify= stays the single verdict authority. +- *Architecture fit* — the new probe is additive beside the existing tiers; the classifier additions sit ahead of the generic not-running rule and leave the rest untouched. Weak point: ordering — the new verdicts must precede =nm-restart= or a masked/rival case mis-classifies. +- *Config surface* — none new. N/A. +- *Documentation plan* — module docstrings, as the package does today. The wall is the user documentation. +- *Dev tooling* — =make test= and the net panel smoke cover it; the control-plane probe needs injected =systemctl= state, a fixture shape the package already uses elsewhere. +- *Rollout, compatibility & rollback* — additive; =net doctor= with no new fault behaves as today. The auth profile-modify and keyfile-perms fixes change persisted NM state, so both are Confirm-tier and reversible by the user. +- *External APIs & deps* — =systemctl is-active= and =nmcli= are already used by the engine; =systemctl is-enabled= and reads/=stat= under =/etc/NetworkManager/system-connections= are new calls (bounded with an explicit timeout, like every existing probe). No new packages. The exact rival-manager set (dhcpcd/networkd/iwd) is verified against the live system before Phase 0. + +* Risks, rabbit holes, and drawbacks + +The rival-manager check can false-positive if a rival service is active but not actually contending for the same interface (e.g. =systemd-networkd= managing a container bridge while NM owns wifi). The probe must scope the conflict to the link the doctor is diagnosing, not merely "is networkd running" — otherwise it cries wolf on a legitimate split. This is the main correctness rabbit hole and wants a test with a bridge-only networkd. + +The auth-cluster boundary between "fix" and "guide" is a slope. SAE and the hidden flag are safe because they are deterministic and reversible; the temptation is to add "just one more" auto-fix until the doctor is editing enterprise profiles it shouldn't. The Non-Goal is the guardrail; hold it. + +Coupling to the shared privilege-model task means this spec can't fully land until that model exists somewhere. Sequence it: whichever panel lands the shared code first, the others depend on it. + +* Review and iteration history + +** 2026-07-11 Sat @ 00:08:41 -0500 — Craig Jennings — Author +- What: drafted the net doctor expansion from the network half of the failure taxonomy. v1 adds the control-plane cluster (rival-manager, NM-masked, keyfile-perms) and sharpens the auth verdict; the flaky/drops cluster is staged to vNext. +- Why: the taxonomy showed the net doctor already reaches six of eight clusters, and the two it misses (control plane, auth naming) are where its verdicts are wrong or vague today. +- Artifacts: [[file:../design/2026-07-10-net-bt-failure-taxonomy.org][the net/bt failure taxonomy]]; code read across =~/.dotfiles/net/src/net/= (=classify.py=, =diag.py=, =doctor.py=, =repair.py=, =priv.py=). + +** 2026-07-11 Sat @ 00:59:30 -0500 — Claude Code (archsetup) — Reviewer +- What: ran spec-review. Rubric =Not ready=. Recorded five findings, one =:blocking:= (connection-name redaction assumed but absent) and four non-blocking accuracy/integration corrections. The three proposed decisions remain open. +- Why: the design is sound and the code read confirmed both target gaps are real (=is-enabled= and =system-connections= reads do not exist in the engine today), so the expansion is well-grounded. What holds the rubric is the privacy claim the spec states as already-true (=redact.py= redacts SSID/MAC/IP/secret-keys/portal_url only — not connection names, =redact.py:9-53=) and the still-open decisions. Once the redaction finding is dispositioned and Craig accepts the three decisions, this reaches =Ready with caveats= (the cross-panel shared privilege-model code is the remaining sequencing caveat, already recorded in Risks). +- Artifacts: engine facts verified across =classify.py= (:96-97, :55-62), =diag.py= (:518-537, bounded probes), =doctor.py= (:112-116 auth-reason), =cmd.py:27= (=is-active= only), =priv.py:150=/=repair.py:695= (dispatch registry), =redact.py:9-53=. Findings in =* Review findings=. + +** 2026-07-11 Sat @ 01:20:00 -0500 — Claude Code (archsetup) — Responder +- What: ran spec-response. Dispositioned all five findings (four accepts, one accept-with-modify: the redaction scrubs at the copy/=--json= surface, not the on-screen wall) and closed all three decisions as accepted. Folded the redaction requirement into Design, the Security dimension, Phase 1, and a new acceptance criterion; reworded the =nm-restart=/control-plane motivation and the six-cluster evidence onto the verified facts; named the =VERBS=/=ACTIONS= dispatch integration and tightened the new verbs to narrowly-scoped commands. Both =[/]= cookies now read complete. +- Why: convergence toward implementation-ready. The redaction modify avoids blanking the network name on the user's own screen while still scrubbing the shareable text — a connection is usually named after its SSID, so leaving it unredacted leaks the already-redacted SSID by a side channel. +- Artifacts: findings =[5/5]=, decisions =[3/3]=. Scope expanded only by the redaction obligation, which is filed as an acceptance criterion and a Phase 1 deliverable, so the readiness rubric holds. Awaiting re-review to flip DRAFT → READY. + +** 2026-07-11 Sat @ 01:45:00 -0500 — Claude Code (archsetup) — Reviewer + Responder (round 2) +- What: a skeptical adversarial re-review returned Not ready with two blockers, both in the parts round 1 claimed to resolve. (1) The redaction "copy/--json surface" I asserted in round 1 does not exist — SSID redaction is event-log-only, =--json= is raw, the copyable report scrubs MAC/IP only. (2) =rival-manager= marked =needs-user-action= could never run its fix, since the doctor only repairs =fixable= outcomes (=doctor.py:181=). Fixed both, plus two non-blocking corrections (auth signal repointed at =manage.py:36-47= + =nmcli.py:164=; masked-NM early-return path). Findings now =[9/9]=. +- Why: the first round introduced a factual error by asserting a redaction surface without verifying =redact.py='s structure. Round 2 verified every claim against the engine before writing. The redaction resolution is now a de-scope: v1 keeps parity with the pre-existing link-step behavior and the systemic redaction gap is a separate task — a scope call worth Craig's eye. +- Artifacts: engine re-verified at =doctor.py:181= (fix gate), =cli.py= (raw =--json=), =report.py= / =redact.py= (MAC-IP-only scrub), =diag.py:103= (link evidence), =diag.py:503-511= (NM-down early return), =manage.py:36-47= / =nmcli.py:164= (auth signals). Awaiting a third re-review. + +** 2026-07-11 Sat @ 02:00:00 -0500 — Claude Code (archsetup) — Reviewer (round 3) +- What: third skeptical adversarial re-review. Verdict =Ready with caveats=, no blocking findings. Verified all five round-2 resolutions against the engine: redaction parity is coherent and buildable (no criterion assumes unbuilt redaction), rival-manager as =fixable= preserves terminal-first ordering with no loop (=doctor.py:184-195=), the auth signals (=manage.py:46= SAE, =nmcli.py:159-176= hidden/SECURITY) distinguish all three cases, and the masked-NM check is addable in the early-return path (=diag.py:503-511=). Flipped DRAFT → READY. +- Why: the loop terminates at the rubric, not at exhaustion. Two independent skeptical passes plus a code re-verification found no remaining blocker. The one named caveat — Phase 1 needs the shared privilege model or =--fix= runs ungated (=priv.py:150=, =_attempt= ungated) — is honestly recorded in Risks, Decision 3, and now a hard ordering gate on Phase 1; Phase 0 is fully buildable now. +- Artifacts: findings =[9/9]=, decisions =[3/3]=. Non-blocking: keyfile-perms on an inactive profile names a connection the disconnected link step wouldn't — same data class, covered by the separate redaction task. diff --git a/scripts/testing/net-scenarios/10-nm-masked.sh b/scripts/testing/net-scenarios/10-nm-masked.sh new file mode 100644 index 0000000..d3b382b --- /dev/null +++ b/scripts/testing/net-scenarios/10-nm-masked.sh @@ -0,0 +1,29 @@ +# shellcheck shell=bash disable=SC2034 # sourced by run-net-scenarios.sh +# Scenario: a masked NetworkManager is unmasked and started. +# +# Masking NetworkManager makes the generic "isn't running" verdict useless — +# systemctl can't start a masked unit — so the doctor must name the mask and +# unmask first. Break by masking; the fix's chain should unmask then start. +SCENARIO_DESC="masked NetworkManager is unmasked and started" +SCENARIO_GROUP="control-plane" + +scenario_break() { + nexec "systemctl mask NetworkManager" +} + +# The verdict the read-only diagnose should produce before any fix runs. +scenario_diagnose_expect() { + ndoctor_json ".outcome" | grep -qx fixable \ + && ndoctor_json '.report.control_plane.nm_state' | grep -qx masked +} + +scenario_fix() { + nfix # net doctor --fix +} + +scenario_assert() { + # Unmasked (the state flip, testable anywhere) AND active (needs a real + # NetworkManager, so a container without netlink can only show the unmask). + ! nexec "systemctl is-enabled NetworkManager | grep -qx masked" \ + && nexec "systemctl is-active NetworkManager >/dev/null" +} diff --git a/scripts/testing/net-scenarios/20-rival-manager.sh b/scripts/testing/net-scenarios/20-rival-manager.sh new file mode 100644 index 0000000..0d7836b --- /dev/null +++ b/scripts/testing/net-scenarios/20-rival-manager.sh @@ -0,0 +1,27 @@ +# shellcheck shell=bash disable=SC2034 # sourced by run-net-scenarios.sh +# Scenario: a rival network manager active alongside NetworkManager is disabled. +# +# dhcpcd (or systemd-networkd / iwd) running next to NM fights it for the link. +# Break by enabling dhcpcd; the fix disables it, and the chain resets to +# reconnect. Requires dhcpcd installed in the target. +SCENARIO_DESC="an active rival manager (dhcpcd) is disabled" +SCENARIO_GROUP="control-plane" + +scenario_break() { + nexec "systemctl enable --now dhcpcd" +} + +scenario_diagnose_expect() { + # The verdict fires only when the link is also failing; in a fresh target + # with no configured uplink that holds, so the rival should be named. + ndoctor_json ".report.control_plane.rivals | index(\"dhcpcd\")" \ + | grep -vqx null +} + +scenario_fix() { + nfix +} + +scenario_assert() { + ! nexec "systemctl is-active dhcpcd >/dev/null 2>&1" +} diff --git a/scripts/testing/net-scenarios/30-keyfile-perms.sh b/scripts/testing/net-scenarios/30-keyfile-perms.sh new file mode 100644 index 0000000..8149f87 --- /dev/null +++ b/scripts/testing/net-scenarios/30-keyfile-perms.sh @@ -0,0 +1,30 @@ +# shellcheck shell=bash disable=SC2034 # sourced by run-net-scenarios.sh +# Scenario: a profile keyfile with unsafe permissions is set back to 0600 root. +# +# NetworkManager silently ignores a world-readable keyfile, so the profile +# never activates. Break by chmod 0644 on an existing profile's keyfile; the +# fix chmods it back to 0600 root. The doctor must run as root to read the +# keyfile dir, so this scenario runs the doctor with sudo. Requires at least +# one saved profile in the target (NET_SCENARIO_PROFILE names it). +SCENARIO_DESC="an unsafe-perms profile keyfile is restored to 0600 root" +SCENARIO_GROUP="control-plane" + +_keyfile() { + echo "/etc/NetworkManager/system-connections/${NET_SCENARIO_PROFILE:?set NET_SCENARIO_PROFILE to a saved profile name}.nmconnection" +} + +scenario_break() { + nexec "chmod 0644 '$(_keyfile)'" +} + +scenario_diagnose_expect() { + ndoctor_json ".report.control_plane.keyfile.perms_ok" | grep -qx false +} + +scenario_fix() { + nfix +} + +scenario_assert() { + nexec "[ \"\$(stat -c '%a %U' '$(_keyfile)')\" = '600 root' ]" +} diff --git a/scripts/testing/run-net-scenarios.sh b/scripts/testing/run-net-scenarios.sh new file mode 100755 index 0000000..7197e37 --- /dev/null +++ b/scripts/testing/run-net-scenarios.sh @@ -0,0 +1,115 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-3.0-or-later +# Run net-doctor control-plane repair scenarios against a booted VM. +# Author: Craig Jennings <craigmartinjennings@gmail.com> +# License: GNU GPLv3 +# +# Live verification for net Phase 1's three privileged fixes (unmask-nm, +# disable-rival, chmod-keyfile). Each scenario breaks the target in a known +# way, runs the real `net doctor --fix` inside it, and asserts the post-state. +# These states are dangerous on a daily driver (masking NM kills the network), +# so they run against a DISPOSABLE VM you can revert, never ratio/velox. +# +# The target must be a booted VM (a real kernel + network stack — NetworkManager +# does not run reliably in an nspawn container) reachable over ssh as root, with +# NetworkManager and dhcpcd installed. The runner rsyncs the net + panelkit +# source trees in and runs the doctor from them via a small wrapper (no install +# needed), matching the stowed shim. jq is used host-side to read the JSON. +# +# STATUS: first-draft harness. The scenario break/fix/assert logic is the +# reviewed part; the ssh transport plumbing has not been exercised against a +# live target yet and may need a shakeout on the first real run. +# +# Usage: run-net-scenarios.sh --target root@HOST [--list] [--profile NAME] +# --target ssh destination of the booted VM (required unless --list) +# --profile a saved NM profile name for the keyfile scenario +# (else the keyfile scenario is skipped) +# --list print the scenario plan and exit +# +# Reverting the VM to a clean snapshot between runs is the caller's job; each +# scenario also restores what it broke on the way out. + +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +SCENARIO_DIR="${NET_SCENARIO_DIR:-$SCRIPT_DIR/net-scenarios}" +DOTFILES="${DOTFILES:-$HOME/.dotfiles}" +T_NET="/root/net-scenario-tree" # where the source trees land in the target + +TARGET="" +LIST_ONLY=false +export NET_SCENARIO_PROFILE="${NET_SCENARIO_PROFILE:-}" + +while [[ $# -gt 0 ]]; do + case $1 in + --target) TARGET="$2"; shift 2 ;; + --profile) NET_SCENARIO_PROFILE="$2"; shift 2 ;; + --list) LIST_ONLY=true; shift ;; + *) echo "usage: $0 --target root@HOST [--list] [--profile NAME]" >&2; exit 1 ;; + esac +done + +mapfile -t S_FILES < <(find "$SCENARIO_DIR" -maxdepth 1 -name '*.sh' | sort) + +if [ "$LIST_ONLY" = true ]; then + echo "net doctor scenario plan:" + for f in "${S_FILES[@]}"; do + ( . "$f"; printf ' %-24s %s\n' "$(basename "$f" .sh)" "$SCENARIO_DESC" ) + done + exit 0 +fi + +[ -n "$TARGET" ] || { echo "--target root@HOST is required" >&2; exit 1; } +command -v jq >/dev/null || { echo "jq is required host-side" >&2; exit 1; } + +SSH() { ssh -o BatchMode=yes "$TARGET" "$@"; } + +# ── target helpers (available to the scenario scripts) ─────────────────── +# nexec runs a payload in the target as root; the doctor runs through the +# /root/net-run wrapper synced below, so no fragile inline quoting is needed. +nexec() { SSH "bash -lc $(printf '%q' "$*")"; } +nfix() { nexec "/root/net-run doctor --fix --json"; } +ndoctor_json() { nexec "/root/net-run doctor --json" | jq -er "$1"; } + +info() { printf ' %s\n' "$*"; } +pass() { printf ' PASS %s\n' "$*"; } +fail() { printf ' FAIL %s\n' "$*"; } + +# ── sync the source trees + the doctor wrapper into the target ─────────── +echo "==> syncing net + panelkit into $TARGET:$T_NET" +SSH "mkdir -p $T_NET/net $T_NET/panelkit" +rsync -a -e "ssh -o BatchMode=yes" "$DOTFILES/net/src" "$TARGET:$T_NET/net/" +rsync -a -e "ssh -o BatchMode=yes" "$DOTFILES/panelkit/src" "$TARGET:$T_NET/panelkit/" +# The wrapper: run as root, so NET_SUDO empty (commands run directly) and +# PANELKIT_SUDO=sudo (root's `sudo -n true` succeeds -> the model resolves RUN). +SSH "cat > /root/net-run" <<EOF +#!/bin/bash +exec env PYTHONPATH=$T_NET/net/src:$T_NET/panelkit/src NET_SUDO= PANELKIT_SUDO=sudo \\ + python3 -c 'import sys; from net.cli import main; sys.exit(main(sys.argv[1:]))' "\$@" +EOF +SSH "chmod +x /root/net-run" + +# ── run each scenario ──────────────────────────────────────────────────── +fails=0 +for f in "${S_FILES[@]}"; do + name="$(basename "$f" .sh)" + # shellcheck disable=SC1090 + ( . "$f" + case "$name" in + *keyfile*) [ -n "$NET_SCENARIO_PROFILE" ] || { info "skip $name (no --profile)"; exit 0; } ;; + esac + echo "== $name — $SCENARIO_DESC" + scenario_break || { fail "$name: break failed"; exit 1; } + if declare -F scenario_diagnose_expect >/dev/null; then + if scenario_diagnose_expect; then pass "$name: diagnose named it" + else fail "$name: diagnose did NOT name it (inspect net doctor --json)"; fi + fi + scenario_fix || info "$name: net doctor --fix returned non-zero" + if scenario_assert; then pass "$name: repaired" + else fail "$name: NOT repaired"; exit 1; fi + ) || fails=$((fails + 1)) +done + +echo +[ "$fails" -eq 0 ] && echo "all scenarios passed" || echo "$fails scenario(s) failed" +exit "$fails" @@ -21,6 +21,45 @@ The vocabulary is open — topic tags are coined as needed — so these are conv - *Effort / autonomy*: =:quick:= a spare-moment fix (minutes, not a sitting); =:solo:= Claude can carry it end to end — there's a build path, a test path, and no upfront decision needed (a leftover manual spot-check doesn't disqualify it). - *Topic / area* (open): the subsystem a task touches — e.g. =:hyprland:= =:waybar:= =:mpd:= =:music:= =:network:= =:tooling:= =:llm:= =:eask:= =:pocketbook:= =:cmail:=. Coin a new one when it aids filtering. * Archsetup Open Work +** TODO [#C] Net diagnostics leak connection names + SSIDs into copyable report and --json :bug:dotfiles:network:solo: +The net doctor's copyable report (=report.py=, =scrub_text=) scrubs only MAC/IP, and =net diag/doctor --json= (=cli.py=) dumps the raw dict with no redaction. SSID redaction lives only in the event log (=redact_event=, gated on =redact_ssid=, default off). So a connection name (usually the SSID) appears in the clear in the link-step evidence and in every =--json= consumer — the copyable report is exactly the text a user pastes into a bug report. Secrets (PSK/password/token/portal URL) are already stripped, so this is names, not credentials: Minor severity, graded on severity alone per the privacy carve-out. + +Split out of the 2026-07-11 net-doctor-expansion spec review: that spec's new rival-manager/keyfile-perms verdicts keep parity with this pre-existing behavior rather than half-solve it. Fix shape: extend redaction to cover the connection name + keyfile basename across the copyable report and =--json= (one systemic pass, not per-verdict), with a redaction test. Engine-wide, so it wants one coherent change rather than being bolted onto the expansion work. + +** TODO [#B] Net doctor expansion v1 — build the READY spec :feature:dotfiles:network: +:PROPERTIES: +:SPEC_ID: ce29b103-ed9d-4f56-bf8c-9ed8fe680ff3 +:END: +Build the [[file:docs/specs/2026-07-11-net-doctor-expansion-spec.org][net doctor expansion]] (DOING). Adds the control-plane cluster (rival-manager / nm-masked / keyfile-perms) and a sharper auth verdict to the shipped net doctor (=~/.dotfiles/net/=). Archsetup owns the dotfiles work end to end — edit, test, commit, and push in =~/.dotfiles=, then drop an inbox note. Phases 1-2 are gated on the shared cross-panel run-time privilege model, which doesn't exist yet; Phase 0 builds now. +*** 2026-07-11 Sat @ 02:47:47 -0500 Built the read-only control-plane probe +New module =net/src/net/control_plane.py= plus =diag.py= wiring, on dotfiles main (=21ca3ff=, pushed). =net diagnose= now carries a =control_plane= key in its envelope with three read-only signals: NetworkManager state (masked/failed/stopped/active, finer than the plain =is-active= the ladder used), any rival manager active alongside NM (dhcpcd, systemd-networkd, iwd via =systemctl is-active=), and the active profile's keyfile permissions (=stat= under an env-overridable =NET_NM_CONNECTIONS= root, default =/etc/NetworkManager/system-connections=). Detection only: no classifier change, no new step, no bearing on =overall=. Surfaced via =net diag --json=. The masked/failed read also runs in the nmcli-down early-return path, or a masked NM would short-circuit before the probe. Every read bounded through =cmd.run='s timeout; degrades to unknown / no-rival / None on an unreadable tool or unstattable file, so a probe that can't see never invents a fault. As a normal user the real 0700 root:root dir is unstattable, so the keyfile signal reads unknown (readable only under root, e.g. the doctor's privileged path). 21 new tests, full =make test= green; =/review-code= approved (two Minor items, both Phase 1 concerns already tracked in the spec: link-scoping the rivals and guarding the keyfile path in the chmod fix). Inbox note sent to dotfiles. +*** DOING [#C] Phase 1 — control-plane verdicts + privilege model +Unblocked now that panelkit ships. =classify= gains =rival-manager=/=nm-masked=/=keyfile-perms= (all =fixable=), ordered ahead of the generic not-running rule, with the masked check reachable in the NM-down early-return path. Fixes register as =VERBS= + =ACTIONS= (=disable-rival=/=unmask-nm=/=chmod-keyfile=), each narrowly scoped. Live verification needs a real rival/masked/bad-keyfile state (a VM or a deliberately-broken host), so not solo. +**** 2026-07-11 Sat @ 04:32:06 -0500 nm-masked verdict shipped + first panelkit integration +On dotfiles main (=d73eba6=, pushed). The clean, unambiguous verdict of the three, done first to prove the panelkit-in-net rails: =classify= reads =control_plane.nm_state=; on =masked= it returns a fixable =unmask-nm= verdict (=remedy_class="privileged"=) ahead of the generic nm-service rule (=nm-restart= can't start a masked unit). New priv verb =unmask-nm= (=systemctl unmask NetworkManager=), =repair_unmask_nm= (unmask → start → verify), narration entry. The doctor resolves a privileged verdict through =panelkit.resolve()= before running: can't-elevate (GUI, no passwordless, no tty) degrades to the manual guide instead of attempting; existing tiers (no remedy_class) run unchanged. The =net= shim + test add =panelkit/src= to sys.path. Design call shipped (flagged for Craig): CLI =net doctor --fix= is the deliberate act satisfying the Confirm floor, so no CLI prompt added; panelkit contributes the run/prompt/guide degradation on the CLI; the GUI arm-press stays its confirm (panel wiring pending). 765 net tests + 65 suites green, review-code clean. Inbox note sent. +**** 2026-07-11 Sat @ 05:16:24 -0500 rival-manager + keyfile-perms verdicts shipped +On dotfiles main (=c9f8604=, pushed). Craig approved proceeding with the default precedence. =classify= adds =disable-rival= (rival dhcpcd/networkd/iwd active + a failing link) and =chmod-keyfile= (bad-perms keyfile + a failing link), both =remedy_class="privileged"= through the same panelkit gate. PRECEDENCE settled: both gate on a local-link symptom (never nag next to a working link), sit behind rfkill/service (more fundamental) and ahead of the generic reset (naming the cause beats a blind reconnect); tested (rfkill beats rival, rival beats reset). Priv verbs =disable-rival= (three-unit allowlist), =chmod-keyfile=/=chown-keyfile= (paths validated under system-connections, no traversal). Repairs re-derive their target (=active_rivals= / the active connection). =fake-systemctl= gained a state dir so a disable flips the re-check. 782 net tests + 65 suites green, review-code clean. Two coverage edges deferred to the VM: the keyfile fix's root-owned pass (only reproduces under root), and whether disable-rival needs a follow-on reset to restore the link. All three Phase 1 verdicts now fake-complete; VM live-verification is the remaining piece (see the sub-task). +**** TODO Net Phase 1 live verification — run the scenario harness against a VM +The harness is BUILT (archsetup =a364c1e=): =scripts/testing/net-scenarios/{10-nm-masked,20-rival-manager,30-keyfile-perms}.sh= (break/fix/assert + the expected diagnose verdict) and =run-net-scenarios.sh= (rsyncs net+panelkit into a target over ssh, drives the scenarios). Scenario logic reviewed; the ssh transport plumbing is UNEXERCISED (marked first-draft) and wants a shakeout on the first real run. What remains needs Craig: a booted VM (a real kernel + network stack — NM does NOT run reliably in nspawn, so a container only shows the mask-symlink/keyfile-mode half, not "NM active") with NetworkManager + dhcpcd installed, reachable over ssh as root. Then =run-net-scenarios.sh --target root@HOST --profile <saved-profile>=. The two live questions: disable-rival was already chained to a follow-on reset (=be15a81=), so the run confirms whether the reset fires or NM auto-recovers; and the keyfile root-owned pass (only reproduces under root). The by-hand equivalent is the "Manual testing and validation" → "Net doctor privileged fixes" checklist. +*** TODO [#C] Phase 2 — sharpened auth verdict +Auth-reason classifier reading profile key-mgmt (=manage.py=) + scanned SECURITY (=nmcli.py=): SAE / hidden / enterprise / generic. SAE and hidden become =fixable= profile-modify (Confirm-tier); the rest stay =needs-user-action= with a cause-named message. Pairwise over (reason × profile-state). The classify logic is agent-buildable with fakes; the live profile-modify fix needs a real WPA3/hidden network to verify, so that part is manual. +*** TODO [#C] Phase 3 — flip the net spec to IMPLEMENTED +On v1 completion: flip the spec's status heading to IMPLEMENTED (dated history line + mirror) and log the vNext items (flaky/drops cluster, DoT/DNSSEC verdict, profile hygiene) to todo.org. + +** TODO [#B] Bt doctor expansion v1 — build the READY spec :feature:dotfiles:bluetooth: +:PROPERTIES: +:SPEC_ID: 3d4d61c4-e5df-44e9-b8e0-40b31452c3f7 +:END: +Build the [[file:docs/specs/2026-07-11-bt-doctor-expansion-spec.org][bt doctor expansion]] (DOING). Adds a dmesg firmware-hint probe (names the missing blob on a no-adapter fault) and a boot-enablement probe (catches an adapter disabled at boot) to the shipped bt doctor (=~/.dotfiles/bluetooth/=). Phase 2 is gated on the shared cross-panel privilege model; Phases 0-1 build now. Archsetup owns the dotfiles work end to end. +*** 2026-07-11 Sat @ 03:06:32 -0500 Built the two read-only probes +New module =bluetooth/src/bt/probes.py= plus =doctor.py= wiring, on dotfiles main (=d19fdca=, pushed). Two reads the diagnose chain never did: =firmware_hint()= scans the current boot's kernel log for per-vendor firmware-load failures (Intel ibt-*.sfi, MediaTek BT_RAM_CODE, Realtek rtl_bt, Broadcom .hcd, Qualcomm QCA), returning the named blob via a bounded =cmd.run(journalctl -k)= that reuses the =doctor.py:84= precedent; =boot_enablement()= reads three boot-persistence signals (bluez AutoEnable from main.conf [Policy], =systemctl is-enabled bluetooth=, whether TLP lists bluetooth in =DEVICES_TO_DISABLE_ON_STARTUP=). =diagnose()= gates the firmware read to the no-adapter branch and the boot read to the soft-blocked/powered-off branch, so a healthy run reads neither; the raw signals ride a new =probes= key that =doctor()= carries into =--json=. Detection only: no verdict, formatter, or repair change (that's Phase 1). Every read degrades to None on an unreadable tool/file, so a probe that can't see never invents a fault. AutoEnable absent/unset reads None, not false, matching bluez's compiled default of true, so only an explicit =AutoEnable=false= is the fault. New env roots for tests (=BT_MAIN_CONF=, =BT_TLP_CONF=, defaulting to absent temp paths in the Sandbox base so no test reads real /etc); =fake-journalctl= branches on =-k=, =fake-systemctl= answers =is-enabled bluetooth=. 125 bt tests, full =make test= green; =/review-code= approved (no Critical/Important; one Minor noting the firmware read also covers the btctl-unavailable branch, harmless). Inbox note sent to dotfiles. +*** 2026-07-11 Sat @ 03:16:59 -0500 Built the firmware-hint Guide verdict +On dotfiles main (=f05a9b4=, pushed). The no-adapter step now names the blob: a new =_no_adapter_step= consults =probes.firmware_hint()= on a genuine no-adapter fault and, on a per-vendor signature match, sets =evidence= to "no Bluetooth adapter found — <Vendor> firmware <blob> failed to load" and =next_action= to "update linux-firmware and reboot", tagged with a new =code="no-adapter-firmware"= so a =--json= consumer can branch without string-matching. A clean log keeps the generic hardware/driver verdict. A Guide, not a repair: the step carries no =repair= action, so =--fix= never touches it, and it needs no privilege model (so Phase 1 lands independently of the shared cross-panel model). A missing =bluetoothctl= (=BtctlError=) short-circuits before the firmware read, so the verdict fires only on a real no-adapter fault, not a broken install — this also tightened Phase 0 (which read the log on both None branches) to the genuine no-adapter case. =_mk= gained a uniform =code= key (default None) added to every diagnose step, mirroring the existing =repair= key; no test asserts an exact step key-set, verified. =format_doctor_human= already renders =evidence=/=next_action=, so no formatter change. 133 bt tests (+8), full =make test= green; =/review-code= clean. Inbox note sent to dotfiles. +*** TODO [#C] Phase 2 — persistent-power verdict + fix +Blocked on the shared cross-panel privilege model. =powered-off-persistent= distinct from =powered-off=; the fix sets AutoEnable / enables the service / edits tlp, Privileged/Confirm. Needs an idempotent INI setter for main.conf ([Policy], preserve comments) and a list-edit for tlp.conf's =DEVICES_TO_DISABLE_ON_STARTUP= — real work, not argv one-liners. Hard gate: no Privileged fix before the Confirm floor. Persistence only proves out across a real reboot, so verification is manual. +*** TODO [#C] Phase 3 — flip the bt spec to IMPLEMENTED +On v1 completion: flip the spec's status heading to IMPLEMENTED (dated history line + mirror) and log the vNext items (stale-bond signature, connection-parameter hints, bt-audio-profile expansion) to todo.org. + ** TODO [#D] Maintenance console vNext :feature: Deferred from the [[file:docs/specs/2026-07-07-maintenance-console-spec.org][spec]] (v1 ships determinate remedies only): AI/workflow assistance on read-only metrics (the vLater tier — failed-unit diagnosis, journal-error fix suggestions, OOM investigation); SIGKILL escalation for TERM-survivors on the KILL lever; live streaming meters on evidence rows where a count could be a level. @@ -73,6 +112,16 @@ Copy text is per panel but one rule: it pastes as that panel's CLI prints, so th Consistent with the 2026-07-07 scope note on the sibling task above: net's compact glyph overlay is what standardizes, not maint's wide COPY-key header row. +** DOING [#B] Run-time privilege model, standard across every panel doctor :feature:dotfiles: +The audio input/output doctor is gaining a run-time privilege model (see [[file:docs/specs/2026-07-10-audio-doctor-input-side-spec.org][docs/specs/2026-07-10-audio-doctor-input-side-spec.org]], decision "The doctor may use sudo, resolved by context at run time"). Craig's call, 2026-07-10: make it a standard, "revise the other panels to be consistent with these changes." + +The model: a doctor resolves its privilege at run time from three signals — passwordless sudo available (=sudo -n true=, which never hangs), a tty to prompt at, and whether it is the GUI panel. Four remedy classes: Auto (user-scope, reversible), Privileged (needs sudo — runs where passwordless, prompts on a CLI tty, degrades to Guide in a GUI with neither), Reboot-tail (run the applicable part, then instruct the reboot), and Guide (physical/BIOS/wait-for-upstream, nothing to run). Safety floor: every Privileged and Reboot-tail remedy defaults to Confirm or Arm tier, never silent Auto, because passwordless sudo is not consequence-free. + +The shared helper is built (see the dated entry below). What remains is per-panel adoption: wire each doctor's remedies through =panelkit.privmodel.resolve()= and audit them against the Confirm/Arm floor, and reconcile maint's =priv.py= build/fire table with the model rather than leaving its implicit always-passwordless assumption. That wiring lives in the per-panel fix phases (net Phase 1, bt Phase 2, audio input/output), each needing a real privileged host to verify =--fix= end to end, so none is agent-solo. + +*** 2026-07-11 Sat @ 03:48:57 -0500 Built the shared privilege model (panelkit) +On dotfiles main (=b987820=, pushed). Craig chose to share a library, not per-panel copies. New =panelkit= package (=panelkit/src/panelkit/=): pure library code in the dotfiles tree, no CLI, no stow package — a consumer reaches it by adding =panelkit/src= to =sys.path= alongside its own =<panel>/src= and =from panelkit import privmodel=. Self-contained (its own =cmd.py=) so a panel plus panelkit stays a coherent unit (keeps the pluggable-panel option open). =privmodel=: four classes (=auto=/=privileged=/=reboot-tail=/=guide=), =resolve(remedy_class, ctx)= → a =Resolution= (=RUN=/=PROMPT=/=GUIDE_ONLY=, =confirm=, =reboot_after=, =reason=) over a =PrivContext= (passwordless via =sudo -n true=, tty, is-GUI). Safety floor enforced and property-tested across all 8 context combos: every privileged/reboot-tail remedy that runs requires a deliberate confirmation, never silent auto. =PANELKIT_SUDO= test seam (="true"=/="false"=), =detect_context()= + =plan()= convenience. 19 tests, full =make test= green (65 suites), review-code clean. Inbox note sent to dotfiles. NOT wired to a consumer — that's the per-panel adoption above. + ** TODO [#B] Scrolling/Carousel layout: frame fit + wrap-around :hyprland: :PROPERTIES: @@ -637,6 +686,38 @@ Craig's standing checklist of everything that isn't agent-verifiable. Each child Priority and type tag added by that audit: the task carried neither, which kept the project's largest live container out of the agenda entirely. +*** Net doctor privileged fixes (net Phase 1) — run in a disposable VM/container +What we're verifying: the three control-plane fixes actually repair a real broken state under real sudo. These states are dangerous on a daily driver (masking NM kills the network), so run them in a throwaway VM or booted nspawn with NetworkManager installed and passwordless sudo, NOT on ratio/velox. Each: break the state, run =net doctor --fix=, confirm the repair. +**** nm-masked → unmask-nm +What we're verifying: a masked NetworkManager is unmasked and started, not met with a doomed nm-restart. +#+begin_src sh :results output +sudo systemctl mask NetworkManager +net doctor --json | python3 -c 'import sys,json; d=json.load(sys.stdin); print(d["outcome"], d.get("next_action"))' +#+end_src +- Expected (diagnose): the verdict names the masked NM (outcome fixable, action unmask-nm), not the generic "NetworkManager isn't running". +#+begin_src sh :results output +net doctor --fix --json | python3 -c 'import sys,json; d=json.load(sys.stdin); print("fixed=",d["fixed"],"attempts=",[a["id"] for a in d["attempts"]])' +systemctl is-enabled NetworkManager; systemctl is-active NetworkManager +#+end_src +Expected: the fix attempts unmask-nm, NetworkManager is no longer masked (is-enabled != masked) and is active. +**** rival-manager → disable-rival +What we're verifying: a rival manager active alongside NM is disabled, and the doctor names it rather than blindly resetting. +#+begin_src sh :results output +sudo systemctl enable --now dhcpcd # a rival that fights NM for the link +net doctor --fix --json | python3 -c 'import sys,json; d=json.load(sys.stdin); print(d["outcome"], [a["id"] for a in d["attempts"]])' +systemctl is-active dhcpcd +#+end_src +Expected: the verdict/fix is disable-rival, and dhcpcd reads inactive afterward. Note whether the link comes back on its own or needs a follow-up =net doctor --fix= (the un-chained-reset question — if it needs the reset, chain "disable-rival": ["disable-rival","reset"] in FIX_CHAINS). +**** keyfile-perms → chmod-keyfile +What we're verifying: a profile keyfile with unsafe perms is set back to 0600 root so NM stops ignoring it. Needs root (the system-connections dir is 0700 root:root), so run the doctor as root or via sudo. +- Pick an existing profile name (its keyfile is /etc/NetworkManager/system-connections/<name>.nmconnection). +#+begin_src sh :results output +sudo chmod 0644 /etc/NetworkManager/system-connections/<PROFILE>.nmconnection +sudo net doctor --fix --json | python3 -c 'import sys,json; d=json.load(sys.stdin); print(d["outcome"], [a["id"] for a in d["attempts"]])' +sudo stat -c '%a %U' /etc/NetworkManager/system-connections/<PROFILE>.nmconnection +#+end_src +Expected: the verdict/fix is chmod-keyfile and the keyfile reads =600 root= afterward. + *** Maintenance console — in-person checklist Re-homed here by the 2026-07-09 audit: this was a second top-level "Manual testing and validation" parent. One parent, per verification.md. Priority and the :test: tag now live on the parent. Promoted from the build parent when it closed 2026-07-08 — Craig's checklist, runs once in person. Collects everything not agent-verifiable; populate per verification.md as phases land. Known so far: panel look-and-feel vs the E5 prototype (Craig's eyeball); arm-press wording reads right at the moment of use; results-wall readability during a real doctor run; a real UPDATE through the armed guard (and the TTY path once); REBOOT offer after an update; velox in-person glyph check (battery %, charging glyph, low-charge red); SET 80% charge limit sticks across a charge cycle; KILL on a real memory hog. @@ -1160,6 +1241,21 @@ Live verification on ratio replaced the planned VM run for remedies 1, 4 and 5, Left open, not a v1 gap: mpv played silently while the doctor read healthy. The stack was genuinely fine, and per-application stream routing is an explicit spec Non-Goal. Worth a task only if it recurs. +Second sighting, 2026-07-10: Chrome stopped recognizing the microphone while the stack was healthy (Shure MV7+ default, unmuted, 82%, PTT off). Restarting Chrome fixed it; nothing on the machine was touched. Both sightings share a shape the doctor cannot see: the graph is fine and one client cannot use it. A third sighting turns this into a design question, namely whether the doctor should say "the stack is fine, the fault is in the application" rather than a bare healthy. + +** TODO [#B] The audio doctor never checks the microphone :bug:audio:solo: +The classifier is output-only. =diag.probe_semantic= already collects =default_source= and =default_source_present=, and =classify.py= reads neither: the word "source" appears once in the whole module, in the graph row that counts them. So a muted mic, a default source naming an unplugged device, or a mic at zero volume all classify as =healthy=, and the verdict prints "the default output is present and audible" while the input side goes unexamined. Found 2026-07-10 while asking whether the doctor would have caught Chrome losing the mic. It would not have. + +Not a scope decision. The spec's Non-Goals never say the doctor is output-only, and its Summary calls it a doctor for "a broken sound stack". + +Work: mirror the sink rules onto the source. =probe_semantic= gains =default_source_muted= / =default_source_volume=; the classifier gains the source cases; =classify.findings()= gains an input row beside its =defaults= row, so the CLI wall and the GUI wall both grow it for free. + +Two things not to get wrong. An absent microphone is legitimate on a desktop, so "no input devices" must never be a fault the way =no-output-devices= is. And a monitor source is a legitimate default source (recording desktop audio), which is why =probe_semantic= passes =include_monitors=True= — inheriting the panel's display filter here would call a working setup broken. + +Specced 2026-07-10 after discussion with Craig, and the design grew past the original gap: [[file:docs/specs/2026-07-10-audio-doctor-input-side-spec.org][docs/specs/2026-07-10-audio-doctor-input-side-spec.org]] (DRAFT, four decisions open). A doctor key per direction, a kernel-level capture probe below PipeWire, PTT-aware muting, and a direction-aware guard. The precedence question the build would have faced is gone: a doctor per direction means the user's press says which side they came to fix. + +Parent spec: [[file:docs/specs/2026-07-09-audio-doctor-spec.org][docs/specs/2026-07-09-audio-doctor-spec.org]] (IMPLEMENTED). This is a v1 gap found after the fact, not a phase of it. + ** TODO [#D] Telega coredump recurrence tell :bug:maint: :PROPERTIES: :LAST_REVIEWED: 2026-07-09 |