diff options
Diffstat (limited to 'docs/design/2026-07-10-net-bt-failure-taxonomy.org')
| -rw-r--r-- | docs/design/2026-07-10-net-bt-failure-taxonomy.org | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/docs/design/2026-07-10-net-bt-failure-taxonomy.org b/docs/design/2026-07-10-net-bt-failure-taxonomy.org index 74790c6..4d57b86 100644 --- a/docs/design/2026-07-10-net-bt-failure-taxonomy.org +++ b/docs/design/2026-07-10-net-bt-failure-taxonomy.org @@ -96,6 +96,7 @@ Six layers, mirroring the net doctor's probe ladder (link → IP/DHCP → gatewa - Another daemon overwrites resolv.conf (yes). DNS works then breaks (or breaks after VPN up/down) as dhcpcd/openvpn/openresolv rewrites resolv.conf. Multiple tools claim it with no coordination. Fix: pick one manager (openresolv =resolvconf=NO=, dhcpcd =nohook resolv.conf=), point resolv.conf at the stub, restart resolved. [[https://github.com/adrienverge/openfortivpn/issues/674][openfortivpn 674]] - nsswitch.conf hosts line broken (yes). All resolution fails, or LAN/mDNS names never resolve; the hosts line lacks =resolve=/=dns= in the right order or references an uninstalled nss module. Fix: set =hosts: mymachines resolve [!UNAVAIL=return] files myhostname dns=. [[https://man.archlinux.org/man/nss-resolve.8.en][nss-resolve]] - Avahi/.local mDNS not resolving (yes). *.local names don't resolve though unicast DNS works. nss-mdns not wired in, or resolved's built-in mDNS collides with avahi. Fix: install nss-mdns, add =mdns_minimal [NOTFOUND=return]= before =resolve=, enable avahi-daemon, disable resolved MulticastDNS if both run. [[https://wiki.archlinux.org/title/Avahi][archwiki avahi]] +- Clock skew breaks DNS itself, and NTP cannot recover it (yes; field-observed 2026-08-19, velox, not from the 2026-07-10 sweep). Nothing resolves at all — not a slow lookup, a dead one — after a boot with a wrong clock. =DNSSEC=yes= validates RRSIG inception/expiry windows against the wall clock, so a clock weeks off fails every query before it leaves the machine. Measured on velox 2026-08-19 with the clock wound back 27 days: resolved logged =signature-expired= against the root DNSKEY and every DS beneath it, and resolution died outright. =DNSOverTLS=yes= is *not* what bites, despite being the obvious suspect — the DoT handshake to =1.1.1.1:853= verified clean at that same clock, because a resolver certificate is good for about a year while an RRSIG window is days to weeks. A skew large enough to break DNSSEC normally leaves the certificate valid. The trap is the recovery path: NTP daemons name their servers by hostname (=pool 2.arch.pool.ntp.org=, =NTP=time.cloudflare.com=), so the daemon that would fix the clock needs the DNS that the clock is breaking. Neither side moves and the machine cannot self-heal — diagnosis needs a second device. Distinguish from the plain clock-skew entry in the egress layer by where it bites: that one has working DNS and failing HTTPS, this one has no DNS at all. Confirm with =dig @1.1.1.1 example.com +short=, which goes out plain UDP/53 and bypasses resolved entirely; an answer there with resolved still failing puts the fault in the validation layer, not the network. Fix: set the clock by hand (=timedatectl set-time=), then =resolvectl flush-caches=. =DNSSEC=allow-downgrade= does *not* help here, which is worth knowing because it is the obvious reach: resolved downgrades when a server lacks DNSSEC support, and a signature-window failure is a validation failure rather than a support failure, so no downgrade fires. Measured on velox: six retries over eighteen seconds, plus =resolvectl reset-server-features=, all dead. The only cure is correcting the clock, which is why the NTP source has to be reachable without DNS. Prevent by giving the NTP daemon at least one source addressed by IP, which needs neither DNS nor a certificate — =server 162.159.200.1 iburst= in a chrony drop-in. Note =timedatectl set-ntp true= is *not* a fix here: it starts a daemon that still cannot resolve its pool. ** Egress / captive portal / MTU / proxy / clock / upstream @@ -107,7 +108,7 @@ Six layers, mirroring the net doctor's probe ladder (link → IP/DHCP → gatewa - PPPoE / VPN link with a lower MTU not clamped (no). Browsing works but big transfers / some HTTPS hang. A PPPoE (1492) or VPN path has a smaller MTU and the too-large segments get dropped. Fix: set the tunnel/link MTU down (=.mtu 1420= for VPN, 1492 for PPPoE) or MSS-clamp on the gateway. [[https://thelineman.ca/articles/article-8-mtu-vpn-mss][vpn mtu/mss]] - Stale http_proxy env var points at a dead proxy (no). Every curl/wget/pacman fails though the network is fine; browsers may work. A leftover =http_proxy= points at an offline/off-network proxy. Fix: unset the vars, remove the export from =~/.profile= / =/etc/environment=. [[https://everything.curl.dev/usingcurl/proxies/env.html][curl proxy env]] - Unreachable PAC file off the corporate network hangs everything (no). Away from the office the browser stalls with no error. A system proxy set to "automatic" with a PAC URL that only resolves on the corporate LAN blocks waiting instead of falling back to DIRECT. Fix: switch system proxy to None (=gsettings … org.gnome.system.proxy mode 'none'=) or clear the PAC URL. [[https://bugzilla.mozilla.org/show_bug.cgi?id=1121800][ff pac hang]] -- Clock skew breaks every TLS handshake (yes). "Your connection is not private" on every HTTPS site though ping/DNS work; the clock is hours/years off. A dual-boot Windows RTC-localtime, unsynced NTP, or a dead CMOS battery leaves the clock wrong. Fix: =timedatectl set-ntp true= (=set-local-rtc 0= on dual-boot), replace the CMOS battery if it recurs. [[https://wiki.archlinux.org/title/System_time][archwiki system time]] +- Clock skew breaks every TLS handshake (yes). "Your connection is not private" on every HTTPS site though ping/DNS work; the clock is hours/years off. A dual-boot Windows RTC-localtime, unsynced NTP, or a dead CMOS battery leaves the clock wrong. Fix: =timedatectl set-ntp true= (=set-local-rtc 0= on dual-boot), replace the CMOS battery if it recurs. This entry assumes DNS still works; when the resolver runs DoT or DNSSEC the same skew kills DNS first and =set-ntp true= cannot recover it — see the clock/DNS deadlock in the DNS layer. [[https://wiki.archlinux.org/title/System_time][archwiki system time]] - Firewall default-deny drops all egress (yes). No traffic leaves right after enabling a firewall, or after both ufw and firewalld are on; even DNS fails. A default outgoing-deny policy, or two firewalls fighting over nftables. Fix: allow egress (=ufw default allow outgoing=) and run only one firewall. [[https://wiki.archlinux.org/title/Uncomplicated_Firewall][archwiki ufw]] - VPN kill-switch / leftover iptables rule strangles egress after VPN drops (yes; distinct from the route-capture case). Internet dies the moment the VPN disconnects and never returns until reboot. A kill-switch rule pinned traffic to tun0 and the leftover rule keeps dropping everything on the real interface. Fix: flush the stale rules (=iptables -F; iptables -P OUTPUT ACCEPT=, or restart the firewall), reconnect. [[https://bbs.archlinux.org/viewtopic.php?id=300104][arch ufw killswitch]] - IPv6 egress broken while IPv4 works (no; the egress angle of the broken-v6 family). Pages load slowly/intermittently; IPv4-only hosts are fine. The network advertises IPv6 with no working route and Happy Eyeballs keeps trying the dead AAAA path. Fix: =nmcli con modify <con> ipv6.method disabled= until the network's IPv6 is fixed. [[https://help.ubuntu.com/community/WebBrowsingSlowIPv6IPv4][ubuntu slow ipv6]] @@ -309,6 +310,7 @@ Probe: dns-config + resolver-health + dns-resolve + the doctor's dns-test (which - VPN split-DNS not applied :: AUTO — =resolvectl domain/default-route= on the VPN link. - IPv6 AAAA lookups stall :: AUTO — disable IPv6 on the link (or the single-request option). Also cluster 8. - Another daemon overwrites resolv.conf :: PRIV — pick one manager, point resolv.conf at the stub. +- Clock skew breaks DNSSEC validation, NTP deadlocked behind it :: PRIV — set the clock by hand, flush caches; prevent with an IP-addressed NTP source. The doctor must reach this verdict *before* any resolved restart, which cannot help and reads as a loop. - nsswitch.conf hosts line / avahi mDNS broken :: PRIV — fix the hosts line, install nss-mdns. ** Cluster 6 — names resolve, egress blocked |