diff options
Diffstat (limited to 'todo.org')
| -rw-r--r-- | todo.org | 1275 |
1 files changed, 1026 insertions, 249 deletions
@@ -75,6 +75,260 @@ Distinct from the =VERIFY [#A] Rotate the credentials exposed by the 2026-08-09 dotfiles leak= under the cgit audit — that one covers credentials a crawler already took from a public repo. This one is a local default never changed. Both are rotation work; neither substitutes for the other. +** TODO [#B] Qt apps render oversized on velox :bug:velox:solo: +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +From the roam inbox, Craig's words: "qt apps look huge on velox. how do we make +it look better on this particular machine, and not change ratio. it seems they +should have different QT configs." + +The shape is per-machine Qt scaling. velox is a high-DPI Framework panel and +ratio drives ordinary-DPI monitors, so one global Qt scale factor cannot suit +both. The fix has to be host-scoped rather than a value written into the shared +config, which is the same tier split the dotfiles already use. + +Grading: Minor severity (apps work, they are just the wrong size) x every user +every time (every Qt app launch on velox) = P2 = [#B]. + +*** 2026-08-19 Wed @ 15:05:00 -0700 Root cause found and fixed; needs a logout to take effect +velox's =conf.d/local.conf= scaled the panel twice. The monitor line sets +=1.566667= and the same file exported =QT_SCALE_FACTOR,1.5= and =GDK_SCALE,1.5=, +and Qt 6 on Wayland already takes its scale from the compositor, so the two +multiplied. Measured rather than reasoned: with the override Qt reports a +960x640 logical screen, without it 1440x960, and 2256/1.566667 is exactly 1440. +That is 1.5x too large, which matches "huge" precisely. + +Those env lines were not careless. The comment above them explains they existed +to compensate for =xwayland:force_zero_scaling = true= in the shared +hyprland.conf, which makes XWayland clients render unscaled and tiny. The +approach was what failed: an env var reaches every app, so fixing XWayland broke +every native Wayland client. Removing the vars alone would have traded "Qt huge" +for "Zoom tiny", so velox now turns =force_zero_scaling= off for itself instead. +XWayland scales through the compositor there, coming out correctly sized and +slightly soft. ratio is untouched and needs nothing, its monitor being scale 1. + +=force_zero_scaling= took effect on =hyprctl reload=. The env removal will not: +Hyprland applies =env== lines with setenv at parse time and never unsets them, +so the running compositor still hands 1.5 to everything it spawns. Craig has to +log out and back in. + +*** VERIFY Is CALIBRE_OVERRIDE_DPI still needed after the scaling fix? +The same file pins =CALIBRE_OVERRIDE_DPI,96= with the comment "calibre renders +oversized at the 1.57 compositor scale". Calibre is a Qt app, so that was almost +certainly this same double-scaling seen through one application and worked +around per-app rather than at the root. With the multiplier gone, the pin is +probably redundant and may now render calibre too small. + +Left in place rather than removed on a guess, since it was validated at 96 on +2026-06-27 and calibre has its own DPI handling. Worth opening calibre after the +next login and deciding by eye. + +The cursor entry in the same file records this identical failure a third time: +"Pre-scaling it (the old 36 = 24 x 1.5) double-applied on top of the +compositor's scale." Three instances of one mistake in one file, two previously +fixed in isolation without anyone naming the pattern. + +** TODO [#B] Function keys issue media actions instead of F-keys :bug:velox: +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +From the roam inbox, Craig's words: "function keys should issue F+number +functionality rather than their media functionality when the button is hit. +currently it's reversed and I have to hit function and the f button for F+number +functionality." + +Check first whether this belongs to archsetup at all. On a Framework the Fn-lock +is a firmware-level toggle held in the keyboard itself (Fn+Esc on most +revisions), not something the OS sets, in which case this is one keystroke +rather than a change here. If it is instead a hid/keyboard-module quirk, it is +ours. + +Grading: Minor severity (the keys work, they are on the wrong layer, and there +is a workaround) x every user every time (every F-key press) = P2 = [#B]. + +** TODO [#C] Waybar panels launch expanded instead of collapsed :bug:dotfiles:waybar: +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +From the roam inbox, Craig's words: "waybar panels should start up collapsed. +currently both the left and the right waybar panels launch expanded." + +Panel source is =~/.dotfiles=. Its heading in the roam inbox read "archsetup." +with a period rather than a colon, so the routing prefix did not match cleanly; +claimed on the plain reading of the text. + +Grading: Cosmetic severity (presentation only, nothing is lost) x every user +every time (every session start) = P3 = [#C]. + +** VERIFY [#C] The visible analog clock avoids being dragged :velox: +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +From the roam inbox, captured verbatim: "the visible analog clock avoids being +dragged. ask me about this." + +Filed as a VERIFY because the capture asks for a conversation rather than +describing a defect. What is the clock avoiding being dragged by, and is the +avoidance the bug or the intended behaviour? + +** TODO [#C] A failed hostname lookup takes seven seconds :bug: +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +=getent hosts fake-vm= takes about 7.2 seconds to return not-found on velox. +Measured repeatedly with the cache flushed between runs. Anything that looks up +a name that does not exist pays it: an ssh typo, shell completion, a script +probing for a host. + +Not caused by the DNSSEC change. A/B measured today, cache flushed each time: +7691ms and 7232ms on =allow-downgrade= against 6804ms and 7482ms on =yes=, so +the setting makes no difference and this predates it. The likely shape is the +tailnet search domain (=search tailf3bb8c.ts.net=) being tried first, then the +two DoT upstreams, each with its own timeout, before NXDOMAIN comes back. + +Found because it blew a 20-second timeout in +=tests.net-scenarios.test_run_net_scenarios=, which shells out to ssh a +deliberately-bogus =root@fake-vm=. That suite passes on its own and the failure +did not recur, so the timeout needed this latency plus the DNS disruption from +the clock testing running alongside it. Worth knowing that the suite sits close +enough to the edge for a slow resolver to tip it. + +Grading: Minor severity (nothing behaves wrong, it just waits) x some users +sometimes (every failed lookup, which is occasional rather than constant) = P3 = +[#C]. + +** TODO [#A] Reseat velox input-cover ribbon — phantom power button :bug:velox:hardware: +DEADLINE: <2026-08-26 Wed> +:PROPERTIES: +:CREATED: [2026-08-13 Thu] +:LAST_REVIEWED: 2026-08-13 +:END: +Machine off, lift the input cover (Framework QR-guided procedure, 5 +fasteners), reseat its ribbon connector to the mainboard — disturbed in the +2026-08-13 board swap. Root cause of every "mystery reboot" that day: +chassis flex (flash-drive touch, ethernet bump, lid partially lowered) +fired phantom power-button presses — journalctl -b -1 showed "Power key +pressed short." → orderly logind poweroff, then the glitching button +powered it back on. While in there, reseat the USB expansion cards too — +the flaky slot (two hard resets, one no-enumeration) is likely the same +flex problem. +THIRD SYMPTOM (2026-08-13 evening): touchpad delivers ZERO input events — +15s synchronized libinput debug-events capture while swiping caught +nothing, though i2c enumeration and a driver rebind handshake are clean. +Signature of a dead interrupt line on the same ribbon. Keyboard + power +LED lines work; BT mouse is the interim pointer. +ESCALATED 2026-08-13 21:00: a fourth event killed the machine THROUGH the +shield. Previous boot's journal ends mid-line (tailscaled chatter) with no +shutdown sequence at all — a hard power cut, not logind acting. So the +glitch now reaches the EC/hardware power path, which no software setting +can intercept. The reseat is the only fix, and this is a +lose-work-without-warning failure mode, not an inconvenience. +Interim shield (already live): /etc/systemd/logind.conf.d/powerkey.conf +sets HandlePowerKey=ignore — phantom presses log but do nothing; EC-level +10s hold still force-cuts. Consider keeping it even after the repair. +Verify after reseat: flex the chassis edges + partially lower the lid, then +grep the journal for new "Power key pressed" lines — zero means fixed. +Must be done before the Sunday flight — a phantom press mid-travel with the +shield on is survivable, but the connector should not be trusted at 30,000 +feet on the loose setting. + +*** 2026-08-19 Wed @ 14:40:00 -0700 Retracted: the RTC reset is not this task's, and I should not have filed it here +I attributed the 2026-08-19 network outage to this ribbon earlier today. Craig +pushed back — he reseated it before the trip to get the touchpad working — and +he is right. The evidence does not support the attribution and some of it points +the other way. + +What actually holds. Boot -3 ended at 01:33:18 with no shutdown sequence: no +power-off target, no unmounting. The next boot's kernel line reads =rtc_cmos +00:01: setting system clock to 2025-01-01T00:00:16 UTC=, a firmware default, so +the RTC was reset rather than drifted. No firmware update was applied +(=fwupdmgr get-history= is empty) and the battery is fine. + +What refutes the ribbon. This boot logged *zero* =Power key pressed= events, and +so did the four boots before it. The phantom-press symptom had genuinely stopped +after 08-15, exactly as the 08-16 session recorded. The earlier events logged a +power-key press and an orderly poweroff; this logged neither, which makes it a +different signature, not a worse version of the same one. + +What I got wrong methodologically: I anchored on the most salient open hardware +task and read association as evidence. I even wrote "I can't prove it is the +same connector" and then filed it here anyway, which is the tell. + +Two things I checked and can rule out. There were no OOM kills — the 3,433 +matching lines are a systemd unit named "Periodically re-score Claude Code +processes for the OOM-killer" firing on a timer, not memory pressure, and there +is not a single "Killed process" line. Thermal is clean; the only mentions are +boot-time zone registration at 34C and 45C. + +One real thing the same window did surface, tracked separately: a python3 crash +loop, 251 core dumps in the final ten minutes, SIGABRT with =XFreeThreads= and +=PyEval_RestoreThread= in the trace. It does not explain the RTC, because +software cannot clear it, but it is its own problem. + +The open question that would settle the RTC is for Craig, not the journal: a +long power-button hold on a Framework triggers an EC-level reset that clears the +RTC, which fits a wedged machine being forced off. A 4-second hold would not. + +*** 2026-08-17 Mon @ 19:57:42 -0700 Not done, and the two symptoms now disagree +The reseat did not happen before the flight, and velox is travelling. The +deadline blew past on 08-14. + +The two symptoms have separated, which is worth recording because it changes +what the evidence proves. The phantom presses have stopped: fifteen "Power key +pressed" entries between 08-14 04:29 and 08-15 20:04, then nothing at all +across five boots including today's. The touchpad has not — there is still no +touchpad node under =/dev/input/by-path/=, which is the same dead interrupt +line the body describes. + +So the quiet power button is not evidence the connector reseated itself. The +interrupt line is the symptom that cannot be masked in software, and it is +still dead, so the ribbon is still unseated. The most likely reason the +presses stopped is that the machine has been sitting on hotel surfaces instead +of being carried and flexed. + +The interim shield is still live (=HandlePowerKey=ignore=), and the escalation +note stands: an EC-level glitch cuts power below systemd regardless of it. +*** 2026-08-15 Sat @ 23:05:00 -0500 The reseat did happen, and the touchpad came back — this contradicts the 08-17 read +Recording this because a parallel session concluded on 08-17 that the reseat had +not happened and the touchpad was still dead. Both halves were done and verified +that night, so the two accounts disagree and the disagreement should be visible +rather than silently resolved by whichever session committed last. + +What was done: the input-cover ribbon was reseated first, which fixed the +phantom power button — the 22:09 boot logged zero =Power key pressed= lines +after Craig flexed the chassis, against nine on the boot before. The touchpad +did not change, because the input-cover ribbon is not its connector. The 4-pin +connector beside the printed =TOUCHPAD= label is silkscreened =PIN 1-2 GND / +PIN 3-4 VCC= — pure power, so it cannot carry i2c or an interrupt. Reseating the +ribbon that actually crosses to the mainboard fixed it. + +Measured, not assumed: the touchpad interrupt (=amd_gpio= pin 8) went from 0 +counts across all 24 CPUs to 1795, and =i2c_hid_acpi ... did not ack reset +within 1000 ms= disappeared from the boot log. Craig confirmed the pointer moved. + +*Why the 08-17 probe likely misread it:* it checked for a node under +=/dev/input/by-path/=. i2c-HID touchpads frequently get no =by-path= symlink +even when fully working, so its absence is not evidence of a dead interrupt +line. The falsifiable check is the interrupt count in =/proc/interrupts= while +the pad is being touched, or the reset message in =dmesg=. + +*Left open rather than closed* — velox was refusing ssh at merge time on 08-20, +so the current state could not be re-verified, and a later regression cannot be +ruled out. One second of Craig's time settles it: move the pointer. If it works, +close this; if it does not, the interrupt line went back down and that is new +information. + ** DOING [#A] Velox reinstall — DR test of archangel + archsetup :velox:chore: DEADLINE: <2026-08-15 Sat> :PROPERTIES: @@ -116,6 +370,274 @@ hand-copied key sprawl; a firm RAM carve-out so builds don't fight the ZFS ARC; headless only — desktop-coupled sessions stay on ratio/velox. Build deliberately AFTER the vacation, not before Sunday. Companion idea (cheaper, complementary): put ratio on the UPS. +** TODO [#B] post-rebuild-check: route every probe through one guarded helper :refactor:solo: +:PROPERTIES: +:CREATED: [2026-08-17 Mon] +:LAST_REVIEWED: 2026-08-17 +:END: +The script works and is well tested, but its shape keeps producing the same +bug. Across three review rounds the reviewer found FOUR separate instances of +"the probe failed and the check reported ok", each in a different place: +=systemctl= in check 1, the enablement read in check 2, =find= in check 3, and +=grep= in check 4. A fifth was latent in an unguarded staged write. Every one +was individually fixed, and I only stopped finding more because someone kept +looking. + +That is a design problem rather than four bugs. The script has five +hand-written probes, and each one has to remember to branch on its own exit +status. Nothing enforces it, nothing fails a review that forgets it, and the +failure is invisible because the wrong behaviour is a clean "ok". + +Shape: one helper every probe must go through, which cannot return a value +without an explicit success, so that "I could not read this" is +unrepresentable as "nothing to report". Roughly: + +: probe "<what>" <command...> # sets a value on success, records a finding otherwise + +Then each check consumes the helper's result rather than a raw command +substitution, and a new check written later inherits the discipline instead of +having to re-derive it. Worth pairing with a test that asserts no check can +report ok when its probe exits non-zero, generically, so the fifth instance is +caught by the suite rather than by a reviewer. + +Not urgent: the current version is correct as far as anyone has found, ships +with 58 tests, and proved itself on a genuinely wedged machine. This is +prevention. + +Grading: Minor severity (no known live defect, the risk is future) x +most-users-frequently (every future edit to this script) = P3 = [#C]... except +the failure mode is silent and the script's whole job is catching silent +failures, so a regression here is uniquely undetectable. P2 = [#B]. + +:solo: — the surface is one script and its suite, the refactor is +behaviour-preserving, and the existing 58 tests plus a mutation battery are +the objective check that it stayed so. +** TODO [#A] powerprofilesctl crashes on a loop since ppd was masked :bug:velox:dotfiles: +:PROPERTIES: +:CREATED: [2026-08-17 Mon] +:LAST_REVIEWED: 2026-08-17 +:END: +Something polls power state every 10-30 seconds, and each poll runs +=powerprofilesctl get=, which SIGABRTs. 47 coredumps on velox on 2026-08-17 +alone, the earliest at 08:34, four in one minute while I was watching. + +Cause is the 2026-08-16 fix that masked =power-profiles-daemon= so TLP +survives on laptops. That fix is right and stays. What it did not account for +is the settings module's power backing +(=~/.dotfiles/settings/src/settings/power.py=), which shells out to +=powerprofilesctl=. Against a masked unit the D-Bus activation fails with +=NameHasNoOwner ... unit is masked=, and the caller aborts rather than +degrading. + +Run by hand the same command exits 0 and prints the error, so the abort is +context-dependent and the caller needs finding before the fix is written. +Ratio does not mask ppd, which is why this is velox-only and why it appeared +the day after the masking. + +Costs: journal spam, coredump disk churn, and repeated failed D-Bus +activations on a travelling laptop's battery. It is also the leading suspect +for the wedged user manager filed below. + +Fix shape: =power.py= should treat a masked or unavailable ppd as a +first-class "no profile control here" state rather than an error path, and +the poller should stop retrying a unit it has been told is masked. The +machine-level half is already correct. + +Grading: Major severity (a crash loop burning battery and filling the +journal, silently) x every user every time on any laptop with the TLP fix +applied = P1 = [#A]. + +*** 2026-08-17 Mon @ 19:57:42 -0700 The loop stopped at the reboot; the defect did not +velox rebooted at 16:04 and there have been zero coredumps since, against 47 +in the twelve hours before it. So the loop is not currently burning anything. + +That is not a fix, and the distinction matters for whoever picks this up. +=powerprofilesctl get= still fails exactly as recorded — =NameHasNoOwner ... +unit is masked= — so every precondition for the loop is intact and it returns +whenever the caller next polls. What the reboot cleared is the caller's state, +not the bug. + +Narrowed the search the body asks for: =power.py= is the *only* file in +dotfiles that shells out to =powerprofilesctl= (=SETTINGS_POWERPROFILESCTL=, +line 14), so the caller is inside the settings module rather than waybar or a +timer. Worth knowing that the coredumps are =powerprofilesctl= itself aborting +— it is a python script, which is why they log as =/usr/bin/python3.14= +SIGABRT rather than under its own name. + +Grade unchanged. The matrix inputs did not move: the severity is what happens +while the machine is in that state, and the frequency row is every laptop +carrying the TLP fix. A quiet interval since a reboot is not a frequency +change. +** TODO [#B] velox's systemd --user spins at 96% and cannot resolve unit files :bug:velox: +:PROPERTIES: +:CREATED: [2026-08-17 Mon] +:LAST_REVIEWED: 2026-08-17 +:END: +Live on velox 2026-08-17 from about 10:29. =systemd --user= (pid 2235) sits +in state R at 96% CPU, measured over a 3-second sample rather than taken from +the lifetime average. It stopped logging at 10:29, so its timers appear to +have stopped firing too. + +The split is the diagnostic: =systemctl --user list-units= still returns +instantly, while =is-enabled=, =cat=, =show=, and =list-unit-files= all hang +indefinitely. So the manager answers from its in-memory unit list and wedges +on anything that has to resolve unit files. It is spinning in userspace, not +blocked on I/O (=/proc/2235/wchan= is 0, no syscall pending). + +Remedies tried, neither worked: =systemctl --user daemon-reexec= hangs like +every other unit-file call, and the signal form (=kill -59=, SIGRTMIN+25) +was accepted but changed nothing. The next step is a logout/login or reboot, +which is Craig's call because it closes his running session. I deliberately +did not kill the manager: that would tear down the graphical session and +everything under it. + +Suspected cause is the powerprofilesctl crash loop filed above, whose +repeated activation attempts against a masked unit are the only new load on +this machine. I cannot prove it, and I have to name the other candidate +honestly: my own =post-rebuild-check= runs called =systemctl --user +is-enabled= roughly thirty times per run over several runs, and the wedge +appeared during that window. The crash loop predates those runs by an hour +and a half, which is why it is the leading suspect rather than the certain +one. + +What it costs: unit-file operations are unavailable, user timers appear +stopped, and a core is pinned on a laptop running on battery. + +Grading: Major severity (a pinned core and stopped user timers, invisible +unless you look) x rare edge case (one machine, specific conditions) = P2 = +[#B]... except that this is a live, ongoing drain on a travelling machine +rather than a latent defect, so it takes [#A] until the machine is back to +normal. Re-grade to [#B] once resolved and the question is only prevention. + +*** 2026-08-17 Mon @ 19:57:42 -0700 The reboot cleared it; re-graded [#A] to [#B] as the task instructed +velox rebooted at 16:04. The wedge is gone: =systemctl --user is-enabled +roam-sync.timer= now answers =enabled= in well under a second, where every +unit-file call hung indefinitely before, and =list-timers= shows +calendar-sync, roam-sync and agenda-render-cache all firing on schedule +again. So the remedy the task named — a logout or reboot — was taken and +worked. + +Nothing here was diagnosed further, which means the cause is still unproven +and both candidates in the body stand. What is left is prevention, and the +task's own grading says that is [#B]: the live-drain argument was the only +thing holding it at [#A], and the drain has stopped. Re-graded per that +instruction rather than by a fresh judgment. + +Reproducing it deliberately is the open question, and it is not obviously +worth doing — it costs a wedged session to learn something the crash-loop fix +may make moot. +** TODO [#A] The installer clones my two working repos shallow and read-only :bug:velox: +:PROPERTIES: +:CREATED: [2026-08-17 Mon] +:LAST_REVIEWED: 2026-08-17 +:END: +=archsetup:1432= clones the user's archsetup repo and =archsetup:1445= clones +dotfiles, both with =--depth 1=. Those are not build directories. They are the +two repos I actively develop in, and on velox they came back from the +2026-08-13 rebuild with 7 commits of history each instead of 851. + +Found 2026-08-17, and found the worst way: I ran the credential-file history +check that the GitHub-release task asks for, and it reported all five files +absent from history with a clean exit. The real answer is that this clone +cannot see the history those files live in. A shallow clone does not error on +=git log -- <path>=, it answers "no commits" — so a security question came back +falsely clean, and nothing about the output said otherwise. + +Everything else it breaks is quieter: =git log=, =blame=, =bisect=, and any +archaeology past the boundary. The tree looks completely normal, which is why +this survived four days on the machine. + +The right shape is already in the codebase. =scripts/post-install.sh:42-51= +takes depth as a per-repo argument and defaults to a full clone, so wallpaper +gets =--depth 1= and org does not. The AUR build clones (=archsetup:855=, +=:1673=, =:1677=) are correctly shallow and stay that way. Only the two +user-repo sites change. + +*Second defect, same two lines, found 2026-08-17 while pushing:* the dotfiles +clone could not push at all. =archsetup:245= defaults =dotfiles_repo= to +=https://git.cjennings.net/dotfiles.git=, the public read-only endpoint, so +=git push= returned 403. Ratio uses =git@cjennings.net:dotfiles.git= and +archsetup's own clone uses the matching ssh form, so velox was the odd one out +purely because it was the machine rebuilt by the installer. Repointed velox's +remote and pushed. + +That half needs a decision rather than a fix, which is why this task is no +longer =:solo:=. The https default is *correct for a stranger* installing +archsetup, who has no ssh key on the server, and this repo is being prepared +for public release. It is wrong for my own machines, which need to push. The +override already exists (=DOTFILES_REPO=, documented in +=archsetup.conf.example=), so the question is only where my personal value +lives: a config the personal ISO bakes in, a post-install step, or a detection +that prefers ssh when a key is present. Craig's call. + +*Decided 2026-08-19: the ISO bakes the value, and a check nets the rest.* +=archsetup:240= has the identical default for =archsetup_repo=, so this was +always two repos rather than one. I ruled out detection — archsetup never +restores =~/.ssh=, so key-presence at clone time depends on ordering it +doesn't control, and "any key means ssh" would break a stranger who has an +unrelated one. I ruled out a bare post-install step for the reason this whole +class of bug exists: manual steps don't get run, which is why this sat four +days. So the personal ISO carries =ARCHSETUP_REPO= / =DOTFILES_REPO= in the +ssh form (noted on the secrets/ISO task), and =post-rebuild-check= check 8 +flags any working repo still on the read-only endpoint — covering curl|bash +and stock-ISO installs, which the ISO value cannot reach. + +Repair on a machine already built: =git fetch --unshallow= in each repo, and +=git remote set-url origin git@cjennings.net:<repo>.git= for dotfiles. + +Grading: Major severity (two working repos silently missing their history on +the machine I develop on, and it returns confidently wrong answers to history +questions rather than failing) x every user every time (every fresh install, +both daily drivers) = P1 = [#A]. + +Not :solo:. The depth half is (two lines plus tests in the existing +=tests/installer-steps/= shape, verifiable by asserting the clone command +carries no =--depth= for these two repos). The remote-URL half needs the +decision above, so the task as a whole waits on it. Split it in two if the +depth fix is wanted sooner. +*** 2026-08-19 Wed @ 23:05:00 -0700 Dropped --depth from both user-repo clones +=archsetup:1462= and =:1475= now clone full history; +=tests/installer-steps/test_clone_user_repos.py= covers it with 8 cases, and +one of them asserts the AUR build clones still carry =--depth 1= so the fix +can't be over-applied by a careless repo-wide sed. Both my repos on velox were +already unshallowed by hand last session, so this is prevention rather than +repair. +*** 2026-08-19 Wed @ 23:05:00 -0700 Settled the remote-URL half and netted it +See the decision recorded above. The ISO half is a note on the secrets/ISO +task; the net is =post-rebuild-check= check 8, which ships now. +** TODO [#B] post-rebuild-check needs a reference-host mode :feature:velox:solo: +:PROPERTIES: +:CREATED: [2026-08-17 Mon] +:LAST_REVIEWED: 2026-08-17 +:END: +=scripts/post-rebuild-check= ships and works, but its first live run on velox +2026-08-17 showed the output is mostly steady state rather than drift. Of the +8 findings that survived three rounds of false-positive removal, comparing +against ratio says exactly ONE is real: =obsbot-wb-guard= is enabled on ratio +and merely linked on velox, which is the deliberate deferral recorded +2026-08-16. The other three unit findings (=emacs=, =geoclue-agent=, +=obs-record-watchdog.timer=) are linked on ratio too, and the three =.claude= +absences are absent on ratio too. + +So the signal-to-noise is about 1:7, and the thing that separates them is a +comparison against the other daily driver — the same discipline that kept the +2026-08-16 session honest when check 4 read as nine projects missing +=CLAUDE.md= and ratio turned out to be missing the identical files. + +Shape: =--reference-host <host>= runs the same five checks on the far machine +over tailscale (ssh, read-only) and reports only the *differences*. Findings +present on both machines are steady state and get summarized as a count rather +than listed. Falls back to the current standalone behavior when the reference +host is unreachable, and says so. + +Grading: Minor severity (the tool works and its findings are accurate; they +are just buried) x every use = P3 = [#C]... except that a check nobody reads +is a check that isn't run, which is the failure mode the whole task existed to +close. Most-users-frequently x Major = P2 = [#B]. + +:solo: — the checks exist, the ssh path is proven (the 2026-08-17 session ran +exactly this comparison by hand), and correctness is verifiable locally by +diffing the two reports. ** TODO [#C] screen-lock test suite red on ratio :bug:test:dotfiles: :PROPERTIES: :CREATED: [2026-08-13 Thu] @@ -263,7 +785,7 @@ users sometimes = P3 = [#C]. ** TODO [#B] Land the rescued emacs-wttrin commit :chore:velox: :PROPERTIES: :CREATED: [2026-08-14 Fri] -:LAST_REVIEWED: 2026-08-14 +:LAST_REVIEWED: 2026-08-17 :END: bf0457f "feat: add wttrin-hide-follow-line to hide the wttr.in follow line" (2026-06-24) was the only genuinely unpushed commit anywhere on the old @@ -273,6 +795,17 @@ git bundle before the disk was wiped: To land it: clone emacs-wttrin, =git fetch <bundle> --branches=, review the commit, then push to git@cjennings.net:emacs-wttrin.git. Delete the bundle once it's on the remote. + +*** 2026-08-17 Mon @ 19:57:42 -0700 Re-checked: still unlanded, and the bundle is still the only copy +Cloned the remote bare and asked it for the object directly: =git cat-file -t +bf0457f= returns "Not a valid object name", so the commit has never reached +=git@cjennings.net:emacs-wttrin.git=. Remote =main= is =ee8fdeb=. + +That makes =working/velox-reinstall/wttrin-bf0457f.bundle= the sole surviving +copy of 103 insertions across three files, on one laptop that is travelling. +Worth doing sooner than its =[#B]= suggests for that reason alone, and it also +pins the working directory open — the reinstall task cannot file its artifacts +away while this bundle is still load-bearing. ** TODO [#B] archsetup doesn't clone rulesets :bug:velox: DEADLINE: <2026-08-15 Sat> :PROPERTIES: @@ -334,6 +867,60 @@ archangel+archsetup ISO that's already ~80% built. Two ISO modes: generic don't start the migration until the credentials are rotated. Not started. Not :solo: — repo standup and history rewrite are Craig's calls; promote to a real spec (spec-create) when work resumes. + +*Also bake the push-capable repo URLs into the personal ISO* (decided +2026-08-19). =archsetup:240= and =:245= default =archsetup_repo= and +=dotfiles_repo= to =https://git.cjennings.net/...=, the anonymous read-only +endpoint. That default is right for a stranger installing archsetup — no key on +the server — and wrong for my machines, which have to push: velox came back +from its rebuild unable to push either repo, and I only found out at a 403 four +days later. I decided against detecting an ssh key in the installer, because +archsetup never restores =~/.ssh= (I do that by hand), so key-presence at clone +time depends on ordering the installer doesn't control, and a naive "any key +means ssh" would break a stranger who happens to have one. The override already +exists and is documented — =ARCHSETUP_REPO= / =DOTFILES_REPO= in +=archsetup.conf.example= — so the personal ISO just needs to carry the ssh +form of both, alongside the secrets bundle. The generic ISO keeps the https +default untouched. + +The gap that leaves is a curl|bash or stock-ISO install, which takes the https +default straight back. =post-rebuild-check= check 8 covers that path — it flags +a working repo whose origin is the read-only endpoint — so the ISO value is the +fix and the check is the net under it. +** TODO [#B] Settings toggles reset silently at session start :bug:dotfiles: +:PROPERTIES: +:CREATED: [2026-07-28 Tue] +:LAST_REVIEWED: 2026-07-28 +:END: +Craig, from the roam inbox 2026-07-28: "launching into wayland doesn't honor previous caffeine settings ...or I expect any other settings in the desktop settings module." Captured right after the 08:59 reboot. + +Confirmed, and it generalizes past caffeine. The settings module splits cleanly into two halves, and only one of them persists. + +Persisted, in =~/.config/desktop-settings/state.json= (=store.py= =DEFAULTS=): program slots, idle-tripper stages, wallpaper. These come back correctly. + +Not persisted — every one is derived live from a process or a compositor runtime option, so a session restart resets it to whatever =hyprland.conf= establishes: +- Caffeine — =caffeine_state()= is =pgrep -x hypridle= inverted, and =hyprland.conf:73= runs =exec-once = pkill -x hypridle; hypridle=. So every launch unconditionally starts hypridle, which means caffeine is *always* OFF after login. There is no code path that could restore it ON. +- Auto-dim — =dim_state()= reads =hyprctl getoption decoration:dim_inactive=, a compositor runtime value that resets to the config default on restart. +- Night light — =state()= is =pgrep -x gammastep=; the process dies with the session. +- DND — =dunstctl=; dunst restarts fresh from =exec-once=. +- Power profile / brightness — owned by powerprofilesctl and systemd-backlight, outside this module's scope. + +Verified live 10 minutes after the reboot: hypridle running (caffeine OFF), dim =false=, gammastep not running, dnd =false=, power =balanced=. Every toggle sat at its factory position. + +The failure is silent, which is what makes it bite: nothing tells you the value you set was discarded. That is the mechanism behind the 2026-07-27 lockout, where Craig believed caffeine was on and the screen locked anyway. + +Grading: Major severity (the panel's core promise is holding these values, and the reset is silent and total across all four toggles) x most users frequently (every session start resets them, though it only harms when a deliberate non-default was set) = P2 = [#B]. + +Not :solo: — the fix needs Craig's call on *which* toggles should persist and whether persistence is per-toggle opt-in. Restoring night light at 3pm or caffeine on a laptop are both plausibly wrong, so this is a preference question, not a derivable one. The mechanism itself (extend =store.py= with a =toggles= block, restore on session start) is mechanical once that's settled. + +Related: =[#B] Caffeine state is unreadable on both surfaces= covers display accuracy — whether the surfaces report the truth. This covers whether the value survives at all. Distinct bugs, same subsystem. + +*** Side finding — gammastep loses a startup race and nothing relaunches it +=hyprland.conf:75= runs =exec-once = gammastep=, but no gammastep process is alive. Today's three launch logs tell the story: =gammastep-2026-07-28-090034.log= carries "Wayland connection experienced a fatal error: -1 / Temperature adjustment failed", and the other two are empty. + +Launched by hand afterward it runs fine and survives, so gammastep is not broken — it loses a race against compositor readiness at session start. Nothing relaunches it, so night light is simply off for the whole session, silently. (An earlier read of this said night light "has likely never worked from the config". That was wrong: the failure is a startup race, not a permanent break.) + +Worth its own task — the fix is a readiness wait or a retry around that exec-once, not a persistence change. Filed here for now because it surfaced during this investigation. ** VERIFY [#A] Pre-vacation fix list — morning review SCHEDULED: <2026-08-08 Sat> :PROPERTIES: @@ -373,72 +960,6 @@ items needing your call say so. if you disagree). Found tonight, low priority: the orchestrator sequence pin can't see an added-but-unstubbed call (it caught drops only) — worth a harness hardening pass someday. -** TODO [#B] Encrypted mail-password files are world-writable :bug:security:dotfiles:quick: -:PROPERTIES: -:CREATED: [2026-08-15 Sat] -:LAST_REVIEWED: 2026-08-15 -:END: -=~/.config/.gmailpass.gpg= and =~/.config/.dmailpass.gpg= resolve to mode 777 -in the dotfiles repo, on both daily drivers. Noticed by a .emacs.d session -2026-08-14 while comparing the machines after the velox rebuild (inbox handoff, -PROCESSED). - -The contents are gpg-encrypted, so this is not an exposure of the passwords -themselves — it is that any local process can *overwrite* a credential file -without complaint. Fix the mode in the dotfiles repo so a fresh stow lands it -correctly, not just =chmod= on the two live machines, or the next rebuild -reintroduces it. - -Grading: Minor severity (the contents stay encrypted, so the harm once the -state is entered is tampering rather than disclosure, and it needs local access -already) x every user, every time (the wrong mode ships from the repo, so every -machine has it after every install) = P2 = [#B]. Grading the frequency row on -"how often does something actually overwrite it" would double-count the rarity -the severity band already carries. - -While in there, archsetup owns dotfiles work end to end (notes.org, Craig -2026-07-04): make the edit, test, commit and push dotfiles from here, then drop -a note in =~/.dotfiles/inbox/=. - -** TODO [#B] Desktop settings don't survive a session restart :bug:dotfiles:hyprland: -:PROPERTIES: -:CREATED: [2026-07-28 Tue] -:LAST_REVIEWED: 2026-07-28 -:END: -Craig, from the roam inbox 2026-07-28: "launching into wayland doesn't honor previous caffeine settings ...or I expect any other settings in the desktop settings module." Captured right after the 08:59 reboot. - -Confirmed, and it generalizes past caffeine. The settings module splits cleanly into two halves, and only one of them persists. - -Persisted, in =~/.config/desktop-settings/state.json= (=store.py= =DEFAULTS=): program slots, idle-tripper stages, wallpaper. These come back correctly. - -Not persisted — every one is derived live from a process or a compositor runtime option, so a session restart resets it to whatever =hyprland.conf= establishes: -- Caffeine — =caffeine_state()= is =pgrep -x hypridle= inverted, and =hyprland.conf:73= runs =exec-once = pkill -x hypridle; hypridle=. So every launch unconditionally starts hypridle, which means caffeine is *always* OFF after login. There is no code path that could restore it ON. -- Auto-dim — =dim_state()= reads =hyprctl getoption decoration:dim_inactive=, a compositor runtime value that resets to the config default on restart. -- Night light — =state()= is =pgrep -x gammastep=; the process dies with the session. -- DND — =dunstctl=; dunst restarts fresh from =exec-once=. -- Power profile / brightness — owned by powerprofilesctl and systemd-backlight, outside this module's scope. - -Verified live 10 minutes after the reboot: hypridle running (caffeine OFF), dim =false=, gammastep not running, dnd =false=, power =balanced=. Every toggle sat at its factory position. - -The failure is silent, which is what makes it bite: nothing tells you the value you set was discarded. That is the mechanism behind the 2026-07-27 lockout, where Craig believed caffeine was on and the screen locked anyway. - -Grading: Major severity (the panel's core promise is holding these values, and the reset is silent and total across all four toggles) x most users frequently (every session start resets them, though it only harms when a deliberate non-default was set) = P2 = [#B]. - -Not :solo: — the fix needs Craig's call on *which* toggles should persist and whether persistence is per-toggle opt-in. Restoring night light at 3pm or caffeine on a laptop are both plausibly wrong, so this is a preference question, not a derivable one. The mechanism itself (extend =store.py= with a =toggles= block, restore on session start) is mechanical once that's settled. - -Related: =[#B] Caffeine state is unreadable on both surfaces= covers display accuracy — whether the surfaces report the truth. This covers whether the value survives at all. Distinct bugs, same subsystem. - -Recovered 2026-08-14: the heading was overwritten in =ce28d35= when a new task -was inserted at the top of Open Work, and the headless body then rode the -podman task into Resolved when that one was archived. Restored here. - -*** Side finding — gammastep loses a startup race and nothing relaunches it -=hyprland.conf:75= runs =exec-once = gammastep=, but no gammastep process is alive. Today's three launch logs tell the story: =gammastep-2026-07-28-090034.log= carries "Wayland connection experienced a fatal error: -1 / Temperature adjustment failed", and the other two are empty. - -Launched by hand afterward it runs fine and survives, so gammastep is not broken — it loses a race against compositor readiness at session start. Nothing relaunches it, so night light is simply off for the whole session, silently. (An earlier read of this said night light "has likely never worked from the config". That was wrong: the failure is a startup race, not a permanent break.) - -Worth its own task — the fix is a readiness wait or a retry around that exec-once, not a persistence change. Filed here for now because it surfaced during this investigation. - ** TODO [#C] Re-apply the active program at session start :refactor:dotfiles:hyprland: :PROPERTIES: :CREATED: [2026-07-30 Thu] @@ -576,23 +1097,6 @@ Handoff from home (2026-07-25), originally combining the 2026-06-07 stale-compos 2. On =Upgrade= of =fontconfig=, =freetype2=, or =harfbuzz=, run =/usr/bin/fc-cache -f= after the transaction. The fontconfig 2.17→2.18 cache-format change left stale cache-9 files that crashed Qt6 apps in =FcCharSetHasChar= until the system font cache was rebuilt. Acceptance: hook files are source-controlled and installed by archsetup; package/operation/action fields are asserted from the generated hook text; the reminder is print-only and exits successfully; the font hook runs only after successful matching upgrades and invokes the absolute =fc-cache= path. Validate with the fast installer tests plus a disposable pacman-hook parser/install check when practical. -** TODO [#D] net-scenarios harness times out under back-to-back suite runs :test:tooling: -:PROPERTIES: -:LAST_REVIEWED: 2026-07-24 -:END: -=tests/net-scenarios/test_run_net_scenarios.py= errored on all 5 tests twice during round 12, each time =subprocess.TimeoutExpired= after its 20s budget on =scripts/testing/run-net-scenarios.sh --target root@fake-vm=. Both occurrences were in =make test-unit= runs launched immediately after a previous full run. It then passed 6 runs in a row (3 on a pristine tree, 3 with the round-12 change), and standalone it finishes in 0.09s, so this is not a regression from any code change. - -The harness stubs =ssh=, =rsync= and =jq= onto =PATH=, so nothing should touch the network at all — which is what makes a 20s timeout suspicious rather than merely slow. Worth reproducing under load before deciding whether the fix is a larger timeout or a real hang in the script. Evidence logs from the round: =/tmp/tu.log= and =/tmp/tu2.log= (tmpfs, gone after reboot). - -Not graded on the bug matrix: it is test infrastructure, not the shipped codebase. - -*** 2026-08-08 Sat @ 05:05:00 -0500 Recurred under concurrent load, same signature -All 5 tests hit the 20s TimeoutExpired again during a =make test-unit= run -that overlapped two review subagents running their own suites on the box. -Standalone immediately after: 0.095s, all pass; the following quiet-machine -full run was clean. Confirms the load-sensitivity read — reproduce under -deliberate load before choosing between a bigger budget and a real hang. - ** VERIFY Should coredump entries group as one journal-digest row per binary? :maint: :PROPERTIES: :LAST_REVIEWED: 2026-07-24 @@ -632,6 +1136,21 @@ Found by sentry (2026-07-25), verified by exercising. =hyprland/.local/bin/wayba Repro: a conf with =America/Chicago|Home=, =Not/AZone|Bad=, =Europe/London|London= renders =tooltip: ""= (Home and London gone too). Grade: minor severity (one module's tooltip blanks, no data loss) x rare edge case (a malformed conf row) = P4 = [#D]. Fix: wrap the per-row =ZoneInfo=/=datetime= in a try/except and =continue=, so a typo drops only that row and the valid zones still render. Solo + quick: the script already has an env-override test harness (=WAYBAR_TIME_EPOCH=, =WAYBAR_WORLDCLOCK_CONF=), so a red-first test is cheap. +** TODO [#C] obsbot-wb-guard polls forever on machines with no OBSBOT :bug:dotfiles:quick:solo: +:PROPERTIES: +:LAST_REVIEWED: 2026-08-16 +:END: +=obsbot-wb-guard.service= is =WantedBy=graphical-session.target= and lives in the shared =common/= stow tier, so it starts on every machine. Its main path is =while :; do check_once; sleep 2; done=, and =check_once= returns early when the camera node is absent. On a machine with no OBSBOT attached that is a process waking every two seconds forever to do nothing, which on a laptop is battery spend for zero benefit. No restart loop, though: the loop never exits, so =Restart=on-failure= never fires. + +Found 2026-08-16 on velox, after enabling it to match ratio and then having to disable it again by hand. A per-machine disable is the wrong shape, because it drifts velox from ratio permanently and a re-stow or a future audit will just put it back. + +Fix: give the unit =ConditionPathExists= on the camera node (=/dev/v4l/by-id/usb-Remo_Tech_Co.__Ltd._OBSBOT_PW106-video-index0=, the same default the script uses) so systemd skips it on any machine without the camera and starts it normally on ratio. Then re-enable it on velox, where it will simply be skipped. Note the limit: a camera plugged in later will not start it until the next login, which is the right trade against a permanent poll. + +Careful when disabling by hand in the meantime: =systemctl --user disable= on a *linked* unit deletes the unit symlink, and that symlink is stow-managed, so a bare disable silently removes a file from the dotfiles stow tree. Restore the link afterward or re-stow. + +Grade: minor severity (wasted wakeups and battery, no data loss, no failure) x every boot on any machine without the camera = P3 = [#C]. + +Solo: buildable here (archsetup owns dotfiles end-to-end), verifiable by the agent (assert the unit is skipped on velox and still active on ratio), and no design call left open. ** TODO [#C] Auto-dim status forgotten on layout change :bug:dotfiles: :PROPERTIES: :LAST_REVIEWED: 2026-07-25 @@ -702,18 +1221,38 @@ doc above (not published, since they map the setup). Follow-ons: the rotation VERIFY above, velox reconcile on return, the secrets-repo split (top of Open Work), the wireguard =.gitignore= bug (line ~191), the cgit move (below), and a pre-receive secret-scan hook so this can't recur. -*** TODO [#A] velox: reconcile its clones after the history rewrite -velox was offline for repair during the 2026-08-09 purge, so its clones still -hold the pre-rewrite history and are diverged from the rewritten remotes. On -its return: force-fetch + rebase local work onto the rewritten main in both -repos (or re-clone), force-update the local tag, local-gc, before its next -push. Also on the velox riders on the sleep/suspend task. +*** 2026-08-17 Mon @ 19:57:42 -0700 Moot — the 08-13 wipe re-cloned velox from the rewritten remotes +This asked velox to reconcile clones that no longer exist. The machine was +wiped and reinstalled on 2026-08-13, so every repo on it was cloned fresh +*after* the purge and never held the pre-rewrite history at all. The runbook +anticipated this ("fresh clones automatically carry the post-purge rewritten +git history"); nobody closed the task once the reinstall took that route. + +Verified rather than assumed: both repos are level with =origin/main= today — +archsetup at =6faa31c=, dotfiles at =65940f2=, both trees clean. + +One thing the reinstall did leave, and it is filed separately: the installer +cloned both repos =--depth 1=, so the history was present-but-truncated until +today's =git fetch --unshallow= (see the shallow-clone =[#A]=). A reconcile +against the rewritten remote was still unnecessary — a shallow clone of the +right history is not a diverged clone of the wrong one. ** TODO [#B] Move archsetup off cgit to cjennings@cjennings.net :chore:security: :PROPERTIES: -:LAST_REVIEWED: 2026-07-21 +:LAST_REVIEWED: 2026-08-17 :END: Decided (Craig, 2026-07-20): move the archsetup repo off the public cgit host (git@cjennings.net, scan-path /var/git) to Craig's private account remote cjennings@cjennings.net, so it is no longer world-cloneable. This is the archsetup-specific fix for the cgit-exposure finding above. Plan: create a bare repo under cjennings's control off the cgit scan-path (e.g. =~cjennings/git/archsetup.git=); push current main + tags there; migrate the post-receive hook that publishes the installer to =/var/www/cjennings/archsetup= so curl-install keeps working (the single published file stays public by design; only the repo goes private); update the origin remote on ratio and velox to =cjennings@cjennings.net:git/archsetup.git=; remove =/var/git/archsetup.git= so cgit no longer serves it. Verify: anonymous =git clone https://git.cjennings.net/archsetup.git= fails, the new private clone works from both machines, and the curl-install URL still returns the installer. Keep the two daily drivers' remotes in sync (daily-drivers rule). + +*** 2026-08-17 Mon @ 19:57:42 -0700 Re-checked: unstarted, and the exposure is confirmed live +Ran the task's own verification step as it stands today, which is the honest +way to check an unstarted task rather than reading its body back. Anonymous +=git ls-remote https://git.cjennings.net/archsetup.git= succeeded with no +credentials and returned =6faa31c= — this afternoon's HEAD. So the repo is +still world-cloneable and current to the commit, not a stale published +snapshot. + +=origin= on this machine is still =git@cjennings.net:archsetup.git=, the cgit +account, so nothing has moved. Everything in the plan stands unchanged. ** TODO [#B] Velox boot-failure retrospective — upgrade guard gaps :bug:zfs:maint: :PROPERTIES: :LAST_REVIEWED: 2026-07-21 @@ -1165,15 +1704,28 @@ Verify (manual, live): see Manual testing and validation. *** 2026-07-09 Thu @ 16:32:54 -0500 Audit reconcile: Phase 4 is filed on the dotfiles side, waiting on them The dotfiles project accepted the Phase 4 handoff and filed it as a =[#C]= task in their own =todo.org= (their note, 2026-07-08 16:56): the help-text audit + panel help affordance, the user-guide/README, and the ratio rollout doc. Not started there. They ping when it lands, and this task's Phase 4 child closes then. Nothing to do here meanwhile. -*** TODO Phase 4 — docs + rollout :network:blocked: -Deliverable: in-app help (=net --help= + per-command, panel help affordance); -README/user-guide (commands, indicator states, panel, config keys, make targets, -troubleshooting from the failure table, rollback); archsetup Hyprland dep install +*** 2026-08-17 Mon @ 19:57:42 -0700 Landed on the dotfiles side; the block is cleared +dotfiles shipped it as =138da7b= and closed its own task, so this one closes +with it and the =:blocked:= tag comes off. Found by checking their =todo.org= +rather than waiting for the ping — their close-out note says "archsetup pinged +so its Phase 4 task can close", so the handoff worked and only this end was +left open. + +All three acceptance criteria are met on their side: the help audit found and +fixed a stale =net repair= action list (nine of nineteen actions were named; +both the CLI help and =repair.py='s docstring now generate from the ACTIONS +registry), =net/README.md= covers every command plus the recovery targets, and +the ratio rollout is documented with both daily drivers verified current. + +They split the panel help affordance out rather than inventing it — no sibling +panel has one, so its shape is a design call. It is tracked on their side, not +here. + +Original deliverable, for the record: in-app help (=net --help= + per-command, +panel help affordance); README/user-guide; archsetup Hyprland dep install (=gtk4-layer-shell=, =python-gobject=, =speedtest-go-bin=); ratio manual dep + -stow step. -Verify: =net --help= and each subcommand complete; user-guide covers every command -+ the recovery targets. -Build handed off to the dotfiles project 2026-07-04 (=~/.dotfiles/inbox/2026-07-04-1305-from-archsetup-phase4-handoff.md=): archsetup deps confirmed installed, the remaining help/user-guide/rollout-doc work is in the net package. dotfiles pings back when it lands. +stow step. Handed off 2026-07-04 with the archsetup deps already confirmed +installed. *** TODO Phase 5 — VPN / WireGuard CLI fold (vNext) :network: Rescoped 2026-07-04 (audit): the tunnels track already shipped most of the original Phase 5. Panel tunnel bring-up/down and detection landed (dotfiles 2d9d060 probes tailscale/NM-wireguard/Proton; 21db05a brings overlays up/down from the panel's Tunnels sub-view; 31ba056 diagnose/doctor understand tunnel routes; archsetup 2e40781 wireguard config import; the net-panel-other-interfaces spec is IMPLEMENTED). What remains for Phase 5 is only the =net vpn ...= CLI subcommand — cli.py still has no vpn/tunnel parser. Fold the panel's existing tunnel operations into a CLI surface; spec separately when picked up. @@ -1435,6 +1987,29 @@ Add kernel parameter: ~rtc_cmos.use_acpi_alarm=1~ (will become systemd default) Consider: ~acpi_mask_gpe=0x1A~ for battery drain, suspend-then-hibernate config See Framework community notes on logind.conf and sleep.conf settings +*** 2026-08-17 Mon @ 19:57:42 -0700 Four of the five riders are done; WireGuard is the one left +The riders were written for "when velox returns from repair". It came back as +a full reinstall instead, and the installer carried most of them, so I checked +each on the live machine rather than reading the list back: + +- tlp radio-enable — done. =/etc/tlp.d/01-custom.conf:10= carries + =DEVICES_TO_ENABLE_ON_STARTUP="bluetooth wifi"=, written by the installer. +- touchpad auto-detection — the dotfiles half is done: =touchpad-auto + --detect= prints =pixa3854:00-093a:0274-touchpad=. Read that carefully + though — it names the device the config expects, not a device delivering + events. The touchpad is still dead on the ribbon fault, so this rider is + satisfied and the hardware still is not. +- podman socket — done, =podman.socket= is enabled. +- camera udev — done, =72-usb-passthrough-cameras.rules= is installed. +- *wolf WireGuard — not done, and it is the one that was time-critical.* No + =~/.config/wireguard/wolf.conf.gpg= and no WireGuard profile in + NetworkManager. The 08-08 decision set this up specifically so velox could + reach home from the road, on the argument that it is cheap at home and + expensive from a hotel. velox is now in the hotel. + +The suspend work itself is untouched — no kernel parameter, no drain +measurement. Only the riders moved. + ** TODO [#B] Manual testing and validation :test: :PROPERTIES: :LAST_REVIEWED: 2026-07-09 @@ -1443,6 +2018,33 @@ Craig's standing checklist of everything that isn't agent-verifiable. Each child Priority and type tag added by that audit: the task carried neither, which kept the project's largest live container out of the agenda entirely. +*** Clock/DNS deadlock: does the next abrupt power loss strand velox again? +What we're verifying: that the machine survives an RTC reset unattended. Not the +coin cell, which is new with the 2026-08-13 mainboard and is ruled out. The RTC +did not drift on 2026-08-19, it was reset to exactly 2025-01-01T00:00:16 by an +abrupt power loss at 01:33:18 that left no shutdown sequence in the journal. + +This one can't be scheduled. Run the block the next time velox comes up after an +unexpected power loss, before touching the clock. +#+begin_src sh :results output +echo "--- what did the RTC read at this boot? ---" +journalctl -b 0 | grep -m1 'rtc_cmos.*setting system clock' +echo "--- did systemd have to advance the clock to its build epoch? ---" +journalctl --list-boots | tail -3 +echo "--- sources: is an IP-addressed one selected? ---" +chronyc -n sources +echo "--- clock + DNS ---" +timedatectl | grep -iE 'Local time|RTC time|synchronized' +getent hosts gnu.org || echo "DNS DEAD" +#+end_src +Expected: even if the RTC came up at 2025-01-01 and systemd advanced the clock +to 2026-07-23, chrony reached 162.159.200.1 without DNS, stepped the clock to +now, and names resolve. You did nothing. + +If instead the clock is still wrong or DNS is dead, the fix did not hold in the +field despite holding under a simulated skew. Capture that whole block and +promote this to a top-level TODO. + *** Floating layout: freeze positions, border flash, glyph, exit to master What we're verifying: the rebuilt floating mode (Super+Shift+F) floats every window on the workspace via per-window setfloating (the old workspaceopt allfloat was deprecated and no-op'd, which is why nothing floated), freezes each in place, flashes the border gold on entry and exit, flips the waybar glyph to the floating icon, and exits to master. Live-verified on a headless output already (windows floated in place, dragged to overlap, glyph read Floating, toggled back clean); this is the on-your-own-monitor confirmation. - Go to a workspace with 2-3 tiled windows in master. @@ -2036,7 +2638,7 @@ NOTE (2026-07-04 audit): the "four-tab panel" framing predates the instrument-co ** DOING [#B] Prepare for GitHub open-source release :PROPERTIES: -:LAST_REVIEWED: 2026-07-09 +:LAST_REVIEWED: 2026-08-17 :END: Remove personal info, credentials, and code quality issues before publishing. *** 2026-07-21 Tue @ 08:00:00 -0500 Audit reconcile: the four assets/ "& Claude" author lines are fixed @@ -2093,6 +2695,19 @@ Recommend: fresh repo for GitHub (keep cjennings.net remote with full history). History is now 589 commits (the 2026-05-11 note's "275" is stale). Only the calendar-feed file has been filter-repo'd so far (2026-05-20). The five credential files remain in history at their pre-=b10cba5= paths: =.tidal-dl.token.json= (5 commits), =calibre/smtp.py.json= (6), =transmission/settings.json= (5), =.msmtprc= (8), =.mbsyncrc= (9). None are tracked in the current tree. The scrub-or-fresh-repo decision still stands. ***** 2026-07-04 Sat @ 11:48:24 -0500 Count refresh — history now 565 commits; re-verify the 5-file claim before scrubbing The 2026-07-04 audit found the history is now 565 commits, down from the 589 recorded above. Because the count dropped, re-verify that the five credential files are still present in history (re-run the per-file =git log --all -- <path>= check) before relying on the scrub scope — the earlier count is stale and the file set may have moved. +***** 2026-08-17 Mon @ 10:20:00 -0700 Corrected the paths — every prior check has been querying paths that never existed +The five filenames recorded above are not the paths these files live at, and =git log -- <path>= answers "no commits" for a path it has never seen rather than erroring. So the checks return a clean result and mean nothing. The real paths, from =git log --all --name-only --diff-filter=A= over the full history, all sit under the pre-migration =dotfiles/= tree: + +- =dotfiles/system/.msmtprc= (3 commits) +- =dotfiles/system/.mbsyncrc= (2) +- =dotfiles/system/.config/calibre/smtp.py.json= (2) +- =dotfiles/system/.config/transmission/settings.json= (2) +- =dotfiles/system/.config/.tidal-dl.token.json= (2) +- =dotfiles/system/.config/.tidal-dl.json= (2) — a *sixth* file, never recorded here + +Use those paths for any future check, not the bare filenames. History is 891 commits; none of the six are in the current tree. The scrub-or-fresh-repo decision still stands and its scope is six files, not five. + +This surfaced while re-verifying on velox, where the check ALSO returned a false clean for a second, unrelated reason: the clone was shallow (7 commits), so it could not see the history either way. Both failures produce the same confident zero. Filed as =[#A] The installer shallow-clones the two repos I develop in=. ***** 2026-07-21 Tue @ 08:00:00 -0500 Re-verified: history now 851 commits; five files still present, per-file counts dropped 2026-07-21 audit re-verification. History is now 851 commits (=git rev-list --all --count=). The five credential files are still in history but at fewer commits each than the 2026-06-28 record: =.tidal-dl.token.json= 3 (was 5), =calibre/smtp.py.json= 4 (was 6), =transmission/settings.json= 3 (was 5), =.msmtprc= 5 (was 8), =.mbsyncrc= 6 (was 9). None are in the current tree. The scrub-or-fresh-repo decision still stands; the scope is smaller than recorded. @@ -2224,9 +2839,11 @@ From the roam inbox (routed 2026-07-13): the networking panel should track speed ** TODO [#C] zfs base VM image build failure: ZFS DKMS module missing :bug:zfs: :PROPERTIES: -:LAST_REVIEWED: 2026-07-09 +:LAST_REVIEWED: 2026-08-17 :END: =FS_PROFILE=zfs make test-vm-base= fails inside the VM at initramfs time: archangel reports "ZFS module not found! DKMS build may have failed" against the installed kernel (linux-lts 6.18.38 at the 2026-07-08 attempt). Consequences: the maint scenario harness's zfs lane (Phase 12) is filtered but unexercised, and a real zfs bare-metal install via archangel would plausibly hit the same wall. Priority per the bug matrix: Major severity (zfs install path broken) × some-users-sometimes = P3. When fixed, run =FS_PROFILE=zfs bash scripts/testing/run-maint-scenarios.sh --list= and add zfs scenario files (zpool scrub / autotrim / snapshot destroy) to the harness. +*** 2026-08-17 Mon @ 10:08:51 -0700 Rechecked: archzfs still on 2.3.3, still blocked +Ran the unblock check from the diagnosis below: archzfs' x86_64 index still serves only =zfs-dkms-2.3.3=. The first release supporting 6.18 is 2.4.0, so the blocking condition is unchanged and there is still nothing on our side to fix. Recheck again with the same one-liner. *** 2026-07-14 Tue @ 01:40:48 -0500 Diagnosed: OpenZFS/kernel version skew, blocked on archzfs Reproduced in ~1 minute of install: =dkms install zfs/2.3.3 -k 6.18.38-2-lts= exits 1 during pacstrap. Root cause confirmed: OpenZFS 2.3.3's META declares Linux-Maximum 6.15, and the VM installs linux-lts 6.18.38. The first release supporting 6.18 is 2.4.0 (2.4.1 covers 6.19), and archzfs currently serves only zfs-dkms 2.3.3-1 — nothing on our side to fix. Unblock condition: archzfs publishes zfs-dkms ≥2.4.0; recheck with =curl -s https://archzfs.com/archzfs/x86_64/ | grep -o 'zfs-dkms-[0-9.]*'=, then rerun =FS_PROFILE=zfs make test-vm-base=. @@ -2266,10 +2883,12 @@ Craig's roam capture 2026-07-20, routed via .emacs.d sentry inbox-zero as archse Craig: the camera works (bought for being Linux-friendly), it just needs configuring. There IS a config panel — =cameractrls= 0.6.10 is installed (a GTK GUI for camera controls: exposure, white balance, PTZ, focus, framing) plus =v4l-utils= for the CLI path. Caveat found 2026-07-21: no =/dev/video*= device is present right now, so the camera isn't currently plugged in / its UVC node isn't enumerated. Task: with the camera connected, confirm it enumerates as a /dev/video node, then set defaults in cameractrls. Small, mostly a live-hardware step. ** TODO [#C] Re-check python-lyricsgenius --skipinteg workaround :chore:solo: :PROPERTIES: -:LAST_REVIEWED: 2026-07-09 +:LAST_REVIEWED: 2026-08-17 :END: archsetup installs =python-lyricsgenius= with =--mflags --skipinteg=, skipping makepkg integrity + PGP checks — a workaround originally for an expired-signature issue upstream (surfaced by the 2026-06-23 --noconfirm audit). Periodically test whether the cause has cleared: if a plain =aur_install python-lyricsgenius= builds without complaint, drop the =--skipinteg= workaround. Removal needs a real AUR build to confirm, so it isn't a blind change. +*** 2026-08-17 Mon @ 10:08:51 -0700 Rechecked: still needed, cause unchanged +Fresh AUR clone, =makepkg --verifysource= on 3.7.0-1: the PyPI tarball passes, =LICENSE.txt= still FAILS its b2sum. The PKGBUILD still pins the license at github master, so its checksum drifts whenever upstream touches the file. =--skipinteg= stays. *** 2026-07-23 Thu @ 02:20:00 -0500 Rechecked: still needed, unchanged Fresh AUR clone, =makepkg --verifysource= on 3.7.0-1 (PKGBUILD still unchanged since the last check): the PyPI tarball passes, =LICENSE.txt= still FAILS its b2sum. Same structural cause — the source pins the license at github master, so its checksum drifts whenever upstream touches the file. =--skipinteg= stays. Nothing to change in the installer. @@ -2371,6 +2990,119 @@ Re-graded =[#C]= → =[#D]= per the bug matrix. There is no defect to fix here; The maintenance console's coredump metric flagged telega-server on ratio (8 coredumps) and velox (18). Root cause was a version skew: the Dockerized =zevlg/telega-server:latest= is frozen at the 2026-06-05 build while the installed elisp lagged at 20260513, so the newer server's plist parser choked on the older elisp's output. .emacs.d fixed it by upgrading telega to 20260706 on both machines (docker kept, =docker pull= is a no-op against the frozen image). Host-coredump pollution should stop. If zevlg later pushes a =:latest= that outruns the installed elisp, the skew and the coredumps recur — the tell is a fresh =tdat_plist_value:500= assertion in =~/.telega/telega-server.log=. The durable escape is a host-native pinned TDLib build, at the cost of an AUR source build. * Archsetup Resolved +** DONE [#B] Velox touchpad interrupt line is dead — needs a part or a BIOS fix :bug:velox:hardware: +CLOSED: [2026-08-15 Sat] +:PROPERTIES: +:CREATED: [2026-08-15 Sat] +:LAST_REVIEWED: 2026-08-15 +:END: +*Fixed 2026-08-15 23:05 by reseating the correct connector* — a seating fault +all along, no part needed. Verified at the kernel level on the 23:05 boot: the +=did not ack reset within 1000 ms= message is gone (clean handshake), and the +interrupt count went 0 → 1795. Power-key events also zero, so both faults from +the mainboard swap are closed. + +What made this take three attempts is worth keeping: two of the connectors on +that board were decoys. The input-cover ribbon looked like the obvious suspect +and fixing it *did* resolve the power button, which made it look like the whole +answer. Then the 4-pin connector next to the printed =TOUCHPAD= label looked +like the touchpad's own — and its cable is silkscreened =PIN 1-2 - GND / +PIN 3-4 - VCC=, four contacts of pure power, incapable of carrying i2c or an +interrupt. Reading that silkscreen off the photo is what ruled it out and sent +the search to the ribbon that actually crosses to the mainboard. + +The ordered touchpad becomes a spare, which is what Craig wanted from it anyway. +The diagnostic path below is left intact — it is the reusable part: =dmesg= +for the i2c-HID reset message and the interrupt count in =/proc/interrupts= +together separate "device absent" from "device present but its interrupt line is +open", and a live USB separates hardware from software in two minutes. +Split from the ribbon-reseat task 2026-08-15 once the reseat fixed the power +button and left this untouched — they are two faults, not one. + +*Diagnosed to the interrupt line specifically, with software eliminated.* +- The i2c *data* path works. =i2c_hid_acpi= read the HID descriptor, returned + the right product ID (=093A:0274=), =hid-multitouch= bound, and input6/7/8/9 + were created. A descriptor read is a real bus transaction, so the device is + electrically present and answering. +- The *interrupt* path never fires. IRQ 81, =amd_gpio= hwirq 8, level-triggered, + =actions=PIXA3854:00= — the handler is correctly registered on the pin the + firmware names. Count is 0 across all 24 CPUs, including during active + swiping. +- =dmesg=: =i2c_hid_acpi i2c-PIXA3854:00: device did not ack reset within 1000 ms=. + The i2c-HID reset handshake is acknowledged *by the device asserting the + interrupt*, so the first operation needing that line already failed at boot, + before anything touched the pad. That is why the fault reproduces on any boot + in ten seconds. +- *Software ruled out by live USB.* Same "did not ack reset" message and no + pointer movement under Ubuntu's kernel (2026-08-15). Not a driver, not + libinput, not Hyprland, not this install. + +Three candidates remain, all needing a part or firmware: +1. Open conductor on the touchpad's own cable or a bad contact at either end. + Framework sells "Touchpad Cable" as a discrete spare, so it is separately + replaceable — and the input-cover ribbon reseat would not have touched it. +2. The touchpad module's interrupt output is dead while its i2c slave still + answers. Indistinguishable from 1 without swapping parts. +3. Firmware naming the wrong GPIO. The DSDT says =amd_gpio= pin 8; if this + board revision routes the interrupt elsewhere, the kernel watches a pin that + never toggles. Plausible because the mainboard is days old to this machine + and its firmware already needed the PSR workaround. BIOS is 03.05 + (2025-10-30); kernel 6.18.44-1-lts. + +*The connector that was reseated is NOT the touchpad's — confirmed from the +board photo.* Craig reseated the 4-pin connector near the printed word +=TOUCHPAD=. Its cable is silkscreened =PIN 1-2 - GND / PIN 3-4 - VCC= — four +contacts, all of them power. No clock, no data, no interrupt; almost certainly +the keyboard backlight feed. An i2c-HID touchpad cannot run through it, so that +reseat could never have fixed this, and *the free retry remains untried*. +Photo: [[file:working/velox-touchpad-interrupt/touchpad-module-underside-2026-08-15.jpg][working/velox-touchpad-interrupt/touchpad-module-underside-2026-08-15.jpg]]. + +Visible on that board: the controller IC marked =PCT3854= (matching the kernel's +=PIXA3854=), a larger =CON3= carrying a blue-backed ribbon with "26" marked +beside it, a white ZIF past the Framework QR label, and a further connector at +the board's end. The one that matters is whichever ribbon physically *leaves the +input cover and reaches the mainboard* — that is the touchpad cable, and its far +end is the press-fit connector at the board. Reseat both ends of that one before +fitting any new part. + +*BIOS 04.02 exists but does not look relevant.* Checked 2026-08-15 with velox +on AC at 90%: fwupd offers 0.0.3.5 → 0.0.4.2. Read the changelog — the only +touchpad line is haptic-touchpad support for the Laptop 13 *Pro* chassis, and +this machine has a conventional PixArt =PIXA3854=. The rest is BIOS Setup +layout, option naming, TPM behavior, iGPU defaults, PMF slider. Nothing about +GPIO routing or interrupt configuration. So candidate 3's cheap test is weaker +than it looked when it was filed sight-unseen; still worth doing (unlisted +fixes happen, and ACPI tables change), just no longer the front-runner. +Deliberately deferred past the flight — a cleared NVRAM is the failure that +started this whole rebuild. Boot-entry recovery reference captured at +[[file:working/velox-reinstall/velox-uefi-boot-entry-reference.org][working/velox-reinstall/velox-uefi-boot-entry-reference.org]]. + +Order of attack on return, cheapest first: reseat the touchpad's *own* press +connector at the mainboard (free, untried) → BIOS 04.02 → fit the replacement +touchpad. Craig's call 2026-08-15: order the parts now anyway, since they are +worth holding as spares regardless of which candidate wins. + +*What to order.* The replacement *Touchpad* ships with the Touchpad Cable +pre-installed, so that single part covers candidates 1 and 2 together — no need +to buy both to cover both. A bare Touchpad Cable is worth adding only as a cheap +spare. The *Input Cover* is a different and more expensive part, and nothing +points at it: the keyboard works, so the input-cover ribbon is carrying signal. +Framework's marketplace renders its catalogue in JavaScript, so prices could not +be read programmatically — search "Touchpad" under Laptop 13 parts. + +*Also worth a Framework support ticket* — the touchpad died coincident with +their mainboard swap, which may put it inside whatever recourse that carries. + +Grading: Major severity (a laptop's built-in pointer is entirely dead — the +counter-argument is that an external mouse is a complete workaround, which +would make it Minor; I took Major because losing the integrated pointer degrades +the machine's portability, which is the whole point of the laptop) x every user, +every time = P1 = [#A]. Filed [#B] rather than [#A] only because an [#A] must +carry a date and Craig's return date isn't known yet — date it and raise it to +[#A] when it is. + +Workaround in the meantime: Bluetooth mouse, already in use. + ** DONE [#A] Tracked WireGuard private keys in repo — public leak, resolved :bug:security:network: CLOSED: [2026-07-20 Mon] @@ -3657,159 +4389,204 @@ CLOSED: [2026-08-08 Sat] Killed at the 2026-08-08 task review: an undated annual intention that never fired — pain points get surfaced organically as they bite. Once-yearly systematic inventory of known deficiencies and friction points in current toolset -** DONE [#A] Reseat velox input-cover ribbon — phantom power button :bug:velox:hardware: -CLOSED: [2026-08-15 Sat] DEADLINE: <2026-08-14 Fri> -:PROPERTIES: -:CREATED: [2026-08-13 Thu] -:LAST_REVIEWED: 2026-08-13 -:END: -Machine off, lift the input cover (Framework QR-guided procedure, 5 -fasteners), reseat its ribbon connector to the mainboard — disturbed in the -2026-08-13 board swap. Root cause of every "mystery reboot" that day: -chassis flex (flash-drive touch, ethernet bump, lid partially lowered) -fired phantom power-button presses — journalctl -b -1 showed "Power key -pressed short." → orderly logind poweroff, then the glitching button -powered it back on. While in there, reseat the USB expansion cards too — -the flaky slot (two hard resets, one no-enumeration) is likely the same -flex problem. -THIRD SYMPTOM (2026-08-13 evening): touchpad delivers ZERO input events — -15s synchronized libinput debug-events capture while swiping caught -nothing, though i2c enumeration and a driver rebind handshake are clean. -Signature of a dead interrupt line on the same ribbon. Keyboard + power -LED lines work; BT mouse is the interim pointer. -ESCALATED 2026-08-13 21:00: a fourth event killed the machine THROUGH the -shield. Previous boot's journal ends mid-line (tailscaled chatter) with no -shutdown sequence at all — a hard power cut, not logind acting. So the -glitch now reaches the EC/hardware power path, which no software setting -can intercept. The reseat is the only fix, and this is a -lose-work-without-warning failure mode, not an inconvenience. -Interim shield (already live): /etc/systemd/logind.conf.d/powerkey.conf -sets HandlePowerKey=ignore — phantom presses log but do nothing; EC-level -10s hold still force-cuts. Consider keeping it even after the repair. -Verify after reseat: flex the chassis edges + partially lower the lid, then -grep the journal for new "Power key pressed" lines — zero means fixed. -Must be done before the Sunday flight — a phantom press mid-travel with the -shield on is survivable, but the connector should not be trusted at 30,000 -feet on the loose setting. -*** 2026-08-15 Sat @ 22:30:00 -0500 Reseated the ribbon; the power button is fixed and the touchpad is not -Done the night before the flight. The power-button half worked: after the -reseat I flexed the chassis and the 22:09 boot logged *zero* "Power key -pressed" lines, against nine on the previous boot. That is real rather than the -shield masking it — =HandlePowerKey=ignore= was already live during those nine, -so logind logs what it suppresses. Short sample (minutes); worth re-checking -after a day of uptime. - -The touchpad did not change, which separated the two symptoms and disproved the -one-fault model they were filed under. Split out as its own task below. -** DONE [#B] Velox touchpad interrupt line is dead — needs a part or a BIOS fix :bug:velox:hardware: -CLOSED: [2026-08-15 Sat] +** CANCELLED [#B] agent-text relay reports success for a message that went nowhere :bug: +CLOSED: [2026-08-19 Wed] +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +Not a defect. rulesets refuted it with measurements and I reproduced theirs +before accepting: on velox, whose account store is empty, +=signal-cli -a +15550000000 send= exits 1 with "User +15550000000 is not +registered", and =ssh 100.71.182.1 'exit 7'= returns 7, so a non-zero code +propagates faithfully back through the relay. The loop's +=[ "$rc" -eq 0 ] && break= therefore advances to the next host exactly as +intended. signal-cli fails closed. + +I filed this off a conditional in their handoff — ".emacs.d raised a case +neither of you tested ... *if* signal-cli send exits zero against an empty +account store" — and turned the "if" into a graded [#B] with a =:blocked:= tag +on another project, without running the one command that settles it. The +machine that proves it was in front of me the whole time. Their ask is fair and +I am recording it rather than the outcome alone: verify before filing a defect +against someone else's work, especially one carrying a blocking tag. +** DONE [#B] Clock/DNS bootstrap deadlock — recovery needs a second device :bug:velox: +CLOSED: [2026-08-19 Wed] +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +The installer wrote both halves of a deadlock. =configure_dns= pins +=DNSOverTLS=yes= with =DNSSEC=yes=, and both validate against the wall clock; +the chrony step enables chronyd without writing a config, so the machine runs +Arch's stock one whose only source is =pool 2.arch.pool.ntp.org= — a hostname. +Boot with a wrong clock and DoT certificate validation fails, so nothing +resolves; chrony then cannot resolve its pool, so the clock stays wrong. +Neither side moves. It caught velox on the road 2026-08-19 and had to be +diagnosed from a phone. + +Fixed at the root: the installer now writes +=/etc/chrony.d/10-bootstrap-ip-ntp.conf= with two IP-addressed Cloudflare +sources and points stock chrony.conf at the drop-in. An address needs no DNS +and carries no certificate, so the escape hatch holds whatever broke the clock. +velox has the same drop-in applied live, verified with =chronyc -n sources= +(=162.159.200.1= selected) and =timedatectl= reporting synchronized. + +What is left here is the part I could not verify: the decisive test is a full +power-down and cold boot, confirming the clock corrects itself untouched. See +the manual-testing entry. Until that runs, the fix is sound by construction +rather than demonstrated. + +Grading: Critical severity (total loss of network — no DNS means no egress, and +recovery needs a second device) x some users sometimes (only machines that boot +with a wrong clock, which is any RTC fault, BIOS reset, or drained cell) = P2 = +[#B]. Graded on the being-in-it, not the getting-into-it: once the machine is in +this state it is fully offline with no local path out. + +*** 2026-08-19 Wed @ 12:25:00 -0700 Reproduced it, and the mechanism was not what either of us said +I wound velox's clock back 27 days with chronyd stopped and watched it fail. +Resolution died outright, and plain UDP/53 to 1.1.1.1 kept answering throughout +— the discriminator the doctor keys on, confirmed live rather than reasoned. + +The cause is DNSSEC, not DNS-over-TLS. resolved logged =signature-expired= +against the root DNSKEY and every DS beneath it. The DoT handshake to +=1.1.1.1:853= verified clean at that same clock, and the Cloudflare certificate +runs Dec 2025 to Dec 2026, so it was never outside its window. An RRSIG window +is days to weeks and a certificate is good for a year, so a skew that breaks +DNSSEC normally leaves DoT untouched. The phone session blamed the certificate +and I carried that forward into the first commit; both were wrong. + +=DNSSEC=allow-downgrade= does not rescue it either, which matters because it is +the obvious reach and it is what ratio runs. resolved downgrades when a server +lacks DNSSEC support, and a signature-window failure is a validation failure, so +no downgrade fires. Six retries over eighteen seconds plus +=resolvectl reset-server-features=, all dead. I briefly believed otherwise off a +test whose success was a cache hit (=Data from: cache network=). + +So ratio was exposed after all, and I have given it the same drop-in. Its +=162.159.200.1= is selected and its clock is synchronized. + +The fix itself is verified end to end: with the clock wound back and no DNS at +all, chronyd reached the IP-addressed source and stepped the clock from +2026-07-23 straight back to 2026-08-19. That is the whole claim, demonstrated +rather than argued. + +Also settled: the clock landed on 2026-07-23 because that is systemd 261.2's +build date to the minute (=/usr/lib/systemd/systemd=, 10:43:59), and systemd +advances a garbage RTC to its own build epoch at boot. Not timesyncd's +last-good-sync timestamp, which cannot be it — timesyncd is disabled here. That +also confirms the RTC really was reading earlier than that, so the coin cell +stays the prime suspect. + +*** 2026-08-19 Wed @ 10:12:00 -0700 Root fix, doctor verdict, and taxonomy entry landed +The installer carries the drop-in; =post-rebuild-check= grew a sixth check that +fails a machine whose every NTP source is a hostname; the net failure taxonomy +gained the mode in its DNS layer plus a cluster 5 triage line, and its existing +egress-layer clock entry now says outright that its remedy does not apply when +DoT or DNSSEC is on. + +The doctor half is in dotfiles: =classify.py= reached "DNS not resolving → net +repair dns-test" here, which cannot help, because every public resolver fails +the same clock-sensitive validation — so the doctor sent you round a loop. It +now emits a =clock-dns= row ahead of the generic DNS verdict. Detection is +deliberately DNS-free: a local =timedatectl= read for sync state, and a bypass +query addressed by IP over plain UDP/53 to tell "resolved is refusing to +validate" apart from "DNS is genuinely dead". +** DONE [#C] DNSSEC strictness on the travelling laptop :velox: +CLOSED: [2026-08-19 Wed] +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +Craig chose =allow-downgrade= everywhere. Applied to velox, ratio, and the +installer, and ratio's =DNSOverTLS= tightened from =opportunistic= to =yes= in +the same pass, so all three now agree: encrypted DNS always, validation +best-effort. + +The reasoning that settled it: the deadlock is fixed by the IP-addressed NTP +source, and =allow-downgrade= was measured not to help with it at all. What +=allow-downgrade= does buy is the venue-resolver case the taxonomy documents, +where =yes= turns a resolver that mangles DNSSEC records into no answer at all. +That is a hotel and airport problem, so it is velox's problem, and the +encryption is the half worth being strict about. +** CANCELLED [#C] Branch network policy on laptop vs desktop in the installer :feature: +CLOSED: [2026-08-19 Wed] +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +Cancelled because the decision above emptied it. All three motivating cases now +want the same value on every machine: =DNSSEC=allow-downgrade=, a stable +per-network wifi MAC, and an IP-addressed NTP source. A branch with nothing to +put on either side is machinery built for a divergence that does not exist, and +it would be the kind of scaffolding that rots unread. + +Worth keeping the observation, which is the part with a shelf life: when a +network default does need to differ by machine class, the test already exists. +=ls /sys/class/power_supply/BAT*= is what =prune_waybar_battery=, the ppd mask, +and the TLP config all key on. Reopen this then rather than building it now. +** DONE [#C] Automate the clock/DNS deadlock repair in the net doctor :feature: +CLOSED: [2026-08-19 Wed] +:PROPERTIES: +:CREATED: [2026-08-19 Wed] +:LAST_REVIEWED: 2026-08-19 +:END: + +Shipped as the =clock-ip-ntp= repair, and the verdict is =fixable= rather than +terminal. Both open questions got answered by driving a real deadlock instead of +reasoning about it: =chronyc add server= returns =200 OK= against a running +chronyd, and =makestep= needs a sample to land, so it took four calls and about +eight seconds rather than working on the first. The repair retries accordingly. + +Verified end to end on velox against a genuine deadlock (wrong clock, chronyd +running with only an unresolvable hostname source, DNS dead): the repair +corrected the clock in 6.1 seconds and DNS came back. + +The live run also caught a defect no unit test would have. The doctor reported +"Saved password for SpectrumSetup-3C was rejected" — because +=_recent_auth_failure= greps =journalctl --since -5min=, and a clock weeks off +windows onto a different incident's entries. It would have sent Craig to +re-enter a password that was never wrong. Fixed twice over: the journal half is +now skipped when the clock is untrustworthy, and the clock verdict is ordered +above the auth verdict, since everything below it reasons over timestamps that +only mean something once the clock is right. Airplane mode and hard rfkill stay +above, being physical states the clock has no bearing on. +** DONE [#D] net-scenarios harness times out under back-to-back suite runs :test:tooling: +CLOSED: [2026-08-19 Wed] :PROPERTIES: -:CREATED: [2026-08-15 Sat] -:LAST_REVIEWED: 2026-08-15 +:LAST_REVIEWED: 2026-07-24 :END: -*Fixed 2026-08-15 23:05 by reseating the correct connector* — a seating fault -all along, no part needed. Verified at the kernel level on the 23:05 boot: the -=did not ack reset within 1000 ms= message is gone (clean handshake), and the -interrupt count went 0 → 1795. Power-key events also zero, so both faults from -the mainboard swap are closed. - -What made this take three attempts is worth keeping: two of the connectors on -that board were decoys. The input-cover ribbon looked like the obvious suspect -and fixing it *did* resolve the power button, which made it look like the whole -answer. Then the 4-pin connector next to the printed =TOUCHPAD= label looked -like the touchpad's own — and its cable is silkscreened =PIN 1-2 - GND / -PIN 3-4 - VCC=, four contacts of pure power, incapable of carrying i2c or an -interrupt. Reading that silkscreen off the photo is what ruled it out and sent -the search to the ribbon that actually crosses to the mainboard. - -The ordered touchpad becomes a spare, which is what Craig wanted from it anyway. -The diagnostic path below is left intact — it is the reusable part: =dmesg= -for the i2c-HID reset message and the interrupt count in =/proc/interrupts= -together separate "device absent" from "device present but its interrupt line is -open", and a live USB separates hardware from software in two minutes. -Split from the ribbon-reseat task 2026-08-15 once the reseat fixed the power -button and left this untouched — they are two faults, not one. - -*Diagnosed to the interrupt line specifically, with software eliminated.* -- The i2c *data* path works. =i2c_hid_acpi= read the HID descriptor, returned - the right product ID (=093A:0274=), =hid-multitouch= bound, and input6/7/8/9 - were created. A descriptor read is a real bus transaction, so the device is - electrically present and answering. -- The *interrupt* path never fires. IRQ 81, =amd_gpio= hwirq 8, level-triggered, - =actions=PIXA3854:00= — the handler is correctly registered on the pin the - firmware names. Count is 0 across all 24 CPUs, including during active - swiping. -- =dmesg=: =i2c_hid_acpi i2c-PIXA3854:00: device did not ack reset within 1000 ms=. - The i2c-HID reset handshake is acknowledged *by the device asserting the - interrupt*, so the first operation needing that line already failed at boot, - before anything touched the pad. That is why the fault reproduces on any boot - in ten seconds. -- *Software ruled out by live USB.* Same "did not ack reset" message and no - pointer movement under Ubuntu's kernel (2026-08-15). Not a driver, not - libinput, not Hyprland, not this install. - -Three candidates remain, all needing a part or firmware: -1. Open conductor on the touchpad's own cable or a bad contact at either end. - Framework sells "Touchpad Cable" as a discrete spare, so it is separately - replaceable — and the input-cover ribbon reseat would not have touched it. -2. The touchpad module's interrupt output is dead while its i2c slave still - answers. Indistinguishable from 1 without swapping parts. -3. Firmware naming the wrong GPIO. The DSDT says =amd_gpio= pin 8; if this - board revision routes the interrupt elsewhere, the kernel watches a pin that - never toggles. Plausible because the mainboard is days old to this machine - and its firmware already needed the PSR workaround. BIOS is 03.05 - (2025-10-30); kernel 6.18.44-1-lts. - -*The connector that was reseated is NOT the touchpad's — confirmed from the -board photo.* Craig reseated the 4-pin connector near the printed word -=TOUCHPAD=. Its cable is silkscreened =PIN 1-2 - GND / PIN 3-4 - VCC= — four -contacts, all of them power. No clock, no data, no interrupt; almost certainly -the keyboard backlight feed. An i2c-HID touchpad cannot run through it, so that -reseat could never have fixed this, and *the free retry remains untried*. -Photo: [[file:working/velox-touchpad-interrupt/touchpad-module-underside-2026-08-15.jpg][working/velox-touchpad-interrupt/touchpad-module-underside-2026-08-15.jpg]]. +=tests/net-scenarios/test_run_net_scenarios.py= errored on all 5 tests twice during round 12, each time =subprocess.TimeoutExpired= after its 20s budget on =scripts/testing/run-net-scenarios.sh --target root@fake-vm=. Both occurrences were in =make test-unit= runs launched immediately after a previous full run. It then passed 6 runs in a row (3 on a pristine tree, 3 with the round-12 change), and standalone it finishes in 0.09s, so this is not a regression from any code change. -Visible on that board: the controller IC marked =PCT3854= (matching the kernel's -=PIXA3854=), a larger =CON3= carrying a blue-backed ribbon with "26" marked -beside it, a white ZIF past the Framework QR label, and a further connector at -the board's end. The one that matters is whichever ribbon physically *leaves the -input cover and reaches the mainboard* — that is the touchpad cable, and its far -end is the press-fit connector at the board. Reseat both ends of that one before -fitting any new part. +The harness stubs =ssh=, =rsync= and =jq= onto =PATH=, so nothing should touch the network at all — which is what makes a 20s timeout suspicious rather than merely slow. Worth reproducing under load before deciding whether the fix is a larger timeout or a real hang in the script. Evidence logs from the round: =/tmp/tu.log= and =/tmp/tu2.log= (tmpfs, gone after reboot). -*BIOS 04.02 exists but does not look relevant.* Checked 2026-08-15 with velox -on AC at 90%: fwupd offers 0.0.3.5 → 0.0.4.2. Read the changelog — the only -touchpad line is haptic-touchpad support for the Laptop 13 *Pro* chassis, and -this machine has a conventional PixArt =PIXA3854=. The rest is BIOS Setup -layout, option naming, TPM behavior, iGPU defaults, PMF slider. Nothing about -GPIO routing or interrupt configuration. So candidate 3's cheap test is weaker -than it looked when it was filed sight-unseen; still worth doing (unlisted -fixes happen, and ACPI tables change), just no longer the front-runner. -Deliberately deferred past the flight — a cleared NVRAM is the failure that -started this whole rebuild. Boot-entry recovery reference captured at -[[file:working/velox-reinstall/velox-uefi-boot-entry-reference.org][working/velox-reinstall/velox-uefi-boot-entry-reference.org]]. +Not graded on the bug matrix: it is test infrastructure, not the shipped codebase. -Order of attack on return, cheapest first: reseat the touchpad's *own* press -connector at the mainboard (free, untried) → BIOS 04.02 → fit the replacement -touchpad. Craig's call 2026-08-15: order the parts now anyway, since they are -worth holding as spares regardless of which candidate wins. +*** 2026-08-08 Sat @ 05:05:00 -0500 Recurred under concurrent load, same signature +All 5 tests hit the 20s TimeoutExpired again during a =make test-unit= run +that overlapped two review subagents running their own suites on the box. +Standalone immediately after: 0.095s, all pass; the following quiet-machine +full run was clean. Confirms the load-sensitivity read — reproduce under +deliberate load before choosing between a bigger budget and a real hang. -*What to order.* The replacement *Touchpad* ships with the Touchpad Cable -pre-installed, so that single part covers candidates 1 and 2 together — no need -to buy both to cover both. A bare Touchpad Cable is worth adding only as a cheap -spare. The *Input Cover* is a different and more expensive part, and nothing -points at it: the keyboard works, so the input-cover ribbon is carrying signal. -Framework's marketplace renders its catalogue in JavaScript, so prices could not -be read programmatically — search "Touchpad" under Laptop 13 parts. -*Also worth a Framework support ticket* — the touchpad died coincident with -their mainboard swap, which may put it inside whatever recourse that carries. +*** 2026-08-19 Wed @ 14:50:00 -0700 Root-caused and fixed: inherited stdin, not load +Not load, and not the network. The harness stubs ssh as =cat >/dev/null=, which +drains stdin to EOF. With no explicit stdin the stub inherits whatever the test +runner had, so it returned instantly when stdin was redirected and blocked +forever when it was a terminal or a live pipe. All five tests then burned their +20-second budget. -Grading: Major severity (a laptop's built-in pointer is entirely dead — the -counter-argument is that an external mouse is a complete workaround, which -would make it Minor; I took Major because losing the integrated pointer degrades -the machine's portability, which is the whole point of the laptop) x every user, -every time = P1 = [#A]. Filed [#B] rather than [#A] only because an [#A] must -carry a date and Craig's return date isn't known yet — date it and raise it to -[#A] when it is. +That is why it looked like a load effect: a run launched immediately after +another inherited a different stdin than a standalone invocation. A/B measured +today — =make test-unit </dev/null= exits 0, the same target with an open pipe +on stdin hangs on all five. The note above guessed at "a larger timeout or a +real hang in the script" and it was neither. -Workaround in the meantime: Bluetooth mouse, already in use. +Fixed by pinning =stdin=subprocess.DEVNULL= in =run_script=. Verified both ways: +the previously-failing open-pipe case and the redirected case both pass in +0.08s, and a full =make test-unit= under a live pipe is clean across 50 suites. |