diff options
Diffstat (limited to 'todo.org')
| -rw-r--r-- | todo.org | 467 |
1 files changed, 290 insertions, 177 deletions
@@ -404,127 +404,6 @@ Alternatives if it drags on: change Signal's tray setting so it keeps a window (=~/.config/Signal/ephemeral.json= =system-tray-setting=), or run a waybar carrying the #5240 fallback. -** TODO [#A] Reseat velox input-cover ribbon — phantom power button :bug:velox:hardware: -DEADLINE: <2026-08-26 Wed> -:PROPERTIES: -:CREATED: [2026-08-13 Thu] -:LAST_REVIEWED: 2026-08-13 -:END: -Machine off, lift the input cover (Framework QR-guided procedure, 5 -fasteners), reseat its ribbon connector to the mainboard — disturbed in the -2026-08-13 board swap. Root cause of every "mystery reboot" that day: -chassis flex (flash-drive touch, ethernet bump, lid partially lowered) -fired phantom power-button presses — journalctl -b -1 showed "Power key -pressed short." → orderly logind poweroff, then the glitching button -powered it back on. While in there, reseat the USB expansion cards too — -the flaky slot (two hard resets, one no-enumeration) is likely the same -flex problem. -THIRD SYMPTOM (2026-08-13 evening): touchpad delivers ZERO input events — -15s synchronized libinput debug-events capture while swiping caught -nothing, though i2c enumeration and a driver rebind handshake are clean. -Signature of a dead interrupt line on the same ribbon. Keyboard + power -LED lines work; BT mouse is the interim pointer. -ESCALATED 2026-08-13 21:00: a fourth event killed the machine THROUGH the -shield. Previous boot's journal ends mid-line (tailscaled chatter) with no -shutdown sequence at all — a hard power cut, not logind acting. So the -glitch now reaches the EC/hardware power path, which no software setting -can intercept. The reseat is the only fix, and this is a -lose-work-without-warning failure mode, not an inconvenience. -Interim shield (already live): /etc/systemd/logind.conf.d/powerkey.conf -sets HandlePowerKey=ignore — phantom presses log but do nothing; EC-level -10s hold still force-cuts. Consider keeping it even after the repair. -Verify after reseat: flex the chassis edges + partially lower the lid, then -grep the journal for new "Power key pressed" lines — zero means fixed. -Must be done before the Sunday flight — a phantom press mid-travel with the -shield on is survivable, but the connector should not be trusted at 30,000 -feet on the loose setting. - -*** 2026-08-19 Wed @ 14:40:00 -0700 Retracted: the RTC reset is not this task's, and I should not have filed it here -I attributed the 2026-08-19 network outage to this ribbon earlier today. Craig -pushed back — he reseated it before the trip to get the touchpad working — and -he is right. The evidence does not support the attribution and some of it points -the other way. - -What actually holds. Boot -3 ended at 01:33:18 with no shutdown sequence: no -power-off target, no unmounting. The next boot's kernel line reads =rtc_cmos -00:01: setting system clock to 2025-01-01T00:00:16 UTC=, a firmware default, so -the RTC was reset rather than drifted. No firmware update was applied -(=fwupdmgr get-history= is empty) and the battery is fine. - -What refutes the ribbon. This boot logged *zero* =Power key pressed= events, and -so did the four boots before it. The phantom-press symptom had genuinely stopped -after 08-15, exactly as the 08-16 session recorded. The earlier events logged a -power-key press and an orderly poweroff; this logged neither, which makes it a -different signature, not a worse version of the same one. - -What I got wrong methodologically: I anchored on the most salient open hardware -task and read association as evidence. I even wrote "I can't prove it is the -same connector" and then filed it here anyway, which is the tell. - -Two things I checked and can rule out. There were no OOM kills — the 3,433 -matching lines are a systemd unit named "Periodically re-score Claude Code -processes for the OOM-killer" firing on a timer, not memory pressure, and there -is not a single "Killed process" line. Thermal is clean; the only mentions are -boot-time zone registration at 34C and 45C. - -One real thing the same window did surface, tracked separately: a python3 crash -loop, 251 core dumps in the final ten minutes, SIGABRT with =XFreeThreads= and -=PyEval_RestoreThread= in the trace. It does not explain the RTC, because -software cannot clear it, but it is its own problem. - -The open question that would settle the RTC is for Craig, not the journal: a -long power-button hold on a Framework triggers an EC-level reset that clears the -RTC, which fits a wedged machine being forced off. A 4-second hold would not. - -*** 2026-08-17 Mon @ 19:57:42 -0700 Not done, and the two symptoms now disagree -The reseat did not happen before the flight, and velox is travelling. The -deadline blew past on 08-14. - -The two symptoms have separated, which is worth recording because it changes -what the evidence proves. The phantom presses have stopped: fifteen "Power key -pressed" entries between 08-14 04:29 and 08-15 20:04, then nothing at all -across five boots including today's. The touchpad has not — there is still no -touchpad node under =/dev/input/by-path/=, which is the same dead interrupt -line the body describes. - -So the quiet power button is not evidence the connector reseated itself. The -interrupt line is the symptom that cannot be masked in software, and it is -still dead, so the ribbon is still unseated. The most likely reason the -presses stopped is that the machine has been sitting on hotel surfaces instead -of being carried and flexed. - -The interim shield is still live (=HandlePowerKey=ignore=), and the escalation -note stands: an EC-level glitch cuts power below systemd regardless of it. -*** 2026-08-15 Sat @ 23:05:00 -0500 The reseat did happen, and the touchpad came back — this contradicts the 08-17 read -Recording this because a parallel session concluded on 08-17 that the reseat had -not happened and the touchpad was still dead. Both halves were done and verified -that night, so the two accounts disagree and the disagreement should be visible -rather than silently resolved by whichever session committed last. - -What was done: the input-cover ribbon was reseated first, which fixed the -phantom power button — the 22:09 boot logged zero =Power key pressed= lines -after Craig flexed the chassis, against nine on the boot before. The touchpad -did not change, because the input-cover ribbon is not its connector. The 4-pin -connector beside the printed =TOUCHPAD= label is silkscreened =PIN 1-2 GND / -PIN 3-4 VCC= — pure power, so it cannot carry i2c or an interrupt. Reseating the -ribbon that actually crosses to the mainboard fixed it. - -Measured, not assumed: the touchpad interrupt (=amd_gpio= pin 8) went from 0 -counts across all 24 CPUs to 1795, and =i2c_hid_acpi ... did not ack reset -within 1000 ms= disappeared from the boot log. Craig confirmed the pointer moved. - -*Why the 08-17 probe likely misread it:* it checked for a node under -=/dev/input/by-path/=. i2c-HID touchpads frequently get no =by-path= symlink -even when fully working, so its absence is not evidence of a dead interrupt -line. The falsifiable check is the interrupt count in =/proc/interrupts= while -the pad is being touched, or the reset message in =dmesg=. - -*Left open rather than closed* — velox was refusing ssh at merge time on 08-20, -so the current state could not be re-verified, and a later regression cannot be -ruled out. One second of Craig's time settles it: move the pointer. If it works, -close this; if it does not, the interrupt line went back down and that is new -information. - ** DOING [#A] Velox reinstall — DR test of archangel + archsetup :velox:chore: DEADLINE: <2026-08-15 Sat> :PROPERTIES: @@ -1321,17 +1200,26 @@ measuring it. Verify placement at the time of the move rather than trusting a recorded list. ** TODO [#B] Velox boot-failure retrospective — upgrade guard gaps :bug:zfs:maint: :PROPERTIES: -:LAST_REVIEWED: 2026-07-21 +:LAST_REVIEWED: 2026-08-26 :END: Post-mortem for the 2026-07-15 velox no-kernel boot failure, from the archsetup/maint code review: - maint's UPDATE remedy runs a plain =yay -Syu --noconfirm= (remedies.py:297). The live-update guard (guard.py) only matches mesa/hyprland (the 2026-06-07 live-swap class) — it never checks /boot, kernel, initramfs, or mkinitcpio exit. No post-upgrade /boot assertion exists. An interrupted kernel transaction slips straight through. - Add a post-upgrade /boot assertion: after a transaction touching linux/linux-*, confirm vmlinuz-* + initramfs-*.img present and mkinitcpio exit 0; refuse to end the run (or page Craig) otherwise. Would have caught this. - Sanoid-vs-actual dataset drift: configure_zfs_snapshots configures zroot/var/log + zroot/var/lib/pacman as separate datasets; velox's actual layout has neither separate (/var/log sits inside zroot/var). Reconcile. - CONFIRMED (2026-07-21): the pre-pacman snapshot hook fired on velox — the 2026-07-15 no-kernel boot was recovered via the pre-pacman ZFS snapshot rollback, and velox is back on the tailnet running linux-lts 6.18.38 with initramfs present (2026-07-19 session). Root-cause hook-ordering fix shipped separately. Still open: the post-upgrade /boot assertion in guard.py and the sanoid-vs-actual dataset drift reconcile (the two bullets above). +*** 2026-08-26 Wed @ 22:35:01 -0600 The /boot assertion now lives in the topgrade spec; the dataset drift is what remains here +The post-upgrade /boot assertion is covered by the kernel-modules-check gate in +[[file:docs/specs/2026-08-25-topgrade-guarded-upgrade-spec.org][the topgrade guarded-upgrade spec]] +(dkms built for the new kernel, initramfs newer than vmlinuz, pre-pacman +snapshot on a ZFS root), which ships with that spec's Phase 1 rather than here. +What this task still owns is the sanoid-vs-actual dataset drift: whether to +split zroot/var/log and zroot/var/lib/pacman out as configure_zfs_snapshots +assumes, or change the config to match the layout velox actually has. That is +a call I have not made, so the task stays [#B] and not solo. -** TODO [#B] Assess a Hyprland left-drag window gesture :feature:hyprland: +** TODO [#C] Assess a Hyprland left-drag window gesture :feature:hyprland: :PROPERTIES: -:LAST_REVIEWED: 2026-07-21 +:LAST_REVIEWED: 2026-08-26 :END: Evaluate whether a global left-click drag can move ordinary windows without breaking application selection, text interaction, or Wayland security @@ -1380,15 +1268,15 @@ the audit plus wiring plus docs runs past thirty minutes on its own. ** TODO [#B] Add storage-capacity signals to the maintenance module :feature:maint: :PROPERTIES: -:LAST_REVIEWED: 2026-07-21 +:LAST_REVIEWED: 2026-08-26 :END: Investigate capacity and growth diagnostics for full disks, identify the appropriate remedies, and incorporate a clear storage signal into the maintenance console. -** TODO [#B] Add per-channel controls to the audio panel :feature:audio: +** TODO [#C] Add per-channel controls to the audio panel :feature:audio: :PROPERTIES: -:LAST_REVIEWED: 2026-07-21 +:LAST_REVIEWED: 2026-08-26 :END: Expose channel-level input and output volume controls without losing the existing device-level workflow. @@ -2071,6 +1959,122 @@ filed here at Craig's direction so the sleep task carries it. Nothing diagnosed yet — first step is =journalctl -b -1 -u systemd-suspend-then-hibernate= around those timestamps to see which device reported busy. +*** 2026-08-26 Wed @ 16:16:08 -0600 Diagnosed the hibernate battery drain: three separate faults, one task each +Craig hibernated twice in ten days and found the battery dead both times. Read +all 39 boots since the 08-13 reinstall, upower's charge history +(=/var/lib/upower/history-charge-Framewo-55-03F5.dat=, root-only, starts +08-19), sysfs, and the scripts inside =/efi/EFI/ZBM/zfsbootmenu.efi=. + +Hibernate is configured right and has worked: five hibernate+resume cycles +since reinstall (08-13, 08-17 14:31, and three suspend-then-hibernate cycles on +08-20/21). The two fatal events are the two overnight explicit +=systemctl hibernate= runs, 08-17 22:20 and 08-21 19:58. Both journals end at +"PM: hibernation: hibernation entry"; the next power-ons (08-18 10:13, 08-22 +17:22) were fresh boots whose resume hook found no image, no later swapon +reported a leftover suspend signature, and on 08-22 the battery read 2% at +power-on. The 08-23/24 night was on AC and not a battery death (three suspends +failed to enter, machine awake all night at the charge limit; the 09:57 end was +three power-key presses and a hard cut at 63%). The 08-19 death was the +caffeine/hypridle one already diagnosed. + +Three faults, tracked as the children below: +- Hibernate hard-freezes on entry (documented on Framework 13 AMD incl. Ryzen + AI 300: black screen, never powers off, intermittent, amdgpu-side). Fits + everything: the freeze precedes the swap signature, so the next boot is + fresh, and a frozen laptop at ~5 W empties 44.7 Wh in ~8 h. Unprovable from + logs by nature; the alternative (completed hibernate, unattended power-on to + the ZBM passphrase prompt) predicts a surviving image, which neither boot + had — see the VERIFY. +- ZFS ARC starves the hibernate image: "Image allocation is 8118265 pages + short" today 14:09, "390678 pages short" 08-20 09:14. ARC 58 GB of 93, + =zfs_arc_max=0= so =c_max= = RAM − 1 GiB; the kernel must free RAM − + =image_size= (37.4 GB) ≈ 56 GB. systemd falls back to s2idle and retries + every 90 min, so suspend-then-hibernate never actually hibernates. +- The SD card reader (090c:3350, =sda=, no media) can block suspend entirely: + "Freezing remaining freezable tasks failed after 20s (wq_busy=1)", pending + =disk_events_workfn= on =events_freezable_pwr_efficient=, three times on + 08-23/24. On battery that is a dead laptop by morning. + +Mistake worth remembering: =journalctl --since … -k= silently limits itself to +the current boot (=-k= implies =-b=); cross-boot kernel facts need +=_TRANSPORT=kernel= or an explicit =-b=. + +*** TODO Hibernate entry freeze — confirm under observation, then mitigate :bug:velox:hibernate: +Interim rule until this closes: do not hibernate unattended on battery. Shut +down, or suspend on AC. + +What is known: the two dead-battery hibernates match the Framework 13 AMD +"hard freeze on hibernate entry" reports (community threads 69516 and 53860, +Arch bbs 293242): screen black, power LED on, never powers off; intermittent +(one report: every 6–7 cycles); TTM/amdgpu warnings; improved by newer +=linux-firmware=; no confirmed fix. Board A9, BIOS 03.05, linux-lts 6.18.46, +=amdgpu.dcdebugmask=0x610= already on the cmdline. + +Confirm first: the "Hibernate entry freeze: five observed cycles on AC" test +under Manual testing and validation. A failed cycle there is the proof the +journal cannot give. + +Mitigations to try in order once confirmed, one at a time, re-running the +cycles after each: (1) =linux-firmware= at current, then =linux-firmware-git= +if the freeze persists; (2) =/sys/power/disk= = =shutdown= instead of +=platform= (a systemd =HibernateMode=shutdown= drop-in), which skips the ACPI +S4 path some Framework users found hanging; (3) a newer kernel (=linux= vs +=linux-lts=) for the amdgpu delta; (4) unload =mt7925e= in a pre-sleep hook +if the freeze survives the first three. Not =:solo:=: each cycle needs a +person watching the power LED. + +*** TODO ZFS ARC starves the hibernate image — cap it or shrink it pre-hibernate :bug:zfs:velox:solo: +The arithmetic: the kernel preallocates RAM − =image_size= pages before +snapshotting; with 93 GB RAM and the default =image_size= (2/5 of RAM, +37.4 GB) that is ~56 GB, and only free memory plus what shrinkers give back +counts. ARC was 58 GB today and the ZFS shrinker released little inside the +preallocation window, so it came up 31 GiB short. Nothing in +=/etc/modprobe.d/= sets =zfs_arc_max=. + +Two fixes, either or both: +- Cap the ARC: =options zfs zfs_arc_max=<bytes>= in =/etc/modprobe.d/zfs.conf= + (16 GiB leaves ~70 GB reclaimable) plus =echo <bytes> > + /sys/module/zfs/parameters/zfs_arc_max= for the running system. +- Or a =/usr/lib/systemd/system-sleep/= pre hook for the hibernate class that + lowers =zfs_arc_max=, waits for =size= in + =/proc/spl/kstat/zfs/arcstats= to fall, and restores it post-sleep. Keeps + the big ARC while awake. +- Raising =image_size= toward the kernel's ceiling (about half of RAM) also + shrinks the demand; combine with the cap. +Install it through archsetup so the next rebuild carries it (velox-only: ratio +has no swap partition). + +Verify: after the change =arcstats size= drops below the cap within seconds; +then one live suspend-then-hibernate cycle on AC with the delay temporarily +short shows "hibernation exit" and no "Image allocation … short" line in the +journal. That live cycle rides the entry-freeze test above; the ARC half is +checkable without it. + +*** TODO SD card reader media polling can block suspend :bug:velox:solo: +The reader (USB 090c:3350 Silicon Motion, =sda=, "Media removed, stopped +polling" at boot yet =events_poll_msecs= = −1 → default 2000 ms) left a +=disk_events_workfn= item pending on the freezable workqueue three times on +08-23/24, and the freezer gives up after 20 s: "Failed to put system to +sleep … Device or resource busy". Same symptom as the flaky expansion slot in +the ribbon task; a stalled poll never completes. + +Fix: a udev rule for that vendor/product setting +=ATTR{events_poll_msecs}="0"= (or =block.events_dfl_poll_msecs=0= on the +cmdline if every removable disk should stop polling), shipped by archsetup. +Verify with =rtcwake -m mem -s 20= on AC: journal shows "PM: suspend entry" +and "PM: suspend exit" with no "Freezing remaining freezable tasks failed", +and =/sys/block/sda/events_poll_msecs= reads 0 after a replug. Pulling the +card before sleeping is the manual workaround meanwhile. + +*** VERIFY After the 08-17 and 08-21 dead batteries, did the first power-on hang, or boot straight to a fresh login? +Decides between the two mechanisms. An entry freeze leaves no image, so the +next power-on boots straight through. A completed hibernate followed by an +unattended power-on (phantom power button, ZBM passphrase prompt until dead) +leaves the image in place, so the next power-on would try to resume — and the +only way that ends in the fresh boots the journal shows is a hung resume that +got force-cut. If both power-ons went straight to a fresh login, the freeze +is the answer. + ** TODO [#A] Port Hyprland config to Lua before 0.57 drops .conf support :hyprland:dotfiles: SCHEDULED: <2026-08-25 Tue> :PROPERTIES: @@ -2911,6 +2915,32 @@ NOTE (2026-07-04 audit): the "four-tab panel" framing predates the instrument-co - Expected: ↓/↑ Mbps + ping + server shown inline. - Byte-rate→Mbps unit: VERIFIED 2026-06-30 (velox). Raw =speedtest-go --json= dl_speed read ~3.66M, unambiguously bytes/s (29 down / 80 up Mbps); =net speedtest= reported 33.62 / 77.99 through the wired path. =BYTES_PER_SEC = True= + =* 8 / 1e6= are correct, no flip needed. Remaining here is only that the panel renders the inline result. +*** Hibernate entry freeze: five observed hibernate cycles on AC +What we're verifying: whether velox hard-freezes on hibernate entry (black +screen, power LED on, never powers off), the documented Framework 13 AMD +failure that fits both dead-battery events. The journal cannot show it; a +person watching the LED can. +- Plug in AC, lid open, nothing important unsaved. +- Note the cycle number, then hibernate from a terminal: +#+begin_src sh :results output +date; systemctl hibernate +#+end_src +- Watch: the screen goes black; within about two minutes the power LED goes + off and the fans stop. +- Press power, enter the ZBM passphrase, and confirm the same session comes + back (windows still open). +- Check that the cycle was a real hibernate and not a fallback: +#+begin_src sh :results output +journalctl -b -o short-iso | grep -E "systemd-sleep|hibernation (entry|exit)|Image allocation|Failed to put" | tail -6 +#+end_src +- Repeat until five cycles are logged. +Expected: all five cycles power off within two minutes and resume into the +same session, with "hibernation exit" and no "Image allocation … short" line. +A cycle where the screen stays black with the power LED on for more than five +minutes is the entry freeze: hold power for 10 s, and write down the cycle +number and whether the keyboard backlight was lit. A cycle that instead comes +straight back with "Cannot allocate memory" is the ARC task, not a freeze. + ** DOING [#B] Prepare for GitHub open-source release :PROPERTIES: :LAST_REVIEWED: 2026-08-17 @@ -3061,7 +3091,7 @@ Grading (2026-08-25 review): Major severity — the doctor's verdict is silently ** TODO [#C] Weather chip color signals unclear + unenforced :bug:dotfiles:waybar:weather: :PROPERTIES: -:LAST_REVIEWED: 2026-07-21 +:LAST_REVIEWED: 2026-08-26 :END: From the roam inbox (2026-07-20): the shipped Waybar weather chip's comfort coloring reads as noise — it shows amber for no clear reason, and some items are bolded, which isn't a legible signal. Craig's intended scheme (every item except the arrow key colored by whether the weather is comfortable; NO bold or italic anywhere): - Normal — all text white: temp in 60-85; condition sunny/clear/etc. @@ -3126,13 +3156,13 @@ Reproduced in ~1 minute of install: =dkms install zfs/2.3.3 -k 6.18.38-2-lts= ex ** TODO [#C] Waybar collapse control: replace the triangle glyph :feature:waybar: :PROPERTIES: -:LAST_REVIEWED: 2026-07-14 +:LAST_REVIEWED: 2026-08-26 :END: From the 2026-07-04 roam capture. The waybar collapse mechanism (click the triangle, the bar sections redisplay shortened) works, but the triangle glyph doesn't match the instrument-console aesthetic the panels now use. Replace it with something in keeping with the console look. Aesthetic decision — bring Craig two or three concrete glyph/style options (a machined chevron, a console-key style expander, an engraved caret) before wiring. Dotfiles waybar config (handled per the archsetup-owns-dotfiles rule). Raised alongside the net-panel/audio speedrun; deferred from it because the glyph choice is a taste call. ** TODO [#C] Net panel: driver-health diagnostic tier :feature:network: :PROPERTIES: -:LAST_REVIEWED: 2026-07-14 +:LAST_REVIEWED: 2026-08-26 :END: Follow-up from the 2026-07-04 net-panel hardening speedrun (Craig's cj question on the no-WiFi item). The shipped no-wifi-hardware verdict covers "no adapter at all." This tier covers "adapter present but the driver is wedged": read-only health signals — =ip link= (device present but no-carrier / down), =dmesg= / =journalctl -k= for firmware-load failures, =rfkill= for a hard block, =modinfo= / =lsmod= for the driver module — classified before a generic reset. Remedy actions: a privileged =modprobe -r <mod> && modprobe <mod>= reload of the wifi driver, and a firmware-package pointer when the failure is a missing/failed firmware load. Dotfiles net-package work (handled per the archsetup-owns-dotfiles rule). Design pass first to decide whether it's worth a repair tier vs a needs-user-action pointer. @@ -3267,6 +3297,136 @@ Re-graded =[#C]= → =[#D]= per the bug matrix. There is no defect to fix here; The maintenance console's coredump metric flagged telega-server on ratio (8 coredumps) and velox (18). Root cause was a version skew: the Dockerized =zevlg/telega-server:latest= is frozen at the 2026-06-05 build while the installed elisp lagged at 20260513, so the newer server's plist parser choked on the older elisp's output. .emacs.d fixed it by upgrading telega to 20260706 on both machines (docker kept, =docker pull= is a no-op against the frozen image). Host-coredump pollution should stop. If zevlg later pushes a =:latest= that outruns the installed elisp, the skew and the coredumps recur — the tell is a fresh =tdat_plist_value:500= assertion in =~/.telega/telega-server.log=. The durable escape is a host-native pinned TDLib build, at the cost of an AUR source build. * Archsetup Resolved +** DONE [#A] Reseat velox input-cover ribbon — phantom power button :bug:velox:hardware: +CLOSED: [2026-08-26 Wed] DEADLINE: <2026-08-26 Wed> +:PROPERTIES: +:CREATED: [2026-08-13 Thu] +:LAST_REVIEWED: 2026-08-13 +:END: +Machine off, lift the input cover (Framework QR-guided procedure, 5 +fasteners), reseat its ribbon connector to the mainboard — disturbed in the +2026-08-13 board swap. Root cause of every "mystery reboot" that day: +chassis flex (flash-drive touch, ethernet bump, lid partially lowered) +fired phantom power-button presses — journalctl -b -1 showed "Power key +pressed short." → orderly logind poweroff, then the glitching button +powered it back on. While in there, reseat the USB expansion cards too — +the flaky slot (two hard resets, one no-enumeration) is likely the same +flex problem. +THIRD SYMPTOM (2026-08-13 evening): touchpad delivers ZERO input events — +15s synchronized libinput debug-events capture while swiping caught +nothing, though i2c enumeration and a driver rebind handshake are clean. +Signature of a dead interrupt line on the same ribbon. Keyboard + power +LED lines work; BT mouse is the interim pointer. +ESCALATED 2026-08-13 21:00: a fourth event killed the machine THROUGH the +shield. Previous boot's journal ends mid-line (tailscaled chatter) with no +shutdown sequence at all — a hard power cut, not logind acting. So the +glitch now reaches the EC/hardware power path, which no software setting +can intercept. The reseat is the only fix, and this is a +lose-work-without-warning failure mode, not an inconvenience. +Interim shield (already live): /etc/systemd/logind.conf.d/powerkey.conf +sets HandlePowerKey=ignore — phantom presses log but do nothing; EC-level +10s hold still force-cuts. Consider keeping it even after the repair. +Verify after reseat: flex the chassis edges + partially lower the lid, then +grep the journal for new "Power key pressed" lines — zero means fixed. +Must be done before the Sunday flight — a phantom press mid-travel with the +shield on is survivable, but the connector should not be trusted at 30,000 +feet on the loose setting. + +*** 2026-08-19 Wed @ 14:40:00 -0700 Retracted: the RTC reset is not this task's, and I should not have filed it here +I attributed the 2026-08-19 network outage to this ribbon earlier today. Craig +pushed back — he reseated it before the trip to get the touchpad working — and +he is right. The evidence does not support the attribution and some of it points +the other way. + +What actually holds. Boot -3 ended at 01:33:18 with no shutdown sequence: no +power-off target, no unmounting. The next boot's kernel line reads =rtc_cmos +00:01: setting system clock to 2025-01-01T00:00:16 UTC=, a firmware default, so +the RTC was reset rather than drifted. No firmware update was applied +(=fwupdmgr get-history= is empty) and the battery is fine. + +What refutes the ribbon. This boot logged *zero* =Power key pressed= events, and +so did the four boots before it. The phantom-press symptom had genuinely stopped +after 08-15, exactly as the 08-16 session recorded. The earlier events logged a +power-key press and an orderly poweroff; this logged neither, which makes it a +different signature, not a worse version of the same one. + +What I got wrong methodologically: I anchored on the most salient open hardware +task and read association as evidence. I even wrote "I can't prove it is the +same connector" and then filed it here anyway, which is the tell. + +Two things I checked and can rule out. There were no OOM kills — the 3,433 +matching lines are a systemd unit named "Periodically re-score Claude Code +processes for the OOM-killer" firing on a timer, not memory pressure, and there +is not a single "Killed process" line. Thermal is clean; the only mentions are +boot-time zone registration at 34C and 45C. + +One real thing the same window did surface, tracked separately: a python3 crash +loop, 251 core dumps in the final ten minutes, SIGABRT with =XFreeThreads= and +=PyEval_RestoreThread= in the trace. It does not explain the RTC, because +software cannot clear it, but it is its own problem. + +The open question that would settle the RTC is for Craig, not the journal: a +long power-button hold on a Framework triggers an EC-level reset that clears the +RTC, which fits a wedged machine being forced off. A 4-second hold would not. + +*** 2026-08-17 Mon @ 19:57:42 -0700 Not done, and the two symptoms now disagree +The reseat did not happen before the flight, and velox is travelling. The +deadline blew past on 08-14. + +The two symptoms have separated, which is worth recording because it changes +what the evidence proves. The phantom presses have stopped: fifteen "Power key +pressed" entries between 08-14 04:29 and 08-15 20:04, then nothing at all +across five boots including today's. The touchpad has not — there is still no +touchpad node under =/dev/input/by-path/=, which is the same dead interrupt +line the body describes. + +So the quiet power button is not evidence the connector reseated itself. The +interrupt line is the symptom that cannot be masked in software, and it is +still dead, so the ribbon is still unseated. The most likely reason the +presses stopped is that the machine has been sitting on hotel surfaces instead +of being carried and flexed. + +The interim shield is still live (=HandlePowerKey=ignore=), and the escalation +note stands: an EC-level glitch cuts power below systemd regardless of it. +*** 2026-08-15 Sat @ 23:05:00 -0500 The reseat did happen, and the touchpad came back — this contradicts the 08-17 read +Recording this because a parallel session concluded on 08-17 that the reseat had +not happened and the touchpad was still dead. Both halves were done and verified +that night, so the two accounts disagree and the disagreement should be visible +rather than silently resolved by whichever session committed last. + +What was done: the input-cover ribbon was reseated first, which fixed the +phantom power button — the 22:09 boot logged zero =Power key pressed= lines +after Craig flexed the chassis, against nine on the boot before. The touchpad +did not change, because the input-cover ribbon is not its connector. The 4-pin +connector beside the printed =TOUCHPAD= label is silkscreened =PIN 1-2 GND / +PIN 3-4 VCC= — pure power, so it cannot carry i2c or an interrupt. Reseating the +ribbon that actually crosses to the mainboard fixed it. + +Measured, not assumed: the touchpad interrupt (=amd_gpio= pin 8) went from 0 +counts across all 24 CPUs to 1795, and =i2c_hid_acpi ... did not ack reset +within 1000 ms= disappeared from the boot log. Craig confirmed the pointer moved. + +*Why the 08-17 probe likely misread it:* it checked for a node under +=/dev/input/by-path/=. i2c-HID touchpads frequently get no =by-path= symlink +even when fully working, so its absence is not evidence of a dead interrupt +line. The falsifiable check is the interrupt count in =/proc/interrupts= while +the pad is being touched, or the reset message in =dmesg=. + +*Left open rather than closed* — velox was refusing ssh at merge time on 08-20, +so the current state could not be re-verified, and a later regression cannot be +ruled out. One second of Craig's time settles it: move the pointer. If it works, +close this; if it does not, the interrupt line went back down and that is new +information. + +*** 2026-08-26 Wed @ 22:30:46 -0600 Closed: the reseat was done on 08-15 and the task was never marked +I reseated the ribbon on 2026-08-15 and never closed this. The 08-15 entry +above already records the verification: zero =Power key pressed= lines on the +22:09 boot after flexing the chassis, the touchpad interrupt count back up +once the right connector was reseated. This boot shows zero presses as well. +The interim shield (=HandlePowerKey=ignore= in +=/etc/systemd/logind.conf.d/powerkey.conf=) is still live; I'm leaving it in +place, since a phantom press with it on costs nothing and without it costs +the session. ** DONE [#B] Velox touchpad interrupt line is dead — needs a part or a BIOS fix :bug:velox:hardware: CLOSED: [2026-08-15 Sat] :PROPERTIES: @@ -3381,53 +3541,6 @@ carry a date and Craig's return date isn't known yet — date it and raise it to Workaround in the meantime: Bluetooth mouse, already in use. -** DONE [#B] Weather tooltip caching :feature:waybar:weather:solo: -CLOSED: [2026-07-25 Sat 10:53] -From the roam inbox (Craig, claimed 2026-07-22): retrieve the weather tooltip data once per hour and cache it. If the network is unavailable, display the cached tooltip with explanatory text saying so. Dotfiles-side work (archsetup owns the lifecycle); touches common/.local/bin/weather. -Verified complete in the 2026-07-25 batch: the weather CLI already had the hourly default TTL, fresh-cache no-fetch path, stale fallback, and explicit offline footer. Its 33-test suite and the full dotfiles suite pass. -** DONE [#B] Settings gear becomes four device toggles :feature:waybar:dotfiles:solo: -CLOSED: [2026-07-25 Sat 10:53] -From the roam inbox (Craig, claimed 2026-07-23): the waybar gear should become four icons — touchpad, mouse, webcam, and a notification bubble. Clicking each toggles that setting directly. The first three turn red when disabled; the bubble turns red when DND is enabled. - -Today =custom/settings= (=hyprland/.config/waybar/config=) is one gear glyph () whose only job is =on-click: settings-panel=. The toggles themselves already exist and are tested — the settings package owns touchpad, mouse, and webcam (=webcam.py= is the USB-authorized kill switch from 2026-07-22), so this is a bar-side surface over existing backends rather than new capability. - -Note the state-polarity split when wiring the colors: three read "red = off" and DND reads "red = on". That asymmetry is deliberate (red means "something is disabled that normally isn't, or suppressed that normally isn't"), so encode it per-icon rather than deriving one rule. - -Decided 2026-07-23 (Craig): the gear STAYS alongside the four toggles as the panel launcher. So the bar's right side grows from 12 modules to 16 — the four toggles are net-new, the gear keeps its =on-click: settings-panel=. Open sub-question for build time, not blocking: whether the four toggles are four separate waybar modules or one custom module rendering four glyphs (fewer layout entries, one exec). Pick at build; the four-module shape is simplest and matches how mic/net already sit as individual modules. -Shipped in the 2026-07-25 batch as four independent JSON modules over the existing verified settings backends. Touchpad, mouse, and webcam turn terracotta when disabled; DND uses the deliberate inverse polarity; unavailable hardware dims. The gear remains the panel launcher. The live and Dupre theme CSS copies stay byte-identical. -** DONE [#C] Wallpaper panel selection and scroll state :feature:dotfiles:solo: -CLOSED: [2026-07-25 Sat 10:53] -:PROPERTIES: -:LAST_REVIEWED: 2026-07-25 -:END: -From the roam inbox (Craig, 2026-07-25). Screenshot: =~/pictures/screenshots/2026-07-25_013041.png=. Three related behaviors in the settings wallpaper panel (=settings/src/settings/wallpaper.py=): -1. Open at the wallpaper currently displayed, not the top of the list. -2. Highlight that wallpaper as selected in the scrollable pane while it shows in the preview. -3. Keep the scroll position when a picture is selected. Today selecting a picture snaps the scroll back to the top, which is the bug half of this. -Grade: minor scroll-reset defect x every panel selection = P3 = [#C]; the open-at-current and select-current behaviors are enhancements at the same level. One type tag, so filed =:feature:= with the scroll-reset called out as the bug. Solo: buildable in the settings GTK panel, agent-verifiable via headless capture plus the wallpaper.py tests, no design call — swww query gives the current wallpaper, and scroll-position preservation and row selection are standard GTK. -Shipped in the 2026-07-25 batch. The panel queries =awww query= off the UI thread, prefers the actually displayed image over stale stored state, highlights it, scrolls it into view on first open, and remembers the horizontal adjustment across selection-triggered rebuilds. -** DONE [#C] Net tooltip IPs and line order :feature:waybar:network:solo: -CLOSED: [2026-07-25 Sat 10:53] -From the roam inbox (Craig, claimed 2026-07-23): in the wifi hover, add the internal IP, external IP, and gateway IP just below the Interface line; move the Signal line to just above the keyboard-shortcuts line. Design constraint: the bar's hot path does no network I/O (status.py deliberately skips _address_facts on the 2s beat) — internal IP + gateway can ride cheap local reads, but the external IP must come from a cache the connectivity probe refreshes, never a live lookup in waybar-net. -Shipped in the 2026-07-25 batch. The slow connectivity probe caches local addressing and a validated external IP with the network identity; the Waybar hot path only reads that valid cache. Tooltip order is Interface, internal/external/gateway IPs, connectivity detail, throughput, Signal, shortcut. -** DONE [#B] Dupre Kit merge — casting additions :feature:tooling:solo: -CLOSED: [2026-07-25 Sat 10:53] -Fold docs/prototypes/dupre-kit-additions.js back into the kit proper: detentFader (NEW — multi-detent slide attenuator with speedbump drag physics: magnet + escape hysteresis, parked tick glow) and the drumRoller redefinition (UPGRADE — 1..N channels and min/max range; stock hardcodes two drums and throws on one, defaults reproduce stock exactly) and the guardedToggle redefinition (UPGRADE — lever throws with rotateX so it flips toward the viewer instead of the stock 180° planar spin that sweeps sideways mid-transition; contract unchanged). Merge means: builders into widgets.js, the additions CSS into DUPRE_CSS, additions-scoped gradients into the shared defs plate, gallery cards for both in panel-widget-gallery.html, and POLICY entries. Origin: the desktop-settings casting sitting 2026-07-21 — Craig's direction is that components get finished by being needed ("the ones needed most will have had the most attention"), so more additions may accrue here before the merge; batch them. -Shipped in the 2026-07-25 batch. =widgets.js= now owns all three builders, shared gradients/CSS, contracts, and policy records; additions no longer redefines them when older casting pages load it. The gallery has a three-detent fader card and a three-channel 0–12 drum demonstration (112 cards total). Static ownership tests, JS syntax checks, and the complete headless interaction probe pass. -** DONE [#C] Maint live-refresh hairline replacement :feature:maint:solo: -CLOSED: [2026-07-25 Sat 10:53] -:PROPERTIES: -:LAST_REVIEWED: 2026-07-14 -:END: -From the roam inbox (routed 2026-07-13): the memory-killer section seemed to update too often, and "it's a bit unclear what the line is doing; consider something else." Diagnosis (2026-07-14): the data cadence is already the requested 3s (gui live tier, _LIVE_SECONDS); the perceived churn is the live-refresh hairline — the 2px bar under the live sections that drains full-to-empty over each 3s window, redrawn at 150ms (gui._hair_tick, viewmodel.refresh_fraction). It exists to tell a stale board from a frozen one (2026-07-09), but it reads as constant unexplained motion. Design call for Craig: replace the draining line with something whose meaning is legible — candidates: a dot that blinks once per refresh, a "3s" age caption that only appears when refresh is overdue, slowing the drain redraw, or dropping the indicator on live tiers and keeping it only when data goes stale. Keep the stale-vs-frozen distinguishability that motivated the hairline. -*** 2026-07-21 Tue @ 08:35:00 -0500 Decided (Craig): silent-until-stale age caption -Replace the draining 2px hairline with an age caption that shows ONLY when refresh is overdue (e.g. "3s", "8s" once past the expected window) and shows nothing while the board is healthy. This keeps the stale-vs-frozen signal — a frozen board surfaces a growing age number, a live one stays clean — while removing the constant motion the hairline created. Implementation (dotfiles, archsetup-owned): drop =gui._hair_tick= / the hairline draw, add an overdue-age caption driven off =viewmodel.refresh_fraction= (or the last-refresh timestamp) rendered only past the live window. Now unblocked; needs a live visual check on the panel after. -Shipped in the 2026-07-25 batch. The animated draw area and 150ms timer are gone; the memory section header stays silent through the healthy three-second window, then shows a once-per-second growing age caption. Pure boundary tests and the full maintenance suite pass. -** DONE [#D] Test-framework + prototype refactor cluster :refactor:solo: -CLOSED: [2026-07-25 Sat 10:53] -Grading: no behavior change; parking lot. Refactors from the S5-S7 audit, distinct from the installer refactor rollup above. -scripts/testing/run-test.sh + run-test-baremetal.sh duplicate the run/poll/report skeleton and have drifted (VM uses setsid + copy helpers, baremetal uses nohup + hand-rolled sshpass scp) — extract the shared core so baremetal inherits the sturdier paths; run-maint-nspawn.sh:66 + run-maint-scenarios.sh:78 duplicate the transport-independent _scenario_var/_validate_scenario/run_scenario (a sourced lib/maint-scenario.sh); run-test.sh:251,265 uses two different mechanisms (pgrep vs ps|grep) for the same liveness check; docs/prototypes/gen_tokens.py:78 repeats the section-iteration skeleton across four emitters; gallery-widget.el:95,136 hardcodes SVG arc/hub path strings that duplicate the cx/cy/radius geometry (dial desyncs silently on a constant change); gallery-widget.el:72,84 leans on the private svg--append. See findings doc (S5, S6, S7). -Completed test-first in the 2026-07-25 batch. QEMU and bare-metal runners share liveness/report helpers; maintenance transports share scenario validation/execution; token emitters share ordered section traversal; and the Emacs SVG gauge shares semicircle geometry and uses the public DOM append API. Every fast Python/ERT suite passes. ** DONE [#B] Two agent sessions sharing one git repo :chore:tooling: CLOSED: [2026-07-26 Sun] Craig approved the shared-rules-layer solution on 2026-07-26. |