From 38200c6683e55860b044568cd70004dcbc7c4031 Mon Sep 17 00:00:00 2001 From: Craig Jennings Date: Sat, 4 Jul 2026 15:38:00 -0500 Subject: docs(specs): adopt status-heading lifecycle convention across specs Migrate 29 legacy specs off the old shape (a status suffix in the filename plus a :STATUS: property drawer) onto the docs-lifecycle status heading: a top-level heading carrying the org lifecycle keyword and a dated history line, with the two #+TODO sequences in the header. Dropping the -doing/-implemented/-superseded suffixes means a status change no longer forces a rename and link surgery. Each keyword comes from the spec's own recorded status. The four specs already on the heading form are untouched, and every inbound reference now points at the new names. The status board is one grep: rg '^\* (DRAFT|READY|DOING|IMPLEMENTED|SUPERSEDED|CANCELLED) ' docs/specs/ --- docs/design/signal-client-review.org | 2 +- docs/design/utility-inventory.org | 6 +- docs/specs/ai-kb-spec.org | 12 +- docs/specs/ai-vterm-spec-superseded.org | 163 --- docs/specs/ai-vterm-spec.org | 167 +++ .../specs/cache-helper-design-spec-implemented.org | 169 --- docs/specs/cache-helper-design-spec.org | 173 +++ docs/specs/company-to-corfu-migration-spec.org | 12 +- docs/specs/coverage-spec-implemented.org | 210 ---- docs/specs/coverage-spec.org | 214 ++++ docs/specs/debug-profiling-spec.org | 12 +- docs/specs/dev-setup-project-spec.org | 12 +- docs/specs/dupre-clear-theme-spec.org | 12 +- ...face-font-diagnostic-popup-spec-implemented.org | 197 ---- docs/specs/face-font-diagnostic-popup-spec.org | 200 ++++ ...eck-modeline-customization-spec-implemented.org | 319 ----- .../specs/flycheck-modeline-customization-spec.org | 323 ++++++ docs/specs/gloss-spec-doing.org | 320 ----- docs/specs/gloss-spec.org | 323 ++++++ docs/specs/google-keep-emacs-integration-spec.org | 9 +- docs/specs/init-load-graph-spec-doing.org | 833 ------------- docs/specs/init-load-graph-spec.org | 837 +++++++++++++ .../specs/keybinding-console-safety-spec-doing.org | 943 --------------- docs/specs/keybinding-console-safety-spec.org | 947 +++++++++++++++ docs/specs/messenger-unification-spec.org | 13 +- docs/specs/music-config-without-emms-spec.org | 12 +- docs/specs/org-faces-spec-implemented.org | 154 --- docs/specs/org-faces-spec.org | 157 +++ docs/specs/signal-client-spec-doing.org | 254 ---- docs/specs/signal-client-spec.org | 257 ++++ .../specs/theme-studio-completion-preview-spec.org | 11 +- docs/specs/theme-studio-nerd-icons-colors-spec.org | 9 +- .../theme-studio-package-faces-spec-doing.org | 590 ---------- docs/specs/theme-studio-package-faces-spec.org | 594 ++++++++++ .../theme-studio-palette-generator-spec-doing.org | 298 ----- docs/specs/theme-studio-palette-generator-spec.org | 301 +++++ ...o-perceptual-color-metrics-spec-implemented.org | 580 ---------- .../theme-studio-perceptual-color-metrics-spec.org | 584 ++++++++++ docs/specs/theme-studio-preview-locate-spec.org | 13 +- .../theme-studio-seeding-engine-spec-doing.org | 354 ------ docs/specs/theme-studio-seeding-engine-spec.org | 358 ++++++ ...eme-studio-semantic-theme-architecture-spec.org | 15 +- docs/specs/theme-studio-structured-output-spec.org | 13 +- docs/specs/utility-consolidation-spec-doing.org | 1220 ------------------- docs/specs/utility-consolidation-spec.org | 1224 ++++++++++++++++++++ ...vterm-to-ghostel-migration-spec-implemented.org | 424 ------- docs/specs/vterm-to-ghostel-migration-spec.org | 428 +++++++ 47 files changed, 7197 insertions(+), 7081 deletions(-) delete mode 100644 docs/specs/ai-vterm-spec-superseded.org create mode 100644 docs/specs/ai-vterm-spec.org delete mode 100644 docs/specs/cache-helper-design-spec-implemented.org create mode 100644 docs/specs/cache-helper-design-spec.org delete mode 100644 docs/specs/coverage-spec-implemented.org create mode 100644 docs/specs/coverage-spec.org delete mode 100644 docs/specs/face-font-diagnostic-popup-spec-implemented.org create mode 100644 docs/specs/face-font-diagnostic-popup-spec.org delete mode 100644 docs/specs/flycheck-modeline-customization-spec-implemented.org create mode 100644 docs/specs/flycheck-modeline-customization-spec.org delete mode 100644 docs/specs/gloss-spec-doing.org create mode 100644 docs/specs/gloss-spec.org delete mode 100644 docs/specs/init-load-graph-spec-doing.org create mode 100644 docs/specs/init-load-graph-spec.org delete mode 100644 docs/specs/keybinding-console-safety-spec-doing.org create mode 100644 docs/specs/keybinding-console-safety-spec.org delete mode 100644 docs/specs/org-faces-spec-implemented.org create mode 100644 docs/specs/org-faces-spec.org delete mode 100644 docs/specs/signal-client-spec-doing.org create mode 100644 docs/specs/signal-client-spec.org delete mode 100644 docs/specs/theme-studio-package-faces-spec-doing.org create mode 100644 docs/specs/theme-studio-package-faces-spec.org delete mode 100644 docs/specs/theme-studio-palette-generator-spec-doing.org create mode 100644 docs/specs/theme-studio-palette-generator-spec.org delete mode 100644 docs/specs/theme-studio-perceptual-color-metrics-spec-implemented.org create mode 100644 docs/specs/theme-studio-perceptual-color-metrics-spec.org delete mode 100644 docs/specs/theme-studio-seeding-engine-spec-doing.org create mode 100644 docs/specs/theme-studio-seeding-engine-spec.org delete mode 100644 docs/specs/utility-consolidation-spec-doing.org create mode 100644 docs/specs/utility-consolidation-spec.org delete mode 100644 docs/specs/vterm-to-ghostel-migration-spec-implemented.org create mode 100644 docs/specs/vterm-to-ghostel-migration-spec.org diff --git a/docs/design/signal-client-review.org b/docs/design/signal-client-review.org index 7e8a73e9..e1ac462e 100644 --- a/docs/design/signal-client-review.org +++ b/docs/design/signal-client-review.org @@ -5,7 +5,7 @@ * Scope reviewed - =.ai/workflows/spec-review.org=. -- =docs/specs/signal-client-spec-doing.org=, including the base design, open-question dispositions, initiate-message workflow, architecture additions, accepted caveats, test plan, scope summary, and readiness rubric. +- =docs/specs/signal-client-spec.org=, including the base design, open-question dispositions, initiate-message workflow, architecture additions, accepted caveats, test plan, scope summary, and readiness rubric. - =modules/signal-config.el=, including =cj/signal--parse-contacts=, notify-suppression helpers, private config loading, and current =use-package signel= wiring. - =~/code/signel/signel.el=, including =signel-start=, =signel--send-rpc=, =signel--dispatch=, =signel--handle-error=, =signel--handle-receive=, =signel--insert-msg=, =signel--insert-system-msg=, =signel--send-input=, =signel-chat=, and dashboard commands. - =tests/test-signal-config.el=, covering contact parsing and notify-suppression helpers. diff --git a/docs/design/utility-inventory.org b/docs/design/utility-inventory.org index 8438a592..9b811429 100644 --- a/docs/design/utility-inventory.org +++ b/docs/design/utility-inventory.org @@ -4,7 +4,7 @@ * Status -Living inventory. Phase 1 of [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec-doing.org]]. Records the current state of helpers identified in the spec's Candidate Extraction Table plus any new candidates discovered during module walkthroughs. Decisions become concrete tasks in =todo.org= for Phase 2+. +Living inventory. Phase 1 of [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec.org]]. Records the current state of helpers identified in the spec's Candidate Extraction Table plus any new candidates discovered during module walkthroughs. Decisions become concrete tasks in =todo.org= for Phase 2+. * Scope @@ -82,7 +82,7 @@ Caller counts in the inventory below reflect grep results from 2026-05-10. The c | Symbol | File | Vis | Deps | Side effects | Proposed home / name | Callers (modules) | Tests | Pri | Decision | Rationale | |--------+------+-----+------+--------------+----------------------+-------------------+-------+-----+----------+-----------| -| =cj/modeline-vc-cache-*= helpers (key/get/put/clear/valid-p) | =modeline-config.el:108-140= | private | buffer-local vars | mutates buffer-local state | =cj-cache.el= / =cj/cache-valid-p=, =cj/cache-get=, =cj/cache-put=, =cj/cache-clear= | 1 (within file) | =test-modeline-config-vc-cache.el= | Medium | Defer | Good pattern, but variable-local cache shape differs from the agenda/refile caches. Needs design before extraction. Spec calls out a Phase 5 design addendum at =docs/specs/cache-helper-design-spec-implemented.org=. | +| =cj/modeline-vc-cache-*= helpers (key/get/put/clear/valid-p) | =modeline-config.el:108-140= | private | buffer-local vars | mutates buffer-local state | =cj-cache.el= / =cj/cache-valid-p=, =cj/cache-get=, =cj/cache-put=, =cj/cache-clear= | 1 (within file) | =test-modeline-config-vc-cache.el= | Medium | Defer | Good pattern, but variable-local cache shape differs from the agenda/refile caches. Needs design before extraction. Spec calls out a Phase 5 design addendum at =docs/specs/cache-helper-design-spec.org=. | | agenda/refile cache vars and build flags | =org-agenda-config.el=, =org-refile-config.el= | n/a | timers, file scans | scans filesystem, sets vars | =cj-cache.el= / =cj/cache-value-or-rebuild= | 2 | none | Medium | Defer | TTL/build/invalidate lifecycle; higher risk than the modeline cache. Same Phase 5 work. | ** Logging / Warnings @@ -144,7 +144,7 @@ These become =todo.org= entries (or update existing ones) as Phase 2 starts. ** Deferred (track in =todo.org= but no commit yet) -- Cache abstraction (modeline + agenda/refile) -- needs Phase 5 design addendum at =docs/specs/cache-helper-design-spec-implemented.org=. +- Cache abstraction (modeline + agenda/refile) -- needs Phase 5 design addendum at =docs/specs/cache-helper-design-spec.org=. - =cj/--open-with-is-launcher-p= -- move when external-open ownership is finalized. - =cj/log-silently= rename -- low value; do during incidental =system-lib= work. - HTML/text helpers (=strip-html=, =clean-text=) -- defer until a second consumer. diff --git a/docs/specs/ai-kb-spec.org b/docs/specs/ai-kb-spec.org index fbd35ca5..b35b3ac6 100644 --- a/docs/specs/ai-kb-spec.org +++ b/docs/specs/ai-kb-spec.org @@ -1,11 +1,15 @@ -:PROPERTIES: -:ID: 03742426-35ce-41c5-aed7-d4e248e91833 -:STATUS: not-started -:END: #+TITLE: Design: AI Knowledge Base (ai-kb) #+AUTHOR: Craig Jennings #+DATE: 2026-05-24 #+OPTIONS: toc:nil num:nil +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Design: AI Knowledge Base (ai-kb) +:PROPERTIES: +:ID: 03742426-35ce-41c5-aed7-d4e248e91833 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from existing :STATUS: not-started * Status diff --git a/docs/specs/ai-vterm-spec-superseded.org b/docs/specs/ai-vterm-spec-superseded.org deleted file mode 100644 index 0b6bfb86..00000000 --- a/docs/specs/ai-vterm-spec-superseded.org +++ /dev/null @@ -1,163 +0,0 @@ -:PROPERTIES: -:ID: 3abd0270-e87c-42b7-9b3a-ef60300db99d -:STATUS: superseded -:END: -#+TITLE: Design: ai-vterm — in-Emacs Claude launcher -#+AUTHOR: Craig Jennings -#+DATE: 2026-05-07 -#+OPTIONS: toc:nil num:nil - -* Status - -Draft. - -* Problem - -Claude Code currently launches outside Emacs via the =ai= shell script, which builds candidate projects from =~/.emacs.d=, =~/code/*=, =~/projects/*= (anything with =.ai/protocols.org=), opens each in a tmux window, and runs =claude "Read .ai/protocols.org and follow all instructions."= per project. The shell-out pulls focus to a terminal, and tmux's horizontal split is the wrong shape for a code-on-left, Claude-on-right reading layout. - -The in-Emacs alternative today is =vterm-toggle= at F12, which uses a horizontal bottom split via =display-buffer-at-bottom=. No project picker, no per-project session model, no vertical split. - -Building this in Emacs eliminates the context switch and gives the side-by-side layout that matches how the work is actually read. - -* Non-Goals - -- Replicating the =ai= script's git prep / auto-pull. Phase A.0 of the startup workflow already handles pulls at session start. -- A multi-project session switcher with its own UI. =consult-buffer= and the buffer list already navigate between =claude [...]= buffers. -- Replacing =vterm-toggle= at F12. The existing bottom-split flow stays for non-AI shells. -- Tab-bar or frame-per-project layouts. -- Auto-launching tmux inside the AI vterm. Claude under tmux adds a session-management layer for no benefit here. - -* Approaches Considered - -** Recommended: wrap =vterm= directly with per-project named buffers + a parallel display rule - -A new module =modules/ai-vterm.el= adds a command that picks a Claude-template project, opens (or reuses) a vterm buffer named =claude []=, and lets a new =display-buffer-alist= entry route any buffer matching that prefix to a right-side window. Multiple projects produce multiple coexisting buffers, all sharing the same right-side slot. Switching among them is a buffer-switch, not a kill-and-recreate. - -Pros: -- Same package (=vterm=) as the existing config. -- Per-project buffers run simultaneously without conflict. -- Right-side placement is one =display-buffer-alist= entry. -- Existing windmove (Shift-arrows) handles code↔Claude focus toggling. =buffer-move= (C-M-arrows) handles side-swap. Neither needs new bindings. - -Cons: -- Re-implements toggle/show-hide logic that =vterm-toggle= would handle for free. Acceptable because =vterm-toggle= is built around one toggle-able buffer, and the per-project model is what's wanted. - -** Rejected: wrap =vterm-toggle= - -=vterm-toggle='s contract is one buffer toggled visible/hidden. Per-project buffers running simultaneously is outside that contract. Wrapping it would mean fighting the abstraction. - -** Rejected: project-per-tab via =tab-bar-mode= - -Each project gets its own tab. Matches the =ai= / tmux model cleanly, but adds tab-bar UI that isn't in current use. Bigger lifestyle change for a one-window task. - -** Rejected: frame-per-project - -Each Claude session opens in a new Emacs frame. Hyprland-native, clean isolation, but frame creation under Wayland has historical jank, and it breaks the easy windmove flow between code and Claude. - -** Rejected: window-configuration-per-project - -Save and restore named window configs (code buffers + Claude vterm together). Preserves the surrounding thinking environment, but window configs go stale when buffers die, and it adds a parallel mechanism to project.el. Overkill for v1. - -* Design - -** Architecture - -New module =modules/ai-vterm.el=. Required after =eshell-vterm-config= in =init.el= so =vterm= is loaded. - -Components: - -| Function | Kind | Responsibility | -|----------+------+----------------| -| =cj/--ai-vterm-candidates= | pure | Walks =~/.emacs.d=, =~/code/*=, =~/projects/*=; returns abs paths containing =.ai/protocols.org= | -| =cj/--ai-vterm-pick-project= | interactive helper | =completing-read= over candidates; returns picked path | -| =cj/--ai-vterm-buffer-name= | pure | =(format "claude [%s]" basename)= | -| =cj/--ai-vterm-show-or-create= | internal | Given dir + name: display existing buffer, or create vterm + send claude command | -| =cj/ai-vterm= | interactive entry | Composes picker + show-or-create | - -The =display-buffer-alist= entry is added at module load: - -#+begin_src emacs-lisp -(add-to-list 'display-buffer-alist - '("\\`claude \\[" - (display-buffer-in-side-window) - (side . right) - (window-width . 0.5) - (dedicated . t))) -#+end_src - -** Data Flow - -On =M-x cj/ai-vterm=: - -1. Pick a project via =completing-read=. Display in =~/relative= form. Return absolute path. -2. Compute buffer name: =claude []=. -3. Branch: - - *Buffer exists with live process* → =display-buffer= it. Side-window rule routes it to the right slot. - - *Buffer exists, dead process* → kill it (log last 200 chars to =*Messages*=), then fall through to create. - - *No buffer* → =let=-bind =default-directory= to picked dir and =vterm-buffer-name= to computed name; call =(vterm)=. After process is live, send =claude "Read .ai/protocols.org and follow all instructions."= via =vterm-send-string= + =vterm-send-return=. -4. =select-window= on the displayed window so point lands in Claude. =C-u= prefix shows without selecting. - -After this, all navigation is handled by existing global bindings: Shift-arrows (windmove) for focus, C-M-arrows (=buffer-move=) for directional side-swap. - -** Error Handling - -| Case | Response | -|------+----------| -| Picker cancelled (=quit=) | Silent no-op | -| No candidates found | =user-error= naming the search roots | -| Picked dir disappeared between scan and launch | =user-error= naming the path | -| Existing buffer with dead process | Kill + recreate; log last 200 chars | -| Side-window already showing a different =claude [...]= | =display-buffer= swaps which buffer occupies the slot; hidden one keeps running | -| =vterm= not installed | Module fails to load loudly (no graceful degradation) | - -** Per-project tmux sessions - -The launch command sent to a fresh AI-vterm shell is - -#+begin_example -tmux new-session -A -s -c '; exec bash' -#+end_example - -- =-A= reattaches to an existing session of the same name instead of creating a new one. So a second F9 on the same project after an Emacs crash brings the running Claude back without spawning a duplicate. -- =-s = names the session after the project's directory basename. =tmux ls= shows the active sessions by project name. -- =-c = sets the start directory for new sessions (ignored on attach). -- =exec bash= tails the shell command so the tmux window survives Claude exiting -- the session stays alive with a bare prompt for recovery, and reattach is unaffected. - -** Tmux Auto-Launch Suppression - -The existing =cj/vterm-launch-tmux= on =vterm-mode-hook= types =tmux\n= unconditionally for any vterm buffer. AI-vterm =let='s a dynamic =cj/--ai-vterm-suppress-tmux= flag around =(vterm)= so the hook skips its bare =tmux\n= and the project-named launch command (above) runs instead. - -** Testing - -Pure helpers tested against real inputs: - -- =cj/--ai-vterm-buffer-name= — Normal, Boundary (trailing slash, dot-prefix dirs, spaces in basenames), Error (degenerate paths). -- =cj/--ai-vterm-candidates= — temp directory tree built with =make-temp-file= + =make-directory=, fake =.ai/protocols.org= markers. Assert returned paths, ignored entries. - -Internal with mocked boundary: - -- =cj/--ai-vterm-show-or-create= — =cl-letf= on =vterm= to skip process spawn; assert buffer name, =default-directory=, claude argv via captured =vterm-send-string= calls. Two branches (exists vs creates) tested with mocked =process-live-p=. - -Display rule: - -- After =add-to-list=, =display-buffer= on a buffer named =claude [test]= lands in a window with =(window-parameter w 'window-side) = 'right=. - -Test files: - -- =tests/test-ai-vterm--candidates.el= -- =tests/test-ai-vterm--buffer-name.el= -- =tests/test-ai-vterm--show-or-create.el= -- =tests/test-ai-vterm--display-rule.el= - -Smoke test (=:slow= tag, excluded from default suite): launch against a fixture, verify live process. - -* Open Questions - -- [ ] Default split width — 50/50 vs 60/40 weighted to code. Starting with 50/50. -- [X] Keybinding — F9. Replaces the prior =cj/toggle-gptel= binding on F9; gptel moves to C-F9. - -* Next Steps - -- TDD implementation in this order: =buffer-name= → =candidates= → =show-or-create= → display rule → interactive entry. -- Wire into =init.el= after =eshell-vterm-config=. -- Pick a keybinding once the command is shipped. diff --git a/docs/specs/ai-vterm-spec.org b/docs/specs/ai-vterm-spec.org new file mode 100644 index 00000000..7015a862 --- /dev/null +++ b/docs/specs/ai-vterm-spec.org @@ -0,0 +1,167 @@ +#+TITLE: Design: ai-vterm — in-Emacs Claude launcher +#+AUTHOR: Craig Jennings +#+DATE: 2026-05-07 +#+OPTIONS: toc:nil num:nil +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* SUPERSEDED Design: ai-vterm — in-Emacs Claude launcher +:PROPERTIES: +:ID: 3abd0270-e87c-42b7-9b3a-ef60300db99d +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword SUPERSEDED from existing :STATUS: superseded + -superseded filename (Craig's prior determination) + +* Status + +Draft. + +* Problem + +Claude Code currently launches outside Emacs via the =ai= shell script, which builds candidate projects from =~/.emacs.d=, =~/code/*=, =~/projects/*= (anything with =.ai/protocols.org=), opens each in a tmux window, and runs =claude "Read .ai/protocols.org and follow all instructions."= per project. The shell-out pulls focus to a terminal, and tmux's horizontal split is the wrong shape for a code-on-left, Claude-on-right reading layout. + +The in-Emacs alternative today is =vterm-toggle= at F12, which uses a horizontal bottom split via =display-buffer-at-bottom=. No project picker, no per-project session model, no vertical split. + +Building this in Emacs eliminates the context switch and gives the side-by-side layout that matches how the work is actually read. + +* Non-Goals + +- Replicating the =ai= script's git prep / auto-pull. Phase A.0 of the startup workflow already handles pulls at session start. +- A multi-project session switcher with its own UI. =consult-buffer= and the buffer list already navigate between =claude [...]= buffers. +- Replacing =vterm-toggle= at F12. The existing bottom-split flow stays for non-AI shells. +- Tab-bar or frame-per-project layouts. +- Auto-launching tmux inside the AI vterm. Claude under tmux adds a session-management layer for no benefit here. + +* Approaches Considered + +** Recommended: wrap =vterm= directly with per-project named buffers + a parallel display rule + +A new module =modules/ai-vterm.el= adds a command that picks a Claude-template project, opens (or reuses) a vterm buffer named =claude []=, and lets a new =display-buffer-alist= entry route any buffer matching that prefix to a right-side window. Multiple projects produce multiple coexisting buffers, all sharing the same right-side slot. Switching among them is a buffer-switch, not a kill-and-recreate. + +Pros: +- Same package (=vterm=) as the existing config. +- Per-project buffers run simultaneously without conflict. +- Right-side placement is one =display-buffer-alist= entry. +- Existing windmove (Shift-arrows) handles code↔Claude focus toggling. =buffer-move= (C-M-arrows) handles side-swap. Neither needs new bindings. + +Cons: +- Re-implements toggle/show-hide logic that =vterm-toggle= would handle for free. Acceptable because =vterm-toggle= is built around one toggle-able buffer, and the per-project model is what's wanted. + +** Rejected: wrap =vterm-toggle= + +=vterm-toggle='s contract is one buffer toggled visible/hidden. Per-project buffers running simultaneously is outside that contract. Wrapping it would mean fighting the abstraction. + +** Rejected: project-per-tab via =tab-bar-mode= + +Each project gets its own tab. Matches the =ai= / tmux model cleanly, but adds tab-bar UI that isn't in current use. Bigger lifestyle change for a one-window task. + +** Rejected: frame-per-project + +Each Claude session opens in a new Emacs frame. Hyprland-native, clean isolation, but frame creation under Wayland has historical jank, and it breaks the easy windmove flow between code and Claude. + +** Rejected: window-configuration-per-project + +Save and restore named window configs (code buffers + Claude vterm together). Preserves the surrounding thinking environment, but window configs go stale when buffers die, and it adds a parallel mechanism to project.el. Overkill for v1. + +* Design + +** Architecture + +New module =modules/ai-vterm.el=. Required after =eshell-vterm-config= in =init.el= so =vterm= is loaded. + +Components: + +| Function | Kind | Responsibility | +|----------+------+----------------| +| =cj/--ai-vterm-candidates= | pure | Walks =~/.emacs.d=, =~/code/*=, =~/projects/*=; returns abs paths containing =.ai/protocols.org= | +| =cj/--ai-vterm-pick-project= | interactive helper | =completing-read= over candidates; returns picked path | +| =cj/--ai-vterm-buffer-name= | pure | =(format "claude [%s]" basename)= | +| =cj/--ai-vterm-show-or-create= | internal | Given dir + name: display existing buffer, or create vterm + send claude command | +| =cj/ai-vterm= | interactive entry | Composes picker + show-or-create | + +The =display-buffer-alist= entry is added at module load: + +#+begin_src emacs-lisp +(add-to-list 'display-buffer-alist + '("\\`claude \\[" + (display-buffer-in-side-window) + (side . right) + (window-width . 0.5) + (dedicated . t))) +#+end_src + +** Data Flow + +On =M-x cj/ai-vterm=: + +1. Pick a project via =completing-read=. Display in =~/relative= form. Return absolute path. +2. Compute buffer name: =claude []=. +3. Branch: + - *Buffer exists with live process* → =display-buffer= it. Side-window rule routes it to the right slot. + - *Buffer exists, dead process* → kill it (log last 200 chars to =*Messages*=), then fall through to create. + - *No buffer* → =let=-bind =default-directory= to picked dir and =vterm-buffer-name= to computed name; call =(vterm)=. After process is live, send =claude "Read .ai/protocols.org and follow all instructions."= via =vterm-send-string= + =vterm-send-return=. +4. =select-window= on the displayed window so point lands in Claude. =C-u= prefix shows without selecting. + +After this, all navigation is handled by existing global bindings: Shift-arrows (windmove) for focus, C-M-arrows (=buffer-move=) for directional side-swap. + +** Error Handling + +| Case | Response | +|------+----------| +| Picker cancelled (=quit=) | Silent no-op | +| No candidates found | =user-error= naming the search roots | +| Picked dir disappeared between scan and launch | =user-error= naming the path | +| Existing buffer with dead process | Kill + recreate; log last 200 chars | +| Side-window already showing a different =claude [...]= | =display-buffer= swaps which buffer occupies the slot; hidden one keeps running | +| =vterm= not installed | Module fails to load loudly (no graceful degradation) | + +** Per-project tmux sessions + +The launch command sent to a fresh AI-vterm shell is + +#+begin_example +tmux new-session -A -s -c '; exec bash' +#+end_example + +- =-A= reattaches to an existing session of the same name instead of creating a new one. So a second F9 on the same project after an Emacs crash brings the running Claude back without spawning a duplicate. +- =-s = names the session after the project's directory basename. =tmux ls= shows the active sessions by project name. +- =-c = sets the start directory for new sessions (ignored on attach). +- =exec bash= tails the shell command so the tmux window survives Claude exiting -- the session stays alive with a bare prompt for recovery, and reattach is unaffected. + +** Tmux Auto-Launch Suppression + +The existing =cj/vterm-launch-tmux= on =vterm-mode-hook= types =tmux\n= unconditionally for any vterm buffer. AI-vterm =let='s a dynamic =cj/--ai-vterm-suppress-tmux= flag around =(vterm)= so the hook skips its bare =tmux\n= and the project-named launch command (above) runs instead. + +** Testing + +Pure helpers tested against real inputs: + +- =cj/--ai-vterm-buffer-name= — Normal, Boundary (trailing slash, dot-prefix dirs, spaces in basenames), Error (degenerate paths). +- =cj/--ai-vterm-candidates= — temp directory tree built with =make-temp-file= + =make-directory=, fake =.ai/protocols.org= markers. Assert returned paths, ignored entries. + +Internal with mocked boundary: + +- =cj/--ai-vterm-show-or-create= — =cl-letf= on =vterm= to skip process spawn; assert buffer name, =default-directory=, claude argv via captured =vterm-send-string= calls. Two branches (exists vs creates) tested with mocked =process-live-p=. + +Display rule: + +- After =add-to-list=, =display-buffer= on a buffer named =claude [test]= lands in a window with =(window-parameter w 'window-side) = 'right=. + +Test files: + +- =tests/test-ai-vterm--candidates.el= +- =tests/test-ai-vterm--buffer-name.el= +- =tests/test-ai-vterm--show-or-create.el= +- =tests/test-ai-vterm--display-rule.el= + +Smoke test (=:slow= tag, excluded from default suite): launch against a fixture, verify live process. + +* Open Questions + +- [ ] Default split width — 50/50 vs 60/40 weighted to code. Starting with 50/50. +- [X] Keybinding — F9. Replaces the prior =cj/toggle-gptel= binding on F9; gptel moves to C-F9. + +* Next Steps + +- TDD implementation in this order: =buffer-name= → =candidates= → =show-or-create= → display rule → interactive entry. +- Wire into =init.el= after =eshell-vterm-config=. +- Pick a keybinding once the command is shipped. diff --git a/docs/specs/cache-helper-design-spec-implemented.org b/docs/specs/cache-helper-design-spec-implemented.org deleted file mode 100644 index 27c818dc..00000000 --- a/docs/specs/cache-helper-design-spec-implemented.org +++ /dev/null @@ -1,169 +0,0 @@ -:PROPERTIES: -:ID: 647c5101-21c2-47bb-aaa7-72c757f45fb7 -:STATUS: implemented -:END: -#+TITLE: Cache Helper Design Addendum -#+AUTHOR: Craig Jennings -#+DATE: 2026-05-10 - -* Status - -Phase 5 design addendum to [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec-doing.org]]. Specifies the cache API to extract before any code moves. - -* Problem - -Two modules carry a parallel cache implementation today: - -- =modules/org-agenda-config.el= caches the agenda file list. -- =modules/org-refile-config.el= caches refile targets. - -Both share the same shape (lines map between modules): - -| Element | Purpose | -|---------|---------| -| =VAR-cache= | the cached value | -| =VAR-cache-time= | float-time when last built | -| =VAR-cache-ttl= | seconds to retain (default 3600) | -| =VAR-building= | non-nil while an async build is in progress | -| =cj/build-VAR (&optional force-rebuild)= | "use cache if valid, else rebuild" | - -The build function does: - -1. Check whether cache is valid: cache and cache-time set, FORCE-REBUILD nil, age < TTL. -2. If valid: assign to the consumer's variable, log cache-hit, return. -3. If a background build is running: log "waiting...", continue (no second build). -4. Otherwise: set building flag, run the rebuild closure inside =condition-case=, on success update cache+time, on error log message; the =unwind-protect= clears the building flag. - -The consumer's variable (e.g. =org-agenda-files=, =org-refile-targets=) is updated as a side effect. - -A separate cache pattern exists in =modules/modeline-config.el= for VC info, but it's *buffer-local*, *key-based* (the file path), and not TTL-based. It will not migrate to the same helper -- see "Out of scope" below. - -* Goals - -- One source of truth for the TTL+building build pattern. -- Consumers shrink from ~30 lines of cache plumbing to a single =cj/cache-value-or-rebuild= call. -- Behavior preserved for agenda and refile. -- Tests cover rebuild / TTL hit / TTL miss / nil cache value / build error / building-flag cleanup. - -* Out of scope - -- Modeline VC cache (=cj/modeline-vc-cache-*=). Buffer-local + key-based + non-TTL invalidation. Different lifecycle. The spec calls this out explicitly: consider only after global cache behavior is stable. Defer to a future round. -- Generic memoization. We're not introducing function memoization or LRU. Just the specific "rebuild a long-running computation behind a TTL" pattern. - -* API - -** =cj/cache-make= - -#+begin_src emacs-lisp -(cj/cache-make &key ttl) -#+end_src - -Return a fresh cache state object. TTL is in seconds; defaults to 3600. - -The state is a plist with keys =:value=, =:time=, =:ttl=, =:building=. Consumers store the state in a single =defvar=. - -** =cj/cache-valid-p= - -#+begin_src emacs-lisp -(cj/cache-valid-p cache) -#+end_src - -Return non-nil when CACHE has a non-nil value, a non-nil time, and (now - time) < ttl. - -** =cj/cache-value-or-rebuild= - -#+begin_src emacs-lisp -(cj/cache-value-or-rebuild cache build-fn - &key force-rebuild - on-hit - on-build-start - on-build-success - on-build-error) -#+end_src - -The main entry point. Returns the cached value when valid (and FORCE-REBUILD is nil); otherwise calls BUILD-FN to compute a new value, updates CACHE, and returns the result. - -The four optional callbacks let the consumer log without the helper printing on its behalf: - -- =on-hit (value)= -- the helper found a valid cache. -- =on-build-start ()= -- about to call BUILD-FN. -- =on-build-success (value)= -- BUILD-FN returned cleanly. -- =on-build-error (err)= -- BUILD-FN signaled. After this fires the helper rethrows so the caller sees the error. - -The =:building= flag is set before BUILD-FN runs and cleared inside an =unwind-protect= regardless of outcome. The flag is exposed read-only via =cj/cache-building-p= so callers can log "build in progress" without poking at the plist directly. - -** =cj/cache-building-p= - -#+begin_src emacs-lisp -(cj/cache-building-p cache) -#+end_src - -Return non-nil when a build is currently in progress on CACHE. Used by callers that want to log "waiting for background build" before invoking =cj/cache-value-or-rebuild=. - -** =cj/cache-invalidate= - -#+begin_src emacs-lisp -(cj/cache-invalidate cache) -#+end_src - -Reset the cache to "no value, no time". TTL is preserved. - -* Consumer Shape (after migration) - -The agenda module's ~30 lines of cache plumbing become roughly: - -#+begin_src emacs-lisp -(defvar cj/--org-agenda-files-cache (cj/cache-make :ttl 3600)) - -(defun cj/build-org-agenda-list (&optional force-rebuild) - "..." - (interactive "P") - (when (cj/cache-building-p cj/--org-agenda-files-cache) - (cj/log-silently "Waiting for background agenda build to complete...")) - (let ((files - (cj/cache-value-or-rebuild - cj/--org-agenda-files-cache - (lambda () (cj/--scan-org-agenda-files)) - :force-rebuild force-rebuild - :on-hit (lambda (v) (cj/log-silently - "Using cached agenda files (%d files)" (length v))) - :on-build-start (lambda () (cj/log-silently - "Rebuilding agenda files (slow)...")) - :on-build-success (lambda (v) (cj/log-silently - "Built agenda files (%d files)" - (length v))) - :on-build-error (lambda (err) (cj/log-silently - "Agenda build failed: %s" err))))) - (setq org-agenda-files files) - files)) -#+end_src - -The =cj/--scan-org-agenda-files= helper holds the slow filesystem walk; the existing in-place expression body moves there with no behavior change. - -* Migration order - -Per the spec: agenda first, refile second. Each is its own commit: - -1. Add =modules/cj-cache.el= with the API and tests. No call-site changes. -2. Migrate =org-agenda-config.el= to the helper. Verify behavior with the existing async-cache tests. -3. Migrate =org-refile-config.el= the same way. - -* Testing - -For the helper itself, =tests/test-cj-cache.el=: - -- Normal: hit returns cached value; miss calls BUILD-FN. -- TTL: build at t=0, request at t=ttl-1 hits; request at t=ttl+1 rebuilds. -- FORCE-REBUILD wins over a valid cache. -- nil from BUILD-FN is stored (cache-valid-p returns t). This matches today's behavior -- a build that legitimately produces nil should not loop. *Decision point*: the current implementations actually treat "cache value nil" as "cache invalid" (line 131 of agenda, line 66 of refile both check =(and cache cache-time ...)=). Preserve that to avoid a behavior change: the new helper's =cj/cache-valid-p= treats a nil :value as "not valid". That's the safer default; consumers that need "nil is a real value" can migrate to a sentinel later. -- :building flag is set during BUILD-FN, cleared after success. -- :building flag is cleared even when BUILD-FN signals. -- Each callback fires once per appropriate path (hit / start / success / error). - -For the migrated consumers: the existing async-cache and rebuild tests run unchanged after the migration. No new test files for agenda/refile are required as part of Phase 5 itself; they got their tests when the original cache was added. - -* Risks - -- *Behavior drift on cache-hit logging.* The current code logs "Using cached agenda files (N files)" via =cj/log-silently=. The migration preserves that exact message via =:on-hit=. Verify by tail-ing =*Messages*= during a manual smoke test. -- *Building-flag leak.* The current code uses =unwind-protect= to clear =VAR-building= even on error. The helper does the same. The test "building flag cleared on error" pins this contract. -- *Async timer interaction.* Both modules schedule background builds via =run-with-idle-timer=. The migration leaves those scheduling forms in the consumer; only the cache-or-build core moves. No changes to startup timing. diff --git a/docs/specs/cache-helper-design-spec.org b/docs/specs/cache-helper-design-spec.org new file mode 100644 index 00000000..5bfb661b --- /dev/null +++ b/docs/specs/cache-helper-design-spec.org @@ -0,0 +1,173 @@ +#+TITLE: Cache Helper Design Addendum +#+AUTHOR: Craig Jennings +#+DATE: 2026-05-10 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Cache Helper Design Addendum +:PROPERTIES: +:ID: 647c5101-21c2-47bb-aaa7-72c757f45fb7 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword IMPLEMENTED from existing :STATUS: implemented + -implemented filename (Craig's prior determination) + +* Status + +Phase 5 design addendum to [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec.org]]. Specifies the cache API to extract before any code moves. + +* Problem + +Two modules carry a parallel cache implementation today: + +- =modules/org-agenda-config.el= caches the agenda file list. +- =modules/org-refile-config.el= caches refile targets. + +Both share the same shape (lines map between modules): + +| Element | Purpose | +|---------|---------| +| =VAR-cache= | the cached value | +| =VAR-cache-time= | float-time when last built | +| =VAR-cache-ttl= | seconds to retain (default 3600) | +| =VAR-building= | non-nil while an async build is in progress | +| =cj/build-VAR (&optional force-rebuild)= | "use cache if valid, else rebuild" | + +The build function does: + +1. Check whether cache is valid: cache and cache-time set, FORCE-REBUILD nil, age < TTL. +2. If valid: assign to the consumer's variable, log cache-hit, return. +3. If a background build is running: log "waiting...", continue (no second build). +4. Otherwise: set building flag, run the rebuild closure inside =condition-case=, on success update cache+time, on error log message; the =unwind-protect= clears the building flag. + +The consumer's variable (e.g. =org-agenda-files=, =org-refile-targets=) is updated as a side effect. + +A separate cache pattern exists in =modules/modeline-config.el= for VC info, but it's *buffer-local*, *key-based* (the file path), and not TTL-based. It will not migrate to the same helper -- see "Out of scope" below. + +* Goals + +- One source of truth for the TTL+building build pattern. +- Consumers shrink from ~30 lines of cache plumbing to a single =cj/cache-value-or-rebuild= call. +- Behavior preserved for agenda and refile. +- Tests cover rebuild / TTL hit / TTL miss / nil cache value / build error / building-flag cleanup. + +* Out of scope + +- Modeline VC cache (=cj/modeline-vc-cache-*=). Buffer-local + key-based + non-TTL invalidation. Different lifecycle. The spec calls this out explicitly: consider only after global cache behavior is stable. Defer to a future round. +- Generic memoization. We're not introducing function memoization or LRU. Just the specific "rebuild a long-running computation behind a TTL" pattern. + +* API + +** =cj/cache-make= + +#+begin_src emacs-lisp +(cj/cache-make &key ttl) +#+end_src + +Return a fresh cache state object. TTL is in seconds; defaults to 3600. + +The state is a plist with keys =:value=, =:time=, =:ttl=, =:building=. Consumers store the state in a single =defvar=. + +** =cj/cache-valid-p= + +#+begin_src emacs-lisp +(cj/cache-valid-p cache) +#+end_src + +Return non-nil when CACHE has a non-nil value, a non-nil time, and (now - time) < ttl. + +** =cj/cache-value-or-rebuild= + +#+begin_src emacs-lisp +(cj/cache-value-or-rebuild cache build-fn + &key force-rebuild + on-hit + on-build-start + on-build-success + on-build-error) +#+end_src + +The main entry point. Returns the cached value when valid (and FORCE-REBUILD is nil); otherwise calls BUILD-FN to compute a new value, updates CACHE, and returns the result. + +The four optional callbacks let the consumer log without the helper printing on its behalf: + +- =on-hit (value)= -- the helper found a valid cache. +- =on-build-start ()= -- about to call BUILD-FN. +- =on-build-success (value)= -- BUILD-FN returned cleanly. +- =on-build-error (err)= -- BUILD-FN signaled. After this fires the helper rethrows so the caller sees the error. + +The =:building= flag is set before BUILD-FN runs and cleared inside an =unwind-protect= regardless of outcome. The flag is exposed read-only via =cj/cache-building-p= so callers can log "build in progress" without poking at the plist directly. + +** =cj/cache-building-p= + +#+begin_src emacs-lisp +(cj/cache-building-p cache) +#+end_src + +Return non-nil when a build is currently in progress on CACHE. Used by callers that want to log "waiting for background build" before invoking =cj/cache-value-or-rebuild=. + +** =cj/cache-invalidate= + +#+begin_src emacs-lisp +(cj/cache-invalidate cache) +#+end_src + +Reset the cache to "no value, no time". TTL is preserved. + +* Consumer Shape (after migration) + +The agenda module's ~30 lines of cache plumbing become roughly: + +#+begin_src emacs-lisp +(defvar cj/--org-agenda-files-cache (cj/cache-make :ttl 3600)) + +(defun cj/build-org-agenda-list (&optional force-rebuild) + "..." + (interactive "P") + (when (cj/cache-building-p cj/--org-agenda-files-cache) + (cj/log-silently "Waiting for background agenda build to complete...")) + (let ((files + (cj/cache-value-or-rebuild + cj/--org-agenda-files-cache + (lambda () (cj/--scan-org-agenda-files)) + :force-rebuild force-rebuild + :on-hit (lambda (v) (cj/log-silently + "Using cached agenda files (%d files)" (length v))) + :on-build-start (lambda () (cj/log-silently + "Rebuilding agenda files (slow)...")) + :on-build-success (lambda (v) (cj/log-silently + "Built agenda files (%d files)" + (length v))) + :on-build-error (lambda (err) (cj/log-silently + "Agenda build failed: %s" err))))) + (setq org-agenda-files files) + files)) +#+end_src + +The =cj/--scan-org-agenda-files= helper holds the slow filesystem walk; the existing in-place expression body moves there with no behavior change. + +* Migration order + +Per the spec: agenda first, refile second. Each is its own commit: + +1. Add =modules/cj-cache.el= with the API and tests. No call-site changes. +2. Migrate =org-agenda-config.el= to the helper. Verify behavior with the existing async-cache tests. +3. Migrate =org-refile-config.el= the same way. + +* Testing + +For the helper itself, =tests/test-cj-cache.el=: + +- Normal: hit returns cached value; miss calls BUILD-FN. +- TTL: build at t=0, request at t=ttl-1 hits; request at t=ttl+1 rebuilds. +- FORCE-REBUILD wins over a valid cache. +- nil from BUILD-FN is stored (cache-valid-p returns t). This matches today's behavior -- a build that legitimately produces nil should not loop. *Decision point*: the current implementations actually treat "cache value nil" as "cache invalid" (line 131 of agenda, line 66 of refile both check =(and cache cache-time ...)=). Preserve that to avoid a behavior change: the new helper's =cj/cache-valid-p= treats a nil :value as "not valid". That's the safer default; consumers that need "nil is a real value" can migrate to a sentinel later. +- :building flag is set during BUILD-FN, cleared after success. +- :building flag is cleared even when BUILD-FN signals. +- Each callback fires once per appropriate path (hit / start / success / error). + +For the migrated consumers: the existing async-cache and rebuild tests run unchanged after the migration. No new test files for agenda/refile are required as part of Phase 5 itself; they got their tests when the original cache was added. + +* Risks + +- *Behavior drift on cache-hit logging.* The current code logs "Using cached agenda files (N files)" via =cj/log-silently=. The migration preserves that exact message via =:on-hit=. Verify by tail-ing =*Messages*= during a manual smoke test. +- *Building-flag leak.* The current code uses =unwind-protect= to clear =VAR-building= even on error. The helper does the same. The test "building flag cleared on error" pins this contract. +- *Async timer interaction.* Both modules schedule background builds via =run-with-idle-timer=. The migration leaves those scheduling forms in the consumer; only the cache-or-build core moves. No changes to startup timing. diff --git a/docs/specs/company-to-corfu-migration-spec.org b/docs/specs/company-to-corfu-migration-spec.org index a7b059a3..ef094937 100644 --- a/docs/specs/company-to-corfu-migration-spec.org +++ b/docs/specs/company-to-corfu-migration-spec.org @@ -1,11 +1,15 @@ -:PROPERTIES: -:ID: 68733ba2-37a7-4a7b-bfaa-b845d82ff1e7 -:STATUS: not-started -:END: #+TITLE: Design: Migrate from Company to Corfu (with prescient integration) #+AUTHOR: Craig Jennings #+DATE: 2026-05-15 #+OPTIONS: toc:nil num:nil +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Design: Migrate from Company to Corfu (with prescient integration) +:PROPERTIES: +:ID: 68733ba2-37a7-4a7b-bfaa-b845d82ff1e7 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from existing :STATUS: not-started * Status diff --git a/docs/specs/coverage-spec-implemented.org b/docs/specs/coverage-spec-implemented.org deleted file mode 100644 index 65734fb3..00000000 --- a/docs/specs/coverage-spec-implemented.org +++ /dev/null @@ -1,210 +0,0 @@ -:PROPERTIES: -:ID: 7d7f4486-fad7-4f0a-bd9a-775bd4cd8f7e -:STATUS: implemented -:END: -#+TITLE: Design: Coverage Reporting -#+AUTHOR: Craig Jennings -#+DATE: 2026-04-22 - -* Status - -Implemented for Elisp. - -The shipped path is local-first: =make coverage= produces -=.coverage/simplecov.json= with Undercover, and =cj/coverage-report= reads that -artifact to show either diff-aware coverage or a whole-project summary from -Emacs. Python, TypeScript, and Go backends remain future work. - -* Problem - -Before this work, there was no quick way to answer "are the lines I just -changed actually covered by tests?" Line-level coverage for the *whole* -project was also missing, and there was no local artifact to inspect. - -The primary user-facing need is the first one: point-in-time feedback on -in-flight changes, triggered from Emacs. The implemented system also supports a -whole-project summary and writes a local SimpleCov JSON artifact. - -The tooling should be pluggable so the same workflow covers Elisp today and Python, TypeScript, and Go later — without rebuilding the UI for each language. - -* Non-Goals - -- Continuous in-buffer overlays (fringe marks, line highlights). Parked over performance concerns. -- Mutation testing or any signal other than line coverage. -- CI integration beyond emitting a simplecov JSON artifact. No coveralls, no GitHub Actions wiring. -- Shadowing or replacing existing test-running commands (=make test=, =make test-file=, etc.). - -* Approaches Considered - -** Recommended: diff-aware report with pluggable backends - -Core engine reads a simplecov JSON file, shells to ~git diff~ at a selectable scope, intersects, and displays the result in a compilation-mode-derived buffer. Language-specific "backends" each produce simplecov in their own way and register themselves with the core. - -*Pros:* Directly serves the primary use case. Simplecov is broadly supported across language coverage tools, and Undercover's ~:merge-report t~ option works for simplecov (but not for LCOV), which is essential for the per-file coverage-run strategy. Compilation-mode inheritance gives free =next-error= / =previous-error= navigation. - -*Note on format:* An earlier draft of this design used LCOV. That was changed to simplecov after discovering that Undercover's LCOV writer does not implement report-merging — per-file coverage runs would require custom merge logic or an external ~lcov~ tool. Simplecov's native merge-report support made it the cleaner fit without changing anything about the pluggable backend story. - -*Cons:* More code than a "just run coverage and read the output" approach. Backend registry adds one layer of indirection (small — ~30 lines). - -** Rejected: non-interactive pre-commit hook - -Would run coverage on every commit and report uncovered-changed-lines to stderr. Literal fit for the use case but adds a long delay to every commit and offers no way to inspect non-staged scopes. - -** Rejected: coverage as a =review-code= skill criterion - -Would fold coverage into the existing pre-commit review skill. Clean in principle, but couples =review-code= to Emacs-specific tooling and makes ad-hoc inspection (outside a review) awkward. - -** Rejected: mutation testing instead of line coverage - -Stronger signal than coverage but minutes-to-hours runtime on the current 265-file suite, and no polished Elisp tool exists. Different conversation. - -* Design - -** Architecture - -Three files: - -- =modules/coverage-core.el= — engine + backend registry + user-facing command. Language-agnostic. -- =modules/coverage-elisp.el= — the initial backend. Registers itself on load. -- (Future) =modules/coverage-python.el=, =coverage-typescript.el=, =coverage-go.el= — each ~30 lines, self-registering. - -=init.el= requires the core and the active backends. - -*** Elisp coverage producer - -For the Elisp backend, =make coverage= is the only supported producer of the -coverage artifact. It removes stale compiled files for instrumented sources, -then runs each unit test file in its own batch Emacs process. Before loading -the test file, the Makefile loads =tests/run-coverage-file.el=, which -initializes packages and configures Undercover: - -#+begin_src emacs-lisp -(undercover "modules/*.el" - "gptel-tools/*.el" - (:report-format 'simplecov) - (:report-file ".coverage/simplecov.json") - (:merge-report t) - (:send-report nil)) -#+end_src - -Undercover is therefore the instrumentation layer: it instruments -=modules/*.el= and =gptel-tools/*.el=, records Edebug stop-point hits while -tests execute, and writes the line hit arrays. SimpleCov is the local -interchange format consumed by the rest of this design. The split-per-test-file -Makefile strategy depends on =:merge-report t=; Undercover can merge SimpleCov -reports across separate Emacs processes, while its LCOV writer cannot merge -reports. This is the concrete reason the artifact is -=.coverage/simplecov.json= rather than =coverage.lcov=. - -The Makefile excludes tests that are incompatible with instrumented source -loading, such as byte-compilation checks. If =.coverage/simplecov.json= is not -created, the coverage run is considered failed; downstream report commands -should not infer partial coverage from a missing artifact. - -*** Backend protocol - -Each backend is a plist registered into =cj/coverage-backends=: - -#+begin_src emacs-lisp -(:name 'elisp - :detect (lambda () ...) ; non-nil if current project matches - :run (lambda (cb) ...) ; kick off coverage build; invoke CB with report path - :report-path (lambda () ...)) ; where the simplecov JSON lives (for re-reading without running) -#+end_src - -Detection precedence: =.dir-locals.el= override (=cj/coverage-backend= set to a backend name), then project-root fingerprints (=go.mod=, =pyproject.toml=, =package.json=, =.el= files + Makefile, etc.). First =:detect= that matches wins. No silent fallback — if nothing matches, the command errors with guidance. - -*** Pure helpers - -- =cj/--coverage-parse-simplecov FILE= → hash-table ={file → covered-line-set}=. -- =cj/--coverage-changed-lines SCOPE BASE= → hash-table ={file → changed-line-set}= by shelling a =git diff --unified=0= for the selected scope and parsing hunk headers. -- =cj/--coverage-intersect COVERED CHANGED= → per-file records with three buckets: covered, uncovered, not-tracked. - -These helpers are pure and covered by focused ERT tests. - -** Data Flow - -1. User invokes =cj/coverage-report= (bound to =F7=). -2. Core resolves the backend for the current project. -3. =completing-read= prompts for scope: - - "Working tree — all uncommitted changes" - - "Staged — about to commit" - - "Branch vs parent" (uses =@{upstream}= unless a caller passes an explicit base to the helper) - - "Branch vs main" (explicit) - - "Whole project — all executable lines" -4. If =simplecov.json= is missing, prompt to run coverage. A prefix argument - (=C-u F7=) forces a fresh run. Otherwise the existing report is used as-is. -5. Parse simplecov, compute changed lines or all executable lines, intersect. -6. Display a report buffer in a mode derived from =compilation-mode=. - -** Persistence - -- =.coverage/simplecov.json= at the project root, gitignored. Overwritten on each run. -- No long-term storage. Historical tracking is explicitly out of scope for v1. - -** Error Handling - -*Pre-flight:* -- No backend matches → =user-error= with instructions to register a backend or set =.dir-locals.el=. -- =.dir-locals.el= names an unknown backend → error listing registered backends. -- Not in a git repository → error; don't swallow git's stderr. -- Branch comparison on a repo with no common ancestor (orphan branch, shallow - clone missing the fork point, or missing upstream) reports the underlying git - failure. - -*During the coverage run:* -- Backend =:run= fails (test failure, Make error) → keep the =compile= buffer visible, do *not* proceed to display a report. Partial data is worse than no data. -- Run completes but no simplecov.json produced → error naming the expected path. - -*Post-flight classification:* three buckets, not two. -- *Covered* — changed line in the simplecov covered-line set. -- *Uncovered* — changed line in a tracked file but not covered. -- *Not tracked* — changed file isn't in the simplecov data at all (test files, READMEs, config). Reported separately — don't conflate "coverage didn't look here" with "tests didn't exercise this code." - -*Happy-path degenerates:* -- Zero changed lines in scope → "No changes in this scope; nothing to report." -- All changed lines covered → "N of N changed lines covered. " - -** Keybindings - -*Global:* -- =F7= → =cj/coverage-report= (prompts scope, shows report). -- =C-u F7= → force re-run regardless of report freshness. - -*In the report buffer* (compilation-mode derived, most inherited for free): -- =RET= → jump to source under point. -- =n= / =p= → next / previous uncovered line. -- =q= → bury buffer. - -*Globally available via compilation-mode integration:* -- =M-g n= / =M-g p= → =next-error= / =previous-error= on the last compilation buffer. -- =C-x `= → visit next uncovered line without leaving the current buffer. - -The =F4=–=F7= developer block currently uses =F6= for project-aware test -dispatch and =F7= for coverage. - -** Testing - -*Pure helpers, fully tested* (Normal / Boundary / Error for each): -- =cj/--coverage-parse-simplecov= — handcrafted simplecov JSON in temp files; empty object, all-null coverage arrays, spaces in filenames, multiple test-name keys unioned, malformed JSON. -- =cj/--coverage-simplecov-executable-lines= — whole-project executable-line set, including zero-hit executable lines. -- =cj/--coverage-changed-lines= — =cl-letf= over =shell-command-to-string= to return canned =git diff= output; single hunk, new-file hunk, deletion-only hunk, binary marker, no-diff case. -- =cj/--coverage-intersect= — pure table-in / table-out; covered ⊇ changed, unknown files, nil/empty inputs. -- =cj/--coverage-format-report= and =cj/--coverage-format-summary= — report text and whole-project per-file summary. - -*Backend registry, structurally tested:* -- =cj/coverage-backend-for-project ROOT= — synthetic temp project roots with marker files; assert correct backend. Registration-order test: two backends match, first-registered wins. - -*Not tested:* -- =cj/coverage-report= interactive command — one smoke test with a prepared simplecov report and a stubbed git-diff. No tests for the prompt UI or the compilation-buffer display. -- The elisp backend's =:run= function — shells to =make coverage=; integration-test-shaped, low value, slow. Skipped by design. - -* Current Limitations - -- =make coverage= runs all unit test files except known instrumentation - conflicts. It does not try to select only tests related to changed modules. -- Existing reports are not checked for staleness. Use =C-u F7= or - =make coverage= when a fresh report matters. -- Only the Elisp backend is implemented. -- There is no CI coverage publishing. The generated - =.coverage/simplecov.json= file is local and gitignored. diff --git a/docs/specs/coverage-spec.org b/docs/specs/coverage-spec.org new file mode 100644 index 00000000..e2ac4b3c --- /dev/null +++ b/docs/specs/coverage-spec.org @@ -0,0 +1,214 @@ +#+TITLE: Design: Coverage Reporting +#+AUTHOR: Craig Jennings +#+DATE: 2026-04-22 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Design: Coverage Reporting +:PROPERTIES: +:ID: 7d7f4486-fad7-4f0a-bd9a-775bd4cd8f7e +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword IMPLEMENTED from existing :STATUS: implemented + -implemented filename (Craig's prior determination) + +* Status + +Implemented for Elisp. + +The shipped path is local-first: =make coverage= produces +=.coverage/simplecov.json= with Undercover, and =cj/coverage-report= reads that +artifact to show either diff-aware coverage or a whole-project summary from +Emacs. Python, TypeScript, and Go backends remain future work. + +* Problem + +Before this work, there was no quick way to answer "are the lines I just +changed actually covered by tests?" Line-level coverage for the *whole* +project was also missing, and there was no local artifact to inspect. + +The primary user-facing need is the first one: point-in-time feedback on +in-flight changes, triggered from Emacs. The implemented system also supports a +whole-project summary and writes a local SimpleCov JSON artifact. + +The tooling should be pluggable so the same workflow covers Elisp today and Python, TypeScript, and Go later — without rebuilding the UI for each language. + +* Non-Goals + +- Continuous in-buffer overlays (fringe marks, line highlights). Parked over performance concerns. +- Mutation testing or any signal other than line coverage. +- CI integration beyond emitting a simplecov JSON artifact. No coveralls, no GitHub Actions wiring. +- Shadowing or replacing existing test-running commands (=make test=, =make test-file=, etc.). + +* Approaches Considered + +** Recommended: diff-aware report with pluggable backends + +Core engine reads a simplecov JSON file, shells to ~git diff~ at a selectable scope, intersects, and displays the result in a compilation-mode-derived buffer. Language-specific "backends" each produce simplecov in their own way and register themselves with the core. + +*Pros:* Directly serves the primary use case. Simplecov is broadly supported across language coverage tools, and Undercover's ~:merge-report t~ option works for simplecov (but not for LCOV), which is essential for the per-file coverage-run strategy. Compilation-mode inheritance gives free =next-error= / =previous-error= navigation. + +*Note on format:* An earlier draft of this design used LCOV. That was changed to simplecov after discovering that Undercover's LCOV writer does not implement report-merging — per-file coverage runs would require custom merge logic or an external ~lcov~ tool. Simplecov's native merge-report support made it the cleaner fit without changing anything about the pluggable backend story. + +*Cons:* More code than a "just run coverage and read the output" approach. Backend registry adds one layer of indirection (small — ~30 lines). + +** Rejected: non-interactive pre-commit hook + +Would run coverage on every commit and report uncovered-changed-lines to stderr. Literal fit for the use case but adds a long delay to every commit and offers no way to inspect non-staged scopes. + +** Rejected: coverage as a =review-code= skill criterion + +Would fold coverage into the existing pre-commit review skill. Clean in principle, but couples =review-code= to Emacs-specific tooling and makes ad-hoc inspection (outside a review) awkward. + +** Rejected: mutation testing instead of line coverage + +Stronger signal than coverage but minutes-to-hours runtime on the current 265-file suite, and no polished Elisp tool exists. Different conversation. + +* Design + +** Architecture + +Three files: + +- =modules/coverage-core.el= — engine + backend registry + user-facing command. Language-agnostic. +- =modules/coverage-elisp.el= — the initial backend. Registers itself on load. +- (Future) =modules/coverage-python.el=, =coverage-typescript.el=, =coverage-go.el= — each ~30 lines, self-registering. + +=init.el= requires the core and the active backends. + +*** Elisp coverage producer + +For the Elisp backend, =make coverage= is the only supported producer of the +coverage artifact. It removes stale compiled files for instrumented sources, +then runs each unit test file in its own batch Emacs process. Before loading +the test file, the Makefile loads =tests/run-coverage-file.el=, which +initializes packages and configures Undercover: + +#+begin_src emacs-lisp +(undercover "modules/*.el" + "gptel-tools/*.el" + (:report-format 'simplecov) + (:report-file ".coverage/simplecov.json") + (:merge-report t) + (:send-report nil)) +#+end_src + +Undercover is therefore the instrumentation layer: it instruments +=modules/*.el= and =gptel-tools/*.el=, records Edebug stop-point hits while +tests execute, and writes the line hit arrays. SimpleCov is the local +interchange format consumed by the rest of this design. The split-per-test-file +Makefile strategy depends on =:merge-report t=; Undercover can merge SimpleCov +reports across separate Emacs processes, while its LCOV writer cannot merge +reports. This is the concrete reason the artifact is +=.coverage/simplecov.json= rather than =coverage.lcov=. + +The Makefile excludes tests that are incompatible with instrumented source +loading, such as byte-compilation checks. If =.coverage/simplecov.json= is not +created, the coverage run is considered failed; downstream report commands +should not infer partial coverage from a missing artifact. + +*** Backend protocol + +Each backend is a plist registered into =cj/coverage-backends=: + +#+begin_src emacs-lisp +(:name 'elisp + :detect (lambda () ...) ; non-nil if current project matches + :run (lambda (cb) ...) ; kick off coverage build; invoke CB with report path + :report-path (lambda () ...)) ; where the simplecov JSON lives (for re-reading without running) +#+end_src + +Detection precedence: =.dir-locals.el= override (=cj/coverage-backend= set to a backend name), then project-root fingerprints (=go.mod=, =pyproject.toml=, =package.json=, =.el= files + Makefile, etc.). First =:detect= that matches wins. No silent fallback — if nothing matches, the command errors with guidance. + +*** Pure helpers + +- =cj/--coverage-parse-simplecov FILE= → hash-table ={file → covered-line-set}=. +- =cj/--coverage-changed-lines SCOPE BASE= → hash-table ={file → changed-line-set}= by shelling a =git diff --unified=0= for the selected scope and parsing hunk headers. +- =cj/--coverage-intersect COVERED CHANGED= → per-file records with three buckets: covered, uncovered, not-tracked. + +These helpers are pure and covered by focused ERT tests. + +** Data Flow + +1. User invokes =cj/coverage-report= (bound to =F7=). +2. Core resolves the backend for the current project. +3. =completing-read= prompts for scope: + - "Working tree — all uncommitted changes" + - "Staged — about to commit" + - "Branch vs parent" (uses =@{upstream}= unless a caller passes an explicit base to the helper) + - "Branch vs main" (explicit) + - "Whole project — all executable lines" +4. If =simplecov.json= is missing, prompt to run coverage. A prefix argument + (=C-u F7=) forces a fresh run. Otherwise the existing report is used as-is. +5. Parse simplecov, compute changed lines or all executable lines, intersect. +6. Display a report buffer in a mode derived from =compilation-mode=. + +** Persistence + +- =.coverage/simplecov.json= at the project root, gitignored. Overwritten on each run. +- No long-term storage. Historical tracking is explicitly out of scope for v1. + +** Error Handling + +*Pre-flight:* +- No backend matches → =user-error= with instructions to register a backend or set =.dir-locals.el=. +- =.dir-locals.el= names an unknown backend → error listing registered backends. +- Not in a git repository → error; don't swallow git's stderr. +- Branch comparison on a repo with no common ancestor (orphan branch, shallow + clone missing the fork point, or missing upstream) reports the underlying git + failure. + +*During the coverage run:* +- Backend =:run= fails (test failure, Make error) → keep the =compile= buffer visible, do *not* proceed to display a report. Partial data is worse than no data. +- Run completes but no simplecov.json produced → error naming the expected path. + +*Post-flight classification:* three buckets, not two. +- *Covered* — changed line in the simplecov covered-line set. +- *Uncovered* — changed line in a tracked file but not covered. +- *Not tracked* — changed file isn't in the simplecov data at all (test files, READMEs, config). Reported separately — don't conflate "coverage didn't look here" with "tests didn't exercise this code." + +*Happy-path degenerates:* +- Zero changed lines in scope → "No changes in this scope; nothing to report." +- All changed lines covered → "N of N changed lines covered. " + +** Keybindings + +*Global:* +- =F7= → =cj/coverage-report= (prompts scope, shows report). +- =C-u F7= → force re-run regardless of report freshness. + +*In the report buffer* (compilation-mode derived, most inherited for free): +- =RET= → jump to source under point. +- =n= / =p= → next / previous uncovered line. +- =q= → bury buffer. + +*Globally available via compilation-mode integration:* +- =M-g n= / =M-g p= → =next-error= / =previous-error= on the last compilation buffer. +- =C-x `= → visit next uncovered line without leaving the current buffer. + +The =F4=–=F7= developer block currently uses =F6= for project-aware test +dispatch and =F7= for coverage. + +** Testing + +*Pure helpers, fully tested* (Normal / Boundary / Error for each): +- =cj/--coverage-parse-simplecov= — handcrafted simplecov JSON in temp files; empty object, all-null coverage arrays, spaces in filenames, multiple test-name keys unioned, malformed JSON. +- =cj/--coverage-simplecov-executable-lines= — whole-project executable-line set, including zero-hit executable lines. +- =cj/--coverage-changed-lines= — =cl-letf= over =shell-command-to-string= to return canned =git diff= output; single hunk, new-file hunk, deletion-only hunk, binary marker, no-diff case. +- =cj/--coverage-intersect= — pure table-in / table-out; covered ⊇ changed, unknown files, nil/empty inputs. +- =cj/--coverage-format-report= and =cj/--coverage-format-summary= — report text and whole-project per-file summary. + +*Backend registry, structurally tested:* +- =cj/coverage-backend-for-project ROOT= — synthetic temp project roots with marker files; assert correct backend. Registration-order test: two backends match, first-registered wins. + +*Not tested:* +- =cj/coverage-report= interactive command — one smoke test with a prepared simplecov report and a stubbed git-diff. No tests for the prompt UI or the compilation-buffer display. +- The elisp backend's =:run= function — shells to =make coverage=; integration-test-shaped, low value, slow. Skipped by design. + +* Current Limitations + +- =make coverage= runs all unit test files except known instrumentation + conflicts. It does not try to select only tests related to changed modules. +- Existing reports are not checked for staleness. Use =C-u F7= or + =make coverage= when a fresh report matters. +- Only the Elisp backend is implemented. +- There is no CI coverage publishing. The generated + =.coverage/simplecov.json= file is local and gitignored. diff --git a/docs/specs/debug-profiling-spec.org b/docs/specs/debug-profiling-spec.org index 5961071b..3492d3a2 100644 --- a/docs/specs/debug-profiling-spec.org +++ b/docs/specs/debug-profiling-spec.org @@ -1,10 +1,14 @@ -:PROPERTIES: -:ID: c713b431-ae14-498d-aba9-b84d52f981b6 -:STATUS: not-started -:END: #+TITLE: Design: debug-profiling.el module #+AUTHOR: Craig Jennings #+DATE: 2026-04-26 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Design: debug-profiling.el module +:PROPERTIES: +:ID: c713b431-ae14-498d-aba9-b84d52f981b6 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from existing :STATUS: not-started * Status diff --git a/docs/specs/dev-setup-project-spec.org b/docs/specs/dev-setup-project-spec.org index 5d64f368..058784a5 100644 --- a/docs/specs/dev-setup-project-spec.org +++ b/docs/specs/dev-setup-project-spec.org @@ -1,10 +1,14 @@ -:PROPERTIES: -:ID: 596fce5d-1bab-46e7-8567-d4a2e0923091 -:STATUS: not-started -:END: #+TITLE: Design: cj/dev-setup-project #+AUTHOR: Craig Jennings #+DATE: 2026-04-22 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Design: cj/dev-setup-project +:PROPERTIES: +:ID: 596fce5d-1bab-46e7-8567-d4a2e0923091 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from existing :STATUS: not-started * Status diff --git a/docs/specs/dupre-clear-theme-spec.org b/docs/specs/dupre-clear-theme-spec.org index 578eb240..8027ee2a 100644 --- a/docs/specs/dupre-clear-theme-spec.org +++ b/docs/specs/dupre-clear-theme-spec.org @@ -1,10 +1,14 @@ -:PROPERTIES: -:ID: 20df7f50-4759-47ba-9782-8dd25a2e173e -:STATUS: not-started -:END: #+TITLE: dupre-clear — a contrast-first AAA sibling theme #+AUTHOR: Craig Jennings #+DATE: 2026-06-07 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT dupre-clear — a contrast-first AAA sibling theme +:PROPERTIES: +:ID: 20df7f50-4759-47ba-9782-8dd25a2e173e +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from existing :STATUS: not-started * Status diff --git a/docs/specs/face-font-diagnostic-popup-spec-implemented.org b/docs/specs/face-font-diagnostic-popup-spec-implemented.org deleted file mode 100644 index 3e8fadcd..00000000 --- a/docs/specs/face-font-diagnostic-popup-spec-implemented.org +++ /dev/null @@ -1,197 +0,0 @@ -:PROPERTIES: -:ID: 98f065cf-8bd5-46a0-ac24-da94d66855ad -:STATUS: implemented -:END: -#+TITLE: Face and Font Diagnostic Popup — Spec -#+AUTHOR: Craig Jennings -#+DATE: 2026-06-14 -#+TODO: TODO | DONE SUPERSEDED CANCELLED - -* Metadata - -| Status | implemented | -|----------+---------------------------------------------------| -| Owner | Craig Jennings | -|----------+---------------------------------------------------| -| Related | [[file:../../todo.org][todo.org: Face and font diagnostic popup at point]] | - -* Summary - -A read-only command that, for the character at point in an ordinary buffer, pops up everything that determines how that character is painted: the full face stack, the effective merged attributes, the real font versus the declared family, and where each attribute came from (theme, config, or inheritance). It exists to answer one question fast — "why does this text look wrong under the theme, and is the fault the theme, my config, or a fallback?" - -* Problem / Context - -Theme work in this config keeps hitting the same wall: a glyph renders in the wrong color and there's no quick way to see why. The cursor showed gold in auto-dimmed buffers; elfeed rendered all-white ignoring its theme assignments. Each of those is a different layer failing — a face remap, an overlay, an unspecified attribute falling through to the default — and the built-in tools don't separate those layers or trace provenance. - -What paints a character is a merge of several sources resolved by the redisplay engine: the default face, then text-property faces (=face= / =font-lock-face=), then overlay faces stacked by priority, all rewritten by any =face-remapping-alist= entries, and finally a font chosen by the fontset that can differ from the face's declared =:family=. To debug a theme issue you need to see each layer, the merged result, and — for each face — whether its current attributes came from the active theme, from config, or from an =:inherit= chain that bottoms out at the default. - -=describe-char= and =C-u C-x ​== show the character, its faces as links, and the font, but they don't separate the stack by source, don't surface active remaps, and don't trace attribute provenance. The gap is exactly the part that distinguishes a theme bug from a config bug. - -* Goals and Non-Goals - -** Goals -- For the character at point, show the face stack separated by source (text-property, overlay-by-priority, active remaps, default). -- Show the effective merged attributes — the value that wins for each attribute. -- Show the real font (=font-at=) next to the face's declared =:family=, to expose fontset substitution. -- For each face, trace provenance: which theme(s) and/or config set each attribute, the =:inherit= chain, and the unspecified→fallback resolution. -- Present it in a read-only, navigable help-style buffer obeying the project's unified popup placement and dismissal rules. -- Degrade gracefully in out-of-scope buffers: show what can be read plus a banner naming the foreign color source — never a bare refusal. - -** Non-Goals -- No editing of faces, themes, or attributes. This is a diagnostic, not an editor; theme-studio owns editing. -- No reimplementation of =describe-char='s general character report (display tables, composition, char properties beyond faces). -- No coverage of color sources outside the theme/face system as first-class (terminal ANSI palettes, document HTML/CSS, image buffers) — surfaced, not analyzed. -- No persistence, history, or export of diagnostic output. - -** Scope tiers -- v1: char-at-point diagnosis with an optional region-scan mode; the five info groups below; the help-style popup; graceful out-of-scope handling. -- Out of scope: terminal-ANSI buffers, image/PDF buffers, and shr/document-rendered buffers as analyzed targets (they get the banner + best-effort dump). -- vNext: interactivity — "send this face to theme-studio", jump-to-theme-spec actions, and any write path. Logged to todo.org. - -* Design - -The command — call it =cj/describe-face-at-point= (final name an open detail) — reads the character at point and builds a report buffer in five groups. It never mutates buffer or frame state. - -** For the user (what the popup shows) - -1. *Character context.* The character, its codepoint and Unicode name, and its script. Script is what explains fontset routing, so it earns its place even though it's one line. - -2. *Face stack, by source.* The layers that contribute, in merge order, each labeled by where it comes from: - - text-property faces: the =face= and =font-lock-face= properties at point, in list order, anonymous specs shown inline; - - overlay faces: every overlay covering point that carries a =face=, sorted by overlay priority, with a best-effort owner label; - - active remaps: the =face-remapping-alist= entries that apply to faces in the stack (this is the auto-dim layer); - - the default face underneath. - Source separation is the diagnostic — "is this from a text property, an overlay, or a remap?" is half the answer. - -3. *Effective merged attributes.* The winning value per attribute (family, height, weight, slant, foreground, background, underline, overline, strike-through, box, inverse-video). This is what actually paints. - -4. *Real font vs declared family.* The font =font-at= reports as actually used, next to the merged =:family=. A mismatch means the fontset substituted (emoji, CJK, a missing glyph) — a common "why is this one character different" cause. - -5. *Per-face provenance.* For each named face in the stack: which theme(s) set its attributes (=theme-face= property), whether config overrode it (=saved-face= / =customized-face= / a runtime =set-face-attribute=), the =:inherit= chain, and for each unspecified attribute the resolution trace — "=:foreground= unspecified → not set by any theme → no inherit → default fg." That last trace is the direct read on the elfeed-white class of bug. - -The report is a read-only buffer in a dedicated mode, with named faces rendered as buttons that re-run the command's per-face section or call =describe-face= (navigation only — no edits in v1). - -** For the implementer (how it's built) - -A pure core plus a thin interactive wrapper, per the project's interactive/internal split: - -- =cj/--face-diagnosis-at (pos &optional buffer)= → a plist describing the five groups. No prompts, no display. This is the testable unit. -- =cj/describe-face-at-point= (interactive) → calls the core at point, renders the plist into the help buffer, places the window per the unified popup rules. -- Region mode → maps the core over the distinct face-runs in the active region and concatenates. - -Data sources, by group: -- Stack: =get-text-property= for =face= / =font-lock-face=; =overlays-at= filtered to those with a =face=, sorted by =overlay-get … 'priority=; =face-remapping-alist= (buffer-local) intersected with the stack; =get-char-property= as a cross-check on the merged text-prop+overlay face. -- Merged attributes: see the open decision below — Emacs exposes no single "final merged plist" call, so the core folds the ordered stack itself. -- Real font: =font-at=, then =query-font= / =font-info= for its family and name; nil under =--batch=, handled as "unavailable". -- Provenance: =(get FACE 'theme-face)= for theme spec history, =saved-face= / =customized-face= / =face--attribute-from-frame= comparisons for config overrides, and =face-attribute= with the inherit-following argument to produce the resolution trace. - -Buffer classification (group 0, decides scope handling): a predicate inspects =major-mode= derivation and known markers to bucket the buffer as theme-faced (analyze fully), terminal-ANSI, document-shr, or image/no-text. Out-of-scope buckets still render groups 1–2 best-effort and prepend a banner naming the color source. - -* Alternatives Considered - -** Presentation: childframe / posframe popup -- Good, because it floats near point and looks modern. -- Bad, because the report is tall and structured; a childframe is cramped, doesn't scroll naturally, and fights the existing unified-popup policy. -- Neutral, because a posframe could wrap the same render function later if wanted. - -** Presentation: which-key-style transient strip -- Good, because it's lightweight. -- Bad, because it can't hold five groups of structured, navigable, copyable text. Wrong tool for a report. - -** Reuse: extend describe-char instead of a new command -- Good, because describe-char already resolves faces and the font and renders links. -- Bad, because its output is fixed and character-report-shaped; the value here is source-separation and provenance, which would mean rewriting most of its body anyway. Better to study =descr-text.el= for the font/face resolution mechanics and build a focused command than to graft onto a general one. -- Neutral, because we still reuse the same primitives it uses (=font-at=, =get-char-property=). - -** Scope: analyze every buffer uniformly -- Good, because no classifier to write. -- Bad, because in a terminal or an shr buffer the provenance trace is misleading — the color isn't from the theme, so "theme didn't set it" reads as a theme bug when it isn't. The banner exists precisely to stop that false read. - -* Decisions [7/7] -** DONE Granularity: char-at-point with optional region scan -- Context: precise diagnosis wants one character; occasionally you want a whole region surveyed. -- Decision: We will default to the character at point and offer a region-scan mode over the distinct face-runs when a region is active. -- Consequences: easier — the common case is one precise report; harder — region mode must dedupe face-runs and concatenate without flooding the buffer. - -** DONE Provenance is core v1 -- Context: provenance (theme vs config vs inherit, unspecified→fallback) is the whole reason to build this over describe-char. -- Decision: We will treat the per-face provenance trace as required v1 content, not a follow-up. -- Consequences: easier — the tool actually answers theme-vs-config; harder — provenance extraction is the most intricate part and carries Emacs-version risk on the =theme-face= / =saved-face= internals. - -** DONE Include the real-font (fontset) layer -- Context: a face's =:family= can differ from the font actually chosen for a glyph. -- Decision: We will show =font-at='s real font next to the declared family. -- Consequences: easier — catches substitution bugs; harder — =font-at= is nil in batch, so tests must tolerate "unavailable". - -** DONE Presentation: read-only help-style buffer under the unified popup rules -- Context: the report is tall and structured and benefits from scrolling, copy, and face links. -- Decision: We will render into a dedicated read-only buffer and place/dismiss it via the project's unified popup placement and dismissal rules. -- Consequences: easier — idiomatic, navigable, consistent with other popups; harder — depends on the unified-popup policy, whose placement thresholds are still being settled in its own task. - -** DONE Interactivity is vNext -- Context: a "send face to theme-studio" bridge is attractive but is editing-adjacent. -- Decision: We will ship v1 read-only; the theme-studio bridge and any write path are vNext. -- Consequences: easier — v1 stays a safe pure diagnostic; harder — users must round-trip through theme-studio by hand until vNext. - -** DONE Out-of-scope buffers: classify and show everything, with a banner -- Context: a hard refuse in a terminal/shr/image buffer is unhelpful and hides information. -- Decision: We will classify the buffer, render what we can, and prepend a banner naming the foreign color source instead of refusing. -- Consequences: easier — maximal information always, and the boundary teaches itself; harder — the classifier must recognize the buffer buckets reliably enough that the banner isn't wrong. - -** DONE Effective-attribute computation approach -- Owner / by-when: Claude / before Phase 2 implementation. -- Context: Emacs exposes no public call returning the final merged attribute plist for a position (text props + overlays + remaps as the C redisplay merges them). The tool has to produce the "what actually paints" values itself. -- Decision: We fold the ordered, remap-expanded stack manually with =face-attribute=, treating overlays-over-text-props-over-default and applying remaps, and label the merged result as "computed" — accepting that exotic edge cases (relative heights, deep =:inherit= ordering) may diverge slightly from the engine. Alternative under consideration: lift the resolution mechanics from =descr-text.el= / =face-at-point= rather than hand-rolling. -- Consequences: easier — a single explicit merge we can unit-test; harder — fidelity to the real engine isn't guaranteed for corner cases, so the spec stays "Ready with caveats" until the approach is pinned. - -*** Discussion -- Resolved 2026-06-15: implemented as the hand-fold in =cj/--face-diag-merged-attributes= (overlays over text-props over default, remaps expanded ahead of their base), labeled "computed". Pinned by fixtures in =test-face-diagnostic.el= -- overlay-over-text-prop, a default remap, and a face-symbol attribute all resolve correctly. Exotic relative-height / deep-inherit cases may still diverge, accepted per the decision. - -* Implementation phases - -** Phase 1 — Core read model + buffer classifier -=cj/--face-diagnosis-at= returns the plist for groups 0–2 (classification, character context, face stack by source). Pure, no display. Unit-tested against temp-buffer fixtures with planted text properties, overlays, and remaps. Tree stays green. - -** Phase 2 — Merged attributes + real font -Extend the core with group 3 (effective merged attributes, per the resolved computation decision) and group 4 (=font-at= vs declared family, "unavailable" under batch). Unit-tested on the merge fixtures. - -** Phase 3 — Provenance trace -Add group 5: theme/config/inherit provenance and the unspecified→fallback resolution per face. Tested with fixtures that set a face via a loaded theme, via =set-face-attribute=, and leave one attribute unspecified. - -** Phase 4 — Render + popup wiring -The interactive =cj/describe-face-at-point=, the read-only mode with face buttons, region-scan mode, and placement/dismissal via the unified popup rules. Smoke-tested live; the render function tested on a captured plist. - -* Acceptance criteria -- [ ] On a normal prog/text buffer, the popup shows all five groups for the character at point. -- [ ] An overlay face (e.g. region) at point appears in the stack, labeled as an overlay, above the text-property faces. -- [ ] An active =face-remapping-alist= remap (e.g. under auto-dim) appears as the remap layer and is reflected in the merged result. -- [ ] A face with an unspecified =:foreground= shows the resolution trace down to its actual fallback. -- [ ] A glyph using a substituted font (e.g. an emoji) shows a real-font ≠ declared-family mismatch. -- [ ] In a terminal/shr/image buffer, the popup shows a banner naming the color source and still renders what it can. -- [ ] The core (=cj/--face-diagnosis-at=) returns its plist with no prompts and no display side effects, and passes under =make test= (=--batch=). - -* Readiness dimensions -- Data model & ownership: all data is read live from buffer/overlay/face/font state; nothing user-authored, generated, or persisted. The report plist is ephemeral. -- Errors, empty states & failure: no character at point (empty buffer / eob) → a clear "nothing at point" message; =font-at= nil under batch → "font: unavailable (batch)"; out-of-scope buffer → banner, not error. No silent data loss (read-only tool). -- Security & privacy: N/A — reads visible buffer text and face metadata; logs nothing; no credentials. -- Observability: the tool *is* the observability surface. Its own failures surface as in-buffer messages naming the missing piece (e.g. "font backend unavailable"). -- Performance & scale: single character is trivial; region mode is bounded by distinct face-runs in the region — cap or warn past a threshold so a whole-buffer region doesn't generate a huge report. No live/remote dependency. -- Reuse & lost opportunities: reuses =font-at=, =get-char-property=, =face-attribute=, =theme-face=/=saved-face= internals, and the project's unified-popup policy and interactive/internal split. Studies =descr-text.el= rather than forking it. -- Architecture fit & weak points: integration points are the unified-popup placement policy (in flux) and the face/theme internals (=theme-face=, =saved-face=) which are version-sensitive — isolate them behind small accessors so an Emacs-version change touches one place. -- Config surface: the region-run cap is the one likely knob, with a safe default. Possibly a toggle for whether out-of-scope buffers render best-effort or just the banner. -- Documentation plan: a docstring on the command, the keybinding noted in the keybinding map, and a CLAUDE.md/notes pointer only if a non-obvious gotcha surfaces. No user manual needed. -- Dev tooling: existing =make test= / byte-compile / live-reload loop; no new targets. -- Rollout, compatibility & rollback: additive new command + one keybinding; nothing persisted or migrated; rollback is removing the module. No compatibility surface. -- External APIs & deps: N/A — pure Emacs primitives, no external API or package dependency. - -* Risks, Rabbit Holes, and Drawbacks -- *Merge fidelity* (the open decision): a hand-folded attribute merge may diverge from the redisplay engine on exotic cases. Dodge: validate against =describe-char= on a handful of fixtures; label the result "computed"; don't claim pixel-exactness. -- *Provenance internals*: =theme-face= / =saved-face= are not a stable public contract. Dodge: isolate behind accessors; tolerate missing properties as "unknown source" rather than erroring. -- *Unified-popup dependency*: that policy's placement thresholds aren't settled. Dodge: code to the policy's interface, accept whatever defaults it lands on; don't invent a parallel placement scheme here. -- *Overlay owner labeling*: overlays don't record their creator. Dodge: best-effort label from known marker properties; fall back to "(overlay)" without guessing. - -* Review and iteration history -** 2026-06-14 Sun @ 22:30:00 -0500 — Claude (for Craig) — author -- What: initial draft. -- Why: theme debugging keeps hitting layered face/font issues with no tool that separates the layers or traces provenance; agreed to spec before building. -- Artifacts: [[file:../../todo.org][todo.org: Face and font diagnostic popup at point]]; motivating bugs — gold-text-in-auto-dim, elfeed-ignores-theme. diff --git a/docs/specs/face-font-diagnostic-popup-spec.org b/docs/specs/face-font-diagnostic-popup-spec.org new file mode 100644 index 00000000..aae355f9 --- /dev/null +++ b/docs/specs/face-font-diagnostic-popup-spec.org @@ -0,0 +1,200 @@ +#+TITLE: Face and Font Diagnostic Popup — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-14 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Face and Font Diagnostic Popup — Spec +:PROPERTIES: +:ID: 98f065cf-8bd5-46a0-ac24-da94d66855ad +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword IMPLEMENTED from existing :STATUS: implemented + -implemented filename (Craig's prior determination) + +* Metadata + +| Status | implemented | +|----------+---------------------------------------------------| +| Owner | Craig Jennings | +|----------+---------------------------------------------------| +| Related | [[file:../../todo.org][todo.org: Face and font diagnostic popup at point]] | + +* Summary + +A read-only command that, for the character at point in an ordinary buffer, pops up everything that determines how that character is painted: the full face stack, the effective merged attributes, the real font versus the declared family, and where each attribute came from (theme, config, or inheritance). It exists to answer one question fast — "why does this text look wrong under the theme, and is the fault the theme, my config, or a fallback?" + +* Problem / Context + +Theme work in this config keeps hitting the same wall: a glyph renders in the wrong color and there's no quick way to see why. The cursor showed gold in auto-dimmed buffers; elfeed rendered all-white ignoring its theme assignments. Each of those is a different layer failing — a face remap, an overlay, an unspecified attribute falling through to the default — and the built-in tools don't separate those layers or trace provenance. + +What paints a character is a merge of several sources resolved by the redisplay engine: the default face, then text-property faces (=face= / =font-lock-face=), then overlay faces stacked by priority, all rewritten by any =face-remapping-alist= entries, and finally a font chosen by the fontset that can differ from the face's declared =:family=. To debug a theme issue you need to see each layer, the merged result, and — for each face — whether its current attributes came from the active theme, from config, or from an =:inherit= chain that bottoms out at the default. + +=describe-char= and =C-u C-x ​== show the character, its faces as links, and the font, but they don't separate the stack by source, don't surface active remaps, and don't trace attribute provenance. The gap is exactly the part that distinguishes a theme bug from a config bug. + +* Goals and Non-Goals + +** Goals +- For the character at point, show the face stack separated by source (text-property, overlay-by-priority, active remaps, default). +- Show the effective merged attributes — the value that wins for each attribute. +- Show the real font (=font-at=) next to the face's declared =:family=, to expose fontset substitution. +- For each face, trace provenance: which theme(s) and/or config set each attribute, the =:inherit= chain, and the unspecified→fallback resolution. +- Present it in a read-only, navigable help-style buffer obeying the project's unified popup placement and dismissal rules. +- Degrade gracefully in out-of-scope buffers: show what can be read plus a banner naming the foreign color source — never a bare refusal. + +** Non-Goals +- No editing of faces, themes, or attributes. This is a diagnostic, not an editor; theme-studio owns editing. +- No reimplementation of =describe-char='s general character report (display tables, composition, char properties beyond faces). +- No coverage of color sources outside the theme/face system as first-class (terminal ANSI palettes, document HTML/CSS, image buffers) — surfaced, not analyzed. +- No persistence, history, or export of diagnostic output. + +** Scope tiers +- v1: char-at-point diagnosis with an optional region-scan mode; the five info groups below; the help-style popup; graceful out-of-scope handling. +- Out of scope: terminal-ANSI buffers, image/PDF buffers, and shr/document-rendered buffers as analyzed targets (they get the banner + best-effort dump). +- vNext: interactivity — "send this face to theme-studio", jump-to-theme-spec actions, and any write path. Logged to todo.org. + +* Design + +The command — call it =cj/describe-face-at-point= (final name an open detail) — reads the character at point and builds a report buffer in five groups. It never mutates buffer or frame state. + +** For the user (what the popup shows) + +1. *Character context.* The character, its codepoint and Unicode name, and its script. Script is what explains fontset routing, so it earns its place even though it's one line. + +2. *Face stack, by source.* The layers that contribute, in merge order, each labeled by where it comes from: + - text-property faces: the =face= and =font-lock-face= properties at point, in list order, anonymous specs shown inline; + - overlay faces: every overlay covering point that carries a =face=, sorted by overlay priority, with a best-effort owner label; + - active remaps: the =face-remapping-alist= entries that apply to faces in the stack (this is the auto-dim layer); + - the default face underneath. + Source separation is the diagnostic — "is this from a text property, an overlay, or a remap?" is half the answer. + +3. *Effective merged attributes.* The winning value per attribute (family, height, weight, slant, foreground, background, underline, overline, strike-through, box, inverse-video). This is what actually paints. + +4. *Real font vs declared family.* The font =font-at= reports as actually used, next to the merged =:family=. A mismatch means the fontset substituted (emoji, CJK, a missing glyph) — a common "why is this one character different" cause. + +5. *Per-face provenance.* For each named face in the stack: which theme(s) set its attributes (=theme-face= property), whether config overrode it (=saved-face= / =customized-face= / a runtime =set-face-attribute=), the =:inherit= chain, and for each unspecified attribute the resolution trace — "=:foreground= unspecified → not set by any theme → no inherit → default fg." That last trace is the direct read on the elfeed-white class of bug. + +The report is a read-only buffer in a dedicated mode, with named faces rendered as buttons that re-run the command's per-face section or call =describe-face= (navigation only — no edits in v1). + +** For the implementer (how it's built) + +A pure core plus a thin interactive wrapper, per the project's interactive/internal split: + +- =cj/--face-diagnosis-at (pos &optional buffer)= → a plist describing the five groups. No prompts, no display. This is the testable unit. +- =cj/describe-face-at-point= (interactive) → calls the core at point, renders the plist into the help buffer, places the window per the unified popup rules. +- Region mode → maps the core over the distinct face-runs in the active region and concatenates. + +Data sources, by group: +- Stack: =get-text-property= for =face= / =font-lock-face=; =overlays-at= filtered to those with a =face=, sorted by =overlay-get … 'priority=; =face-remapping-alist= (buffer-local) intersected with the stack; =get-char-property= as a cross-check on the merged text-prop+overlay face. +- Merged attributes: see the open decision below — Emacs exposes no single "final merged plist" call, so the core folds the ordered stack itself. +- Real font: =font-at=, then =query-font= / =font-info= for its family and name; nil under =--batch=, handled as "unavailable". +- Provenance: =(get FACE 'theme-face)= for theme spec history, =saved-face= / =customized-face= / =face--attribute-from-frame= comparisons for config overrides, and =face-attribute= with the inherit-following argument to produce the resolution trace. + +Buffer classification (group 0, decides scope handling): a predicate inspects =major-mode= derivation and known markers to bucket the buffer as theme-faced (analyze fully), terminal-ANSI, document-shr, or image/no-text. Out-of-scope buckets still render groups 1–2 best-effort and prepend a banner naming the color source. + +* Alternatives Considered + +** Presentation: childframe / posframe popup +- Good, because it floats near point and looks modern. +- Bad, because the report is tall and structured; a childframe is cramped, doesn't scroll naturally, and fights the existing unified-popup policy. +- Neutral, because a posframe could wrap the same render function later if wanted. + +** Presentation: which-key-style transient strip +- Good, because it's lightweight. +- Bad, because it can't hold five groups of structured, navigable, copyable text. Wrong tool for a report. + +** Reuse: extend describe-char instead of a new command +- Good, because describe-char already resolves faces and the font and renders links. +- Bad, because its output is fixed and character-report-shaped; the value here is source-separation and provenance, which would mean rewriting most of its body anyway. Better to study =descr-text.el= for the font/face resolution mechanics and build a focused command than to graft onto a general one. +- Neutral, because we still reuse the same primitives it uses (=font-at=, =get-char-property=). + +** Scope: analyze every buffer uniformly +- Good, because no classifier to write. +- Bad, because in a terminal or an shr buffer the provenance trace is misleading — the color isn't from the theme, so "theme didn't set it" reads as a theme bug when it isn't. The banner exists precisely to stop that false read. + +* Decisions [7/7] +** DONE Granularity: char-at-point with optional region scan +- Context: precise diagnosis wants one character; occasionally you want a whole region surveyed. +- Decision: We will default to the character at point and offer a region-scan mode over the distinct face-runs when a region is active. +- Consequences: easier — the common case is one precise report; harder — region mode must dedupe face-runs and concatenate without flooding the buffer. + +** DONE Provenance is core v1 +- Context: provenance (theme vs config vs inherit, unspecified→fallback) is the whole reason to build this over describe-char. +- Decision: We will treat the per-face provenance trace as required v1 content, not a follow-up. +- Consequences: easier — the tool actually answers theme-vs-config; harder — provenance extraction is the most intricate part and carries Emacs-version risk on the =theme-face= / =saved-face= internals. + +** DONE Include the real-font (fontset) layer +- Context: a face's =:family= can differ from the font actually chosen for a glyph. +- Decision: We will show =font-at='s real font next to the declared family. +- Consequences: easier — catches substitution bugs; harder — =font-at= is nil in batch, so tests must tolerate "unavailable". + +** DONE Presentation: read-only help-style buffer under the unified popup rules +- Context: the report is tall and structured and benefits from scrolling, copy, and face links. +- Decision: We will render into a dedicated read-only buffer and place/dismiss it via the project's unified popup placement and dismissal rules. +- Consequences: easier — idiomatic, navigable, consistent with other popups; harder — depends on the unified-popup policy, whose placement thresholds are still being settled in its own task. + +** DONE Interactivity is vNext +- Context: a "send face to theme-studio" bridge is attractive but is editing-adjacent. +- Decision: We will ship v1 read-only; the theme-studio bridge and any write path are vNext. +- Consequences: easier — v1 stays a safe pure diagnostic; harder — users must round-trip through theme-studio by hand until vNext. + +** DONE Out-of-scope buffers: classify and show everything, with a banner +- Context: a hard refuse in a terminal/shr/image buffer is unhelpful and hides information. +- Decision: We will classify the buffer, render what we can, and prepend a banner naming the foreign color source instead of refusing. +- Consequences: easier — maximal information always, and the boundary teaches itself; harder — the classifier must recognize the buffer buckets reliably enough that the banner isn't wrong. + +** DONE Effective-attribute computation approach +- Owner / by-when: Claude / before Phase 2 implementation. +- Context: Emacs exposes no public call returning the final merged attribute plist for a position (text props + overlays + remaps as the C redisplay merges them). The tool has to produce the "what actually paints" values itself. +- Decision: We fold the ordered, remap-expanded stack manually with =face-attribute=, treating overlays-over-text-props-over-default and applying remaps, and label the merged result as "computed" — accepting that exotic edge cases (relative heights, deep =:inherit= ordering) may diverge slightly from the engine. Alternative under consideration: lift the resolution mechanics from =descr-text.el= / =face-at-point= rather than hand-rolling. +- Consequences: easier — a single explicit merge we can unit-test; harder — fidelity to the real engine isn't guaranteed for corner cases, so the spec stays "Ready with caveats" until the approach is pinned. + +*** Discussion +- Resolved 2026-06-15: implemented as the hand-fold in =cj/--face-diag-merged-attributes= (overlays over text-props over default, remaps expanded ahead of their base), labeled "computed". Pinned by fixtures in =test-face-diagnostic.el= -- overlay-over-text-prop, a default remap, and a face-symbol attribute all resolve correctly. Exotic relative-height / deep-inherit cases may still diverge, accepted per the decision. + +* Implementation phases + +** Phase 1 — Core read model + buffer classifier +=cj/--face-diagnosis-at= returns the plist for groups 0–2 (classification, character context, face stack by source). Pure, no display. Unit-tested against temp-buffer fixtures with planted text properties, overlays, and remaps. Tree stays green. + +** Phase 2 — Merged attributes + real font +Extend the core with group 3 (effective merged attributes, per the resolved computation decision) and group 4 (=font-at= vs declared family, "unavailable" under batch). Unit-tested on the merge fixtures. + +** Phase 3 — Provenance trace +Add group 5: theme/config/inherit provenance and the unspecified→fallback resolution per face. Tested with fixtures that set a face via a loaded theme, via =set-face-attribute=, and leave one attribute unspecified. + +** Phase 4 — Render + popup wiring +The interactive =cj/describe-face-at-point=, the read-only mode with face buttons, region-scan mode, and placement/dismissal via the unified popup rules. Smoke-tested live; the render function tested on a captured plist. + +* Acceptance criteria +- [ ] On a normal prog/text buffer, the popup shows all five groups for the character at point. +- [ ] An overlay face (e.g. region) at point appears in the stack, labeled as an overlay, above the text-property faces. +- [ ] An active =face-remapping-alist= remap (e.g. under auto-dim) appears as the remap layer and is reflected in the merged result. +- [ ] A face with an unspecified =:foreground= shows the resolution trace down to its actual fallback. +- [ ] A glyph using a substituted font (e.g. an emoji) shows a real-font ≠ declared-family mismatch. +- [ ] In a terminal/shr/image buffer, the popup shows a banner naming the color source and still renders what it can. +- [ ] The core (=cj/--face-diagnosis-at=) returns its plist with no prompts and no display side effects, and passes under =make test= (=--batch=). + +* Readiness dimensions +- Data model & ownership: all data is read live from buffer/overlay/face/font state; nothing user-authored, generated, or persisted. The report plist is ephemeral. +- Errors, empty states & failure: no character at point (empty buffer / eob) → a clear "nothing at point" message; =font-at= nil under batch → "font: unavailable (batch)"; out-of-scope buffer → banner, not error. No silent data loss (read-only tool). +- Security & privacy: N/A — reads visible buffer text and face metadata; logs nothing; no credentials. +- Observability: the tool *is* the observability surface. Its own failures surface as in-buffer messages naming the missing piece (e.g. "font backend unavailable"). +- Performance & scale: single character is trivial; region mode is bounded by distinct face-runs in the region — cap or warn past a threshold so a whole-buffer region doesn't generate a huge report. No live/remote dependency. +- Reuse & lost opportunities: reuses =font-at=, =get-char-property=, =face-attribute=, =theme-face=/=saved-face= internals, and the project's unified-popup policy and interactive/internal split. Studies =descr-text.el= rather than forking it. +- Architecture fit & weak points: integration points are the unified-popup placement policy (in flux) and the face/theme internals (=theme-face=, =saved-face=) which are version-sensitive — isolate them behind small accessors so an Emacs-version change touches one place. +- Config surface: the region-run cap is the one likely knob, with a safe default. Possibly a toggle for whether out-of-scope buffers render best-effort or just the banner. +- Documentation plan: a docstring on the command, the keybinding noted in the keybinding map, and a CLAUDE.md/notes pointer only if a non-obvious gotcha surfaces. No user manual needed. +- Dev tooling: existing =make test= / byte-compile / live-reload loop; no new targets. +- Rollout, compatibility & rollback: additive new command + one keybinding; nothing persisted or migrated; rollback is removing the module. No compatibility surface. +- External APIs & deps: N/A — pure Emacs primitives, no external API or package dependency. + +* Risks, Rabbit Holes, and Drawbacks +- *Merge fidelity* (the open decision): a hand-folded attribute merge may diverge from the redisplay engine on exotic cases. Dodge: validate against =describe-char= on a handful of fixtures; label the result "computed"; don't claim pixel-exactness. +- *Provenance internals*: =theme-face= / =saved-face= are not a stable public contract. Dodge: isolate behind accessors; tolerate missing properties as "unknown source" rather than erroring. +- *Unified-popup dependency*: that policy's placement thresholds aren't settled. Dodge: code to the policy's interface, accept whatever defaults it lands on; don't invent a parallel placement scheme here. +- *Overlay owner labeling*: overlays don't record their creator. Dodge: best-effort label from known marker properties; fall back to "(overlay)" without guessing. + +* Review and iteration history +** 2026-06-14 Sun @ 22:30:00 -0500 — Claude (for Craig) — author +- What: initial draft. +- Why: theme debugging keeps hitting layered face/font issues with no tool that separates the layers or traces provenance; agreed to spec before building. +- Artifacts: [[file:../../todo.org][todo.org: Face and font diagnostic popup at point]]; motivating bugs — gold-text-in-auto-dim, elfeed-ignores-theme. diff --git a/docs/specs/flycheck-modeline-customization-spec-implemented.org b/docs/specs/flycheck-modeline-customization-spec-implemented.org deleted file mode 100644 index 59567be6..00000000 --- a/docs/specs/flycheck-modeline-customization-spec-implemented.org +++ /dev/null @@ -1,319 +0,0 @@ -:PROPERTIES: -:ID: 76979608-956e-474f-90a8-8d0c958101a0 -:STATUS: implemented -:END: -#+TITLE: Design: Flycheck modeline customization -#+AUTHOR: Craig Jennings -#+DATE: 2026-05-15 -#+OPTIONS: toc:nil num:nil - -* Status - -Draft. Supersedes the earlier =flycheck-modeline-customization-spec.org= -draft in =.ai/= (2025-11-14), which used stale line numbers and conflated -Option 3's risky-local-variable requirement with Option 4. - -* Problem - -Flycheck's status (error / warning counts, "checking" indicator) is not -visible in the custom modeline. The cause is a deliberate choice in -=modules/modeline-config.el=: =mode-line-format= is built from explicit -segments (=cj/modeline-buffer-name=, =cj/modeline-position=, -=cj/modeline-vc-branch=, etc.) and does not include =minor-mode-alist= -or =mode-line-modes=. Flycheck publishes its lighter into -=minor-mode-alist=, so the custom modeline never picks it up. - -The fix is to add a flycheck-aware segment to =mode-line-format=. - -* Goals - -1. Flycheck status appears in the custom modeline when =flycheck-mode= is on. -2. The display picks up flycheck's existing color logic (error count in =error= face, warning count in =warning= face). -3. The display gates on active window, matching the convention used by =cj/modeline-vc-branch= and =cj/modeline-misc-info=. -4. The customization is small enough that swapping prefix / success indicator is a one-line edit. - -* Non-Goals - -- A "minor modes" segment that surfaces every lighter from =minor-mode-alist=. Flycheck is the specific case we care about; the rest stay invisible. -- Reworking =flycheck-config.el= beyond the two =:custom= additions. -- Adding flycheck-side checkers or changing what gets checked. - -* Current State - -** =modules/flycheck-config.el:47-97= - -#+begin_src emacs-lisp -(use-package flycheck - :defer t - :commands (flycheck-list-errors cj/flycheck-list-errors) - :hook ((sh-mode emacs-lisp-mode) . flycheck-mode) - :bind (:map cj/custom-keymap ("?" . cj/flycheck-list-errors)) - :custom - (checkdoc-arguments - '(("sentence-end-double-space" nil) - ("warn-escape" nil))) - :config - ...) -#+end_src - -No flycheck-modeline customization. Defaults are in force: - -| Variable | Default | -|-------------------------------------+-------------------------------------------| -| =flycheck-mode-line-prefix= | ="FlyC"= | -| =flycheck-mode-success-indicator= | =":0"= | -| =flycheck-mode-line-color= | =t= (apply error / warning faces) | -| =flycheck-mode-line= | ='(:eval (flycheck-mode-line-status-text))= | - -** =modules/modeline-config.el:220-237= - -=mode-line-format= layout (left → right, with right-align edge): - -#+begin_src emacs-lisp -(setq-default mode-line-format - '("%e" - " " - cj/modeline-major-mode - " " - cj/modeline-buffer-name - " " - cj/modeline-position - mode-line-format-right-align - (:eval (when (fboundp 'cj/recording-modeline-indicator) - (cj/recording-modeline-indicator))) - cj/modeline-vc-branch - " " - cj/modeline-misc-info - " ")) -#+end_src - -Risky-local-variable list (=modeline-config.el:240-246=): - -#+begin_src emacs-lisp -(dolist (construct '(cj/modeline-buffer-name - cj/modeline-position - cj/modeline-vc-branch - cj/modeline-vc-faces - cj/modeline-major-mode - cj/modeline-misc-info)) - (put construct 'risky-local-variable t)) -#+end_src - -Note: =cj/modeline-vc-branch= and =cj/modeline-misc-info= both gate on -=(mode-line-window-selected-p)= so they appear only in the active window. - -** Flycheck lighter outputs (for reference) - -Flycheck status text values that =flycheck-mode-line-status-text= -returns, depending on =flycheck-last-status-change= and current errors: - -| Status | Display (with default prefix / indicator) | -|------------------------------+----------------------------------------------------| -| Not yet checked | =FlyC= | -| Currently checking | =FlyC*= | -| Finished, no errors | =FlyC:0= | -| Finished, 3 errors, 5 warns | =FlyC:3|5= | -| Checker errored | =FlyC!= | -| Interrupted | =FlyC.= | -| Suspicious | =FlyC?= | -| No checker available | =FlyC-= | - -With =flycheck-mode-line-color= = =t= (the default), the count portion -is colored: error count in the =error= face, warning count in =warning=. - -* Approaches Considered - -** Option 1 (Reject): customize prefix / indicator only - -Setting =flycheck-mode-line-prefix= and =flycheck-mode-success-indicator= -in =:custom= changes the lighter content, but the lighter still publishes -to =minor-mode-alist=, which the custom modeline doesn't read. The lighter -becomes prettier wherever it does show (e.g. doom-modeline if reinstated) -but not here. Doesn't solve the visibility problem. - -** Option 2 (Reject): add the raw =flycheck-mode-line= variable - -Inserting =flycheck-mode-line= into =mode-line-format= directly works, -but the form has no =flycheck-mode= guard. In a buffer where flycheck -isn't loaded or not enabled, the =:eval (flycheck-mode-line-status-text)= -call still fires and either errors or returns junk. Needs a wrapping -guard, which is what Option 4 does. - -** Option 3 (Reject for now): custom segment with full control - -Define =cj/modeline-flycheck= as a =defvar-local= holding a =(:eval ...)= -form that pulls error / warning counts directly from -=flycheck-current-errors=, builds a per-status string, propertizes it -with =error= / =warning= faces, and returns it. Reimplements what -=flycheck-mode-line-status-text= already does, with bespoke formatting. - -Pros: full control over format. Cons: maintenance burden, drifts from -flycheck's status model if flycheck changes it. - -If the Option 4 result ever stops being good enough -- e.g. you want a -different layout (=E:3 W:5= instead of =:3|5=) -- come back to this. -Until then, more code than the problem deserves. - -** Option 4 (Recommended): hybrid -- customize variables + add guarded segment - -Two changes: - -1. =modules/flycheck-config.el= =:custom= block gets prefix and success-indicator overrides. (Optional: also =flycheck-mode-line-color=.) - -2. =modules/modeline-config.el= adds a small =(:eval ...)= form inline in =mode-line-format= that guards on =flycheck-mode= and calls =(flycheck-mode-line-status-text)= directly. - -Pros: minimal code, uses flycheck's logic verbatim, prefix / indicator -swappable with a one-line edit, picks up flycheck's face colors -automatically. - -Cons: layout fixed to flycheck's =PREFIX[indicator|counts]= shape. -Acceptable. - -* Recommended Implementation (Option 4) - -** Step 1: =modules/flycheck-config.el= - -Add to the =:custom= block (currently lines 55-59 in the file): - -#+begin_src emacs-lisp -;; Modeline customization (rendered via mode-line-format in modeline-config.el). -(flycheck-mode-line-prefix "🐛") -(flycheck-mode-success-indicator " ✓") -;; flycheck-mode-line-color stays t (default) so counts keep their face coloring. -#+end_src - -Prefix and success indicator are taste; the **Emoji Reference** section -below catalogs the candidates. Note that the prefix emoji itself does -not inherit the =error= / =warning= face -- only the count portion does -(via =flycheck-mode-line-color=). That trade-off is fine for a static -prefix; an emoji prefix gives a recognizable shape that you scan for, -and the colored count carries the alert signal. - -** Step 2: =modules/modeline-config.el= - -Insert a =(:eval ...)= form into =mode-line-format= (currently lines -220-237). Recommended placement: between the recording indicator and -=cj/modeline-vc-branch= so flycheck status sits with the other -right-aligned status segments. - -After the change, the right-side block reads: - -#+begin_src emacs-lisp -;; RIGHT SIDE -mode-line-format-right-align -(:eval (when (fboundp 'cj/recording-modeline-indicator) - (cj/recording-modeline-indicator))) -(:eval (when (and (mode-line-window-selected-p) - (bound-and-true-p flycheck-mode)) - (flycheck-mode-line-status-text))) -" " -cj/modeline-vc-branch -" " -cj/modeline-misc-info -" ") -#+end_src - -Two design choices baked in: - -- =(mode-line-window-selected-p)= gates the segment to the active window, matching the convention used by =cj/modeline-vc-branch= and =cj/modeline-misc-info=. -- =(bound-and-true-p flycheck-mode)= prevents the function call in buffers where flycheck never loaded; safer than asking =flycheck-mode= directly. - -** Risky-local-variable: not needed here - -This implementation places =(:eval ...)= inline inside =mode-line-format= -rather than wrapping it in a =defvar-local=. Inline forms are evaluated -by mode-line processing without a risky-local-variable marker. The -existing risky list (=modeline-config.el:240-246=) does not need to -grow. - -(If you ever refactor this to a named segment -- =defvar-local cj/modeline-flycheck= -- then add it to the risky list. Option 3 above is the path that needs that step.) - -* Emoji Reference - -** Prefix candidates (=flycheck-mode-line-prefix=) - -| Glyph | Codepoint | Name | -|-------+-----------+---------------------------------------| -| 🪰 | U+1FAB0 | FLY (literal "fly" for flycheck) | -| 🐛 | U+1F41B | BUG (recommended -- broadest font support) | -| 🐞 | U+1F41E | LADY BEETLE | -| ⚠ | U+26A0 | WARNING SIGN | -| 🔍 | U+1F50D | MAGNIFYING GLASS | -| 📝 | U+1F4DD | MEMO | -| ✓ | U+2713 | CHECK MARK (text) | - -🪰 (U+1FAB0) is from Unicode 13.0 (2020) and needs an up-to-date emoji -font. 🐛 (U+1F41B) is older and renders everywhere. Default to 🐛 unless -the fly is a strong preference and the GUI fonts are known to cover it. - -** Success indicator candidates (=flycheck-mode-success-indicator=) - -| Glyph | Codepoint | Name | -|-------+-----------+---------------------------------------| -| ✓ | U+2713 | CHECK MARK (text) | -| ✔ | U+2714 | HEAVY CHECK MARK | -| ✅ | U+2705 | WHITE HEAVY CHECK MARK (green box) | -| 🟢 | U+1F7E2 | GREEN CIRCLE | -| ⭐ | U+2B50 | WHITE MEDIUM STAR | - -Note the leading space in the recommended setting (=" ✓"=): flycheck -joins the prefix and the success indicator with no separator, so a -leading space in the indicator gives breathing room between the emoji -prefix and the check mark. - -** Suggested combinations - -| Mood | Prefix | Success indicator | Result example | -|---------------------+--------+-------------------+----------------| -| Recommended default | 🐛 | " ✓" | =🐛 ✓= / =🐛:3|5= | -| Literal Flycheck | 🪰 | " ✓" | =🪰 ✓= / =🪰:3|5= | -| Minimal | "" | " ✓" | = ✓= / =:3|5= | -| Status light | "" | " 🟢" | = 🟢= / =:3|5= | - -* Testing - -** Manual - -1. Open =modules/flycheck-config.el= (an =emacs-lisp-mode= buffer with =flycheck-mode= auto-enabled per the existing =:hook=). The right side of the modeline shows the prefix + success indicator when there are no errors. -2. Introduce a deliberate parse error (drop a paren). Save. The modeline updates to show =:1|0= (or whatever count) in the =error= face. -3. Trigger =M-x flycheck-buffer= in a fresh =sh-mode= buffer. The "currently checking" state (=PREFIX*=) flashes briefly before settling on success or counts. -4. Open a second window onto the same buffer (=C-x 2=). The flycheck segment appears in the active window only; the inactive copy drops it. Confirms the active-window gate. -5. Open a buffer where flycheck never engages (e.g. =*scratch*= in fundamental-mode, or a =dired= buffer). No segment, no errors. -6. Run =cj/flycheck-prose-on-demand= in an org buffer (=C-; ?= in org-mode). The LanguageTool checker engages and the segment appears with prose-error counts. - -** Regression watch - -- The custom-modeline width should not jump distractingly as flycheck cycles "checking → finished". The status text is short (one to seven chars), so this should be invisible -- worth a glance. -- Inactive-window display: confirm the segment disappears, not just greys out. The current pattern is "hide entirely" via the =mode-line-window-selected-p= guard. -- =cj/modeline-misc-info= keeps showing chime / notification text. The flycheck segment sits to its left; verify the visual order matches the spec. - -* Files to Modify - -- =modules/flycheck-config.el= -- add two =:custom= lines. -- =modules/modeline-config.el= -- insert one =(:eval ...)= form into =mode-line-format=. - -Two-line / one-form change. No new tests required (the existing tests -don't lock the modeline content; they exercise behavior elsewhere). If -you want a smoke test, add one assertion in =tests/test-modeline-config.el= -(if that file exists or you create it) that =mode-line-format='s sexp -contains a form mentioning =flycheck-mode-line-status-text=. Optional. - -* Risks - -| Risk | Mitigation | -|-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------| -| Emoji renders as a tofu square in terminal Emacs | The user runs GUI Emacs primarily; if terminal use matters, set the prefix to a text glyph (=""= or =":"=) instead. | -| Modeline width thrash when flycheck transitions running → finished | Status text is one to seven chars; jitter is negligible. Confirm during manual testing. | -| Prefix emoji doesn't pick up =error= / =warning= face | Expected: =flycheck-mode-line-color= colors the count portion only. The static prefix is intentionally unstyled. If you want a colored prefix, switch to Option 3. | -| Flycheck not yet loaded when modeline first evaluates | The =(bound-and-true-p flycheck-mode)= guard returns nil in that case, the =(:eval ...)= returns nil, mode-line skips the slot. | -| Active-window gate is wrong for some workflow (e.g. multi-window comparison) | Drop =(mode-line-window-selected-p)=. One-line change. Decide after living with the default. | - -* Rollback - -Revert the commit. Two-file change, no schema impact. Idempotent. - -* Effort estimate - -S (under 1 hour). Two lines in =flycheck-config.el=, one form in -=modeline-config.el=, plus the manual verification walk-through. The -emoji selection is the time sink, not the code. diff --git a/docs/specs/flycheck-modeline-customization-spec.org b/docs/specs/flycheck-modeline-customization-spec.org new file mode 100644 index 00000000..2a58b447 --- /dev/null +++ b/docs/specs/flycheck-modeline-customization-spec.org @@ -0,0 +1,323 @@ +#+TITLE: Design: Flycheck modeline customization +#+AUTHOR: Craig Jennings +#+DATE: 2026-05-15 +#+OPTIONS: toc:nil num:nil +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Design: Flycheck modeline customization +:PROPERTIES: +:ID: 76979608-956e-474f-90a8-8d0c958101a0 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword IMPLEMENTED from existing :STATUS: implemented + -implemented filename (Craig's prior determination) + +* Status + +Draft. Supersedes the earlier =flycheck-modeline-customization-spec.org= +draft in =.ai/= (2025-11-14), which used stale line numbers and conflated +Option 3's risky-local-variable requirement with Option 4. + +* Problem + +Flycheck's status (error / warning counts, "checking" indicator) is not +visible in the custom modeline. The cause is a deliberate choice in +=modules/modeline-config.el=: =mode-line-format= is built from explicit +segments (=cj/modeline-buffer-name=, =cj/modeline-position=, +=cj/modeline-vc-branch=, etc.) and does not include =minor-mode-alist= +or =mode-line-modes=. Flycheck publishes its lighter into +=minor-mode-alist=, so the custom modeline never picks it up. + +The fix is to add a flycheck-aware segment to =mode-line-format=. + +* Goals + +1. Flycheck status appears in the custom modeline when =flycheck-mode= is on. +2. The display picks up flycheck's existing color logic (error count in =error= face, warning count in =warning= face). +3. The display gates on active window, matching the convention used by =cj/modeline-vc-branch= and =cj/modeline-misc-info=. +4. The customization is small enough that swapping prefix / success indicator is a one-line edit. + +* Non-Goals + +- A "minor modes" segment that surfaces every lighter from =minor-mode-alist=. Flycheck is the specific case we care about; the rest stay invisible. +- Reworking =flycheck-config.el= beyond the two =:custom= additions. +- Adding flycheck-side checkers or changing what gets checked. + +* Current State + +** =modules/flycheck-config.el:47-97= + +#+begin_src emacs-lisp +(use-package flycheck + :defer t + :commands (flycheck-list-errors cj/flycheck-list-errors) + :hook ((sh-mode emacs-lisp-mode) . flycheck-mode) + :bind (:map cj/custom-keymap ("?" . cj/flycheck-list-errors)) + :custom + (checkdoc-arguments + '(("sentence-end-double-space" nil) + ("warn-escape" nil))) + :config + ...) +#+end_src + +No flycheck-modeline customization. Defaults are in force: + +| Variable | Default | +|-------------------------------------+-------------------------------------------| +| =flycheck-mode-line-prefix= | ="FlyC"= | +| =flycheck-mode-success-indicator= | =":0"= | +| =flycheck-mode-line-color= | =t= (apply error / warning faces) | +| =flycheck-mode-line= | ='(:eval (flycheck-mode-line-status-text))= | + +** =modules/modeline-config.el:220-237= + +=mode-line-format= layout (left → right, with right-align edge): + +#+begin_src emacs-lisp +(setq-default mode-line-format + '("%e" + " " + cj/modeline-major-mode + " " + cj/modeline-buffer-name + " " + cj/modeline-position + mode-line-format-right-align + (:eval (when (fboundp 'cj/recording-modeline-indicator) + (cj/recording-modeline-indicator))) + cj/modeline-vc-branch + " " + cj/modeline-misc-info + " ")) +#+end_src + +Risky-local-variable list (=modeline-config.el:240-246=): + +#+begin_src emacs-lisp +(dolist (construct '(cj/modeline-buffer-name + cj/modeline-position + cj/modeline-vc-branch + cj/modeline-vc-faces + cj/modeline-major-mode + cj/modeline-misc-info)) + (put construct 'risky-local-variable t)) +#+end_src + +Note: =cj/modeline-vc-branch= and =cj/modeline-misc-info= both gate on +=(mode-line-window-selected-p)= so they appear only in the active window. + +** Flycheck lighter outputs (for reference) + +Flycheck status text values that =flycheck-mode-line-status-text= +returns, depending on =flycheck-last-status-change= and current errors: + +| Status | Display (with default prefix / indicator) | +|------------------------------+----------------------------------------------------| +| Not yet checked | =FlyC= | +| Currently checking | =FlyC*= | +| Finished, no errors | =FlyC:0= | +| Finished, 3 errors, 5 warns | =FlyC:3|5= | +| Checker errored | =FlyC!= | +| Interrupted | =FlyC.= | +| Suspicious | =FlyC?= | +| No checker available | =FlyC-= | + +With =flycheck-mode-line-color= = =t= (the default), the count portion +is colored: error count in the =error= face, warning count in =warning=. + +* Approaches Considered + +** Option 1 (Reject): customize prefix / indicator only + +Setting =flycheck-mode-line-prefix= and =flycheck-mode-success-indicator= +in =:custom= changes the lighter content, but the lighter still publishes +to =minor-mode-alist=, which the custom modeline doesn't read. The lighter +becomes prettier wherever it does show (e.g. doom-modeline if reinstated) +but not here. Doesn't solve the visibility problem. + +** Option 2 (Reject): add the raw =flycheck-mode-line= variable + +Inserting =flycheck-mode-line= into =mode-line-format= directly works, +but the form has no =flycheck-mode= guard. In a buffer where flycheck +isn't loaded or not enabled, the =:eval (flycheck-mode-line-status-text)= +call still fires and either errors or returns junk. Needs a wrapping +guard, which is what Option 4 does. + +** Option 3 (Reject for now): custom segment with full control + +Define =cj/modeline-flycheck= as a =defvar-local= holding a =(:eval ...)= +form that pulls error / warning counts directly from +=flycheck-current-errors=, builds a per-status string, propertizes it +with =error= / =warning= faces, and returns it. Reimplements what +=flycheck-mode-line-status-text= already does, with bespoke formatting. + +Pros: full control over format. Cons: maintenance burden, drifts from +flycheck's status model if flycheck changes it. + +If the Option 4 result ever stops being good enough -- e.g. you want a +different layout (=E:3 W:5= instead of =:3|5=) -- come back to this. +Until then, more code than the problem deserves. + +** Option 4 (Recommended): hybrid -- customize variables + add guarded segment + +Two changes: + +1. =modules/flycheck-config.el= =:custom= block gets prefix and success-indicator overrides. (Optional: also =flycheck-mode-line-color=.) + +2. =modules/modeline-config.el= adds a small =(:eval ...)= form inline in =mode-line-format= that guards on =flycheck-mode= and calls =(flycheck-mode-line-status-text)= directly. + +Pros: minimal code, uses flycheck's logic verbatim, prefix / indicator +swappable with a one-line edit, picks up flycheck's face colors +automatically. + +Cons: layout fixed to flycheck's =PREFIX[indicator|counts]= shape. +Acceptable. + +* Recommended Implementation (Option 4) + +** Step 1: =modules/flycheck-config.el= + +Add to the =:custom= block (currently lines 55-59 in the file): + +#+begin_src emacs-lisp +;; Modeline customization (rendered via mode-line-format in modeline-config.el). +(flycheck-mode-line-prefix "🐛") +(flycheck-mode-success-indicator " ✓") +;; flycheck-mode-line-color stays t (default) so counts keep their face coloring. +#+end_src + +Prefix and success indicator are taste; the **Emoji Reference** section +below catalogs the candidates. Note that the prefix emoji itself does +not inherit the =error= / =warning= face -- only the count portion does +(via =flycheck-mode-line-color=). That trade-off is fine for a static +prefix; an emoji prefix gives a recognizable shape that you scan for, +and the colored count carries the alert signal. + +** Step 2: =modules/modeline-config.el= + +Insert a =(:eval ...)= form into =mode-line-format= (currently lines +220-237). Recommended placement: between the recording indicator and +=cj/modeline-vc-branch= so flycheck status sits with the other +right-aligned status segments. + +After the change, the right-side block reads: + +#+begin_src emacs-lisp +;; RIGHT SIDE +mode-line-format-right-align +(:eval (when (fboundp 'cj/recording-modeline-indicator) + (cj/recording-modeline-indicator))) +(:eval (when (and (mode-line-window-selected-p) + (bound-and-true-p flycheck-mode)) + (flycheck-mode-line-status-text))) +" " +cj/modeline-vc-branch +" " +cj/modeline-misc-info +" ") +#+end_src + +Two design choices baked in: + +- =(mode-line-window-selected-p)= gates the segment to the active window, matching the convention used by =cj/modeline-vc-branch= and =cj/modeline-misc-info=. +- =(bound-and-true-p flycheck-mode)= prevents the function call in buffers where flycheck never loaded; safer than asking =flycheck-mode= directly. + +** Risky-local-variable: not needed here + +This implementation places =(:eval ...)= inline inside =mode-line-format= +rather than wrapping it in a =defvar-local=. Inline forms are evaluated +by mode-line processing without a risky-local-variable marker. The +existing risky list (=modeline-config.el:240-246=) does not need to +grow. + +(If you ever refactor this to a named segment -- =defvar-local cj/modeline-flycheck= -- then add it to the risky list. Option 3 above is the path that needs that step.) + +* Emoji Reference + +** Prefix candidates (=flycheck-mode-line-prefix=) + +| Glyph | Codepoint | Name | +|-------+-----------+---------------------------------------| +| 🪰 | U+1FAB0 | FLY (literal "fly" for flycheck) | +| 🐛 | U+1F41B | BUG (recommended -- broadest font support) | +| 🐞 | U+1F41E | LADY BEETLE | +| ⚠ | U+26A0 | WARNING SIGN | +| 🔍 | U+1F50D | MAGNIFYING GLASS | +| 📝 | U+1F4DD | MEMO | +| ✓ | U+2713 | CHECK MARK (text) | + +🪰 (U+1FAB0) is from Unicode 13.0 (2020) and needs an up-to-date emoji +font. 🐛 (U+1F41B) is older and renders everywhere. Default to 🐛 unless +the fly is a strong preference and the GUI fonts are known to cover it. + +** Success indicator candidates (=flycheck-mode-success-indicator=) + +| Glyph | Codepoint | Name | +|-------+-----------+---------------------------------------| +| ✓ | U+2713 | CHECK MARK (text) | +| ✔ | U+2714 | HEAVY CHECK MARK | +| ✅ | U+2705 | WHITE HEAVY CHECK MARK (green box) | +| 🟢 | U+1F7E2 | GREEN CIRCLE | +| ⭐ | U+2B50 | WHITE MEDIUM STAR | + +Note the leading space in the recommended setting (=" ✓"=): flycheck +joins the prefix and the success indicator with no separator, so a +leading space in the indicator gives breathing room between the emoji +prefix and the check mark. + +** Suggested combinations + +| Mood | Prefix | Success indicator | Result example | +|---------------------+--------+-------------------+----------------| +| Recommended default | 🐛 | " ✓" | =🐛 ✓= / =🐛:3|5= | +| Literal Flycheck | 🪰 | " ✓" | =🪰 ✓= / =🪰:3|5= | +| Minimal | "" | " ✓" | = ✓= / =:3|5= | +| Status light | "" | " 🟢" | = 🟢= / =:3|5= | + +* Testing + +** Manual + +1. Open =modules/flycheck-config.el= (an =emacs-lisp-mode= buffer with =flycheck-mode= auto-enabled per the existing =:hook=). The right side of the modeline shows the prefix + success indicator when there are no errors. +2. Introduce a deliberate parse error (drop a paren). Save. The modeline updates to show =:1|0= (or whatever count) in the =error= face. +3. Trigger =M-x flycheck-buffer= in a fresh =sh-mode= buffer. The "currently checking" state (=PREFIX*=) flashes briefly before settling on success or counts. +4. Open a second window onto the same buffer (=C-x 2=). The flycheck segment appears in the active window only; the inactive copy drops it. Confirms the active-window gate. +5. Open a buffer where flycheck never engages (e.g. =*scratch*= in fundamental-mode, or a =dired= buffer). No segment, no errors. +6. Run =cj/flycheck-prose-on-demand= in an org buffer (=C-; ?= in org-mode). The LanguageTool checker engages and the segment appears with prose-error counts. + +** Regression watch + +- The custom-modeline width should not jump distractingly as flycheck cycles "checking → finished". The status text is short (one to seven chars), so this should be invisible -- worth a glance. +- Inactive-window display: confirm the segment disappears, not just greys out. The current pattern is "hide entirely" via the =mode-line-window-selected-p= guard. +- =cj/modeline-misc-info= keeps showing chime / notification text. The flycheck segment sits to its left; verify the visual order matches the spec. + +* Files to Modify + +- =modules/flycheck-config.el= -- add two =:custom= lines. +- =modules/modeline-config.el= -- insert one =(:eval ...)= form into =mode-line-format=. + +Two-line / one-form change. No new tests required (the existing tests +don't lock the modeline content; they exercise behavior elsewhere). If +you want a smoke test, add one assertion in =tests/test-modeline-config.el= +(if that file exists or you create it) that =mode-line-format='s sexp +contains a form mentioning =flycheck-mode-line-status-text=. Optional. + +* Risks + +| Risk | Mitigation | +|-----------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------| +| Emoji renders as a tofu square in terminal Emacs | The user runs GUI Emacs primarily; if terminal use matters, set the prefix to a text glyph (=""= or =":"=) instead. | +| Modeline width thrash when flycheck transitions running → finished | Status text is one to seven chars; jitter is negligible. Confirm during manual testing. | +| Prefix emoji doesn't pick up =error= / =warning= face | Expected: =flycheck-mode-line-color= colors the count portion only. The static prefix is intentionally unstyled. If you want a colored prefix, switch to Option 3. | +| Flycheck not yet loaded when modeline first evaluates | The =(bound-and-true-p flycheck-mode)= guard returns nil in that case, the =(:eval ...)= returns nil, mode-line skips the slot. | +| Active-window gate is wrong for some workflow (e.g. multi-window comparison) | Drop =(mode-line-window-selected-p)=. One-line change. Decide after living with the default. | + +* Rollback + +Revert the commit. Two-file change, no schema impact. Idempotent. + +* Effort estimate + +S (under 1 hour). Two lines in =flycheck-config.el=, one form in +=modeline-config.el=, plus the manual verification walk-through. The +emoji selection is the time sink, not the code. diff --git a/docs/specs/gloss-spec-doing.org b/docs/specs/gloss-spec-doing.org deleted file mode 100644 index 320b83eb..00000000 --- a/docs/specs/gloss-spec-doing.org +++ /dev/null @@ -1,320 +0,0 @@ -:PROPERTIES: -:ID: 295f9969-ccef-4df9-945b-9e08d8069daf -:STATUS: doing -:END: -#+TITLE: Design — gloss (Glossary Lookup with Online-Sourced Selection) -#+DATE: 2026-04-28 -#+STATUS: Draft - -* Problem - -A personal glossary inside Emacs, modelled on the existing =quick-sdcv= UX (=C-h d=) but for self-curated terms rather than packaged dictionaries. =C-h g= prompts for a term (defaulting to word-at-point), looks it up in a single git-tracked org file, and shows the definition in a side buffer that =q= dismisses. On a local miss, the package fetches candidate definitions from an online source, lets the user pick one, and saves it with provenance. The same org file feeds =org-drill= for spaced-repetition study. - -The pain point: domain jargon — government acronyms, technical terms, philosophy vocabulary, project-specific names — doesn't live in any general dictionary, so existing tools like =quick-sdcv= can't help. A personal glossary that grows by use (encounter term → save it once → it's permanently looked-up-able and study-card-able) closes that gap. - -* Non-Goals - -The following are explicitly out of scope for v1. Each is a defensible v2+ topic on its own. - -- *Multi-language support.* English only. Wiktionary returns French/Latin/etc. — v1 ignores everything but the =en= key. -- *Synonyms, cross-references, related terms.* Even when the upstream source returns them, v1 stores only the picked definition. -- *Audio pronunciation.* Not fetched, not played. -- *Etymology, usage notes, parsed examples.* Discarded during HTML strip. -- *Multiple glossaries / domain separation.* One file, one glossary. -- *Backup or sync infrastructure.* Delegated to git on whatever path =gloss-file= points at. -- *Org-drill scheduling control.* The exporter prepares entries; =org-drill= itself runs unmodified. - -In scope (kept after triage): edit-in-place via =C-h g e=, which jumps to the source file at the entry's heading. - -* Approaches Considered - -Six approaches evaluated during brainstorm. Three conventional, three tail samples for diversity. - -** Recommended: Layered multi-module package - -Five =.el= files, each owning one concern: =gloss-core= (data), =gloss-fetch= (network), =gloss-display= (UI), =gloss-drill= (drill export), =gloss= (orchestration entry point). Each layer mocks at its own natural boundary; no layer mocks another layer's internals. - -*Why this over the alternatives.* The codebase already prefers layering — =coverage-core= + =coverage-elisp= split, Hugo pure-helpers + interactive wrappers, LSP file-watch defvar + function. The four concerns (data, fetch, display, drill) have genuinely different test boundaries (file I/O, HTTP, mode UI, =org-element=). Mixing them in one file would force overmocking, which the project's testing rules flag as a smell. The package is also public-style — clear module boundaries reward cold readers. - -*What's traded away.* About 30 minutes more structural setup at the start, in exchange for boilerplate that may never pay off if the package stays personal forever. Cheap trade against the testing and reading wins. - -** Rejected: Single-file quick-sdcv-clone - -One =.el= file (~400 lines) covering all four concerns. Simplest path, lowest dependency footprint, but everything (data, HTTP, mode definition, drill) cohabits a single namespace. Test isolation gets awkward; refactor cost grows when one piece needs replacing. - -** Rejected: Backend-pluggable registry - -A =glossary-backend= protocol covering both local-org and online sources, with =lookup= / =save= / =list= operations. Local and online become interchangeable backends. Real future-proofing, but for v1 with two backends and probably never a third, the protocol is overkill — YAGNI risk. The forward-compat shape we did adopt (the =gloss-fetch-sources= registry, see Architecture) gets the same benefit at a fraction of the design weight, scoped only to where source variety is real. - -** Rejected: quick-sdcv + generated StarDict - -Round-trip the org file through StarDict format on save; reuse =quick-sdcv='s UI verbatim. Reuses 100% of an existing UI but loses provenance metadata in the round-trip, fights drill (which reads org, not StarDict), and forces a binary intermediate format for what should be a plain-text data store. - -** Rejected: Org-roam node per term - -Each entry is its own =org-roam= node. Free fuzzy/exact title search, free backlinks. But it's a heavy dependency for an otherwise self-contained package, file-explodes (1000 terms = 1000 files), and contradicts the locked single-file storage decision. - -** Rejected: Lazy-reactive minor mode - -Passive recognition — =gloss-mode= scans buffer text for known terms, underlines them, hover/click reveals definitions. Different and arguably more-natural mental model, but it reframes the brief (active =C-h g= lookup is what was asked for) and doesn't naturally support online fallback or auto-add. Probably belongs as a v3 feature on top of the layered architecture, not as the architecture itself. - -* Design - -** Architecture - -Five =.el= files: - -#+begin_example -gloss-core.el data layer — org file I/O + in-memory cache -gloss-fetch.el network layer — Wiktionary REST + HTML strip -gloss-display.el UI layer — side buffer + picker -gloss-drill.el drill export — :drill: tag + twosided property -gloss.el entry point — defcustoms, prefix keymap, user commands -#+end_example - -*Public API by layer.* - -=gloss-core=: =gloss-core-lookup TERM=, =gloss-core-save TERM DEFINITION SOURCE=, =gloss-core-list=, =gloss-core-find-buffer-position TERM=. - -=gloss-fetch=: =gloss-fetch-definitions TERM= → =(:ok DEFS) | (:empty :no-defs SOURCES :failed SOURCES)=. Internally a registry: =gloss-fetch--sources= alist (source-symbol → fetcher function), walked in order per the user-facing =gloss-fetch-sources= defcustom. - -=gloss-display=: =gloss-display-show-entry TERM BODY=, =gloss-display-pick-definition TERM DEFINITIONS=. Defines =gloss-mode= (derived from =special-mode=, =q= quits). - -=gloss-drill=: =gloss-drill-export-all=, =gloss-drill-untag-all=. Operates on the org file via =org-element=. - -=gloss=: =defcustom gloss-file= (path), =gloss-prefix-map= for =C-h g=, user commands =gloss-lookup=, =gloss-add=, =gloss-edit=, =gloss-fetch-online=, =gloss-drill-export=. - -** Data Flow - -*Shapes.* - -A definition (in flight from fetch through display to save) is a plist: - -#+begin_src emacs-lisp -(:source wiktionary :text "Reference to something earlier in the discourse...") -#+end_src - -An entry (saved in cache and on disk) is a plist: - -#+begin_src emacs-lisp -(:term "anaphora" - :body "Reference to something earlier in the discourse..." - :source wiktionary - :added "2026-04-28" - :marker #) -#+end_src - -The cache is a hash table, term-string → entry-plist. The org file is the source of truth; the cache is a read-side index. - -*Lookup flow (=C-h g=).* - -1. Read input — word-at-point if available, else minibuffer prompt. -2. =gloss-core-lookup TERM=. Cache loaded if cold. -3. Hit → =gloss-display-show-entry=. Done. -4. Miss → silent fall-through to =gloss-fetch-definitions TERM=. -5. Orchestrate on result: - - 0 definitions or all-failures → side buffer message (see Error Handling). - - 1 definition → auto-save via =gloss-core-save=, then =gloss-display-show-entry=. - - >1 definitions → =gloss-display-pick-definition= → user picks → =gloss-core-save= → =gloss-display-show-entry=. - -*Add flow (=C-h g a=).* - -=gloss-add= prompts for term and body (small temp buffer for multi-line body, =C-c C-c= accepts). =gloss-core-save TERM BODY 'manual=. Then =gloss-display-show-entry=. - -*Edit flow (=C-h g e=).* - -=gloss-edit= resolves the term to a buffer position via =gloss-core-find-buffer-position=. Opens the org file at that heading in the *source* buffer (not the side buffer). User edits inline. On save, the buffer-local =after-save-hook= refreshes the cache for that single term. - -*Drill export (=C-h g D=).* - -=gloss-drill-export-all= walks the org file via =org-element=, ensures every term heading has =:drill:= tag and =:DRILL_CARD_TYPE: twosided= property. =M-x org-drill= runs the session — gloss does not wrap or invoke =org-drill= itself. - -** Persistence - -*File shape.* Single org file at =gloss-file= (default: =(expand-file-name "gloss.org" (or org-directory user-emacs-directory))=). One =* term= heading per entry, alphabetical order maintained on insert. Each entry has a =:PROPERTIES:= drawer with =:SOURCE:= and =:ADDED:=. Body is plain text immediately under the heading. - -#+begin_example -#+TITLE: Glossary -#+STARTUP: showall - -* anaphora -:PROPERTIES: -:SOURCE: wiktionary -:ADDED: 2026-04-28 -:END: -Reference to something earlier in the discourse... - -* SBIR -:PROPERTIES: -:SOURCE: wiktionary -:ADDED: 2026-04-28 -:END: -Initialism of Small Business Innovation Research... -#+end_example - -After =gloss-drill-export-all=, the heading line gains a =:drill:= tag and the properties drawer gains =:DRILL_CARD_TYPE: twosided=. - -*Cache lifecycle.* Hash table loaded lazily on first lookup of the session. Populated by reading =gloss-file= once and parsing with =org-element-parse-buffer=. Subsequent lookups hit the cache directly. - -*Cache invalidation.* Four triggers, in order of cost: - -1. =gloss-core-save= mutates the cache directly when it writes. -2. *mtime check on every lookup.* =file-attributes= the file before each =gloss-core-lookup= returns; if mtime > cached-mtime, reload before answering. Sub-millisecond cost; catches every out-of-band edit (other Emacs session, =git pull=, hand-edit, =sed=). -3. =gloss-edit='s buffer-local =after-save-hook= updates the single edited term immediately; overlaps with #2 but doesn't wait for the next lookup. -4. Manual =gloss-reload= command — nuclear option for paranoia. - -=file-notify-add-watch= rejected: platform-specific backend, async callback complicates the model, mtime path is already sub-millisecond. - -*Write strategy.* Append-on-add via direct buffer editing (=find-file-noselect=, insert at the alphabetically-correct heading position, save, kill the buffer if not previously open). No journal, no temp file — org-mode's =auto-save-mode= and the user's git tracking provide durability. Single-user, single-Emacs assumed; concurrent access isn't a concern. - -*Alphabetical order.* Maintained on insert via case-insensitive string compare. Cheap; the file stays diff-clean (only the inserted block changes). - -** Error Handling - -*Per-source status taxonomy.* Five internal values; three user-facing rollups. - -#+begin_src emacs-lisp -;; Internal per-source result: -(:source SYM :status STATUS :reason STRING) - -;; STATUS values: -;; :ok :defs (def1 def2 ...) — success -;; :no-defs — server reached, term not there (HTTP 404 or empty 200) -;; :unreachable — network problem (DNS, refused, timeout) -;; :server-error — HTTP 5xx, malformed JSON, schema mismatch, HTTP 4xx other than 404/429 -;; :rate-limited — HTTP 429 -#+end_src - -*=:reason= strings* carry the technical detail (=timeout (5s)=, =HTTP 503=, =malformed JSON: ...=) and land in =*gloss-debug*=. They are never user-facing. - -*User-facing rollup.* =gloss-fetch-definitions= aggregates per-source results into: - -#+begin_src emacs-lisp -(:ok DEFS) ;; any source returned >=1 def -(:empty :no-defs (...) :failed (...)) ;; everything else -#+end_src - -=:failed= unions =:unreachable=, =:server-error=, =:rate-limited=. - -| Result shape | Message | -|-------------------------------------------+--------------------------------------------------------------------| -| Every source =:no-defs=, none failed | "No definition for X in Wiktionary." | -| Every source failed, none =:no-defs= | "Couldn't reach Wiktionary." | -| Mix of =:no-defs= and failures | "No definition in Wiktionary; couldn't reach DictionaryAPI." | -| Any =:ok= with defs | Silent on others — picker shows what came back | - -When v2 starts surfacing =:rate-limited= regularly, the rollup wording will gain a third visible category. v1 with no-key Wiktionary doesn't need it. - -*libxml as a precondition, not a per-source failure.* First time =gloss-fetch-definitions= runs, probe =(libxml-parse-html-region 1 1)= on a temp buffer. If unavailable, online fetching is disabled package-wide for the session with a one-shot =user-error=: "Online fetch requires Emacs built with libxml2; manual add still works." Subsequent online attempts in the session short-circuit to that message. - -*Partial-success on per-sense HTML failures.* If libxml is available but fails on a specific sense's content, drop that sense and return the rest. Source status stays =:ok= with N-1 entries; the dropped sense logs to =*gloss-debug*=. A single bad sense doesn't poison the whole source. - -*Storage failures.* First call creates =gloss-file= and any missing parent directory with a =#+TITLE: Glossary= header. Permission denied raises =user-error= naming the path. Corrupt org file (=org-element-parse-buffer= raises) preserves the existing cache and surfaces "glossary file corrupt at line N; cache not refreshed" — operations fall back to the stale cache until the user fixes the file and runs =gloss-reload=. Term collision (saving an existing term) prompts: replace, append-with-separator, or cancel. - -*Drill.* =org-drill= checked via =featurep= before export runs. If absent: =user-error= with install hint. - -*User cancellations.* =C-g= during the picker → no save, side buffer shows the local-miss state. Empty term input from =gloss-add= → re-prompt once, then abort silently. Cancelled at the term-collision prompt → no write. - -** Testing - -Per-function test files; three categories (Normal/Boundary/Error) per function. TDD by default. Real production code via =require=, never inlined. - -*=gloss-core=.* Temp files + real =org-element-parse-buffer=. No mocking — exercises the actual file I/O and parser. - -#+begin_example -test-gloss-core--lookup.el -test-gloss-core--save.el -test-gloss-core--invalidate-on-mtime.el -test-gloss-core--corrupt-file-preserves-cache.el -test-gloss-core--alphabetical-insert.el -test-gloss-core--first-call-creates-file.el -#+end_example - -*=gloss-fetch=.* =cl-letf= mock on =url-retrieve-synchronously=, injecting canned response buffers. Captured Wiktionary fixtures in =tests/fixtures/wiktionary-*.json= — real responses for SBIR, anaphora, API, frozen once, replayed forever. - -#+begin_example -test-gloss-fetch--definitions-200-returns-ok.el -test-gloss-fetch--definitions-404-returns-no-defs.el -test-gloss-fetch--definitions-500-returns-server-error.el -test-gloss-fetch--definitions-timeout-returns-unreachable.el -test-gloss-fetch--strip-html.el -test-gloss-fetch--multi-source-walks-registry.el -test-gloss-fetch--libxml-probe.el -#+end_example - -*=gloss-display=.* The candidate-formatting helper =gloss-display--format-candidate PLIST → "[wiktionary] text..."= is pure → full N/B/E coverage. =gloss-display-show-entry= and =gloss-mode= get one smoke test each (Emacs already tests =switch-to-buffer= and major-mode definition). - -#+begin_example -test-gloss-display--format-candidate.el -test-gloss-display--show-entry-smoke.el -#+end_example - -*=gloss-drill=.* Temp file + real =org-element=. Tests assert tag/property changes on entries. - -#+begin_example -test-gloss-drill--export-all-tags-untagged.el -test-gloss-drill--export-all-skips-already-tagged.el -test-gloss-drill--export-all-no-orgdrill-installed.el -test-gloss-drill--untag-all.el -#+end_example - -*=gloss=.* The orchestration policy =gloss--orchestrate-fetch-result RESULT → SYMBOL= is a pure pattern-matcher. Tested with shaped inputs covering every result variant. - -#+begin_example -test-gloss--orchestrate-fetch-result.el -#+end_example - -*Integration tests.* Three small ones, each with a docstring naming participants per project convention. - -#+begin_example -test-integration-gloss-lookup-flow-local-hit.el -test-integration-gloss-lookup-flow-online-fall-through.el -test-integration-gloss-lookup-flow-online-failure.el -#+end_example - -*Coverage targets.* 90%+ on =gloss-core=, =gloss-fetch=, =gloss-drill=, and pure helpers in =gloss-display= / =gloss=. 70%+ on display mode-glue. Overall ≥80%. - -** Observability - -*=*gloss-debug*= log buffer.* Off until =gloss-debug= defcustom is non-nil, or session-only =gloss-toggle-debug= flips it. One timestamped, layer-prefixed line per significant event. - -#+begin_example -2026-04-28 11:14:02 [fetch:wiktionary] GET /API → 200, 12 senses -2026-04-28 11:14:02 [fetch:wiktionary] sense 7 HTML parse failed, dropping -2026-04-28 11:14:02 [core] cache hit for "anaphora" -2026-04-28 11:14:09 [core] mtime change detected, reloading cache (47 terms) -2026-04-28 11:14:11 [save] "API" → wiktionary, 11 alts not saved -#+end_example - -Per-source statuses from Error Handling land here verbatim. No personal data beyond user-supplied terms. - -*=*Messages*= for user-facing events.* Saves, picker-shown, "no definition found" messages — short single-line =message= calls, persisted in =*Messages*= via Emacs idiom. Strict separation: =*Messages*= for things the user did or asked for; =*gloss-debug*= for everything else. - -*Inspection commands.* - -- =gloss-list-terms= — completing-read over every term in the cache. Pick one to jump to it. -- =gloss-stats= — small buffer summarizing total terms, breakdown by =:source=, count of drill-tagged entries, file size, cache mtime. - -No metrics export, no telemetry, no profiling hooks — v3 territory if the package ever needs them. - -* Open Questions (will become ADRs) - -Each was decided during the brainstorm. Listed for traceability; each becomes an ADR in the gloss repo's =docs/decisions/=. - -- [ ] *ADR-1: storage path default* → =(expand-file-name "gloss.org" (or org-directory user-emacs-directory))=. Rationale: respects the user's existing =org-directory= convention; falls back gracefully. -- [ ] *ADR-2: auto-fetch on local miss* → silent fall-through with graceful network-failure path. Rationale: y/n prompt is yes 99% of the time and an annoyance the other 1%; the offline case is better handled by detecting the failure than by pre-asking permission. -- [ ] *ADR-3: drill direction* → =:DRILL_CARD_TYPE: twosided=. Rationale: tests both recognition and recall over time without doubling the deck. -- [ ] *ADR-4: HTML strip strategy* → =libxml-parse-html-region= (plain text only, no italic/bold preservation). Rationale: more robust than regex on edge cases; libxml2 is standard on Linux/Mac; ~30 lines. - -* Next Steps - -1. *Scaffold the repo.* =~/code/gloss= with the claude-template structure: =.ai/= and =todo.org= and =inbox/= gitignored, =Makefile= for tests/lint/compile, =README.org= placeholder, =LICENSE=, package skeleton (=gloss.el= with package-header autoload entry). -2. *Set up remotes.* Bare repo on cjennings.net at =/var/cjennings/git/gloss.git/= with the existing post-receive hook pattern that mirrors to =github.com/cjennings/gloss=. -3. *Decompose into todo.org tasks.* One TODO per layer, in implementation order: core → fetch → display → drill → entry-point → integration tests → README. Each task carries its acceptance criteria from this design. -4. *Implement v1 layer by layer*, TDD per project rules. Run =/start-work= once per task. -5. *First-week shakedown.* Use the package on real terms for a week. File issues against any rough edges as v1.1 tasks. -6. *Record the four ADRs* in =docs/decisions/= once the repo exists. - -* Status - -Draft. Pending: repo scaffold, ADR records, implementation. diff --git a/docs/specs/gloss-spec.org b/docs/specs/gloss-spec.org new file mode 100644 index 00000000..06a7bf50 --- /dev/null +++ b/docs/specs/gloss-spec.org @@ -0,0 +1,323 @@ +#+TITLE: Design — gloss (Glossary Lookup with Online-Sourced Selection) +#+DATE: 2026-04-28 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Design — gloss (Glossary Lookup with Online-Sourced Selection) +:PROPERTIES: +:ID: 295f9969-ccef-4df9-945b-9e08d8069daf +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DOING from existing :STATUS: doing + +* Problem + +A personal glossary inside Emacs, modelled on the existing =quick-sdcv= UX (=C-h d=) but for self-curated terms rather than packaged dictionaries. =C-h g= prompts for a term (defaulting to word-at-point), looks it up in a single git-tracked org file, and shows the definition in a side buffer that =q= dismisses. On a local miss, the package fetches candidate definitions from an online source, lets the user pick one, and saves it with provenance. The same org file feeds =org-drill= for spaced-repetition study. + +The pain point: domain jargon — government acronyms, technical terms, philosophy vocabulary, project-specific names — doesn't live in any general dictionary, so existing tools like =quick-sdcv= can't help. A personal glossary that grows by use (encounter term → save it once → it's permanently looked-up-able and study-card-able) closes that gap. + +* Non-Goals + +The following are explicitly out of scope for v1. Each is a defensible v2+ topic on its own. + +- *Multi-language support.* English only. Wiktionary returns French/Latin/etc. — v1 ignores everything but the =en= key. +- *Synonyms, cross-references, related terms.* Even when the upstream source returns them, v1 stores only the picked definition. +- *Audio pronunciation.* Not fetched, not played. +- *Etymology, usage notes, parsed examples.* Discarded during HTML strip. +- *Multiple glossaries / domain separation.* One file, one glossary. +- *Backup or sync infrastructure.* Delegated to git on whatever path =gloss-file= points at. +- *Org-drill scheduling control.* The exporter prepares entries; =org-drill= itself runs unmodified. + +In scope (kept after triage): edit-in-place via =C-h g e=, which jumps to the source file at the entry's heading. + +* Approaches Considered + +Six approaches evaluated during brainstorm. Three conventional, three tail samples for diversity. + +** Recommended: Layered multi-module package + +Five =.el= files, each owning one concern: =gloss-core= (data), =gloss-fetch= (network), =gloss-display= (UI), =gloss-drill= (drill export), =gloss= (orchestration entry point). Each layer mocks at its own natural boundary; no layer mocks another layer's internals. + +*Why this over the alternatives.* The codebase already prefers layering — =coverage-core= + =coverage-elisp= split, Hugo pure-helpers + interactive wrappers, LSP file-watch defvar + function. The four concerns (data, fetch, display, drill) have genuinely different test boundaries (file I/O, HTTP, mode UI, =org-element=). Mixing them in one file would force overmocking, which the project's testing rules flag as a smell. The package is also public-style — clear module boundaries reward cold readers. + +*What's traded away.* About 30 minutes more structural setup at the start, in exchange for boilerplate that may never pay off if the package stays personal forever. Cheap trade against the testing and reading wins. + +** Rejected: Single-file quick-sdcv-clone + +One =.el= file (~400 lines) covering all four concerns. Simplest path, lowest dependency footprint, but everything (data, HTTP, mode definition, drill) cohabits a single namespace. Test isolation gets awkward; refactor cost grows when one piece needs replacing. + +** Rejected: Backend-pluggable registry + +A =glossary-backend= protocol covering both local-org and online sources, with =lookup= / =save= / =list= operations. Local and online become interchangeable backends. Real future-proofing, but for v1 with two backends and probably never a third, the protocol is overkill — YAGNI risk. The forward-compat shape we did adopt (the =gloss-fetch-sources= registry, see Architecture) gets the same benefit at a fraction of the design weight, scoped only to where source variety is real. + +** Rejected: quick-sdcv + generated StarDict + +Round-trip the org file through StarDict format on save; reuse =quick-sdcv='s UI verbatim. Reuses 100% of an existing UI but loses provenance metadata in the round-trip, fights drill (which reads org, not StarDict), and forces a binary intermediate format for what should be a plain-text data store. + +** Rejected: Org-roam node per term + +Each entry is its own =org-roam= node. Free fuzzy/exact title search, free backlinks. But it's a heavy dependency for an otherwise self-contained package, file-explodes (1000 terms = 1000 files), and contradicts the locked single-file storage decision. + +** Rejected: Lazy-reactive minor mode + +Passive recognition — =gloss-mode= scans buffer text for known terms, underlines them, hover/click reveals definitions. Different and arguably more-natural mental model, but it reframes the brief (active =C-h g= lookup is what was asked for) and doesn't naturally support online fallback or auto-add. Probably belongs as a v3 feature on top of the layered architecture, not as the architecture itself. + +* Design + +** Architecture + +Five =.el= files: + +#+begin_example +gloss-core.el data layer — org file I/O + in-memory cache +gloss-fetch.el network layer — Wiktionary REST + HTML strip +gloss-display.el UI layer — side buffer + picker +gloss-drill.el drill export — :drill: tag + twosided property +gloss.el entry point — defcustoms, prefix keymap, user commands +#+end_example + +*Public API by layer.* + +=gloss-core=: =gloss-core-lookup TERM=, =gloss-core-save TERM DEFINITION SOURCE=, =gloss-core-list=, =gloss-core-find-buffer-position TERM=. + +=gloss-fetch=: =gloss-fetch-definitions TERM= → =(:ok DEFS) | (:empty :no-defs SOURCES :failed SOURCES)=. Internally a registry: =gloss-fetch--sources= alist (source-symbol → fetcher function), walked in order per the user-facing =gloss-fetch-sources= defcustom. + +=gloss-display=: =gloss-display-show-entry TERM BODY=, =gloss-display-pick-definition TERM DEFINITIONS=. Defines =gloss-mode= (derived from =special-mode=, =q= quits). + +=gloss-drill=: =gloss-drill-export-all=, =gloss-drill-untag-all=. Operates on the org file via =org-element=. + +=gloss=: =defcustom gloss-file= (path), =gloss-prefix-map= for =C-h g=, user commands =gloss-lookup=, =gloss-add=, =gloss-edit=, =gloss-fetch-online=, =gloss-drill-export=. + +** Data Flow + +*Shapes.* + +A definition (in flight from fetch through display to save) is a plist: + +#+begin_src emacs-lisp +(:source wiktionary :text "Reference to something earlier in the discourse...") +#+end_src + +An entry (saved in cache and on disk) is a plist: + +#+begin_src emacs-lisp +(:term "anaphora" + :body "Reference to something earlier in the discourse..." + :source wiktionary + :added "2026-04-28" + :marker #) +#+end_src + +The cache is a hash table, term-string → entry-plist. The org file is the source of truth; the cache is a read-side index. + +*Lookup flow (=C-h g=).* + +1. Read input — word-at-point if available, else minibuffer prompt. +2. =gloss-core-lookup TERM=. Cache loaded if cold. +3. Hit → =gloss-display-show-entry=. Done. +4. Miss → silent fall-through to =gloss-fetch-definitions TERM=. +5. Orchestrate on result: + - 0 definitions or all-failures → side buffer message (see Error Handling). + - 1 definition → auto-save via =gloss-core-save=, then =gloss-display-show-entry=. + - >1 definitions → =gloss-display-pick-definition= → user picks → =gloss-core-save= → =gloss-display-show-entry=. + +*Add flow (=C-h g a=).* + +=gloss-add= prompts for term and body (small temp buffer for multi-line body, =C-c C-c= accepts). =gloss-core-save TERM BODY 'manual=. Then =gloss-display-show-entry=. + +*Edit flow (=C-h g e=).* + +=gloss-edit= resolves the term to a buffer position via =gloss-core-find-buffer-position=. Opens the org file at that heading in the *source* buffer (not the side buffer). User edits inline. On save, the buffer-local =after-save-hook= refreshes the cache for that single term. + +*Drill export (=C-h g D=).* + +=gloss-drill-export-all= walks the org file via =org-element=, ensures every term heading has =:drill:= tag and =:DRILL_CARD_TYPE: twosided= property. =M-x org-drill= runs the session — gloss does not wrap or invoke =org-drill= itself. + +** Persistence + +*File shape.* Single org file at =gloss-file= (default: =(expand-file-name "gloss.org" (or org-directory user-emacs-directory))=). One =* term= heading per entry, alphabetical order maintained on insert. Each entry has a =:PROPERTIES:= drawer with =:SOURCE:= and =:ADDED:=. Body is plain text immediately under the heading. + +#+begin_example +#+TITLE: Glossary +#+STARTUP: showall + +* anaphora +:PROPERTIES: +:SOURCE: wiktionary +:ADDED: 2026-04-28 +:END: +Reference to something earlier in the discourse... + +* SBIR +:PROPERTIES: +:SOURCE: wiktionary +:ADDED: 2026-04-28 +:END: +Initialism of Small Business Innovation Research... +#+end_example + +After =gloss-drill-export-all=, the heading line gains a =:drill:= tag and the properties drawer gains =:DRILL_CARD_TYPE: twosided=. + +*Cache lifecycle.* Hash table loaded lazily on first lookup of the session. Populated by reading =gloss-file= once and parsing with =org-element-parse-buffer=. Subsequent lookups hit the cache directly. + +*Cache invalidation.* Four triggers, in order of cost: + +1. =gloss-core-save= mutates the cache directly when it writes. +2. *mtime check on every lookup.* =file-attributes= the file before each =gloss-core-lookup= returns; if mtime > cached-mtime, reload before answering. Sub-millisecond cost; catches every out-of-band edit (other Emacs session, =git pull=, hand-edit, =sed=). +3. =gloss-edit='s buffer-local =after-save-hook= updates the single edited term immediately; overlaps with #2 but doesn't wait for the next lookup. +4. Manual =gloss-reload= command — nuclear option for paranoia. + +=file-notify-add-watch= rejected: platform-specific backend, async callback complicates the model, mtime path is already sub-millisecond. + +*Write strategy.* Append-on-add via direct buffer editing (=find-file-noselect=, insert at the alphabetically-correct heading position, save, kill the buffer if not previously open). No journal, no temp file — org-mode's =auto-save-mode= and the user's git tracking provide durability. Single-user, single-Emacs assumed; concurrent access isn't a concern. + +*Alphabetical order.* Maintained on insert via case-insensitive string compare. Cheap; the file stays diff-clean (only the inserted block changes). + +** Error Handling + +*Per-source status taxonomy.* Five internal values; three user-facing rollups. + +#+begin_src emacs-lisp +;; Internal per-source result: +(:source SYM :status STATUS :reason STRING) + +;; STATUS values: +;; :ok :defs (def1 def2 ...) — success +;; :no-defs — server reached, term not there (HTTP 404 or empty 200) +;; :unreachable — network problem (DNS, refused, timeout) +;; :server-error — HTTP 5xx, malformed JSON, schema mismatch, HTTP 4xx other than 404/429 +;; :rate-limited — HTTP 429 +#+end_src + +*=:reason= strings* carry the technical detail (=timeout (5s)=, =HTTP 503=, =malformed JSON: ...=) and land in =*gloss-debug*=. They are never user-facing. + +*User-facing rollup.* =gloss-fetch-definitions= aggregates per-source results into: + +#+begin_src emacs-lisp +(:ok DEFS) ;; any source returned >=1 def +(:empty :no-defs (...) :failed (...)) ;; everything else +#+end_src + +=:failed= unions =:unreachable=, =:server-error=, =:rate-limited=. + +| Result shape | Message | +|-------------------------------------------+--------------------------------------------------------------------| +| Every source =:no-defs=, none failed | "No definition for X in Wiktionary." | +| Every source failed, none =:no-defs= | "Couldn't reach Wiktionary." | +| Mix of =:no-defs= and failures | "No definition in Wiktionary; couldn't reach DictionaryAPI." | +| Any =:ok= with defs | Silent on others — picker shows what came back | + +When v2 starts surfacing =:rate-limited= regularly, the rollup wording will gain a third visible category. v1 with no-key Wiktionary doesn't need it. + +*libxml as a precondition, not a per-source failure.* First time =gloss-fetch-definitions= runs, probe =(libxml-parse-html-region 1 1)= on a temp buffer. If unavailable, online fetching is disabled package-wide for the session with a one-shot =user-error=: "Online fetch requires Emacs built with libxml2; manual add still works." Subsequent online attempts in the session short-circuit to that message. + +*Partial-success on per-sense HTML failures.* If libxml is available but fails on a specific sense's content, drop that sense and return the rest. Source status stays =:ok= with N-1 entries; the dropped sense logs to =*gloss-debug*=. A single bad sense doesn't poison the whole source. + +*Storage failures.* First call creates =gloss-file= and any missing parent directory with a =#+TITLE: Glossary= header. Permission denied raises =user-error= naming the path. Corrupt org file (=org-element-parse-buffer= raises) preserves the existing cache and surfaces "glossary file corrupt at line N; cache not refreshed" — operations fall back to the stale cache until the user fixes the file and runs =gloss-reload=. Term collision (saving an existing term) prompts: replace, append-with-separator, or cancel. + +*Drill.* =org-drill= checked via =featurep= before export runs. If absent: =user-error= with install hint. + +*User cancellations.* =C-g= during the picker → no save, side buffer shows the local-miss state. Empty term input from =gloss-add= → re-prompt once, then abort silently. Cancelled at the term-collision prompt → no write. + +** Testing + +Per-function test files; three categories (Normal/Boundary/Error) per function. TDD by default. Real production code via =require=, never inlined. + +*=gloss-core=.* Temp files + real =org-element-parse-buffer=. No mocking — exercises the actual file I/O and parser. + +#+begin_example +test-gloss-core--lookup.el +test-gloss-core--save.el +test-gloss-core--invalidate-on-mtime.el +test-gloss-core--corrupt-file-preserves-cache.el +test-gloss-core--alphabetical-insert.el +test-gloss-core--first-call-creates-file.el +#+end_example + +*=gloss-fetch=.* =cl-letf= mock on =url-retrieve-synchronously=, injecting canned response buffers. Captured Wiktionary fixtures in =tests/fixtures/wiktionary-*.json= — real responses for SBIR, anaphora, API, frozen once, replayed forever. + +#+begin_example +test-gloss-fetch--definitions-200-returns-ok.el +test-gloss-fetch--definitions-404-returns-no-defs.el +test-gloss-fetch--definitions-500-returns-server-error.el +test-gloss-fetch--definitions-timeout-returns-unreachable.el +test-gloss-fetch--strip-html.el +test-gloss-fetch--multi-source-walks-registry.el +test-gloss-fetch--libxml-probe.el +#+end_example + +*=gloss-display=.* The candidate-formatting helper =gloss-display--format-candidate PLIST → "[wiktionary] text..."= is pure → full N/B/E coverage. =gloss-display-show-entry= and =gloss-mode= get one smoke test each (Emacs already tests =switch-to-buffer= and major-mode definition). + +#+begin_example +test-gloss-display--format-candidate.el +test-gloss-display--show-entry-smoke.el +#+end_example + +*=gloss-drill=.* Temp file + real =org-element=. Tests assert tag/property changes on entries. + +#+begin_example +test-gloss-drill--export-all-tags-untagged.el +test-gloss-drill--export-all-skips-already-tagged.el +test-gloss-drill--export-all-no-orgdrill-installed.el +test-gloss-drill--untag-all.el +#+end_example + +*=gloss=.* The orchestration policy =gloss--orchestrate-fetch-result RESULT → SYMBOL= is a pure pattern-matcher. Tested with shaped inputs covering every result variant. + +#+begin_example +test-gloss--orchestrate-fetch-result.el +#+end_example + +*Integration tests.* Three small ones, each with a docstring naming participants per project convention. + +#+begin_example +test-integration-gloss-lookup-flow-local-hit.el +test-integration-gloss-lookup-flow-online-fall-through.el +test-integration-gloss-lookup-flow-online-failure.el +#+end_example + +*Coverage targets.* 90%+ on =gloss-core=, =gloss-fetch=, =gloss-drill=, and pure helpers in =gloss-display= / =gloss=. 70%+ on display mode-glue. Overall ≥80%. + +** Observability + +*=*gloss-debug*= log buffer.* Off until =gloss-debug= defcustom is non-nil, or session-only =gloss-toggle-debug= flips it. One timestamped, layer-prefixed line per significant event. + +#+begin_example +2026-04-28 11:14:02 [fetch:wiktionary] GET /API → 200, 12 senses +2026-04-28 11:14:02 [fetch:wiktionary] sense 7 HTML parse failed, dropping +2026-04-28 11:14:02 [core] cache hit for "anaphora" +2026-04-28 11:14:09 [core] mtime change detected, reloading cache (47 terms) +2026-04-28 11:14:11 [save] "API" → wiktionary, 11 alts not saved +#+end_example + +Per-source statuses from Error Handling land here verbatim. No personal data beyond user-supplied terms. + +*=*Messages*= for user-facing events.* Saves, picker-shown, "no definition found" messages — short single-line =message= calls, persisted in =*Messages*= via Emacs idiom. Strict separation: =*Messages*= for things the user did or asked for; =*gloss-debug*= for everything else. + +*Inspection commands.* + +- =gloss-list-terms= — completing-read over every term in the cache. Pick one to jump to it. +- =gloss-stats= — small buffer summarizing total terms, breakdown by =:source=, count of drill-tagged entries, file size, cache mtime. + +No metrics export, no telemetry, no profiling hooks — v3 territory if the package ever needs them. + +* Open Questions (will become ADRs) + +Each was decided during the brainstorm. Listed for traceability; each becomes an ADR in the gloss repo's =docs/decisions/=. + +- [ ] *ADR-1: storage path default* → =(expand-file-name "gloss.org" (or org-directory user-emacs-directory))=. Rationale: respects the user's existing =org-directory= convention; falls back gracefully. +- [ ] *ADR-2: auto-fetch on local miss* → silent fall-through with graceful network-failure path. Rationale: y/n prompt is yes 99% of the time and an annoyance the other 1%; the offline case is better handled by detecting the failure than by pre-asking permission. +- [ ] *ADR-3: drill direction* → =:DRILL_CARD_TYPE: twosided=. Rationale: tests both recognition and recall over time without doubling the deck. +- [ ] *ADR-4: HTML strip strategy* → =libxml-parse-html-region= (plain text only, no italic/bold preservation). Rationale: more robust than regex on edge cases; libxml2 is standard on Linux/Mac; ~30 lines. + +* Next Steps + +1. *Scaffold the repo.* =~/code/gloss= with the claude-template structure: =.ai/= and =todo.org= and =inbox/= gitignored, =Makefile= for tests/lint/compile, =README.org= placeholder, =LICENSE=, package skeleton (=gloss.el= with package-header autoload entry). +2. *Set up remotes.* Bare repo on cjennings.net at =/var/cjennings/git/gloss.git/= with the existing post-receive hook pattern that mirrors to =github.com/cjennings/gloss=. +3. *Decompose into todo.org tasks.* One TODO per layer, in implementation order: core → fetch → display → drill → entry-point → integration tests → README. Each task carries its acceptance criteria from this design. +4. *Implement v1 layer by layer*, TDD per project rules. Run =/start-work= once per task. +5. *First-week shakedown.* Use the package on real terms for a week. File issues against any rough edges as v1.1 tasks. +6. *Record the four ADRs* in =docs/decisions/= once the repo exists. + +* Status + +Draft. Pending: repo scaffold, ADR records, implementation. diff --git a/docs/specs/google-keep-emacs-integration-spec.org b/docs/specs/google-keep-emacs-integration-spec.org index 376522ab..96fd83e5 100644 --- a/docs/specs/google-keep-emacs-integration-spec.org +++ b/docs/specs/google-keep-emacs-integration-spec.org @@ -1,7 +1,14 @@ #+TITLE: Google Keep <-> Emacs integration — Spec #+AUTHOR: Craig Jennings & Claude #+DATE: 2026-06-24 -#+TODO: TODO | DONE SUPERSEDED CANCELLED +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Google Keep <-> Emacs integration — Spec +:PROPERTIES: +:ID: 4c796fb9-1d3e-42a9-9b76-eb286eee8732 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DOING from Metadata Status: v1 implemented, v2 next -> work ongoing * Metadata | Status | v1 implemented (Phases 1-3); live setup pending; v2 next | diff --git a/docs/specs/init-load-graph-spec-doing.org b/docs/specs/init-load-graph-spec-doing.org deleted file mode 100644 index 05dd9e0a..00000000 --- a/docs/specs/init-load-graph-spec-doing.org +++ /dev/null @@ -1,833 +0,0 @@ -:PROPERTIES: -:ID: e1fd137e-e164-42f4-a658-f4d32fbe3228 -:STATUS: doing -:END: -#+TITLE: Design: Untangle the init.el Load Graph -#+AUTHOR: Craig Jennings -#+DATE: 2026-05-04 - -* Status - -Draft. Specification only. No load-order implementation is part of this design -document. - -* Problem - -=init.el= is currently both the startup script and the dependency graph. It -eagerly requires almost every module in a fixed order, so many modules work -because some earlier require happened to define a variable, keymap, path -constant, hook owner, package, or helper function. - -That creates four practical problems: - -- Standalone module loading is unreliable. A module may byte-compile but fail at - runtime unless enough of =init.el= was loaded first. -- Startup has unnecessary work. Optional workflows, heavy packages, timers, - network-facing integrations, and media tools load even when not used. -- Side effects are hard to audit. Keybindings, timers, global hooks, server - setup, package configuration, and command definitions are mixed together. -- Test boundaries are blurry. Tests often need to simulate init order instead of - loading the unit under test directly. - -The target is not "lazy load everything." The target is an explicit, testable -load graph where eager startup is a small documented set, optional workflows -load from commands/hooks/autoloads, and module dependencies are declared by the -modules that use them. - -* Goals - -- Make module ownership obvious: libraries, keymap ownership, package - configuration, commands, and startup side effects should be distinguishable. -- Make dependencies explicit with ordinary =require=, =autoload=, or documented - hook/package boundaries. -- Reduce eager startup load without breaking existing keybindings or daily - workflows. -- Keep the migration incremental and reversible. Each batch should be small - enough to test and inspect. -- Preserve interactive behavior for configured workflows, including calendar - sync, Org capture/agenda, mail, F-keys, and media commands. -- Improve testability: modules should either load directly or fail with a clear - missing external package/config message. - -* Non-Goals - -- Rewriting the whole configuration into one framework or literate init. -- Removing =use-package=. This design assumes package config modules continue to - use it where appropriate. -- Eliminating all top-level forms. Some top-level configuration is appropriate, - especially for foundational Emacs settings and hook registration. -- Solving package bootstrap in =early-init.el=. That is tracked by the separate - "Move package bootstrap out of =early-init.el= where possible" project. -- Rotating calendar feed URLs or designing secret storage beyond the local - calendar config path already introduced. Token rotation remains a separate - security task. -- Consolidating all scattered utility helpers. Utility consolidation is a - sibling project because it changes helper ownership, tests, and call sites - without necessarily changing startup load order. - -* Principles - -** Eager Requires Are Allowed Only With A Reason - -An eager require in =init.el= should satisfy one of these conditions: - -- It establishes basic Emacs behavior needed for the rest of startup. -- It defines shared constants or helpers used by many eager modules. -- It owns the global key prefix/keymap registration system. -- It configures core UI behavior that should be visible in the first frame. -- It starts a user-approved startup service that cannot be triggered lazily. - -Everything else should be a candidate for autoload, hook-based loading, -=with-eval-after-load=, or a command wrapper. - -** Modules Declare What They Use - -If a module calls a function or reads a variable at runtime, it should not rely -on init order unless that dependency is an explicit startup contract. - -Preferred dependency forms: - -- Runtime dependency: =(require 'module)=. -- Optional runtime dependency: =(require 'module nil t)= with a clear degraded - behavior. -- Macro/compile-time dependency: =(eval-when-compile (require 'module))=. -- Command-only dependency: =(autoload 'command "module" nil t)= or a lazy - command wrapper. -- Package-bound dependency: =use-package :after=, =:hook=, =:commands=, or - =with-eval-after-load=. - -Avoid test-only shims in production modules such as "define this keymap if it -does not exist." Tests should provide stubs or load the real owner. - -** Utility Extraction Should Stay Small And Evidence-Based - -Some hidden dependencies exist because generic helpers live in feature modules -where they were first needed. Moving those helpers into =system-lib= can make -dependencies clearer, but utility extraction should not become part of every -load-order change by default. - -Extract a helper only when: - -- at least two callers need substantially the same behavior, -- the helper can stay dependency-light enough for foundation startup, -- tests can move with the helper, -- the extraction is atomic and easy to review. - -Avoid building a broad utility suite speculatively. Prefer one helper, one -tested extraction, one commit. - -** Keymaps Have Owners - -=keybindings.el= should own global prefixes, especially =cj/custom-keymap= and -the =C-;= prefix. Feature modules may define local maps or command maps, but -registration into global prefixes should go through a small convention/helper so -load order is not a hidden dependency. - -** Side Effects Are Named And Isolated - -Side effects include: - -- starting timers, -- starting processes, -- calling network-facing sync/fetch commands, -- setting global keybindings, -- mutating global hooks, -- opening files/buffers, -- enabling global modes, -- loading large packages solely for optional commands. - -Each side effect should have one of: - -- a documented eager reason, -- an interactive command, -- a hook/package boundary, -- a noninteractive/batch guard, -- a test that proves the side effect does not happen in the wrong context. - -* Target Architecture - -** Layer 0: Early Startup - -Owned by =early-init.el=. Should remain limited to startup mechanics that must -happen before package/UI initialization. - -Examples: - -- package archive/bootstrap policy, -- native-comp/cache startup knobs that must be early, -- disabling expensive default UI before first frame. - -This design does not refactor =early-init.el= except to avoid adding new load -graph responsibilities to it. - -** Layer 1: Foundation - -Small eager set required before most other modules can safely load. - -Expected contents: - -- =system-lib= -- =user-constants= -- =host-environment= -- =system-defaults= -- =keyboard-compat= -- =keybindings= -- maybe =config-utilities=, if debug helpers are intentionally eager - -Foundation modules should be able to load in batch mode without package, -network, timer, or UI-package side effects. - -Adding a new Layer 1 module requires a coordinated update to the -=system-lib.el= dependency budget in [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec-doing.org]]. - -Topic libraries introduced by the utility project join Layer 1 only when their -first consumer is foundation-eager. Otherwise they are Layer 2 and loaded by an -explicit =require= from their eager consumers. Add each new topic library to the -module category table before migrating its first consumer. - -** Layer 2: Core UX - -Eager or near-eager modules that shape the first interactive session. - -Expected contents: - -- basic text/editing defaults, -- core UI frame/theme/font/modeline behavior, -- selection/completion framework, -- F-key development entry points, -- VC/test/coverage command entry points. - -Core UX modules may configure packages, but heavy features should still use -=:commands=, =:hook=, or =:defer= where practical. - -** Layer 3: Domain Workflows - -Org, programming, mail, browser, media, AI, and integration modules. These -should generally load through hooks, commands, package =:after= clauses, or -workflow-specific entry commands. - -Examples: - -- Org capture/agenda can remain eager if the user's daily workflow needs it, - but exporters and optional extensions can be deferred. -- Language modules should load from mode hooks or file associations, not because - every startup might edit every language. -- Mail/media/AI/rest tools should register commands eagerly if needed, then load - heavy packages only on use. - -** Layer 4: Optional And Experimental - -Entertainment, modules in test, diagnostics, and rarely used tools. These should -not be required by default unless the user explicitly chooses that behavior. - -Examples: - -- =games-config= -- =music-config= -- =lorem-optimum= -- =gloss-config= -- optional IRC/Slack/feed/media modules when not in active use - -* Module Categories - -This is a first-pass classification to guide implementation. It is not an -architectural truth table; each module should be confirmed while refactoring. - -Category key: - -- =F= foundation or shared library/config. -- =C= core eager UX. -- =P= package configuration that should usually be hook/command/package loaded. -- =D= domain workflow that may have a user-visible eager reason. -- =S= startup side-effect or timer/process owner. -- =O= optional, entertainment, experimental, or rarely used. -- =L= pure-ish library/command helpers that should be easy to load directly. - -| Module | Category | Expected final load shape | Notes | -|--------+----------+---------------------------+-------| -| =early-init= | F | early | Layer 0; see Non-Goals. | -| =system-lib= | F/L | eager | Low-level helpers. Keep side-effect free. | -| =cj-process= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 3. | -| =cj-org-text= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 6. | -| =cj-cache= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 7. | -| =user-constants= | F | eager, then split | Split pure path constants from directory creation/failure behavior. | -| =host-environment= | F/L | eager | Predicate helpers. | -| =system-defaults= | F/S | eager | Owns global Emacs defaults, server/recentf/minibuffer hooks. | -| =keyboard-compat= | F/S | eager | Terminal/GUI keyboard setup hooks. | -| =keybindings= | F/C | eager | Owner of =cj/custom-keymap= and global prefixes. | -| =config-utilities= | C/O | eager or command-loaded | Debug keymap may be eager; heavy org parsing commands can lazy require. | -| =custom-case= | L/C | autoload commands + key registration | Text command helper. | -| =custom-comments= | L/C | autoload commands + key registration | Text command helper. | -| =custom-datetime= | L/C | autoload commands + key registration | Text command helper. | -| =custom-buffer-file= | L/C | eager only if remaps required | Has file/process helpers and keymap registration. | -| =custom-line-paragraph= | L/C | autoload commands + key registration | Requires =expand-region= at command boundary if possible. | -| =custom-misc= | L/C | autoload commands + key registration | Misc commands. | -| =custom-ordering= | L/C | autoload commands + key registration | Text command helper. | -| =custom-text-enclose= | L/C | autoload commands + key registration | Text command helper. | -| =custom-whitespace= | L/C | autoload commands + key registration | Text command helper. | -| =external-open= | L/D | autoload commands | Runtime requires environment/process helpers explicitly. | -| =media-utils= | D | command-loaded | Downloads/players should run only by command. | -| =auth-config= | F/D | eager or package-after | Auth setup may be core; GPG commands should remain commands. | -| =keyboard-macros= | C | eager or keymap-only | Lightweight command/key owner. | -| =system-utils= | L/C | eager or command-loaded | Timers/process monitor utilities. | -| =text-config= | C/P | eager hooks | General text defaults and package config. | -| =undead-buffers= | C | eager if remaps desired | Global kill-buffer remaps. | -| =browser-config= | D/P | command/package-loaded | Browser workflow. | -| =coverage-core= | C/L | eager command entry | F7 entry point and backend registry. | -| =coverage-elisp= | C/P | eager after core | Backend registration; keep cheap. | -| =dev-fkeys= | C | eager | F4/F6 command entry points. | -| =ui-config= | C/S | eager | Cursor/UI defaults; post-command hook should be documented. | -| =ui-theme= | C | eager + explicit startup call | Theme load stays explicit in init. | -| =ui-navigation= | C/P | eager | Window keybindings and winner/buffer-move config. | -| =font-config= | C/P/S | eager or first-frame | Font hooks/font installation checks need guards. | -| =selection-framework= | C/P | eager | Completion stack; likely core UX. | -| =modeline-config= | C/S | eager | Mode line and VC cache hooks. | -| =mousetrap-mode= | C | eager if global behavior desired | Prevents accidental mouse edits. | -| =popper-config= | C/P | eager if enabled, else remove/defer | Existing disabled-state question remains. | -| =chrono-tools= | D/P | command-loaded | Calendar/timer commands; sound path dependency explicit. | -| =diff-config= | C/P | eager or package-loaded | Diff/merge UX. | -| =erc-config= | O/D/P | command-loaded | IRC should not be startup load by default. | -| =slack-config= | O/D/P | command-loaded | Slack package/auth and which-key registration should be after-load. | -| =eshell + term-config= | D/P | command/hook-loaded | Shell/terminal packages. | -| =help-utils= | L/D | autoload commands | Search/help commands. | -| =help-config= | C/P | eager or after help | Info/man/help config. | -| =tramp-config= | D/P | package-loaded | Remote shell configuration. | -| =calibredb-epub-config= | O/D/P | command-loaded | Ebook workflow. | -| =dashboard-config= | C/S | eager only if startup dashboard desired | Opens/initializes landing page behavior. | -| =dirvish-config= | D/P | command/hook-loaded | File manager; runtime constants explicit. | -| =dwim-shell-config= | D/P | command-loaded | Shell commands from Dired/Dirvish. | -| =elfeed-config= | O/D/P | command-loaded | Feed reader/podcast workflow. | -| =eww-config= | D/P | command-loaded | Web browsing helpers. | -| =flyspell-and-abbrev= | C/P | hooks | Text-mode spelling/abbrev. | -| =httpd-config= | O/D/P | command-loaded | Local web server. | -| =latex-config= | D/P | hook-loaded | Existing WIP comment should become tasks or be removed. | -| =mail-config= | D/P | command-loaded or eager by choice | Heavy mu4e/org-msg; daily workflow may justify eager command registration. | -| =markdown-config= | D/P | mode-loaded | Markdown package config. | -| =pdf-config= | D/P | file/mode-loaded | Heavy PDF packages should load on PDF open. | -| =quick-video-capture= | O/D/S | command/protocol-loaded | Top-level timers should be removed or guarded. | -| =video-audio-recording= | O/D/S | command-loaded | External process/device probing only on command. | -| =transcription-config= | O/D/P | command-loaded | Auth/process workflow. | -| =weather-config= | O/D/P | command-loaded | Optional command. | -| =prog-general= | C/P/S | eager or hooks | Projectile, treesit policy, LSP ownership concerns. | -| =test-runner= | C/L | eager command entry | Test keymap and project-scoped state. | -| =vc-config= | C/P | eager command entry | Magit/git keymap; clone command hardening separate. | -| =flycheck-config= | C/P | hooks | General linting. | -| =prog-training= | O/D/P | command-loaded | Exercism/Leetcode optional. | -| =prog-c= | D/P | mode-loaded | C hooks and compile command. | -| =prog-go= | D/P | mode-loaded | Go hooks/LSP. | -| =prog-lisp= | D/P | mode-loaded | Lisp package config. | -| =prog-lsp= | C/P | package policy owner | Should consolidate generic LSP policy. | -| =prog-shell= | D/P/S | mode-loaded | after-save executable hook should be opt-in or scoped. | -| =prog-python= | D/P | mode-loaded | Python hooks/LSP. | -| =prog-webdev= | D/P | mode-loaded | Webdev modes/LSP. | -| =prog-json= | D/P | mode-loaded | JSON formatting/mode config. | -| =prog-yaml= | D/P | mode-loaded | YAML formatting/mode config. | -| =org-config= | C/D/P | eager | Core Org behavior likely eager. | -| =org-agenda-config= | D/S | eager by workflow, timers guarded | Agenda cache lifecycle project tracks cleanup. | -| =org-babel-config= | D/P | after Org | Babel languages package config. | -| =org-capture-config= | D/P | eager if capture hot path | Protocol/capture templates. | -| =org-contacts-config= | D/P | after Org/mail | Contacts workflow. | -| =org-drill-config= | O/D/P | command-loaded | Optional drill workflow. | -| =org-export-config= | D/P | command-loaded | Export packages/processes. | -| =hugo-config= | D/P | command-loaded | Blog workflow. | -| =org-reveal-config= | O/D/P | command-loaded | Presentation workflow. | -| =org-refile-config= | D/S | eager by workflow, timers guarded | Refile cache lifecycle project tracks cleanup. | -| =org-roam-config= | D/P/S | eager by workflow | Capture/finalize hooks, db. | -| =org-webclipper= | O/D/P | protocol/command-loaded | Global temp state cleanup tracked separately. | -| =org-noter-config= | O/D/P | command-loaded | PDF notes workflow. | -| =ai-config= | D/P | command-loaded | GPTel commands; avoid loading all AI tooling at startup. | -| =ai-conversations= | D/L/S | after gptel | Autosave hook and persistence path need coverage. | -| =restclient-config= | D/P | command-loaded | API exploration. | -| =calendar-sync= | D/S | eager only if configured, batch safe | Private config path and noninteractive guard exist. | -| =reconcile-open-repos= | D/S | command-loaded | Repo scanning/reconciliation should not run at startup. | -| =local-repository= | O/D/P | command-loaded | Local package mirror workflow. | -| =music-config= | O/D/P/S | command-loaded | EMMS/keymap optional, hooks only after EMMS. | -| =games-config= | O | command-loaded | Optional. | -| =lorem-optimum= | O/L | command-loaded | Module in test. | -| =jumper= | O/L | command-loaded | Navigation helper. | -| =system-commands= | D/S | command-loaded | High-impact commands; defensive work tracked separately. | -| =gloss-config= | O/D/P | command-loaded | Glossary workflow. | -| =wrap-up= | S | eager if desired | End-of-startup buffer bury timer. | -| =ledger-config= | O/D/P | mode-loaded | Not currently required by init. | -| =mu4e-org-contacts-integration= | D/L | after mu4e/org-contacts | Loaded by mail workflow. | -| =mu4e-org-contacts-setup= | D/L | after mu4e/org-contacts | Setup helper. | -| =org-agenda-config-debug= | O/L | command/debug-loaded | Debug helper. | -| =show-kill-ring= | O/L | command-loaded | Not currently required by init. | - -* Module File Header Standard - -Each module should eventually declare its load-graph contract in its own -commentary header. The category table above is the seed view; module headers -are the contributor-facing contract that travels with the code. - -Required header lines, after =;;; Commentary:=: - -1. =;; Layer: <0|1|2|3|4> ().= -2. =;; Category: =. -3. =;; Load shape: =. -4. =;; Eager reason:= one-line justification when load shape is =eager=, - omitted otherwise. -5. =;; Top-level side effects:= timer, process, hook, package, network, - buffer mutation, file write, or =none=. -6. =;; Runtime requires:= explicit runtime module/package list. -7. =;; Direct test load: =, with a brief reason when not - =yes=. - -Optional: - -- =;; See also:= references to tests and design docs. - -Worked example: - -#+begin_src emacs-lisp -;;; calendar-sync.el --- One-way calendar synchronization to Org -*- lexical-binding: t; -*- -;; -;;; Commentary: -;; -;; Layer: 3 (Domain Workflow). -;; Category: D/S. -;; Load shape: eager only when calendar-sync.local.el configures calendars. -;; Eager reason: daily-driver workflow; user expects calendars synced at first -;; session. Top-level startup is guarded so batch/test loads do not start -;; timers or network fetches. -;; Top-level side effects: timer, network fetch, file writes to calendar Org -;; files. Guarded by noninteractive/config checks. -;; Runtime requires: user-constants, seq, subr-x. -;; Direct test load: yes (batch-safe; private config is optional). -;; -;; See also: docs/specs/init-load-graph-spec-doing.org, tests/test-calendar-sync.el. -;; -;;; Code: -#+end_src - -Phase 1 should annotate every module required by =init.el= with this header. -Later validation can assert that every required module declares the seven -required lines. - -* Proposed Load Shape - -Migration commits should use conventional commit prefixes consistently: - -- =refactor:= for behavior-preserving load-order, dependency, keymap, and lazy - loading migrations. -- =feat:= only when adding a new user-visible capability. -- =test:= for test-only follow-up work. -- =docs:= for spec, inventory, design updates, and module-header annotations, - even when those annotations touch =modules/*.el= files. - -Default deferral mechanism: - -- Prefer =use-package :commands= for command-driven deferrals. -- Prefer =use-package :mode= when loading is file-extension or major-mode - driven. -- Prefer =use-package :hook= when the consumer is a mode-hook function. -- Use explicit =(autoload 'command "module" nil t)= only when the command is - not naturally owned by a =use-package= form. - -** Phase 1: Inventory And Contracts - -Do not change load order yet. - -1. Keep the current eager =init.el= order. -2. Create/maintain =docs/design/module-inventory.org= as a living inventory - with: - - module name, - - category, - - eager/deferred target, - - known runtime dependencies, - - top-level side effects, - - tests that cover standalone load or command behavior. -3. Annotate every module required by =init.el= with the module header standard. -4. Convert vague comments in =init.el= into tasks or remove them: - - =latex-config= "WIP need to fix", - - =prog-shell= "combine elsewhere", - - "Modules In Test" section. -5. Add lightweight standalone-load smoke tests for the lowest-level modules. - -Inventory rules: - -- The module table in this spec seeds the inventory. -- =docs/design/module-inventory.org= is the living per-module truth after Phase - 1 starts. -- Every module required by =init.el= must be represented before Phase 2 starts. -- Discoveries during later phases update the inventory. -- This inventory is independent from the helper inventory owned by - [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec-doing.org]]. - -Exit criteria: - -- Every module required by =init.el= has a category and target load shape. -- Every eager survivor has a documented reason. -- The inventory identifies top-level timers/process/network-ish side effects. -- Every module required by =init.el= has the required load-graph header lines. - -** Phase 2: Explicit Dependencies - -Still do not significantly change startup behavior. - -1. For each module batch, load it directly in batch mode. -2. Fix hidden dependencies by adding real =require=, =autoload=, or package - boundaries. -3. Remove production shims that only exist because tests load modules in an - incomplete environment. -4. If a keymap dependency is hidden, document it and make the dependency - explicit with =require= or =autoload=. Do not refactor into the registration - convention until Phase 3. When the hidden dependency is on - =cj/custom-keymap= itself, add =(require 'keybindings)= to the consuming - module; Phase 3 replaces these direct dependencies with the registration - API. -5. When a hidden dependency is really a duplicated generic helper, either: - - hand the extraction to the utility-consolidation sibling project when it - is in scope there, or - - leave it in place and record it under that project. - -Suggested order: - -- Foundation and libraries. -- Text/editing command modules. -- UI modules. -- Programming modules. -- Org modules. -- Optional integrations. - -Exit criteria: - -- Direct module load either succeeds or fails with a clear missing external - package/config message. -- =make test-file FILE=test-all-comp-errors.el= passes. -- New tests cover any helper extracted while fixing dependencies. -- Helper extraction remains dependency-light and does not pull heavy packages - into foundation startup. - -** Phase 3: Keymap Registration Boundary - -Introduce a small keymap registration API before deferring many feature modules. - -Possible API: - -#+begin_src emacs-lisp -(defun cj/register-prefix-map (key map label) - "Register MAP under KEY in `cj/custom-keymap' with LABEL for which-key." - ...) - -(defun cj/register-command (key command label) - "Register COMMAND under KEY in `cj/custom-keymap' with LABEL for which-key." - ...) -#+end_src - -Design rules: - -- =keybindings.el= owns =cj/custom-keymap= and the global =C-;= binding. -- Feature modules may define maps and commands without mutating global keys - directly. -- Which-key labels must be registered after which-key loads. -- Tests can assert key resolution without loading every feature package. - -Exit criteria: - -- Modules no longer need to assume =cj/custom-keymap= exists at top level - except through the registration API. -- Existing =C-;= bindings continue to resolve. -- Which-key labels for documented prefixes remain available. - -** Phase 4: Defer Low-Risk Optional Modules - -Start with modules that are unlikely to affect first-frame startup. - -Candidate batch: - -- =games-config= -- =music-config= -- =weather-config= -- =gloss-config= -- =lorem-optimum= -- =jumper= -- =httpd-config= -- =prog-training= - -For each module: - -1. Keep its user-facing command/key available via the default deferral mechanism - above. -2. Move package loading into =use-package :commands=, =:hook=, =:mode=, or an - explicit autoload/wrapper only when the default does not fit. -3. Run targeted tests and an interactive smoke check. - -Exit criteria: - -- Startup no longer requires the module eagerly. -- User command still works from a fresh Emacs session. -- Module-specific tests pass. - -** Phase 5: Defer Heavy Domain Modules - -Candidate batch: - -- =pdf-config= -- =calibredb-epub-config= -- =video-audio-recording= -- =transcription-config= -- =mail-config= -- =ai-config= -- =restclient-config= -- =elfeed-config= -- =erc-config= -- =slack-config= - -These need more care because they often combine package setup, auth, keymaps, -processes, hooks, and user workflows. - -Exit criteria for each: - -- Commands are discoverable before package load. -- Package load happens through the default deferral mechanism: command, hook, - mode, or explicit startup opt-in. -- Auth and private config are not read until necessary unless the user opts in. -- Batch/test startup does not start network/process work. - -Private config opt-in follows the =calendar-sync.local.el= precedent: a module -reads =.local.el= when readable, the file is gitignored, and the -module degrades cleanly when the file is missing. Token rotation is a separate -security task; this convention is about config presence, not secret protection. - -** Phase 6: Revisit Org And Programming Eagerness - -Org and programming modules are daily-use, so the goal is not blindly deferring -everything. - -Programming target: - -- Keep generic programming defaults and F-key command entry points available. -- Load language-specific modules by major mode. -- Consolidate generic LSP policy under =prog-lsp=. - - Move to =prog-lsp=: global LSP toggles such as =lsp-idle-delay=, - =lsp-log-io=, =lsp-enable-folding=, =lsp-enable-snippet=, - =lsp-headerline-breadcrumb-enable=, and file-watch ignore lists. - - Keep per-language: server client settings such as - =lsp-clients-clangd-args= and =lsp-pyright-*=, plus language-mode hook - wiring. -- Tree-sitter grammar auto-install is always on; the project policy is global - allow. =treesit-auto-install= is =t= without per-language conditionals. - -Org target: - -- Keep these daily first-session workflows eager: =org-config=, - =org-agenda-config=, =org-capture-config=, =org-refile-config=, - =calendar-sync= when local config is present, and =org-roam-config=. -- Defer exporters, reveal, drill, noter, webclipper, and optional publishing - pieces behind commands/hooks. -- Normalize agenda/refile cache lifecycle before changing timer behavior. This - is behavioral normalization within the load-graph project; the shared - =cj-cache.el= extraction is owned by utility-consolidation Phase 5 and may - follow. - -The =prog-lsp= consolidation and tree-sitter policy decisions are owned by this -load-graph project. Utility consolidation owns reusable helper extraction, not -programming policy. - -Exit criteria: - -- Common daily Org/programming workflows work from a fresh session. -- Optional exporters/languages load when used. -- Timers are guarded in batch/test contexts. - -* Adjacent Project: Utility Consolidation - -The review of this spec identified a related but distinct architectural -problem: helper functions are scattered across feature modules, sometimes with -duplicated behavior. This matters to the load graph because modules can become -coupled to whichever feature file happened to define a useful helper first. - -This should be tracked as a sibling project, not folded into the load-graph -project. The load-graph project asks "when and why does this module load?" The -utility consolidation project asks "which module should own this reusable -behavior?" Those questions overlap, but their changes have different risk and -rollback shapes. - -This sibling project can run beside Phase 2. When explicit-dependency work finds -a generic duplicated helper, the sibling project owns the extraction commit when -the helper is in scope for that project. See -[[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec-doing.org]] for candidate -helpers, naming rules, dependency budgets, migration phases, and test policy. - -* Testing Strategy - -** Static/Batch Tests - -Add or extend tests for: - -- Direct module load smoke tests for modules in each batch. -- Header validation: every module required by =init.el= declares the seven - required load-graph header lines. - - Test file: =tests/test-init-module-headers.el=. - - Assertion shape: inspect every module required by =init.el=, read its - commentary header, and fail with the missing line names for any absent - required header line. -- Keymap registration: prefix maps and commands resolve without requiring the - feature implementation package. -- No startup timers/processes in batch for side-effect modules. -- =init.el= startup smoke in batch, where possible. -- Byte/native compile smoke via existing =test-all-comp-errors.el=. - -Test files for this project use =test-init-.el=, for example -=test-init-module-headers.el= and =test-init-keymap-registration.el=. This keeps -load-graph validation tests distinct from per-module unit tests. - -Header validation runs directly against module files. It does not depend on the -final =docs/design/module-inventory.org= format, which remains a Phase 1 -authoring decision. - -** Automated Smoke Checks - -Automate every smoke item that can run in batch: - -- Important keybindings resolve to the intended command symbols, including - =C-;= prefixes and F4/F6/F7 entry points. -- Org capture and agenda command entry points load or produce expected - batch-safe guidance. -- Calendar sync status reports configured/no-config state without starting - timers or network fetches in batch. -- Optional commands touched in the batch autoload and resolve. -- Non-graphical interactive flows use =execute-kbd-macro= or - =with-simulated-input= where practical. - -These checks should run under =make test= for every migration commit. - -** Manual Smoke Checks - -Each migration batch should be followed by an interactive restart and checklist: - -- First frame appears with expected theme/font/modeline. -- =C-;= prefix appears and key descriptions are present. -- Magit opens. -- Mail command opens or gives expected package/config guidance. -- Refile target lookup works in an interactive session. -- Any optional command changed in the batch runs end to end. -- If daemon mode is part of normal use, run the visual checklist once via - regular =emacs= and once via =emacsclient= against a running daemon. - -** Performance Checks - -Before and after major batches: - -- Record =emacs-init-time=. -- Record a startup profile baseline and diff, preferably with =benchmark-init= - if enabled for the phase. -- =benchmark-init= is installed via package.el. The activation block in - =early-init.el= is commented; uncomment it locally during phases that need - profiling and do not commit the activation. Profile output goes to - =.profile/=, which should stay gitignored. -- Suggested workflow: - - =make profile-baseline= records =emacs-init-time= and a startup profile to - =.profile/baseline.txt=. - - =make profile-diff= records the current run and compares it to the phase - baseline. -- Keep a simple note of eagerly loaded feature count from - =cj/info-loaded-features= or equivalent. - -Performance is a supporting signal. Correctness and explicit dependencies are -the primary acceptance criteria. Startup regressions larger than roughly 50 ms -against the phase baseline should be investigated and explained; after several -stable baseline runs, this can become a stricter gate. - -* Acceptance Criteria - -The project is complete when: - -- =init.el= contains only documented eager requires and explicit startup calls. -- Optional modules no longer load merely because Emacs started. -- Each module required by =init.el= has a category and eager/deferred rationale. -- Modules that remain eager have no hidden dependencies on arbitrary earlier - init order. -- Global key registration has a central owner/convention. -- Top-level timers/process/network work is either removed, guarded, or - documented as intentional. -- Full =make test= passes. -- Byte/native compile smoke passes. -- Interactive startup checklist passes. - -* Risks And Mitigations - -** Risk: Breaking muscle-memory keybindings - -Mitigation: - -- Change key registration mechanics before changing bindings. -- Add keymap resolution tests for important prefixes. -- Keep a per-batch manual keybinding checklist. - -** Risk: Lazy-loaded packages miss early hook setup - -Mitigation: - -- Prefer =use-package :hook= and =:mode= over ad hoc lazy command bodies for mode - packages. -- Add tests that inspect hook contents where possible. -- Smoke-test opening representative files. - -** Risk: Daily workflows silently stop starting - -Mitigation: - -- Distinguish "safe default" from "local opt-in" for workflows like calendar - sync. -- Use ignored/local config files for private eager opt-ins. -- Report missing config clearly. - -** Risk: Batch tests differ from interactive startup - -Mitigation: - -- Guard timers/process/network work with =noninteractive= only when that is the - intended distinction. -- Add at least one interactive checklist per migration batch. - -** Risk: Refactor becomes too broad - -Mitigation: - -- One batch, one module family. -- Do not mix dependency fixes, keybinding redesign, and package lazy-loading in - the same commit unless tightly coupled. -- Keep rollback easy by preserving user-facing commands and using wrappers. - -* Implementation Backlog - -The project in =todo.org= should remain the source of task state. This design -supports these implementation tickets: - -1. Classify modules by role and startup requirement. -2. Add explicit module dependencies before changing load order. -3. Centralize custom keymap registration. -4. Defer low-risk optional modules. -5. Defer heavy document/media/integration modules. -6. Revisit programming module LSP/tree-sitter ownership. -7. Revisit Org module cache/timer and optional extension loading. -8. Retire or rewrite stale =init.el= comments. -9. Create a sibling utility consolidation project with an inventory pass and - first helper extractions. - -* Open Questions - -- Should =config-utilities= remain eager because debug commands are useful - during startup work, or should it become command-loaded after this project? -- Should local/private opt-ins share one file, or should modules keep - workflow-specific local files such as =calendar-sync.local.el=? -- Should the module inventory become machine-readable for validation, or is an - org table enough? Decide during Phase 1 based on inventory authoring - experience. -- Should =init.el= ultimately become declarative sections plus an explicit - startup contract list? - -* Next Steps - -1. Use this document as the reference for the =Classify modules by role and - startup requirement= task. -2. Build the first inventory directly from the module table above, correcting - category guesses while inspecting each file. -3. Do not defer a module until its direct runtime dependencies are explicit. -4. Implement keymap registration before deferring feature modules that currently - mutate =cj/custom-keymap= at top level. -5. Create the sibling utility consolidation project before Phase 2 work begins, - so duplicated helpers found during dependency cleanup have a clear place to - land. diff --git a/docs/specs/init-load-graph-spec.org b/docs/specs/init-load-graph-spec.org new file mode 100644 index 00000000..0feebfc9 --- /dev/null +++ b/docs/specs/init-load-graph-spec.org @@ -0,0 +1,837 @@ +#+TITLE: Design: Untangle the init.el Load Graph +#+AUTHOR: Craig Jennings +#+DATE: 2026-05-04 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Design: Untangle the init.el Load Graph +:PROPERTIES: +:ID: e1fd137e-e164-42f4-a658-f4d32fbe3228 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DOING from existing :STATUS: doing + +* Status + +Draft. Specification only. No load-order implementation is part of this design +document. + +* Problem + +=init.el= is currently both the startup script and the dependency graph. It +eagerly requires almost every module in a fixed order, so many modules work +because some earlier require happened to define a variable, keymap, path +constant, hook owner, package, or helper function. + +That creates four practical problems: + +- Standalone module loading is unreliable. A module may byte-compile but fail at + runtime unless enough of =init.el= was loaded first. +- Startup has unnecessary work. Optional workflows, heavy packages, timers, + network-facing integrations, and media tools load even when not used. +- Side effects are hard to audit. Keybindings, timers, global hooks, server + setup, package configuration, and command definitions are mixed together. +- Test boundaries are blurry. Tests often need to simulate init order instead of + loading the unit under test directly. + +The target is not "lazy load everything." The target is an explicit, testable +load graph where eager startup is a small documented set, optional workflows +load from commands/hooks/autoloads, and module dependencies are declared by the +modules that use them. + +* Goals + +- Make module ownership obvious: libraries, keymap ownership, package + configuration, commands, and startup side effects should be distinguishable. +- Make dependencies explicit with ordinary =require=, =autoload=, or documented + hook/package boundaries. +- Reduce eager startup load without breaking existing keybindings or daily + workflows. +- Keep the migration incremental and reversible. Each batch should be small + enough to test and inspect. +- Preserve interactive behavior for configured workflows, including calendar + sync, Org capture/agenda, mail, F-keys, and media commands. +- Improve testability: modules should either load directly or fail with a clear + missing external package/config message. + +* Non-Goals + +- Rewriting the whole configuration into one framework or literate init. +- Removing =use-package=. This design assumes package config modules continue to + use it where appropriate. +- Eliminating all top-level forms. Some top-level configuration is appropriate, + especially for foundational Emacs settings and hook registration. +- Solving package bootstrap in =early-init.el=. That is tracked by the separate + "Move package bootstrap out of =early-init.el= where possible" project. +- Rotating calendar feed URLs or designing secret storage beyond the local + calendar config path already introduced. Token rotation remains a separate + security task. +- Consolidating all scattered utility helpers. Utility consolidation is a + sibling project because it changes helper ownership, tests, and call sites + without necessarily changing startup load order. + +* Principles + +** Eager Requires Are Allowed Only With A Reason + +An eager require in =init.el= should satisfy one of these conditions: + +- It establishes basic Emacs behavior needed for the rest of startup. +- It defines shared constants or helpers used by many eager modules. +- It owns the global key prefix/keymap registration system. +- It configures core UI behavior that should be visible in the first frame. +- It starts a user-approved startup service that cannot be triggered lazily. + +Everything else should be a candidate for autoload, hook-based loading, +=with-eval-after-load=, or a command wrapper. + +** Modules Declare What They Use + +If a module calls a function or reads a variable at runtime, it should not rely +on init order unless that dependency is an explicit startup contract. + +Preferred dependency forms: + +- Runtime dependency: =(require 'module)=. +- Optional runtime dependency: =(require 'module nil t)= with a clear degraded + behavior. +- Macro/compile-time dependency: =(eval-when-compile (require 'module))=. +- Command-only dependency: =(autoload 'command "module" nil t)= or a lazy + command wrapper. +- Package-bound dependency: =use-package :after=, =:hook=, =:commands=, or + =with-eval-after-load=. + +Avoid test-only shims in production modules such as "define this keymap if it +does not exist." Tests should provide stubs or load the real owner. + +** Utility Extraction Should Stay Small And Evidence-Based + +Some hidden dependencies exist because generic helpers live in feature modules +where they were first needed. Moving those helpers into =system-lib= can make +dependencies clearer, but utility extraction should not become part of every +load-order change by default. + +Extract a helper only when: + +- at least two callers need substantially the same behavior, +- the helper can stay dependency-light enough for foundation startup, +- tests can move with the helper, +- the extraction is atomic and easy to review. + +Avoid building a broad utility suite speculatively. Prefer one helper, one +tested extraction, one commit. + +** Keymaps Have Owners + +=keybindings.el= should own global prefixes, especially =cj/custom-keymap= and +the =C-;= prefix. Feature modules may define local maps or command maps, but +registration into global prefixes should go through a small convention/helper so +load order is not a hidden dependency. + +** Side Effects Are Named And Isolated + +Side effects include: + +- starting timers, +- starting processes, +- calling network-facing sync/fetch commands, +- setting global keybindings, +- mutating global hooks, +- opening files/buffers, +- enabling global modes, +- loading large packages solely for optional commands. + +Each side effect should have one of: + +- a documented eager reason, +- an interactive command, +- a hook/package boundary, +- a noninteractive/batch guard, +- a test that proves the side effect does not happen in the wrong context. + +* Target Architecture + +** Layer 0: Early Startup + +Owned by =early-init.el=. Should remain limited to startup mechanics that must +happen before package/UI initialization. + +Examples: + +- package archive/bootstrap policy, +- native-comp/cache startup knobs that must be early, +- disabling expensive default UI before first frame. + +This design does not refactor =early-init.el= except to avoid adding new load +graph responsibilities to it. + +** Layer 1: Foundation + +Small eager set required before most other modules can safely load. + +Expected contents: + +- =system-lib= +- =user-constants= +- =host-environment= +- =system-defaults= +- =keyboard-compat= +- =keybindings= +- maybe =config-utilities=, if debug helpers are intentionally eager + +Foundation modules should be able to load in batch mode without package, +network, timer, or UI-package side effects. + +Adding a new Layer 1 module requires a coordinated update to the +=system-lib.el= dependency budget in [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec.org]]. + +Topic libraries introduced by the utility project join Layer 1 only when their +first consumer is foundation-eager. Otherwise they are Layer 2 and loaded by an +explicit =require= from their eager consumers. Add each new topic library to the +module category table before migrating its first consumer. + +** Layer 2: Core UX + +Eager or near-eager modules that shape the first interactive session. + +Expected contents: + +- basic text/editing defaults, +- core UI frame/theme/font/modeline behavior, +- selection/completion framework, +- F-key development entry points, +- VC/test/coverage command entry points. + +Core UX modules may configure packages, but heavy features should still use +=:commands=, =:hook=, or =:defer= where practical. + +** Layer 3: Domain Workflows + +Org, programming, mail, browser, media, AI, and integration modules. These +should generally load through hooks, commands, package =:after= clauses, or +workflow-specific entry commands. + +Examples: + +- Org capture/agenda can remain eager if the user's daily workflow needs it, + but exporters and optional extensions can be deferred. +- Language modules should load from mode hooks or file associations, not because + every startup might edit every language. +- Mail/media/AI/rest tools should register commands eagerly if needed, then load + heavy packages only on use. + +** Layer 4: Optional And Experimental + +Entertainment, modules in test, diagnostics, and rarely used tools. These should +not be required by default unless the user explicitly chooses that behavior. + +Examples: + +- =games-config= +- =music-config= +- =lorem-optimum= +- =gloss-config= +- optional IRC/Slack/feed/media modules when not in active use + +* Module Categories + +This is a first-pass classification to guide implementation. It is not an +architectural truth table; each module should be confirmed while refactoring. + +Category key: + +- =F= foundation or shared library/config. +- =C= core eager UX. +- =P= package configuration that should usually be hook/command/package loaded. +- =D= domain workflow that may have a user-visible eager reason. +- =S= startup side-effect or timer/process owner. +- =O= optional, entertainment, experimental, or rarely used. +- =L= pure-ish library/command helpers that should be easy to load directly. + +| Module | Category | Expected final load shape | Notes | +|--------+----------+---------------------------+-------| +| =early-init= | F | early | Layer 0; see Non-Goals. | +| =system-lib= | F/L | eager | Low-level helpers. Keep side-effect free. | +| =cj-process= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 3. | +| =cj-org-text= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 6. | +| =cj-cache= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 7. | +| =user-constants= | F | eager, then split | Split pure path constants from directory creation/failure behavior. | +| =host-environment= | F/L | eager | Predicate helpers. | +| =system-defaults= | F/S | eager | Owns global Emacs defaults, server/recentf/minibuffer hooks. | +| =keyboard-compat= | F/S | eager | Terminal/GUI keyboard setup hooks. | +| =keybindings= | F/C | eager | Owner of =cj/custom-keymap= and global prefixes. | +| =config-utilities= | C/O | eager or command-loaded | Debug keymap may be eager; heavy org parsing commands can lazy require. | +| =custom-case= | L/C | autoload commands + key registration | Text command helper. | +| =custom-comments= | L/C | autoload commands + key registration | Text command helper. | +| =custom-datetime= | L/C | autoload commands + key registration | Text command helper. | +| =custom-buffer-file= | L/C | eager only if remaps required | Has file/process helpers and keymap registration. | +| =custom-line-paragraph= | L/C | autoload commands + key registration | Requires =expand-region= at command boundary if possible. | +| =custom-misc= | L/C | autoload commands + key registration | Misc commands. | +| =custom-ordering= | L/C | autoload commands + key registration | Text command helper. | +| =custom-text-enclose= | L/C | autoload commands + key registration | Text command helper. | +| =custom-whitespace= | L/C | autoload commands + key registration | Text command helper. | +| =external-open= | L/D | autoload commands | Runtime requires environment/process helpers explicitly. | +| =media-utils= | D | command-loaded | Downloads/players should run only by command. | +| =auth-config= | F/D | eager or package-after | Auth setup may be core; GPG commands should remain commands. | +| =keyboard-macros= | C | eager or keymap-only | Lightweight command/key owner. | +| =system-utils= | L/C | eager or command-loaded | Timers/process monitor utilities. | +| =text-config= | C/P | eager hooks | General text defaults and package config. | +| =undead-buffers= | C | eager if remaps desired | Global kill-buffer remaps. | +| =browser-config= | D/P | command/package-loaded | Browser workflow. | +| =coverage-core= | C/L | eager command entry | F7 entry point and backend registry. | +| =coverage-elisp= | C/P | eager after core | Backend registration; keep cheap. | +| =dev-fkeys= | C | eager | F4/F6 command entry points. | +| =ui-config= | C/S | eager | Cursor/UI defaults; post-command hook should be documented. | +| =ui-theme= | C | eager + explicit startup call | Theme load stays explicit in init. | +| =ui-navigation= | C/P | eager | Window keybindings and winner/buffer-move config. | +| =font-config= | C/P/S | eager or first-frame | Font hooks/font installation checks need guards. | +| =selection-framework= | C/P | eager | Completion stack; likely core UX. | +| =modeline-config= | C/S | eager | Mode line and VC cache hooks. | +| =mousetrap-mode= | C | eager if global behavior desired | Prevents accidental mouse edits. | +| =popper-config= | C/P | eager if enabled, else remove/defer | Existing disabled-state question remains. | +| =chrono-tools= | D/P | command-loaded | Calendar/timer commands; sound path dependency explicit. | +| =diff-config= | C/P | eager or package-loaded | Diff/merge UX. | +| =erc-config= | O/D/P | command-loaded | IRC should not be startup load by default. | +| =slack-config= | O/D/P | command-loaded | Slack package/auth and which-key registration should be after-load. | +| =eshell + term-config= | D/P | command/hook-loaded | Shell/terminal packages. | +| =help-utils= | L/D | autoload commands | Search/help commands. | +| =help-config= | C/P | eager or after help | Info/man/help config. | +| =tramp-config= | D/P | package-loaded | Remote shell configuration. | +| =calibredb-epub-config= | O/D/P | command-loaded | Ebook workflow. | +| =dashboard-config= | C/S | eager only if startup dashboard desired | Opens/initializes landing page behavior. | +| =dirvish-config= | D/P | command/hook-loaded | File manager; runtime constants explicit. | +| =dwim-shell-config= | D/P | command-loaded | Shell commands from Dired/Dirvish. | +| =elfeed-config= | O/D/P | command-loaded | Feed reader/podcast workflow. | +| =eww-config= | D/P | command-loaded | Web browsing helpers. | +| =flyspell-and-abbrev= | C/P | hooks | Text-mode spelling/abbrev. | +| =httpd-config= | O/D/P | command-loaded | Local web server. | +| =latex-config= | D/P | hook-loaded | Existing WIP comment should become tasks or be removed. | +| =mail-config= | D/P | command-loaded or eager by choice | Heavy mu4e/org-msg; daily workflow may justify eager command registration. | +| =markdown-config= | D/P | mode-loaded | Markdown package config. | +| =pdf-config= | D/P | file/mode-loaded | Heavy PDF packages should load on PDF open. | +| =quick-video-capture= | O/D/S | command/protocol-loaded | Top-level timers should be removed or guarded. | +| =video-audio-recording= | O/D/S | command-loaded | External process/device probing only on command. | +| =transcription-config= | O/D/P | command-loaded | Auth/process workflow. | +| =weather-config= | O/D/P | command-loaded | Optional command. | +| =prog-general= | C/P/S | eager or hooks | Projectile, treesit policy, LSP ownership concerns. | +| =test-runner= | C/L | eager command entry | Test keymap and project-scoped state. | +| =vc-config= | C/P | eager command entry | Magit/git keymap; clone command hardening separate. | +| =flycheck-config= | C/P | hooks | General linting. | +| =prog-training= | O/D/P | command-loaded | Exercism/Leetcode optional. | +| =prog-c= | D/P | mode-loaded | C hooks and compile command. | +| =prog-go= | D/P | mode-loaded | Go hooks/LSP. | +| =prog-lisp= | D/P | mode-loaded | Lisp package config. | +| =prog-lsp= | C/P | package policy owner | Should consolidate generic LSP policy. | +| =prog-shell= | D/P/S | mode-loaded | after-save executable hook should be opt-in or scoped. | +| =prog-python= | D/P | mode-loaded | Python hooks/LSP. | +| =prog-webdev= | D/P | mode-loaded | Webdev modes/LSP. | +| =prog-json= | D/P | mode-loaded | JSON formatting/mode config. | +| =prog-yaml= | D/P | mode-loaded | YAML formatting/mode config. | +| =org-config= | C/D/P | eager | Core Org behavior likely eager. | +| =org-agenda-config= | D/S | eager by workflow, timers guarded | Agenda cache lifecycle project tracks cleanup. | +| =org-babel-config= | D/P | after Org | Babel languages package config. | +| =org-capture-config= | D/P | eager if capture hot path | Protocol/capture templates. | +| =org-contacts-config= | D/P | after Org/mail | Contacts workflow. | +| =org-drill-config= | O/D/P | command-loaded | Optional drill workflow. | +| =org-export-config= | D/P | command-loaded | Export packages/processes. | +| =hugo-config= | D/P | command-loaded | Blog workflow. | +| =org-reveal-config= | O/D/P | command-loaded | Presentation workflow. | +| =org-refile-config= | D/S | eager by workflow, timers guarded | Refile cache lifecycle project tracks cleanup. | +| =org-roam-config= | D/P/S | eager by workflow | Capture/finalize hooks, db. | +| =org-webclipper= | O/D/P | protocol/command-loaded | Global temp state cleanup tracked separately. | +| =org-noter-config= | O/D/P | command-loaded | PDF notes workflow. | +| =ai-config= | D/P | command-loaded | GPTel commands; avoid loading all AI tooling at startup. | +| =ai-conversations= | D/L/S | after gptel | Autosave hook and persistence path need coverage. | +| =restclient-config= | D/P | command-loaded | API exploration. | +| =calendar-sync= | D/S | eager only if configured, batch safe | Private config path and noninteractive guard exist. | +| =reconcile-open-repos= | D/S | command-loaded | Repo scanning/reconciliation should not run at startup. | +| =local-repository= | O/D/P | command-loaded | Local package mirror workflow. | +| =music-config= | O/D/P/S | command-loaded | EMMS/keymap optional, hooks only after EMMS. | +| =games-config= | O | command-loaded | Optional. | +| =lorem-optimum= | O/L | command-loaded | Module in test. | +| =jumper= | O/L | command-loaded | Navigation helper. | +| =system-commands= | D/S | command-loaded | High-impact commands; defensive work tracked separately. | +| =gloss-config= | O/D/P | command-loaded | Glossary workflow. | +| =wrap-up= | S | eager if desired | End-of-startup buffer bury timer. | +| =ledger-config= | O/D/P | mode-loaded | Not currently required by init. | +| =mu4e-org-contacts-integration= | D/L | after mu4e/org-contacts | Loaded by mail workflow. | +| =mu4e-org-contacts-setup= | D/L | after mu4e/org-contacts | Setup helper. | +| =org-agenda-config-debug= | O/L | command/debug-loaded | Debug helper. | +| =show-kill-ring= | O/L | command-loaded | Not currently required by init. | + +* Module File Header Standard + +Each module should eventually declare its load-graph contract in its own +commentary header. The category table above is the seed view; module headers +are the contributor-facing contract that travels with the code. + +Required header lines, after =;;; Commentary:=: + +1. =;; Layer: <0|1|2|3|4> ().= +2. =;; Category: =. +3. =;; Load shape: =. +4. =;; Eager reason:= one-line justification when load shape is =eager=, + omitted otherwise. +5. =;; Top-level side effects:= timer, process, hook, package, network, + buffer mutation, file write, or =none=. +6. =;; Runtime requires:= explicit runtime module/package list. +7. =;; Direct test load: =, with a brief reason when not + =yes=. + +Optional: + +- =;; See also:= references to tests and design docs. + +Worked example: + +#+begin_src emacs-lisp +;;; calendar-sync.el --- One-way calendar synchronization to Org -*- lexical-binding: t; -*- +;; +;;; Commentary: +;; +;; Layer: 3 (Domain Workflow). +;; Category: D/S. +;; Load shape: eager only when calendar-sync.local.el configures calendars. +;; Eager reason: daily-driver workflow; user expects calendars synced at first +;; session. Top-level startup is guarded so batch/test loads do not start +;; timers or network fetches. +;; Top-level side effects: timer, network fetch, file writes to calendar Org +;; files. Guarded by noninteractive/config checks. +;; Runtime requires: user-constants, seq, subr-x. +;; Direct test load: yes (batch-safe; private config is optional). +;; +;; See also: docs/specs/init-load-graph-spec.org, tests/test-calendar-sync.el. +;; +;;; Code: +#+end_src + +Phase 1 should annotate every module required by =init.el= with this header. +Later validation can assert that every required module declares the seven +required lines. + +* Proposed Load Shape + +Migration commits should use conventional commit prefixes consistently: + +- =refactor:= for behavior-preserving load-order, dependency, keymap, and lazy + loading migrations. +- =feat:= only when adding a new user-visible capability. +- =test:= for test-only follow-up work. +- =docs:= for spec, inventory, design updates, and module-header annotations, + even when those annotations touch =modules/*.el= files. + +Default deferral mechanism: + +- Prefer =use-package :commands= for command-driven deferrals. +- Prefer =use-package :mode= when loading is file-extension or major-mode + driven. +- Prefer =use-package :hook= when the consumer is a mode-hook function. +- Use explicit =(autoload 'command "module" nil t)= only when the command is + not naturally owned by a =use-package= form. + +** Phase 1: Inventory And Contracts + +Do not change load order yet. + +1. Keep the current eager =init.el= order. +2. Create/maintain =docs/design/module-inventory.org= as a living inventory + with: + - module name, + - category, + - eager/deferred target, + - known runtime dependencies, + - top-level side effects, + - tests that cover standalone load or command behavior. +3. Annotate every module required by =init.el= with the module header standard. +4. Convert vague comments in =init.el= into tasks or remove them: + - =latex-config= "WIP need to fix", + - =prog-shell= "combine elsewhere", + - "Modules In Test" section. +5. Add lightweight standalone-load smoke tests for the lowest-level modules. + +Inventory rules: + +- The module table in this spec seeds the inventory. +- =docs/design/module-inventory.org= is the living per-module truth after Phase + 1 starts. +- Every module required by =init.el= must be represented before Phase 2 starts. +- Discoveries during later phases update the inventory. +- This inventory is independent from the helper inventory owned by + [[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec.org]]. + +Exit criteria: + +- Every module required by =init.el= has a category and target load shape. +- Every eager survivor has a documented reason. +- The inventory identifies top-level timers/process/network-ish side effects. +- Every module required by =init.el= has the required load-graph header lines. + +** Phase 2: Explicit Dependencies + +Still do not significantly change startup behavior. + +1. For each module batch, load it directly in batch mode. +2. Fix hidden dependencies by adding real =require=, =autoload=, or package + boundaries. +3. Remove production shims that only exist because tests load modules in an + incomplete environment. +4. If a keymap dependency is hidden, document it and make the dependency + explicit with =require= or =autoload=. Do not refactor into the registration + convention until Phase 3. When the hidden dependency is on + =cj/custom-keymap= itself, add =(require 'keybindings)= to the consuming + module; Phase 3 replaces these direct dependencies with the registration + API. +5. When a hidden dependency is really a duplicated generic helper, either: + - hand the extraction to the utility-consolidation sibling project when it + is in scope there, or + - leave it in place and record it under that project. + +Suggested order: + +- Foundation and libraries. +- Text/editing command modules. +- UI modules. +- Programming modules. +- Org modules. +- Optional integrations. + +Exit criteria: + +- Direct module load either succeeds or fails with a clear missing external + package/config message. +- =make test-file FILE=test-all-comp-errors.el= passes. +- New tests cover any helper extracted while fixing dependencies. +- Helper extraction remains dependency-light and does not pull heavy packages + into foundation startup. + +** Phase 3: Keymap Registration Boundary + +Introduce a small keymap registration API before deferring many feature modules. + +Possible API: + +#+begin_src emacs-lisp +(defun cj/register-prefix-map (key map label) + "Register MAP under KEY in `cj/custom-keymap' with LABEL for which-key." + ...) + +(defun cj/register-command (key command label) + "Register COMMAND under KEY in `cj/custom-keymap' with LABEL for which-key." + ...) +#+end_src + +Design rules: + +- =keybindings.el= owns =cj/custom-keymap= and the global =C-;= binding. +- Feature modules may define maps and commands without mutating global keys + directly. +- Which-key labels must be registered after which-key loads. +- Tests can assert key resolution without loading every feature package. + +Exit criteria: + +- Modules no longer need to assume =cj/custom-keymap= exists at top level + except through the registration API. +- Existing =C-;= bindings continue to resolve. +- Which-key labels for documented prefixes remain available. + +** Phase 4: Defer Low-Risk Optional Modules + +Start with modules that are unlikely to affect first-frame startup. + +Candidate batch: + +- =games-config= +- =music-config= +- =weather-config= +- =gloss-config= +- =lorem-optimum= +- =jumper= +- =httpd-config= +- =prog-training= + +For each module: + +1. Keep its user-facing command/key available via the default deferral mechanism + above. +2. Move package loading into =use-package :commands=, =:hook=, =:mode=, or an + explicit autoload/wrapper only when the default does not fit. +3. Run targeted tests and an interactive smoke check. + +Exit criteria: + +- Startup no longer requires the module eagerly. +- User command still works from a fresh Emacs session. +- Module-specific tests pass. + +** Phase 5: Defer Heavy Domain Modules + +Candidate batch: + +- =pdf-config= +- =calibredb-epub-config= +- =video-audio-recording= +- =transcription-config= +- =mail-config= +- =ai-config= +- =restclient-config= +- =elfeed-config= +- =erc-config= +- =slack-config= + +These need more care because they often combine package setup, auth, keymaps, +processes, hooks, and user workflows. + +Exit criteria for each: + +- Commands are discoverable before package load. +- Package load happens through the default deferral mechanism: command, hook, + mode, or explicit startup opt-in. +- Auth and private config are not read until necessary unless the user opts in. +- Batch/test startup does not start network/process work. + +Private config opt-in follows the =calendar-sync.local.el= precedent: a module +reads =.local.el= when readable, the file is gitignored, and the +module degrades cleanly when the file is missing. Token rotation is a separate +security task; this convention is about config presence, not secret protection. + +** Phase 6: Revisit Org And Programming Eagerness + +Org and programming modules are daily-use, so the goal is not blindly deferring +everything. + +Programming target: + +- Keep generic programming defaults and F-key command entry points available. +- Load language-specific modules by major mode. +- Consolidate generic LSP policy under =prog-lsp=. + - Move to =prog-lsp=: global LSP toggles such as =lsp-idle-delay=, + =lsp-log-io=, =lsp-enable-folding=, =lsp-enable-snippet=, + =lsp-headerline-breadcrumb-enable=, and file-watch ignore lists. + - Keep per-language: server client settings such as + =lsp-clients-clangd-args= and =lsp-pyright-*=, plus language-mode hook + wiring. +- Tree-sitter grammar auto-install is always on; the project policy is global + allow. =treesit-auto-install= is =t= without per-language conditionals. + +Org target: + +- Keep these daily first-session workflows eager: =org-config=, + =org-agenda-config=, =org-capture-config=, =org-refile-config=, + =calendar-sync= when local config is present, and =org-roam-config=. +- Defer exporters, reveal, drill, noter, webclipper, and optional publishing + pieces behind commands/hooks. +- Normalize agenda/refile cache lifecycle before changing timer behavior. This + is behavioral normalization within the load-graph project; the shared + =cj-cache.el= extraction is owned by utility-consolidation Phase 5 and may + follow. + +The =prog-lsp= consolidation and tree-sitter policy decisions are owned by this +load-graph project. Utility consolidation owns reusable helper extraction, not +programming policy. + +Exit criteria: + +- Common daily Org/programming workflows work from a fresh session. +- Optional exporters/languages load when used. +- Timers are guarded in batch/test contexts. + +* Adjacent Project: Utility Consolidation + +The review of this spec identified a related but distinct architectural +problem: helper functions are scattered across feature modules, sometimes with +duplicated behavior. This matters to the load graph because modules can become +coupled to whichever feature file happened to define a useful helper first. + +This should be tracked as a sibling project, not folded into the load-graph +project. The load-graph project asks "when and why does this module load?" The +utility consolidation project asks "which module should own this reusable +behavior?" Those questions overlap, but their changes have different risk and +rollback shapes. + +This sibling project can run beside Phase 2. When explicit-dependency work finds +a generic duplicated helper, the sibling project owns the extraction commit when +the helper is in scope for that project. See +[[id:fc2e3926-b4a1-4b45-92eb-20841e13f655][utility-consolidation-spec.org]] for candidate +helpers, naming rules, dependency budgets, migration phases, and test policy. + +* Testing Strategy + +** Static/Batch Tests + +Add or extend tests for: + +- Direct module load smoke tests for modules in each batch. +- Header validation: every module required by =init.el= declares the seven + required load-graph header lines. + - Test file: =tests/test-init-module-headers.el=. + - Assertion shape: inspect every module required by =init.el=, read its + commentary header, and fail with the missing line names for any absent + required header line. +- Keymap registration: prefix maps and commands resolve without requiring the + feature implementation package. +- No startup timers/processes in batch for side-effect modules. +- =init.el= startup smoke in batch, where possible. +- Byte/native compile smoke via existing =test-all-comp-errors.el=. + +Test files for this project use =test-init-.el=, for example +=test-init-module-headers.el= and =test-init-keymap-registration.el=. This keeps +load-graph validation tests distinct from per-module unit tests. + +Header validation runs directly against module files. It does not depend on the +final =docs/design/module-inventory.org= format, which remains a Phase 1 +authoring decision. + +** Automated Smoke Checks + +Automate every smoke item that can run in batch: + +- Important keybindings resolve to the intended command symbols, including + =C-;= prefixes and F4/F6/F7 entry points. +- Org capture and agenda command entry points load or produce expected + batch-safe guidance. +- Calendar sync status reports configured/no-config state without starting + timers or network fetches in batch. +- Optional commands touched in the batch autoload and resolve. +- Non-graphical interactive flows use =execute-kbd-macro= or + =with-simulated-input= where practical. + +These checks should run under =make test= for every migration commit. + +** Manual Smoke Checks + +Each migration batch should be followed by an interactive restart and checklist: + +- First frame appears with expected theme/font/modeline. +- =C-;= prefix appears and key descriptions are present. +- Magit opens. +- Mail command opens or gives expected package/config guidance. +- Refile target lookup works in an interactive session. +- Any optional command changed in the batch runs end to end. +- If daemon mode is part of normal use, run the visual checklist once via + regular =emacs= and once via =emacsclient= against a running daemon. + +** Performance Checks + +Before and after major batches: + +- Record =emacs-init-time=. +- Record a startup profile baseline and diff, preferably with =benchmark-init= + if enabled for the phase. +- =benchmark-init= is installed via package.el. The activation block in + =early-init.el= is commented; uncomment it locally during phases that need + profiling and do not commit the activation. Profile output goes to + =.profile/=, which should stay gitignored. +- Suggested workflow: + - =make profile-baseline= records =emacs-init-time= and a startup profile to + =.profile/baseline.txt=. + - =make profile-diff= records the current run and compares it to the phase + baseline. +- Keep a simple note of eagerly loaded feature count from + =cj/info-loaded-features= or equivalent. + +Performance is a supporting signal. Correctness and explicit dependencies are +the primary acceptance criteria. Startup regressions larger than roughly 50 ms +against the phase baseline should be investigated and explained; after several +stable baseline runs, this can become a stricter gate. + +* Acceptance Criteria + +The project is complete when: + +- =init.el= contains only documented eager requires and explicit startup calls. +- Optional modules no longer load merely because Emacs started. +- Each module required by =init.el= has a category and eager/deferred rationale. +- Modules that remain eager have no hidden dependencies on arbitrary earlier + init order. +- Global key registration has a central owner/convention. +- Top-level timers/process/network work is either removed, guarded, or + documented as intentional. +- Full =make test= passes. +- Byte/native compile smoke passes. +- Interactive startup checklist passes. + +* Risks And Mitigations + +** Risk: Breaking muscle-memory keybindings + +Mitigation: + +- Change key registration mechanics before changing bindings. +- Add keymap resolution tests for important prefixes. +- Keep a per-batch manual keybinding checklist. + +** Risk: Lazy-loaded packages miss early hook setup + +Mitigation: + +- Prefer =use-package :hook= and =:mode= over ad hoc lazy command bodies for mode + packages. +- Add tests that inspect hook contents where possible. +- Smoke-test opening representative files. + +** Risk: Daily workflows silently stop starting + +Mitigation: + +- Distinguish "safe default" from "local opt-in" for workflows like calendar + sync. +- Use ignored/local config files for private eager opt-ins. +- Report missing config clearly. + +** Risk: Batch tests differ from interactive startup + +Mitigation: + +- Guard timers/process/network work with =noninteractive= only when that is the + intended distinction. +- Add at least one interactive checklist per migration batch. + +** Risk: Refactor becomes too broad + +Mitigation: + +- One batch, one module family. +- Do not mix dependency fixes, keybinding redesign, and package lazy-loading in + the same commit unless tightly coupled. +- Keep rollback easy by preserving user-facing commands and using wrappers. + +* Implementation Backlog + +The project in =todo.org= should remain the source of task state. This design +supports these implementation tickets: + +1. Classify modules by role and startup requirement. +2. Add explicit module dependencies before changing load order. +3. Centralize custom keymap registration. +4. Defer low-risk optional modules. +5. Defer heavy document/media/integration modules. +6. Revisit programming module LSP/tree-sitter ownership. +7. Revisit Org module cache/timer and optional extension loading. +8. Retire or rewrite stale =init.el= comments. +9. Create a sibling utility consolidation project with an inventory pass and + first helper extractions. + +* Open Questions + +- Should =config-utilities= remain eager because debug commands are useful + during startup work, or should it become command-loaded after this project? +- Should local/private opt-ins share one file, or should modules keep + workflow-specific local files such as =calendar-sync.local.el=? +- Should the module inventory become machine-readable for validation, or is an + org table enough? Decide during Phase 1 based on inventory authoring + experience. +- Should =init.el= ultimately become declarative sections plus an explicit + startup contract list? + +* Next Steps + +1. Use this document as the reference for the =Classify modules by role and + startup requirement= task. +2. Build the first inventory directly from the module table above, correcting + category guesses while inspecting each file. +3. Do not defer a module until its direct runtime dependencies are explicit. +4. Implement keymap registration before deferring feature modules that currently + mutate =cj/custom-keymap= at top level. +5. Create the sibling utility consolidation project before Phase 2 work begins, + so duplicated helpers found during dependency cleanup have a clear place to + land. diff --git a/docs/specs/keybinding-console-safety-spec-doing.org b/docs/specs/keybinding-console-safety-spec-doing.org deleted file mode 100644 index 4a1dec81..00000000 --- a/docs/specs/keybinding-console-safety-spec-doing.org +++ /dev/null @@ -1,943 +0,0 @@ -:PROPERTIES: -:ID: 540bf06b-16b8-46c6-b459-c40d1b9c795d -:STATUS: doing -:END: -#+TITLE: Keymap Consolidation — Spec -#+AUTHOR: Craig Jennings -#+DATE: 2026-06-12 - -* Metadata -| Status | doing | -|----------+--------------------------------------------------------------------| -| Owner | Craig Jennings | -|----------+--------------------------------------------------------------------| -| Reviewer | TBD (multi-reviewer cycle) | -|----------+--------------------------------------------------------------------| -| Related | [[file:../../todo.org][todo.org]] — "M-S- launcher keys" task (to be reclassified, Phase 0) | -|----------+--------------------------------------------------------------------| - -* Summary - -Some commonly-used window/layout commands are bound to =M-S-= chords that only work in GUI frames, via a fragile =key-translation-map= layer that already caused a regression. - -The primary work consolidates the common commands under the =cj/custom-keymap= personal keymap and retires the fragile translation layer — independent of any prefix choice. Console reachability is then a one-line, *optional* follow-on: bind that one keymap to a single console-safe prefix (a =Control=+key, or a free =M-=; candidates in Appendix C), used everywhere. Per Path 2 (2026-06-13), the work proceeds up to the point of assigning that prefix and stops there; the assignment is a deferred phase Craig takes when he picks the key. - -The aim: consolidate the common commands into one keymap and retire the translation block now, leaving a single, optional console-safe prefix to switch on later. - -* Problem / Context - -A subset of common commands is bound to =M-S-= chords (Meta + Shift + lowercase letter). Pressing Meta+Shift+e emits the event =M-E= (uppercase Meta), but the command is bound to =M-S-e=; the bridge between them is a =key-translation-map= entry that =modules/keyboard-compat.el= installs *only* in GUI frames (=env-gui-p=). So these chords are dead in terminal frames and dead in the Linux console. - -Craig does not use terminal or console Emacs often, but falls back to the console in emergencies (a broken graphical session). When common keys are unavailable there, the editor stops being usable for the emergency and he has to switch tools. For *uncommon* commands, =M-x= is an acceptable fallback; for *common* ones it is not. - -How each key family actually behaves across the three contexts (the facts the design turns on): - -| Context | Meta sent as | =M-S-e= (as bound) | =M-E= (uppercase Meta) | =C-;= | -|-------------------------+--------------+-------------------------+-------------------------+-------------------------| -| GUI frame | native event | reached only via the | intercepted by the | works natively | -| | | GUI translation map | translation map | | -|-------------------------+--------------+-------------------------+-------------------------+-------------------------| -| Terminal emulator | ESC prefix | dead (keypress emits | works (ESC E), if no | works if the emulator | -| (xterm-family) | | =M-E=, binding is on | translation intercepts | speaks | -| | | =M-S-e=) | | modifyOtherKeys/kitty | -| | | | | (recent Emacs | -| | | | | auto-enables for | -| | | | | xterm-family) | -|-------------------------+--------------+-------------------------+-------------------------+-------------------------| -| Linux console | ESC prefix | dead (same reason) | works (ESC E) | DEAD — semicolon is not | -| (TERM=linux) | | | | a control char; cannot | -| | | | | be transmitted | -|-------------------------+--------------+-------------------------+-------------------------+-------------------------| - -Three consequences: =M-S-e= is dead outside GUI by construction; =C-;= is solid in GUI, conditional in terminal emulators, and dead in the Linux console (so it cannot be the *only* home for console-critical commands); and =M-E= plus function keys and =C-c= sequences are transmittable everywhere, which is the material to build a console-safe path from. - -** The regression that triggered this - -Commit =4a1ecf64= "fixed" three launcher keys (=eww=/=elfeed=/=calibredb=) by rebinding them from =M-S-e/r/b= to =M-E/M-R/M-B=. It was wrong, and three review passes missed it because they all used =key-binding=, which consults keymaps only and ignores =key-translation-map=. The original audit "verified dead in the live daemon" with that blind check (false positive); the fix bound =M-E= but left the =M-E -> M-S-e= translation entry in place, so in GUI the keypress is rewritten to the now-unbound =M-S-e= and the launchers break on the next restart; and the new test asserted =(key-binding (kbd "M-E"))=, passing against a configuration broken at the keyboard. It only appears to work in the running daemon because the pre-fix binding is still loaded as stale state — the stale-daemon trap. - -The lesson is encoded into the acceptance criteria: real reachability is not =key-binding= when a translation map participates. - -* Goals and Non-Goals - -** Goals -- Every *commonly used* command is reachable in GUI, terminal emulators, and the Linux console. -- One canonical personal command surface, so console-reachability is solved once at the prefix level rather than per command. -- Retire the =keyboard-compat.el= =M-uppercase -> M-S-lowercase= translation block, the root of the fragility. -- Keep daily ergonomics: high-frequency commands keep a fast chord in GUI. - -** Non-Goals -- Making *every* binding console-safe. Uncommon commands may live on =M-x= only. -- A ground-up keymap redesign. This is about reachability and retiring one fragile mechanism. -- Defeating the Linux virtual console's hard limits (it cannot transmit =C-;=, and Meta+Shift behaviour varies). The design routes around them. - -** Scope tiers -- *v1 (primary — Phases 0–2):* - - revert =4a1ecf64= (Phase 0, unblocks the push); - - prune the forgotten keybindings Craig marks in Appendix D; - - migrate the common window/layout =M-S-= commands into =cj/custom-keymap=; - - drop the uncommon chords to =M-x=; - - retire the translation block; - - translation-aware tests. -- *Deferred / optional (Phase 3):* - - bind =cj/custom-keymap= to a single console-safe prefix (D1/D3) once Craig picks the key — the console-reachability payoff, switched on later. -- *Out of scope:* - - enabling modifyOtherKeys/kitty for terminal emulators (helps terminals, not the console; orthogonal). -- *vNext:* auditing non-=M-S-= GUI-only chords (modified function keys, =C-+/C-==) for console behaviour. - -* Design - -Craig's first choice (review, 2026-06-12): instead of two prefixes, retrain muscle memory onto *one* console-safe prefix that works everywhere — a =Control= + home-row key that is lightly used or easily/intuitively rebound. =cj/custom-keymap= moves from =C-;= to that single prefix (=C-;= may stay bound during the transition since one keymap can carry many prefixes). The candidate list is Appendix C; the standout home-row candidate is =C-l=. The two-prefix design below is the documented fallback if no single prefix proves acceptable. - -The personal command surface is already a single keymap object, =cj/custom-keymap= (=modules/keybindings.el=), bound to =C-;=. The whole design rests on one Emacs fact: a keymap is an object and can be bound to more than one prefix. So console-reachability is a *prefix* problem, not a per-command problem. - -For a user (single-prefix path): you reach your personal commands with the one console-safe prefix in GUI, terminal, and console alike — same menu, same keys after the prefix, nothing per-context to remember. (Fallback two-prefix path: =C-;= in the GUI as today, plus a second console-safe prefix anywhere.) - -For the implementer: add one line — =(keymap-global-set "" cj/custom-keymap)= — and the entire tree in Appendix A becomes reachable through it; nothing per-command. The work then is to move the *common* console-dead commands (the window/layout =M-S-= subset, Appendix B) *into* =cj/custom-keymap= so they inherit that reachability, drop the *uncommon* =M-S-= chords to =M-x=, and delete the now-unused translation block. High-frequency window commands additionally keep a fast chord so daily GUI use doesn't regress to a 3-key sequence (Decision D4). - -The console-dead common set is window/layout work, which has no =C-;= sub-prefix today, so v1 adds one (a new window sub-map; letter is Decision D5). The =C-c=/=C-h=/=C-z=/=C-x= and plain function-key bindings already work in the console and stay where they are. - -* Alternatives Considered - -** A — Revert 4a1ecf64 and keep the translation layer as the end state -- Good, because it is the smallest change and restores correctness immediately. -- Bad, because it keeps 18 keys on the GUI-only mechanism that already bit us and - leaves the console-dead problem unsolved. -- Neutral, because the revert itself is still needed as Phase 0; this option just - stops there. - -** B — Migrate the whole family to direct uppercase-Meta, delete the translation block, no C-; move -- Good, because it preserves every single-chord and =M-E= (ESC + uppercase) is - transmittable in GUI, terminal, and console alike. -- Bad, because it bets the emergency-console guarantee on Meta+Shift behaving - cleanly on every console keyboard, which is probable but not certain, and it - gives the common commands no robust prefix-based fallback. -- Neutral, because it still deletes the translation block (shared with the chosen - design) and could be layered onto the frequent-chord subset (see D4 Option B). - -** C — Enable an enhanced keyboard protocol (modifyOtherKeys / kitty) so C-; works in terminals -- Good, because it makes =C-;= itself work in capable terminal emulators. -- Bad, because it does nothing for the Linux virtual console (a hard limit), and - adds a terminal-capability dependency. -- Neutral, because it is orthogonal and could be added later without conflicting. - -** Chosen — one map, two prefixes (consolidate common commands under C-;, add a console-safe alt prefix) -- Good, because console-reachability is solved once at the prefix; it depends on - exactly one prefix working, and that prefix is chosen to be bulletproof. -- Bad, because moved commands cost a muscle-memory transition, and a pure - sub-prefix path is 3 keys (mitigated by D4 for the frequent ones). -- Neutral, because it still requires the revert (Phase 0) and the translation- - block deletion (shared with B). - -* Decisions [3/5] - -Each decision is a TODO task. It flips to DONE when Craig agrees with the call; if -he doesn't, it stays TODO and the discussion continues under its =*** Discussion= -child header. - -*Gate (Path 2).* The decisions split by which work they block. D2, D4, and D5 gate -the *primary* work (Phases 0–2: revert, prune, consolidate, retire the translation -block); the spec is implementation-ready for that work once those three are DONE. D1 -and D3 (the console-safe prefix) gate *only* the optional Phase 3 — they can stay -TODO indefinitely without blocking the consolidation. So yes: the work proceeds all -the way to the point of assigning the prefix and stops there, exactly as Craig asked, -even if D1/D3 are never decided. The =[n/5]= cookie tracks the overall tally; full -=ready= (including Phase 3) still needs all five. - -** DONE D1 — One map, one console-safe prefix (single-prefix primary; two-prefix fallback) -CLOSED: [2026-06-13 Sat 00:20] -- Owner / by-when: Craig / review cycle -- Context: the common console-dead commands need to be reachable in the console; - =C-;= alone is dead there; per-command console bindings would not scale. -- Decision (revised): We keep =cj/custom-keymap= as the single personal surface. - *Primary (Craig's first choice):* rebind it to ONE console-safe prefix — a - =Control= + lightly-used home-row key (Appendix C; standout =C-l=) — used in GUI, - terminal, and console alike, retraining muscle memory off =C-;=. =C-;= may stay - bound during the transition. *Fallback:* if no single prefix is acceptable, bind - the map to both =C-;= (GUI) and one console-safe alternate prefix (D3). -- Consequences: easier — one prefix to make console-safe, whole tree travels, and - the single-prefix path needs no per-context mnemonic; harder — every - console-critical command must actually live under =cj/custom-keymap= (so the - common =M-S-= set is still migrated in), and the single-prefix path costs a - full =C-;= → new-prefix muscle-memory transition. -*** Discussion -- Direction agreed by Craig 2026-06-12: single-prefix primary, two-prefix fallback. -- Deferred by Craig 2026-06-13 (Path 2): the console-safe prefix becomes the optional - Phase 3, not part of the primary work. The consolidation (Phases 0–2) lands without - it, so D1 no longer blocks anything until Craig chooses to do Phase 3. It stays TODO - as the marker for "decide the prefix later." The phases are rewritten accordingly, - and the keybinding audit Craig asked for lives in Appendix D. - -** DONE D2 — Migrate only the common (window/layout) M-S- set; drop the uncommon to M-x -CLOSED: [2026-06-13 Sat 00:22] -- Owner / by-when: Craig / review cycle -- Context: of the 18 =M-S-= commands, only window/layout control is plausibly - needed in an emergency console session; apps and one-off tools are not. -- Decision: We will move the window/layout subset (=M-S-o/m/v/h/t/u/z=, and - =M-S-k= pending review) into =C-;=, and remove the other ten =M-S-= chords, - leaving those commands on =M-x=. -- Consequences: easier — shrinks the translation block to nothing, focuses the - console surface on essentials; harder — the dropped commands lose a chord; - =show-kill-ring='s classification is a judgment call. -*** Discussion -- Not yet reviewed by Craig. Open: confirm the window/layout subset to migrate - (incl. =M-S-k= show-kill-ring's common/uncommon call) and that the other ten - drop to =M-x=. Flip to DONE on Craig's sign-off. - - -** TODO D3 — The console-safe prefix (pick from Appendix C) -- Owner / by-when: Craig / review cycle -- Context: under D1's single-prefix primary, this prefix is THE personal-keymap - prefix; under the two-prefix fallback it is the second (alternate) binding. It - must transmit in the Linux console, where only =Control= + letter chords carry - (and TAB/RET/LF/ESC/DEL collisions and =C-g= are excluded). Full candidate - analysis is Appendix C. -- Decision: For the single-prefix path, =C-l= is the standout (home-row, - console-safe, default =recenter-top-bottom= is light and trivially relocatable); - =C-q= / =C-o= / =C-t= are off-home-row runners-up. For the two-prefix fallback, - =C-c ;= (=C-c= transmits; =;= is a plain key; mnemonic mirror of =C-;=) stays the - recommendation. Craig picks the prefix. -- Consequences: easier — solves console reachability for the whole tree at one - binding; harder — a single =Control=+letter prefix displaces its default command - (relocate =recenter-top-bottom= if =C-l=), and =C-l= is also bound to - =vertico-insert= inside the minibuffer (=selection-framework.el:42=) — minibuffer- - local, so no conflict with a global prefix, but worth noting. -*** Discussion -- Open: Craig picks the prefix. Recommendation =C-l= (only clean home-row option); - runners-up =C-q= / =C-o= / =C-t=. Flip to DONE on the pick. D1 closes with it. - - -#+begin_src cj: comment -it's not going to be C-l. That's too hard of a habit for me to kick right now. I'd rather go C-c ; altogether -- even in GUI -- than have C-l do the wrong thing when I hit it. We'll find something. But it's not decided yet. Change the status of this decision to waiting. -#+end_src - -** DONE D4 — Fast-chord strategy for high-frequency window ops -CLOSED: [2026-06-13 Sat 00:25] -- Owner / by-when: Craig / review cycle -- Context: =split-and-follow-right/below= and =undo-kill-buffer= are pressed - constantly; a 3-key =C-; v= sequence is a real downgrade. -- Decision: We will (Option A) keep a fast GUI chord for the frequent commands in - addition to their =C-;= entry, OR (Option B) bind them to direct uppercase-Meta - single chords and retire the translation block. Review picks. -- Consequences: A — preserves speed, but the fast chord may itself be GUI-only - unless it is a function key; B — single chord works in all three contexts but - leans on console Meta+Shift. -*** Discussion -- Open: Craig picks Option A (keep fast GUI chord) vs Option B (direct - uppercase-Meta single chords). Note: if D3 lands a single console-safe prefix, - Option B's console rationale weakens. Flip to DONE on the pick. - -#+begin_src cj: comment -we can simply revert -#+end_src - -** TODO D5 — Window sub-prefix and apps disposition -- Owner / by-when: Craig / review cycle -- Context: window/layout has no =C-;= sub-prefix. Free single lowercase letters are - =i q u y z= (=g= is calendar, =h= is Hugo — both taken); most uppercase is free. - =C-; L= is reserved for the Pearl/Linear package — do NOT reuse it. The four apps - (=eww=/=elfeed=/=calibredb=/=wttrin=) could go to =M-x= or a launcher sub-prefix. - - #+begin_src cj: comment - add a listing of the keybindings we're discussing. I don't know what the window/layout keybindings you're discussing. It's not shift arrow keys, is it? - #+end_src -- Decision: We will add a window sub-prefix under =C-;= (letter TBD from the free - set). Apps: Craig decided the launcher commands get real keys under a launcher - sub-prefix (=e/f/b/w= leaves), NOT =M-x=. Sub-prefix letter TBD; the freed - =C-; a t= (ai-assistant toggle, see Phase 0) is one candidate location if the - apps belong nearer the AI tools. Both sub-prefix letters are Craig's pick. -- Consequences: easier — groups window ops and launcher apps discoverably under - which-key, and the launcher apps inherit console reachability for free; harder — - spends two scarce top-level =C-;= letters from the small free set. -*** Discussion -- Apps half agreed by Craig 2026-06-12: launcher sub-prefix, not =M-x=. Open: the - window sub-prefix letter and the launcher sub-prefix letter, both from the free - set {=i q u y z=} + uppercase (NOT =L= — Pearl). Flip to DONE once both letters - are chosen. - -* Implementation phases - -Path 2 (Craig, 2026-06-13): Phase 0 is a *pure revert* that unblocks the held push; the migration follows, and the console-safe prefix is an *optional, deferred* phase. Everything proceeds up to the point of assigning the prefix (end of Phase 2) and stops there; Phase 3 is the optional assignment once Craig picks the prefix. So the consolidation does not wait on the prefix decision (D1/D3); only Phase 3 does. - -** Phase 0 — Revert the regression (unblocks the push) -Revert =4a1ecf64= and nothing more: restore =M-S-e/r/b= in the three modules and delete the flawed test (=tests/test-launcher-meta-shift-keys.el=), leaving a clean, correct baseline. Reclassify the "M-S- launcher keys" task as not-a-bug — the keys worked via the GUI translation layer. This is the only step the held 12-commit stack needs before it can push. Per Path 2, the launchers get reverted to =M-S-= here and move to their new homes in Phase 2 — the accepted small throwaway (3 bindings) of not waiting on the full move-map. - -The flawed test asserts the launcher bindings with =key-binding= alone: - -#+begin_src emacs-lisp -(should (eq (key-binding (kbd "M-E")) 'eww)) -(should (eq (key-binding (kbd "M-R")) 'cj/elfeed-open)) -(should (eq (key-binding (kbd "M-B")) 'calibredb)) -#+end_src - -=key-binding= consults keymaps only and ignores =key-translation-map=, so the test passes even though the GUI translation entry =M-E -> M-S-e= rewrites the keypress back to the now-unbound =M-S-e=. It cannot see the rewrite, so it certifies a configuration that is broken at the keyboard. Phase 2's translation-aware assertion replaces it. - -** Phase 1 — Audit and prune forgotten keybindings (Appendix D) -Appendix D inventories every keybinding Craig has set outside the =C-;= tree and the =M-S-= family — the place to catch chords set-and-forgotten. Craig checks the boxes for the bindings to retire; remove those. Independent cleanup, and a good moment to clear cruft before the migration. Tree working. - -** Phase 2 — Consolidate: migrate the common set, retire the translation block -The primary deliverable, needing *no* console-safe-prefix decision. Migrate the window/layout =M-S-= subset into =cj/custom-keymap= under a new window sub-prefix (D5); add the launcher sub-prefix (D5) with the =eww=/=elfeed=/=calibredb=/=wttrin= leaves (freeing =C-; a t= — the =cj/toggle-gptel= ai-assistant toggle, =ai-config.el:541=, unfinished and far less used than the =ai-term= F9 launcher — if the letter is tight); apply the fast-chord strategy (D4); drop the ten uncommon =M-S-= chords to =M-x= (D2); delete =keyboard-compat.el='s translation block and its hook (keep the arrow-key =input-decode-map= setup); add the translation-aware tests (see Acceptance criteria) and update the docs. At the end of Phase 2 the work is "done" per Craig's stop point. Tree working. - -** Phase 3 — (OPTIONAL, deferred) Bind the console-safe prefix -Only once Craig picks the prefix (D1/D3, Appendix C). Bind =cj/custom-keymap= to it — =(keymap-global-set "" cj/custom-keymap)= — and if the pick is =C-l=, relocate its default =recenter-top-bottom= first. This is the console-reachability payoff: the whole tree becomes reachable in =emacs -nw= and the Linux console through one prefix. Verify in a *fresh* session, not the live daemon. May be deferred indefinitely; the consolidation stands on its own without it. - -* Acceptance criteria -- [ ] The whole =cj/custom-keymap= tree is reachable in a GUI frame, an =emacs -nw= xterm-family terminal, and the Linux virtual console via the alt prefix. -- [ ] The final "common" commands are reachable in all three contexts. -- [ ] =keyboard-compat.el='s translation block is gone; no command depends on it. -- [ ] For any chord claimed to run command X, tests assert BOTH =(key-binding (kbd CHORD))= AND =(lookup-key key-translation-map (kbd CHORD))= are consistent (the latter =nil=, or pointing where intended). =key-binding= alone is insufficient — it is what let =4a1ecf64= through. -- [ ] Reachability is verified in a *fresh* frame/session, not the live daemon (the stale-daemon trap masks results). -- [ ] =make test= fully green (the 4 pre-existing =test-dupre-theme= failures are tracked separately and out of scope). - -* Readiness dimensions -- Data model & ownership: keybindings are user-authored code in =modules/=; - =cj/custom-keymap= is the owned surface. Nothing generated/cached/remote; - nothing persists. -- Errors, empty states & failure: N/A — a missing command symbol surfaces as a - load-time =void-function=, caught by byte-compile and the launch smoke test. -- Security & privacy: N/A — no credentials or sensitive data. -- Observability: which-key shows each prefix's menu; =C-h k= / =describe-bindings= - report the live binding; the translation-aware test reports reachability. -- Performance & scale: N/A — keymap lookup is constant-time; one extra prefix - binding has no measurable cost. -- Reuse & lost opportunities: reuse Emacs's native multi-prefix keymap binding - (one keymap object, two prefix keys) instead of duplicating bindings; reuse - which-key and the existing =cj/register-prefix-map= / =cj/register-command= - helpers. Deletes (does not wrap) the bespoke translation layer. -- Architecture fit & weak points: integration points are =keybindings.el= - (=cj/custom-keymap=, the register helpers), =keyboard-compat.el= (translation - block to delete; arrow-key decode to keep), and the per-module =:bind= / - register calls for the migrated commands. Weak point: the stale-daemon trap can - mask whether a change actually works — mitigated by verifying in a fresh - =-nw=/console session (acceptance criterion). -- Config surface: the console-safe alt prefix (D3) and the window sub-prefix - letter (D5) are the only new knobs; both are constants set once in config. -- Documentation plan: update the =keyboard-compat.el= header (it documents the - retired translation table); note the moved/dropped keys wherever keybindings - are documented. No user-facing migration doc beyond that. -- Dev tooling: existing =make test= / byte-compile / launch smoke cover it; the - new translation-aware assertion is an ERT test like the others. -- Rollout, compatibility & rollback: user-facing keybinding change; rollback is - =git revert=. No persisted data, no public API, no external state. The only - compatibility cost is Craig's muscle memory for the moved/dropped keys — - a transition note, not a migration. -- External APIs & deps: N/A — no external APIs; no new dependencies. - -* Risks, Rabbit Holes, and Drawbacks -- *Muscle-memory disruption* for moved/dropped keys. Dodge: keep fast chords for the highest-frequency commands (D4); accept =M-x= only for genuinely uncommon ones. -- *Console Meta+Shift uncertainty* if D4 Option B is chosen. Dodge: the prefix path (D1/D3) does not depend on it, so the emergency guarantee holds regardless of the fast-chord choice. -- *Stale-daemon trap* masking test results — the exact failure mode behind the regression. Dodge: the acceptance criteria mandate verification in a fresh frame/session and a translation-aware assertion. - -* References / Appendix - -** Appendix A — Full C-; keybinding tree (live, 2026-06-12) - -Dumped from the running daemon by walking =cj/custom-keymap= recursively. -Format: chord — command — what it does. - -*** Top-level leaves (directly on C-;) -- C-; ) — cj/jump-to-matching-paren — jump to the matching paren -- C-; / — cj/replace-fraction-glyphs — replace 1/2-style fractions with glyphs -- C-; ? — cj/flycheck-list-errors — list flycheck errors for the buffer -- C-; A — align-regexp — align region by a regexp -- C-; B — cj/choose-browser — pick the default browser -- C-; f — cj/format-region-or-buffer — format region or whole buffer -- C-; k — cj/org-babel-toggle-confirm — toggle the org-babel eval confirmation -- C-; P — cj/projectile-reset-cmds — reset projectile's cached project commands -- C-; SPC — cj/switch-to-previous-buffer — toggle to the previous buffer -- C-; T — cj/telega — open Telegram (telega) -- C-; | — display-fill-column-indicator-mode — toggle the fill-column rule -- C-; # c — cj/count-characters-buffer-or-region — count characters -- C-; # w — cj/count-words-buffer-or-region — count words - -*** C-; ! — System commands -- C-; ! ! — cj/system-command-menu — the system-command transient menu -- C-; ! e — cj/system-cmd-restart-emacs — restart Emacs -- C-; ! E — cj/system-cmd-exit-emacs — exit Emacs -- C-; ! l — cj/system-cmd-lock — lock the screen -- C-; ! L — cj/system-cmd-logout — log out of the session -- C-; ! r — cj/system-cmd-reboot — reboot -- C-; ! s — cj/system-cmd-shutdown — shut down -- C-; ! S — cj/system-cmd-suspend — suspend - -*** C-; a — AI / gptel -- C-; a . — cj/gptel-add-this-buffer — add current buffer to the gptel context -- C-; a A — cj/gptel-autosave-toggle — toggle conversation autosave -- C-; a b — cj/gptel-browse-conversations — browse saved conversations -- C-; a B — cj/gptel-switch-backend — switch the LLM backend -- C-; a c — cj/gptel-context-clear — clear the gptel context -- C-; a d — cj/gptel-delete-conversation — delete a saved conversation -- C-; a f — cj/gptel-add-file — add a file to the context -- C-; a l — cj/gptel-load-conversation — load a saved conversation -- C-; a m — cj/gptel-change-model — change the model -- C-; a M — gptel-menu — the gptel transient menu -- C-; a p — gptel-system-prompt — edit the system prompt -- C-; a q — cj/gptel-quick-ask — quick one-off ask -- C-; a r — cj/gptel-rewrite-with-directive — rewrite region with a directive -- C-; a R — cj/gptel-rewrite-redo-with-different-directive — redo rewrite, new directive -- C-; a s — cj/gptel-save-conversation — save the conversation -- C-; a t — cj/toggle-gptel — toggle the gptel chat buffer -- C-; a x — cj/gptel-clear-buffer — clear the chat buffer - -*** C-; b — Buffer & file operations -- C-; b — cj/window-resize-sticky — sticky window resize (arrow keys) -- C-; b b — cj/clear-to-bottom-of-buffer — clear from point to end -- C-; b c b — cj/copy-to-bottom-of-buffer — copy point-to-end -- C-; b c t — cj/copy-to-top-of-buffer — copy point-to-start -- C-; b c w — cj/copy-whole-buffer — copy the whole buffer -- C-; b d — cj/delete-buffer-and-file — delete the buffer and its file -- C-; b D — cj/diff-buffer-with-file — diff buffer against its file on disk -- C-; b e — eval-buffer — eval the buffer -- C-; b E — cj/view-email-in-buffer — view the buffer as email -- C-; b g — revert-buffer — revert from disk -- C-; b k — cj/kill-buffer-and-window — kill buffer and close its window -- C-; b K — cj/kill-other-window-buffer — kill the other window's buffer -- C-; b l — cj/copy-link-to-buffer-file — copy an org link to the file -- C-; b m — cj/move-buffer-and-file — move/rename buffer + file -- C-; b n — cj/copy-buffer-name — copy the buffer name -- C-; b o — cj/xdg-open — open the file with the system handler -- C-; b O — cj/open-this-file-with — open with a chosen program -- C-; b p — cj/copy-buffer-source-as-kill — copy buffer source -- C-; b P — cj/print-buffer-ps — print the buffer (PostScript) -- C-; b r — cj/rename-buffer-and-file — rename buffer + file -- C-; b s — mark-whole-buffer — select all -- C-; b S — write-file — write/save-as -- C-; b t — cj/clear-to-top-of-buffer — clear from start to point -- C-; b w — cj/view-buffer-in-eww — render the buffer in EWW -- C-; b x — erase-buffer — erase the buffer - -*** C-; c — Case -- C-; c l — cj/downcase-dwim — downcase (dwim) -- C-; c t — cj/title-case-region — title-case the region -- C-; c u — cj/upcase-dwim — upcase (dwim) - -*** C-; C — Comments -- C-; C - — cj/comment-hyphen — hyphen divider comment -- C-; C b — cj/comment-box — boxed comment -- C-; C c — cj/comment-inline-border — inline bordered comment -- C-; C d — cj/delete-buffer-comments — delete all comments in the buffer -- C-; C h — cj/comment-heavy-box — heavy box comment -- C-; C n — cj/comment-block-banner — block banner comment -- C-; C p — cj/comment-padded-divider — padded divider comment -- C-; C r — cj/comment-reformat — reformat a comment -- C-; C s — cj/comment-simple-divider — simple divider comment -- C-; C u — cj/comment-unicode-box — unicode box comment - -*** C-; d — Date / time insertion -- C-; d d — cj/insert-sortable-date — insert YYYY-MM-DD -- C-; d D — cj/insert-readable-date — insert a human-readable date -- C-; d r — cj/insert-readable-date-time — readable date + time -- C-; d s — cj/insert-sortable-date-time — sortable date + time -- C-; d t — cj/insert-sortable-time — sortable time -- C-; d T — cj/insert-readable-time — readable time - -*** C-; D — Org-drill (flashcards) -- C-; D c — cj/drill-capture — capture a drill question -- C-; D e — cj/drill-edit — open a drill file to edit -- C-; D f — cj/drill-this-file — drill the current file -- C-; D r — cj/drill-refile — refile into a drill file -- C-; D R — org-drill-resume — resume a drill session -- C-; D s — cj/drill-start — start a drill session - -*** C-; e — Email (mu4e) -- C-; e s — cj/mu4e-save-attachment-here — save attachment to current dir -- C-; e S — cj/mu4e-save-all-attachments — save all attachments -- C-; e m — cj/mu4e-save-some-attachments — save selected attachments -- C-; e {c,d,g} {i,l,s,u} — mu4e maildir searches: account {c=cmail, d=dmail, - g=gmail} x view {i=inbox, l=large >5M, s=starred/flagged, u=unread} - -*** C-; E — ERC (IRC) -- C-; E b — cj/erc-switch-to-buffer-with-completion — switch ERC buffer -- C-; E c — cj/erc-join-channel-with-completion — join a channel -- C-; E C — cj/erc-connect-server-with-completion — connect to a server -- C-; E l — cj/erc-connected-servers — list connected servers -- C-; E q — erc-part-from-channel — leave a channel -- C-; E Q — erc-quit-server — quit a server - -*** C-; g — Calendar sync (Google Calendar) -- C-; g s — calendar-sync-now — sync now -- C-; g S — calendar-sync-start — start auto-sync -- C-; g x — calendar-sync-stop — stop auto-sync -- C-; g t — calendar-sync-toggle — toggle auto-sync -- C-; g i — calendar-sync-status — sync status - -*** C-; h — Hugo (website/blog) -- C-; h n — cj/hugo-new-post — new post -- C-; h d — cj/hugo-open-draft — open a draft -- C-; h D — cj/hugo-toggle-draft — toggle a post's draft flag -- C-; h e — cj/hugo-export-post — export a post -- C-; h p — cj/hugo-preview — preview the site -- C-; h P — cj/hugo-publish — publish the site -- C-; h o — cj/hugo-open-blog-dir — open the blog dir in Emacs -- C-; h O — cj/hugo-open-blog-dir-external — open the blog dir externally - -*** C-; j — Jump to files -- C-; j c — cj/jump-to-contacts ; C-; j g — cj/jump-to-gcal -- C-; j i — cj/jump-to-inbox ; C-; j I — cj/jump-to-emacs-init -- C-; j m — cj/jump-to-macros ; C-; j n — cj/jump-to-reading-notes -- C-; j r — cj/jump-to-reference ; C-; j s — cj/jump-to-schedule -- C-; j w — cj/jump-to-webclipped - -*** C-; L — Pearl (Linear tickets) [RESERVED — do not reuse] -- C-; L … — pearl-prefix-map — Pearl/Linear ticket commands (lazy sub-map) -- =C-; L= is reserved as the Pearl (Linear integration) leader key. Sub-prefix - letter picks (D5) must avoid it. - -*** C-; l — Line & paragraph -- C-; l c — duplicate line/region (comment variant) ; C-; l d — cj/duplicate-line-or-region -- C-; l j — cj/join-line-or-region ; C-; l J — cj/join-paragraph -- C-; l r — cj/remove-lines-containing ; C-; l R — cj/remove-duplicate-lines-region-or-buffer -- C-; l u — cj/underscore-line - -*** C-; m — Music (EMMS) -- C-; m m — cj/music-playlist-toggle ; C-; m M — cj/music-playlist-show -- C-; m SPC — emms-pause ; C-; m s — emms-stop -- C-; m n — cj/music-next ; C-; m p — cj/music-previous -- C-; m a — cj/music-fuzzy-select-and-add ; C-; m g — emms-playlist-mode-go -- C-; m r — emms-toggle-repeat-playlist ; C-; m t — emms-toggle-repeat-track -- C-; m x — cj/music-toggle-consume ; C-; m z — emms-toggle-random-playlist -- C-; m Z — emms-shuffle ; C-; m R — cj/music-create-radio-station - -*** C-; M — Signal (signel) -- C-; M m — cj/signel-message — message a contact -- C-; M s — cj/signel-message-self — note to self -- C-; M SPC — cj/signel-connect — start/connect the daemon -- C-; M d — signel-dashboard — the Signal dashboard -- C-; M q — signel-stop — stop the daemon - -*** C-; n — Org-noter -- C-; n t — cj/org-noter-start — start noter on the document -- C-; n n — cj/org-noter-insert-note-dwim — insert a note (dwim) - -*** C-; o — Ordering / text transforms -- C-; o a — cj/arrayify ; C-; o j — cj/arrayify-json ; C-; o p — cj/arrayify-python -- C-; o u — cj/unarrayify ; C-; o l — cj/listify ; C-; o L — cj/comma-separated-text-to-lines -- C-; o A — cj/alphabetize-region ; C-; o r — cj/reverse-lines ; C-; o n — cj/number-lines -- C-; o q — cj/toggle-quotes ; C-; o o — cj/org-sort-by-todo-and-priority - -*** C-; p — reveal.js presentations -- C-; p n — cj/reveal-new ; C-; p h — cj/reveal-insert-header ; C-; p H — cj/reveal-remove-headers -- C-; p e — cj/reveal-export ; C-; p SPC — cj/reveal-present -- C-; p p — cj/reveal-preview-start ; C-; p s — cj/reveal-preview-stop - -*** C-; r — Recording (audio/video) -- C-; r a — cj/audio-recording-toggle ; C-; r v — cj/video-recording-toggle -- C-; r s — cj/recording-quick-setup ; C-; r S — cj/recording-select-devices -- C-; r d — cj/recording-list-devices ; C-; r l — cj/recording-adjust-volumes -- C-; r w — cj/recording-show-active-audio -- C-; r t b/m/s — cj/recording-test-both / -mic / -monitor - -*** C-; R — restclient -- C-; R n — cj/restclient-new-buffer ; C-; R o — cj/restclient-open-file - -*** C-; s — Enclose / surround / indent -- C-; s s — cj/surround-word-or-region ; C-; s u — cj/unwrap-word-or-region -- C-; s w — cj/wrap-word-or-region ; C-; s i — cj/indent-lines-in-region-or-buffer -- C-; s d — cj/dedent-lines-in-region-or-buffer ; C-; s a — cj/append-to-lines-in-region-or-buffer -- C-; s p — cj/prepend-to-lines-in-region-or-buffer -- C-; s I — change-inner ; C-; s O — change-outer - -*** C-; t — Test runner -- C-; t r — cj/test-run-smart ; C-; t R — cj/test-run-all ; C-; t . — cj/run-test-at-point -- C-; t a — cj/test-focus-add ; C-; t b — cj/test-focus-add-this-buffer-file -- C-; t c — cj/test-focus-clear ; C-; t v — cj/test-view-focused -- C-; t L — cj/test-load-all ; C-; t t — cj/test-toggle-mode - -*** C-; v — Version control (git / forge) -- C-; v c — cj/git-clone-clipboard-url ; C-; v d — cj/goto-git-gutter-diff-hunks -- C-; v t — cj/git-timemachine ; C-; v f — forge-pull ; C-; v r — forge-list-pullreqs -- C-; v i c — cj/forge-create-issue ; C-; v i l — forge-list-issues - -*** C-; w — Whitespace -- C-; w c — cj/collapse-whitespace-line-or-region ; C-; w d — cj/delete-all-whitespace -- C-; w l — cj/delete-blank-lines-region-or-buffer ; C-; w 1 — cj/ensure-single-blank-line -- C-; w r — cj/remove-leading-trailing-whitespace ; C-; w - — cj/hyphenate-whitespace-in-region -- C-; w t — untabify ; C-; w T — tabify - -*** C-; x — Terminal (ghostel) -- C-; x t — cj/term-toggle ; C-; x N — ghostel (new) ; C-; x c — cj/term-copy-mode-dwim -- C-; x h — cj/term-tmux-history ; C-; x l — ghostel-clear-scrollback -- C-; x n — ghostel-next-prompt ; C-; x p — ghostel-previous-prompt -- C-; x q — ghostel-send-next-key - -** Appendix B — The M-S- family (18 keys) - -All bound as =M-S-= and reachable in GUI only, via the -=keyboard-compat.el= translation layer. Format: chord — command — what it does — -source module. - -- M-S-o — cj/kill-other-window — kill the other window's buffer and close it — undead-buffers.el -- M-S-m — cj/kill-all-other-buffers-and-windows — close all other windows, kill their buffers — undead-buffers.el -- M-S-y — yank-media — paste an image/media object from the clipboard — keybindings.el -- M-S-f — fontaine-set-preset — switch the font preset — font-config.el -- M-S-w — wttrin — show the weather report — weather-config.el -- M-S-e — eww — open the EWW web browser — eww-config.el -- M-S-l — cj/switch-themes — select/cycle the theme — ui-theme.el -- M-S-r — cj/elfeed-open — open the Elfeed RSS reader — elfeed-config.el -- M-S-v — cj/split-and-follow-right — split window right and move focus there — ui-navigation.el -- M-S-h — cj/split-and-follow-below — split window below and move focus there — ui-navigation.el -- M-S-t — toggle-window-split — toggle horizontal/vertical split — ui-navigation.el -- M-S-z — cj/undo-kill-buffer — reopen the most-recently-killed file buffer — ui-navigation.el -- M-S-u — winner-undo — undo the last window-configuration change — ui-navigation.el -- M-S-d — dwim-shell-commands-menu — DWIM shell-command menu on marked files — dwim-shell-config.el -- M-S-i — edit-indirect-region — edit the region in an indirect buffer — text-config.el -- M-S-c — time-zones — show the world-clock / time-zones view — chrono-tools.el -- M-S-b — calibredb — open the Calibre ebook library — calibredb-epub-config.el -- M-S-k — show-kill-ring — browse the kill ring — show-kill-ring.el - -Note: =4a1ecf64= (in-flight, reverted in Phase 0) currently leaves -=eww=/=elfeed=/=calibredb= mis-bound to =M-E=/=M-R=/=M-B=; the table lists the -intended/original =M-S-= bindings. - -** Appendix C — Console-safe single-prefix candidates (D1/D3) - -Craig's first choice (D1) is one =Control=+key prefix that works in GUI, terminal, -and the Linux console, ideally a lightly-used home-row key. Console transmittability -is the gate. Two classes of chord transmit in =TERM=linux=: - -1. =Control= + letter (ASCII 1–26). Several collide with terminal control characters - and so cannot serve as a distinct prefix — =C-i=/TAB, =C-j=/LF, =C-m=/RET, - =C-[=/ESC, =C-h=/DEL — and =C-g= (=keyboard-quit=) is sacred and excluded. -2. =Meta= + key, which the console sends as an *ESC prefix* (=M-x= = ESC then x). - This is why the Problem table above shows =M-E= working as "ESC E" in the console. - So a plain =M-= prefix is console-safe too — and unlike the broken =M-S-= - family, an unshifted =M-= binds directly with no =key-translation-map= in the - path. The catch is finding a free one: the Meta namespace is crowded (Appendix D - shows =M-*=, =M-+=, =M-#=, =M-P=, =M-t=, and the whole =M-g=/=M-s=/=M-e=/=M-r= - consult family taken), so a free Meta prefix would be punctuation (=M-\\=, =M-/=…), - not a letter, and it carries the usual ESC-prefix timing caveat in terminals. - -=Control= + *non-letter* punctuation (=C-;=, =C-'=, =C-.=…) does NOT transmit in the -console — the character isn't a control code. So =C-'= is a non-starter on two counts: -dead in the console like =C-;=, and already bound (=cj/flyspell-then-abbrev=, globally -at =flyspell-and-abbrev.el:253= and in =org-mode-map= at =:258=). =Control=+letter -(the table below) stays the cleanest path; a free =M-= is the viable -runner-up class if Craig prefers Meta. - -| Candidate | Home-row | Console-safe | Default binding | Verdict + note | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-l | yes | yes | recenter-top-bottom | TOP. Home-row, light default, | -| | | | | trivially relocated. Also | -| | | | | vertico-insert in the minibuffer | -| | | | | (selection-framework.el:42) — | -| | | | | minibuffer-local, no global | -| | | | | conflict. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-q | no | yes | quoted-insert | Strong runner-up. Very light | -| | | | | default; trivial rebind. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-o | no | yes | open-line | Strong runner-up. Light default; | -| | | | | easy rebind. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-t | no | yes | transpose-chars | Strong runner-up. Light default; | -| | | | | easy rebind. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-k | yes | yes | kill-line | Possible. Home-row, but kill-to-eol | -| | | | | is muscle memory — medium retrain | -| | | | | friction. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-s | yes | yes | cj/consult-line-or-repeat | Possible, but already a useful | -| | | | (selection-framework.el:265) | rebind; using it as a prefix | -| | | | | reverses that. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-a | yes | yes | move-beginning-of-line | Reject. Essential editing reflex. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-d | yes | yes | delete-char | Reject. Essential. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-f | yes | yes | forward-char | Reject. Essential. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-h | yes | collides (DEL) | help-command | Reject. Console DEL collision; help | -| | | | | is frequent. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-j | yes | collides (LF) | newline | Reject. LF control char; cannot | -| | | | | transmit distinctly. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-g | yes | sacred | keyboard-quit | Reject. Universal escape; never | -| | | | | reuse. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-z | no | yes | suspend-frame (live prefix; C-z F = | Reject. Already an extended prefix. | -| | | | fonts, font-config.el:300) | | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| C-' | no | no | cj/flyspell-then-abbrev | Reject. Punctuation — dead in the | -| | | | (flyspell-and-abbrev.el:253) | console like C-;; and already bound | -| | | | | (also org-mode-map :258). | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| -| M- | n/a | yes (ESC-prefix) | — (Meta namespace crowded; see | Viable runner-up class. Console-safe | -| | | | Appendix D) | via ESC-prefix, no translation | -| | | | | layer, distinct from the broken | -| | | | | M-S-. Needs a free M-punctuation | -| | | | | (M-\\, M-/); ESC-timing caveat in | -| | | | | terminals. | -|-----------+----------+------------------+-------------------------------------+--------------------------------------| - -Recommendation: =C-l= is the single best fit — the only clean home-row option (every -other home-row letter is essential, a collision, sacred, or already repurposed), -console-safe, and its default =recenter-top-bottom= is light and trivially relocated. -=C-q= / =C-o= / =C-t= are equally console-safe and lightly bound if Craig prefers to -keep all home-row defaults; they cost a right-hand reach off home row. If Craig would -rather a Meta prefix, a free =M-= (=M-\\=, =M-/=) is the viable runner-up -class — console-safe via ESC-prefix and free of the translation layer — at the cost of -the ESC-timing caveat. =C-'= is out (console-dead and already taken). Craig picks. - -** Appendix D — Personal keybindings set outside C-; (audit for pruning) - -Every keybinding Craig has set *outside* the =C-;= tree (Appendix A) and the =M-S-= -family (Appendix B), grouped by context. Check a box to mark that binding — or a -whole group — for removal in Phase 1. Boxes start unchecked; Craig marks them. -Inventoried 2026-06-13. Some =:bind (:map …)= package-integration maps (lsp-mode, -c-mode-base, python-ts, json-ts, outline-minor, magit-blame, quick-sdcv, cj/vc-map) -have large package-managed binding lists not enumerated here. - -- [ ] Global bindings - - [ ] C-+ — text-scale-increase — (font-config.el:306) - - [ ] C-= — text-scale-increase — (font-config.el:307) - - [ ] C-_ — text-scale-decrease — (font-config.el:308) - - [ ] C-- — text-scale-decrease — (font-config.el:309) - - [ ] C-x C-f — find-file — (keybindings.el:147) - - [ ] C-x \ — sort-lines — (keybindings.el:160) - - [ ] C-x u — undo-reminder-message — (keybindings.el:164) - - [ ] — keyboard-escape-quit — (keybindings.el:156) - - [ ] — cj/title-case-region — (custom-case.el:124) - - [ ] — cj/kill-buffer-or-bury-alive — (undead-buffers.el:55) - - [ ] — ibuffer — (system-utils.el:147) - - [ ] — cj/disabled — (system-defaults.el:191) - - [ ] C-z — prefix map (suspend-frame replacement) — (keybindings.el:148) - - [ ] C-z F — cj/display-available-fonts — (font-config.el:300) - - [ ] C-h A — cj/local-arch-wiki-search — (help-utils.el:82) - - [ ] C-h D s — devdocs-search — (help-utils.el:40) - - [ ] C-h D b — devdocs-peruse — (help-utils.el:41) - - [ ] C-h D l — devdocs-lookup — (help-utils.el:42) - - [ ] C-h D i — devdocs-install — (help-utils.el:43) - - [ ] C-h D d — devdocs-delete — (help-utils.el:44) - - [ ] C-h D u — devdocs-update-all — (help-utils.el:45) - - [ ] C-h P — list-packages — (help-config.el:31) - - [ ] C-h i — cj/browse-info-files — (help-config.el:90) - - [ ] C-c b — cj/eval-buffer-with-confirmation-or-error-message — (system-utils.el:57) - - [ ] C-c C — cj/org-contacts-map prefix — (org-contacts-config.el:271) - - [ ] C-c d — cj/debug-config-keymap prefix — (config-utilities.el:28) - - [ ] C-c f — cj/flyspell-toggle — (flyspell-and-abbrev.el:252) - - [ ] C-c l — org-store-link — (org-config.el:58) - - [ ] C-c m — mu4e — (mail-config.el:125) - - [ ] C-c M — mouse-trap-mode — (mousetrap-mode.el:275) - - [ ] C-' — cj/flyspell-then-abbrev — (flyspell-and-abbrev.el:253) - - [ ] C-s — cj/consult-line-or-repeat — (selection-framework.el:265) - - [ ] M-* — calculator — (keybindings.el:152) - - [ ] M-+ — balance-windows — (ui-navigation.el:67) - - [ ] M-P — cj/check-for-open-work — (reconcile-open-repos.el:221) - - [ ] C-c n d — org-roam-dailies-map prefix — (org-roam-config.el:94) - - [ ] C-c n I — cj/org-roam-node-insert-immediate — (org-roam-config.el:131) -- [ ] Function keys - - [ ] — cj/dashboard-only — (dashboard-config.el:158) - - [ ] — call-last-kbd-macro — (keyboard-macros.el:131) - - [ ] C- — cj/kbd-macro-start-or-end — (keyboard-macros.el:130) - - [ ] M- — cj/save-maybe-edit-macro — (keyboard-macros.el:132) - - [ ] s- — cj/open-macros-file — (keyboard-macros.el:133) - - [ ] — cj/f4-compile-and-run — (dev-fkeys.el:535) - - [ ] C- — cj/f4-compile-only — (dev-fkeys.el:536) - - [ ] M- — cj/f4-clean-rebuild — (dev-fkeys.el:537) - - [ ] S- — recompile — (dev-fkeys.el:538) - - [ ] — cj/f6-test-runner — (dev-fkeys.el:539) - - [ ] C- — cj/f6-current-file-tests — (dev-fkeys.el:540) - - [ ] S- (Python) — cj/python-mypy — (prog-python.el:103) - - [ ] S- (Shell) — cj/shell-run-shellcheck — (prog-shell.el:98) - - [ ] S- (Go) — cj/go-staticcheck — (prog-go.el:102) - - [ ] S- (C) — cj/disabled — (prog-c.el:158) - - [ ] S- (Python) — cj/python-debug — (prog-python.el:106) - - [ ] S- (Shell) — cj/disabled — (prog-shell.el:101) - - [ ] S- (Go) — cj/go-debug — (prog-go.el:105) - - [ ] S- (C) — gdb — (prog-c.el:161) - - [ ] — cj/coverage-report — (coverage-core.el:537) - - [ ] — cj/main-agenda-display — (org-agenda-config.el:418) - - [ ] C- — cj/todo-list-single-project — (org-agenda-config.el:269) - - [ ] M- — cj/todo-list-from-this-buffer — (org-agenda-config.el:283) - - [ ] s- — cj/todo-list-all-agenda-files — (org-agenda-config.el:244) - - [ ] — cj/ai-term — (ai-term.el:920) - - [ ] C- — cj/ai-term-pick-project — (ai-term.el:921) - - [ ] M- — cj/ai-term-close — (ai-term.el:922) - - [ ] C-S- — cj/ai-term-close — (ai-term.el:923) - - [ ] — cj/music-playlist-toggle — (music-config.el:910) - - [ ] C- — cj/server-shutdown — (system-utils.el:105) - - [ ] — cj/term-toggle — (term-config.el:383) - - [ ] C- — eshell-toggle — (eshell-config.el:161) -- [ ] use-package :bind (global) - - [ ] C-c L — slime — (prog-lisp.el:151) - - [ ] C-c G — geiser-guile — (prog-lisp.el:172) - - [ ] C-h L — leetcode — (prog-training.el:35) - - [ ] C-h M — man — (help-config.el:49) - - [ ] C-h T — tldr — (help-utils.el:53) - - [ ] C-h W — wiki-summary — (help-utils.el:58) - - [ ] C-` — accent-company — (text-config.el:122) - - [ ] C-x M-f — sudo-edit — (system-utils.el:66) - - [ ] C-x g — magit-status — (vc-config.el:34) - - [ ] C-c s i — consult-yasnippet — (selection-framework.el:191) - - [ ] M-# — calendar — (chrono-tools.el:38) - - [ ] M-t — tmr-prefix-map — (chrono-tools.el:110) - - [ ] C-M-p — proced — (system-utils.el:183) -- [ ] Vertico / selection framework - - [ ] C-h C-k — free-keys — (keybindings.el:129) - - [ ] C-j (vertico-map) — vertico-next — (selection-framework.el:40) - - [ ] C-k (vertico-map) — vertico-previous — (selection-framework.el:41) - - [ ] C-l (vertico-map) — vertico-insert — (selection-framework.el:42) - - [ ] RET (vertico-map) — vertico-exit — (selection-framework.el:43) - - [ ] C-RET (vertico-map) — vertico-exit-input — (selection-framework.el:44) - - [ ] M-RET (vertico-map) — minibuffer-force-complete-and-exit — (selection-framework.el:45) - - [ ] TAB (vertico-map) — minibuffer-complete — (selection-framework.el:46) -- [ ] Consult (global) - - [ ] C-c h — consult-history — (selection-framework.el:64) - - [ ] C-x M-: — consult-complex-command — (selection-framework.el:66) - - [ ] C-x b — consult-buffer — (selection-framework.el:67) - - [ ] C-x 4 b — consult-buffer-other-window — (selection-framework.el:68) - - [ ] C-x 5 b — consult-buffer-other-frame — (selection-framework.el:69) - - [ ] C-x r b — consult-bookmark — (selection-framework.el:70) - - [ ] C-x p b — consult-project-buffer — (selection-framework.el:71) - - [ ] M-g e — consult-compile-error — (selection-framework.el:73) - - [ ] M-g f — consult-flymake — (selection-framework.el:74) - - [ ] M-g g — consult-goto-line — (selection-framework.el:75) - - [ ] M-g M-g — consult-goto-line — (selection-framework.el:76) - - [ ] M-g o — consult-outline — (selection-framework.el:77) - - [ ] M-g m — consult-mark — (selection-framework.el:78) - - [ ] M-g k — consult-global-mark — (selection-framework.el:79) - - [ ] M-g i — consult-imenu — (selection-framework.el:80) - - [ ] M-g I — consult-imenu-multi — (selection-framework.el:81) - - [ ] M-s d — consult-find — (selection-framework.el:83) - - [ ] M-s D — consult-locate — (selection-framework.el:84) - - [ ] M-s g — consult-grep — (selection-framework.el:85) - - [ ] M-s G — consult-git-grep — (selection-framework.el:86) - - [ ] M-s r — consult-ripgrep — (selection-framework.el:87) - - [ ] M-s l — consult-line — (selection-framework.el:88) - - [ ] M-s L — consult-line-multi — (selection-framework.el:89) - - [ ] M-s k — consult-keep-lines — (selection-framework.el:90) - - [ ] M-s u — consult-focus-lines — (selection-framework.el:91) - - [ ] M-s e — consult-isearch-history — (selection-framework.el:93) -- [ ] Isearch / minibuffer search - - [ ] M-e (isearch-mode-map) — consult-isearch-history — (selection-framework.el:95) - - [ ] M-s e (isearch-mode-map) — consult-isearch-history — (selection-framework.el:96) - - [ ] M-s l (isearch-mode-map) — consult-line — (selection-framework.el:97) - - [ ] M-s L (isearch-mode-map) — consult-line-multi — (selection-framework.el:98) - - [ ] M-s (minibuffer-local-map) — consult-history — (selection-framework.el:101) - - [ ] M-r (minibuffer-local-map) — consult-history — (selection-framework.el:102) -- [ ] PDF view mode - - [ ] M — pdf-view-midnight-minor-mode — (pdf-config.el:49) - - [ ] m — bookmark-set — (pdf-config.el:50) - - [ ] C-= — pdf-view-enlarge — (pdf-config.el:51) - - [ ] C-- — pdf-view-shrink — (pdf-config.el:52) - - [ ] C-c l — org-store-link — (pdf-config.el:53) - - [ ] z — cj/open-file-with-command zathura — (pdf-config.el:54) - - [ ] j — image-next-line — (pdf-config.el:56) - - [ ] k — image-previous-line — (pdf-config.el:57) - - [ ] — image-next-line — (pdf-config.el:58) - - [ ] — image-previous-line — (pdf-config.el:59) - - [ ] i — cj/org-noter-insert-note-dwim — (pdf-config.el:61) - - [ ] C- — pdf-view-next-page-command + image-bob — (pdf-config.el:63) - - [ ] C- — pdf-view-previous-page-command + image-eob — (pdf-config.el:65) -- [ ] Ediff mode - - [ ] j (ediff-mode-map) — ediff-next-difference — (diff-config.el:54) - - [ ] k (ediff-mode-map) — ediff-previous-difference — (diff-config.el:55) -- [ ] Org / org-related - - [ ] C-' (org-mode-map) — cj/flyspell-then-abbrev — (flyspell-and-abbrev.el:258) - - [ ] S- (org-mouse-map) — cj/org-follow-link-at-mouse-same-window — (org-config.el:338) - - [ ] (org-mouse-map) — cj/org-follow-link-at-mouse-same-window — (org-config.el:339) -- [ ] Dired / dirvish - - [ ] G (dired-mode-map) — cj/deadgrep-here — (prog-general.el:277) - - [ ] M-D (dirvish-mode-map) — dwim-shell-commands-menu — (dwim-shell-config.el:934) - - [ ] + (dirvish-mode-map) — cj/music-add-dired-selection — (music-config.el:597) - - [ ] T (dired/dirvish-mode-map) — cj/transcribe-media-at-point — (transcription-config.el:463/467) - - [ ] (dirvish-mode-map) — dirvish-side — (dirvish-config.el:481) -- [ ] Shell / terminal - - [ ] C-r (eshell-mode-map) — cj/eshell-history-search — (eshell-config.el:202) - - [ ] (eshell-hist-mode-map) — previous-line — (eshell-config.el:99) - - [ ] (eshell-hist-mode-map) — next-line — (eshell-config.el:100) -- [ ] Ghostel terminal - - [ ] (ghostel-mode-map) — cj/ai-term — (ai-term.el:932) - - [ ] C- (ghostel-mode-map) — cj/ai-term-pick-project — (ai-term.el:933) - - [ ] M- (ghostel-mode-map) — cj/ai-term-close — (ai-term.el:934) - - [ ] C-S- (ghostel-mode-map) — cj/ai-term-close — (ai-term.el:935) - - [ ] (ghostel-mode-map) — cj/term-toggle — (term-config.el:415) - - [ ] C-SPC (ghostel-mode-map) — cj/term-send-C-SPC — (term-config.el:416) -- [ ] Version control / magit - - [ ] M-g (git-commit-mode-map) — gptel-magit-generate-message — (ai-config.el:498) - - [ ] N (magit-mode-map) — forge-pull — (vc-config.el:125) -- [ ] Help / docs modes - - [ ] b (devdocs-mode-map) — devdocs-go-back — (help-utils.el:47) - - [ ] f (devdocs-mode-map) — devdocs-go-forward — (help-utils.el:48) -- [ ] Org-roam dailies - - [ ] Y (org-roam-dailies-map) — org-roam-dailies-capture-yesterday — (org-roam-config.el:92) - - [ ] T (org-roam-dailies-map) — org-roam-dailies-capture-tomorrow — (org-roam-config.el:93) -- [ ] Other mode maps - - [ ] C- (slack-message-compose-buffer-mode-map) — slack-message-send-from-buffer — (slack-config.el:297) - - [ ] q (dashboard-mode-map) — nil (unbound) — (dashboard-config.el:223) - - [ ] q (show-kill-ring-mode-map) — show-kill-ring-exit — (show-kill-ring.el:67) - - [ ] (markdown-mode-map) — markdown-preview — (markdown-config.el:24) - - [ ] — dwim-shell-command — (dwim-shell-config.el:204) -- [ ] key-translation-map / input-decode-map - - [ ] input-decode-map ESC [ A — [up] — (keyboard-compat.el:109) - - [ ] input-decode-map ESC [ B — [down] — (keyboard-compat.el:110) - - [ ] input-decode-map ESC [ C — [right] — (keyboard-compat.el:111) - - [ ] input-decode-map ESC [ D — [left] — (keyboard-compat.el:112) - - [ ] input-decode-map ESC O A — [up] — (keyboard-compat.el:115) - - [ ] input-decode-map ESC O B — [down] — (keyboard-compat.el:116) - - [ ] input-decode-map ESC O C — [right] — (keyboard-compat.el:117) - - [ ] input-decode-map ESC O D — [left] — (keyboard-compat.el:118) -- [ ] Jumper - - [ ] jumper-prefix-key (computed at runtime) — jumper-map — (jumper.el:270) [computed key — exact binding depends on the variable value] - -Note: the global =M-S-= family is intentionally excluded (Appendix B). The -arrow-key =input-decode-map= entries are the terminal setup the spec keeps (not the -translation block being retired). =C-l= appears only minibuffer-local in -=vertico-map=, consistent with Appendix C. - -* Review and iteration history -** 2026-06-12 Fri @ 11:21:56 -0500 — Craig Jennings — author -- What: initial draft. Problem, three-context analysis, the 4a1ecf64 regression - as motivating evidence, the one-map/two-prefix design, alternatives, five - open decisions, phased plan, acceptance criteria, readiness dimensions, and the - full C-; tree + M-S- family appendices. -- Why: a touched key family broke in GUI and is dead in console; the fix path is - cross-cutting (18 keys, a translation layer to retire, a console-safety - architecture) with real trade-offs, so it clears the spec bar. -- Artifacts: docs/specs/keybinding-console-safety-spec-doing.org; supersedes the - pre-template draft docs/design/keybinding-console-safety.org. -** 2026-06-12 Fri @ 18:30:30 -0500 — Craig Jennings — review response -- What: processed Craig's four review comments. Recorded his first-choice - direction — one console-safe =Control=+key prefix used everywhere (single-prefix - primary; the two-prefix design is now the documented fallback) — in the Summary, - Design, and D1. Added Appendix C, the console-safe single-prefix candidate table - (standout =C-l=; runners-up =C-q=/=C-o=/=C-t=). Reframed D3 around that pick. - Named the flawed test (=tests/test-launcher-meta-shift-keys.el=) and quoted its - =key-binding=-only assertion in Phase 0. Recorded Craig's decision that the - launcher apps (=eww=/=elfeed=/=calibredb=/=wttrin=) get new keys under a launcher - sub-prefix, not =M-x= (D2/D5, Phases 0/2), with =C-; a t= (=cj/toggle-gptel=, - =ai-config.el:541=) flagged as freeable space. Reserved =C-; L= for Pearl in - Appendix A and D5. -- Why: Craig's review shifted the architecture from two-prefix to a single unified - console-safe prefix and resolved the apps disposition; the spec had to carry the - candidate data he asked for and reflect the choices through the phases. -- Open: the specific prefix (Appendix C), the window and launcher sub-prefix - letters (D5) remain Craig's picks. D1–D5 still State: proposed pending those. -** 2026-06-12 Fri @ 18:43:25 -0500 — Craig Jennings — decisions-as-TODO convention -- What: switched the Decisions section to org TODO tasks. Each decision is =** TODO - Dn=, flips to =DONE= when Craig agrees, stays TODO with a =*** Discussion= child - thread when not. Added a =[0/5]= statistics cookie and a gate: spec Status cannot - reach =ready= while any decision is TODO. Current status: all 5 TODO (none fully - agreed — D1 awaits the prefix lock, D2 unreviewed, D3 awaits the prefix pick, D4 - awaits the A/B pick, D5's apps half agreed but both sub-prefix letters open). -- Why: replaces the inline =State: proposed/accepted= field with an org-native, - agenda-visible task + discussion-thread workflow, and makes the - all-decisions-resolved gate explicit and machine-checkable. -** 2026-06-13 Sat @ 00:18:09 -0500 — Craig Jennings — Path 2 restructure + audit appendix -- What: processed three more review comments. Restructured the phases to Path 2: - Phase 0 is a pure revert that unblocks the held push; Phase 1 prunes forgotten - keybindings (Appendix D); Phase 2 is the consolidation (migrate the common set, - retire the translation block) — the primary deliverable; Phase 3 (bind the - console-safe prefix) is now OPTIONAL and deferred until Craig picks the key. The - Decisions gate split accordingly: D2/D4/D5 gate the primary work, D1/D3 gate only - Phase 3, so the work runs to the prefix-assignment point and stops there. - Corrected Appendix C's premise (Meta transmits in the console as an ESC prefix, so - =M-= is a viable console-safe class); added the =C-'= row (rejected — - console-dead and already bound to flyspell) and the =M-= row. Added Appendix - D: every personal keybinding set outside the =C-;= tree and the =M-S-= family, as a - checkbox pruning tree (~190 bindings, inventoried by a read-only sweep). -- Why: Craig pivoted to landing the consolidation first and treating the - console-safe prefix as a later switch-on, and wanted a one-time audit of his - set-and-forgotten keybindings while the keymap work was open. -- Open: D1–D5 still TODO; the prefix (D1/D3) is now non-blocking. Phase 0 revert - pending so the push can proceed. diff --git a/docs/specs/keybinding-console-safety-spec.org b/docs/specs/keybinding-console-safety-spec.org new file mode 100644 index 00000000..c7d1baf7 --- /dev/null +++ b/docs/specs/keybinding-console-safety-spec.org @@ -0,0 +1,947 @@ +#+TITLE: Keymap Consolidation — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-12 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Keymap Consolidation — Spec +:PROPERTIES: +:ID: 540bf06b-16b8-46c6-b459-c40d1b9c795d +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DOING from existing :STATUS: doing + +* Metadata +| Status | doing | +|----------+--------------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+--------------------------------------------------------------------| +| Reviewer | TBD (multi-reviewer cycle) | +|----------+--------------------------------------------------------------------| +| Related | [[file:../../todo.org][todo.org]] — "M-S- launcher keys" task (to be reclassified, Phase 0) | +|----------+--------------------------------------------------------------------| + +* Summary + +Some commonly-used window/layout commands are bound to =M-S-= chords that only work in GUI frames, via a fragile =key-translation-map= layer that already caused a regression. + +The primary work consolidates the common commands under the =cj/custom-keymap= personal keymap and retires the fragile translation layer — independent of any prefix choice. Console reachability is then a one-line, *optional* follow-on: bind that one keymap to a single console-safe prefix (a =Control=+key, or a free =M-=; candidates in Appendix C), used everywhere. Per Path 2 (2026-06-13), the work proceeds up to the point of assigning that prefix and stops there; the assignment is a deferred phase Craig takes when he picks the key. + +The aim: consolidate the common commands into one keymap and retire the translation block now, leaving a single, optional console-safe prefix to switch on later. + +* Problem / Context + +A subset of common commands is bound to =M-S-= chords (Meta + Shift + lowercase letter). Pressing Meta+Shift+e emits the event =M-E= (uppercase Meta), but the command is bound to =M-S-e=; the bridge between them is a =key-translation-map= entry that =modules/keyboard-compat.el= installs *only* in GUI frames (=env-gui-p=). So these chords are dead in terminal frames and dead in the Linux console. + +Craig does not use terminal or console Emacs often, but falls back to the console in emergencies (a broken graphical session). When common keys are unavailable there, the editor stops being usable for the emergency and he has to switch tools. For *uncommon* commands, =M-x= is an acceptable fallback; for *common* ones it is not. + +How each key family actually behaves across the three contexts (the facts the design turns on): + +| Context | Meta sent as | =M-S-e= (as bound) | =M-E= (uppercase Meta) | =C-;= | +|-------------------------+--------------+-------------------------+-------------------------+-------------------------| +| GUI frame | native event | reached only via the | intercepted by the | works natively | +| | | GUI translation map | translation map | | +|-------------------------+--------------+-------------------------+-------------------------+-------------------------| +| Terminal emulator | ESC prefix | dead (keypress emits | works (ESC E), if no | works if the emulator | +| (xterm-family) | | =M-E=, binding is on | translation intercepts | speaks | +| | | =M-S-e=) | | modifyOtherKeys/kitty | +| | | | | (recent Emacs | +| | | | | auto-enables for | +| | | | | xterm-family) | +|-------------------------+--------------+-------------------------+-------------------------+-------------------------| +| Linux console | ESC prefix | dead (same reason) | works (ESC E) | DEAD — semicolon is not | +| (TERM=linux) | | | | a control char; cannot | +| | | | | be transmitted | +|-------------------------+--------------+-------------------------+-------------------------+-------------------------| + +Three consequences: =M-S-e= is dead outside GUI by construction; =C-;= is solid in GUI, conditional in terminal emulators, and dead in the Linux console (so it cannot be the *only* home for console-critical commands); and =M-E= plus function keys and =C-c= sequences are transmittable everywhere, which is the material to build a console-safe path from. + +** The regression that triggered this + +Commit =4a1ecf64= "fixed" three launcher keys (=eww=/=elfeed=/=calibredb=) by rebinding them from =M-S-e/r/b= to =M-E/M-R/M-B=. It was wrong, and three review passes missed it because they all used =key-binding=, which consults keymaps only and ignores =key-translation-map=. The original audit "verified dead in the live daemon" with that blind check (false positive); the fix bound =M-E= but left the =M-E -> M-S-e= translation entry in place, so in GUI the keypress is rewritten to the now-unbound =M-S-e= and the launchers break on the next restart; and the new test asserted =(key-binding (kbd "M-E"))=, passing against a configuration broken at the keyboard. It only appears to work in the running daemon because the pre-fix binding is still loaded as stale state — the stale-daemon trap. + +The lesson is encoded into the acceptance criteria: real reachability is not =key-binding= when a translation map participates. + +* Goals and Non-Goals + +** Goals +- Every *commonly used* command is reachable in GUI, terminal emulators, and the Linux console. +- One canonical personal command surface, so console-reachability is solved once at the prefix level rather than per command. +- Retire the =keyboard-compat.el= =M-uppercase -> M-S-lowercase= translation block, the root of the fragility. +- Keep daily ergonomics: high-frequency commands keep a fast chord in GUI. + +** Non-Goals +- Making *every* binding console-safe. Uncommon commands may live on =M-x= only. +- A ground-up keymap redesign. This is about reachability and retiring one fragile mechanism. +- Defeating the Linux virtual console's hard limits (it cannot transmit =C-;=, and Meta+Shift behaviour varies). The design routes around them. + +** Scope tiers +- *v1 (primary — Phases 0–2):* + - revert =4a1ecf64= (Phase 0, unblocks the push); + - prune the forgotten keybindings Craig marks in Appendix D; + - migrate the common window/layout =M-S-= commands into =cj/custom-keymap=; + - drop the uncommon chords to =M-x=; + - retire the translation block; + - translation-aware tests. +- *Deferred / optional (Phase 3):* + - bind =cj/custom-keymap= to a single console-safe prefix (D1/D3) once Craig picks the key — the console-reachability payoff, switched on later. +- *Out of scope:* + - enabling modifyOtherKeys/kitty for terminal emulators (helps terminals, not the console; orthogonal). +- *vNext:* auditing non-=M-S-= GUI-only chords (modified function keys, =C-+/C-==) for console behaviour. + +* Design + +Craig's first choice (review, 2026-06-12): instead of two prefixes, retrain muscle memory onto *one* console-safe prefix that works everywhere — a =Control= + home-row key that is lightly used or easily/intuitively rebound. =cj/custom-keymap= moves from =C-;= to that single prefix (=C-;= may stay bound during the transition since one keymap can carry many prefixes). The candidate list is Appendix C; the standout home-row candidate is =C-l=. The two-prefix design below is the documented fallback if no single prefix proves acceptable. + +The personal command surface is already a single keymap object, =cj/custom-keymap= (=modules/keybindings.el=), bound to =C-;=. The whole design rests on one Emacs fact: a keymap is an object and can be bound to more than one prefix. So console-reachability is a *prefix* problem, not a per-command problem. + +For a user (single-prefix path): you reach your personal commands with the one console-safe prefix in GUI, terminal, and console alike — same menu, same keys after the prefix, nothing per-context to remember. (Fallback two-prefix path: =C-;= in the GUI as today, plus a second console-safe prefix anywhere.) + +For the implementer: add one line — =(keymap-global-set "" cj/custom-keymap)= — and the entire tree in Appendix A becomes reachable through it; nothing per-command. The work then is to move the *common* console-dead commands (the window/layout =M-S-= subset, Appendix B) *into* =cj/custom-keymap= so they inherit that reachability, drop the *uncommon* =M-S-= chords to =M-x=, and delete the now-unused translation block. High-frequency window commands additionally keep a fast chord so daily GUI use doesn't regress to a 3-key sequence (Decision D4). + +The console-dead common set is window/layout work, which has no =C-;= sub-prefix today, so v1 adds one (a new window sub-map; letter is Decision D5). The =C-c=/=C-h=/=C-z=/=C-x= and plain function-key bindings already work in the console and stay where they are. + +* Alternatives Considered + +** A — Revert 4a1ecf64 and keep the translation layer as the end state +- Good, because it is the smallest change and restores correctness immediately. +- Bad, because it keeps 18 keys on the GUI-only mechanism that already bit us and + leaves the console-dead problem unsolved. +- Neutral, because the revert itself is still needed as Phase 0; this option just + stops there. + +** B — Migrate the whole family to direct uppercase-Meta, delete the translation block, no C-; move +- Good, because it preserves every single-chord and =M-E= (ESC + uppercase) is + transmittable in GUI, terminal, and console alike. +- Bad, because it bets the emergency-console guarantee on Meta+Shift behaving + cleanly on every console keyboard, which is probable but not certain, and it + gives the common commands no robust prefix-based fallback. +- Neutral, because it still deletes the translation block (shared with the chosen + design) and could be layered onto the frequent-chord subset (see D4 Option B). + +** C — Enable an enhanced keyboard protocol (modifyOtherKeys / kitty) so C-; works in terminals +- Good, because it makes =C-;= itself work in capable terminal emulators. +- Bad, because it does nothing for the Linux virtual console (a hard limit), and + adds a terminal-capability dependency. +- Neutral, because it is orthogonal and could be added later without conflicting. + +** Chosen — one map, two prefixes (consolidate common commands under C-;, add a console-safe alt prefix) +- Good, because console-reachability is solved once at the prefix; it depends on + exactly one prefix working, and that prefix is chosen to be bulletproof. +- Bad, because moved commands cost a muscle-memory transition, and a pure + sub-prefix path is 3 keys (mitigated by D4 for the frequent ones). +- Neutral, because it still requires the revert (Phase 0) and the translation- + block deletion (shared with B). + +* Decisions [3/5] + +Each decision is a TODO task. It flips to DONE when Craig agrees with the call; if +he doesn't, it stays TODO and the discussion continues under its =*** Discussion= +child header. + +*Gate (Path 2).* The decisions split by which work they block. D2, D4, and D5 gate +the *primary* work (Phases 0–2: revert, prune, consolidate, retire the translation +block); the spec is implementation-ready for that work once those three are DONE. D1 +and D3 (the console-safe prefix) gate *only* the optional Phase 3 — they can stay +TODO indefinitely without blocking the consolidation. So yes: the work proceeds all +the way to the point of assigning the prefix and stops there, exactly as Craig asked, +even if D1/D3 are never decided. The =[n/5]= cookie tracks the overall tally; full +=ready= (including Phase 3) still needs all five. + +** DONE D1 — One map, one console-safe prefix (single-prefix primary; two-prefix fallback) +CLOSED: [2026-06-13 Sat 00:20] +- Owner / by-when: Craig / review cycle +- Context: the common console-dead commands need to be reachable in the console; + =C-;= alone is dead there; per-command console bindings would not scale. +- Decision (revised): We keep =cj/custom-keymap= as the single personal surface. + *Primary (Craig's first choice):* rebind it to ONE console-safe prefix — a + =Control= + lightly-used home-row key (Appendix C; standout =C-l=) — used in GUI, + terminal, and console alike, retraining muscle memory off =C-;=. =C-;= may stay + bound during the transition. *Fallback:* if no single prefix is acceptable, bind + the map to both =C-;= (GUI) and one console-safe alternate prefix (D3). +- Consequences: easier — one prefix to make console-safe, whole tree travels, and + the single-prefix path needs no per-context mnemonic; harder — every + console-critical command must actually live under =cj/custom-keymap= (so the + common =M-S-= set is still migrated in), and the single-prefix path costs a + full =C-;= → new-prefix muscle-memory transition. +*** Discussion +- Direction agreed by Craig 2026-06-12: single-prefix primary, two-prefix fallback. +- Deferred by Craig 2026-06-13 (Path 2): the console-safe prefix becomes the optional + Phase 3, not part of the primary work. The consolidation (Phases 0–2) lands without + it, so D1 no longer blocks anything until Craig chooses to do Phase 3. It stays TODO + as the marker for "decide the prefix later." The phases are rewritten accordingly, + and the keybinding audit Craig asked for lives in Appendix D. + +** DONE D2 — Migrate only the common (window/layout) M-S- set; drop the uncommon to M-x +CLOSED: [2026-06-13 Sat 00:22] +- Owner / by-when: Craig / review cycle +- Context: of the 18 =M-S-= commands, only window/layout control is plausibly + needed in an emergency console session; apps and one-off tools are not. +- Decision: We will move the window/layout subset (=M-S-o/m/v/h/t/u/z=, and + =M-S-k= pending review) into =C-;=, and remove the other ten =M-S-= chords, + leaving those commands on =M-x=. +- Consequences: easier — shrinks the translation block to nothing, focuses the + console surface on essentials; harder — the dropped commands lose a chord; + =show-kill-ring='s classification is a judgment call. +*** Discussion +- Not yet reviewed by Craig. Open: confirm the window/layout subset to migrate + (incl. =M-S-k= show-kill-ring's common/uncommon call) and that the other ten + drop to =M-x=. Flip to DONE on Craig's sign-off. + + +** TODO D3 — The console-safe prefix (pick from Appendix C) +- Owner / by-when: Craig / review cycle +- Context: under D1's single-prefix primary, this prefix is THE personal-keymap + prefix; under the two-prefix fallback it is the second (alternate) binding. It + must transmit in the Linux console, where only =Control= + letter chords carry + (and TAB/RET/LF/ESC/DEL collisions and =C-g= are excluded). Full candidate + analysis is Appendix C. +- Decision: For the single-prefix path, =C-l= is the standout (home-row, + console-safe, default =recenter-top-bottom= is light and trivially relocatable); + =C-q= / =C-o= / =C-t= are off-home-row runners-up. For the two-prefix fallback, + =C-c ;= (=C-c= transmits; =;= is a plain key; mnemonic mirror of =C-;=) stays the + recommendation. Craig picks the prefix. +- Consequences: easier — solves console reachability for the whole tree at one + binding; harder — a single =Control=+letter prefix displaces its default command + (relocate =recenter-top-bottom= if =C-l=), and =C-l= is also bound to + =vertico-insert= inside the minibuffer (=selection-framework.el:42=) — minibuffer- + local, so no conflict with a global prefix, but worth noting. +*** Discussion +- Open: Craig picks the prefix. Recommendation =C-l= (only clean home-row option); + runners-up =C-q= / =C-o= / =C-t=. Flip to DONE on the pick. D1 closes with it. + + +#+begin_src cj: comment +it's not going to be C-l. That's too hard of a habit for me to kick right now. I'd rather go C-c ; altogether -- even in GUI -- than have C-l do the wrong thing when I hit it. We'll find something. But it's not decided yet. Change the status of this decision to waiting. +#+end_src + +** DONE D4 — Fast-chord strategy for high-frequency window ops +CLOSED: [2026-06-13 Sat 00:25] +- Owner / by-when: Craig / review cycle +- Context: =split-and-follow-right/below= and =undo-kill-buffer= are pressed + constantly; a 3-key =C-; v= sequence is a real downgrade. +- Decision: We will (Option A) keep a fast GUI chord for the frequent commands in + addition to their =C-;= entry, OR (Option B) bind them to direct uppercase-Meta + single chords and retire the translation block. Review picks. +- Consequences: A — preserves speed, but the fast chord may itself be GUI-only + unless it is a function key; B — single chord works in all three contexts but + leans on console Meta+Shift. +*** Discussion +- Open: Craig picks Option A (keep fast GUI chord) vs Option B (direct + uppercase-Meta single chords). Note: if D3 lands a single console-safe prefix, + Option B's console rationale weakens. Flip to DONE on the pick. + +#+begin_src cj: comment +we can simply revert +#+end_src + +** TODO D5 — Window sub-prefix and apps disposition +- Owner / by-when: Craig / review cycle +- Context: window/layout has no =C-;= sub-prefix. Free single lowercase letters are + =i q u y z= (=g= is calendar, =h= is Hugo — both taken); most uppercase is free. + =C-; L= is reserved for the Pearl/Linear package — do NOT reuse it. The four apps + (=eww=/=elfeed=/=calibredb=/=wttrin=) could go to =M-x= or a launcher sub-prefix. + + #+begin_src cj: comment + add a listing of the keybindings we're discussing. I don't know what the window/layout keybindings you're discussing. It's not shift arrow keys, is it? + #+end_src +- Decision: We will add a window sub-prefix under =C-;= (letter TBD from the free + set). Apps: Craig decided the launcher commands get real keys under a launcher + sub-prefix (=e/f/b/w= leaves), NOT =M-x=. Sub-prefix letter TBD; the freed + =C-; a t= (ai-assistant toggle, see Phase 0) is one candidate location if the + apps belong nearer the AI tools. Both sub-prefix letters are Craig's pick. +- Consequences: easier — groups window ops and launcher apps discoverably under + which-key, and the launcher apps inherit console reachability for free; harder — + spends two scarce top-level =C-;= letters from the small free set. +*** Discussion +- Apps half agreed by Craig 2026-06-12: launcher sub-prefix, not =M-x=. Open: the + window sub-prefix letter and the launcher sub-prefix letter, both from the free + set {=i q u y z=} + uppercase (NOT =L= — Pearl). Flip to DONE once both letters + are chosen. + +* Implementation phases + +Path 2 (Craig, 2026-06-13): Phase 0 is a *pure revert* that unblocks the held push; the migration follows, and the console-safe prefix is an *optional, deferred* phase. Everything proceeds up to the point of assigning the prefix (end of Phase 2) and stops there; Phase 3 is the optional assignment once Craig picks the prefix. So the consolidation does not wait on the prefix decision (D1/D3); only Phase 3 does. + +** Phase 0 — Revert the regression (unblocks the push) +Revert =4a1ecf64= and nothing more: restore =M-S-e/r/b= in the three modules and delete the flawed test (=tests/test-launcher-meta-shift-keys.el=), leaving a clean, correct baseline. Reclassify the "M-S- launcher keys" task as not-a-bug — the keys worked via the GUI translation layer. This is the only step the held 12-commit stack needs before it can push. Per Path 2, the launchers get reverted to =M-S-= here and move to their new homes in Phase 2 — the accepted small throwaway (3 bindings) of not waiting on the full move-map. + +The flawed test asserts the launcher bindings with =key-binding= alone: + +#+begin_src emacs-lisp +(should (eq (key-binding (kbd "M-E")) 'eww)) +(should (eq (key-binding (kbd "M-R")) 'cj/elfeed-open)) +(should (eq (key-binding (kbd "M-B")) 'calibredb)) +#+end_src + +=key-binding= consults keymaps only and ignores =key-translation-map=, so the test passes even though the GUI translation entry =M-E -> M-S-e= rewrites the keypress back to the now-unbound =M-S-e=. It cannot see the rewrite, so it certifies a configuration that is broken at the keyboard. Phase 2's translation-aware assertion replaces it. + +** Phase 1 — Audit and prune forgotten keybindings (Appendix D) +Appendix D inventories every keybinding Craig has set outside the =C-;= tree and the =M-S-= family — the place to catch chords set-and-forgotten. Craig checks the boxes for the bindings to retire; remove those. Independent cleanup, and a good moment to clear cruft before the migration. Tree working. + +** Phase 2 — Consolidate: migrate the common set, retire the translation block +The primary deliverable, needing *no* console-safe-prefix decision. Migrate the window/layout =M-S-= subset into =cj/custom-keymap= under a new window sub-prefix (D5); add the launcher sub-prefix (D5) with the =eww=/=elfeed=/=calibredb=/=wttrin= leaves (freeing =C-; a t= — the =cj/toggle-gptel= ai-assistant toggle, =ai-config.el:541=, unfinished and far less used than the =ai-term= F9 launcher — if the letter is tight); apply the fast-chord strategy (D4); drop the ten uncommon =M-S-= chords to =M-x= (D2); delete =keyboard-compat.el='s translation block and its hook (keep the arrow-key =input-decode-map= setup); add the translation-aware tests (see Acceptance criteria) and update the docs. At the end of Phase 2 the work is "done" per Craig's stop point. Tree working. + +** Phase 3 — (OPTIONAL, deferred) Bind the console-safe prefix +Only once Craig picks the prefix (D1/D3, Appendix C). Bind =cj/custom-keymap= to it — =(keymap-global-set "" cj/custom-keymap)= — and if the pick is =C-l=, relocate its default =recenter-top-bottom= first. This is the console-reachability payoff: the whole tree becomes reachable in =emacs -nw= and the Linux console through one prefix. Verify in a *fresh* session, not the live daemon. May be deferred indefinitely; the consolidation stands on its own without it. + +* Acceptance criteria +- [ ] The whole =cj/custom-keymap= tree is reachable in a GUI frame, an =emacs -nw= xterm-family terminal, and the Linux virtual console via the alt prefix. +- [ ] The final "common" commands are reachable in all three contexts. +- [ ] =keyboard-compat.el='s translation block is gone; no command depends on it. +- [ ] For any chord claimed to run command X, tests assert BOTH =(key-binding (kbd CHORD))= AND =(lookup-key key-translation-map (kbd CHORD))= are consistent (the latter =nil=, or pointing where intended). =key-binding= alone is insufficient — it is what let =4a1ecf64= through. +- [ ] Reachability is verified in a *fresh* frame/session, not the live daemon (the stale-daemon trap masks results). +- [ ] =make test= fully green (the 4 pre-existing =test-dupre-theme= failures are tracked separately and out of scope). + +* Readiness dimensions +- Data model & ownership: keybindings are user-authored code in =modules/=; + =cj/custom-keymap= is the owned surface. Nothing generated/cached/remote; + nothing persists. +- Errors, empty states & failure: N/A — a missing command symbol surfaces as a + load-time =void-function=, caught by byte-compile and the launch smoke test. +- Security & privacy: N/A — no credentials or sensitive data. +- Observability: which-key shows each prefix's menu; =C-h k= / =describe-bindings= + report the live binding; the translation-aware test reports reachability. +- Performance & scale: N/A — keymap lookup is constant-time; one extra prefix + binding has no measurable cost. +- Reuse & lost opportunities: reuse Emacs's native multi-prefix keymap binding + (one keymap object, two prefix keys) instead of duplicating bindings; reuse + which-key and the existing =cj/register-prefix-map= / =cj/register-command= + helpers. Deletes (does not wrap) the bespoke translation layer. +- Architecture fit & weak points: integration points are =keybindings.el= + (=cj/custom-keymap=, the register helpers), =keyboard-compat.el= (translation + block to delete; arrow-key decode to keep), and the per-module =:bind= / + register calls for the migrated commands. Weak point: the stale-daemon trap can + mask whether a change actually works — mitigated by verifying in a fresh + =-nw=/console session (acceptance criterion). +- Config surface: the console-safe alt prefix (D3) and the window sub-prefix + letter (D5) are the only new knobs; both are constants set once in config. +- Documentation plan: update the =keyboard-compat.el= header (it documents the + retired translation table); note the moved/dropped keys wherever keybindings + are documented. No user-facing migration doc beyond that. +- Dev tooling: existing =make test= / byte-compile / launch smoke cover it; the + new translation-aware assertion is an ERT test like the others. +- Rollout, compatibility & rollback: user-facing keybinding change; rollback is + =git revert=. No persisted data, no public API, no external state. The only + compatibility cost is Craig's muscle memory for the moved/dropped keys — + a transition note, not a migration. +- External APIs & deps: N/A — no external APIs; no new dependencies. + +* Risks, Rabbit Holes, and Drawbacks +- *Muscle-memory disruption* for moved/dropped keys. Dodge: keep fast chords for the highest-frequency commands (D4); accept =M-x= only for genuinely uncommon ones. +- *Console Meta+Shift uncertainty* if D4 Option B is chosen. Dodge: the prefix path (D1/D3) does not depend on it, so the emergency guarantee holds regardless of the fast-chord choice. +- *Stale-daemon trap* masking test results — the exact failure mode behind the regression. Dodge: the acceptance criteria mandate verification in a fresh frame/session and a translation-aware assertion. + +* References / Appendix + +** Appendix A — Full C-; keybinding tree (live, 2026-06-12) + +Dumped from the running daemon by walking =cj/custom-keymap= recursively. +Format: chord — command — what it does. + +*** Top-level leaves (directly on C-;) +- C-; ) — cj/jump-to-matching-paren — jump to the matching paren +- C-; / — cj/replace-fraction-glyphs — replace 1/2-style fractions with glyphs +- C-; ? — cj/flycheck-list-errors — list flycheck errors for the buffer +- C-; A — align-regexp — align region by a regexp +- C-; B — cj/choose-browser — pick the default browser +- C-; f — cj/format-region-or-buffer — format region or whole buffer +- C-; k — cj/org-babel-toggle-confirm — toggle the org-babel eval confirmation +- C-; P — cj/projectile-reset-cmds — reset projectile's cached project commands +- C-; SPC — cj/switch-to-previous-buffer — toggle to the previous buffer +- C-; T — cj/telega — open Telegram (telega) +- C-; | — display-fill-column-indicator-mode — toggle the fill-column rule +- C-; # c — cj/count-characters-buffer-or-region — count characters +- C-; # w — cj/count-words-buffer-or-region — count words + +*** C-; ! — System commands +- C-; ! ! — cj/system-command-menu — the system-command transient menu +- C-; ! e — cj/system-cmd-restart-emacs — restart Emacs +- C-; ! E — cj/system-cmd-exit-emacs — exit Emacs +- C-; ! l — cj/system-cmd-lock — lock the screen +- C-; ! L — cj/system-cmd-logout — log out of the session +- C-; ! r — cj/system-cmd-reboot — reboot +- C-; ! s — cj/system-cmd-shutdown — shut down +- C-; ! S — cj/system-cmd-suspend — suspend + +*** C-; a — AI / gptel +- C-; a . — cj/gptel-add-this-buffer — add current buffer to the gptel context +- C-; a A — cj/gptel-autosave-toggle — toggle conversation autosave +- C-; a b — cj/gptel-browse-conversations — browse saved conversations +- C-; a B — cj/gptel-switch-backend — switch the LLM backend +- C-; a c — cj/gptel-context-clear — clear the gptel context +- C-; a d — cj/gptel-delete-conversation — delete a saved conversation +- C-; a f — cj/gptel-add-file — add a file to the context +- C-; a l — cj/gptel-load-conversation — load a saved conversation +- C-; a m — cj/gptel-change-model — change the model +- C-; a M — gptel-menu — the gptel transient menu +- C-; a p — gptel-system-prompt — edit the system prompt +- C-; a q — cj/gptel-quick-ask — quick one-off ask +- C-; a r — cj/gptel-rewrite-with-directive — rewrite region with a directive +- C-; a R — cj/gptel-rewrite-redo-with-different-directive — redo rewrite, new directive +- C-; a s — cj/gptel-save-conversation — save the conversation +- C-; a t — cj/toggle-gptel — toggle the gptel chat buffer +- C-; a x — cj/gptel-clear-buffer — clear the chat buffer + +*** C-; b — Buffer & file operations +- C-; b — cj/window-resize-sticky — sticky window resize (arrow keys) +- C-; b b — cj/clear-to-bottom-of-buffer — clear from point to end +- C-; b c b — cj/copy-to-bottom-of-buffer — copy point-to-end +- C-; b c t — cj/copy-to-top-of-buffer — copy point-to-start +- C-; b c w — cj/copy-whole-buffer — copy the whole buffer +- C-; b d — cj/delete-buffer-and-file — delete the buffer and its file +- C-; b D — cj/diff-buffer-with-file — diff buffer against its file on disk +- C-; b e — eval-buffer — eval the buffer +- C-; b E — cj/view-email-in-buffer — view the buffer as email +- C-; b g — revert-buffer — revert from disk +- C-; b k — cj/kill-buffer-and-window — kill buffer and close its window +- C-; b K — cj/kill-other-window-buffer — kill the other window's buffer +- C-; b l — cj/copy-link-to-buffer-file — copy an org link to the file +- C-; b m — cj/move-buffer-and-file — move/rename buffer + file +- C-; b n — cj/copy-buffer-name — copy the buffer name +- C-; b o — cj/xdg-open — open the file with the system handler +- C-; b O — cj/open-this-file-with — open with a chosen program +- C-; b p — cj/copy-buffer-source-as-kill — copy buffer source +- C-; b P — cj/print-buffer-ps — print the buffer (PostScript) +- C-; b r — cj/rename-buffer-and-file — rename buffer + file +- C-; b s — mark-whole-buffer — select all +- C-; b S — write-file — write/save-as +- C-; b t — cj/clear-to-top-of-buffer — clear from start to point +- C-; b w — cj/view-buffer-in-eww — render the buffer in EWW +- C-; b x — erase-buffer — erase the buffer + +*** C-; c — Case +- C-; c l — cj/downcase-dwim — downcase (dwim) +- C-; c t — cj/title-case-region — title-case the region +- C-; c u — cj/upcase-dwim — upcase (dwim) + +*** C-; C — Comments +- C-; C - — cj/comment-hyphen — hyphen divider comment +- C-; C b — cj/comment-box — boxed comment +- C-; C c — cj/comment-inline-border — inline bordered comment +- C-; C d — cj/delete-buffer-comments — delete all comments in the buffer +- C-; C h — cj/comment-heavy-box — heavy box comment +- C-; C n — cj/comment-block-banner — block banner comment +- C-; C p — cj/comment-padded-divider — padded divider comment +- C-; C r — cj/comment-reformat — reformat a comment +- C-; C s — cj/comment-simple-divider — simple divider comment +- C-; C u — cj/comment-unicode-box — unicode box comment + +*** C-; d — Date / time insertion +- C-; d d — cj/insert-sortable-date — insert YYYY-MM-DD +- C-; d D — cj/insert-readable-date — insert a human-readable date +- C-; d r — cj/insert-readable-date-time — readable date + time +- C-; d s — cj/insert-sortable-date-time — sortable date + time +- C-; d t — cj/insert-sortable-time — sortable time +- C-; d T — cj/insert-readable-time — readable time + +*** C-; D — Org-drill (flashcards) +- C-; D c — cj/drill-capture — capture a drill question +- C-; D e — cj/drill-edit — open a drill file to edit +- C-; D f — cj/drill-this-file — drill the current file +- C-; D r — cj/drill-refile — refile into a drill file +- C-; D R — org-drill-resume — resume a drill session +- C-; D s — cj/drill-start — start a drill session + +*** C-; e — Email (mu4e) +- C-; e s — cj/mu4e-save-attachment-here — save attachment to current dir +- C-; e S — cj/mu4e-save-all-attachments — save all attachments +- C-; e m — cj/mu4e-save-some-attachments — save selected attachments +- C-; e {c,d,g} {i,l,s,u} — mu4e maildir searches: account {c=cmail, d=dmail, + g=gmail} x view {i=inbox, l=large >5M, s=starred/flagged, u=unread} + +*** C-; E — ERC (IRC) +- C-; E b — cj/erc-switch-to-buffer-with-completion — switch ERC buffer +- C-; E c — cj/erc-join-channel-with-completion — join a channel +- C-; E C — cj/erc-connect-server-with-completion — connect to a server +- C-; E l — cj/erc-connected-servers — list connected servers +- C-; E q — erc-part-from-channel — leave a channel +- C-; E Q — erc-quit-server — quit a server + +*** C-; g — Calendar sync (Google Calendar) +- C-; g s — calendar-sync-now — sync now +- C-; g S — calendar-sync-start — start auto-sync +- C-; g x — calendar-sync-stop — stop auto-sync +- C-; g t — calendar-sync-toggle — toggle auto-sync +- C-; g i — calendar-sync-status — sync status + +*** C-; h — Hugo (website/blog) +- C-; h n — cj/hugo-new-post — new post +- C-; h d — cj/hugo-open-draft — open a draft +- C-; h D — cj/hugo-toggle-draft — toggle a post's draft flag +- C-; h e — cj/hugo-export-post — export a post +- C-; h p — cj/hugo-preview — preview the site +- C-; h P — cj/hugo-publish — publish the site +- C-; h o — cj/hugo-open-blog-dir — open the blog dir in Emacs +- C-; h O — cj/hugo-open-blog-dir-external — open the blog dir externally + +*** C-; j — Jump to files +- C-; j c — cj/jump-to-contacts ; C-; j g — cj/jump-to-gcal +- C-; j i — cj/jump-to-inbox ; C-; j I — cj/jump-to-emacs-init +- C-; j m — cj/jump-to-macros ; C-; j n — cj/jump-to-reading-notes +- C-; j r — cj/jump-to-reference ; C-; j s — cj/jump-to-schedule +- C-; j w — cj/jump-to-webclipped + +*** C-; L — Pearl (Linear tickets) [RESERVED — do not reuse] +- C-; L … — pearl-prefix-map — Pearl/Linear ticket commands (lazy sub-map) +- =C-; L= is reserved as the Pearl (Linear integration) leader key. Sub-prefix + letter picks (D5) must avoid it. + +*** C-; l — Line & paragraph +- C-; l c — duplicate line/region (comment variant) ; C-; l d — cj/duplicate-line-or-region +- C-; l j — cj/join-line-or-region ; C-; l J — cj/join-paragraph +- C-; l r — cj/remove-lines-containing ; C-; l R — cj/remove-duplicate-lines-region-or-buffer +- C-; l u — cj/underscore-line + +*** C-; m — Music (EMMS) +- C-; m m — cj/music-playlist-toggle ; C-; m M — cj/music-playlist-show +- C-; m SPC — emms-pause ; C-; m s — emms-stop +- C-; m n — cj/music-next ; C-; m p — cj/music-previous +- C-; m a — cj/music-fuzzy-select-and-add ; C-; m g — emms-playlist-mode-go +- C-; m r — emms-toggle-repeat-playlist ; C-; m t — emms-toggle-repeat-track +- C-; m x — cj/music-toggle-consume ; C-; m z — emms-toggle-random-playlist +- C-; m Z — emms-shuffle ; C-; m R — cj/music-create-radio-station + +*** C-; M — Signal (signel) +- C-; M m — cj/signel-message — message a contact +- C-; M s — cj/signel-message-self — note to self +- C-; M SPC — cj/signel-connect — start/connect the daemon +- C-; M d — signel-dashboard — the Signal dashboard +- C-; M q — signel-stop — stop the daemon + +*** C-; n — Org-noter +- C-; n t — cj/org-noter-start — start noter on the document +- C-; n n — cj/org-noter-insert-note-dwim — insert a note (dwim) + +*** C-; o — Ordering / text transforms +- C-; o a — cj/arrayify ; C-; o j — cj/arrayify-json ; C-; o p — cj/arrayify-python +- C-; o u — cj/unarrayify ; C-; o l — cj/listify ; C-; o L — cj/comma-separated-text-to-lines +- C-; o A — cj/alphabetize-region ; C-; o r — cj/reverse-lines ; C-; o n — cj/number-lines +- C-; o q — cj/toggle-quotes ; C-; o o — cj/org-sort-by-todo-and-priority + +*** C-; p — reveal.js presentations +- C-; p n — cj/reveal-new ; C-; p h — cj/reveal-insert-header ; C-; p H — cj/reveal-remove-headers +- C-; p e — cj/reveal-export ; C-; p SPC — cj/reveal-present +- C-; p p — cj/reveal-preview-start ; C-; p s — cj/reveal-preview-stop + +*** C-; r — Recording (audio/video) +- C-; r a — cj/audio-recording-toggle ; C-; r v — cj/video-recording-toggle +- C-; r s — cj/recording-quick-setup ; C-; r S — cj/recording-select-devices +- C-; r d — cj/recording-list-devices ; C-; r l — cj/recording-adjust-volumes +- C-; r w — cj/recording-show-active-audio +- C-; r t b/m/s — cj/recording-test-both / -mic / -monitor + +*** C-; R — restclient +- C-; R n — cj/restclient-new-buffer ; C-; R o — cj/restclient-open-file + +*** C-; s — Enclose / surround / indent +- C-; s s — cj/surround-word-or-region ; C-; s u — cj/unwrap-word-or-region +- C-; s w — cj/wrap-word-or-region ; C-; s i — cj/indent-lines-in-region-or-buffer +- C-; s d — cj/dedent-lines-in-region-or-buffer ; C-; s a — cj/append-to-lines-in-region-or-buffer +- C-; s p — cj/prepend-to-lines-in-region-or-buffer +- C-; s I — change-inner ; C-; s O — change-outer + +*** C-; t — Test runner +- C-; t r — cj/test-run-smart ; C-; t R — cj/test-run-all ; C-; t . — cj/run-test-at-point +- C-; t a — cj/test-focus-add ; C-; t b — cj/test-focus-add-this-buffer-file +- C-; t c — cj/test-focus-clear ; C-; t v — cj/test-view-focused +- C-; t L — cj/test-load-all ; C-; t t — cj/test-toggle-mode + +*** C-; v — Version control (git / forge) +- C-; v c — cj/git-clone-clipboard-url ; C-; v d — cj/goto-git-gutter-diff-hunks +- C-; v t — cj/git-timemachine ; C-; v f — forge-pull ; C-; v r — forge-list-pullreqs +- C-; v i c — cj/forge-create-issue ; C-; v i l — forge-list-issues + +*** C-; w — Whitespace +- C-; w c — cj/collapse-whitespace-line-or-region ; C-; w d — cj/delete-all-whitespace +- C-; w l — cj/delete-blank-lines-region-or-buffer ; C-; w 1 — cj/ensure-single-blank-line +- C-; w r — cj/remove-leading-trailing-whitespace ; C-; w - — cj/hyphenate-whitespace-in-region +- C-; w t — untabify ; C-; w T — tabify + +*** C-; x — Terminal (ghostel) +- C-; x t — cj/term-toggle ; C-; x N — ghostel (new) ; C-; x c — cj/term-copy-mode-dwim +- C-; x h — cj/term-tmux-history ; C-; x l — ghostel-clear-scrollback +- C-; x n — ghostel-next-prompt ; C-; x p — ghostel-previous-prompt +- C-; x q — ghostel-send-next-key + +** Appendix B — The M-S- family (18 keys) + +All bound as =M-S-= and reachable in GUI only, via the +=keyboard-compat.el= translation layer. Format: chord — command — what it does — +source module. + +- M-S-o — cj/kill-other-window — kill the other window's buffer and close it — undead-buffers.el +- M-S-m — cj/kill-all-other-buffers-and-windows — close all other windows, kill their buffers — undead-buffers.el +- M-S-y — yank-media — paste an image/media object from the clipboard — keybindings.el +- M-S-f — fontaine-set-preset — switch the font preset — font-config.el +- M-S-w — wttrin — show the weather report — weather-config.el +- M-S-e — eww — open the EWW web browser — eww-config.el +- M-S-l — cj/switch-themes — select/cycle the theme — ui-theme.el +- M-S-r — cj/elfeed-open — open the Elfeed RSS reader — elfeed-config.el +- M-S-v — cj/split-and-follow-right — split window right and move focus there — ui-navigation.el +- M-S-h — cj/split-and-follow-below — split window below and move focus there — ui-navigation.el +- M-S-t — toggle-window-split — toggle horizontal/vertical split — ui-navigation.el +- M-S-z — cj/undo-kill-buffer — reopen the most-recently-killed file buffer — ui-navigation.el +- M-S-u — winner-undo — undo the last window-configuration change — ui-navigation.el +- M-S-d — dwim-shell-commands-menu — DWIM shell-command menu on marked files — dwim-shell-config.el +- M-S-i — edit-indirect-region — edit the region in an indirect buffer — text-config.el +- M-S-c — time-zones — show the world-clock / time-zones view — chrono-tools.el +- M-S-b — calibredb — open the Calibre ebook library — calibredb-epub-config.el +- M-S-k — show-kill-ring — browse the kill ring — show-kill-ring.el + +Note: =4a1ecf64= (in-flight, reverted in Phase 0) currently leaves +=eww=/=elfeed=/=calibredb= mis-bound to =M-E=/=M-R=/=M-B=; the table lists the +intended/original =M-S-= bindings. + +** Appendix C — Console-safe single-prefix candidates (D1/D3) + +Craig's first choice (D1) is one =Control=+key prefix that works in GUI, terminal, +and the Linux console, ideally a lightly-used home-row key. Console transmittability +is the gate. Two classes of chord transmit in =TERM=linux=: + +1. =Control= + letter (ASCII 1–26). Several collide with terminal control characters + and so cannot serve as a distinct prefix — =C-i=/TAB, =C-j=/LF, =C-m=/RET, + =C-[=/ESC, =C-h=/DEL — and =C-g= (=keyboard-quit=) is sacred and excluded. +2. =Meta= + key, which the console sends as an *ESC prefix* (=M-x= = ESC then x). + This is why the Problem table above shows =M-E= working as "ESC E" in the console. + So a plain =M-= prefix is console-safe too — and unlike the broken =M-S-= + family, an unshifted =M-= binds directly with no =key-translation-map= in the + path. The catch is finding a free one: the Meta namespace is crowded (Appendix D + shows =M-*=, =M-+=, =M-#=, =M-P=, =M-t=, and the whole =M-g=/=M-s=/=M-e=/=M-r= + consult family taken), so a free Meta prefix would be punctuation (=M-\\=, =M-/=…), + not a letter, and it carries the usual ESC-prefix timing caveat in terminals. + +=Control= + *non-letter* punctuation (=C-;=, =C-'=, =C-.=…) does NOT transmit in the +console — the character isn't a control code. So =C-'= is a non-starter on two counts: +dead in the console like =C-;=, and already bound (=cj/flyspell-then-abbrev=, globally +at =flyspell-and-abbrev.el:253= and in =org-mode-map= at =:258=). =Control=+letter +(the table below) stays the cleanest path; a free =M-= is the viable +runner-up class if Craig prefers Meta. + +| Candidate | Home-row | Console-safe | Default binding | Verdict + note | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-l | yes | yes | recenter-top-bottom | TOP. Home-row, light default, | +| | | | | trivially relocated. Also | +| | | | | vertico-insert in the minibuffer | +| | | | | (selection-framework.el:42) — | +| | | | | minibuffer-local, no global | +| | | | | conflict. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-q | no | yes | quoted-insert | Strong runner-up. Very light | +| | | | | default; trivial rebind. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-o | no | yes | open-line | Strong runner-up. Light default; | +| | | | | easy rebind. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-t | no | yes | transpose-chars | Strong runner-up. Light default; | +| | | | | easy rebind. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-k | yes | yes | kill-line | Possible. Home-row, but kill-to-eol | +| | | | | is muscle memory — medium retrain | +| | | | | friction. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-s | yes | yes | cj/consult-line-or-repeat | Possible, but already a useful | +| | | | (selection-framework.el:265) | rebind; using it as a prefix | +| | | | | reverses that. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-a | yes | yes | move-beginning-of-line | Reject. Essential editing reflex. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-d | yes | yes | delete-char | Reject. Essential. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-f | yes | yes | forward-char | Reject. Essential. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-h | yes | collides (DEL) | help-command | Reject. Console DEL collision; help | +| | | | | is frequent. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-j | yes | collides (LF) | newline | Reject. LF control char; cannot | +| | | | | transmit distinctly. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-g | yes | sacred | keyboard-quit | Reject. Universal escape; never | +| | | | | reuse. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-z | no | yes | suspend-frame (live prefix; C-z F = | Reject. Already an extended prefix. | +| | | | fonts, font-config.el:300) | | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| C-' | no | no | cj/flyspell-then-abbrev | Reject. Punctuation — dead in the | +| | | | (flyspell-and-abbrev.el:253) | console like C-;; and already bound | +| | | | | (also org-mode-map :258). | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| +| M- | n/a | yes (ESC-prefix) | — (Meta namespace crowded; see | Viable runner-up class. Console-safe | +| | | | Appendix D) | via ESC-prefix, no translation | +| | | | | layer, distinct from the broken | +| | | | | M-S-. Needs a free M-punctuation | +| | | | | (M-\\, M-/); ESC-timing caveat in | +| | | | | terminals. | +|-----------+----------+------------------+-------------------------------------+--------------------------------------| + +Recommendation: =C-l= is the single best fit — the only clean home-row option (every +other home-row letter is essential, a collision, sacred, or already repurposed), +console-safe, and its default =recenter-top-bottom= is light and trivially relocated. +=C-q= / =C-o= / =C-t= are equally console-safe and lightly bound if Craig prefers to +keep all home-row defaults; they cost a right-hand reach off home row. If Craig would +rather a Meta prefix, a free =M-= (=M-\\=, =M-/=) is the viable runner-up +class — console-safe via ESC-prefix and free of the translation layer — at the cost of +the ESC-timing caveat. =C-'= is out (console-dead and already taken). Craig picks. + +** Appendix D — Personal keybindings set outside C-; (audit for pruning) + +Every keybinding Craig has set *outside* the =C-;= tree (Appendix A) and the =M-S-= +family (Appendix B), grouped by context. Check a box to mark that binding — or a +whole group — for removal in Phase 1. Boxes start unchecked; Craig marks them. +Inventoried 2026-06-13. Some =:bind (:map …)= package-integration maps (lsp-mode, +c-mode-base, python-ts, json-ts, outline-minor, magit-blame, quick-sdcv, cj/vc-map) +have large package-managed binding lists not enumerated here. + +- [ ] Global bindings + - [ ] C-+ — text-scale-increase — (font-config.el:306) + - [ ] C-= — text-scale-increase — (font-config.el:307) + - [ ] C-_ — text-scale-decrease — (font-config.el:308) + - [ ] C-- — text-scale-decrease — (font-config.el:309) + - [ ] C-x C-f — find-file — (keybindings.el:147) + - [ ] C-x \ — sort-lines — (keybindings.el:160) + - [ ] C-x u — undo-reminder-message — (keybindings.el:164) + - [ ] — keyboard-escape-quit — (keybindings.el:156) + - [ ] — cj/title-case-region — (custom-case.el:124) + - [ ] — cj/kill-buffer-or-bury-alive — (undead-buffers.el:55) + - [ ] — ibuffer — (system-utils.el:147) + - [ ] — cj/disabled — (system-defaults.el:191) + - [ ] C-z — prefix map (suspend-frame replacement) — (keybindings.el:148) + - [ ] C-z F — cj/display-available-fonts — (font-config.el:300) + - [ ] C-h A — cj/local-arch-wiki-search — (help-utils.el:82) + - [ ] C-h D s — devdocs-search — (help-utils.el:40) + - [ ] C-h D b — devdocs-peruse — (help-utils.el:41) + - [ ] C-h D l — devdocs-lookup — (help-utils.el:42) + - [ ] C-h D i — devdocs-install — (help-utils.el:43) + - [ ] C-h D d — devdocs-delete — (help-utils.el:44) + - [ ] C-h D u — devdocs-update-all — (help-utils.el:45) + - [ ] C-h P — list-packages — (help-config.el:31) + - [ ] C-h i — cj/browse-info-files — (help-config.el:90) + - [ ] C-c b — cj/eval-buffer-with-confirmation-or-error-message — (system-utils.el:57) + - [ ] C-c C — cj/org-contacts-map prefix — (org-contacts-config.el:271) + - [ ] C-c d — cj/debug-config-keymap prefix — (config-utilities.el:28) + - [ ] C-c f — cj/flyspell-toggle — (flyspell-and-abbrev.el:252) + - [ ] C-c l — org-store-link — (org-config.el:58) + - [ ] C-c m — mu4e — (mail-config.el:125) + - [ ] C-c M — mouse-trap-mode — (mousetrap-mode.el:275) + - [ ] C-' — cj/flyspell-then-abbrev — (flyspell-and-abbrev.el:253) + - [ ] C-s — cj/consult-line-or-repeat — (selection-framework.el:265) + - [ ] M-* — calculator — (keybindings.el:152) + - [ ] M-+ — balance-windows — (ui-navigation.el:67) + - [ ] M-P — cj/check-for-open-work — (reconcile-open-repos.el:221) + - [ ] C-c n d — org-roam-dailies-map prefix — (org-roam-config.el:94) + - [ ] C-c n I — cj/org-roam-node-insert-immediate — (org-roam-config.el:131) +- [ ] Function keys + - [ ] — cj/dashboard-only — (dashboard-config.el:158) + - [ ] — call-last-kbd-macro — (keyboard-macros.el:131) + - [ ] C- — cj/kbd-macro-start-or-end — (keyboard-macros.el:130) + - [ ] M- — cj/save-maybe-edit-macro — (keyboard-macros.el:132) + - [ ] s- — cj/open-macros-file — (keyboard-macros.el:133) + - [ ] — cj/f4-compile-and-run — (dev-fkeys.el:535) + - [ ] C- — cj/f4-compile-only — (dev-fkeys.el:536) + - [ ] M- — cj/f4-clean-rebuild — (dev-fkeys.el:537) + - [ ] S- — recompile — (dev-fkeys.el:538) + - [ ] — cj/f6-test-runner — (dev-fkeys.el:539) + - [ ] C- — cj/f6-current-file-tests — (dev-fkeys.el:540) + - [ ] S- (Python) — cj/python-mypy — (prog-python.el:103) + - [ ] S- (Shell) — cj/shell-run-shellcheck — (prog-shell.el:98) + - [ ] S- (Go) — cj/go-staticcheck — (prog-go.el:102) + - [ ] S- (C) — cj/disabled — (prog-c.el:158) + - [ ] S- (Python) — cj/python-debug — (prog-python.el:106) + - [ ] S- (Shell) — cj/disabled — (prog-shell.el:101) + - [ ] S- (Go) — cj/go-debug — (prog-go.el:105) + - [ ] S- (C) — gdb — (prog-c.el:161) + - [ ] — cj/coverage-report — (coverage-core.el:537) + - [ ] — cj/main-agenda-display — (org-agenda-config.el:418) + - [ ] C- — cj/todo-list-single-project — (org-agenda-config.el:269) + - [ ] M- — cj/todo-list-from-this-buffer — (org-agenda-config.el:283) + - [ ] s- — cj/todo-list-all-agenda-files — (org-agenda-config.el:244) + - [ ] — cj/ai-term — (ai-term.el:920) + - [ ] C- — cj/ai-term-pick-project — (ai-term.el:921) + - [ ] M- — cj/ai-term-close — (ai-term.el:922) + - [ ] C-S- — cj/ai-term-close — (ai-term.el:923) + - [ ] — cj/music-playlist-toggle — (music-config.el:910) + - [ ] C- — cj/server-shutdown — (system-utils.el:105) + - [ ] — cj/term-toggle — (term-config.el:383) + - [ ] C- — eshell-toggle — (eshell-config.el:161) +- [ ] use-package :bind (global) + - [ ] C-c L — slime — (prog-lisp.el:151) + - [ ] C-c G — geiser-guile — (prog-lisp.el:172) + - [ ] C-h L — leetcode — (prog-training.el:35) + - [ ] C-h M — man — (help-config.el:49) + - [ ] C-h T — tldr — (help-utils.el:53) + - [ ] C-h W — wiki-summary — (help-utils.el:58) + - [ ] C-` — accent-company — (text-config.el:122) + - [ ] C-x M-f — sudo-edit — (system-utils.el:66) + - [ ] C-x g — magit-status — (vc-config.el:34) + - [ ] C-c s i — consult-yasnippet — (selection-framework.el:191) + - [ ] M-# — calendar — (chrono-tools.el:38) + - [ ] M-t — tmr-prefix-map — (chrono-tools.el:110) + - [ ] C-M-p — proced — (system-utils.el:183) +- [ ] Vertico / selection framework + - [ ] C-h C-k — free-keys — (keybindings.el:129) + - [ ] C-j (vertico-map) — vertico-next — (selection-framework.el:40) + - [ ] C-k (vertico-map) — vertico-previous — (selection-framework.el:41) + - [ ] C-l (vertico-map) — vertico-insert — (selection-framework.el:42) + - [ ] RET (vertico-map) — vertico-exit — (selection-framework.el:43) + - [ ] C-RET (vertico-map) — vertico-exit-input — (selection-framework.el:44) + - [ ] M-RET (vertico-map) — minibuffer-force-complete-and-exit — (selection-framework.el:45) + - [ ] TAB (vertico-map) — minibuffer-complete — (selection-framework.el:46) +- [ ] Consult (global) + - [ ] C-c h — consult-history — (selection-framework.el:64) + - [ ] C-x M-: — consult-complex-command — (selection-framework.el:66) + - [ ] C-x b — consult-buffer — (selection-framework.el:67) + - [ ] C-x 4 b — consult-buffer-other-window — (selection-framework.el:68) + - [ ] C-x 5 b — consult-buffer-other-frame — (selection-framework.el:69) + - [ ] C-x r b — consult-bookmark — (selection-framework.el:70) + - [ ] C-x p b — consult-project-buffer — (selection-framework.el:71) + - [ ] M-g e — consult-compile-error — (selection-framework.el:73) + - [ ] M-g f — consult-flymake — (selection-framework.el:74) + - [ ] M-g g — consult-goto-line — (selection-framework.el:75) + - [ ] M-g M-g — consult-goto-line — (selection-framework.el:76) + - [ ] M-g o — consult-outline — (selection-framework.el:77) + - [ ] M-g m — consult-mark — (selection-framework.el:78) + - [ ] M-g k — consult-global-mark — (selection-framework.el:79) + - [ ] M-g i — consult-imenu — (selection-framework.el:80) + - [ ] M-g I — consult-imenu-multi — (selection-framework.el:81) + - [ ] M-s d — consult-find — (selection-framework.el:83) + - [ ] M-s D — consult-locate — (selection-framework.el:84) + - [ ] M-s g — consult-grep — (selection-framework.el:85) + - [ ] M-s G — consult-git-grep — (selection-framework.el:86) + - [ ] M-s r — consult-ripgrep — (selection-framework.el:87) + - [ ] M-s l — consult-line — (selection-framework.el:88) + - [ ] M-s L — consult-line-multi — (selection-framework.el:89) + - [ ] M-s k — consult-keep-lines — (selection-framework.el:90) + - [ ] M-s u — consult-focus-lines — (selection-framework.el:91) + - [ ] M-s e — consult-isearch-history — (selection-framework.el:93) +- [ ] Isearch / minibuffer search + - [ ] M-e (isearch-mode-map) — consult-isearch-history — (selection-framework.el:95) + - [ ] M-s e (isearch-mode-map) — consult-isearch-history — (selection-framework.el:96) + - [ ] M-s l (isearch-mode-map) — consult-line — (selection-framework.el:97) + - [ ] M-s L (isearch-mode-map) — consult-line-multi — (selection-framework.el:98) + - [ ] M-s (minibuffer-local-map) — consult-history — (selection-framework.el:101) + - [ ] M-r (minibuffer-local-map) — consult-history — (selection-framework.el:102) +- [ ] PDF view mode + - [ ] M — pdf-view-midnight-minor-mode — (pdf-config.el:49) + - [ ] m — bookmark-set — (pdf-config.el:50) + - [ ] C-= — pdf-view-enlarge — (pdf-config.el:51) + - [ ] C-- — pdf-view-shrink — (pdf-config.el:52) + - [ ] C-c l — org-store-link — (pdf-config.el:53) + - [ ] z — cj/open-file-with-command zathura — (pdf-config.el:54) + - [ ] j — image-next-line — (pdf-config.el:56) + - [ ] k — image-previous-line — (pdf-config.el:57) + - [ ] — image-next-line — (pdf-config.el:58) + - [ ] — image-previous-line — (pdf-config.el:59) + - [ ] i — cj/org-noter-insert-note-dwim — (pdf-config.el:61) + - [ ] C- — pdf-view-next-page-command + image-bob — (pdf-config.el:63) + - [ ] C- — pdf-view-previous-page-command + image-eob — (pdf-config.el:65) +- [ ] Ediff mode + - [ ] j (ediff-mode-map) — ediff-next-difference — (diff-config.el:54) + - [ ] k (ediff-mode-map) — ediff-previous-difference — (diff-config.el:55) +- [ ] Org / org-related + - [ ] C-' (org-mode-map) — cj/flyspell-then-abbrev — (flyspell-and-abbrev.el:258) + - [ ] S- (org-mouse-map) — cj/org-follow-link-at-mouse-same-window — (org-config.el:338) + - [ ] (org-mouse-map) — cj/org-follow-link-at-mouse-same-window — (org-config.el:339) +- [ ] Dired / dirvish + - [ ] G (dired-mode-map) — cj/deadgrep-here — (prog-general.el:277) + - [ ] M-D (dirvish-mode-map) — dwim-shell-commands-menu — (dwim-shell-config.el:934) + - [ ] + (dirvish-mode-map) — cj/music-add-dired-selection — (music-config.el:597) + - [ ] T (dired/dirvish-mode-map) — cj/transcribe-media-at-point — (transcription-config.el:463/467) + - [ ] (dirvish-mode-map) — dirvish-side — (dirvish-config.el:481) +- [ ] Shell / terminal + - [ ] C-r (eshell-mode-map) — cj/eshell-history-search — (eshell-config.el:202) + - [ ] (eshell-hist-mode-map) — previous-line — (eshell-config.el:99) + - [ ] (eshell-hist-mode-map) — next-line — (eshell-config.el:100) +- [ ] Ghostel terminal + - [ ] (ghostel-mode-map) — cj/ai-term — (ai-term.el:932) + - [ ] C- (ghostel-mode-map) — cj/ai-term-pick-project — (ai-term.el:933) + - [ ] M- (ghostel-mode-map) — cj/ai-term-close — (ai-term.el:934) + - [ ] C-S- (ghostel-mode-map) — cj/ai-term-close — (ai-term.el:935) + - [ ] (ghostel-mode-map) — cj/term-toggle — (term-config.el:415) + - [ ] C-SPC (ghostel-mode-map) — cj/term-send-C-SPC — (term-config.el:416) +- [ ] Version control / magit + - [ ] M-g (git-commit-mode-map) — gptel-magit-generate-message — (ai-config.el:498) + - [ ] N (magit-mode-map) — forge-pull — (vc-config.el:125) +- [ ] Help / docs modes + - [ ] b (devdocs-mode-map) — devdocs-go-back — (help-utils.el:47) + - [ ] f (devdocs-mode-map) — devdocs-go-forward — (help-utils.el:48) +- [ ] Org-roam dailies + - [ ] Y (org-roam-dailies-map) — org-roam-dailies-capture-yesterday — (org-roam-config.el:92) + - [ ] T (org-roam-dailies-map) — org-roam-dailies-capture-tomorrow — (org-roam-config.el:93) +- [ ] Other mode maps + - [ ] C- (slack-message-compose-buffer-mode-map) — slack-message-send-from-buffer — (slack-config.el:297) + - [ ] q (dashboard-mode-map) — nil (unbound) — (dashboard-config.el:223) + - [ ] q (show-kill-ring-mode-map) — show-kill-ring-exit — (show-kill-ring.el:67) + - [ ] (markdown-mode-map) — markdown-preview — (markdown-config.el:24) + - [ ] — dwim-shell-command — (dwim-shell-config.el:204) +- [ ] key-translation-map / input-decode-map + - [ ] input-decode-map ESC [ A — [up] — (keyboard-compat.el:109) + - [ ] input-decode-map ESC [ B — [down] — (keyboard-compat.el:110) + - [ ] input-decode-map ESC [ C — [right] — (keyboard-compat.el:111) + - [ ] input-decode-map ESC [ D — [left] — (keyboard-compat.el:112) + - [ ] input-decode-map ESC O A — [up] — (keyboard-compat.el:115) + - [ ] input-decode-map ESC O B — [down] — (keyboard-compat.el:116) + - [ ] input-decode-map ESC O C — [right] — (keyboard-compat.el:117) + - [ ] input-decode-map ESC O D — [left] — (keyboard-compat.el:118) +- [ ] Jumper + - [ ] jumper-prefix-key (computed at runtime) — jumper-map — (jumper.el:270) [computed key — exact binding depends on the variable value] + +Note: the global =M-S-= family is intentionally excluded (Appendix B). The +arrow-key =input-decode-map= entries are the terminal setup the spec keeps (not the +translation block being retired). =C-l= appears only minibuffer-local in +=vertico-map=, consistent with Appendix C. + +* Review and iteration history +** 2026-06-12 Fri @ 11:21:56 -0500 — Craig Jennings — author +- What: initial draft. Problem, three-context analysis, the 4a1ecf64 regression + as motivating evidence, the one-map/two-prefix design, alternatives, five + open decisions, phased plan, acceptance criteria, readiness dimensions, and the + full C-; tree + M-S- family appendices. +- Why: a touched key family broke in GUI and is dead in console; the fix path is + cross-cutting (18 keys, a translation layer to retire, a console-safety + architecture) with real trade-offs, so it clears the spec bar. +- Artifacts: docs/specs/keybinding-console-safety-spec.org; supersedes the + pre-template draft docs/design/keybinding-console-safety.org. +** 2026-06-12 Fri @ 18:30:30 -0500 — Craig Jennings — review response +- What: processed Craig's four review comments. Recorded his first-choice + direction — one console-safe =Control=+key prefix used everywhere (single-prefix + primary; the two-prefix design is now the documented fallback) — in the Summary, + Design, and D1. Added Appendix C, the console-safe single-prefix candidate table + (standout =C-l=; runners-up =C-q=/=C-o=/=C-t=). Reframed D3 around that pick. + Named the flawed test (=tests/test-launcher-meta-shift-keys.el=) and quoted its + =key-binding=-only assertion in Phase 0. Recorded Craig's decision that the + launcher apps (=eww=/=elfeed=/=calibredb=/=wttrin=) get new keys under a launcher + sub-prefix, not =M-x= (D2/D5, Phases 0/2), with =C-; a t= (=cj/toggle-gptel=, + =ai-config.el:541=) flagged as freeable space. Reserved =C-; L= for Pearl in + Appendix A and D5. +- Why: Craig's review shifted the architecture from two-prefix to a single unified + console-safe prefix and resolved the apps disposition; the spec had to carry the + candidate data he asked for and reflect the choices through the phases. +- Open: the specific prefix (Appendix C), the window and launcher sub-prefix + letters (D5) remain Craig's picks. D1–D5 still State: proposed pending those. +** 2026-06-12 Fri @ 18:43:25 -0500 — Craig Jennings — decisions-as-TODO convention +- What: switched the Decisions section to org TODO tasks. Each decision is =** TODO + Dn=, flips to =DONE= when Craig agrees, stays TODO with a =*** Discussion= child + thread when not. Added a =[0/5]= statistics cookie and a gate: spec Status cannot + reach =ready= while any decision is TODO. Current status: all 5 TODO (none fully + agreed — D1 awaits the prefix lock, D2 unreviewed, D3 awaits the prefix pick, D4 + awaits the A/B pick, D5's apps half agreed but both sub-prefix letters open). +- Why: replaces the inline =State: proposed/accepted= field with an org-native, + agenda-visible task + discussion-thread workflow, and makes the + all-decisions-resolved gate explicit and machine-checkable. +** 2026-06-13 Sat @ 00:18:09 -0500 — Craig Jennings — Path 2 restructure + audit appendix +- What: processed three more review comments. Restructured the phases to Path 2: + Phase 0 is a pure revert that unblocks the held push; Phase 1 prunes forgotten + keybindings (Appendix D); Phase 2 is the consolidation (migrate the common set, + retire the translation block) — the primary deliverable; Phase 3 (bind the + console-safe prefix) is now OPTIONAL and deferred until Craig picks the key. The + Decisions gate split accordingly: D2/D4/D5 gate the primary work, D1/D3 gate only + Phase 3, so the work runs to the prefix-assignment point and stops there. + Corrected Appendix C's premise (Meta transmits in the console as an ESC prefix, so + =M-= is a viable console-safe class); added the =C-'= row (rejected — + console-dead and already bound to flyspell) and the =M-= row. Added Appendix + D: every personal keybinding set outside the =C-;= tree and the =M-S-= family, as a + checkbox pruning tree (~190 bindings, inventoried by a read-only sweep). +- Why: Craig pivoted to landing the consolidation first and treating the + console-safe prefix as a later switch-on, and wanted a one-time audit of his + set-and-forgotten keybindings while the keymap work was open. +- Open: D1–D5 still TODO; the prefix (D1/D3) is now non-blocking. Phase 0 revert + pending so the push can proceed. diff --git a/docs/specs/messenger-unification-spec.org b/docs/specs/messenger-unification-spec.org index 92985f59..c92ba1a7 100644 --- a/docs/specs/messenger-unification-spec.org +++ b/docs/specs/messenger-unification-spec.org @@ -1,11 +1,14 @@ -:PROPERTIES: -:ID: 4bfc2011-8ffc-4765-8886-91df12141171 -:STATUS: not-started -:END: #+TITLE: Messenger Unification — Shared Window Placement and Key Conventions #+AUTHOR: Craig Jennings & Claude #+DATE: 2026-06-11 -#+STATUS: Draft — decisions 1-9 settled (Craig, 2026-06-11/12); held open for further ideas before Ready +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Messenger Unification — Shared Window Placement and Key Conventions +:PROPERTIES: +:ID: 4bfc2011-8ffc-4765-8886-91df12141171 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from existing :STATUS: not-started (held open for more ideas) * Problem diff --git a/docs/specs/music-config-without-emms-spec.org b/docs/specs/music-config-without-emms-spec.org index 32fd6736..c63706e5 100644 --- a/docs/specs/music-config-without-emms-spec.org +++ b/docs/specs/music-config-without-emms-spec.org @@ -1,10 +1,14 @@ -:PROPERTIES: -:ID: 423bc355-18d3-4e39-9e7a-f768b865d95b -:STATUS: not-started -:END: #+TITLE: Design: music-config Without EMMS #+AUTHOR: Craig Jennings #+DATE: 2026-05-15 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Design: music-config Without EMMS +:PROPERTIES: +:ID: 423bc355-18d3-4e39-9e7a-f768b865d95b +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from existing :STATUS: not-started * Status diff --git a/docs/specs/org-faces-spec-implemented.org b/docs/specs/org-faces-spec-implemented.org deleted file mode 100644 index c8855906..00000000 --- a/docs/specs/org-faces-spec-implemented.org +++ /dev/null @@ -1,154 +0,0 @@ -:PROPERTIES: -:ID: 35578114-8c29-43af-97a2-fdfea01a802e -:STATUS: implemented -:END: -#+TITLE: Org Header-Row Faces — Spec -#+AUTHOR: Craig Jennings -#+DATE: 2026-06-15 -#+TODO: TODO | DONE SUPERSEDED CANCELLED - -* Metadata -| Status | implemented | -|----------+----------------------------------------------------------------| -| Owner | Craig Jennings | -|----------+----------------------------------------------------------------| -| Reviewer | Craig Jennings | -|----------+----------------------------------------------------------------| -| Related | [[file:../../todo.org][todo.org: org-faces module + theme-studio app]] | -|----------+----------------------------------------------------------------| - -* Summary - -A small config module, =org-faces-config.el=, that defines named, theme-agnostic faces for org's TODO keywords and priority cookies, wires them through =org-todo-keyword-faces= and =org-priority-faces=, and a matching theme-studio app titled "org-faces" so each is an editable, previewable element. It makes the agenda header row (keyword plus priority) a per-element themeable layer that reads as clearly custom, not built-in org. - -* Problem / Context - -Per-keyword and per-priority coloring was stripped from =org-config.el= earlier, so today every keyword (TODO, DOING, …) renders in org's built-in =org-todo= / =org-done=, and every priority cookie ([#A] through [#D]) renders in the single =org-priority= face — one color for all four. There's no way to color them individually. - -The dupre theme does carry a parallel custom set (=dupre-org-todo=, =dupre-org-priority-a..d=, with =-dim= variants), but it's theme-specific and currently unwired, so it colors nothing. And theme-studio has no surface for these at all: its org-mode app exposes only the built-in =org-priority=, and the preview shows a single [#A]. - -The immediate driver is theme-testing the agenda: the header row is the densest, most-scanned part of the agenda, and right now its keyword/priority colors can't be tuned in theme-studio or distinguished from built-in org faces. The user wants each keyword and priority to be its own themeable element, in its own clearly-custom namespace, surfaced in its own theme-studio section. - -* Goals and Non-Goals - -** Goals -- Each keyword (TODO, PROJECT, DOING, WAITING, VERIFY, STALLED, DELEGATED, FAILED, DONE, CANCELLED) and each priority (A-D) gets its own named face. -- The faces live in one module (=org-faces-config.el=), in their own prefix, wired through =org-todo-keyword-faces= and =org-priority-faces=. -- theme-studio surfaces them as a dedicated "org-faces" app (own section, alongside elfeed and mu4e) with one editable row per face and a header-row preview. -- They render correctly on any theme (sensible defaults) and are overridden by the generated theme. - -** Non-Goals -- Not editing the built-in org faces — the org-mode app keeps those. -- Not a general org face overhaul; only the header-row keyword + priority set. -- Not retiring or deleting the legacy =dupre-org-*= faces from dupre-faces.el in v1 — auto-dim is only repointed away from them (decision below). - -** Scope tiers -- v1: =org-faces-config.el= (base + =-dim= faces, wiring, init load); =org-faces-*-dim= wired into auto-dim (replacing the orphaned dupre-org-* entries); the theme-studio "org-faces" app (faces, preview, seeds). -- Out of scope: built-in org faces; non-keyword/priority org elements; terminal/document-rendered color sources. -- vNext (log to todo.org): retire or migrate the legacy =dupre-org-*= faces in dupre-faces.el; a grouped-subsection preview. - -* Design - -A new bespoke app, "org-faces", joins theme-studio's application list next to elfeed and mu4e. Its faces are the config's custom header-row set, not org's built-ins, and the =org-faces-= prefix says so on every row. - -** For the user - -Pick "org-faces" in theme-studio's application dropdown. The face table lists each keyword and each priority as a normal editable row — foreground, background, style toggles, box, lock — exactly like every other package face. The preview renders agenda-style lines: each keyword shown in its own face, then a =[#A] [#B] [#C] [#D]= row in the four priority faces, so the ramp is visible and any cookie is click-to-select. Setting a color there writes it into the generated theme; once that theme loads, the real agenda's keywords and priorities pick it up. Because the faces are named =org-faces-todo=, =org-faces-priority-a=, and so on, it's obvious this is the config's layer rather than built-in org. - -** For the implementer - -=org-faces-config.el= defines one =defface= per keyword and per priority (=org-faces-todo= … =org-faces-cancelled=, =org-faces-priority-a= … =org-faces-priority-d=), each with a default foreground so the row is colored out of the box. It then sets =org-todo-keyword-faces= to map each keyword string to its face and =org-priority-faces= to map =?A..?D= to the priority faces. The module is required after org so the faces exist before org applies the maps; the =defface=​s themselves load eagerly, which is what org needs. - -theme-studio side, all mechanical against the existing bespoke-app machinery: -- =face_data.py=: =ORGFACES_FACES= (the face-name list) and =ORGFACES_SEED= (default colors mirroring =org-faces-config.el=). -- =generate.py=: one row in the =_BESPOKE_APPS= spec, =("org-faces","org-faces","orgfaces",ORGFACES_FACES,"org-faces-",ORGFACES_SEED)=. -- =app_inventory.py=: add =org-faces= to =BESPOKE_APPS=. -- =app.js=: =renderOrgFacesPreview= building the keyword lines and the priority-cookie row with =os('org-faces', face, text)=, registered under =orgfaces= in =PACKAGE_PREVIEWS=. -- =build-theme.el= needs no change — the package tier already emits these faces. - -The =org-faces-= prefix is also theme-studio's label-strip prefix, so rows read as "todo", "priority a", etc. - -* Alternatives Considered - -** Reuse the existing dupre-org-* names -- Good, because no new faces are defined. -- Bad, because the names are theme-specific (dupre) and the goal is a theme-agnostic, clearly-custom namespace; they'd still need all the theme-studio wiring. -- Neutral, because the auto-dim mappings already reference them, which both helps (dim variants exist) and hurts (couples the new layer to one theme). - -** Inline specs in org-todo-keyword-faces (no named faces) -- Good, because it's the least code and needs no defface. -- Bad, because theme-studio can only theme *named* faces; inline specs can't be edited or previewed there, which defeats the whole point. -- Neutral, because org supports both forms equally at runtime. - -** Put these in the existing org-mode app rather than a new app -- Good, because one fewer app in the dropdown. -- Bad, because it blurs the built-in-vs-custom line the user explicitly wants drawn. -- Neutral, because the preview would grow rather than a new one being added. - -* Decisions [4/4] - -** DONE Face prefix -- Context: the prefix is the public face namespace and theme-studio's label-strip; it must read as custom, not built-in org. -- Decision: We will use =org-faces-= (e.g. =org-faces-todo=, =org-faces-priority-a=). -- Consequences: easier — cohesive with the module and section name, and a clean label-strip prefix; harder — it still begins with "org", so a newcomer could misread it as built-in. - -** DONE defface defaults vs inherit-only -- Context: should the header row be colored on any theme, or only once a theme sets these faces? -- Decision: We will give each face a real default =:foreground= so it's colored out of the box, overridable by the theme. -- Consequences: easier — works on stock Emacs and any theme, and the theme-studio seed matches the live default; harder — the defaults are theme-blind, so a theme that doesn't override them shows the generic colors rather than its own palette. - -** DONE Auto-dim dim variants -- Context: dupre defines =dupre-org-*= and auto-dim remaps them to =-dim= variants in unfocused windows; rewiring to =org-faces-*= would drop that dim treatment unless it's carried over. -- Decision: We will include =org-faces-*-dim= variants in v1 and wire them into auto-dim, replacing the now-orphaned =dupre-org-*= entries in =auto-dim-config.el=, so keywords and priorities stay legible when a window is dimmed. The legacy =dupre-org-*= faces in =dupre-faces.el= are left defined-but-unused; retiring them stays vNext. -- Consequences: easier — the dim treatment carries onto the new layer and dupre's mapping stops referencing orphaned faces; harder — doubles the face count (base + dim), touches =auto-dim-config.el=, and under the dupre theme the new faces use their defaults until dupre themes =org-faces-*=. - -** DONE Keyword coverage -- Context: the vocabulary has 10 keywords; dupre only ever defined faces for 8. -- Decision: We will give all 10 keywords (including DELEGATED and CANCELLED) their own face. -- Consequences: easier — full control, no surprise fallbacks; harder — two more faces to seed and maintain. - -* Implementation phases - -** Phase 1 — org-faces.el module -Define the base and =-dim= =defface=​s (all 10 keywords + A-D, each with real default foregrounds) and the =org-todo-keyword-faces= / =org-priority-faces= wiring, require it after org. Done when the agenda colors each keyword and priority through the new base faces. Verify with a full Emacs launch (the wiring is :config-adjacent). - -** Phase 2 — auto-dim integration -In =auto-dim-config.el=, replace the =dupre-org-*= → =dupre-org-*-dim= entries with =org-faces-*= → =org-faces-*-dim=, so dimmed windows render the new layer through its dim variants. Done when an unfocused window shows keywords/priorities in their dim colors. - -** Phase 3 — theme-studio org-faces app -Add the =face_data.py= face list and seed (base + dim), the =generate.py= spec row, the =app_inventory.py= entry, and the =app.js= preview plus registration. Done when "org-faces" appears in the dropdown next to elfeed/mu4e, the rows edit, the preview renders, and =make theme-studio-test= is green. - -** Phase 4 — generated-theme round-trip -Set a color for an =org-faces-*= face in theme-studio, =make deploy-wip=, and confirm the real agenda picks it up. Done when the round-trip lands the color in Emacs. - -* Acceptance criteria -- [ ] Each keyword (TODO … CANCELLED) renders in a distinct =org-faces-*= face in the agenda. -- [ ] [#A]-[#D] render in distinct =org-faces-priority-*= faces. -- [ ] theme-studio shows an "org-faces" app beside elfeed/mu4e, one row per face, with a header-row preview. -- [ ] A color set in theme-studio for an =org-faces-*= face appears on the real agenda after =deploy-wip=. -- [ ] =org-faces-config.el= byte-compiles clean and the theme-studio suite is green. - -* Readiness dimensions -- Data model & ownership: faces are user-authored defaults (=org-faces-config.el=) overridden by the generated theme; theme-studio edits the package-tier specs; =org-todo-keyword-faces= / =org-priority-faces= own the keyword/priority wiring. -- Errors, empty states & failure: a keyword with no mapping falls back to org's built-in =org-todo= / =org-done= — acceptable, not silent data loss. N/A otherwise (no I/O). -- Security & privacy: N/A — faces only. -- Observability: the live agenda and the theme-studio preview are the visible surface; a wrong color is self-evident. -- Performance & scale: N/A — about a dozen faces. -- Reuse & lost opportunities: rides org's built-in keyword/priority face hooks and build-theme's existing package tier; no converter changes. -- Architecture fit & weak points: same bespoke-app pattern as elfeed/mu4e. Weak point is load order — faces must exist before org applies the map; eager =defface= at module load covers it. -- Config surface: =org-todo-keyword-faces=, =org-priority-faces= (set by the module), plus the faces themselves; all overridable. -- Documentation plan: a header comment in =org-faces-config.el= and this spec; no user-facing docs needed. -- Dev tooling: existing =make theme-studio-test= and =make deploy-wip= cover build and round-trip. -- Rollout, compatibility & rollback: additive; rollback is removing the module and the theme-studio app. It re-introduces keyword/priority coloring that was deliberately stripped earlier, now as a named themeable layer. -- External APIs & deps: =org-todo-keyword-faces= and =org-priority-faces= are standard org defcustoms (verified); build-theme's package tier already emits inherit/box/style (verified this session). - -* Risks, Rabbit Holes, and Drawbacks -- Poor default colors fight unset themes. Dodge: seed conservatively, lean on theme override. -- Load order: the faces must be defined before org renders the agenda. Dodge: eager defface, require before/with org. -- Scope creep into the dupre/auto-dim migration. Dodge: it's explicitly vNext with its own decision. - -* Review and iteration history -** 2026-06-15 Mon @ 01:51:57 -0500 — Craig — author -- What: initial draft. -- Why: the A-D priority request generalized into a clearly-custom, theme-studio-surfaced header-row face layer; the prefix, default-color policy, and dupre/auto-dim reconciliation are real trade-offs worth settling on paper first. -- Artifacts: [[file:../../todo.org][todo.org org-faces task]]; theme-studio bespoke-app machinery (face_data.py, generate.py, app.js). diff --git a/docs/specs/org-faces-spec.org b/docs/specs/org-faces-spec.org new file mode 100644 index 00000000..94fe7bb4 --- /dev/null +++ b/docs/specs/org-faces-spec.org @@ -0,0 +1,157 @@ +#+TITLE: Org Header-Row Faces — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-15 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Org Header-Row Faces — Spec +:PROPERTIES: +:ID: 35578114-8c29-43af-97a2-fdfea01a802e +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword IMPLEMENTED from existing :STATUS: implemented + -implemented filename (Craig's prior determination) + +* Metadata +| Status | implemented | +|----------+----------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+----------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+----------------------------------------------------------------| +| Related | [[file:../../todo.org][todo.org: org-faces module + theme-studio app]] | +|----------+----------------------------------------------------------------| + +* Summary + +A small config module, =org-faces-config.el=, that defines named, theme-agnostic faces for org's TODO keywords and priority cookies, wires them through =org-todo-keyword-faces= and =org-priority-faces=, and a matching theme-studio app titled "org-faces" so each is an editable, previewable element. It makes the agenda header row (keyword plus priority) a per-element themeable layer that reads as clearly custom, not built-in org. + +* Problem / Context + +Per-keyword and per-priority coloring was stripped from =org-config.el= earlier, so today every keyword (TODO, DOING, …) renders in org's built-in =org-todo= / =org-done=, and every priority cookie ([#A] through [#D]) renders in the single =org-priority= face — one color for all four. There's no way to color them individually. + +The dupre theme does carry a parallel custom set (=dupre-org-todo=, =dupre-org-priority-a..d=, with =-dim= variants), but it's theme-specific and currently unwired, so it colors nothing. And theme-studio has no surface for these at all: its org-mode app exposes only the built-in =org-priority=, and the preview shows a single [#A]. + +The immediate driver is theme-testing the agenda: the header row is the densest, most-scanned part of the agenda, and right now its keyword/priority colors can't be tuned in theme-studio or distinguished from built-in org faces. The user wants each keyword and priority to be its own themeable element, in its own clearly-custom namespace, surfaced in its own theme-studio section. + +* Goals and Non-Goals + +** Goals +- Each keyword (TODO, PROJECT, DOING, WAITING, VERIFY, STALLED, DELEGATED, FAILED, DONE, CANCELLED) and each priority (A-D) gets its own named face. +- The faces live in one module (=org-faces-config.el=), in their own prefix, wired through =org-todo-keyword-faces= and =org-priority-faces=. +- theme-studio surfaces them as a dedicated "org-faces" app (own section, alongside elfeed and mu4e) with one editable row per face and a header-row preview. +- They render correctly on any theme (sensible defaults) and are overridden by the generated theme. + +** Non-Goals +- Not editing the built-in org faces — the org-mode app keeps those. +- Not a general org face overhaul; only the header-row keyword + priority set. +- Not retiring or deleting the legacy =dupre-org-*= faces from dupre-faces.el in v1 — auto-dim is only repointed away from them (decision below). + +** Scope tiers +- v1: =org-faces-config.el= (base + =-dim= faces, wiring, init load); =org-faces-*-dim= wired into auto-dim (replacing the orphaned dupre-org-* entries); the theme-studio "org-faces" app (faces, preview, seeds). +- Out of scope: built-in org faces; non-keyword/priority org elements; terminal/document-rendered color sources. +- vNext (log to todo.org): retire or migrate the legacy =dupre-org-*= faces in dupre-faces.el; a grouped-subsection preview. + +* Design + +A new bespoke app, "org-faces", joins theme-studio's application list next to elfeed and mu4e. Its faces are the config's custom header-row set, not org's built-ins, and the =org-faces-= prefix says so on every row. + +** For the user + +Pick "org-faces" in theme-studio's application dropdown. The face table lists each keyword and each priority as a normal editable row — foreground, background, style toggles, box, lock — exactly like every other package face. The preview renders agenda-style lines: each keyword shown in its own face, then a =[#A] [#B] [#C] [#D]= row in the four priority faces, so the ramp is visible and any cookie is click-to-select. Setting a color there writes it into the generated theme; once that theme loads, the real agenda's keywords and priorities pick it up. Because the faces are named =org-faces-todo=, =org-faces-priority-a=, and so on, it's obvious this is the config's layer rather than built-in org. + +** For the implementer + +=org-faces-config.el= defines one =defface= per keyword and per priority (=org-faces-todo= … =org-faces-cancelled=, =org-faces-priority-a= … =org-faces-priority-d=), each with a default foreground so the row is colored out of the box. It then sets =org-todo-keyword-faces= to map each keyword string to its face and =org-priority-faces= to map =?A..?D= to the priority faces. The module is required after org so the faces exist before org applies the maps; the =defface=​s themselves load eagerly, which is what org needs. + +theme-studio side, all mechanical against the existing bespoke-app machinery: +- =face_data.py=: =ORGFACES_FACES= (the face-name list) and =ORGFACES_SEED= (default colors mirroring =org-faces-config.el=). +- =generate.py=: one row in the =_BESPOKE_APPS= spec, =("org-faces","org-faces","orgfaces",ORGFACES_FACES,"org-faces-",ORGFACES_SEED)=. +- =app_inventory.py=: add =org-faces= to =BESPOKE_APPS=. +- =app.js=: =renderOrgFacesPreview= building the keyword lines and the priority-cookie row with =os('org-faces', face, text)=, registered under =orgfaces= in =PACKAGE_PREVIEWS=. +- =build-theme.el= needs no change — the package tier already emits these faces. + +The =org-faces-= prefix is also theme-studio's label-strip prefix, so rows read as "todo", "priority a", etc. + +* Alternatives Considered + +** Reuse the existing dupre-org-* names +- Good, because no new faces are defined. +- Bad, because the names are theme-specific (dupre) and the goal is a theme-agnostic, clearly-custom namespace; they'd still need all the theme-studio wiring. +- Neutral, because the auto-dim mappings already reference them, which both helps (dim variants exist) and hurts (couples the new layer to one theme). + +** Inline specs in org-todo-keyword-faces (no named faces) +- Good, because it's the least code and needs no defface. +- Bad, because theme-studio can only theme *named* faces; inline specs can't be edited or previewed there, which defeats the whole point. +- Neutral, because org supports both forms equally at runtime. + +** Put these in the existing org-mode app rather than a new app +- Good, because one fewer app in the dropdown. +- Bad, because it blurs the built-in-vs-custom line the user explicitly wants drawn. +- Neutral, because the preview would grow rather than a new one being added. + +* Decisions [4/4] + +** DONE Face prefix +- Context: the prefix is the public face namespace and theme-studio's label-strip; it must read as custom, not built-in org. +- Decision: We will use =org-faces-= (e.g. =org-faces-todo=, =org-faces-priority-a=). +- Consequences: easier — cohesive with the module and section name, and a clean label-strip prefix; harder — it still begins with "org", so a newcomer could misread it as built-in. + +** DONE defface defaults vs inherit-only +- Context: should the header row be colored on any theme, or only once a theme sets these faces? +- Decision: We will give each face a real default =:foreground= so it's colored out of the box, overridable by the theme. +- Consequences: easier — works on stock Emacs and any theme, and the theme-studio seed matches the live default; harder — the defaults are theme-blind, so a theme that doesn't override them shows the generic colors rather than its own palette. + +** DONE Auto-dim dim variants +- Context: dupre defines =dupre-org-*= and auto-dim remaps them to =-dim= variants in unfocused windows; rewiring to =org-faces-*= would drop that dim treatment unless it's carried over. +- Decision: We will include =org-faces-*-dim= variants in v1 and wire them into auto-dim, replacing the now-orphaned =dupre-org-*= entries in =auto-dim-config.el=, so keywords and priorities stay legible when a window is dimmed. The legacy =dupre-org-*= faces in =dupre-faces.el= are left defined-but-unused; retiring them stays vNext. +- Consequences: easier — the dim treatment carries onto the new layer and dupre's mapping stops referencing orphaned faces; harder — doubles the face count (base + dim), touches =auto-dim-config.el=, and under the dupre theme the new faces use their defaults until dupre themes =org-faces-*=. + +** DONE Keyword coverage +- Context: the vocabulary has 10 keywords; dupre only ever defined faces for 8. +- Decision: We will give all 10 keywords (including DELEGATED and CANCELLED) their own face. +- Consequences: easier — full control, no surprise fallbacks; harder — two more faces to seed and maintain. + +* Implementation phases + +** Phase 1 — org-faces.el module +Define the base and =-dim= =defface=​s (all 10 keywords + A-D, each with real default foregrounds) and the =org-todo-keyword-faces= / =org-priority-faces= wiring, require it after org. Done when the agenda colors each keyword and priority through the new base faces. Verify with a full Emacs launch (the wiring is :config-adjacent). + +** Phase 2 — auto-dim integration +In =auto-dim-config.el=, replace the =dupre-org-*= → =dupre-org-*-dim= entries with =org-faces-*= → =org-faces-*-dim=, so dimmed windows render the new layer through its dim variants. Done when an unfocused window shows keywords/priorities in their dim colors. + +** Phase 3 — theme-studio org-faces app +Add the =face_data.py= face list and seed (base + dim), the =generate.py= spec row, the =app_inventory.py= entry, and the =app.js= preview plus registration. Done when "org-faces" appears in the dropdown next to elfeed/mu4e, the rows edit, the preview renders, and =make theme-studio-test= is green. + +** Phase 4 — generated-theme round-trip +Set a color for an =org-faces-*= face in theme-studio, =make deploy-wip=, and confirm the real agenda picks it up. Done when the round-trip lands the color in Emacs. + +* Acceptance criteria +- [ ] Each keyword (TODO … CANCELLED) renders in a distinct =org-faces-*= face in the agenda. +- [ ] [#A]-[#D] render in distinct =org-faces-priority-*= faces. +- [ ] theme-studio shows an "org-faces" app beside elfeed/mu4e, one row per face, with a header-row preview. +- [ ] A color set in theme-studio for an =org-faces-*= face appears on the real agenda after =deploy-wip=. +- [ ] =org-faces-config.el= byte-compiles clean and the theme-studio suite is green. + +* Readiness dimensions +- Data model & ownership: faces are user-authored defaults (=org-faces-config.el=) overridden by the generated theme; theme-studio edits the package-tier specs; =org-todo-keyword-faces= / =org-priority-faces= own the keyword/priority wiring. +- Errors, empty states & failure: a keyword with no mapping falls back to org's built-in =org-todo= / =org-done= — acceptable, not silent data loss. N/A otherwise (no I/O). +- Security & privacy: N/A — faces only. +- Observability: the live agenda and the theme-studio preview are the visible surface; a wrong color is self-evident. +- Performance & scale: N/A — about a dozen faces. +- Reuse & lost opportunities: rides org's built-in keyword/priority face hooks and build-theme's existing package tier; no converter changes. +- Architecture fit & weak points: same bespoke-app pattern as elfeed/mu4e. Weak point is load order — faces must exist before org applies the map; eager =defface= at module load covers it. +- Config surface: =org-todo-keyword-faces=, =org-priority-faces= (set by the module), plus the faces themselves; all overridable. +- Documentation plan: a header comment in =org-faces-config.el= and this spec; no user-facing docs needed. +- Dev tooling: existing =make theme-studio-test= and =make deploy-wip= cover build and round-trip. +- Rollout, compatibility & rollback: additive; rollback is removing the module and the theme-studio app. It re-introduces keyword/priority coloring that was deliberately stripped earlier, now as a named themeable layer. +- External APIs & deps: =org-todo-keyword-faces= and =org-priority-faces= are standard org defcustoms (verified); build-theme's package tier already emits inherit/box/style (verified this session). + +* Risks, Rabbit Holes, and Drawbacks +- Poor default colors fight unset themes. Dodge: seed conservatively, lean on theme override. +- Load order: the faces must be defined before org renders the agenda. Dodge: eager defface, require before/with org. +- Scope creep into the dupre/auto-dim migration. Dodge: it's explicitly vNext with its own decision. + +* Review and iteration history +** 2026-06-15 Mon @ 01:51:57 -0500 — Craig — author +- What: initial draft. +- Why: the A-D priority request generalized into a clearly-custom, theme-studio-surfaced header-row face layer; the prefix, default-color policy, and dupre/auto-dim reconciliation are real trade-offs worth settling on paper first. +- Artifacts: [[file:../../todo.org][todo.org org-faces task]]; theme-studio bespoke-app machinery (face_data.py, generate.py, app.js). diff --git a/docs/specs/signal-client-spec-doing.org b/docs/specs/signal-client-spec-doing.org deleted file mode 100644 index beee0acf..00000000 --- a/docs/specs/signal-client-spec-doing.org +++ /dev/null @@ -1,254 +0,0 @@ -:PROPERTIES: -:ID: 0cabd6ee-c458-47b5-a8af-3ee054b25821 -:STATUS: doing -:END: -#+TITLE: Design: Signal client in Emacs (forked signel) -#+DATE: 2026-05-26 -#+STATUS: Draft - -* Problem -I want a Signal chat client inside Emacs: link it as a secondary device to my phone, pick a contact from my contact list, hold a text 1:1 conversation (read and send), and get a desktop notification on incoming messages, with an optional sound. Signal has no official API, so this is built on =signal-cli=, the mature headless CLI, driven over JSON-RPC. - -* Non-Goals -- Groups, attachments, stickers, reactions, read receipts, typing indicators in the first version (text 1:1 only). The fork base already supports several of these, so they are deferred, not forbidden. -- Replacing the phone as primary. This is a *linked secondary device*, like Signal Desktop. -- Registering a phone number standalone. -- Notifying for the conversation I'm actively viewing. - -* Assumptions -- *Researched fact:* signal-cli (AsamK) is mature, headless, and exposes JSON-RPC; it runs as =signal-cli -a ACCOUNT jsonRpc=. Source: https://github.com/AsamK/signal-cli -- *Researched fact:* signel (keenban) is GPL-3, single-file (642 lines), on MELPA, and already implements the signal-cli JSON-RPC process loop, a read-only chat buffer with guarded prompt, send, sync handling, media rendering, and an active-chats dashboard. Source: https://github.com/keenban/signel -- *Researched fact:* signel is stale — all 40 commits in a ~10-day burst in Jan 2026, nothing since 2026-02-04, an unattended issue tracker (5 open, filed Mar 2026, none answered), no PRs ever, ~26 stars. Highest-severity open bug is #2 (incoming messages clobber in-progress input); no crashes, no signal-cli incompatibility, no data loss. So: fork-and-own, not depend-and-track. -- *Researched fact:* signal-cli is AUR-only on Arch (=yay -S signal-cli=), needs a JRE (OpenJDK 17+, satisfied by the installed OpenJDK 26). -- *Assumption (to confirm before building):* signal-cli must be updated roughly every three months or Signal-Server rejects the client version. (Widely reported; confirm cadence against the project's NEWS when it bites.) -- *Researched fact:* signal-cli =listContacts= returns a contact list in a shape usable for a completing-read picker. Verified 2026-05-26 against the live linked account (94 real contacts; =cj/signal--parse-contacts= ERT-covered). - -* Approaches Considered - -** Recommended: fork signel into ~/code/signel and own it -Clone is already at =~/code/signel=. Wire via =use-package :load-path= like the org-drill and auto-dim-other-buffers forks. The clean 642-line core handles the hard plumbing; layer three focused changes plus integration on top. -- Pros: full control over the exact spec (contact picker, notify-when-not-viewing, sound toggle) in cj/ idioms; the hard JSON-RPC/receive/buffer/media work is already done; upstream is dead-quiet so there is no divergence cost to forking. -- Cons: own the maintenance (the signal-cli update treadmill, reconnect/resync) and signel's existing bugs. - -** Rejected: install signel from MELPA + advice the internals -=(use-package signel :ensure t)=, add the contact picker and link command as additive config, advice =signel--handle-receive= for the notify behavior. -- Why not: the notification change and the #2 input-clobber fix are internal edits; advising them is fragile and ugly. With upstream dead, forking loses nothing and keeps those edits clean. - -** Rejected: custom Emacs client from scratch on signal-cli -- Why not: rewrites the JSON-RPC loop, buffer management, and media that signel already does cleanly. "Read signel as reference then retype it" is forking with extra steps. - -** Rejected: signal-cli-rest-api (Docker) -- Why not: a Docker dependency for a personal Emacs feature is heavy; two moving parts instead of one daemon. - -** Rejected (tail): Signal-as-MCP-tool via gptel -- Why not: agent-mediated messaging, not a chat client; undershoots "pick a contact and chat"; foxl-ai MCP server is v0.1.1 and unproven. - -** Rejected (tail): bridge to ERC via a Signal↔IRC gateway -- Why not: a second daemon plus a bridge to keep alive; double the breakage surface; bridge maturity unverified. - -** Rejected (tail): org-backed (receive-hook writes per-contact org) -- Why not: org is not a live chat surface; reframes the picked option into note-taking. - -* Design - -** Fork integration -- Fork lives at =~/code/signel= (already cloned). New module =modules/signal-config.el= wires it with =use-package signel :load-path "~/code/signel" :ensure nil=, mirroring the org-drill and auto-dim forks. -- Keybindings under a dedicated prefix (candidate =C-; M= for Messages, since =C-; S= is Slack). Commands: start/link, contact picker, dashboard, toggle sound. -- =signel-account= set from a defcustom or authinfo, not hardcoded. - -** Three changes on top of the fork -1. *Contact picker.* New command =cj/signel-pick-contact= (or rename signel's =signel-chat=): call signal-cli =listContacts= over JSON-RPC, cache name→number, present a =completing-read= of names, open the chat buffer for the chosen contact. signel today opens by raw phone number and only lists chats that already received a message. -2. *Linking / auth.* New command =cj/signel-link= wrapping =signal-cli link -n "Emacs"=, capturing the =tsdevice:= URI and rendering it as a scannable QR (via =qrencode= to an image buffer, or a CLI QR) so the phone's Linked Devices can scan it. signel assumes an already-linked account. -3. *Notification behavior.* Edit =signel--handle-receive='s notify block: (a) suppress the notification when the message's chat buffer is the selected window's buffer (actively viewing); (b) route through Craig's =notify= script instead of bare =notifications-notify=; (c) sound off by default behind a defcustom toggle (=cj/signel-notify-sound=, default nil). - -** Folded-in upstream fix -- Issue #2 (incoming messages clobber in-progress input): the redraw in =signel--insert-msg= / =signel--draw-prompt= replaces the prompt region while the user may be mid-type. Preserve and restore any unsent input across the insert. Fix it in the fork since it sits right next to the notification edit. - -** Data flow -signal-cli (linked secondary device) ⇄ JSON-RPC over the subprocess stdio ⇄ signel process filter → dispatch → receive handler → chat buffer insert + notify. Send: chat-buffer prompt → =send= RPC. No persistence beyond what signal-cli stores; Emacs holds session state (contact cache, active chats) in memory. - -** Error handling -- signal-cli not installed / not linked → =user-error= with the remedy (install, or =cj/signel-link=). signel already guards the missing executable and unset account. -- RPC errors map to the originating chat buffer (signel already does this). -- Process death → sentinel logs; add a visible message and a restart hint. - -** Testing -- Pure helpers (contact-list parsing from a fixture JSON, the notify-suppression predicate given a buffer/window state, the input-preserve logic) get ERT unit tests with mocked signal-cli output — no live account needed. -- The live loop (link, receive, send, notify) is verified manually against a linked account (scripted manual checklist), since it needs the phone and a real signal-cli. - -** Observability -- signel already logs RPC traffic to =*signel-log*=. Keep it; it's the diagnostic surface for the update-treadmill breakages. - -* Open Questions -- [ ] Fork-vs-MELPA+advice is decided (fork). Record as an ADR via =arch-decide= if a formal record is wanted. -- [X] Keybinding prefix: =C-; M= (Messages). Decided 2026-05-27 (workflow spec D1). Leaf keys: =m= message, =s= self, =d= dashboard, =l= link, =q= stop, =SPC= connect. -- [X] Account source: defcustom in =signal-config.local.el= (=signel-account=, loaded by =cj/signal-private-config-file=). Decided 2026-05-27. The phone number is an identifier rather than a credential, so a gitignored local-config file is the right home (no GPG prompt at connect time, off the public mirror). -- [X] Fork remote: keep as a local checkout at =~/code/signel= for now. Decided 2026-05-27. Upstream is dead-quiet so there's no remote to track; revisit if/when divergence is large enough that a backup remote on cjennings.net adds value. - -* Next Steps -1. Install signal-cli: =yay -S signal-cli= (interactive, Craig). -2. Link as a secondary device (=cj/signel-link= once built, or =signal-cli link= directly) — scan the QR from the phone. -3. Implement on the fork against the live engine (TDD the pure helpers, manual-verify the live loop) via =/start-work=. -4. archsetup request to add signal-cli to the standard install — sent 2026-05-26. - -* Initiate-message workflow (spec — 2026-05-27) - -This section specs the two requests that matter most right now and the end goal that ties them together: - -1. Wire signel to keybindings. -2. A contact picker keyed by *name*, not phone number, so initiating a chat (including a message to self) is a pick-from-names action. - -End goal: invoke a key, pick a contact by name, land in the chat buffer, type, send — the whole flow intuitive and without rough edges. - -** Current state (what's already built) - -- =cj/signal--parse-contacts= turns signal-cli =listContacts= output into a sorted =(LABEL . RECIPIENT)= alist, where LABEL is "Name (recipient)". Unit-tested against all 94 real contacts. This is the data layer for the name-based picker — done. -- The notify-suppression helpers (=cj/signal--should-notify-p= and friends) and the fork wiring (=use-package signel=, private-config load) are in =modules/signal-config.el=. -- =signel-chat= (signel.el) opens a chat buffer for a recipient but prompts with raw =(interactive "sSignal Recipient (+Phone): ")= — typing a phone number. Replacing that prompt with a name pick is the core of request #2. - -** Happy path - -1. =C-; M m= (or chosen key) invokes =cj/signel-message=. -2. It ensures the daemon is connected, gets the contact list (cached), and runs =completing-read= over names, with "Note to Self" pinned first. -3. Pick a name → resolve to recipient → call =signel-chat=. -4. Chat buffer opens; type at the prompt; send. - -** Pieces to build - -In dependency order (the picker can't be built before the RPC result path exists — see Architecture additions below): - -1. *JSON-RPC success-result dispatch* (fork edit) — signel today routes only =receive= notifications and RPC errors; successful =((id . N) (result . VALUE))= responses have no path. Add a request-callback table and result routing. Everything else depends on this. -2. =cj/signel--ensure-started= — the daemon/link/account guard predicate. -3. =cj/signel--fetch-contacts= — issue =listContacts= via the new callback contract, feed the result through the existing parser, populate the cache. -4. =cj/signel--contact-cache= + =cj/signel-refresh-contacts= — cj-owned picker cache, separate from signel's receive-time map. -5. =cj/signel-message= — the interactive picker command wrapping =signel-chat=. -6. =cj/signel-message-self= — direct "Note to Self" command. -7. The signel =C-; M= prefix keymap. -8. The #2 input-clobber fix (fork edit) covering both =signel--insert-msg= and =signel--insert-system-msg=, since both delete from the prompt line through =point-max=. A mid-type send must survive an incoming message AND a system-error insertion. - -** Decisions (resolved 2026-05-27 — Craig accepted all recommendations) - -Each recommendation below stands as the accepted decision, including D5 (the input-clobber fix is in scope for this workflow). The Options/Why are kept as the record of what was weighed. - -*** D1 — Keymap prefix and layout -Options: -- (a) =C-; M= ("Messages"), per the original Design note — =C-; S= is Slack, =C-; M= is free. -- (b) =C-; G= ("siGnal"). -- (c) Fold into an existing comms prefix. - -Recommendation: (a) =C-; M=. Why: it's already reserved in the design note, "Messages" reads as the general intent (room to add other messaging later), and it dodges the Slack collision. Proposed leaf keys: =m= message (picker), =s= message-self, =d= dashboard, =l= link, =q= stop, =SPC= start/connect. (Final key list itself is low-stakes; the prefix is the real choice.) - -*** D2 — Contact-list freshness -Options: -- (a) Fetch live on every invocation. -- (b) Cache on first use, refresh with an explicit command, auto-invalidate on (re)connect. -- (c) Cache with a TTL. - -Recommendation: (b). Why: =listContacts= over the RPC isn't instant, and "intuitive" means the picker pops immediately. Cache-plus-explicit-refresh keeps it snappy and predictable; invalidating on connect covers the "I added a contact on my phone" case without a guessed TTL. A =cj/signel-refresh-contacts= command (bound under the prefix) handles the rare staleness. - -*** D3 — Message-to-self affordance -Options: -- (a) Pin "Note to Self" as the first entry in the picker. -- (b) A dedicated =cj/signel-message-self= command on its own key. -- (c) Both. - -Recommendation: (c) both. Why: message-to-self is a distinct, frequent intent (it's how you use Signal as a personal scratchpad), so a direct key is the fast path; the pinned picker entry covers discoverability for when you're already in the picker. Low cost to do both since both resolve to the same account recipient. - -*** D4 — Daemon not connected -Options: -- (a) Auto-start/connect the daemon, then proceed. -- (b) Prompt "Signal isn't connected — connect now?" then proceed. -- (c) =user-error= with a hint to run start/link. - -Recommendation: (a) when an account is linked, falling back to (c) when it isn't. Why: "intuitive" means the picker just works when you're set up, so auto-connecting on first use removes a manual step; but the client can't fabricate a link, so an unlinked state has to point you at =cj/signel-link= rather than hang. - -*** D5 — Is the input-clobber bug (#2) in scope here? -Options: -- (a) Fix it as part of this workflow. -- (b) Track it separately, ship the picker + keymap first. - -Recommendation: (a) in scope. Why: your stated bar is "send a message without issues," and the clobber bug corrupts in-progress input the moment a message arrives mid-type — that is the send flow failing. The fork already plans this fix (Design → Folded-in upstream fix), and it sits right next to the notify edit. Shipping the picker while the clobber remains would meet the letter of request #2 but miss the end goal. - -*** D6 — 1:1 only, or groups in the picker? -Options: -- (a) 1:1 contacts only for now. -- (b) Include groups in the same picker. - -Recommendation: (a) 1:1 only. Why: groups are an explicit Non-Goal for v1, and =listContacts= is the 1:1 source; pulling groups in means a second RPC (=listGroups=) and merged labels. Defer to a follow-up, consistent with the rest of the spec. - -** Architecture additions (resolving the 2026-05-27 review blockers) - -The Codex review (=docs/design/signal-client-review.org=) found the workflow above hid three unspecified architecture decisions. Confirmed against the fork: =signel--dispatch= (signel.el:230) handles only =receive= and =error=; a successful =result= response is dropped, and =signel--send-rpc= maps request IDs to buffers for error display only. These resolve those gaps so the build isn't inventing contracts midstream. - -*** JSON-RPC result path (blocker 1) -The picker needs a value back from =listContacts=, which the fork can't currently deliver. -- Add =signel--request-handler-map=, a hash keyed by JSON-RPC id holding a success callback. -- Add =cj/signel--send-rpc-with-callback= (or extend =signel--send-rpc= with an optional success callback) that registers the callback under the request id. -- Extend =signel--dispatch= to route =((id . N) (result . VALUE))= to the registered callback, and to clean up the handler entry on success, on error, and on reconnect (so a dead request can't leak a stale callback). -- =cj/signel--fetch-contacts= consumes this: send =listContacts=, and in the callback parse + cache the result. Picker-facing failures surface as =user-error=; full RPC detail stays in =*signel-log*=. - -*** Daemon / link / account guard (blocker 2) -"Auto-connect when linked, =user-error= when not" needs a real definition of "linked" and of process death. -- =cj/signel--ensure-started= contract: - - Return normally when =(process-live-p (get-process signel--process-name))=. - - When =signel-account= is set but no live process exists, call =signel-start=. - - When =signel-account= is nil, =user-error= with the exact remedy (set it in the private config, or run the future link command — linking is out of scope this pass and done manually for now). - - If startup exits before the first RPC response, fail with a message pointing at =*signel-stderr*= / =*signel-log*= and the manual-link remedy, rather than hanging or surfacing a raw process error. -- "Linked for v1" means: =signel-account= configured in =signal-config.local.el= AND =signal-cli -a ACCOUNT jsonRpc= starts a live process. The client does not separately prove the account is linked on the server; a not-actually-linked account fails at first RPC and routes through the startup-death message above. - -*** Contact cache ownership + invalidation (blocker 3) -- =cj/signel--contact-cache= holds the parsed =(LABEL . RECIPIENT)= picker alist, owned by =signal-config.el=, kept separate from signel's =signel--contact-map= (which is receive-time sender names, a different and noisier source). -- =cj/signel-refresh-contacts= clears and refetches it. -- Auto-invalidate on reconnect by clearing =cj/signel--contact-cache= in the same wrapper/fork edit that starts or restarts the signel process. -- An empty success result ("No Signal contacts returned") is a distinct, user-facing message from an RPC/startup failure; the two must not collapse into the same error. - -*** Note-to-Self recipient (medium) -- v1 resolves "Note to Self" as =signel-chat= / =send= to =signel-account= (the linked number). No special-casing beyond pinning the picker entry and the direct command. -- Manual-verify: sending to =signel-account= lands in the Signal Note-to-Self thread, not as a self-addressed display anomaly. - -*** Synchronous picker over asynchronous fetch (final blocker — resolved 2026-05-27) -=completing-read= is synchronous; =cj/signel--fetch-contacts= is asynchronous via the callback table. On a cold cache the picker has to bridge that gap mid-call. Resolved via pre-warm + bounded block: -- =cj/signel--ensure-started= triggers a background fetch on connect / restart. The fetch's callback populates =cj/signel--contact-cache=; no user-visible step. -- =cj/signel-message= opens =completing-read= immediately when the cache is non-empty. On a cold cache (pre-warm hasn't returned yet), the command kicks off a fetch and calls =accept-process-output= with a bounded timeout (default 3s, =cj/signel-fetch-timeout= defcustom). On result, the picker opens. On timeout, =user-error= "Signal contact fetch timed out — try again, or refresh with =M-x cj/signel-refresh-contacts=" and point at =*signel-log*= for detail. -- Why this shape: warm cache is the common path so the picker feels instant; cold path still completes without a two-step "fetching… try again" UX; the timeout prevents a dead or wedged daemon from hanging Emacs. - -*** Caveats accepted (state at build time, none blocking) -- *JSON-RPC result envelope* — JSON-RPC 2.0 success is =((jsonrpc . "2.0") (id . N) (result . VALUE))=. The parser was verified on a real =listContacts= return on the live linked account, so the envelope keying is observed-correct in practice. Confirm against the next live response when the dispatch lands. -- *Diagnostic logging stance* — =*signel-log*= (signel's existing log) carries RPC traffic, which includes contact names/numbers and message text. Single-user local setup, log lives on disk under Emacs's control: accept-and-state, no redaction beyond what signel already does. Revisit if the log ever gets synced off-machine or the threat model widens. -- *Keymap conflict check* — before binding =C-; M=, verify it's unbound on the global =C-;= map at wiring time. The global =C-;= map is owned by =keybindings.el= (=cj/custom-keymap=); a quick =(keymap-lookup cj/custom-keymap "M")= during the keymap step is enough. - -** Testing - -Unit-testable without a live account (TDD these): the result-dispatch routing (a =result= response with a registered id invokes the callback; handler cleaned up on success/error; an unknown id is a no-op), the live-fetch result handling (mocked RPC JSON → parser, already covered for parsing itself), recipient resolution from a picked label, the note-to-self recipient, the daemon-state guard predicate (=cj/signel--ensure-started= branches: live process, account-set-no-process, account-nil), cache invalidation (refresh clears; empty result vs failure produce distinct outcomes), and *prompt-input preservation across both =signel--insert-msg= and =signel--insert-system-msg=* (regression for the #2 clobber fix and the system-error insertion path). Manual checklist against the linked account: the actual pick → open → type → send round-trip, the clobber fix under a real incoming message, the clobber fix under a real system-error insertion, auto-connect on first use, and that Note-to-Self lands in the right thread. This mirrors the Testing section above (pure helpers ERT, live loop manual). - -** Scope summary - -In scope: =cj/signel-message=, =cj/signel-message-self=, =cj/signel--fetch-contacts=, =cj/signel-refresh-contacts=, the JSON-RPC result-dispatch fork edit, =cj/signel--ensure-started=, the cj-owned contact cache + pre-warm, the =C-; M= keymap, and the #2 clobber fix. Out of scope for this pass: linking/QR (=cj/signel-link=, separate request), groups, and the colon-alignment-style polish. Linking is assumed already done manually for the workflow to be exercised. - -Notification-slice forward-flag: the existing Design notes route notifications through Craig's =notify= script with an optional sound, but the slice-level details — exact =notify= command shape, fallback when =notify= is missing, body truncation, and whether Signal message text is shown verbatim in desktop notifications — are not specified here. Before the notification slice starts, add a short subsection to this spec naming those four. Not in scope for the initiate-message workflow because the notify-suppression predicates already exist and the notification edit isn't on the build path for the picker. - -** Readiness rubric - -*Ready* (2026-05-27 spec-review). All three Codex blockers folded in (Architecture additions); the final sync/async decision resolved as pre-warm + bounded block; three minor caveats stated for build time, none blocking. Implementation order follows the Pieces-to-build list — the JSON-RPC result-dispatch fork edit is step 1, everything else builds on it. - -* Notification slice (spec addendum — 2026-06-11) - -Resolves the four details the forward-flag in Scope summary required before this slice starts. Craig accepted all four recommendations 2026-06-11. - -** The four decisions - -1. *Command shape.* =notify info "Signal: " ""=, launched via =start-process= so the receive filter never blocks on the script. Type =info= (blue icon, confident tone). No =--persist=: messages arrive as a stream and persistent toasts pile up; a missed toast is still in the chat buffer, unlike an alarm. Sound follows =cj/signel-notify-sound= (defcustom, default nil): pass =--silent= unless it is non-nil. -2. *Fallback when =notify= is missing.* Warn once at module load via =cj/executable-find-or-warn= (the established external-tool pattern), and at runtime fall back to the fork's =notifications-notify= call so an incoming message is never silently dropped on a machine without the script. -3. *Body truncation.* Collapse whitespace runs (including newlines) to single spaces and truncate to 120 characters with an ellipsis. Long messages flood the toast and the daemon clips unpredictably; the full text is always in the chat buffer. -4. *Verbatim text.* Show the (truncated) message text verbatim. Same accept-and-state stance as the =*signel-log*= caveat — single-user local machine. Revisit if notifications ever route off-machine. - -** Wiring architecture - -The fork stays generic; the policy lives in =signal-config.el=: - -- *Fork edit* — =signel.el= gains =signel-notify-function= (defvar, default =signel--notify-default= which preserves today's bare =notifications-notify= behavior). =signel--handle-receive='s notify block becomes =(funcall signel-notify-function chat-id sender notify-body)= — chat-id added so a custom function can make per-chat decisions. The fork never references cj/ symbols and remains loadable standalone. -- *cj layer* — =cj/signel--notify= (set as =signel-notify-function= in the use-package =:config=) gates on =cj/signal--should-notify-p= (the existing suppression predicate), formats the body via a pure helper (=cj/signal--format-notify-body=: whitespace collapse + 120-char truncation), and routes: =notify= script when =executable-find= sees it, =notifications-notify= fallback otherwise. - -** Testing - -ERT, no live account: the body formatter (Normal/Boundary — passthrough, newline collapse, exact-limit, over-limit + ellipsis, empty, unicode); =cj/signel--notify= routing (suppressed when the predicate says no; script path args including =--silent= by default and its absence when sound is enabled; fallback path when the script is missing); the fork dispatch (a text message calls =signel-notify-function= with chat-id/sender/body; sticker → "[Sticker]"; attachment → "[Attachment]"; default function preserved). Manual: a real incoming message raises the toast through the script; no toast while viewing that chat in a focused frame. diff --git a/docs/specs/signal-client-spec.org b/docs/specs/signal-client-spec.org new file mode 100644 index 00000000..13f67115 --- /dev/null +++ b/docs/specs/signal-client-spec.org @@ -0,0 +1,257 @@ +#+TITLE: Design: Signal client in Emacs (forked signel) +#+DATE: 2026-05-26 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Design: Signal client in Emacs (forked signel) +:PROPERTIES: +:ID: 0cabd6ee-c458-47b5-a8af-3ee054b25821 +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DOING from existing :STATUS: doing + +* Problem +I want a Signal chat client inside Emacs: link it as a secondary device to my phone, pick a contact from my contact list, hold a text 1:1 conversation (read and send), and get a desktop notification on incoming messages, with an optional sound. Signal has no official API, so this is built on =signal-cli=, the mature headless CLI, driven over JSON-RPC. + +* Non-Goals +- Groups, attachments, stickers, reactions, read receipts, typing indicators in the first version (text 1:1 only). The fork base already supports several of these, so they are deferred, not forbidden. +- Replacing the phone as primary. This is a *linked secondary device*, like Signal Desktop. +- Registering a phone number standalone. +- Notifying for the conversation I'm actively viewing. + +* Assumptions +- *Researched fact:* signal-cli (AsamK) is mature, headless, and exposes JSON-RPC; it runs as =signal-cli -a ACCOUNT jsonRpc=. Source: https://github.com/AsamK/signal-cli +- *Researched fact:* signel (keenban) is GPL-3, single-file (642 lines), on MELPA, and already implements the signal-cli JSON-RPC process loop, a read-only chat buffer with guarded prompt, send, sync handling, media rendering, and an active-chats dashboard. Source: https://github.com/keenban/signel +- *Researched fact:* signel is stale — all 40 commits in a ~10-day burst in Jan 2026, nothing since 2026-02-04, an unattended issue tracker (5 open, filed Mar 2026, none answered), no PRs ever, ~26 stars. Highest-severity open bug is #2 (incoming messages clobber in-progress input); no crashes, no signal-cli incompatibility, no data loss. So: fork-and-own, not depend-and-track. +- *Researched fact:* signal-cli is AUR-only on Arch (=yay -S signal-cli=), needs a JRE (OpenJDK 17+, satisfied by the installed OpenJDK 26). +- *Assumption (to confirm before building):* signal-cli must be updated roughly every three months or Signal-Server rejects the client version. (Widely reported; confirm cadence against the project's NEWS when it bites.) +- *Researched fact:* signal-cli =listContacts= returns a contact list in a shape usable for a completing-read picker. Verified 2026-05-26 against the live linked account (94 real contacts; =cj/signal--parse-contacts= ERT-covered). + +* Approaches Considered + +** Recommended: fork signel into ~/code/signel and own it +Clone is already at =~/code/signel=. Wire via =use-package :load-path= like the org-drill and auto-dim-other-buffers forks. The clean 642-line core handles the hard plumbing; layer three focused changes plus integration on top. +- Pros: full control over the exact spec (contact picker, notify-when-not-viewing, sound toggle) in cj/ idioms; the hard JSON-RPC/receive/buffer/media work is already done; upstream is dead-quiet so there is no divergence cost to forking. +- Cons: own the maintenance (the signal-cli update treadmill, reconnect/resync) and signel's existing bugs. + +** Rejected: install signel from MELPA + advice the internals +=(use-package signel :ensure t)=, add the contact picker and link command as additive config, advice =signel--handle-receive= for the notify behavior. +- Why not: the notification change and the #2 input-clobber fix are internal edits; advising them is fragile and ugly. With upstream dead, forking loses nothing and keeps those edits clean. + +** Rejected: custom Emacs client from scratch on signal-cli +- Why not: rewrites the JSON-RPC loop, buffer management, and media that signel already does cleanly. "Read signel as reference then retype it" is forking with extra steps. + +** Rejected: signal-cli-rest-api (Docker) +- Why not: a Docker dependency for a personal Emacs feature is heavy; two moving parts instead of one daemon. + +** Rejected (tail): Signal-as-MCP-tool via gptel +- Why not: agent-mediated messaging, not a chat client; undershoots "pick a contact and chat"; foxl-ai MCP server is v0.1.1 and unproven. + +** Rejected (tail): bridge to ERC via a Signal↔IRC gateway +- Why not: a second daemon plus a bridge to keep alive; double the breakage surface; bridge maturity unverified. + +** Rejected (tail): org-backed (receive-hook writes per-contact org) +- Why not: org is not a live chat surface; reframes the picked option into note-taking. + +* Design + +** Fork integration +- Fork lives at =~/code/signel= (already cloned). New module =modules/signal-config.el= wires it with =use-package signel :load-path "~/code/signel" :ensure nil=, mirroring the org-drill and auto-dim forks. +- Keybindings under a dedicated prefix (candidate =C-; M= for Messages, since =C-; S= is Slack). Commands: start/link, contact picker, dashboard, toggle sound. +- =signel-account= set from a defcustom or authinfo, not hardcoded. + +** Three changes on top of the fork +1. *Contact picker.* New command =cj/signel-pick-contact= (or rename signel's =signel-chat=): call signal-cli =listContacts= over JSON-RPC, cache name→number, present a =completing-read= of names, open the chat buffer for the chosen contact. signel today opens by raw phone number and only lists chats that already received a message. +2. *Linking / auth.* New command =cj/signel-link= wrapping =signal-cli link -n "Emacs"=, capturing the =tsdevice:= URI and rendering it as a scannable QR (via =qrencode= to an image buffer, or a CLI QR) so the phone's Linked Devices can scan it. signel assumes an already-linked account. +3. *Notification behavior.* Edit =signel--handle-receive='s notify block: (a) suppress the notification when the message's chat buffer is the selected window's buffer (actively viewing); (b) route through Craig's =notify= script instead of bare =notifications-notify=; (c) sound off by default behind a defcustom toggle (=cj/signel-notify-sound=, default nil). + +** Folded-in upstream fix +- Issue #2 (incoming messages clobber in-progress input): the redraw in =signel--insert-msg= / =signel--draw-prompt= replaces the prompt region while the user may be mid-type. Preserve and restore any unsent input across the insert. Fix it in the fork since it sits right next to the notification edit. + +** Data flow +signal-cli (linked secondary device) ⇄ JSON-RPC over the subprocess stdio ⇄ signel process filter → dispatch → receive handler → chat buffer insert + notify. Send: chat-buffer prompt → =send= RPC. No persistence beyond what signal-cli stores; Emacs holds session state (contact cache, active chats) in memory. + +** Error handling +- signal-cli not installed / not linked → =user-error= with the remedy (install, or =cj/signel-link=). signel already guards the missing executable and unset account. +- RPC errors map to the originating chat buffer (signel already does this). +- Process death → sentinel logs; add a visible message and a restart hint. + +** Testing +- Pure helpers (contact-list parsing from a fixture JSON, the notify-suppression predicate given a buffer/window state, the input-preserve logic) get ERT unit tests with mocked signal-cli output — no live account needed. +- The live loop (link, receive, send, notify) is verified manually against a linked account (scripted manual checklist), since it needs the phone and a real signal-cli. + +** Observability +- signel already logs RPC traffic to =*signel-log*=. Keep it; it's the diagnostic surface for the update-treadmill breakages. + +* Open Questions +- [ ] Fork-vs-MELPA+advice is decided (fork). Record as an ADR via =arch-decide= if a formal record is wanted. +- [X] Keybinding prefix: =C-; M= (Messages). Decided 2026-05-27 (workflow spec D1). Leaf keys: =m= message, =s= self, =d= dashboard, =l= link, =q= stop, =SPC= connect. +- [X] Account source: defcustom in =signal-config.local.el= (=signel-account=, loaded by =cj/signal-private-config-file=). Decided 2026-05-27. The phone number is an identifier rather than a credential, so a gitignored local-config file is the right home (no GPG prompt at connect time, off the public mirror). +- [X] Fork remote: keep as a local checkout at =~/code/signel= for now. Decided 2026-05-27. Upstream is dead-quiet so there's no remote to track; revisit if/when divergence is large enough that a backup remote on cjennings.net adds value. + +* Next Steps +1. Install signal-cli: =yay -S signal-cli= (interactive, Craig). +2. Link as a secondary device (=cj/signel-link= once built, or =signal-cli link= directly) — scan the QR from the phone. +3. Implement on the fork against the live engine (TDD the pure helpers, manual-verify the live loop) via =/start-work=. +4. archsetup request to add signal-cli to the standard install — sent 2026-05-26. + +* Initiate-message workflow (spec — 2026-05-27) + +This section specs the two requests that matter most right now and the end goal that ties them together: + +1. Wire signel to keybindings. +2. A contact picker keyed by *name*, not phone number, so initiating a chat (including a message to self) is a pick-from-names action. + +End goal: invoke a key, pick a contact by name, land in the chat buffer, type, send — the whole flow intuitive and without rough edges. + +** Current state (what's already built) + +- =cj/signal--parse-contacts= turns signal-cli =listContacts= output into a sorted =(LABEL . RECIPIENT)= alist, where LABEL is "Name (recipient)". Unit-tested against all 94 real contacts. This is the data layer for the name-based picker — done. +- The notify-suppression helpers (=cj/signal--should-notify-p= and friends) and the fork wiring (=use-package signel=, private-config load) are in =modules/signal-config.el=. +- =signel-chat= (signel.el) opens a chat buffer for a recipient but prompts with raw =(interactive "sSignal Recipient (+Phone): ")= — typing a phone number. Replacing that prompt with a name pick is the core of request #2. + +** Happy path + +1. =C-; M m= (or chosen key) invokes =cj/signel-message=. +2. It ensures the daemon is connected, gets the contact list (cached), and runs =completing-read= over names, with "Note to Self" pinned first. +3. Pick a name → resolve to recipient → call =signel-chat=. +4. Chat buffer opens; type at the prompt; send. + +** Pieces to build + +In dependency order (the picker can't be built before the RPC result path exists — see Architecture additions below): + +1. *JSON-RPC success-result dispatch* (fork edit) — signel today routes only =receive= notifications and RPC errors; successful =((id . N) (result . VALUE))= responses have no path. Add a request-callback table and result routing. Everything else depends on this. +2. =cj/signel--ensure-started= — the daemon/link/account guard predicate. +3. =cj/signel--fetch-contacts= — issue =listContacts= via the new callback contract, feed the result through the existing parser, populate the cache. +4. =cj/signel--contact-cache= + =cj/signel-refresh-contacts= — cj-owned picker cache, separate from signel's receive-time map. +5. =cj/signel-message= — the interactive picker command wrapping =signel-chat=. +6. =cj/signel-message-self= — direct "Note to Self" command. +7. The signel =C-; M= prefix keymap. +8. The #2 input-clobber fix (fork edit) covering both =signel--insert-msg= and =signel--insert-system-msg=, since both delete from the prompt line through =point-max=. A mid-type send must survive an incoming message AND a system-error insertion. + +** Decisions (resolved 2026-05-27 — Craig accepted all recommendations) + +Each recommendation below stands as the accepted decision, including D5 (the input-clobber fix is in scope for this workflow). The Options/Why are kept as the record of what was weighed. + +*** D1 — Keymap prefix and layout +Options: +- (a) =C-; M= ("Messages"), per the original Design note — =C-; S= is Slack, =C-; M= is free. +- (b) =C-; G= ("siGnal"). +- (c) Fold into an existing comms prefix. + +Recommendation: (a) =C-; M=. Why: it's already reserved in the design note, "Messages" reads as the general intent (room to add other messaging later), and it dodges the Slack collision. Proposed leaf keys: =m= message (picker), =s= message-self, =d= dashboard, =l= link, =q= stop, =SPC= start/connect. (Final key list itself is low-stakes; the prefix is the real choice.) + +*** D2 — Contact-list freshness +Options: +- (a) Fetch live on every invocation. +- (b) Cache on first use, refresh with an explicit command, auto-invalidate on (re)connect. +- (c) Cache with a TTL. + +Recommendation: (b). Why: =listContacts= over the RPC isn't instant, and "intuitive" means the picker pops immediately. Cache-plus-explicit-refresh keeps it snappy and predictable; invalidating on connect covers the "I added a contact on my phone" case without a guessed TTL. A =cj/signel-refresh-contacts= command (bound under the prefix) handles the rare staleness. + +*** D3 — Message-to-self affordance +Options: +- (a) Pin "Note to Self" as the first entry in the picker. +- (b) A dedicated =cj/signel-message-self= command on its own key. +- (c) Both. + +Recommendation: (c) both. Why: message-to-self is a distinct, frequent intent (it's how you use Signal as a personal scratchpad), so a direct key is the fast path; the pinned picker entry covers discoverability for when you're already in the picker. Low cost to do both since both resolve to the same account recipient. + +*** D4 — Daemon not connected +Options: +- (a) Auto-start/connect the daemon, then proceed. +- (b) Prompt "Signal isn't connected — connect now?" then proceed. +- (c) =user-error= with a hint to run start/link. + +Recommendation: (a) when an account is linked, falling back to (c) when it isn't. Why: "intuitive" means the picker just works when you're set up, so auto-connecting on first use removes a manual step; but the client can't fabricate a link, so an unlinked state has to point you at =cj/signel-link= rather than hang. + +*** D5 — Is the input-clobber bug (#2) in scope here? +Options: +- (a) Fix it as part of this workflow. +- (b) Track it separately, ship the picker + keymap first. + +Recommendation: (a) in scope. Why: your stated bar is "send a message without issues," and the clobber bug corrupts in-progress input the moment a message arrives mid-type — that is the send flow failing. The fork already plans this fix (Design → Folded-in upstream fix), and it sits right next to the notify edit. Shipping the picker while the clobber remains would meet the letter of request #2 but miss the end goal. + +*** D6 — 1:1 only, or groups in the picker? +Options: +- (a) 1:1 contacts only for now. +- (b) Include groups in the same picker. + +Recommendation: (a) 1:1 only. Why: groups are an explicit Non-Goal for v1, and =listContacts= is the 1:1 source; pulling groups in means a second RPC (=listGroups=) and merged labels. Defer to a follow-up, consistent with the rest of the spec. + +** Architecture additions (resolving the 2026-05-27 review blockers) + +The Codex review (=docs/design/signal-client-review.org=) found the workflow above hid three unspecified architecture decisions. Confirmed against the fork: =signel--dispatch= (signel.el:230) handles only =receive= and =error=; a successful =result= response is dropped, and =signel--send-rpc= maps request IDs to buffers for error display only. These resolve those gaps so the build isn't inventing contracts midstream. + +*** JSON-RPC result path (blocker 1) +The picker needs a value back from =listContacts=, which the fork can't currently deliver. +- Add =signel--request-handler-map=, a hash keyed by JSON-RPC id holding a success callback. +- Add =cj/signel--send-rpc-with-callback= (or extend =signel--send-rpc= with an optional success callback) that registers the callback under the request id. +- Extend =signel--dispatch= to route =((id . N) (result . VALUE))= to the registered callback, and to clean up the handler entry on success, on error, and on reconnect (so a dead request can't leak a stale callback). +- =cj/signel--fetch-contacts= consumes this: send =listContacts=, and in the callback parse + cache the result. Picker-facing failures surface as =user-error=; full RPC detail stays in =*signel-log*=. + +*** Daemon / link / account guard (blocker 2) +"Auto-connect when linked, =user-error= when not" needs a real definition of "linked" and of process death. +- =cj/signel--ensure-started= contract: + - Return normally when =(process-live-p (get-process signel--process-name))=. + - When =signel-account= is set but no live process exists, call =signel-start=. + - When =signel-account= is nil, =user-error= with the exact remedy (set it in the private config, or run the future link command — linking is out of scope this pass and done manually for now). + - If startup exits before the first RPC response, fail with a message pointing at =*signel-stderr*= / =*signel-log*= and the manual-link remedy, rather than hanging or surfacing a raw process error. +- "Linked for v1" means: =signel-account= configured in =signal-config.local.el= AND =signal-cli -a ACCOUNT jsonRpc= starts a live process. The client does not separately prove the account is linked on the server; a not-actually-linked account fails at first RPC and routes through the startup-death message above. + +*** Contact cache ownership + invalidation (blocker 3) +- =cj/signel--contact-cache= holds the parsed =(LABEL . RECIPIENT)= picker alist, owned by =signal-config.el=, kept separate from signel's =signel--contact-map= (which is receive-time sender names, a different and noisier source). +- =cj/signel-refresh-contacts= clears and refetches it. +- Auto-invalidate on reconnect by clearing =cj/signel--contact-cache= in the same wrapper/fork edit that starts or restarts the signel process. +- An empty success result ("No Signal contacts returned") is a distinct, user-facing message from an RPC/startup failure; the two must not collapse into the same error. + +*** Note-to-Self recipient (medium) +- v1 resolves "Note to Self" as =signel-chat= / =send= to =signel-account= (the linked number). No special-casing beyond pinning the picker entry and the direct command. +- Manual-verify: sending to =signel-account= lands in the Signal Note-to-Self thread, not as a self-addressed display anomaly. + +*** Synchronous picker over asynchronous fetch (final blocker — resolved 2026-05-27) +=completing-read= is synchronous; =cj/signel--fetch-contacts= is asynchronous via the callback table. On a cold cache the picker has to bridge that gap mid-call. Resolved via pre-warm + bounded block: +- =cj/signel--ensure-started= triggers a background fetch on connect / restart. The fetch's callback populates =cj/signel--contact-cache=; no user-visible step. +- =cj/signel-message= opens =completing-read= immediately when the cache is non-empty. On a cold cache (pre-warm hasn't returned yet), the command kicks off a fetch and calls =accept-process-output= with a bounded timeout (default 3s, =cj/signel-fetch-timeout= defcustom). On result, the picker opens. On timeout, =user-error= "Signal contact fetch timed out — try again, or refresh with =M-x cj/signel-refresh-contacts=" and point at =*signel-log*= for detail. +- Why this shape: warm cache is the common path so the picker feels instant; cold path still completes without a two-step "fetching… try again" UX; the timeout prevents a dead or wedged daemon from hanging Emacs. + +*** Caveats accepted (state at build time, none blocking) +- *JSON-RPC result envelope* — JSON-RPC 2.0 success is =((jsonrpc . "2.0") (id . N) (result . VALUE))=. The parser was verified on a real =listContacts= return on the live linked account, so the envelope keying is observed-correct in practice. Confirm against the next live response when the dispatch lands. +- *Diagnostic logging stance* — =*signel-log*= (signel's existing log) carries RPC traffic, which includes contact names/numbers and message text. Single-user local setup, log lives on disk under Emacs's control: accept-and-state, no redaction beyond what signel already does. Revisit if the log ever gets synced off-machine or the threat model widens. +- *Keymap conflict check* — before binding =C-; M=, verify it's unbound on the global =C-;= map at wiring time. The global =C-;= map is owned by =keybindings.el= (=cj/custom-keymap=); a quick =(keymap-lookup cj/custom-keymap "M")= during the keymap step is enough. + +** Testing + +Unit-testable without a live account (TDD these): the result-dispatch routing (a =result= response with a registered id invokes the callback; handler cleaned up on success/error; an unknown id is a no-op), the live-fetch result handling (mocked RPC JSON → parser, already covered for parsing itself), recipient resolution from a picked label, the note-to-self recipient, the daemon-state guard predicate (=cj/signel--ensure-started= branches: live process, account-set-no-process, account-nil), cache invalidation (refresh clears; empty result vs failure produce distinct outcomes), and *prompt-input preservation across both =signel--insert-msg= and =signel--insert-system-msg=* (regression for the #2 clobber fix and the system-error insertion path). Manual checklist against the linked account: the actual pick → open → type → send round-trip, the clobber fix under a real incoming message, the clobber fix under a real system-error insertion, auto-connect on first use, and that Note-to-Self lands in the right thread. This mirrors the Testing section above (pure helpers ERT, live loop manual). + +** Scope summary + +In scope: =cj/signel-message=, =cj/signel-message-self=, =cj/signel--fetch-contacts=, =cj/signel-refresh-contacts=, the JSON-RPC result-dispatch fork edit, =cj/signel--ensure-started=, the cj-owned contact cache + pre-warm, the =C-; M= keymap, and the #2 clobber fix. Out of scope for this pass: linking/QR (=cj/signel-link=, separate request), groups, and the colon-alignment-style polish. Linking is assumed already done manually for the workflow to be exercised. + +Notification-slice forward-flag: the existing Design notes route notifications through Craig's =notify= script with an optional sound, but the slice-level details — exact =notify= command shape, fallback when =notify= is missing, body truncation, and whether Signal message text is shown verbatim in desktop notifications — are not specified here. Before the notification slice starts, add a short subsection to this spec naming those four. Not in scope for the initiate-message workflow because the notify-suppression predicates already exist and the notification edit isn't on the build path for the picker. + +** Readiness rubric + +*Ready* (2026-05-27 spec-review). All three Codex blockers folded in (Architecture additions); the final sync/async decision resolved as pre-warm + bounded block; three minor caveats stated for build time, none blocking. Implementation order follows the Pieces-to-build list — the JSON-RPC result-dispatch fork edit is step 1, everything else builds on it. + +* Notification slice (spec addendum — 2026-06-11) + +Resolves the four details the forward-flag in Scope summary required before this slice starts. Craig accepted all four recommendations 2026-06-11. + +** The four decisions + +1. *Command shape.* =notify info "Signal: " ""=, launched via =start-process= so the receive filter never blocks on the script. Type =info= (blue icon, confident tone). No =--persist=: messages arrive as a stream and persistent toasts pile up; a missed toast is still in the chat buffer, unlike an alarm. Sound follows =cj/signel-notify-sound= (defcustom, default nil): pass =--silent= unless it is non-nil. +2. *Fallback when =notify= is missing.* Warn once at module load via =cj/executable-find-or-warn= (the established external-tool pattern), and at runtime fall back to the fork's =notifications-notify= call so an incoming message is never silently dropped on a machine without the script. +3. *Body truncation.* Collapse whitespace runs (including newlines) to single spaces and truncate to 120 characters with an ellipsis. Long messages flood the toast and the daemon clips unpredictably; the full text is always in the chat buffer. +4. *Verbatim text.* Show the (truncated) message text verbatim. Same accept-and-state stance as the =*signel-log*= caveat — single-user local machine. Revisit if notifications ever route off-machine. + +** Wiring architecture + +The fork stays generic; the policy lives in =signal-config.el=: + +- *Fork edit* — =signel.el= gains =signel-notify-function= (defvar, default =signel--notify-default= which preserves today's bare =notifications-notify= behavior). =signel--handle-receive='s notify block becomes =(funcall signel-notify-function chat-id sender notify-body)= — chat-id added so a custom function can make per-chat decisions. The fork never references cj/ symbols and remains loadable standalone. +- *cj layer* — =cj/signel--notify= (set as =signel-notify-function= in the use-package =:config=) gates on =cj/signal--should-notify-p= (the existing suppression predicate), formats the body via a pure helper (=cj/signal--format-notify-body=: whitespace collapse + 120-char truncation), and routes: =notify= script when =executable-find= sees it, =notifications-notify= fallback otherwise. + +** Testing + +ERT, no live account: the body formatter (Normal/Boundary — passthrough, newline collapse, exact-limit, over-limit + ellipsis, empty, unicode); =cj/signel--notify= routing (suppressed when the predicate says no; script path args including =--silent= by default and its absence when sound is enabled; fallback path when the script is missing); the fork dispatch (a text message calls =signel-notify-function= with chat-id/sender/body; sticker → "[Sticker]"; attachment → "[Attachment]"; default function preserved). Manual: a real incoming message raises the toast through the script; no toast while viewing that chat in a focused frame. diff --git a/docs/specs/theme-studio-completion-preview-spec.org b/docs/specs/theme-studio-completion-preview-spec.org index 588f35a9..7d0c2608 100644 --- a/docs/specs/theme-studio-completion-preview-spec.org +++ b/docs/specs/theme-studio-completion-preview-spec.org @@ -1,7 +1,14 @@ #+TITLE: Theme Studio Minibuffer-Completion Preview — Spec #+AUTHOR: Craig Jennings #+DATE: 2026-06-23 -#+TODO: TODO | DONE SUPERSEDED CANCELLED +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Theme Studio Minibuffer-Completion Preview — Spec +:PROPERTIES: +:ID: 2462f067-4c8d-4c33-a5be-54c0abc2eb1d +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DRAFT from Metadata Status: Not ready, review found blockers * Metadata | Status | Not ready — first Codex review found implementation-readiness blockers (2026-06-23) | @@ -195,7 +202,7 @@ Add the caption naming minibuffer-prompt + highlight as living in UI Faces. When - Manual: open theme-studio in Chrome on the owner's inventory and confirm the Vertico section + baseline render, orderless/marginalia toggle, and vertico-current shows no background. * References / Appendix -- Reuse: [[file:theme-studio-preview-locate-spec.org][theme-studio-preview-locate-spec.org]] (hover/click locate), [[file:theme-studio-package-faces-spec-doing.org][theme-studio-package-faces-spec-doing.org]]. +- Reuse: [[file:theme-studio-preview-locate-spec.org][theme-studio-preview-locate-spec.org]] (hover/click locate), [[file:theme-studio-package-faces-spec.org][theme-studio-package-faces-spec.org]]. - Spike: /tmp completion-face-preview.el (verified render; not committed — informs this spec, not grown into it). - Live face values captured 2026-06-23 (WIP theme): minibuffer-prompt #899bb1/#100f0f bold; orderless-match-face-0..3 #cbd0d6 / #c99990 / #c5d4ae / #bea9dc bold italic; vertico-current inherits highlight (#eddba7 bold, no background). diff --git a/docs/specs/theme-studio-nerd-icons-colors-spec.org b/docs/specs/theme-studio-nerd-icons-colors-spec.org index c0f07b6d..94a5d178 100644 --- a/docs/specs/theme-studio-nerd-icons-colors-spec.org +++ b/docs/specs/theme-studio-nerd-icons-colors-spec.org @@ -1,7 +1,14 @@ #+TITLE: Theme-driven nerd-icons colors + theme-studio filetype legend — Spec #+AUTHOR: Craig Jennings & Claude #+DATE: 2026-06-23 -#+TODO: TODO | DONE SUPERSEDED CANCELLED +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* READY Theme-driven nerd-icons colors + theme-studio filetype legend — Spec +:PROPERTIES: +:ID: 6df4e8a3-1fca-452a-9416-3fa0647b8dff +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword READY from Metadata Status: Ready pending Craig's go * Metadata | Status | Ready pending Craig's go — Codex review rounds 1-3 incorporated | diff --git a/docs/specs/theme-studio-package-faces-spec-doing.org b/docs/specs/theme-studio-package-faces-spec-doing.org deleted file mode 100644 index 566f34db..00000000 --- a/docs/specs/theme-studio-package-faces-spec-doing.org +++ /dev/null @@ -1,590 +0,0 @@ -:PROPERTIES: -:ID: 8f37a1fd-cfd3-4b25-92e5-772468092bdc -:STATUS: doing -:END: -#+TITLE: theme-studio — package faces (tier 3), starting with org-mode -#+AUTHOR: Craig Jennings -#+DATE: 2026-06-07 - -* Status - -Spec / Craig's first-round answers folded in (2026-06-07). Proposes a third tier -for the theme-studio (scripts/theme-studio/) that lets a theme colorize -package-specific faces, built one application at a time. v1 apps: org-mode -(incl. org-agenda), magit, elfeed. Codex review incorporated (2026-06-07): added -implementation phases, acceptance criteria, the package-face inventory source -(hybrid, split), and state/export semantics. Rubric now =Ready=. -All opens resolved (Craig, 2026-06-07/08): inheritance is modeled (show each -face's resolved color in the table + preview, override what looks bad); inventory -is hybrid-and-split (org/magit/elfeed bespoke first, generated all-package -inventory as a later phase); the custom color picker is built after tier 3. -Implementation tasks live in =todo.org=. - -* Background — the three tiers - -The theme-studio already models two tiers of faces: - -1. *Syntax* — the font-lock / tree-sitter categories (keyword, string, type, - comment, etc.), in the "code/color assignments" table. -2. *UI* — Emacs's built-in interface faces (cursor, region, mode-line, fringe, - line numbers, isearch, and the rest), in the "ui faces" table with the live - mock-frame preview. - -Tier 3 is *package faces*: faces a package declares with =defface= so a theme -can color the package as it wishes. The running config has 1,146 such faces -across 186 packages (magit 111, lsp-mode 97, telega 91, web-mode 82, org ~30 -core, and a long tail). No theme colors all of them; quality themes hand-pick -the packages the user actually lives in and theme those. - -This spec adds a tier-3 section to the tool, structured so applications are -added one at a time. org-mode ships first. - -* Goal - -A new "package faces" section with: - -1. An *application dropdown* — pick which package's faces to edit. v1 ships - org-mode (including org-agenda), magit, and elfeed; the rest of Craig's - packages (calibredb, ghostel, mu4e, IRC, org-drill, dirvish + dired, slack) - follow one at a time. -2. A *face table* for the selected app — one row per face in the app's complete - set, each with a foreground dropdown, a background dropdown, bold / italic - toggles, an optional inherit, and a relative-height stepper, all drawing from - the same palette as the other tables. Grouped, with a text filter for the - large apps. -3. A *preview pane* for the selected app — a realistic mock of that package - rendered with the live theme, the way the ui-faces mock-frame shows the UI - faces in a buffer. org-mode gets a mock org document. - -The export (=theme.json=) gains a =packages= object so the build step can set -these faces too. - -* UI placement - -A new top-level section under the ui-faces row: - -#+begin_example -

package faces

-[ application: (org-mode v) ] -
- left = the selected app's face table (fg / bg / B / I per face) - right = the selected app's preview pane (e.g. the org document mock) -
-#+end_example - -Same two-column stretch layout as the ui-faces row, so the preview matches the -table's height. - -* Data model - -A single data structure drives everything, keyed by application: - -#+begin_src js -APPS = { - "org-mode": { - label: "org-mode", - faces: [ - // face, human label, default {fg, bg, bold, italic} - ["org-document-title", "document title", {fg:"gold", bold:true}], - ["org-level-1", "heading 1", {fg:"blue", bold:true}], - ["org-level-2", "heading 2", {fg:"gold"}], - ["org-level-3", "heading 3", {fg:"regal"}], - ["org-todo", "TODO keyword", {fg:"terracotta", bold:true}], - ["org-done", "DONE keyword", {fg:"sage", bold:true}], - ["org-link", "link", {fg:"blue"}], // base `link` - ["org-code", "inline code", {fg:"terracotta"}], - ["org-verbatim", "verbatim", {fg:"steel"}], - ["org-block", "src block body", {fg:"white", bg:"bg-dim"}], - ["org-block-begin-line","block delim", {fg:"pewter", bg:"bg-dim"}], - ["org-table", "table", {fg:"steel"}], - ["org-date", "timestamp", {fg:"steel"}], - ["org-tag", "tag", {fg:"tan"}], - ["org-special-keyword","keyword/drawer", {fg:"pewter"}], - ["org-meta-line", "#+meta line", {fg:"pewter"}], - ["org-checkbox", "checkbox", {fg:"gold"}], - ["org-headline-done", "done headline", {fg:"pewter"}], - ], - preview: "org" // names the preview renderer - }, - // magit, elfeed, ... added later with the same shape -} -#+end_src - -Defaults reference palette *names* (blue, gold, ...) resolved to hexes at load, -so a curated app seeds sensibly from the current palette. The user reassigns -any face from the palette dropdowns exactly like the other tables. - -State mirrors the other tiers: a =PKGMAP= of -={app: {face: {fg, bg, bold, italic, inherit, height, source}}}=, edited live, rendered into -the table and the preview. The =APPS= block above shows ~18 org faces only as a -shape illustration; the real org entry is the complete set below. - -** Data model — org face set (complete) - -Per the completeness decision, org's table lists org's entire own =defface= set -(org-faces.el + org-agenda.el), ~88 faces, grouped. Seed defaults for the -prominent groups; the long tail seeds to fg or an =inherit= of its group base, -which the user overrides. The groups (face names verbatim from the running -Emacs): - -- *Document:* org-document-title, org-document-info, org-document-info-keyword -- *Headings:* org-level-1 .. org-level-8, org-headline-todo, org-headline-done -- *Status / keywords:* org-todo, org-done, org-priority, org-tag, org-tag-group, - org-special-keyword, org-drawer, org-property-value, org-checkbox, - org-checkbox-statistics-todo, org-checkbox-statistics-done, org-warning -- *Links / dates / refs:* org-link, org-footnote, org-date, org-sexp-date, - org-date-selected, org-target, org-macro, org-cite, org-cite-key -- *Blocks / code / quote:* org-block, org-block-begin-line, org-block-end-line, - org-code, org-verbatim, org-inline-src-block, org-quote, org-verse, - org-latex-and-related -- *Tables / columns:* org-table, org-table-header, org-table-row, org-formula, - org-column, org-column-title -- *Lists / meta / structure:* org-list-dt, org-meta-line, org-ellipsis, - org-hide, org-indent, org-archived, org-default, org-dispatcher-highlight -- *Agenda — structure & dates:* org-agenda-structure, - org-agenda-structure-secondary, org-agenda-structure-filter, org-agenda-date, - org-agenda-date-today, org-agenda-date-weekend, org-agenda-date-weekend-today, - org-agenda-current-time, org-agenda-done, org-agenda-dimmed-todo-face -- *Agenda — calendar & filters:* org-agenda-calendar-event, - org-agenda-calendar-sexp, org-agenda-calendar-daterange, org-agenda-diary, - org-agenda-clocking, org-agenda-column-dateline, org-agenda-restriction-lock, - org-agenda-filter-category, org-agenda-filter-effort, org-agenda-filter-regexp, - org-agenda-filter-tags -- *Scheduling / deadlines / clock:* org-scheduled, org-scheduled-today, - org-scheduled-previously, org-upcoming-deadline, org-upcoming-distant-deadline, - org-imminent-deadline, org-time-grid, org-clock-overlay, org-mode-line-clock, - org-mode-line-clock-overrun - -The org *preview* below stays a curated document exercising the prominent -faces; the *table* carries the complete set so every face is assignable, even -the ones the preview doesn't draw. magit and elfeed get the same treatment -(complete own-defface set in the table, a bespoke preview for the common faces). - -* The org preview - -A mock org document painted from PKGMAP["org-mode"] plus the palette ground/fg. -One bespoke renderer (=renderOrgPreview()=) drawing a representative document: - -#+begin_example -#+TITLE: Project Notes <- org-document-title -#+AUTHOR: ... <- org-meta-line / document-info - -* Inbox :work: <- org-level-1 + org-tag -** TODO Draft the spec <- org-level-2 + org-todo - SCHEDULED: <2026-06-08 Sun> <- org-special-keyword + org-date -** DONE Ship the tool <- org-level-2 + org-done (headline-done) -*** Heading three <- org-level-3 - A line with =inline code=, <- org-code - ~verbatim~, and a [[link]]. <- org-verbatim + org-link - - [X] a checkbox item <- org-checkbox - - #+begin_src elisp <- org-block-begin-line - (message "hi") <- org-block - #+end_src <- org-block-end-line - - | name | hex | <- org-table (header row org-table-header) - |------+---------| - | blue | #67809c | -#+end_example - -Each marked element is a span colored from the corresponding PKGMAP face. The -preview rebuilds whenever a package face or the palette changes, same as the -mock frame. - -org, magit, and elfeed get bespoke preview renderers (magit -> a status buffer -mock, elfeed -> a search-list mock). Every *other* package is still fully -themeable: its face *table* is always present and editable, only the rich -*preview* is replaced by a generic fallback — each face's name rendered in its -own colors on the ground. So a user can theme every package they have the -moment its face list is added; the bespoke preview is a polish layer on top, not -a gate. This is the v1 answer to "some will want to touch every package." - -* Export schema - -=theme.json= gains a =packages= key: - -#+begin_src json -{ - "name": "dupre", - "palette": [...], - "assignments": {...}, - "bold": [...], "italic": [...], - "ui": {...}, - "packages": { - "org-mode": { - "org-level-1": {"fg":"#67809c","bg":null,"bold":true,"italic":false,"inherit":null,"height":1.3}, - "org-level-2": {"fg":"#e8bd30","bg":null,"bold":false,"italic":false,"inherit":"org-level-1","height":1.2}, - "org-todo": {"fg":"#cb6b4d","bg":null,"bold":true,"italic":false,"inherit":null} - } - } -} -#+end_src - -=inherit= is optional and =null= when absent. When set, the converter writes -=:inherit PARENT= plus only the overridden attributes. - -Only faces the user actually touched (or the curated defaults) are written. The -build step's converter sets each as a normal face. Backward compatible: a file -without =packages= loads fine. - -* Build-step consumption - -The eventual =theme.json= -> =dupre-*.el= converter already owns tiers 1 and 2. -Tier 3 adds, per package face: - -#+begin_src elisp -(org-level-1 ((t (:foreground "#67809c" :weight bold)))) -(org-todo ((t (:foreground "#cb6b4d" :weight bold)))) -#+end_src - -No new converter machinery — package faces are just more faces. This is the -TDD-worthy part (JSON in, valid faces out), same as the rest of the converter. - -* Scope for v1 - -- Build the section, the app dropdown, and the face tables + previews for the - three v1 apps: org-mode (incl. org-agenda), magit, elfeed. -- org's table carries its complete own-defface set (~88 faces, grouped above), - seeded with defaults; the org preview draws the prominent ones. -- Every other installed package is reachable in the dropdown with an editable - face table and the generic fallback preview, so any package can be themed. -- Wire export/import of the =packages= key (with the optional =inherit= and - =height= fields). -- Leave the converter for the separate build-step task (Elisp, per Craig); the - spec only needs the schema to be right. - -* Implementation phases - -Phased so each step ships without a broken intermediate, and the three bespoke -apps don't wait on the all-package inventory. - -1. *State + schema.* Add =PKGMAP= - ({app:{face:{fg,bg,bold,italic,inherit,height,source}}}) and the =APPS= - registry. Extend export/import with the =packages= key; old JSON (no - =packages=) still imports cleanly. No UI yet. -2. *Curated app data.* Complete own-defface face lists + seeded defaults for org - (incl. org-agenda), magit, elfeed, in =APPS= — including heading heights and - the fixed-pitch inherits. Pure data. -3. *Package face table UI.* App selector; grouped rows; fg/bg dropdowns + bold / - italic toggles + optional inherit + a relative-height stepper; per-face and - per-app reset; a text filter (org/magit are large); a contrast readout per - fg/bg. Built on a generalized face-control helper shared with the ui-faces - table, not a fork of =uiSelect=. -4. *Org preview.* =renderOrgPreview()=, live, refreshing on palette/face change. -5. *Magit + elfeed previews.* Bespoke mocks (magit status buffer, elfeed search - list). -6. *Generated all-package inventory* (the "theme every package" path). A build - step queries Emacs for installed packages' faces grouped by package, writes a - data file =generate.py= embeds; the dropdown then lists every package with an - editable table + the generic fallback preview. Lands after phases 1-5 without - blocking the three bespoke apps. -7. *Docs + validation.* README =packages= schema + inventory-refresh command; - regenerate HTML; fixtures + manual checklist. - -Phases 1-5 deliver the three high-value apps fully; phase 6 opens the long tail; -phase 7 documents. - -* Package face inventory source - -*Hybrid, split across phases.* Curated app metadata (org/magit/elfeed: complete -face lists, seeded defaults, bespoke previews) is hand-maintained in =APPS= and -ships in phases 2-5. A *generated* =PACKAGE_FACE_INVENTORY= — produced by a build -step that asks the running Emacs for each installed package's faces grouped by -package, written to a JSON/Python data file =generate.py= embeds — supplies the -generic fallback packages and ships in phase 6. - -Why hybrid and split: the static generator can't discover packages at runtime in -the browser, so "theme every package" needs a generated inventory; but making the -full inventory a prerequisite for the three bespoke apps invites the scope -explosion the review flagged. Splitting it lets v1's core ship first; the -inventory is additive. - -The generated inventory is an *input artifact* to =generate.py= (a committed data -file refreshed by an explicit command), never browser-side discovery. The refresh -command's dependency on a loaded Emacs config is documented. - -Decided (Craig, 2026-06-08): hybrid-and-split, as above. - -* State and export policy - -Each package face object carries a =source= marker so export can tell a seeded -default from a user edit from a deliberate clear: - -#+begin_src js -{ fg:"#67809c", bg:null, bold:true, italic:false, underline:false, strike:false, inherit:null, height:1.0, source:"default" } -// underline / strike: booleans -> the converter writes :underline t / :strike-through t -// height: float multiplier off the base font (1.0 = unchanged); see Relative height -// source: "default" (seeded) | "user" (edited) | "cleared" (user removed a default) -#+end_src - -Export policy: - -- Write =default= and =user= entries. -- Write =cleared= entries — they must suppress a curated default on reload. -- Omit untouched faces that have no default. -- When =inherit= is set, write =inherit= plus only the explicit overrides. -- Write =height= only when it differs from 1.0. -- Preserve package faces present in an imported file but absent from the current - inventory (or warn) — don't silently drop them. - -Import tolerates a missing =packages= key, unknown app keys, unknown face keys, -a missing =inherit=, and a missing =height= (defaults 1.0). A deleted palette -color leaves package face references in the same "(gone)" recoverable state -syntax colors use. Inheritance cycles are rejected (treated as no inheritance) -during preview resolution. - -* Relative height - -Some faces want to be bigger than body text — org headings above all, also -=org-document-title=. A face's =height= field is a *float multiplier* off the -base font (=1.3= = 1.3× the running font, whatever it is), never an absolute -point size, so it stays portable across fonts and machines. =1.0= means -unchanged. The base monospace family is *not* a theme/tool concern — it lives in -=modules/font-config.el=; the tool owns only relative size. - -*Height does not cascade through =inherit=.* This is the one attribute resolved -directly off the face, not through its inherit chain. Emacs multiplies float -heights along an inherit chain, so a level-2 that inherits level-1 (1.3) and -also sets 1.1 would render at 1.43 — almost never what's wanted. Headings should -each size off the *body*, so the seeded defaults set =org-level-1= 1.3, -=org-level-2= 1.2, =org-level-3= 1.15, etc., each independent, and the tool reads -=height= from the face while still resolving *color* through inherit. - -- *Schema:* the =height= float on the face object (above), default 1.0, omitted - from export when 1.0. -- *UI:* a small numeric stepper in the face row (range ~0.8–2.0, step 0.05); - meaningful only for the size-bearing faces but shown on every row at 1.0. -- *Preview:* the row renders at the scaled =font-size= so a heading visibly - grows in the mock. -- *Converter:* writes =:height 1.3= into the face spec when ≠ 1.0. - -Related, same mechanism: org's mixed-pitch faces (=org-block=, =org-code=, -=org-verbatim=, =org-table=, =org-meta-line=, =org-date=) seed =inherit: -"fixed-pitch"= so they stay monospace when a buffer switches to a proportional -font via =variable-pitch-mode= / =mixed-pitch=. The proportional family itself -stays in =font-config.el= (the presets already carry =:variable-pitch-family=); -the tool only carries the fixed-pitch inherit relationship, shown like any other -inherited value. - -* Acceptance criteria - -- Existing =dupre.json= (no =packages= key) imports cleanly. -- Export includes =packages= once defaults or edits exist; - =fg/bg/bold/italic/inherit/height/source= round-trip through import/export. -- A face =height= renders as a scaled font-size in the preview (heading visibly - grows) and is read off the face, not cascaded through =inherit=. -- org, magit, elfeed appear in the app selector with complete grouped face tables. -- (phase 6) generic inventory packages appear with editable tables + fallback - previews, the fallback visibly labeled as generic. -- A palette color update propagates to package faces the same way it does to - syntax / ui faces. -- =python3 scripts/theme-studio/generate.py= rebuilds =theme-studio.html=. -- README documents the =packages= schema, inheritance, and the inventory source. - -* Extensibility (adding the next app) - -1. Add an entry to =APPS= (label, curated face list with palette-name defaults, - preview key). -2. Optionally write a bespoke preview renderer; until then the generic fallback - renders. -3. Nothing else changes — the dropdown, table, export, and import are all - data-driven off =APPS= / =PKGMAP=. - -* Agreed decisions - -Craig's answers to the first review round, baked in (the body sections above -reflect these; this records the decisions): - -1. *Curated set is complete, not iterative.* For org, list its *entire* own - defface set (org-faces.el + org-agenda.el), ~88 faces, not a hand-picked - ~18. The user wants every choice present, not a set that grows on demand. - See "Data model — org face set" for the full grouped list. -2. *Seed curated defaults.* Seed sensible fg/bg and weight per face (headings, - title, TODO/DONE bold; agenda dates and deadlines colored by role). The user - reassigns from there. -3. *App order: org, magit, elfeed for v1.* Then the rest one at a time, drawn - from the packages Craig actually runs: calibredb, ghostel, mu4e, the IRC - client, org-drill, dirvish + dired, slack. A finite "most-used" list gets - picked later; we do not try to do everything at once. -4. *Generic fallback is real, not display-only.* Any package not given a - bespoke preview still gets a fully editable face table (so a user can theme - *every* package they have); only the rich preview is missing, replaced by a - swatch-in-context fallback. Bespoke previews ship for org, magit, elfeed. - -* Inheritance representation (decided) - -Each face carries an optional =inherit= field naming another face (or =null=). -The face's own =fg/bg/bold/italic= are *overrides* layered on top of what it -inherits. - -#+begin_src js -["org-level-2", "heading 2", {inherit:"org-level-1", fg:"gold"}] -// exports as: (org-level-2 ((t (:inherit org-level-1 :foreground "#e8bd30")))) -["org-agenda-date-today", "agenda today", {inherit:"org-agenda-date", bold:true}] -// exports as: (org-agenda-date-today ((t (:inherit org-agenda-date :weight bold)))) -#+end_src - -*Decision (Craig, 2026-06-07): model inheritance, show the resolved result, -override what looks bad.* The point is to see what a face ends up looking like -when it inherits, judge it in the preview, and fix only the ones that look -wrong: - -- Each face's *effective* color is resolved through its inherit chain and shown - in its table row, visibly marked "inherited from " so it reads as - not-explicitly-set. The face's own =fg/bg/bold/italic= are overrides layered - on top. -- The mock preview on the right renders every face with its effective color, so - inherited faces are judged in context, not in the abstract. -- Overriding is one action: assign a color (or toggle weight) and the row flips - from inherited to explicit (=source: "user"=), shown at once in the table and - preview. -- Export writes =:inherit PARENT= for faces left inherited (carrying the - relationship, so they follow the parent the theme also sets) and explicit - attributes for the ones overridden — never a frozen copy of an inherited - color. - -Seeded defaults express the inherit relationships org itself uses out of the box -(heading levels off a base, =org-agenda-date= variants off =org-agenda-date=, -=org-code= / =org-verbatim= off =fixed-pitch=), so the table opens showing -org's real cascade, which the user then tunes. Inheritance cycles resolve to no -inheritance. - -* Custom color picker (proposal) - -Craig wants a custom in-page color picker to replace the native browser swatch. -The native == opens the OS color chooser, which the page -cannot size or restyle; a custom picker is the only way to get a larger, -on-theme picker and to show the palette/contrast in the picker itself. - -Proposed widget — a popup anchored to the swatch, drawn in-page: - -- A *saturation/value square* (click or drag to set S and V) plus a *hue - slider* down the side. Standard HSV picker geometry. -- A *hex field* synced both ways with the square/slider (already exists in the - add-color row; the picker writes to it). -- The current *palette* shown as clickable chips along the bottom, so picking - an existing color is one click and the overlap problem (many roles, one - color) is visible while choosing. -- A live *contrast readout* against the current background (ratio + AAA / AA / - FAIL) updating as the color moves, so a color is judged for legibility at - pick time, not after assignment. -- Sized generously (the native popup's size was the original complaint); opens - on click of the swatch, closes on pick or click-away. - -Implementation: ~120 lines of vanilla JS/canvas (or CSS gradients) for the -square + slider, reusing the existing =rl()= / =contrast()= / =rating()= -helpers for the readout and =normHex()= for the field sync. No dependency. It -replaces the == in the add-color row and, later, becomes the -picker the package-face dropdowns can also invoke. - -It stays *off* the tier-3 critical path: a separate task before or after the -package-face build, not folded into it, since folding it in widens the blast -radius for no dependency benefit. Build it only sooner if package-face editing -proves painful with the native swatch. - -Decided (Craig, 2026-06-08): after tier 3, as its own task. - -* Files touched - -- =scripts/theme-studio/generate.py= — the section, =APPS= data, the package - face table, =renderOrgPreview()=, export/import of =packages=. -- =scripts/theme-studio/theme-studio.html= — regenerated. -- (later) the =theme.json= -> =dupre-*.el= converter (Elisp) — consumes - =packages=. - -* Review dispositions - -Codex review (2026-06-07), =Not ready=. Findings processed: - -- *Modified — generated inventory (high, blocking).* Codex recommended a hybrid - inventory so every installed package is reachable. Accepted the hybrid, but - *split* it: the generated all-package inventory is its own phase (6), after the - three bespoke apps (phases 1-5), rather than a v1 prerequisite. Reason: Codex - named scope explosion as the main risk, and gating org/magit/elfeed on a - full-inventory mechanism is exactly that. The split keeps v1's core shippable - and makes "theme every package" additive. Confirm-with-Craig flagged as an - open. -- *Modified — preview depth (UX obs).* Codex suggested level 4-8 examples in the - org preview. The preview stays a curated document drawing the prominent faces - (incl. a couple of deeper levels as representative); the complete level set - lives in the *table*, which is where every face is assignable. A full 8-level - preview block would bloat the mock without adding assignability. - -Everything else in the review accepted as written: implementation phases, -acceptance criteria, the =source= state field + export policy, curated-vs-complete -wording, keeping the custom picker off the critical path, unknown-import -preservation, the test-strategy fixtures, and the UX/architecture/robustness -observations (grouping + filter, reset controls, package-fg/bg contrast readout, -generalized face-control helper, package style kept inside the package object, -"(gone)" recoverable state, inheritance-cycle rejection). - -* Review and iteration history - -** 2026-06-07 Sun @ 18:17:14 -0500 — Claude Code (emacs-d) — author + responder -- *What:* Folded Craig's first-round cj-comment answers into the body. Curated - org set changed from ~18 to org's complete own-defface set (~88, grouped, incl. - org-agenda). v1 apps fixed to org/magit/elfeed with the rest deferred to a - one-at-a-time list. Generic fallback clarified as a fully editable table for - every package (only the rich preview is bespoke). Answered the inheritance - question with an optional =inherit= field (absolute-default, opt-in cascade) - and added it to the schema. Added a custom-color-picker proposal at Craig's - request. Converted Open questions to Agreed decisions; two genuine opens - remain (inheritance confirm, picker timing). -- *Why:* Craig answered the five open questions inline and asked for the - spec-response fold plus a custom-picker proposal. Completeness over iterative - growth was his explicit call on the curated set. -- *Artifacts:* This spec. The converter language (Elisp) confirmed by Craig. - Note: Craig referenced a Codex spec review, but no - =theme-studio-package-faces-review.org= (or any =*-review.org=) exists in the - repo — the review file could not be found, so only the author-side cj-comment - fold was done; the Codex review remains to be processed when its file is - located. - -** 2026-06-07 Sun @ 18:20:16 -0500 — Codex — reviewer -- *What changed or was recommended:* Created the missing formal review and - marked the spec =Not ready=. The review recommends adding implementation - phases, acceptance criteria, an explicit package-face inventory source, and - default/user/cleared/inherited export semantics. -- *Why:* The direction is coherent and the first-round decisions are folded in, - but v1 now depends on behavior that is not yet implementable from the current - static generator without a defined inventory and state/export contract. -- *Artifacts:* [[file:theme-studio-package-faces-spec-review.org][theme-studio-package-faces-spec-review.org]] - -** 2026-06-07 Sun @ 18:28:02 -0500 — Claude Code (emacs-d) — responder -- *What:* Ran spec-response against the Codex review. Added Implementation - phases (7), Package face inventory source (hybrid, split), State and export - policy (=source= field + export rules), and Acceptance criteria. Wove in the - UX/architecture/robustness observations. Two reasoned modifies (generated - inventory split into its own phase; org preview stays curated rather than - all-8-levels), everything else accepted — see Review dispositions. Status - moved to =Ready with caveats=. Deleted the review file. -- *Why:* The four blocking findings were real implementation-contract gaps; the - inventory split answers Codex's own scope-explosion warning while still - reaching "theme every package." -- *Artifacts:* This spec (Review dispositions section); review file deleted per - the spec-response close-out. Three opens remain for Craig (inheritance confirm, - hybrid-inventory confirm, picker timing). - -** 2026-06-08 Mon @ 00:12:38 -0500 — Codex — reviewer -- *What changed or was recommended:* Reran spec-review after the response pass. - Assigned rubric =Ready= and did not create a new review file. The previous - blockers are now addressed: implementation phases, acceptance criteria, - hybrid/split inventory source, package-face state/export semantics, task - tracking, and the open inheritance/inventory/picker decisions are resolved. -- *Why:* The spec now gives an implementer concrete behavior, phase boundaries, - validation criteria, and deferred-work handling without forcing product - decisions during implementation. -- *Artifacts:* This spec; implementation tasks in [[file:../../todo.org][todo.org]]. - -** 2026-06-08 Mon @ 00:38:23 -0500 — Claude Code (emacs-d) — author -- *What:* Added a relative =height= field to the face schema (float multiplier - off the base font, default 1.0, omitted at 1.0), a new "Relative height" - section, a per-face stepper in the table, preview scaling, and converter - output. Established the rule that =height= is read off the face and does *not* - cascade through =inherit= (Emacs multiplies float heights along the chain). - Noted the mixed-pitch =fixed-pitch= inherits as the same-mechanism companion. - Brought Phase 1's shipped schema plumbing in line with the new field. -- *Why:* Craig asked to fold height in — it matters for org headings above all. - Font *family* stays in =modules/font-config.el=; the theme owns relative size - and the fixed-pitch inherit relationships only. -- *Artifacts:* This spec; =scripts/theme-studio/generate.py= phase-1 plumbing. diff --git a/docs/specs/theme-studio-package-faces-spec.org b/docs/specs/theme-studio-package-faces-spec.org new file mode 100644 index 00000000..4390f0fc --- /dev/null +++ b/docs/specs/theme-studio-package-faces-spec.org @@ -0,0 +1,594 @@ +#+TITLE: theme-studio — package faces (tier 3), starting with org-mode +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-07 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING theme-studio — package faces (tier 3), starting with org-mode +:PROPERTIES: +:ID: 8f37a1fd-cfd3-4b25-92e5-772468092bdc +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DOING from existing :STATUS: doing + +* Status + +Spec / Craig's first-round answers folded in (2026-06-07). Proposes a third tier +for the theme-studio (scripts/theme-studio/) that lets a theme colorize +package-specific faces, built one application at a time. v1 apps: org-mode +(incl. org-agenda), magit, elfeed. Codex review incorporated (2026-06-07): added +implementation phases, acceptance criteria, the package-face inventory source +(hybrid, split), and state/export semantics. Rubric now =Ready=. +All opens resolved (Craig, 2026-06-07/08): inheritance is modeled (show each +face's resolved color in the table + preview, override what looks bad); inventory +is hybrid-and-split (org/magit/elfeed bespoke first, generated all-package +inventory as a later phase); the custom color picker is built after tier 3. +Implementation tasks live in =todo.org=. + +* Background — the three tiers + +The theme-studio already models two tiers of faces: + +1. *Syntax* — the font-lock / tree-sitter categories (keyword, string, type, + comment, etc.), in the "code/color assignments" table. +2. *UI* — Emacs's built-in interface faces (cursor, region, mode-line, fringe, + line numbers, isearch, and the rest), in the "ui faces" table with the live + mock-frame preview. + +Tier 3 is *package faces*: faces a package declares with =defface= so a theme +can color the package as it wishes. The running config has 1,146 such faces +across 186 packages (magit 111, lsp-mode 97, telega 91, web-mode 82, org ~30 +core, and a long tail). No theme colors all of them; quality themes hand-pick +the packages the user actually lives in and theme those. + +This spec adds a tier-3 section to the tool, structured so applications are +added one at a time. org-mode ships first. + +* Goal + +A new "package faces" section with: + +1. An *application dropdown* — pick which package's faces to edit. v1 ships + org-mode (including org-agenda), magit, and elfeed; the rest of Craig's + packages (calibredb, ghostel, mu4e, IRC, org-drill, dirvish + dired, slack) + follow one at a time. +2. A *face table* for the selected app — one row per face in the app's complete + set, each with a foreground dropdown, a background dropdown, bold / italic + toggles, an optional inherit, and a relative-height stepper, all drawing from + the same palette as the other tables. Grouped, with a text filter for the + large apps. +3. A *preview pane* for the selected app — a realistic mock of that package + rendered with the live theme, the way the ui-faces mock-frame shows the UI + faces in a buffer. org-mode gets a mock org document. + +The export (=theme.json=) gains a =packages= object so the build step can set +these faces too. + +* UI placement + +A new top-level section under the ui-faces row: + +#+begin_example +

package faces

+[ application: (org-mode v) ] +
+ left = the selected app's face table (fg / bg / B / I per face) + right = the selected app's preview pane (e.g. the org document mock) +
+#+end_example + +Same two-column stretch layout as the ui-faces row, so the preview matches the +table's height. + +* Data model + +A single data structure drives everything, keyed by application: + +#+begin_src js +APPS = { + "org-mode": { + label: "org-mode", + faces: [ + // face, human label, default {fg, bg, bold, italic} + ["org-document-title", "document title", {fg:"gold", bold:true}], + ["org-level-1", "heading 1", {fg:"blue", bold:true}], + ["org-level-2", "heading 2", {fg:"gold"}], + ["org-level-3", "heading 3", {fg:"regal"}], + ["org-todo", "TODO keyword", {fg:"terracotta", bold:true}], + ["org-done", "DONE keyword", {fg:"sage", bold:true}], + ["org-link", "link", {fg:"blue"}], // base `link` + ["org-code", "inline code", {fg:"terracotta"}], + ["org-verbatim", "verbatim", {fg:"steel"}], + ["org-block", "src block body", {fg:"white", bg:"bg-dim"}], + ["org-block-begin-line","block delim", {fg:"pewter", bg:"bg-dim"}], + ["org-table", "table", {fg:"steel"}], + ["org-date", "timestamp", {fg:"steel"}], + ["org-tag", "tag", {fg:"tan"}], + ["org-special-keyword","keyword/drawer", {fg:"pewter"}], + ["org-meta-line", "#+meta line", {fg:"pewter"}], + ["org-checkbox", "checkbox", {fg:"gold"}], + ["org-headline-done", "done headline", {fg:"pewter"}], + ], + preview: "org" // names the preview renderer + }, + // magit, elfeed, ... added later with the same shape +} +#+end_src + +Defaults reference palette *names* (blue, gold, ...) resolved to hexes at load, +so a curated app seeds sensibly from the current palette. The user reassigns +any face from the palette dropdowns exactly like the other tables. + +State mirrors the other tiers: a =PKGMAP= of +={app: {face: {fg, bg, bold, italic, inherit, height, source}}}=, edited live, rendered into +the table and the preview. The =APPS= block above shows ~18 org faces only as a +shape illustration; the real org entry is the complete set below. + +** Data model — org face set (complete) + +Per the completeness decision, org's table lists org's entire own =defface= set +(org-faces.el + org-agenda.el), ~88 faces, grouped. Seed defaults for the +prominent groups; the long tail seeds to fg or an =inherit= of its group base, +which the user overrides. The groups (face names verbatim from the running +Emacs): + +- *Document:* org-document-title, org-document-info, org-document-info-keyword +- *Headings:* org-level-1 .. org-level-8, org-headline-todo, org-headline-done +- *Status / keywords:* org-todo, org-done, org-priority, org-tag, org-tag-group, + org-special-keyword, org-drawer, org-property-value, org-checkbox, + org-checkbox-statistics-todo, org-checkbox-statistics-done, org-warning +- *Links / dates / refs:* org-link, org-footnote, org-date, org-sexp-date, + org-date-selected, org-target, org-macro, org-cite, org-cite-key +- *Blocks / code / quote:* org-block, org-block-begin-line, org-block-end-line, + org-code, org-verbatim, org-inline-src-block, org-quote, org-verse, + org-latex-and-related +- *Tables / columns:* org-table, org-table-header, org-table-row, org-formula, + org-column, org-column-title +- *Lists / meta / structure:* org-list-dt, org-meta-line, org-ellipsis, + org-hide, org-indent, org-archived, org-default, org-dispatcher-highlight +- *Agenda — structure & dates:* org-agenda-structure, + org-agenda-structure-secondary, org-agenda-structure-filter, org-agenda-date, + org-agenda-date-today, org-agenda-date-weekend, org-agenda-date-weekend-today, + org-agenda-current-time, org-agenda-done, org-agenda-dimmed-todo-face +- *Agenda — calendar & filters:* org-agenda-calendar-event, + org-agenda-calendar-sexp, org-agenda-calendar-daterange, org-agenda-diary, + org-agenda-clocking, org-agenda-column-dateline, org-agenda-restriction-lock, + org-agenda-filter-category, org-agenda-filter-effort, org-agenda-filter-regexp, + org-agenda-filter-tags +- *Scheduling / deadlines / clock:* org-scheduled, org-scheduled-today, + org-scheduled-previously, org-upcoming-deadline, org-upcoming-distant-deadline, + org-imminent-deadline, org-time-grid, org-clock-overlay, org-mode-line-clock, + org-mode-line-clock-overrun + +The org *preview* below stays a curated document exercising the prominent +faces; the *table* carries the complete set so every face is assignable, even +the ones the preview doesn't draw. magit and elfeed get the same treatment +(complete own-defface set in the table, a bespoke preview for the common faces). + +* The org preview + +A mock org document painted from PKGMAP["org-mode"] plus the palette ground/fg. +One bespoke renderer (=renderOrgPreview()=) drawing a representative document: + +#+begin_example +#+TITLE: Project Notes <- org-document-title +#+AUTHOR: ... <- org-meta-line / document-info + +* Inbox :work: <- org-level-1 + org-tag +** TODO Draft the spec <- org-level-2 + org-todo + SCHEDULED: <2026-06-08 Sun> <- org-special-keyword + org-date +** DONE Ship the tool <- org-level-2 + org-done (headline-done) +*** Heading three <- org-level-3 + A line with =inline code=, <- org-code + ~verbatim~, and a [[link]]. <- org-verbatim + org-link + - [X] a checkbox item <- org-checkbox + + #+begin_src elisp <- org-block-begin-line + (message "hi") <- org-block + #+end_src <- org-block-end-line + + | name | hex | <- org-table (header row org-table-header) + |------+---------| + | blue | #67809c | +#+end_example + +Each marked element is a span colored from the corresponding PKGMAP face. The +preview rebuilds whenever a package face or the palette changes, same as the +mock frame. + +org, magit, and elfeed get bespoke preview renderers (magit -> a status buffer +mock, elfeed -> a search-list mock). Every *other* package is still fully +themeable: its face *table* is always present and editable, only the rich +*preview* is replaced by a generic fallback — each face's name rendered in its +own colors on the ground. So a user can theme every package they have the +moment its face list is added; the bespoke preview is a polish layer on top, not +a gate. This is the v1 answer to "some will want to touch every package." + +* Export schema + +=theme.json= gains a =packages= key: + +#+begin_src json +{ + "name": "dupre", + "palette": [...], + "assignments": {...}, + "bold": [...], "italic": [...], + "ui": {...}, + "packages": { + "org-mode": { + "org-level-1": {"fg":"#67809c","bg":null,"bold":true,"italic":false,"inherit":null,"height":1.3}, + "org-level-2": {"fg":"#e8bd30","bg":null,"bold":false,"italic":false,"inherit":"org-level-1","height":1.2}, + "org-todo": {"fg":"#cb6b4d","bg":null,"bold":true,"italic":false,"inherit":null} + } + } +} +#+end_src + +=inherit= is optional and =null= when absent. When set, the converter writes +=:inherit PARENT= plus only the overridden attributes. + +Only faces the user actually touched (or the curated defaults) are written. The +build step's converter sets each as a normal face. Backward compatible: a file +without =packages= loads fine. + +* Build-step consumption + +The eventual =theme.json= -> =dupre-*.el= converter already owns tiers 1 and 2. +Tier 3 adds, per package face: + +#+begin_src elisp +(org-level-1 ((t (:foreground "#67809c" :weight bold)))) +(org-todo ((t (:foreground "#cb6b4d" :weight bold)))) +#+end_src + +No new converter machinery — package faces are just more faces. This is the +TDD-worthy part (JSON in, valid faces out), same as the rest of the converter. + +* Scope for v1 + +- Build the section, the app dropdown, and the face tables + previews for the + three v1 apps: org-mode (incl. org-agenda), magit, elfeed. +- org's table carries its complete own-defface set (~88 faces, grouped above), + seeded with defaults; the org preview draws the prominent ones. +- Every other installed package is reachable in the dropdown with an editable + face table and the generic fallback preview, so any package can be themed. +- Wire export/import of the =packages= key (with the optional =inherit= and + =height= fields). +- Leave the converter for the separate build-step task (Elisp, per Craig); the + spec only needs the schema to be right. + +* Implementation phases + +Phased so each step ships without a broken intermediate, and the three bespoke +apps don't wait on the all-package inventory. + +1. *State + schema.* Add =PKGMAP= + ({app:{face:{fg,bg,bold,italic,inherit,height,source}}}) and the =APPS= + registry. Extend export/import with the =packages= key; old JSON (no + =packages=) still imports cleanly. No UI yet. +2. *Curated app data.* Complete own-defface face lists + seeded defaults for org + (incl. org-agenda), magit, elfeed, in =APPS= — including heading heights and + the fixed-pitch inherits. Pure data. +3. *Package face table UI.* App selector; grouped rows; fg/bg dropdowns + bold / + italic toggles + optional inherit + a relative-height stepper; per-face and + per-app reset; a text filter (org/magit are large); a contrast readout per + fg/bg. Built on a generalized face-control helper shared with the ui-faces + table, not a fork of =uiSelect=. +4. *Org preview.* =renderOrgPreview()=, live, refreshing on palette/face change. +5. *Magit + elfeed previews.* Bespoke mocks (magit status buffer, elfeed search + list). +6. *Generated all-package inventory* (the "theme every package" path). A build + step queries Emacs for installed packages' faces grouped by package, writes a + data file =generate.py= embeds; the dropdown then lists every package with an + editable table + the generic fallback preview. Lands after phases 1-5 without + blocking the three bespoke apps. +7. *Docs + validation.* README =packages= schema + inventory-refresh command; + regenerate HTML; fixtures + manual checklist. + +Phases 1-5 deliver the three high-value apps fully; phase 6 opens the long tail; +phase 7 documents. + +* Package face inventory source + +*Hybrid, split across phases.* Curated app metadata (org/magit/elfeed: complete +face lists, seeded defaults, bespoke previews) is hand-maintained in =APPS= and +ships in phases 2-5. A *generated* =PACKAGE_FACE_INVENTORY= — produced by a build +step that asks the running Emacs for each installed package's faces grouped by +package, written to a JSON/Python data file =generate.py= embeds — supplies the +generic fallback packages and ships in phase 6. + +Why hybrid and split: the static generator can't discover packages at runtime in +the browser, so "theme every package" needs a generated inventory; but making the +full inventory a prerequisite for the three bespoke apps invites the scope +explosion the review flagged. Splitting it lets v1's core ship first; the +inventory is additive. + +The generated inventory is an *input artifact* to =generate.py= (a committed data +file refreshed by an explicit command), never browser-side discovery. The refresh +command's dependency on a loaded Emacs config is documented. + +Decided (Craig, 2026-06-08): hybrid-and-split, as above. + +* State and export policy + +Each package face object carries a =source= marker so export can tell a seeded +default from a user edit from a deliberate clear: + +#+begin_src js +{ fg:"#67809c", bg:null, bold:true, italic:false, underline:false, strike:false, inherit:null, height:1.0, source:"default" } +// underline / strike: booleans -> the converter writes :underline t / :strike-through t +// height: float multiplier off the base font (1.0 = unchanged); see Relative height +// source: "default" (seeded) | "user" (edited) | "cleared" (user removed a default) +#+end_src + +Export policy: + +- Write =default= and =user= entries. +- Write =cleared= entries — they must suppress a curated default on reload. +- Omit untouched faces that have no default. +- When =inherit= is set, write =inherit= plus only the explicit overrides. +- Write =height= only when it differs from 1.0. +- Preserve package faces present in an imported file but absent from the current + inventory (or warn) — don't silently drop them. + +Import tolerates a missing =packages= key, unknown app keys, unknown face keys, +a missing =inherit=, and a missing =height= (defaults 1.0). A deleted palette +color leaves package face references in the same "(gone)" recoverable state +syntax colors use. Inheritance cycles are rejected (treated as no inheritance) +during preview resolution. + +* Relative height + +Some faces want to be bigger than body text — org headings above all, also +=org-document-title=. A face's =height= field is a *float multiplier* off the +base font (=1.3= = 1.3× the running font, whatever it is), never an absolute +point size, so it stays portable across fonts and machines. =1.0= means +unchanged. The base monospace family is *not* a theme/tool concern — it lives in +=modules/font-config.el=; the tool owns only relative size. + +*Height does not cascade through =inherit=.* This is the one attribute resolved +directly off the face, not through its inherit chain. Emacs multiplies float +heights along an inherit chain, so a level-2 that inherits level-1 (1.3) and +also sets 1.1 would render at 1.43 — almost never what's wanted. Headings should +each size off the *body*, so the seeded defaults set =org-level-1= 1.3, +=org-level-2= 1.2, =org-level-3= 1.15, etc., each independent, and the tool reads +=height= from the face while still resolving *color* through inherit. + +- *Schema:* the =height= float on the face object (above), default 1.0, omitted + from export when 1.0. +- *UI:* a small numeric stepper in the face row (range ~0.8–2.0, step 0.05); + meaningful only for the size-bearing faces but shown on every row at 1.0. +- *Preview:* the row renders at the scaled =font-size= so a heading visibly + grows in the mock. +- *Converter:* writes =:height 1.3= into the face spec when ≠ 1.0. + +Related, same mechanism: org's mixed-pitch faces (=org-block=, =org-code=, +=org-verbatim=, =org-table=, =org-meta-line=, =org-date=) seed =inherit: +"fixed-pitch"= so they stay monospace when a buffer switches to a proportional +font via =variable-pitch-mode= / =mixed-pitch=. The proportional family itself +stays in =font-config.el= (the presets already carry =:variable-pitch-family=); +the tool only carries the fixed-pitch inherit relationship, shown like any other +inherited value. + +* Acceptance criteria + +- Existing =dupre.json= (no =packages= key) imports cleanly. +- Export includes =packages= once defaults or edits exist; + =fg/bg/bold/italic/inherit/height/source= round-trip through import/export. +- A face =height= renders as a scaled font-size in the preview (heading visibly + grows) and is read off the face, not cascaded through =inherit=. +- org, magit, elfeed appear in the app selector with complete grouped face tables. +- (phase 6) generic inventory packages appear with editable tables + fallback + previews, the fallback visibly labeled as generic. +- A palette color update propagates to package faces the same way it does to + syntax / ui faces. +- =python3 scripts/theme-studio/generate.py= rebuilds =theme-studio.html=. +- README documents the =packages= schema, inheritance, and the inventory source. + +* Extensibility (adding the next app) + +1. Add an entry to =APPS= (label, curated face list with palette-name defaults, + preview key). +2. Optionally write a bespoke preview renderer; until then the generic fallback + renders. +3. Nothing else changes — the dropdown, table, export, and import are all + data-driven off =APPS= / =PKGMAP=. + +* Agreed decisions + +Craig's answers to the first review round, baked in (the body sections above +reflect these; this records the decisions): + +1. *Curated set is complete, not iterative.* For org, list its *entire* own + defface set (org-faces.el + org-agenda.el), ~88 faces, not a hand-picked + ~18. The user wants every choice present, not a set that grows on demand. + See "Data model — org face set" for the full grouped list. +2. *Seed curated defaults.* Seed sensible fg/bg and weight per face (headings, + title, TODO/DONE bold; agenda dates and deadlines colored by role). The user + reassigns from there. +3. *App order: org, magit, elfeed for v1.* Then the rest one at a time, drawn + from the packages Craig actually runs: calibredb, ghostel, mu4e, the IRC + client, org-drill, dirvish + dired, slack. A finite "most-used" list gets + picked later; we do not try to do everything at once. +4. *Generic fallback is real, not display-only.* Any package not given a + bespoke preview still gets a fully editable face table (so a user can theme + *every* package they have); only the rich preview is missing, replaced by a + swatch-in-context fallback. Bespoke previews ship for org, magit, elfeed. + +* Inheritance representation (decided) + +Each face carries an optional =inherit= field naming another face (or =null=). +The face's own =fg/bg/bold/italic= are *overrides* layered on top of what it +inherits. + +#+begin_src js +["org-level-2", "heading 2", {inherit:"org-level-1", fg:"gold"}] +// exports as: (org-level-2 ((t (:inherit org-level-1 :foreground "#e8bd30")))) +["org-agenda-date-today", "agenda today", {inherit:"org-agenda-date", bold:true}] +// exports as: (org-agenda-date-today ((t (:inherit org-agenda-date :weight bold)))) +#+end_src + +*Decision (Craig, 2026-06-07): model inheritance, show the resolved result, +override what looks bad.* The point is to see what a face ends up looking like +when it inherits, judge it in the preview, and fix only the ones that look +wrong: + +- Each face's *effective* color is resolved through its inherit chain and shown + in its table row, visibly marked "inherited from " so it reads as + not-explicitly-set. The face's own =fg/bg/bold/italic= are overrides layered + on top. +- The mock preview on the right renders every face with its effective color, so + inherited faces are judged in context, not in the abstract. +- Overriding is one action: assign a color (or toggle weight) and the row flips + from inherited to explicit (=source: "user"=), shown at once in the table and + preview. +- Export writes =:inherit PARENT= for faces left inherited (carrying the + relationship, so they follow the parent the theme also sets) and explicit + attributes for the ones overridden — never a frozen copy of an inherited + color. + +Seeded defaults express the inherit relationships org itself uses out of the box +(heading levels off a base, =org-agenda-date= variants off =org-agenda-date=, +=org-code= / =org-verbatim= off =fixed-pitch=), so the table opens showing +org's real cascade, which the user then tunes. Inheritance cycles resolve to no +inheritance. + +* Custom color picker (proposal) + +Craig wants a custom in-page color picker to replace the native browser swatch. +The native == opens the OS color chooser, which the page +cannot size or restyle; a custom picker is the only way to get a larger, +on-theme picker and to show the palette/contrast in the picker itself. + +Proposed widget — a popup anchored to the swatch, drawn in-page: + +- A *saturation/value square* (click or drag to set S and V) plus a *hue + slider* down the side. Standard HSV picker geometry. +- A *hex field* synced both ways with the square/slider (already exists in the + add-color row; the picker writes to it). +- The current *palette* shown as clickable chips along the bottom, so picking + an existing color is one click and the overlap problem (many roles, one + color) is visible while choosing. +- A live *contrast readout* against the current background (ratio + AAA / AA / + FAIL) updating as the color moves, so a color is judged for legibility at + pick time, not after assignment. +- Sized generously (the native popup's size was the original complaint); opens + on click of the swatch, closes on pick or click-away. + +Implementation: ~120 lines of vanilla JS/canvas (or CSS gradients) for the +square + slider, reusing the existing =rl()= / =contrast()= / =rating()= +helpers for the readout and =normHex()= for the field sync. No dependency. It +replaces the == in the add-color row and, later, becomes the +picker the package-face dropdowns can also invoke. + +It stays *off* the tier-3 critical path: a separate task before or after the +package-face build, not folded into it, since folding it in widens the blast +radius for no dependency benefit. Build it only sooner if package-face editing +proves painful with the native swatch. + +Decided (Craig, 2026-06-08): after tier 3, as its own task. + +* Files touched + +- =scripts/theme-studio/generate.py= — the section, =APPS= data, the package + face table, =renderOrgPreview()=, export/import of =packages=. +- =scripts/theme-studio/theme-studio.html= — regenerated. +- (later) the =theme.json= -> =dupre-*.el= converter (Elisp) — consumes + =packages=. + +* Review dispositions + +Codex review (2026-06-07), =Not ready=. Findings processed: + +- *Modified — generated inventory (high, blocking).* Codex recommended a hybrid + inventory so every installed package is reachable. Accepted the hybrid, but + *split* it: the generated all-package inventory is its own phase (6), after the + three bespoke apps (phases 1-5), rather than a v1 prerequisite. Reason: Codex + named scope explosion as the main risk, and gating org/magit/elfeed on a + full-inventory mechanism is exactly that. The split keeps v1's core shippable + and makes "theme every package" additive. Confirm-with-Craig flagged as an + open. +- *Modified — preview depth (UX obs).* Codex suggested level 4-8 examples in the + org preview. The preview stays a curated document drawing the prominent faces + (incl. a couple of deeper levels as representative); the complete level set + lives in the *table*, which is where every face is assignable. A full 8-level + preview block would bloat the mock without adding assignability. + +Everything else in the review accepted as written: implementation phases, +acceptance criteria, the =source= state field + export policy, curated-vs-complete +wording, keeping the custom picker off the critical path, unknown-import +preservation, the test-strategy fixtures, and the UX/architecture/robustness +observations (grouping + filter, reset controls, package-fg/bg contrast readout, +generalized face-control helper, package style kept inside the package object, +"(gone)" recoverable state, inheritance-cycle rejection). + +* Review and iteration history + +** 2026-06-07 Sun @ 18:17:14 -0500 — Claude Code (emacs-d) — author + responder +- *What:* Folded Craig's first-round cj-comment answers into the body. Curated + org set changed from ~18 to org's complete own-defface set (~88, grouped, incl. + org-agenda). v1 apps fixed to org/magit/elfeed with the rest deferred to a + one-at-a-time list. Generic fallback clarified as a fully editable table for + every package (only the rich preview is bespoke). Answered the inheritance + question with an optional =inherit= field (absolute-default, opt-in cascade) + and added it to the schema. Added a custom-color-picker proposal at Craig's + request. Converted Open questions to Agreed decisions; two genuine opens + remain (inheritance confirm, picker timing). +- *Why:* Craig answered the five open questions inline and asked for the + spec-response fold plus a custom-picker proposal. Completeness over iterative + growth was his explicit call on the curated set. +- *Artifacts:* This spec. The converter language (Elisp) confirmed by Craig. + Note: Craig referenced a Codex spec review, but no + =theme-studio-package-faces-review.org= (or any =*-review.org=) exists in the + repo — the review file could not be found, so only the author-side cj-comment + fold was done; the Codex review remains to be processed when its file is + located. + +** 2026-06-07 Sun @ 18:20:16 -0500 — Codex — reviewer +- *What changed or was recommended:* Created the missing formal review and + marked the spec =Not ready=. The review recommends adding implementation + phases, acceptance criteria, an explicit package-face inventory source, and + default/user/cleared/inherited export semantics. +- *Why:* The direction is coherent and the first-round decisions are folded in, + but v1 now depends on behavior that is not yet implementable from the current + static generator without a defined inventory and state/export contract. +- *Artifacts:* [[file:theme-studio-package-faces-spec-review.org][theme-studio-package-faces-spec-review.org]] + +** 2026-06-07 Sun @ 18:28:02 -0500 — Claude Code (emacs-d) — responder +- *What:* Ran spec-response against the Codex review. Added Implementation + phases (7), Package face inventory source (hybrid, split), State and export + policy (=source= field + export rules), and Acceptance criteria. Wove in the + UX/architecture/robustness observations. Two reasoned modifies (generated + inventory split into its own phase; org preview stays curated rather than + all-8-levels), everything else accepted — see Review dispositions. Status + moved to =Ready with caveats=. Deleted the review file. +- *Why:* The four blocking findings were real implementation-contract gaps; the + inventory split answers Codex's own scope-explosion warning while still + reaching "theme every package." +- *Artifacts:* This spec (Review dispositions section); review file deleted per + the spec-response close-out. Three opens remain for Craig (inheritance confirm, + hybrid-inventory confirm, picker timing). + +** 2026-06-08 Mon @ 00:12:38 -0500 — Codex — reviewer +- *What changed or was recommended:* Reran spec-review after the response pass. + Assigned rubric =Ready= and did not create a new review file. The previous + blockers are now addressed: implementation phases, acceptance criteria, + hybrid/split inventory source, package-face state/export semantics, task + tracking, and the open inheritance/inventory/picker decisions are resolved. +- *Why:* The spec now gives an implementer concrete behavior, phase boundaries, + validation criteria, and deferred-work handling without forcing product + decisions during implementation. +- *Artifacts:* This spec; implementation tasks in [[file:../../todo.org][todo.org]]. + +** 2026-06-08 Mon @ 00:38:23 -0500 — Claude Code (emacs-d) — author +- *What:* Added a relative =height= field to the face schema (float multiplier + off the base font, default 1.0, omitted at 1.0), a new "Relative height" + section, a per-face stepper in the table, preview scaling, and converter + output. Established the rule that =height= is read off the face and does *not* + cascade through =inherit= (Emacs multiplies float heights along the chain). + Noted the mixed-pitch =fixed-pitch= inherits as the same-mechanism companion. + Brought Phase 1's shipped schema plumbing in line with the new field. +- *Why:* Craig asked to fold height in — it matters for org headings above all. + Font *family* stays in =modules/font-config.el=; the theme owns relative size + and the fixed-pitch inherit relationships only. +- *Artifacts:* This spec; =scripts/theme-studio/generate.py= phase-1 plumbing. diff --git a/docs/specs/theme-studio-palette-generator-spec-doing.org b/docs/specs/theme-studio-palette-generator-spec-doing.org deleted file mode 100644 index b98e1078..00000000 --- a/docs/specs/theme-studio-palette-generator-spec-doing.org +++ /dev/null @@ -1,298 +0,0 @@ -:PROPERTIES: -:ID: 2df157b8-c7c1-47a9-b080-d9586c6f424c -:STATUS: doing -:END: -#+TITLE: Theme Studio Palette Generator -- Spec -#+AUTHOR: Craig Jennings -#+DATE: 2026-06-14 -#+TODO: TODO | DONE SUPERSEDED CANCELLED - -* Metadata -| Status | doing | -|----------+-------| -| Owner | Craig | -|----------+-------| -| Reviewer | Craig | -|----------+-------| -| Related | [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]] | -|----------+-------| - -* Summary -Theme Studio should grow a constraint-first palette generator that creates preview palette columns from the current theme context, then lets the user explicitly commit individual colors or whole columns. It should reuse the existing color selector as the single-color workbench instead of adding a second picker. - -The v1 feature generates palette columns only. It does not assign faces automatically; once generated colors are applied, they become normal editable palette colors with stable column ids. - -* Problem / Context -Theme Studio now has stable color columns, span controls, OKLCH editing, contrast metrics, DeltaE warnings, locks, package faces, and a live preview. The missing workflow is generating a coherent set of candidate base colors without manually choosing each hue, checking contrast, spanning, and then adjusting by eye. - -Generic palette generators are not a good fit by themselves. They optimize visual harmony before text readability, while Emacs themes need dense syntax colors, UI overlays, selections, search hits, diagnostics, and package faces to remain legible over a fixed ground. - -There is also UI overlap with the existing color selector. The selector already edits one active color, shows a swatch, drives the hex field and OKLCH picker, and adds or updates palette colors. A generator should feed that selector, not duplicate it. - -* Goals and Non-Goals -** Goals -- Generate candidate palette columns from explicit source modes and current theme constraints. -- Default to OKLCH generation so lightness and chroma are predictable. -- Default to a syntax-balanced scheme designed for readable code themes. -- Preview generated columns without mutating the real palette. -- Reuse the current color selector to inspect and tune one generated tile at a time. -- Allow adding a generated tile as a new base color column. -- Allow appending a whole generated column. -- Preserve stable column ids and existing assignments where possible when applying proposals. -- Expose diagnostics for contrast, DeltaE, gamut clamp, and rejected candidates. - -** Non-Goals -- Automatically assign generated colors to syntax/UI/package faces in v1. -- Replace the existing manual picker, swatch, or per-column span controls. -- Import external palettes, CSS colors, image colors, or theme files. Import organization remains separate. -- Generate terminal/ANSI palettes in v1. -- Add OKHSL/OKHSV generation modes in v1. -- Rewrite bg/fg automatically in v1. - -** Scope tiers -- v1: generator panel, pure planner, preview columns, existing color selector integration, add generated tile as base column, append whole generated column, diagnostics, tests, README docs. -- Out of scope: automatic face assignment, external imports, image extraction, terminal colors, rewriting bg/fg. -- vNext: replace selected columns, regenerate selected spans, regenerate generator-owned columns, OKHSL/OKHSV controls, whole-palette harmonization, CVD-aware scoring, named style presets, terminal palette derivation. - -* Design -The generator is a panel above the real palette columns. It contains generation controls and a preview area, but no separate color picker. The existing color selector remains the place where one color is inspected, tuned, named, and committed. - -For the user, the workflow is: - -1. Choose a generator source mode and scheme from compact controls in the generator panel. -2. Click preview. -3. Inspect temporary generated columns above the real palette. -4. Click a generated tile to load it into the existing color selector. -5. Optionally tune that color with the existing hex/OKLCH picker. -6. Commit either one generated tile as a new base column, or a whole generated column as normal palette entries. - -The preview layer is separate from =PALETTE=. Rendering preview columns must not create real palette entries, move existing columns, or change assignments. Preview tiles should look like palette tiles, but have a distinct preview treatment such as a dashed border/header or subtle "preview" label. - -The generator controls should be compact and explicit: - -- =Source= is a segmented control in v1: =bg/fg= and, if implemented in v1, =selected=. =bg/fg= is the default. Each option has a short tooltip: =bg/fg= means "generate from the current ground/foreground constraints"; =selected= means "harmonize around selected palette columns." -- =Scheme= can be a select/dropdown because it has more choices. A scheme is the hue-placement strategy used to propose candidate accents. The choices are not final color decisions; they are starting layouts that the planner filters through contrast, lightness, chroma, gamut, and DeltaE constraints. -- The scheme dropdown should include one-line help in the option title or adjacent help text. For example: =syntax-balanced= means "spread readable code accents across the hue wheel"; =analogous= means "nearby hues"; =triadic= means "three evenly-spaced hue families"; =manual= means "use the hues entered by the user." -- =Accent count= controls how many base columns the proposal tries to generate. The default is 8, with a v1 range of 3-12. -- =Span count= controls how many generated span steps each proposed column includes. The default should be conservative, likely 0 or the current preferred span setting, with a v1 range of 0-4. - -Therefore, the number of generated columns is configurable: it is the =accent count=, subject to rejected candidates. If the user asks for 8 accents and 2 are rejected by constraints, the preview should show 6 generated columns and report 2 rejected candidates in the summary. It should not silently backfill unrelated colors unless that behavior is explicitly added later. - -The current color selector gains a third selection mode: - -- No selection: =+ add color= creates a new manual base column from the selector value. -- Real palette tile selected: =update selected= changes that palette tile or recolors its column as today. -- Generated preview tile selected: the selector shows the generated hex/name, and =+ add color= commits that generated color as a new base color column. =update selected= is disabled unless a real palette tile is selected. - -A small status label near the selector should make this state explicit: - -#+begin_example -editing: new color -editing: palette color blue -editing: generated color blue-2 -#+end_example - -Generated columns are proposed from a source mode: - -- =bg/fg only= is the v1 default. It uses the ground and foreground as constraints, plus the chosen scheme/base hue. -- =selected columns= is v1 optional if cheap; otherwise vNext. It harmonizes around columns the user explicitly selected. -- =whole palette= is vNext. It should not be automatic in v1 because imported, experimental, or throwaway colors could make generation unpredictable. - -Generation is constraint-first. The planner chooses hue candidates from the scheme, then searches for useful OKLCH lightness/chroma values that satisfy the current background, contrast target, DeltaE separation, and gamut constraints. Classical harmony schemes are input layouts, not the final authority. - -For implementers, the feature is a pure planning layer plus DOM rendering/application: - -- The planner receives current palette, ground, source mode, generator config, and locks. -- The planner returns a proposal object and never mutates global state. -- The DOM layer renders proposal columns as preview columns. -- Applying one tile or one column converts proposal members into normal palette entries using existing column-id, span, name-collision, and gone/repoint behavior. - -** Generator config -V1 config fields: - -- sourceMode: bg-fg, selected-columns -- scheme: syntax-balanced, analogous, split-complementary, triadic, tetradic, warm-cool, manual -- baseHue: degrees, used by non-manual schemes -- manualHues: list of degrees, used by manual mode -- accentCount: integer, default 8, range 3-12 -- spanCount: integer, default 0 or current preferred span, range 0-4 in v1 -- textLightnessBand: min/max OKLCH L for text accents -- chromaBias: subdued, balanced, vivid -- contrastTarget: none, WCAG AA, WCAG AAA -- deltaEMin: default to the existing palette warning threshold -- locks: respect locked columns and assignments where apply modes touch existing data - -** Proposal object -The planner returns a proposal object with the generator config, proposed columns, rejected candidates, and a summary. Each proposed column carries a stable column id, display name, base hex, member colors with offsets and clamp/metric data, and column-level diagnostics. Rejected candidates carry the attempted hue, rejection reason, and nearest conflicting column when relevant. The summary includes generated count, clamped count, rejected count, minimum contrast, and minimum DeltaE. - -This shape is intentionally close to the existing palette-column model. Preview rendering should not need a second color model. - -** Applying generated colors -The first v1 apply actions are deliberately small: - -- =Add generated tile as base column=: creates a new normal palette column from the selected preview tile. The new column id is derived from the tile name and suffixes on collision. -- =Append generated column=: adds every member of that preview column after existing real columns. Members keep a stable shared column id. -- =Clear preview=: discards proposal state without changing the real palette. - -The following apply actions are deferred unless v1 implementation is already straightforward: - -- replace selected columns -- regenerate spans only -- regenerate generator-owned columns - -When a generated tile is committed as a base column, it starts as a one-tile column. The user can then span it using the existing column span widget. This keeps the one-color action easy to understand and avoids surprising multi-tile commits from a tile-level button. - -** Display -The panel sits between the color selector row and the committed palette columns. It has: - -- source mode segmented control -- scheme segmented control or select -- base hue/manual hue controls -- accent count and span count numeric controls -- chroma bias control -- contrast target control -- preview and clear-preview buttons -- summary row: min contrast, min DeltaE, clamped count, rejected count - -Preview columns render below the controls and above committed columns. A generated tile click loads the existing selector. A generated column header click loads the column base into the selector. A column-level =append column= button commits the whole preview column. - -* Alternatives Considered -** Generic harmony wheel that writes directly into the palette -- Good, because it is familiar and visually compact. -- Bad, because it mutates real palette state before the user can inspect results, and it optimizes hue harmony before readability. -- Neutral, because a hue wheel can still be useful as an input control inside a preview-first generator. - -** Separate generator-specific color picker -- Good, because preview tuning could be isolated from committed palette editing. -- Bad, because Theme Studio already has a capable single-color selector, and a second picker would duplicate hex, OKLCH, contrast, swatch, and add/update semantics. -- Neutral, because a future advanced generator could add a small detail panel, but v1 should not. - -** Generate from the whole palette by default -- Good, because it can harmonize with everything already on screen. -- Bad, because the palette may contain experiments, imported colors, temporary colors, or intentionally clashing accents; using all of them makes results hard to predict. -- Neutral, because whole-palette harmonization is valuable as an explicit vNext mode. - -** Full automatic face assignment -- Good, because it could produce a near-complete theme in one action. -- Bad, because it crosses into seeding, locks, role maps, and package face behavior that already have separate product decisions. -- Neutral, because the palette generator can feed a later seeding workflow. - -** Add OKHSL/OKHSV now -- Good, because those controls may feel friendlier than raw OKLCH. -- Bad, because v1 already has OKLCH math and the feature risk is workflow/state, not another color model. -- Neutral, because OKHSL/OKHSV remain good vNext candidates. - -* Decisions [5/5] -** DONE Default to preview-first generation -- Context: Generator output can disrupt a carefully tuned palette if it mutates immediately. -- Decision: We will render generated colors as temporary preview columns and require explicit add/append actions. -- Consequences: Easier to inspect and avoid accidental data loss; harder because the UI needs proposal state and apply semantics. - -** DONE Reuse the current color selector -- Context: The existing selector already edits one color, shows metrics, opens the picker, and adds/updates palette colors. -- Decision: We will make generated tile clicks load the existing selector instead of adding a second generator picker. -- Consequences: Easier to keep editing behavior consistent; harder because the selector now needs clear state for new, palette, and generated selections. - -** DONE Keep v1 palette-only -- Context: Automatic assignment would touch syntax, UI, package faces, locks, and seeding rules. -- Decision: We will generate palette columns only in v1 and leave face assignment to existing/manual workflows. -- Consequences: Easier to ship a focused generator; harder because the user still maps colors onto faces. - -** DONE Default generation source to bg/fg only -- Context: Existing palette colors may be experimental or imported; using all of them by default makes generation unpredictable. -- Decision: We will default to bg/fg constraints plus explicit scheme/base hue. Selected-column source can be included if scoped; whole-palette source is vNext. -- Consequences: Easier to understand why a proposal was generated; harder because matching an existing palette requires an explicit source mode. - -** DONE Defer OKHSL/OKHSV to vNext -- Context: OKHSL/OKHSV may be friendlier interaction models, but OKLCH already supports the required perceptual generation math. -- Decision: We will ship OKLCH generation first and consider OKHSL/OKHSV after v1 is usable. -- Consequences: Easier to keep v1 small and rigorous; harder because some users may find OKLCH controls less familiar. - -* Implementation phases -** Phase 1 -- Planner core -Add pure generator functions in =app-core.js= or a new generator module. Inputs are current palette, ground, generator config, source selection, and locks. Outputs are proposal objects. Unit tests cover scheme hue placement, OKLCH candidate generation, gamut clamp reporting, name/id collision handling, and no mutation of input state. - -** Phase 2 -- Candidate scoring -Add bounded scoring/adjustment for contrast target, DeltaE separation, chroma bias, and text lightness band. Unit tests cover rejected candidates, clamped colors, low-chroma distinguishability, and deterministic output for fixed config. - -** Phase 3 -- Generator panel and preview rendering -Add the panel, controls, proposal state, preview columns, summary metrics, and clear-preview behavior. Browser gate: preview creates temporary columns without changing committed =PALETTE=. - -** Phase 4 -- Existing selector integration -Add generated-preview selection state to the color selector. Clicking a preview tile loads its hex/name. The selector status label shows generated-vs-palette-vs-new. =+ add color= commits the selected preview tile as a new one-tile base column. Browser gates cover generated tile selection and add-as-column behavior. - -** Phase 5 -- Append generated column -Add column-level append. The generated column becomes normal palette entries with one stable column id. Collisions suffix names/ids deterministically. Browser gates cover append, collision suffixing, and unchanged existing assignments. - -** Phase 6 -- Persistence and metadata -Round-trip optional generator metadata for applied columns without requiring it for normal palette behavior. Existing palettes without metadata continue to work. Browser gate extends roundtrip coverage. - -** Phase 7 -- Documentation and cleanup -Document generator controls, source modes, preview behavior, selector integration, and limits in =scripts/theme-studio/README.md=. Remove prototype code and keep =make -C scripts/theme-studio test= green. - -* Acceptance criteria -- [ ] Previewing a generated palette does not mutate committed =PALETTE=. -- [ ] Preview columns appear above committed columns and are visually distinct. -- [ ] Clicking a generated tile loads that color into the existing selector. -- [ ] The selector clearly says whether it is editing a new color, palette color, or generated color. -- [ ] =+ add color= on a selected generated tile creates a new one-tile base column. -- [ ] Appending a generated column creates normal editable palette entries with one stable column id. -- [ ] Name and column-id collisions are suffixed deterministically. -- [ ] Generated colors report clamp, contrast, and DeltaE diagnostics. -- [ ] Existing manual palette workflows still work without opening the generator panel. -- [ ] Theme Studio tests cover planner functions, preview rendering, selector integration, apply behavior, and round-trip metadata. - -* Readiness dimensions -- Data model & ownership: Proposal state is transient and generator-owned. Applied colors become normal user-owned palette entries. Optional generator metadata is advisory and must not override manual edits. -- Errors, empty states & failure: Invalid config disables preview with an inline message naming the bad field. No preview shows an empty-state line. Rejected candidates appear in the summary. Apply failures must not partially mutate committed palette state. -- Security & privacy: N/A because generation is local deterministic color math with no credentials, network calls, or private external data. -- Observability: The preview summary shows generated, clamped, rejected, min contrast, and min DeltaE. Tile titles or details expose per-color diagnostics. -- Performance & scale: Expected accent counts are 3-12 bases with 0-4 span steps. Candidate search should remain synchronous and bounded. Broader search, if added later, needs progress/cancel. -- Reuse & lost opportunities: Reuse OKLCH, gamut clamp, contrast, DeltaE, palette columns, span generation, locks, existing selector, and existing browser gates. Do not add a second color math stack or picker. -- Architecture fit & weak points: The weak point is proposal/apply state in the DOM app. Keep planner pure and DOM code limited to rendering/applying proposal objects. -- Config surface: Public knobs are source mode, scheme, base/manual hue, accent count, span count, chroma bias, contrast target, DeltaE threshold, and lightness band. Defaults favor readable dark-theme syntax. -- Documentation plan: Update the Theme Studio README with generator controls, source modes, selector integration, preview/apply behavior, and v1 limits. -- Dev tooling: Use =make -C scripts/theme-studio test= as the primary gate and =make -C scripts/theme-studio coverage= for instrumented JS/generator coverage. -- Rollout, compatibility & rollback: The generator is additive. Existing palettes load unchanged. Preview can be cleared. Applied generated columns can be deleted manually. -- External APIs & deps: N/A because v1 has no external APIs or new dependencies. - -* Risks, Rabbit Holes, and Drawbacks -- Candidate search can become an optimization rabbit hole. V1 should use deterministic bounded search around target OKLCH bands. -- "Syntax-balanced" is subjective. Keep it documented as a heuristic, not a claim of universal taste. -- Selector state can become confusing. The status label and disabled update button are required, not polish. -- Whole-palette harmonization is tempting but should wait until preview/apply basics feel good. -- Optional metadata can drift after manual edits. Treat it as advisory only. - -* Testing / Verification / Rollout -Use the existing Theme Studio test stack: - -- Node tests for planner/scoring/collision/immutability. -- Browser hash gate for preview-only non-mutation. -- Browser hash gate for generated tile -> selector state. -- Browser hash gate for add-generated-tile-as-column. -- Browser hash gate for append generated column. -- Round-trip gate for optional generator metadata. -- Manual Chrome pass on at least one dark palette and one light palette. - -* References / Appendix -- [[file:design/theme-studio-color-harmony.org][theme-studio color harmony explainer]] -- [[id:15db8ae3-fc14-49f3-9ed5-d5ff59790904][perceptual color metrics spec]] -- [[file:theme-studio-palette-ramps-spec.org][palette ramps and contrast safety spec]] -- [[file:theme-studio-palette-columns-spec.org][palette columns spec]] -- [[file:../../todo.org::*theme-studio import organization workflow needs a spec][import organization task]] - -* Review and iteration history -** 2026-06-13 Saturday @ 16:31:01 -0500 -- Craig -- author -- What: Initial draft using the spec-create workflow. -- Why: Palette generation has real design trade-offs around color space, preview/apply behavior, assignment boundaries, and how much generator state should persist. -- Artifacts: [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]]. - -** 2026-06-14 Sunday @ 00:44:00 -0500 -- Craig -- author -- What: Reworked the draft around preview-first generation, existing color selector integration, generated tile add-as-column behavior, and source-mode defaults. -- Why: Craig clarified the desired UX: generated colors should be inspectable/tunable through the existing selector, and committing one generated color should create a normal base column. -- Artifacts: [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]]. - -** 2026-06-14 Sunday @ 01:07:00 -0500 -- Craig -- author -- What: Folded Craig's inline comments into the design, clarifying source/scheme controls, guidance text, the meaning of schemes, configurable accent count, and rejected-candidate behavior. Removed the comment/source blocks. -- Why: The generator UI needed to say what the user actually sees before implementation can proceed. -- Artifacts: [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]]. diff --git a/docs/specs/theme-studio-palette-generator-spec.org b/docs/specs/theme-studio-palette-generator-spec.org new file mode 100644 index 00000000..2d8fc5c6 --- /dev/null +++ b/docs/specs/theme-studio-palette-generator-spec.org @@ -0,0 +1,301 @@ +#+TITLE: Theme Studio Palette Generator -- Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-14 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DOING Theme Studio Palette Generator -- Spec +:PROPERTIES: +:ID: 2df157b8-c7c1-47a9-b080-d9586c6f424c +:END: +- 2026-07-04 Sat @ 15:30:41 -0500 — retrofitted to status-heading convention; keyword DOING from existing :STATUS: doing + +* Metadata +| Status | doing | +|----------+-------| +| Owner | Craig | +|----------+-------| +| Reviewer | Craig | +|----------+-------| +| Related | [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]] | +|----------+-------| + +* Summary +Theme Studio should grow a constraint-first palette generator that creates preview palette columns from the current theme context, then lets the user explicitly commit individual colors or whole columns. It should reuse the existing color selector as the single-color workbench instead of adding a second picker. + +The v1 feature generates palette columns only. It does not assign faces automatically; once generated colors are applied, they become normal editable palette colors with stable column ids. + +* Problem / Context +Theme Studio now has stable color columns, span controls, OKLCH editing, contrast metrics, DeltaE warnings, locks, package faces, and a live preview. The missing workflow is generating a coherent set of candidate base colors without manually choosing each hue, checking contrast, spanning, and then adjusting by eye. + +Generic palette generators are not a good fit by themselves. They optimize visual harmony before text readability, while Emacs themes need dense syntax colors, UI overlays, selections, search hits, diagnostics, and package faces to remain legible over a fixed ground. + +There is also UI overlap with the existing color selector. The selector already edits one active color, shows a swatch, drives the hex field and OKLCH picker, and adds or updates palette colors. A generator should feed that selector, not duplicate it. + +* Goals and Non-Goals +** Goals +- Generate candidate palette columns from explicit source modes and current theme constraints. +- Default to OKLCH generation so lightness and chroma are predictable. +- Default to a syntax-balanced scheme designed for readable code themes. +- Preview generated columns without mutating the real palette. +- Reuse the current color selector to inspect and tune one generated tile at a time. +- Allow adding a generated tile as a new base color column. +- Allow appending a whole generated column. +- Preserve stable column ids and existing assignments where possible when applying proposals. +- Expose diagnostics for contrast, DeltaE, gamut clamp, and rejected candidates. + +** Non-Goals +- Automatically assign generated colors to syntax/UI/package faces in v1. +- Replace the existing manual picker, swatch, or per-column span controls. +- Import external palettes, CSS colors, image colors, or theme files. Import organization remains separate. +- Generate terminal/ANSI palettes in v1. +- Add OKHSL/OKHSV generation modes in v1. +- Rewrite bg/fg automatically in v1. + +** Scope tiers +- v1: generator panel, pure planner, preview columns, existing color selector integration, add generated tile as base column, append whole generated column, diagnostics, tests, README docs. +- Out of scope: automatic face assignment, external imports, image extraction, terminal colors, rewriting bg/fg. +- vNext: replace selected columns, regenerate selected spans, regenerate generator-owned columns, OKHSL/OKHSV controls, whole-palette harmonization, CVD-aware scoring, named style presets, terminal palette derivation. + +* Design +The generator is a panel above the real palette columns. It contains generation controls and a preview area, but no separate color picker. The existing color selector remains the place where one color is inspected, tuned, named, and committed. + +For the user, the workflow is: + +1. Choose a generator source mode and scheme from compact controls in the generator panel. +2. Click preview. +3. Inspect temporary generated columns above the real palette. +4. Click a generated tile to load it into the existing color selector. +5. Optionally tune that color with the existing hex/OKLCH picker. +6. Commit either one generated tile as a new base column, or a whole generated column as normal palette entries. + +The preview layer is separate from =PALETTE=. Rendering preview columns must not create real palette entries, move existing columns, or change assignments. Preview tiles should look like palette tiles, but have a distinct preview treatment such as a dashed border/header or subtle "preview" label. + +The generator controls should be compact and explicit: + +- =Source= is a segmented control in v1: =bg/fg= and, if implemented in v1, =selected=. =bg/fg= is the default. Each option has a short tooltip: =bg/fg= means "generate from the current ground/foreground constraints"; =selected= means "harmonize around selected palette columns." +- =Scheme= can be a select/dropdown because it has more choices. A scheme is the hue-placement strategy used to propose candidate accents. The choices are not final color decisions; they are starting layouts that the planner filters through contrast, lightness, chroma, gamut, and DeltaE constraints. +- The scheme dropdown should include one-line help in the option title or adjacent help text. For example: =syntax-balanced= means "spread readable code accents across the hue wheel"; =analogous= means "nearby hues"; =triadic= means "three evenly-spaced hue families"; =manual= means "use the hues entered by the user." +- =Accent count= controls how many base columns the proposal tries to generate. The default is 8, with a v1 range of 3-12. +- =Span count= controls how many generated span steps each proposed column includes. The default should be conservative, likely 0 or the current preferred span setting, with a v1 range of 0-4. + +Therefore, the number of generated columns is configurable: it is the =accent count=, subject to rejected candidates. If the user asks for 8 accents and 2 are rejected by constraints, the preview should show 6 generated columns and report 2 rejected candidates in the summary. It should not silently backfill unrelated colors unless that behavior is explicitly added later. + +The current color selector gains a third selection mode: + +- No selection: =+ add color= creates a new manual base column from the selector value. +- Real palette tile selected: =update selected= changes that palette tile or recolors its column as today. +- Generated preview tile selected: the selector shows the generated hex/name, and =+ add color= commits that generated color as a new base color column. =update selected= is disabled unless a real palette tile is selected. + +A small status label near the selector should make this state explicit: + +#+begin_example +editing: new color +editing: palette color blue +editing: generated color blue-2 +#+end_example + +Generated columns are proposed from a source mode: + +- =bg/fg only= is the v1 default. It uses the ground and foreground as constraints, plus the chosen scheme/base hue. +- =selected columns= is v1 optional if cheap; otherwise vNext. It harmonizes around columns the user explicitly selected. +- =whole palette= is vNext. It should not be automatic in v1 because imported, experimental, or throwaway colors could make generation unpredictable. + +Generation is constraint-first. The planner chooses hue candidates from the scheme, then searches for useful OKLCH lightness/chroma values that satisfy the current background, contrast target, DeltaE separation, and gamut constraints. Classical harmony schemes are input layouts, not the final authority. + +For implementers, the feature is a pure planning layer plus DOM rendering/application: + +- The planner receives current palette, ground, source mode, generator config, and locks. +- The planner returns a proposal object and never mutates global state. +- The DOM layer renders proposal columns as preview columns. +- Applying one tile or one column converts proposal members into normal palette entries using existing column-id, span, name-collision, and gone/repoint behavior. + +** Generator config +V1 config fields: + +- sourceMode: bg-fg, selected-columns +- scheme: syntax-balanced, analogous, split-complementary, triadic, tetradic, warm-cool, manual +- baseHue: degrees, used by non-manual schemes +- manualHues: list of degrees, used by manual mode +- accentCount: integer, default 8, range 3-12 +- spanCount: integer, default 0 or current preferred span, range 0-4 in v1 +- textLightnessBand: min/max OKLCH L for text accents +- chromaBias: subdued, balanced, vivid +- contrastTarget: none, WCAG AA, WCAG AAA +- deltaEMin: default to the existing palette warning threshold +- locks: respect locked columns and assignments where apply modes touch existing data + +** Proposal object +The planner returns a proposal object with the generator config, proposed columns, rejected candidates, and a summary. Each proposed column carries a stable column id, display name, base hex, member colors with offsets and clamp/metric data, and column-level diagnostics. Rejected candidates carry the attempted hue, rejection reason, and nearest conflicting column when relevant. The summary includes generated count, clamped count, rejected count, minimum contrast, and minimum DeltaE. + +This shape is intentionally close to the existing palette-column model. Preview rendering should not need a second color model. + +** Applying generated colors +The first v1 apply actions are deliberately small: + +- =Add generated tile as base column=: creates a new normal palette column from the selected preview tile. The new column id is derived from the tile name and suffixes on collision. +- =Append generated column=: adds every member of that preview column after existing real columns. Members keep a stable shared column id. +- =Clear preview=: discards proposal state without changing the real palette. + +The following apply actions are deferred unless v1 implementation is already straightforward: + +- replace selected columns +- regenerate spans only +- regenerate generator-owned columns + +When a generated tile is committed as a base column, it starts as a one-tile column. The user can then span it using the existing column span widget. This keeps the one-color action easy to understand and avoids surprising multi-tile commits from a tile-level button. + +** Display +The panel sits between the color selector row and the committed palette columns. It has: + +- source mode segmented control +- scheme segmented control or select +- base hue/manual hue controls +- accent count and span count numeric controls +- chroma bias control +- contrast target control +- preview and clear-preview buttons +- summary row: min contrast, min DeltaE, clamped count, rejected count + +Preview columns render below the controls and above committed columns. A generated tile click loads the existing selector. A generated column header click loads the column base into the selector. A column-level =append column= button commits the whole preview column. + +* Alternatives Considered +** Generic harmony wheel that writes directly into the palette +- Good, because it is familiar and visually compact. +- Bad, because it mutates real palette state before the user can inspect results, and it optimizes hue harmony before readability. +- Neutral, because a hue wheel can still be useful as an input control inside a preview-first generator. + +** Separate generator-specific color picker +- Good, because preview tuning could be isolated from committed palette editing. +- Bad, because Theme Studio already has a capable single-color selector, and a second picker would duplicate hex, OKLCH, contrast, swatch, and add/update semantics. +- Neutral, because a future advanced generator could add a small detail panel, but v1 should not. + +** Generate from the whole palette by default +- Good, because it can harmonize with everything already on screen. +- Bad, because the palette may contain experiments, imported colors, temporary colors, or intentionally clashing accents; using all of them makes results hard to predict. +- Neutral, because whole-palette harmonization is valuable as an explicit vNext mode. + +** Full automatic face assignment +- Good, because it could produce a near-complete theme in one action. +- Bad, because it crosses into seeding, locks, role maps, and package face behavior that already have separate product decisions. +- Neutral, because the palette generator can feed a later seeding workflow. + +** Add OKHSL/OKHSV now +- Good, because those controls may feel friendlier than raw OKLCH. +- Bad, because v1 already has OKLCH math and the feature risk is workflow/state, not another color model. +- Neutral, because OKHSL/OKHSV remain good vNext candidates. + +* Decisions [5/5] +** DONE Default to preview-first generation +- Context: Generator output can disrupt a carefully tuned palette if it mutates immediately. +- Decision: We will render generated colors as temporary preview columns and require explicit add/append actions. +- Consequences: Easier to inspect and avoid accidental data loss; harder because the UI needs proposal state and apply semantics. + +** DONE Reuse the current color selector +- Context: The existing selector already edits one color, shows metrics, opens the picker, and adds/updates palette colors. +- Decision: We will make generated tile clicks load the existing selector instead of adding a second generator picker. +- Consequences: Easier to keep editing behavior consistent; harder because the selector now needs clear state for new, palette, and generated selections. + +** DONE Keep v1 palette-only +- Context: Automatic assignment would touch syntax, UI, package faces, locks, and seeding rules. +- Decision: We will generate palette columns only in v1 and leave face assignment to existing/manual workflows. +- Consequences: Easier to ship a focused generator; harder because the user still maps colors onto faces. + +** DONE Default generation source to bg/fg only +- Context: Existing palette colors may be experimental or imported; using all of them by default makes generation unpredictable. +- Decision: We will default to bg/fg constraints plus explicit scheme/base hue. Selected-column source can be included if scoped; whole-palette source is vNext. +- Consequences: Easier to understand why a proposal was generated; harder because matching an existing palette requires an explicit source mode. + +** DONE Defer OKHSL/OKHSV to vNext +- Context: OKHSL/OKHSV may be friendlier interaction models, but OKLCH already supports the required perceptual generation math. +- Decision: We will ship OKLCH generation first and consider OKHSL/OKHSV after v1 is usable. +- Consequences: Easier to keep v1 small and rigorous; harder because some users may find OKLCH controls less familiar. + +* Implementation phases +** Phase 1 -- Planner core +Add pure generator functions in =app-core.js= or a new generator module. Inputs are current palette, ground, generator config, source selection, and locks. Outputs are proposal objects. Unit tests cover scheme hue placement, OKLCH candidate generation, gamut clamp reporting, name/id collision handling, and no mutation of input state. + +** Phase 2 -- Candidate scoring +Add bounded scoring/adjustment for contrast target, DeltaE separation, chroma bias, and text lightness band. Unit tests cover rejected candidates, clamped colors, low-chroma distinguishability, and deterministic output for fixed config. + +** Phase 3 -- Generator panel and preview rendering +Add the panel, controls, proposal state, preview columns, summary metrics, and clear-preview behavior. Browser gate: preview creates temporary columns without changing committed =PALETTE=. + +** Phase 4 -- Existing selector integration +Add generated-preview selection state to the color selector. Clicking a preview tile loads its hex/name. The selector status label shows generated-vs-palette-vs-new. =+ add color= commits the selected preview tile as a new one-tile base column. Browser gates cover generated tile selection and add-as-column behavior. + +** Phase 5 -- Append generated column +Add column-level append. The generated column becomes normal palette entries with one stable column id. Collisions suffix names/ids deterministically. Browser gates cover append, collision suffixing, and unchanged existing assignments. + +** Phase 6 -- Persistence and metadata +Round-trip optional generator metadata for applied columns without requiring it for normal palette behavior. Existing palettes without metadata continue to work. Browser gate extends roundtrip coverage. + +** Phase 7 -- Documentation and cleanup +Document generator controls, source modes, preview behavior, selector integration, and limits in =scripts/theme-studio/README.md=. Remove prototype code and keep =make -C scripts/theme-studio test= green. + +* Acceptance criteria +- [ ] Previewing a generated palette does not mutate committed =PALETTE=. +- [ ] Preview columns appear above committed columns and are visually distinct. +- [ ] Clicking a generated tile loads that color into the existing selector. +- [ ] The selector clearly says whether it is editing a new color, palette color, or generated color. +- [ ] =+ add color= on a selected generated tile creates a new one-tile base column. +- [ ] Appending a generated column creates normal editable palette entries with one stable column id. +- [ ] Name and column-id collisions are suffixed deterministically. +- [ ] Generated colors report clamp, contrast, and DeltaE diagnostics. +- [ ] Existing manual palette workflows still work without opening the generator panel. +- [ ] Theme Studio tests cover planner functions, preview rendering, selector integration, apply behavior, and round-trip metadata. + +* Readiness dimensions +- Data model & ownership: Proposal state is transient and generator-owned. Applied colors become normal user-owned palette entries. Optional generator metadata is advisory and must not override manual edits. +- Errors, empty states & failure: Invalid config disables preview with an inline message naming the bad field. No preview shows an empty-state line. Rejected candidates appear in the summary. Apply failures must not partially mutate committed palette state. +- Security & privacy: N/A because generation is local deterministic color math with no credentials, network calls, or private external data. +- Observability: The preview summary shows generated, clamped, rejected, min contrast, and min DeltaE. Tile titles or details expose per-color diagnostics. +- Performance & scale: Expected accent counts are 3-12 bases with 0-4 span steps. Candidate search should remain synchronous and bounded. Broader search, if added later, needs progress/cancel. +- Reuse & lost opportunities: Reuse OKLCH, gamut clamp, contrast, DeltaE, palette columns, span generation, locks, existing selector, and existing browser gates. Do not add a second color math stack or picker. +- Architecture fit & weak points: The weak point is proposal/apply state in the DOM app. Keep planner pure and DOM code limited to rendering/applying proposal objects. +- Config surface: Public knobs are source mode, scheme, base/manual hue, accent count, span count, chroma bias, contrast target, DeltaE threshold, and lightness band. Defaults favor readable dark-theme syntax. +- Documentation plan: Update the Theme Studio README with generator controls, source modes, selector integration, preview/apply behavior, and v1 limits. +- Dev tooling: Use =make -C scripts/theme-studio test= as the primary gate and =make -C scripts/theme-studio coverage= for instrumented JS/generator coverage. +- Rollout, compatibility & rollback: The generator is additive. Existing palettes load unchanged. Preview can be cleared. Applied generated columns can be deleted manually. +- External APIs & deps: N/A because v1 has no external APIs or new dependencies. + +* Risks, Rabbit Holes, and Drawbacks +- Candidate search can become an optimization rabbit hole. V1 should use deterministic bounded search around target OKLCH bands. +- "Syntax-balanced" is subjective. Keep it documented as a heuristic, not a claim of universal taste. +- Selector state can become confusing. The status label and disabled update button are required, not polish. +- Whole-palette harmonization is tempting but should wait until preview/apply basics feel good. +- Optional metadata can drift after manual edits. Treat it as advisory only. + +* Testing / Verification / Rollout +Use the existing Theme Studio test stack: + +- Node tests for planner/scoring/collision/immutability. +- Browser hash gate for preview-only non-mutation. +- Browser hash gate for generated tile -> selector state. +- Browser hash gate for add-generated-tile-as-column. +- Browser hash gate for append generated column. +- Round-trip gate for optional generator metadata. +- Manual Chrome pass on at least one dark palette and one light palette. + +* References / Appendix +- [[file:design/theme-studio-color-harmony.org][theme-studio color harmony explainer]] +- [[id:15db8ae3-fc14-49f3-9ed5-d5ff59790904][perceptual color metrics spec]] +- [[file:theme-studio-palette-ramps-spec.org][palette ramps and contrast safety spec]] +- [[file:theme-studio-palette-columns-spec.org][palette columns spec]] +- [[file:../../todo.org::*theme-studio import organization workflow needs a spec][import organization task]] + +* Review and iteration history +** 2026-06-13 Saturday @ 16:31:01 -0500 -- Craig -- author +- What: Initial draft using the spec-create workflow. +- Why: Palette generation has real design trade-offs around color space, preview/apply behavior, assignment boundaries, and how much generator state should persist. +- Artifacts: [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]]. + +** 2026-06-14 Sunday @ 00:44:00 -0500 -- Craig -- author +- What: Reworked the draft around preview-first generation, existing color selector integration, generated tile add-as-column behavior, and source-mode defaults. +- Why: Craig clarified the desired UX: generated colors should be inspectable/tunable through the existing selector, and committing one generated color should create a normal base column. +- Artifacts: [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]]. + +** 2026-06-14 Sunday @ 01:07:00 -0500 -- Craig -- author +- What: Folded Craig's inline comments into the design, clarifying source/scheme controls, guidance text, the meaning of schemes, configurable accent count, and rejected-candidate behavior. Removed the comment/source blocks. +- Why: The generator UI needed to say what the user actually sees before implementation can proceed. +- Artifacts: [[file:../../todo.org::*theme-studio palette generator][theme-studio palette generator task]]. diff --git a/docs/specs/theme-studio-perceptual-color-metrics-spec-implemented.org b/docs/specs/theme-studio-perceptual-color-metrics-spec-implemented.org deleted file mode 100644 index 57a4c70b..00000000 --- a/docs/specs/theme-studio-perceptual-color-metrics-spec-implemented.org +++ /dev/null @@ -1,580 +0,0 @@ -:PROPERTIES: -:ID: 15db8ae3-fc14-49f3-9ed5-d5ff59790904 -:STATUS: implemented -:END: -#+TITLE: theme-studio — perceptual color metrics (OKLCH, APCA, ΔE) -#+AUTHOR: Craig Jennings -#+DATE: 2026-06-08 - -* Status - -Spec / review incorporated (Codex, 2026-06-08, two passes). Adds perceptual color metrics to -the theme-studio (=scripts/theme-studio/=) so it can build deliberately -low-contrast themes (Solarized / Zenburn class) with the same rigor it already -brings to high-contrast WCAG checking. Four additions: an OKLCH color model, a -per-color perceptual-lightness readout, an APCA contrast score alongside the -existing WCAG ratio, and a pairwise ΔE distinguishability check across the -palette. - -Came out of a design conversation comparing the low-contrast school (Solarized, -Zenburn) against Prot's high-contrast Modus themes. The conclusion: a theme has -three independent dials — contrast ratio, overall luminance, and chroma — and -the low-contrast camp turns down the first while Modus leaves it high and turns -down the other two. The current tool only measures the first (WCAG contrast) and -edits color in HSV, whose "lightness" is not perceptually uniform. To build -low-contrast themes by metric rather than by eye, the tool needs -perceptually-uniform lightness and chroma controls plus distinguishability and -polarity-aware contrast measures. - -Rubric: *Ready.* The four v1 product questions are decided in "Agreed decisions -(v1)" below and confirmed by Craig (2026-06-08); the testing strategy was -revised on his direction to a layered pyramid (Node-unit-tested color core + -thin UI hash tests + measured coverage). No remaining blocking ambiguity — the -implementer no longer has to invent product behavior while coding. Implementation -is sequenced into five phases, each independently shippable and tested. Tasks -filed in =todo.org=. - -* Background — the current color model - -The tool today works entirely in sRGB hex, HSV, and WCAG luminance. The relevant -cluster in =generate.py=: - -- =rl(hex)= (line 517) — WCAG relative luminance, via the existing =lin()= - sRGB-linearization helper. -- =contrast(a,b)= (line 519) — the WCAG 2.x ratio =(L1+0.05)/(L2+0.05)=. -- =rating(r)= / =ratingColor(r)= (lines 520-521) — AA (≥4.5) / AAA (≥7) verdict - and its display color. -- =hsv2rgb=, =rgb2hsv=, =hex2rgb=, =rgb2hex=, =normHex= (lines 604-609). -- The picker holds state as =pkH, pkS, pkV= (HSV) and renders an SV box (=#sv=), - a hue strip (=#hue=), crosshairs (=#svcur=, =#huecur=), a hex+contrast readout - (=#pkhex= / =#pkcon=, line 451), the contrast-mask buttons (=.pmode=, state - =pkMode=, values =any= / =aa= / =aaa=), and palette chips (=#pkchips=). -- =drawMask()= (line 613) greys SV-box regions whose contrast against the - background falls below the selected mask threshold (=pkThresh()=). -- Per-face contrast readouts appear across *three* tables — syntax (line 548), - UI (line 1064), and package faces (line 752) — each via =contrast()= + - =rating()=. The package-face tier has grown large since the tool's early - versions (51 packages in the current inventory), so any "add a column to the - table readouts" change now touches that whole surface, not just the two - original tables. - -Two limitations this spec addresses: - -1. *HSV lightness lies.* Two colors at the same HSV =V= can differ markedly in - perceived brightness, so the SV box cannot hold perceived lightness constant - while hue changes — exactly the operation a calm, even palette needs. -2. *WCAG 2.x is a known-flawed contrast model in the low-contrast / dark band.* - Its ratio misjudges contrast most where this work operates, and it is not - polarity-aware: it scores light-on-dark and dark-on-light identically, which - perception does not. WCAG 3 is reworking contrast but is years out — still a - Working Draft in 2026, with the final Recommendation not expected until - roughly 2028–2030 — and its contrast algorithm is undetermined: APCA was moved - *out* of the WCAG 3 draft in 2023 for further evaluation. So APCA enters here - as a well-regarded independent perceptual model used as an additional - diagnostic, not as a coming standard. WCAG 2.x stays the baseline precisely - because nothing has replaced it yet. - -* Goal - -Add four metrics, each a discrete increment: - -1. *OKLCH color model* — perceptually-uniform Lightness / Chroma / Hue, so the - editor can move one axis without disturbing the others, plus a gamut clamp - for OKLCH values outside sRGB. -2. *Perceptual-lightness readout* — show each color's OKLCH L (and C, H) in the - picker, so "low, even lightness steps" becomes a number rather than a guess. -3. *APCA score* — the Accessible Perceptual Contrast Algorithm Lc value - displayed next to the WCAG ratio, as the more trustworthy contrast metric in - the low-contrast band. -4. *Pairwise ΔE check* — perceptual color-difference between every pair of - palette entries, flagging pairs too similar to tell apart, which is the - constraint that keeps a low-chroma / low-lightness-spread palette from - collapsing into mush. - -Non-goals: replacing WCAG (it stays as the compatibility baseline, shown -alongside APCA, which is an additional perceptual diagnostic, not a -replacement); replacing the HSV picker outright (OKLCH is added as a parallel -color model, HSV remains the default); CIEDE2000 in v1 (ΔE-OK is the v1 -difference metric — see vNext). - -* Agreed decisions (v1) - -Settled on author + reviewer alignment and confirmed by Craig (2026-06-08). - -1. *ΔE metric and threshold.* Use ΔE-OK (Euclidean distance in OKLab) on its - native scale (OKLab L is 0..1). Default "too similar" warning threshold is - *0.02* — the just-noticeable-difference floor, so the warning fires only when - two palette colors are genuinely hard to tell apart. The threshold is a named - constant, calibratable in one place. CIEDE2000 (the CIE's 2000 perceptual - color-difference standard — more accurate than plain Euclidean distance, but - ~40 lines of trigonometric lightness/chroma/hue corrections plus a blue-region - rotation term) is deferred to vNext: ΔE-OK is accurate enough to flag - indistinguishable pairs, which is all this check needs, and it is five lines. -2. *Low-contrast preset.* v1 ships *readouts only* (OKLCH, APCA, ΔE). No named - low-contrast preset / mask mode yet. No such preset exists anywhere today — it - would be a new feature: a saved low-contrast target (e.g. an APCA Lc band, or - a contrast ceiling as well as a floor) that masks the palette to a comfortable - range in one click, the way the current any/AA+/AAA buttons mask by a contrast - floor. It is deferred until the raw readouts are in use, because only then is - it clear which band is worth presetting. v1 gives the numbers; the preset - would automate a judgment the numbers first have to inform. -3. *APCA placement.* v1 shows APCA *only in the picker* readout, not in the - syntax/UI/package table contrast cells. Adding it to the tables is - low-complexity once =apca()= exists — the same pattern as the existing - =contrast()= + =rating()= cells, repeated across the three tables — so the - deferral is about table *density*, not difficulty: the package table alone is - 51 packages wide, and a second contrast number per row risks clutter before - it is clear anyone reads it there. Table-wide APCA is a vNext candidate if - picker-only proves too hidden. -4. *Picker default model.* HSV stays the *default* picker model; OKLCH is - opt-in via a color-model control. The reason: HSV is the familiar 2D SV-box - the picker already has, and OKLCH is slider-only until the C×L plane (Phase - 4b) lands — so defaulting to OKLCH before 4b would hand users a worse default - editing surface than they have now. Once 4b ships the C×L plane, making OKLCH - the default becomes a real option worth revisiting; until then, HSV default - keeps the current editing experience intact and makes OKLCH an additive - choice, not a regression. - -* Color-math foundation (Phase 1, prerequisite) - -The pure color math is *extracted into its own importable module* rather than -inlined as loose functions in the page. This is the core architectural change -this spec makes to the test surface: the math is logic, so it gets tested as -logic — directly, in Node, with exhaustive fixtures — and the picker becomes a -thin UI layer over a tested core, not the only way to exercise the math. - -- New file: =scripts/theme-studio/colormath.js= — the pure, side-effect-free - conversion + metric functions, written as an ES module (each =export=-ed), - with a small guard so the same source loads both ways: =import=-ed by the Node - tests and spliced into the page by the generator. -- =generate.py= inlines =colormath.js= into the page's =