aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2026-05-04 04:12:18 -0500
committerCraig Jennings <c@cjennings.net>2026-05-04 04:12:18 -0500
commitb3ef232a486382601b6788f0c1a1edeb3982075d (patch)
treeadb4d20268be674da7f05f0fe711df0ce73b0583 /docs
parent31c913b26a9725595357f678e7197cc7efcd6c53 (diff)
downloaddotemacs-b3ef232a486382601b6788f0c1a1edeb3982075d.tar.gz
dotemacs-b3ef232a486382601b6788f0c1a1edeb3982075d.zip
docs: add init.el load-graph and utility-consolidation specs
I added two sibling design specs in `docs/design/`: `init-load-graph.org` covers untangling `init.el` from its current "everything eager in a fixed order" shape. It defines a layered architecture (early-init / foundation / core UX / domain workflow / optional), a module category table for every required file, a per-file commentary header standard with seven required lines, a six-phase migration plan with exit criteria, and a testing strategy split into automated batch checks, manual smoke checks, and startup performance baselines via `benchmark-init`. `utility-consolidation.org` is the sibling project. It covers extracting reusable helpers from feature modules into `system-lib.el` and a small set of topic libraries (`cj-process.el`, `cj-org-text.el`, `cj-cache.el`). It includes a candidate decision criteria section, a library file header standard with worked example, a candidate extraction table with priorities and proposed names, nine helper groups with API plus behavior contracts, naming rules, migration phases, test relocation policy, and a recommended first-three-commits sequence. Both specs are draft. No code change in this commit. The two projects are intentionally separated because the load-graph project asks "when does this load?" and the consolidation project asks "who owns this helper?". Those are different questions with different rollback shapes. Implementation tracking lives in `todo.org`.
Diffstat (limited to 'docs')
-rw-r--r--docs/design/init-load-graph.org829
-rw-r--r--docs/design/utility-consolidation.org1216
2 files changed, 2045 insertions, 0 deletions
diff --git a/docs/design/init-load-graph.org b/docs/design/init-load-graph.org
new file mode 100644
index 00000000..d4a68f47
--- /dev/null
+++ b/docs/design/init-load-graph.org
@@ -0,0 +1,829 @@
+#+TITLE: Design: Untangle the init.el Load Graph
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-05-04
+
+* Status
+
+Draft. Specification only. No load-order implementation is part of this design
+document.
+
+* Problem
+
+=init.el= is currently both the startup script and the dependency graph. It
+eagerly requires almost every module in a fixed order, so many modules work
+because some earlier require happened to define a variable, keymap, path
+constant, hook owner, package, or helper function.
+
+That creates four practical problems:
+
+- Standalone module loading is unreliable. A module may byte-compile but fail at
+ runtime unless enough of =init.el= was loaded first.
+- Startup has unnecessary work. Optional workflows, heavy packages, timers,
+ network-facing integrations, and media tools load even when not used.
+- Side effects are hard to audit. Keybindings, timers, global hooks, server
+ setup, package configuration, and command definitions are mixed together.
+- Test boundaries are blurry. Tests often need to simulate init order instead of
+ loading the unit under test directly.
+
+The target is not "lazy load everything." The target is an explicit, testable
+load graph where eager startup is a small documented set, optional workflows
+load from commands/hooks/autoloads, and module dependencies are declared by the
+modules that use them.
+
+* Goals
+
+- Make module ownership obvious: libraries, keymap ownership, package
+ configuration, commands, and startup side effects should be distinguishable.
+- Make dependencies explicit with ordinary =require=, =autoload=, or documented
+ hook/package boundaries.
+- Reduce eager startup load without breaking existing keybindings or daily
+ workflows.
+- Keep the migration incremental and reversible. Each batch should be small
+ enough to test and inspect.
+- Preserve interactive behavior for configured workflows, including calendar
+ sync, Org capture/agenda, mail, F-keys, and media commands.
+- Improve testability: modules should either load directly or fail with a clear
+ missing external package/config message.
+
+* Non-Goals
+
+- Rewriting the whole configuration into one framework or literate init.
+- Removing =use-package=. This design assumes package config modules continue to
+ use it where appropriate.
+- Eliminating all top-level forms. Some top-level configuration is appropriate,
+ especially for foundational Emacs settings and hook registration.
+- Solving package bootstrap in =early-init.el=. That is tracked by the separate
+ "Move package bootstrap out of =early-init.el= where possible" project.
+- Rotating calendar feed URLs or designing secret storage beyond the local
+ calendar config path already introduced. Token rotation remains a separate
+ security task.
+- Consolidating all scattered utility helpers. Utility consolidation is a
+ sibling project because it changes helper ownership, tests, and call sites
+ without necessarily changing startup load order.
+
+* Principles
+
+** Eager Requires Are Allowed Only With A Reason
+
+An eager require in =init.el= should satisfy one of these conditions:
+
+- It establishes basic Emacs behavior needed for the rest of startup.
+- It defines shared constants or helpers used by many eager modules.
+- It owns the global key prefix/keymap registration system.
+- It configures core UI behavior that should be visible in the first frame.
+- It starts a user-approved startup service that cannot be triggered lazily.
+
+Everything else should be a candidate for autoload, hook-based loading,
+=with-eval-after-load=, or a command wrapper.
+
+** Modules Declare What They Use
+
+If a module calls a function or reads a variable at runtime, it should not rely
+on init order unless that dependency is an explicit startup contract.
+
+Preferred dependency forms:
+
+- Runtime dependency: =(require 'module)=.
+- Optional runtime dependency: =(require 'module nil t)= with a clear degraded
+ behavior.
+- Macro/compile-time dependency: =(eval-when-compile (require 'module))=.
+- Command-only dependency: =(autoload 'command "module" nil t)= or a lazy
+ command wrapper.
+- Package-bound dependency: =use-package :after=, =:hook=, =:commands=, or
+ =with-eval-after-load=.
+
+Avoid test-only shims in production modules such as "define this keymap if it
+does not exist." Tests should provide stubs or load the real owner.
+
+** Utility Extraction Should Stay Small And Evidence-Based
+
+Some hidden dependencies exist because generic helpers live in feature modules
+where they were first needed. Moving those helpers into =system-lib= can make
+dependencies clearer, but utility extraction should not become part of every
+load-order change by default.
+
+Extract a helper only when:
+
+- at least two callers need substantially the same behavior,
+- the helper can stay dependency-light enough for foundation startup,
+- tests can move with the helper,
+- the extraction is atomic and easy to review.
+
+Avoid building a broad utility suite speculatively. Prefer one helper, one
+tested extraction, one commit.
+
+** Keymaps Have Owners
+
+=keybindings.el= should own global prefixes, especially =cj/custom-keymap= and
+the =C-;= prefix. Feature modules may define local maps or command maps, but
+registration into global prefixes should go through a small convention/helper so
+load order is not a hidden dependency.
+
+** Side Effects Are Named And Isolated
+
+Side effects include:
+
+- starting timers,
+- starting processes,
+- calling network-facing sync/fetch commands,
+- setting global keybindings,
+- mutating global hooks,
+- opening files/buffers,
+- enabling global modes,
+- loading large packages solely for optional commands.
+
+Each side effect should have one of:
+
+- a documented eager reason,
+- an interactive command,
+- a hook/package boundary,
+- a noninteractive/batch guard,
+- a test that proves the side effect does not happen in the wrong context.
+
+* Target Architecture
+
+** Layer 0: Early Startup
+
+Owned by =early-init.el=. Should remain limited to startup mechanics that must
+happen before package/UI initialization.
+
+Examples:
+
+- package archive/bootstrap policy,
+- native-comp/cache startup knobs that must be early,
+- disabling expensive default UI before first frame.
+
+This design does not refactor =early-init.el= except to avoid adding new load
+graph responsibilities to it.
+
+** Layer 1: Foundation
+
+Small eager set required before most other modules can safely load.
+
+Expected contents:
+
+- =system-lib=
+- =user-constants=
+- =host-environment=
+- =system-defaults=
+- =keyboard-compat=
+- =keybindings=
+- maybe =config-utilities=, if debug helpers are intentionally eager
+
+Foundation modules should be able to load in batch mode without package,
+network, timer, or UI-package side effects.
+
+Adding a new Layer 1 module requires a coordinated update to the
+=system-lib.el= dependency budget in [[file:utility-consolidation.org][utility-consolidation.org]].
+
+Topic libraries introduced by the utility project join Layer 1 only when their
+first consumer is foundation-eager. Otherwise they are Layer 2 and loaded by an
+explicit =require= from their eager consumers. Add each new topic library to the
+module category table before migrating its first consumer.
+
+** Layer 2: Core UX
+
+Eager or near-eager modules that shape the first interactive session.
+
+Expected contents:
+
+- basic text/editing defaults,
+- core UI frame/theme/font/modeline behavior,
+- selection/completion framework,
+- F-key development entry points,
+- VC/test/coverage command entry points.
+
+Core UX modules may configure packages, but heavy features should still use
+=:commands=, =:hook=, or =:defer= where practical.
+
+** Layer 3: Domain Workflows
+
+Org, programming, mail, browser, media, AI, and integration modules. These
+should generally load through hooks, commands, package =:after= clauses, or
+workflow-specific entry commands.
+
+Examples:
+
+- Org capture/agenda can remain eager if the user's daily workflow needs it,
+ but exporters and optional extensions can be deferred.
+- Language modules should load from mode hooks or file associations, not because
+ every startup might edit every language.
+- Mail/media/AI/rest tools should register commands eagerly if needed, then load
+ heavy packages only on use.
+
+** Layer 4: Optional And Experimental
+
+Entertainment, modules in test, diagnostics, and rarely used tools. These should
+not be required by default unless the user explicitly chooses that behavior.
+
+Examples:
+
+- =games-config=
+- =music-config=
+- =lorem-optimum=
+- =gloss-config=
+- optional IRC/Slack/feed/media modules when not in active use
+
+* Module Categories
+
+This is a first-pass classification to guide implementation. It is not an
+architectural truth table; each module should be confirmed while refactoring.
+
+Category key:
+
+- =F= foundation or shared library/config.
+- =C= core eager UX.
+- =P= package configuration that should usually be hook/command/package loaded.
+- =D= domain workflow that may have a user-visible eager reason.
+- =S= startup side-effect or timer/process owner.
+- =O= optional, entertainment, experimental, or rarely used.
+- =L= pure-ish library/command helpers that should be easy to load directly.
+
+| Module | Category | Expected final load shape | Notes |
+|--------+----------+---------------------------+-------|
+| =early-init= | F | early | Layer 0; see Non-Goals. |
+| =system-lib= | F/L | eager | Low-level helpers. Keep side-effect free. |
+| =cj-process= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 3. |
+| =cj-org-text= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 6. |
+| =cj-cache= | F/L | TBD per first consumer | Topic library placeholder; see utility-consolidation Group 7. |
+| =user-constants= | F | eager, then split | Split pure path constants from directory creation/failure behavior. |
+| =host-environment= | F/L | eager | Predicate helpers. |
+| =system-defaults= | F/S | eager | Owns global Emacs defaults, server/recentf/minibuffer hooks. |
+| =keyboard-compat= | F/S | eager | Terminal/GUI keyboard setup hooks. |
+| =keybindings= | F/C | eager | Owner of =cj/custom-keymap= and global prefixes. |
+| =config-utilities= | C/O | eager or command-loaded | Debug keymap may be eager; heavy org parsing commands can lazy require. |
+| =custom-case= | L/C | autoload commands + key registration | Text command helper. |
+| =custom-comments= | L/C | autoload commands + key registration | Text command helper. |
+| =custom-datetime= | L/C | autoload commands + key registration | Text command helper. |
+| =custom-buffer-file= | L/C | eager only if remaps required | Has file/process helpers and keymap registration. |
+| =custom-line-paragraph= | L/C | autoload commands + key registration | Requires =expand-region= at command boundary if possible. |
+| =custom-misc= | L/C | autoload commands + key registration | Misc commands. |
+| =custom-ordering= | L/C | autoload commands + key registration | Text command helper. |
+| =custom-text-enclose= | L/C | autoload commands + key registration | Text command helper. |
+| =custom-whitespace= | L/C | autoload commands + key registration | Text command helper. |
+| =external-open= | L/D | autoload commands | Runtime requires environment/process helpers explicitly. |
+| =media-utils= | D | command-loaded | Downloads/players should run only by command. |
+| =auth-config= | F/D | eager or package-after | Auth setup may be core; GPG commands should remain commands. |
+| =keyboard-macros= | C | eager or keymap-only | Lightweight command/key owner. |
+| =system-utils= | L/C | eager or command-loaded | Timers/process monitor utilities. |
+| =text-config= | C/P | eager hooks | General text defaults and package config. |
+| =undead-buffers= | C | eager if remaps desired | Global kill-buffer remaps. |
+| =browser-config= | D/P | command/package-loaded | Browser workflow. |
+| =coverage-core= | C/L | eager command entry | F7 entry point and backend registry. |
+| =coverage-elisp= | C/P | eager after core | Backend registration; keep cheap. |
+| =dev-fkeys= | C | eager | F4/F6 command entry points. |
+| =ui-config= | C/S | eager | Cursor/UI defaults; post-command hook should be documented. |
+| =ui-theme= | C | eager + explicit startup call | Theme load stays explicit in init. |
+| =ui-navigation= | C/P | eager | Window keybindings and winner/buffer-move config. |
+| =font-config= | C/P/S | eager or first-frame | Font hooks/font installation checks need guards. |
+| =selection-framework= | C/P | eager | Completion stack; likely core UX. |
+| =modeline-config= | C/S | eager | Mode line and VC cache hooks. |
+| =mousetrap-mode= | C | eager if global behavior desired | Prevents accidental mouse edits. |
+| =popper-config= | C/P | eager if enabled, else remove/defer | Existing disabled-state question remains. |
+| =chrono-tools= | D/P | command-loaded | Calendar/timer commands; sound path dependency explicit. |
+| =diff-config= | C/P | eager or package-loaded | Diff/merge UX. |
+| =erc-config= | O/D/P | command-loaded | IRC should not be startup load by default. |
+| =slack-config= | O/D/P | command-loaded | Slack package/auth and which-key registration should be after-load. |
+| =eshell-vterm-config= | D/P | command/hook-loaded | Shell/terminal packages. |
+| =help-utils= | L/D | autoload commands | Search/help commands. |
+| =help-config= | C/P | eager or after help | Info/man/help config. |
+| =tramp-config= | D/P | package-loaded | Remote shell configuration. |
+| =calibredb-epub-config= | O/D/P | command-loaded | Ebook workflow. |
+| =dashboard-config= | C/S | eager only if startup dashboard desired | Opens/initializes landing page behavior. |
+| =dirvish-config= | D/P | command/hook-loaded | File manager; runtime constants explicit. |
+| =dwim-shell-config= | D/P | command-loaded | Shell commands from Dired/Dirvish. |
+| =elfeed-config= | O/D/P | command-loaded | Feed reader/podcast workflow. |
+| =eww-config= | D/P | command-loaded | Web browsing helpers. |
+| =flyspell-and-abbrev= | C/P | hooks | Text-mode spelling/abbrev. |
+| =httpd-config= | O/D/P | command-loaded | Local web server. |
+| =latex-config= | D/P | hook-loaded | Existing WIP comment should become tasks or be removed. |
+| =mail-config= | D/P | command-loaded or eager by choice | Heavy mu4e/org-msg; daily workflow may justify eager command registration. |
+| =markdown-config= | D/P | mode-loaded | Markdown package config. |
+| =pdf-config= | D/P | file/mode-loaded | Heavy PDF packages should load on PDF open. |
+| =quick-video-capture= | O/D/S | command/protocol-loaded | Top-level timers should be removed or guarded. |
+| =video-audio-recording= | O/D/S | command-loaded | External process/device probing only on command. |
+| =transcription-config= | O/D/P | command-loaded | Auth/process workflow. |
+| =weather-config= | O/D/P | command-loaded | Optional command. |
+| =prog-general= | C/P/S | eager or hooks | Projectile, treesit policy, LSP ownership concerns. |
+| =test-runner= | C/L | eager command entry | Test keymap and project-scoped state. |
+| =vc-config= | C/P | eager command entry | Magit/git keymap; clone command hardening separate. |
+| =flycheck-config= | C/P | hooks | General linting. |
+| =prog-training= | O/D/P | command-loaded | Exercism/Leetcode optional. |
+| =prog-c= | D/P | mode-loaded | C hooks and compile command. |
+| =prog-go= | D/P | mode-loaded | Go hooks/LSP. |
+| =prog-lisp= | D/P | mode-loaded | Lisp package config. |
+| =prog-lsp= | C/P | package policy owner | Should consolidate generic LSP policy. |
+| =prog-shell= | D/P/S | mode-loaded | after-save executable hook should be opt-in or scoped. |
+| =prog-python= | D/P | mode-loaded | Python hooks/LSP. |
+| =prog-webdev= | D/P | mode-loaded | Webdev modes/LSP. |
+| =prog-json= | D/P | mode-loaded | JSON formatting/mode config. |
+| =prog-yaml= | D/P | mode-loaded | YAML formatting/mode config. |
+| =org-config= | C/D/P | eager | Core Org behavior likely eager. |
+| =org-agenda-config= | D/S | eager by workflow, timers guarded | Agenda cache lifecycle project tracks cleanup. |
+| =org-babel-config= | D/P | after Org | Babel languages package config. |
+| =org-capture-config= | D/P | eager if capture hot path | Protocol/capture templates. |
+| =org-contacts-config= | D/P | after Org/mail | Contacts workflow. |
+| =org-drill-config= | O/D/P | command-loaded | Optional drill workflow. |
+| =org-export-config= | D/P | command-loaded | Export packages/processes. |
+| =hugo-config= | D/P | command-loaded | Blog workflow. |
+| =org-reveal-config= | O/D/P | command-loaded | Presentation workflow. |
+| =org-refile-config= | D/S | eager by workflow, timers guarded | Refile cache lifecycle project tracks cleanup. |
+| =org-roam-config= | D/P/S | eager by workflow | Capture/finalize hooks, db. |
+| =org-webclipper= | O/D/P | protocol/command-loaded | Global temp state cleanup tracked separately. |
+| =org-noter-config= | O/D/P | command-loaded | PDF notes workflow. |
+| =ai-config= | D/P | command-loaded | GPTel commands; avoid loading all AI tooling at startup. |
+| =ai-conversations= | D/L/S | after gptel | Autosave hook and persistence path need coverage. |
+| =restclient-config= | D/P | command-loaded | API exploration. |
+| =calendar-sync= | D/S | eager only if configured, batch safe | Private config path and noninteractive guard exist. |
+| =reconcile-open-repos= | D/S | command-loaded | Repo scanning/reconciliation should not run at startup. |
+| =local-repository= | O/D/P | command-loaded | Local package mirror workflow. |
+| =music-config= | O/D/P/S | command-loaded | EMMS/keymap optional, hooks only after EMMS. |
+| =games-config= | O | command-loaded | Optional. |
+| =lorem-optimum= | O/L | command-loaded | Module in test. |
+| =jumper= | O/L | command-loaded | Navigation helper. |
+| =system-commands= | D/S | command-loaded | High-impact commands; defensive work tracked separately. |
+| =gloss-config= | O/D/P | command-loaded | Glossary workflow. |
+| =wrap-up= | S | eager if desired | End-of-startup buffer bury timer. |
+| =ledger-config= | O/D/P | mode-loaded | Not currently required by init. |
+| =mu4e-org-contacts-integration= | D/L | after mu4e/org-contacts | Loaded by mail workflow. |
+| =mu4e-org-contacts-setup= | D/L | after mu4e/org-contacts | Setup helper. |
+| =org-agenda-config-debug= | O/L | command/debug-loaded | Debug helper. |
+| =show-kill-ring= | O/L | command-loaded | Not currently required by init. |
+
+* Module File Header Standard
+
+Each module should eventually declare its load-graph contract in its own
+commentary header. The category table above is the seed view; module headers
+are the contributor-facing contract that travels with the code.
+
+Required header lines, after =;;; Commentary:=:
+
+1. =;; Layer: <0|1|2|3|4> (<layer name>).=
+2. =;; Category: <F|C|P|D|S|O|L>=.
+3. =;; Load shape: <eager|hook|mode|command|after-load>=.
+4. =;; Eager reason:= one-line justification when load shape is =eager=,
+ omitted otherwise.
+5. =;; Top-level side effects:= timer, process, hook, package, network,
+ buffer mutation, file write, or =none=.
+6. =;; Runtime requires:= explicit runtime module/package list.
+7. =;; Direct test load: <yes|conditional|no>=, with a brief reason when not
+ =yes=.
+
+Optional:
+
+- =;; See also:= references to tests and design docs.
+
+Worked example:
+
+#+begin_src emacs-lisp
+;;; calendar-sync.el --- One-way calendar synchronization to Org -*- lexical-binding: t; -*-
+;;
+;;; Commentary:
+;;
+;; Layer: 3 (Domain Workflow).
+;; Category: D/S.
+;; Load shape: eager only when calendar-sync.local.el configures calendars.
+;; Eager reason: daily-driver workflow; user expects calendars synced at first
+;; session. Top-level startup is guarded so batch/test loads do not start
+;; timers or network fetches.
+;; Top-level side effects: timer, network fetch, file writes to calendar Org
+;; files. Guarded by noninteractive/config checks.
+;; Runtime requires: user-constants, seq, subr-x.
+;; Direct test load: yes (batch-safe; private config is optional).
+;;
+;; See also: docs/design/init-load-graph.org, tests/test-calendar-sync.el.
+;;
+;;; Code:
+#+end_src
+
+Phase 1 should annotate every module required by =init.el= with this header.
+Later validation can assert that every required module declares the seven
+required lines.
+
+* Proposed Load Shape
+
+Migration commits should use conventional commit prefixes consistently:
+
+- =refactor:= for behavior-preserving load-order, dependency, keymap, and lazy
+ loading migrations.
+- =feat:= only when adding a new user-visible capability.
+- =test:= for test-only follow-up work.
+- =docs:= for spec, inventory, design updates, and module-header annotations,
+ even when those annotations touch =modules/*.el= files.
+
+Default deferral mechanism:
+
+- Prefer =use-package :commands= for command-driven deferrals.
+- Prefer =use-package :mode= when loading is file-extension or major-mode
+ driven.
+- Prefer =use-package :hook= when the consumer is a mode-hook function.
+- Use explicit =(autoload 'command "module" nil t)= only when the command is
+ not naturally owned by a =use-package= form.
+
+** Phase 1: Inventory And Contracts
+
+Do not change load order yet.
+
+1. Keep the current eager =init.el= order.
+2. Create/maintain =docs/design/module-inventory.org= as a living inventory
+ with:
+ - module name,
+ - category,
+ - eager/deferred target,
+ - known runtime dependencies,
+ - top-level side effects,
+ - tests that cover standalone load or command behavior.
+3. Annotate every module required by =init.el= with the module header standard.
+4. Convert vague comments in =init.el= into tasks or remove them:
+ - =latex-config= "WIP need to fix",
+ - =prog-shell= "combine elsewhere",
+ - "Modules In Test" section.
+5. Add lightweight standalone-load smoke tests for the lowest-level modules.
+
+Inventory rules:
+
+- The module table in this spec seeds the inventory.
+- =docs/design/module-inventory.org= is the living per-module truth after Phase
+ 1 starts.
+- Every module required by =init.el= must be represented before Phase 2 starts.
+- Discoveries during later phases update the inventory.
+- This inventory is independent from the helper inventory owned by
+ [[file:utility-consolidation.org][utility-consolidation.org]].
+
+Exit criteria:
+
+- Every module required by =init.el= has a category and target load shape.
+- Every eager survivor has a documented reason.
+- The inventory identifies top-level timers/process/network-ish side effects.
+- Every module required by =init.el= has the required load-graph header lines.
+
+** Phase 2: Explicit Dependencies
+
+Still do not significantly change startup behavior.
+
+1. For each module batch, load it directly in batch mode.
+2. Fix hidden dependencies by adding real =require=, =autoload=, or package
+ boundaries.
+3. Remove production shims that only exist because tests load modules in an
+ incomplete environment.
+4. If a keymap dependency is hidden, document it and make the dependency
+ explicit with =require= or =autoload=. Do not refactor into the registration
+ convention until Phase 3. When the hidden dependency is on
+ =cj/custom-keymap= itself, add =(require 'keybindings)= to the consuming
+ module; Phase 3 replaces these direct dependencies with the registration
+ API.
+5. When a hidden dependency is really a duplicated generic helper, either:
+ - hand the extraction to the utility-consolidation sibling project when it
+ is in scope there, or
+ - leave it in place and record it under that project.
+
+Suggested order:
+
+- Foundation and libraries.
+- Text/editing command modules.
+- UI modules.
+- Programming modules.
+- Org modules.
+- Optional integrations.
+
+Exit criteria:
+
+- Direct module load either succeeds or fails with a clear missing external
+ package/config message.
+- =make test-file FILE=test-all-comp-errors.el= passes.
+- New tests cover any helper extracted while fixing dependencies.
+- Helper extraction remains dependency-light and does not pull heavy packages
+ into foundation startup.
+
+** Phase 3: Keymap Registration Boundary
+
+Introduce a small keymap registration API before deferring many feature modules.
+
+Possible API:
+
+#+begin_src emacs-lisp
+(defun cj/register-prefix-map (key map label)
+ "Register MAP under KEY in `cj/custom-keymap' with LABEL for which-key."
+ ...)
+
+(defun cj/register-command (key command label)
+ "Register COMMAND under KEY in `cj/custom-keymap' with LABEL for which-key."
+ ...)
+#+end_src
+
+Design rules:
+
+- =keybindings.el= owns =cj/custom-keymap= and the global =C-;= binding.
+- Feature modules may define maps and commands without mutating global keys
+ directly.
+- Which-key labels must be registered after which-key loads.
+- Tests can assert key resolution without loading every feature package.
+
+Exit criteria:
+
+- Modules no longer need to assume =cj/custom-keymap= exists at top level
+ except through the registration API.
+- Existing =C-;= bindings continue to resolve.
+- Which-key labels for documented prefixes remain available.
+
+** Phase 4: Defer Low-Risk Optional Modules
+
+Start with modules that are unlikely to affect first-frame startup.
+
+Candidate batch:
+
+- =games-config=
+- =music-config=
+- =weather-config=
+- =gloss-config=
+- =lorem-optimum=
+- =jumper=
+- =httpd-config=
+- =prog-training=
+
+For each module:
+
+1. Keep its user-facing command/key available via the default deferral mechanism
+ above.
+2. Move package loading into =use-package :commands=, =:hook=, =:mode=, or an
+ explicit autoload/wrapper only when the default does not fit.
+3. Run targeted tests and an interactive smoke check.
+
+Exit criteria:
+
+- Startup no longer requires the module eagerly.
+- User command still works from a fresh Emacs session.
+- Module-specific tests pass.
+
+** Phase 5: Defer Heavy Domain Modules
+
+Candidate batch:
+
+- =pdf-config=
+- =calibredb-epub-config=
+- =video-audio-recording=
+- =transcription-config=
+- =mail-config=
+- =ai-config=
+- =restclient-config=
+- =elfeed-config=
+- =erc-config=
+- =slack-config=
+
+These need more care because they often combine package setup, auth, keymaps,
+processes, hooks, and user workflows.
+
+Exit criteria for each:
+
+- Commands are discoverable before package load.
+- Package load happens through the default deferral mechanism: command, hook,
+ mode, or explicit startup opt-in.
+- Auth and private config are not read until necessary unless the user opts in.
+- Batch/test startup does not start network/process work.
+
+Private config opt-in follows the =calendar-sync.local.el= precedent: a module
+reads =<module-name>.local.el= when readable, the file is gitignored, and the
+module degrades cleanly when the file is missing. Token rotation is a separate
+security task; this convention is about config presence, not secret protection.
+
+** Phase 6: Revisit Org And Programming Eagerness
+
+Org and programming modules are daily-use, so the goal is not blindly deferring
+everything.
+
+Programming target:
+
+- Keep generic programming defaults and F-key command entry points available.
+- Load language-specific modules by major mode.
+- Consolidate generic LSP policy under =prog-lsp=.
+ - Move to =prog-lsp=: global LSP toggles such as =lsp-idle-delay=,
+ =lsp-log-io=, =lsp-enable-folding=, =lsp-enable-snippet=,
+ =lsp-headerline-breadcrumb-enable=, and file-watch ignore lists.
+ - Keep per-language: server client settings such as
+ =lsp-clients-clangd-args= and =lsp-pyright-*=, plus language-mode hook
+ wiring.
+- Tree-sitter grammar auto-install is always on; the project policy is global
+ allow. =treesit-auto-install= is =t= without per-language conditionals.
+
+Org target:
+
+- Keep these daily first-session workflows eager: =org-config=,
+ =org-agenda-config=, =org-capture-config=, =org-refile-config=,
+ =calendar-sync= when local config is present, and =org-roam-config=.
+- Defer exporters, reveal, drill, noter, webclipper, and optional publishing
+ pieces behind commands/hooks.
+- Normalize agenda/refile cache lifecycle before changing timer behavior. This
+ is behavioral normalization within the load-graph project; the shared
+ =cj-cache.el= extraction is owned by utility-consolidation Phase 5 and may
+ follow.
+
+The =prog-lsp= consolidation and tree-sitter policy decisions are owned by this
+load-graph project. Utility consolidation owns reusable helper extraction, not
+programming policy.
+
+Exit criteria:
+
+- Common daily Org/programming workflows work from a fresh session.
+- Optional exporters/languages load when used.
+- Timers are guarded in batch/test contexts.
+
+* Adjacent Project: Utility Consolidation
+
+The review of this spec identified a related but distinct architectural
+problem: helper functions are scattered across feature modules, sometimes with
+duplicated behavior. This matters to the load graph because modules can become
+coupled to whichever feature file happened to define a useful helper first.
+
+This should be tracked as a sibling project, not folded into the load-graph
+project. The load-graph project asks "when and why does this module load?" The
+utility consolidation project asks "which module should own this reusable
+behavior?" Those questions overlap, but their changes have different risk and
+rollback shapes.
+
+This sibling project can run beside Phase 2. When explicit-dependency work finds
+a generic duplicated helper, the sibling project owns the extraction commit when
+the helper is in scope for that project. See
+[[file:utility-consolidation.org][utility-consolidation.org]] for candidate
+helpers, naming rules, dependency budgets, migration phases, and test policy.
+
+* Testing Strategy
+
+** Static/Batch Tests
+
+Add or extend tests for:
+
+- Direct module load smoke tests for modules in each batch.
+- Header validation: every module required by =init.el= declares the seven
+ required load-graph header lines.
+ - Test file: =tests/test-init-module-headers.el=.
+ - Assertion shape: inspect every module required by =init.el=, read its
+ commentary header, and fail with the missing line names for any absent
+ required header line.
+- Keymap registration: prefix maps and commands resolve without requiring the
+ feature implementation package.
+- No startup timers/processes in batch for side-effect modules.
+- =init.el= startup smoke in batch, where possible.
+- Byte/native compile smoke via existing =test-all-comp-errors.el=.
+
+Test files for this project use =test-init-<feature>.el=, for example
+=test-init-module-headers.el= and =test-init-keymap-registration.el=. This keeps
+load-graph validation tests distinct from per-module unit tests.
+
+Header validation runs directly against module files. It does not depend on the
+final =docs/design/module-inventory.org= format, which remains a Phase 1
+authoring decision.
+
+** Automated Smoke Checks
+
+Automate every smoke item that can run in batch:
+
+- Important keybindings resolve to the intended command symbols, including
+ =C-;= prefixes and F4/F6/F7 entry points.
+- Org capture and agenda command entry points load or produce expected
+ batch-safe guidance.
+- Calendar sync status reports configured/no-config state without starting
+ timers or network fetches in batch.
+- Optional commands touched in the batch autoload and resolve.
+- Non-graphical interactive flows use =execute-kbd-macro= or
+ =with-simulated-input= where practical.
+
+These checks should run under =make test= for every migration commit.
+
+** Manual Smoke Checks
+
+Each migration batch should be followed by an interactive restart and checklist:
+
+- First frame appears with expected theme/font/modeline.
+- =C-;= prefix appears and key descriptions are present.
+- Magit opens.
+- Mail command opens or gives expected package/config guidance.
+- Refile target lookup works in an interactive session.
+- Any optional command changed in the batch runs end to end.
+- If daemon mode is part of normal use, run the visual checklist once via
+ regular =emacs= and once via =emacsclient= against a running daemon.
+
+** Performance Checks
+
+Before and after major batches:
+
+- Record =emacs-init-time=.
+- Record a startup profile baseline and diff, preferably with =benchmark-init=
+ if enabled for the phase.
+- =benchmark-init= is installed via package.el. The activation block in
+ =early-init.el= is commented; uncomment it locally during phases that need
+ profiling and do not commit the activation. Profile output goes to
+ =.profile/=, which should stay gitignored.
+- Suggested workflow:
+ - =make profile-baseline= records =emacs-init-time= and a startup profile to
+ =.profile/baseline.txt=.
+ - =make profile-diff= records the current run and compares it to the phase
+ baseline.
+- Keep a simple note of eagerly loaded feature count from
+ =cj/info-loaded-features= or equivalent.
+
+Performance is a supporting signal. Correctness and explicit dependencies are
+the primary acceptance criteria. Startup regressions larger than roughly 50 ms
+against the phase baseline should be investigated and explained; after several
+stable baseline runs, this can become a stricter gate.
+
+* Acceptance Criteria
+
+The project is complete when:
+
+- =init.el= contains only documented eager requires and explicit startup calls.
+- Optional modules no longer load merely because Emacs started.
+- Each module required by =init.el= has a category and eager/deferred rationale.
+- Modules that remain eager have no hidden dependencies on arbitrary earlier
+ init order.
+- Global key registration has a central owner/convention.
+- Top-level timers/process/network work is either removed, guarded, or
+ documented as intentional.
+- Full =make test= passes.
+- Byte/native compile smoke passes.
+- Interactive startup checklist passes.
+
+* Risks And Mitigations
+
+** Risk: Breaking muscle-memory keybindings
+
+Mitigation:
+
+- Change key registration mechanics before changing bindings.
+- Add keymap resolution tests for important prefixes.
+- Keep a per-batch manual keybinding checklist.
+
+** Risk: Lazy-loaded packages miss early hook setup
+
+Mitigation:
+
+- Prefer =use-package :hook= and =:mode= over ad hoc lazy command bodies for mode
+ packages.
+- Add tests that inspect hook contents where possible.
+- Smoke-test opening representative files.
+
+** Risk: Daily workflows silently stop starting
+
+Mitigation:
+
+- Distinguish "safe default" from "local opt-in" for workflows like calendar
+ sync.
+- Use ignored/local config files for private eager opt-ins.
+- Report missing config clearly.
+
+** Risk: Batch tests differ from interactive startup
+
+Mitigation:
+
+- Guard timers/process/network work with =noninteractive= only when that is the
+ intended distinction.
+- Add at least one interactive checklist per migration batch.
+
+** Risk: Refactor becomes too broad
+
+Mitigation:
+
+- One batch, one module family.
+- Do not mix dependency fixes, keybinding redesign, and package lazy-loading in
+ the same commit unless tightly coupled.
+- Keep rollback easy by preserving user-facing commands and using wrappers.
+
+* Implementation Backlog
+
+The project in =todo.org= should remain the source of task state. This design
+supports these implementation tickets:
+
+1. Classify modules by role and startup requirement.
+2. Add explicit module dependencies before changing load order.
+3. Centralize custom keymap registration.
+4. Defer low-risk optional modules.
+5. Defer heavy document/media/integration modules.
+6. Revisit programming module LSP/tree-sitter ownership.
+7. Revisit Org module cache/timer and optional extension loading.
+8. Retire or rewrite stale =init.el= comments.
+9. Create a sibling utility consolidation project with an inventory pass and
+ first helper extractions.
+
+* Open Questions
+
+- Should =config-utilities= remain eager because debug commands are useful
+ during startup work, or should it become command-loaded after this project?
+- Should local/private opt-ins share one file, or should modules keep
+ workflow-specific local files such as =calendar-sync.local.el=?
+- Should the module inventory become machine-readable for validation, or is an
+ org table enough? Decide during Phase 1 based on inventory authoring
+ experience.
+- Should =init.el= ultimately become declarative sections plus an explicit
+ startup contract list?
+
+* Next Steps
+
+1. Use this document as the reference for the =Classify modules by role and
+ startup requirement= task.
+2. Build the first inventory directly from the module table above, correcting
+ category guesses while inspecting each file.
+3. Do not defer a module until its direct runtime dependencies are explicit.
+4. Implement keymap registration before deferring feature modules that currently
+ mutate =cj/custom-keymap= at top level.
+5. Create the sibling utility consolidation project before Phase 2 work begins,
+ so duplicated helpers found during dependency cleanup have a clear place to
+ land.
diff --git a/docs/design/utility-consolidation.org b/docs/design/utility-consolidation.org
new file mode 100644
index 00000000..b8428380
--- /dev/null
+++ b/docs/design/utility-consolidation.org
@@ -0,0 +1,1216 @@
+#+TITLE: Design: Consolidate Shared Utility Helpers
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-05-04
+
+* Status
+
+Draft. Specification only. No helper extraction is part of this document.
+
+This is the sibling project to [[file:init-load-graph.org][Untangle the init.el Load Graph]]. The load-graph
+project decides when modules load and what dependencies they declare. This
+project decides which module should own reusable helper behavior.
+
+* Framing Questions
+
+Before extracting helpers, ask these questions for each candidate:
+
+1. Is this behavior duplicated, or merely similar?
+2. Does the proposed helper have at least two real consumers?
+3. Can the helper live in a foundation library without pulling heavy packages
+ into startup?
+4. Is the helper pure or mostly pure, or does it create files, processes,
+ timers, buffers, warnings, messages, or network traffic?
+5. Does it need to be available to production code, tests only, or both?
+6. Will extraction make the caller easier to read, or will it hide important
+ domain decisions behind a vague utility name?
+7. Can the tests move with the helper while preserving consumer behavior tests?
+8. Is an existing Emacs primitive already good enough?
+9. Is this a library function, a command helper, or a module-specific private
+ detail?
+10. What compatibility story is needed for existing public =cj/= commands?
+
+The default answer should be "do not extract yet" unless the helper has clear
+reuse pressure and low dependency cost. The exception is a helper that currently
+lives in a clearly wrong dependency layer and blocks the load-graph work; those
+speculative extractions are allowed only when this spec names the expected
+future consumers and the migration keeps the old caller behavior covered.
+
+* Candidate Decision Criteria
+
+Use intent first and implementation shape second. A helper is a real extraction
+candidate when multiple callers are enforcing the same policy, even if the
+current functions were written differently.
+
+Do not reject a candidate just because the implementations differ. Differences
+may be accidental drift, local naming, or missing parameters. Also do not accept
+a candidate just because the implementations look similar. Similar code can
+still represent different workflow policies.
+
+Treat a candidate as real when most of these are true:
+
+- Same job: one domain-neutral sentence describes what both callers are trying
+ to do.
+- Same failure semantics: callers agree on whether failure should return nil,
+ warn, signal =user-error=, skip silently, or fall back.
+- Same side-effect policy: callers agree on whether the helper may message,
+ warn, write files, create buffers, start processes, or mutate state.
+- Same dependency layer: the helper can live somewhere both callers can
+ reasonably require without pulling heavy package/domain dependencies into
+ foundation startup.
+- Differences are parameters, not hidden modes: warning type, feature name,
+ current directory, TTL, or trim behavior are reasonable parameters; broad
+ flags that make one helper behave like several unrelated helpers are a smell.
+- Tests can describe the policy without loading unrelated domain modules.
+- Call sites get clearer because the shared policy has a good name and the
+ caller still owns workflow-specific consequences.
+
+When callers share only part of the intention, extract the shared policy core
+and leave workflow decisions local. For example, executable lookup can be shared
+while mail, programming, and media modules still decide whether a missing tool
+means "disable sync," "skip a hook," or "show an interactive warning."
+
+* Problem
+
+Several modules define helper functions where they were first needed. Some of
+those helpers are truly private. Others are general utilities that now have
+multiple consumers or obvious near-duplicates. This creates architectural drag:
+
+- A feature module becomes the accidental owner of generic behavior.
+- Other modules either duplicate the behavior or depend on a feature module for
+ a helper they should not conceptually require.
+- Tests are harder to place because helper logic is mixed with package config,
+ keybinding setup, timers, external processes, and user commands.
+- The =init.el= load graph stays harder to untangle because helper ownership is
+ not explicit.
+
+The goal is not to build a large personal standard library. The goal is to
+extract a small set of proven helpers into predictable, dependency-light
+libraries with focused tests.
+
+A prior review estimated roughly 221 private helpers across 31 modules. That
+count is useful motivation for an inventory, but it is not the extraction
+target. This project should pull only the helpers whose ownership and reuse
+case are clear.
+
+* Goals
+
+- Identify concrete helper functions that should be moved, renamed, wrapped, or
+ deliberately left alone.
+- Keep foundation helpers dependency-light and safe to load early.
+- Give each helper family a clear home and naming convention.
+- Preserve existing behavior at call sites.
+- Move unit tests with extracted helper behavior.
+- Keep migrations small: one helper family per commit.
+- Improve direct module loading by replacing hidden cross-module assumptions
+ with explicit =require= statements.
+
+* Non-Goals
+
+- Renaming every =cj/= function.
+- Turning command modules into libraries when their behavior is user-facing.
+- Extracting helpers with only one consumer unless they are already in the wrong
+ dependency layer.
+- Replacing useful built-in APIs such as =file-in-directory-p= with wrappers
+ that add no policy.
+- Moving heavy package-specific behavior into =system-lib=.
+- Combining this work with lazy-loading changes in the same commit.
+
+* Existing Library Shape
+
+A file with user-facing interactive commands is a feature/command module, not a
+shared library. Mixed files can keep private helper functions, but those helpers
+should move to =system-lib.el= or a topic library only when reuse pressure and
+the candidate criteria justify extraction.
+
+** =system-lib.el=
+
+Current role: low-level system utility library. It is already expected to be a
+foundation module.
+
+Current functions:
+
+- =cj/executable-exists-p=
+- =cj/log-silently=
+
+Recommended role: foundation helpers that are dependency-light, batch-safe, and
+reasonable to load early.
+
+Good fits:
+
+- executable lookup,
+- shell argument formatting,
+- process execution wrappers built on =process-file=,
+- warning/message convenience wrappers,
+- simple file/path predicates,
+- simple file string read/write helpers.
+
+Bad fits:
+
+- helpers that require Org, mu4e, projectile, dirvish, vc-git, url, gptel, or
+ other package/domain dependencies,
+- helpers that start asynchronous processes/timers at load time,
+- helpers whose semantics are really workflow-specific.
+
+Dependency budget:
+
+- Allowed without additional design: built-ins already available at startup,
+ =subr-x=, =cl-lib=, and =seq=.
+- Allowed only with an explicit note in the helper's section:
+ =host-environment=, because it participates in early foundation/load-order
+ decisions.
+- Not allowed in =system-lib.el=: Org, VC internals such as =vc-git=, Dired
+ implementation packages beyond declarations, url/network libraries, mu4e,
+ projectile, dirvish, gptel, media packages, or any package that would make
+ optional workflows part of foundation startup.
+
+Any helper needing dependencies outside this budget belongs in a topic library
+or the domain module that already owns that dependency.
+
+** =system-utils.el=
+
+Current role: user-facing system commands and Emacs enhancements. It mixes
+commands, package config, keybindings, external open behavior, savehist,
+scratch-buffer setup, dictionary, and proced.
+
+Recommended role: command/config module, not a low-level library.
+
+Potential extraction from here:
+
+- =cj/--file-from-context= should move to =system-lib= as a generic path helper.
+- =cj/--open-with-is-launcher-p= should move if external-open behavior is shared.
+- =cj/identify-external-open-command= should move to =external-open.el= as
+ workflow-owned command resolution.
+
+After extraction, =system-utils.el= should require the library and keep the
+interactive commands.
+
+** =config-utilities.el=
+
+Current role: interactive maintenance/debug commands for this Emacs config.
+
+Recommended role: keep as command/config module. Do not turn it into a generic
+library.
+
+Potential extraction:
+
+- =with-timer= should become =cj/with-timer= only if other modules need the
+ macro.
+- =cj/--delete-compiled-files-in-dir= may become a generic recursive file
+ deletion helper only if another production caller needs the same behavior and
+ the destructive policy is explicit.
+- build-info formatting helpers should stay here; they are command-specific.
+
+** =testutil-general.el=
+
+Current role: test-only filesystem helpers.
+
+Recommended role: keep test harness helpers here unless a production module
+needs the same safety policy. Do not make production code depend on
+=testutil-general=.
+
+Potential production extraction:
+
+- =cj/test--assert-inside-base= -> =cj/path-assert-in-directory=
+- =cj/test--safe-base-dir-p= -> =cj/safe-recursive-delete-root-p=
+
+These should move only when a production destructive workflow needs them.
+
+* Library File Header Standard
+
+Shared library files should document their own scope in the commentary header.
+The design spec records the rationale; the file header is the contributor-facing
+contract that should be visible during ordinary edits.
+
+Required header content:
+
+1. =;; Role:= foundation utility library or topic library role.
+2. =;; Layer:= matching the load-graph architecture.
+3. =;; Dependency budget:= allowed, note-required, and forbidden dependencies.
+4. =;; What belongs here:= concrete helper families.
+5. =;; What does not belong here:= workflow/domain behavior and heavy
+ dependencies.
+6. =;; Adding a helper:= two-consumer rule plus wrong-layer speculative
+ exception.
+7. =;; Naming:= public helper naming policy.
+8. =;; Tests:= expected test-file convention.
+9. =;; Renaming:= obsolete alias policy.
+10. =;; See also:= design docs for rationale.
+
+Worked =system-lib.el= header:
+
+#+begin_src emacs-lisp
+;;; system-lib.el --- Foundation utility helpers -*- lexical-binding: t; -*-
+;;
+;;; Commentary:
+;;
+;; Role: Foundation utility library (Layer 1).
+;;
+;; This file owns reusable, dependency-light helpers used across feature
+;; modules. It loads early in startup and must stay batch-safe.
+;;
+;; Dependency budget:
+;; Allowed without note: built-ins, subr-x, cl-lib, seq.
+;; Allowed with note: host-environment (foundation peer).
+;; Not allowed: Org, vc-git internals, Dired implementation,
+;; url/network libraries, mu4e, projectile,
+;; dirvish, gptel, media packages, or any package
+;; that would attach optional workflows to
+;; foundation startup.
+;;
+;; What belongs here:
+;; - executable lookup (silent predicate or warn-and-return)
+;; - shell argument formatting (readable-when-safe quoting)
+;; - process execution wrappers built on `process-file'
+;; - simple file/path predicates and context resolution
+;; - logging convenience wrappers
+;;
+;; What does not belong here:
+;; - workflow-specific behavior (calendar parsing, Org-roam slug generation,
+;; gptel adapters)
+;; - timers, network requests, or buffer mutations at load time
+;; - helpers that pull a heavy package into foundation startup
+;;
+;; Adding a helper:
+;; Default to "do not extract yet." Extract when at least two callers share
+;; the same job, failure semantics, side-effect policy, and dependency layer.
+;; Speculative extractions are allowed only when a feature module is the
+;; wrong long-term owner; record expected future consumers in
+;; docs/design/utility-inventory.org.
+;;
+;; Naming:
+;; Public helpers use cj/<noun>-<verb> or cj/<domain>-<verb>. Names should
+;; describe policy, not just shape. Do not retain source-module prefixes
+;; after extraction.
+;;
+;; Tests:
+;; tests/test-system-lib-<helper-name>.el, one file per helper. Stub
+;; side-effecting primitives at their boundaries via cl-letf.
+;;
+;; Renaming:
+;; Public helpers in user muscle memory get a one-cycle obsolete alias.
+;; Private helpers rename without alias when all call sites change in the
+;; same commit.
+;;
+;; See also: docs/design/utility-consolidation.org for design rationale.
+;;
+;;; Code:
+#+end_src
+
+Topic libraries such as =cj-process.el=, =cj-org-text.el=, or =cj-cache.el=
+should follow the same shape with a narrower role and dependency budget.
+
+* Proposed Library Layout
+
+Start with single-file growth in =system-lib.el=. Split later only when the file
+becomes too broad or a helper family needs a dependency that should not be
+foundation-eager.
+
+Recommended phases:
+
+1. Grow =system-lib.el= for the first dependency-light helpers.
+2. Keep Org-specific helpers out of =system-lib= unless they can be written with
+ only strings and =subr-x=.
+3. Introduce topic libraries only when there is a clear reason:
+ - =cj-process.el= for process runners if the process API grows beyond a
+ couple functions.
+ - =cj-org-text.el= for Org-safe text helpers if they start requiring Org
+ APIs.
+ - =cj-cache.el= for cache helpers because that abstraction is stateful and
+ distinct from simple system helpers.
+4. Preserve =system-lib.el= as the easy entry point for the low-level set.
+
+Load shape:
+
+- Each topic library declares its load-graph layer in its file header.
+- =cj-process.el= and =cj-org-text.el= are Layer 1 only if their first consumer
+ is foundation-eager; otherwise they are Layer 2 and loaded by explicit
+ =require= from eager consumers.
+- =cj-cache.el= follows the first real cache consumer's layer, likely Layer 2 if
+ modeline/agenda/refile remain eager or near-eager.
+- Coordinate every new topic library with
+ [[file:init-load-graph.org][init-load-graph.org]] before migrating its first consumer.
+
+* Naming Rules
+
+- Library files use =cj-<topic>.el=. The legacy =system-lib.el= name stays for
+ compatibility and serves as the foundation entry point.
+- Public reusable helpers use =cj/<noun>-<verb>= or =cj/<domain>-<verb>=.
+- Private module helpers keep =cj/<module>--<helper>= or
+ =<module>--<helper>=.
+- Do not keep source-module names after extraction. For example,
+ =cj/mail--executable-or-warn= should not become
+ =cj/system-mail-executable-or-warn=.
+- Prefer names that describe policy:
+ - =cj/executable-find-or-warn= is better than =cj/check-program=.
+ - =cj/shell-quote-argument-readable= is better than =cj/shell-quote= because
+ it documents the readable-when-safe policy.
+- Preserve public interactive command names unless there is a separate
+ user-facing rename task.
+- Add obsolete aliases only for functions used outside their defining module or
+ in user muscle memory. Private helpers can be renamed without aliases when
+ all call sites change in the same commit.
+
+* Candidate Extraction Table
+
+This table is intentionally concrete. "Action" describes the recommended end
+state, not necessarily the first commit.
+
+| Current symbol | Current file | Proposed symbol | Proposed home | Action | Priority | Notes |
+|----------------+--------------+-----------------+---------------+--------+----------+-------|
+| =cj/mail--executable-or-warn= | =mail-config.el= | =cj/executable-find-or-warn= | =system-lib.el= | Extract | High | Generalizes missing executable warning; callers include mail, language tools, media/dirvish commands. |
+| =cj/executable-exists-p= | =system-lib.el= | =cj/executable-available-p= | =system-lib.el= | Rename, alias preserved | Medium | New predicate returns boolean. Keep one-cycle obsolete alias/wrapper because the current name is misleading and returns a path. |
+| direct =(executable-find ...)= with silent nil | =prog-c.el=, =prog-go.el=, =prog-python.el=, =prog-shell.el=, =dirvish-config.el=, =browser-config.el=, =mail-config.el= | =cj/executable-find-or-warn= or =cj/executable-available-p= | =system-lib.el= | Migrate selectively | High | Use warnings for user-invoked missing features; keep silent predicates for package =:if= checks when silence is intentional. |
+| =cj/--f6-shell-safe-argument-regexp= | =dev-fkeys.el= | =cj/shell-safe-argument-regexp= | =system-lib.el= | Extract | High | Keep as implementation detail for readable shell quoting. |
+| =cj/--f6-shell-quote-argument= | =dev-fkeys.el= | =cj/shell-quote-argument-readable= | =system-lib.el= | Extract | High | Useful for generated compile/test strings where safe paths should remain readable. |
+| direct =shell-quote-argument= in command strings | =prog-c.el=, =prog-python.el=, =prog-shell.el=, =mail-config.el=, =dirvish-config.el=, =vc-config.el=, =elfeed-config.el=, =system-utils.el= | case-by-case | =system-lib.el= | Audit | Medium | Do not blindly replace; direct quoting is correct when readability is irrelevant or command strings are security-sensitive. |
+| =cj/--coverage-git-string= | =coverage-core.el= | =cj/process-output-or-error= | =system-lib.el= or =cj-process.el= | Extract generic core | High | Generic process-file wrapper: program + argv -> stdout or user-error with status/output. |
+| =cj/--coverage-git-string= | =coverage-core.el= | =cj/git-output-or-error= | =system-lib.el= or =cj-process.el= | Add wrapper | High | Thin wrapper around generic runner with program ="git"=. |
+| =cj/--coverage-git-merge-base= | =coverage-core.el= | keep =cj/--coverage-git-merge-base= | =coverage-core.el= | Keep | Low | Coverage-specific semantics; may call =cj/git-output-or-error=. |
+| =cj/--coverage-git-diff= | =coverage-core.el= | keep =cj/--coverage-git-diff= | =coverage-core.el= | Keep | Low | Coverage-specific =--unified=0= policy. |
+| =cj/--file-from-context= | =system-utils.el= | =cj/file-from-context= | =system-lib.el= | Extract | High | Useful for Dired/current-buffer command helpers. Requires only dired declarations and built-ins. |
+| =cj/--open-with-is-launcher-p= | =system-utils.el= | =cj/external-open-launcher-p= | =external-open.el= | Extract after consumers align | Medium | External-open policy, not core path logic. |
+| =cj/identify-external-open-command= | =system-utils.el= | =cj/external-open-command= | =external-open.el= | Move/rename | Medium | External-open owns command-string resolution; host-environment remains predicate-only. |
+| duplicated OS-open command selection | =system-utils.el=, =dirvish-config.el=, =external-open.el= | =cj/external-open-command= | =external-open.el= | Consolidate | Medium | One source of truth for =xdg-open=, =open=, =start=. |
+| =cj/test--file-in-directory-p= | =test-runner.el= | =file-in-directory-p= (built-in) | built-in | Replace caller with built-in | Medium | Do not create a wrapper unless a real normalization policy emerges. |
+| =cj/test--assert-inside-base= | =testutil-general.el= | =cj/path-assert-in-directory= | =system-lib.el= | Extract only with production caller | Medium | Useful for destructive commands, but currently test-only. |
+| =cj/test--safe-base-dir-p= | =testutil-general.el= | =cj/safe-recursive-delete-root-p= | =system-lib.el= | Extract only with production caller | Medium | Policy-heavy. Should be explicit and well-tested before production use. |
+| =calendar-sync--sanitize-org-body= | =calendar-sync.el= | =cj/org-sanitize-body-text= | =cj-org-text.el= or =system-lib.el= | Extract | High | Already tested; likely useful for webclipper, AI conversations, mail capture. |
+| =calendar-sync--sanitize-org-property-value= | =calendar-sync.el= | =cj/org-sanitize-property-value= | =cj-org-text.el= or =system-lib.el= | Extract | High | String-only behavior; no Org dependency required. |
+| =calendar-sync--sanitize-org-heading= | =calendar-sync.el= | =cj/org-sanitize-heading= | =cj-org-text.el= or =system-lib.el= | Extract | High | Protects outline structure from external text. |
+| =calendar-sync--strip-html= | =calendar-sync.el= | =cj/text-strip-html= | =system-lib.el= or =cj-text.el= | Consider | Medium | Useful beyond calendar, but HTML stripping via regex is intentionally simple and should be documented as such. |
+| =calendar-sync--clean-text= | =calendar-sync.el= | =cj/text-clean-external= | =system-lib.el= or =cj-text.el= | Consider | Medium | Combines ICS unescape + HTML strip today, so it may be too calendar-specific unless split. |
+| =calendar-sync--unescape-ics-text= | =calendar-sync.el= | keep =calendar-sync--unescape-ics-text= | =calendar-sync.el= | Keep | Low | ICS-specific; not a general utility. |
+| =cj/modeline-vc-cache-*= helpers | =modeline-config.el= | =cj/cache-valid-p=, =cj/cache-get=, =cj/cache-put=, =cj/cache-clear= | =cj-cache.el= | Extract later | Medium | Good pattern, but variable-local cache shape differs from Org caches. Needs design before extraction. |
+| agenda/refile cache vars and build flags | =org-agenda-config.el=, =org-refile-config.el= | =cj/cache-value-or-rebuild= or =cj/build-cache= | =cj-cache.el= | Extract later | Medium | Similar TTL/building/invalidation lifecycle. Higher risk than simple helpers. |
+| =cj/log-silently= | =system-lib.el= | =cj/message-log-only= | =system-lib.el= | Rename, alias preserved | Low | Clearer name for discoverability, but low value. Do after higher-priority helpers unless touched nearby. |
+| direct =display-warning= boilerplate | =mail-config.el= and future callers | =cj/display-warning-once= / =cj/warn-once= | =system-lib.el= | Add after second caller | Low | Do not add until repeated formatting or once-only behavior appears. |
+| =with-timer= | =config-utilities.el= | =cj/with-timer= | =system-lib.el= or stay | Defer | Low | Macro is useful, but currently debug-oriented. Extract only after another production caller appears. |
+| =cj/theme-read-file-contents= | =ui-theme.el= | =cj/read-file-string= | =system-lib.el= | Consider | Low | Built-in =insert-file-contents= wrappers are small; extract only if multiple callers emerge. |
+| =cj/theme-write-file-contents= | =ui-theme.el= | =cj/write-file-string= | =system-lib.el= | Consider | Low | Same as above. Keep theme-specific unless reused. |
+| =cj/modeline-string-cut-middle= | =modeline-config.el= | =cj/string-truncate-middle= | =system-lib.el= or =cj-text.el= | Defer | Low | Only one current production caller. Good candidate if completion, headings, or report buffers need it. |
+| =cj/--benchmark-method= | =config-utilities.el= | keep | =config-utilities.el= | Keep | Low | Debug command helper, not general architecture. |
+| =cj/--delete-compiled-files-in-dir= | =config-utilities.el= | =cj/delete-files-recursively-matching= | maybe =system-lib.el= | Defer | Low | Destructive behavior needs a strong second caller and path safety contract. |
+
+* Recommended Helper Groups
+
+** Group 1: Executable Discovery
+
+Home: =system-lib.el=.
+
+Proposed API:
+
+#+begin_src emacs-lisp
+(defun cj/executable-available-p (program)
+ "Return non-nil when PROGRAM resolves to an executable in PATH.")
+
+(defun cj/executable-find-or-warn (program feature &optional warning-type)
+ "Return PROGRAM's executable path, or warn that FEATURE is unavailable.")
+#+end_src
+
+Migration:
+
+- Rename =cj/executable-exists-p= to =cj/executable-available-p= and keep a
+ one-cycle obsolete alias/wrapper for compatibility.
+- Replace =cj/mail--executable-or-warn= first because it is already the exact
+ behavior.
+- Audit language modules:
+ - Keep existing =use-package :if= checks on built-in =executable-find= during
+ this project unless there is a separate load-order reason to change them.
+ - Use =cj/executable-find-or-warn= for interactive commands where the user
+ asked for a feature and needs a clear explanation.
+- Do not warn during startup for every optional language tool unless the feature
+ is explicitly configured to be active.
+
+Behavior:
+
+- =program= is a non-empty string naming an executable.
+- =cj/executable-available-p= returns =t= or =nil=, never the executable path.
+- =cj/executable-find-or-warn= returns the resolved executable path or =nil=.
+- =feature= is a human-readable string used in the warning message.
+- =warning-type= defaults to ='system-lib= unless the caller passes a more
+ specific module symbol.
+- Missing executables warn with level =:warning=.
+- Invalid =program= values (non-string or empty string) signal
+ =wrong-type-argument= via =cl-check-type=. This is a programmer-error path;
+ user-facing error reporting is the caller's responsibility.
+
+Tests:
+
+- program string validation,
+- successful lookup returns path,
+- missing program returns nil,
+- warning type defaults sensibly,
+- warning message includes program and feature.
+
+** Group 2: Shell Command String Helpers
+
+Home: start in =system-lib.el=.
+
+Proposed API:
+
+#+begin_src emacs-lisp
+(defconst cj/shell-safe-argument-regexp "\\`[[:alnum:]_./=+@%:,^-]+\\'")
+
+(defun cj/shell-quote-argument-readable (argument)
+ "Return ARGUMENT unchanged when safe, otherwise shell-quote it.")
+#+end_src
+
+Policy:
+
+- Use this helper only when building shell command strings for display,
+ compilation, or logging and readable safe paths matter.
+- Use plain =shell-quote-argument= when maximum conservatism is preferred and
+ readability does not matter.
+- Prefer argv/process APIs over shell strings for new process execution when
+ possible.
+
+First consumers:
+
+- =dev-fkeys.el= F6 command builder.
+- Candidate later consumers: =prog-c.el= compile command generation,
+ =prog-python.el= test/debug commands, =prog-shell.el= shellcheck command,
+ =mail-config.el= mbsync command.
+
+Justification:
+
+- This is a speculative extraction with one current concrete consumer. It is
+ allowed because =dev-fkeys.el= is not the right long-term owner for shell
+ argument policy, the helper is dependency-free, and several command-building
+ modules already make the same readability/security tradeoff manually.
+
+Behavior:
+
+- =argument= must be a string.
+- Arguments matching =cj/shell-safe-argument-regexp= are returned unchanged.
+- Other arguments are passed to =shell-quote-argument=.
+- This helper is for shell command strings only. New process execution helpers
+ should accept argv lists and should not use this helper internally.
+
+Tests:
+
+- safe paths unchanged,
+- whitespace quoted,
+- shell metacharacters quoted,
+- nil/non-string behavior explicitly errors or normalizes.
+
+** Group 3: Process Execution
+
+Home: =system-lib.el= initially, split to =cj-process.el= if the API grows.
+
+Proposed API:
+
+#+begin_src emacs-lisp
+(cl-defun cj/process-output-or-error
+ (program args &key cwd stdin error-message trim)
+ "Run PROGRAM with ARGS via `process-file' and return stdout.")
+
+(defun cj/git-output-or-error (&rest args)
+ "Run git with ARGS and return stdout, or signal `user-error'.")
+#+end_src
+
+Minimum behavior:
+
+- Accept argv as a list, not a shell command string.
+- Capture stdout/stderr together unless a caller needs them separated.
+- Include program, args, exit status, and trimmed output in failure errors.
+- Optionally bind =default-directory= to =cwd=.
+- Return raw stdout by default; allow =:trim t= for callers that need it.
+
+First migration:
+
+- Extract the generic logic from =cj/--coverage-git-string=.
+- Keep =cj/--coverage-git-merge-base= and =cj/--coverage-git-diff= in
+ =coverage-core.el= because their semantics are coverage-specific.
+
+Justification:
+
+- This is a speculative extraction with one current concrete consumer. It is
+ allowed because =coverage-core.el= is not the right long-term owner for a
+ generic argv/process error-reporting policy, and the helper is a prerequisite
+ for later shell-command hardening in VC, repo reconciliation, Hugo, and
+ language command modules.
+
+Behavior:
+
+- =program= is a non-empty string.
+- =args= is a list of strings.
+- The helper uses =process-file=, not a shell.
+- =cwd=, when non-nil, temporarily binds =default-directory=.
+- =stdin= is out of scope for the first implementation and must be nil. Add a
+ separate design note before supporting string/buffer/file stdin.
+- Stdout and stderr are captured together in a temporary buffer unless a later
+ caller proves separated streams are needed.
+- Exit status =0= is success even when stderr text exists.
+- Exit status non-zero signals =user-error=.
+- Exit status =0= with empty stdout returns =""=, not =nil=.
+- =trim= nil returns raw output. =:trim t= uses =string-trim-right= so leading
+ output whitespace remains intact while common trailing newlines are removed.
+- =error-message= is an optional caller label prepended to the generated error;
+ it does not replace command/status/output details.
+
+Likely later consumers:
+
+- hardened =vc-config.el= clone command,
+- =reconcile-open-repos.el= repository scans,
+- =hugo-config.el= deploy/build commands,
+- language compile/test helpers where argv execution is practical.
+
+Tests:
+
+- success returns stdout,
+- non-zero status signals =user-error=,
+- error includes argv/status/output,
+- cwd is honored,
+- empty output behavior is defined,
+- no shell interpolation occurs.
+
+Table grouping:
+
+- The =cj/process-output-or-error= and =cj/git-output-or-error= rows are one
+ extraction commit. The git helper should be a thin wrapper over the generic
+ process helper.
+
+** Group 4: File/Path Context And Safety
+
+Home: =system-lib.el= for simple predicates; keep test-only setup helpers in
+=testutil-general.el=.
+
+Proposed API:
+
+#+begin_src emacs-lisp
+(defun cj/file-from-context (&optional explicit-filename)
+ "Return a file path from explicit input, current buffer, or Dired point.")
+
+(defun cj/path-assert-in-directory (path directory)
+ "Signal an error unless PATH is inside DIRECTORY.")
+
+(defun cj/safe-recursive-delete-root-p (dir)
+ "Return non-nil when DIR is specific enough to delete recursively.")
+#+end_src
+
+Policy:
+
+- Prefer built-in =file-in-directory-p= directly unless the caller needs a named
+ project policy.
+- Extract =cj/file-from-context= early because it is a useful command helper and
+ already lives in a too-broad command module.
+- Extract deletion safety only when a production destructive command is ready to
+ consume it. The test harness can continue using test-local names until then.
+
+First consumers:
+
+- =system-utils.el= =cj/open-file-with-command= and =cj/xdg-open=.
+- =external-open.el= and =dirvish-config.el= once their file-open behavior is
+ aligned.
+- Production destructive/deploy commands only after policy review.
+
+Behavior:
+
+- =cj/file-from-context= resolves in priority order: explicit filename,
+ current =buffer-file-name=, Dired file at point.
+- It returns =nil= rather than prompting. Interactive commands decide whether
+ to prompt or signal.
+- It may declare Dired functions but must not require a Dired implementation
+ package at load time.
+- =cj/path-assert-in-directory= signals =user-error= with both paths in the
+ message.
+- =cj/safe-recursive-delete-root-p= is not implemented until a production
+ destructive caller is ready to adopt it.
+
+Tests:
+
+- explicit filename wins,
+- buffer file fallback,
+- Dired point fallback,
+- nil when no context exists,
+- path containment handles =..= and symlinks according to documented policy,
+- destructive safe-root rejects =/=, =~/=, =temporary-file-directory=,
+ =user-emacs-directory=, and =default-directory=.
+
+** Group 5: External Open Command Resolution
+
+Home: =external-open.el=.
+
+Proposed API:
+
+#+begin_src emacs-lisp
+(defun cj/external-open-command ()
+ "Return the platform default opener command.")
+
+(defun cj/external-open-launcher-p (command)
+ "Return non-nil when COMMAND should be detached as a desktop launcher.")
+#+end_src
+
+Decision:
+
+- =external-open.el= owns platform opener command resolution because this is
+ workflow policy, not a foundation predicate.
+- =external-open.el= may require =host-environment= for predicates.
+- =host-environment.el= remains predicate-only.
+- =system-lib.el= does not learn external-open workflow semantics.
+- Have =dirvish-config.el= call that owner rather than duplicating OS cases.
+
+Behavior:
+
+- =cj/external-open-command= returns a command string or signals =user-error=
+ for unsupported hosts.
+- The Linux default is =xdg-open=, macOS is =open=, and Windows is =start=.
+ A future helper for explicitly opening folders in Explorer is out of scope
+ for this group.
+- =cj/external-open-launcher-p= returns boolean and has no side effects.
+- The helpers only resolve commands; callers remain responsible for choosing
+ =call-process=, =start-process=, or a shell fallback.
+
+Tests:
+
+- Linux returns =xdg-open=,
+- macOS returns =open=,
+- Windows returns =start=,
+- unsupported host errors clearly,
+- launcher predicate handles =xdg-open=, =open=, =start=.
+
+** Group 6: Org-Safe Text
+
+Home: =cj-org-text.el= from first extraction.
+
+This is also the first topic library to land; the extraction commit
+demonstrates the topic-library header pattern from [[*Library File Header Standard][Library File Header
+Standard]].
+
+Proposed API:
+
+#+begin_src emacs-lisp
+(defun cj/org-sanitize-body-text (text)
+ "Prevent external body TEXT from creating unintended Org headings.")
+
+(defun cj/org-sanitize-property-value (text)
+ "Flatten TEXT for safe use as an Org property value.")
+
+(defun cj/org-sanitize-heading (text)
+ "Flatten TEXT for safe use as an Org heading.")
+#+end_src
+
+Source:
+
+- =calendar-sync--sanitize-org-body=
+- =calendar-sync--sanitize-org-property-value=
+- =calendar-sync--sanitize-org-heading=
+
+Likely consumers:
+
+- =calendar-sync.el= event headings/properties/body,
+- =org-webclipper.el= clipped page titles/content,
+- =ai-conversations.el= model output persisted into Org,
+- =mail-config.el= mail subjects inserted into capture/templates,
+- future external ingest workflows.
+
+Justification:
+
+- This is a speculative extraction with one current concrete consumer family.
+ It is allowed because =calendar-sync.el= is not the right long-term owner for
+ generic external-text Org safety, the helpers are string-only, and several
+ external-ingest workflows need the same policy.
+
+Important distinction:
+
+- =calendar-sync--unescape-ics-text= should stay in =calendar-sync.el= because
+ it is ICS-specific.
+- =calendar-sync--strip-html= may become =cj/text-strip-html= later, but only
+ with a docstring that says it is a lightweight regex cleanup, not a full HTML
+ parser.
+
+Behavior:
+
+- Nil input returns nil.
+- Body text preserves line breaks but replaces leading Org heading stars with
+ dashes so external text cannot create outline entries.
+- Property values and headings flatten newlines and collapse internal
+ whitespace to a single space.
+- Heading sanitization composes body sanitization and property-value
+ flattening.
+- These helpers do not parse Org. They are string guards for external text
+ before insertion into Org structures.
+
+Tests:
+
+- Move the existing calendar sanitizer tests to the new helper names.
+- Add consumer tests showing calendar output still sanitizes correctly.
+- Add webclipper/AI/mail tests only when those modules are migrated.
+
+** Group 7: Cache With TTL And Invalidation
+
+Home: =cj-cache.el=, not =system-lib.el=, once implemented.
+
+Current patterns:
+
+- =modeline-config.el=: per-buffer VC cache with key/time/value/set-p and
+ after-save/after-revert invalidation.
+- =org-agenda-config.el=: global agenda-file cache with TTL and building flag.
+- =org-refile-config.el=: global refile-target cache with TTL and building
+ flag.
+
+Proposed API shape:
+
+#+begin_src emacs-lisp
+(cl-defstruct cj/cache
+ key value timestamp set-p ttl building-p)
+
+(defun cj/cache-valid-p (cache key &optional now)
+ "Return non-nil when CACHE has a value for KEY that has not expired.")
+
+(defun cj/cache-get-or-rebuild (cache key rebuild-fn &optional force)
+ "Return CACHE value for KEY or rebuild it with REBUILD-FN.")
+
+(defun cj/cache-clear (cache)
+ "Clear CACHE state.")
+#+end_src
+
+This API is only illustrative. The real design must decide whether caches are:
+
+- structs stored in one variable,
+- plists,
+- closures,
+- several caller-owned variables with helper predicates.
+
+Recommendation:
+
+- Do not extract cache helpers first.
+- Behavioral normalization (lifecycle alignment between agenda/refile caches)
+ is owned by init-load-graph Phase 6. This project extracts the shared pattern
+ into =cj-cache.el= once that lifecycle alignment lands, or in parallel if the
+ design addendum proves the API can drive the alignment.
+- Then decide whether modeline's buffer-local cache can use the same library or
+ should remain specialized.
+- Phase 5 step 1 produces =docs/design/cache-helper-design.org=. Until that
+ file exists, =cj-cache.el= must not be created. The addendum is the
+ prerequisite for any cache extraction commit.
+
+Tests:
+
+- valid cache hit,
+- forced rebuild,
+- TTL expiration,
+- nil value can be cached distinctly from "not set",
+- rebuild flag is cleared on errors,
+- hook invalidation clears only intended cache scope.
+
+** Group 8: Logging And Warnings
+
+Home: =system-lib.el=.
+
+Proposed API:
+
+#+begin_src emacs-lisp
+(defun cj/message-log-only (format-string &rest args)
+ "Append a formatted message to *Messages* without minibuffer echo.")
+
+(cl-defun cj/display-warning-once (type message &key level key)
+ "Display warning MESSAGE once for KEY.")
+#+end_src
+
+Recommendation:
+
+- Rename =cj/log-silently= to =cj/message-log-only= with an obsolete alias when
+ this low-priority helper is touched. Do not put this rename in the first
+ extraction wave.
+- Do not add =cj/display-warning-once= until there is repeated once-only warning
+ behavior. For ordinary warnings, =display-warning= is already readable.
+
+Behavior:
+
+- =cj/message-log-only= preserves the current =cj/log-silently= behavior:
+ append one formatted message to =*Messages*= without echoing in the
+ minibuffer, ensure the message starts on its own line, and ensure the buffer
+ ends with a newline.
+- =cj/display-warning-once= is not implemented until a second caller proves the
+ need. When implemented, duplicate suppression is process-local and keyed by
+ =(type key)=. =:level= defaults to =:warning=.
+
+Tests:
+
+- log-only inserts into =*Messages*=,
+- warning-once suppresses duplicates by key,
+- warning-once does not suppress unrelated warnings.
+
+** Group 9: File Content Helpers
+
+Home: =system-lib.el= only if reused.
+
+Potential API:
+
+#+begin_src emacs-lisp
+(defun cj/read-file-string (file)
+ "Return FILE contents as a string.")
+
+(defun cj/write-file-string (file contents)
+ "Write CONTENTS to FILE, creating parents if requested by option.")
+#+end_src
+
+Source:
+
+- =cj/theme-read-file-contents=
+- =cj/theme-write-file-contents=
+
+Recommendation:
+
+- Defer. The theme helpers are small and theme-specific today.
+- Extract only when another module needs identical read/write behavior.
+
+* Functions To Leave Alone For Now
+
+- =calendar-sync--parse-*=, =calendar-sync--expand-*=, and
+ =calendar-sync--unescape-ics-text=: calendar/ICS domain logic.
+- =cj/--coverage-parse-simplecov= and =cj/--coverage-parse-diff-output=:
+ coverage domain parsing.
+- =cj/--coverage-git-merge-base= and =cj/--coverage-git-diff=: coverage-specific
+ git policy after the low-level runner is extracted.
+- =cj/modeline-string-cut-middle=: good utility shape, but currently only one
+ real caller.
+- =cj/--generate-roam-slug= and org-roam formatting helpers: Org-roam workflow
+ semantics. These stay local because they are data/workflow-aware, unlike the
+ string-only Org sanitizers that only protect external text from becoming
+ unintended headings or malformed properties.
+- text editing commands in =custom-*.el=: many are reusable commands, but they
+ are user-facing editing features rather than foundation utilities.
+- package configuration helpers in =prog-*.el=: keep mode/package-specific
+ setup close to the package unless a generic policy emerges.
+- test fixture builders in individual test files: keep local unless three or
+ more suites duplicate the same fixture shape.
+
+* Migration Phases
+
+Extraction commits should use conventional commit prefixes consistently:
+
+- =refactor:= for behavior-preserving helper moves/renames and call-site
+ migrations.
+- =feat:= only when adding a new reusable helper for a new user-visible
+ capability.
+- =test:= for test-only follow-up work.
+- =docs:= for spec, inventory, and design addendum updates.
+
+** Phase 1: Inventory And Tags
+
+Create an inventory of private helpers across =modules/=.
+
+Inventory artifact:
+
+- Create =docs/design/utility-inventory.org=.
+- Use an Org table with the fields below.
+- Scope it to the candidate table in this spec plus new candidates discovered
+ during module walkthroughs. It is not required to list every private helper
+ across the whole codebase before Phase 2 can start.
+- Treat the inventory as living documentation. Cleared high-priority candidates
+ may move to Phase 2 before the whole inventory is complete.
+- This inventory is independent from the module-shape inventory maintained by
+ [[file:init-load-graph.org][init-load-graph.org]]. The two projects may walk the same files, but they
+ record different facts in separate artifacts.
+
+For each helper record:
+
+- current symbol,
+- file,
+- public/private status,
+- dependencies,
+- side effects,
+- candidate home,
+- proposed name,
+- real consumers,
+- test file(s),
+- extraction priority,
+- decision: extract, defer, keep, or replace with built-in.
+
+Audit output:
+
+- An =Audit= row produces an inventory decision: =Migrate=, =Leave=, or
+ =Defer=.
+- =Migrate= decisions should create or update a concrete =todo.org= task.
+- =Leave= and =Defer= decisions should record the rationale in the inventory so
+ the same audit is not repeated later.
+
+Exit criteria:
+
+- Every candidate in the table above is represented.
+- Each candidate has at least one specific next action.
+- No code behavior changes.
+
+** Phase 2: Low-Risk Foundation Helpers
+
+Extract helpers that are string/path/process-light and either have direct
+consumers or are explicitly justified as wrong-layer speculative extractions in
+this spec.
+
+Suggested order:
+
+1. =cj/executable-find-or-warn=
+2. =cj/shell-quote-argument-readable=
+3. =cj/process-output-or-error= and =cj/git-output-or-error=
+4. =cj/file-from-context=
+
+Exit criteria:
+
+- Each extraction has moved or added focused tests.
+- Consumer modules explicitly require =system-lib=.
+- Full targeted tests pass after each extraction.
+- Startup does not gain new package dependencies.
+- Existing =use-package :if= checks are not migrated away from built-in
+ predicates unless a separate load-order task explicitly requires it.
+
+** Phase 3: Org-Safe Text Helpers
+
+Extract the calendar Org sanitizers and migrate calendar first.
+
+Then migrate consumers one at a time:
+
+- =org-webclipper.el=,
+- =ai-conversations.el=,
+- =mail-config.el= if mail-to-org behavior needs it.
+
+Exit criteria:
+
+- Existing calendar sanitizer tests pass under new helper names.
+- Calendar integration tests still pass.
+- New consumers have behavior tests showing external text cannot create
+ unintended headings/properties.
+
+** Phase 4: External Open Consolidation
+
+Pick a single owner for default system opener resolution.
+
+Then migrate:
+
+- =system-utils.el=,
+- =dirvish-config.el=,
+- =external-open.el=.
+
+Exit criteria:
+
+- One platform-opener decision point exists.
+- Dired/Dirvish/current-buffer open workflows still work.
+- Host tests cover Linux/macOS/Windows branches via predicate stubs.
+
+** Phase 5: Cache Abstraction
+
+Do this after simpler extractions, because cache abstraction is riskier.
+
+Suggested order:
+
+1. Write a Phase 5 design addendum with the exact cache API. The output of this
+ step is a design document, not code.
+2. Normalize agenda/refile cache code first.
+3. Add tests for rebuild, TTL, nil cache values, and error cleanup.
+4. Consider modeline VC cache only after global cache behavior is stable.
+
+Exit criteria:
+
+- Agenda/refile behavior is unchanged.
+- Building flags clear on errors.
+- Batch startup does not start extra timers/processes.
+- Cache helper does not obscure caller-specific rebuild logic.
+
+** Phase 6: Deferred And Opportunistic Helpers
+
+Consider lower-priority helpers only when a second consumer appears:
+
+- =cj/string-truncate-middle=,
+- =cj/read-file-string= / =cj/write-file-string=,
+- =cj/with-timer=,
+- recursive file deletion helpers,
+- warning-once helpers.
+
+* Test Strategy
+
+- Keep the project convention of focused helper tests.
+- Name new tests after the library/helper, for example:
+ - =tests/test-system-lib-executable-find-or-warn.el=
+ - =tests/test-system-lib-shell-quote-argument-readable.el=
+ - =tests/test-system-lib-process-output-or-error.el=
+ - =tests/test-system-lib-file-from-context.el=
+ - =tests/test-cj-org-text-sanitize-heading.el=
+- Move unit tests with extracted helpers.
+- Keep consumer tests that prove the original workflow still calls the helper
+ correctly.
+- Stub side-effectful primitives with =cl-letf=:
+ - =executable-find=,
+ - =display-warning=,
+ - =process-file=,
+ - Dired file-at-point helpers,
+ - host predicates.
+- For each extraction commit, run targeted tests for the helper and each touched
+ consumer. Run full =make test= before marking the task =VERIFY=.
+
+Interactive coverage:
+
+- For each migrated consumer with a keybinding, add or keep a test asserting
+ =(key-binding (kbd "..."))= resolves to the intended command symbol.
+- For non-trivial interactive flows such as =completing-read= prompts or
+ confirmation dialogs, use =with-simulated-input= where the helper is reachable
+ in batch.
+- Use =execute-kbd-macro= for non-graphical keypress flows where it is simpler
+ than stubbing command internals.
+- Mark visual-only checks, such as modeline appearance, theme rendering, and
+ font rendering, as manual smoke checks in the load-graph project rather than
+ utility helper tests.
+
+Coverage measurement:
+
+- Extracted helpers should meet the project's utility coverage target
+ (currently 90%) measured with =make coverage=.
+- Phase 5 cache work should meet the same target and include explicit
+ unwind/error-path tests for rebuilding flags and invalidation cleanup.
+
+Tooling: Cask, ert-runner, buttercup:
+
+- Keep this project on bare =emacs --batch= plus ERT because this repository is
+ a personal Emacs configuration, not a redistributable package.
+- =with-simulated-input= is acceptable as a test-only dependency for
+ interactive coverage when a migrated helper is reachable through a command
+ path.
+- Cask, ert-runner, buttercup, and ecukes are out of scope unless a future task
+ gives a concrete reason to adopt them.
+
+Header validation:
+
+- Add a smoke test that asserts =modules/system-lib.el= and any future
+ =modules/cj-*.el= library file declares the required library header lines.
+
+Test relocation policy:
+
+- The extraction commit moves the helper, helper unit tests, consumer call
+ sites, and consumer call-site test updates together.
+- Empty old helper test files are deleted in the same commit.
+- If an old consumer test file still has consumer behavior coverage, keep it
+ and remove only the helper-specific tests.
+- Avoid commits that only rename tests without moving behavior unless the rename
+ is too large to review with the helper extraction.
+
+* Acceptance Criteria
+
+This project is complete when:
+
+- =system-lib.el= has a documented role and contains only foundation-safe
+ helpers.
+- Each extracted helper has a stable public name, tests, and explicit consumer
+ =require= statements.
+- Feature modules no longer own generic executable lookup, process execution,
+ shell-readable quoting, or Org-safe text sanitation.
+- External open command resolution has a single owner.
+- Agenda/refile/modeline cache duplication has either been intentionally
+ consolidated or explicitly deferred with rationale.
+- No helper extraction introduces package/network/timer side effects at load
+ time.
+- Phase 2 and Phase 3 migrations record =emacs-init-time= against a phase
+ baseline; regressions around 25 ms or more should be investigated and
+ explained.
+- Full =make test= passes after the final migration.
+
+* Risks And Mitigations
+
+** Risk: Premature abstraction
+
+Mitigation:
+
+- Require at least two real consumers.
+- Keep "defer" as a valid inventory decision.
+- Avoid helpers named around vague concepts such as "do thing safely."
+
+** Risk: Foundation module becomes too broad
+
+Mitigation:
+
+- Keep =system-lib.el= dependency-light.
+- Split topic libraries when a helper family becomes stateful or domain-specific.
+- Track every new =require= added to =system-lib.el=.
+
+** Risk: Behavior changes during rename
+
+Mitigation:
+
+- Move tests first where practical.
+- Preserve old public symbols temporarily with aliases only when needed.
+- Change one helper family per commit.
+- Rename commits that preserve a one-cycle alias use =refactor:= and mention
+ the alias in the commit body; the alias does not need a separate commit.
+
+** Risk: Warnings become noisy at startup
+
+Mitigation:
+
+- Use warning helpers for user-invoked missing features.
+- Keep optional package =:if= checks quiet unless the user explicitly enabled
+ the feature.
+- Do not warn for every absent language tool on startup.
+
+** Risk: Helper calls in =use-package :if= create new load-order requirements
+
+Mitigation:
+
+- Do not migrate =use-package :if= clauses in this project. Keep built-in
+ predicates such as =executable-find= there unless a load-graph task handles
+ the ordering explicitly.
+- Migrate function-body, command-body, and =:config= callers first, where
+ =require 'system-lib= can be ordinary and local.
+- If a future migration needs a helper in =:if=, document the load-order
+ prerequisite in that commit and ensure =system-lib= is required before the
+ =use-package= form is macroexpanded/evaluated.
+
+** Risk: Process helper hides security decisions
+
+Mitigation:
+
+- Prefer argv APIs over shell command strings.
+- Keep shell-string helpers clearly separate from process-file helpers.
+- Document when a caller intentionally uses a shell.
+
+** Risk: Helper API turns out to be wrong
+
+Mitigation:
+
+- Revert the extraction commit, restore the source-module helper name and tests,
+ and file a redesign task.
+- Do not patch a vague or wrong foundation API in place after consumers have
+ started migrating; stabilize the API before the second wave of consumers.
+
+* Open Questions
+
+- Should =system-lib.el= eventually become only an aggregator requiring topic
+ libraries, or remain the primary helper file?
+- Should =system-utils.el= and =config-utilities.el= remain separate command
+ modules after library extraction? Default answer for this project: keep them
+ separate; revisit only during the load-graph refactor.
+- What should the cache helper represent: a struct, a plist, closures, or
+ caller-owned variables plus helper predicates?
+
+Closed alias check:
+
+- A local search across =/home/cjennings/code=, =/home/cjennings/projects=,
+ =/home/cjennings/go=, and this repository found no uses of
+ =cj/executable-exists-p= or =cj/log-silently= outside this Emacs
+ configuration. Alias decisions are therefore for in-repo compatibility and
+ user muscle memory, not external package consumers.
+
+* Recommended First Three Commits
+
+Phase 2 also extracts =cj/file-from-context= as commit 4. It is not in this
+first-three list because it has direct multi-consumer pressure and is not a
+speculative extraction; the three commits below are the speculative or
+high-policy commits that need extra care.
+
+1. Extract =cj/executable-find-or-warn= to =system-lib.el=.
+ - Migrate =mail-config.el=.
+ - Add =tests/test-system-lib-executable-find-or-warn.el=.
+ - Keep optional language/package checks unchanged.
+2. Extract =cj/shell-quote-argument-readable= to =system-lib.el=.
+ - Migrate =dev-fkeys.el=.
+ - Move F6 shell-quote tests or add focused system-lib tests.
+ - Do not replace all =shell-quote-argument= callers yet.
+ - This is a wrong-layer speculative extraction; record expected future
+ consumers in the inventory.
+3. Extract =cj/process-output-or-error= and =cj/git-output-or-error=.
+ - Migrate =coverage-core.el= only.
+ - Keep coverage-specific git wrappers in =coverage-core.el=.
+ - Add tests that stub =process-file=.
+ - This is a wrong-layer speculative extraction; record expected future
+ consumers in the inventory.
+
+These give the project useful proof points without touching stateful cache
+behavior or broader load-order mechanics.