aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2026-04-22 17:00:39 -0500
committerCraig Jennings <c@cjennings.net>2026-04-22 17:00:39 -0500
commit83ac3201023f8736c234da27a0642f21786adcfc (patch)
tree0967e330c5acc0cc99d9ec1f947220f033871847 /docs
parent99080be3310c60eb584a7253c9656e59fed33e69 (diff)
downloaddotemacs-83ac3201023f8736c234da27a0642f21786adcfc.tar.gz
dotemacs-83ac3201023f8736c234da27a0642f21786adcfc.zip
docs: add design docs for coverage and dev-setup-project
Two new design docs for pending todo.org tickets. docs/design/coverage.org describes diff-aware coverage reporting with pluggable backends. Primary use case is pre-commit feedback on in-flight changes. LCOV is the shared output format across languages. docs/design/dev-setup-project.org describes an interactive helper that detects a project's shape and writes per-subdirectory .dir-locals.el files for the F4/F6/F7 dev block, with optional starter Makefile generation. Three-tier detection: existing Makefile, existing package.json or pyproject.toml scripts, or fall-back generation. Both tickets in todo.org reference their design docs via org file: links.
Diffstat (limited to 'docs')
-rw-r--r--docs/design/coverage.org160
-rw-r--r--docs/design/dev-setup-project.org158
2 files changed, 318 insertions, 0 deletions
diff --git a/docs/design/coverage.org b/docs/design/coverage.org
new file mode 100644
index 00000000..a913a2bc
--- /dev/null
+++ b/docs/design/coverage.org
@@ -0,0 +1,160 @@
+#+TITLE: Design: Coverage Reporting
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-04-22
+
+* Status
+
+Draft. Not yet implemented.
+
+* Problem
+
+Before committing or opening a PR, there's no quick way to answer "are the lines I just changed actually covered by tests?" Line-level coverage for the *whole* project is also missing, and there's no artifact to track coverage over time.
+
+The primary user-facing need is the first one: point-in-time feedback on in-flight changes, triggered from Emacs. The other two (whole-project report, long-term artifact) fall out naturally once the primary path exists.
+
+The tooling should be pluggable so the same workflow covers Elisp today and Python, TypeScript, and Go later — without rebuilding the UI for each language.
+
+* Non-Goals
+
+- Continuous in-buffer overlays (fringe marks, line highlights). Parked over performance concerns.
+- Mutation testing or any signal other than line coverage.
+- CI integration beyond emitting an LCOV artifact. No coveralls, no GitHub Actions wiring.
+- Shadowing or replacing existing test-running commands (=make test=, =make test-file=, etc.).
+
+* Approaches Considered
+
+** Recommended: diff-aware report with pluggable backends
+
+Core engine reads an LCOV file, shells to ~git diff~ at a selectable scope, intersects, and displays the result in a compilation-mode-derived buffer. Language-specific "backends" each produce LCOV in their own way and register themselves with the core.
+
+*Pros:* Directly serves the primary use case. LCOV is a universal format, so new languages plug in without touching the core. Compilation-mode inheritance gives free =next-error= / =previous-error= navigation.
+
+*Cons:* More code than a "just run coverage and read the output" approach. Backend registry adds one layer of indirection (small — ~30 lines).
+
+** Rejected: non-interactive pre-commit hook
+
+Would run coverage on every commit and report uncovered-changed-lines to stderr. Literal fit for the use case but adds a long delay to every commit and offers no way to inspect non-staged scopes.
+
+** Rejected: coverage as a =review-code= skill criterion
+
+Would fold coverage into the existing pre-commit review skill. Clean in principle, but couples =review-code= to Emacs-specific tooling and makes ad-hoc inspection (outside a review) awkward.
+
+** Rejected: mutation testing instead of line coverage
+
+Stronger signal than coverage but minutes-to-hours runtime on the current 265-file suite, and no polished Elisp tool exists. Different conversation.
+
+* Design
+
+** Architecture
+
+Three files:
+
+- =modules/coverage-core.el= — engine + backend registry + user-facing command. Language-agnostic.
+- =modules/coverage-elisp.el= — the initial backend. Registers itself on load.
+- (Future) =modules/coverage-python.el=, =coverage-typescript.el=, =coverage-go.el= — each ~30 lines, self-registering.
+
+=init.el= requires the core and the active backends.
+
+*** Backend protocol
+
+Each backend is a plist registered into =cj/coverage-backends=:
+
+#+begin_src emacs-lisp
+(:name 'elisp
+ :detect (lambda () ...) ; non-nil if current project matches
+ :run (lambda (cb) ...) ; kick off coverage build; invoke CB with LCOV path
+ :lcov-path (lambda () ...)) ; where the LCOV lives (for re-reading without running)
+#+end_src
+
+Detection precedence: =.dir-locals.el= override (=cj/coverage-backend= set to a backend name), then project-root fingerprints (=go.mod=, =pyproject.toml=, =package.json=, =.el= files + Makefile, etc.). First =:detect= that matches wins. No silent fallback — if nothing matches, the command errors with guidance.
+
+*** Pure helpers
+
+- =cj/--coverage-parse-lcov FILE= → hash-table ={file → covered-line-set}=.
+- =cj/--coverage-changed-lines SCOPE BASE= → hash-table ={file → changed-line-set}= by shelling a =git diff --unified=0= for the selected scope and parsing hunk headers.
+- =cj/--coverage-intersect COVERED CHANGED= → per-file records with three buckets: covered, uncovered, not-tracked.
+
+All three are pure, fully ERT-tested.
+
+** Data Flow
+
+1. User invokes =cj/coverage-report= (bound to =F7=).
+2. Core resolves the backend for the current project.
+3. =completing-read= prompts for scope:
+ - "Working tree — all uncommitted changes"
+ - "Staged — about to commit"
+ - "Branch vs parent" (uses =cj/coverage-base-branch= → =@{upstream}= → =main= in order)
+ - "Branch vs main" (explicit)
+4. Freshness check: if =lcov.info= is missing, or older than the newest changed file, prompt "Run coverage now?" Yes runs the backend's =:run= asynchronously via =compile=; no reads the stale file anyway.
+5. Parse LCOV, compute changed lines, intersect.
+6. Display a report buffer in a mode derived from =compilation-mode=.
+
+** Persistence
+
+- =.coverage/lcov.info= at the project root, gitignored. Overwritten on each run.
+- No long-term storage. Historical tracking is explicitly out of scope for v1.
+
+** Error Handling
+
+*Pre-flight:*
+- No backend matches → =user-error= with instructions to register a backend or set =.dir-locals.el=.
+- =.dir-locals.el= names an unknown backend → error listing registered backends.
+- Not in a git repository → error; don't swallow git's stderr.
+- "Branch vs main" scope on a repo with no common ancestor (orphan branch, shallow clone missing the fork point) → "no merge base with main" error, suggest "Working tree" or "Staged" scope.
+
+*During the coverage run:*
+- Backend =:run= fails (test failure, Make error) → keep the =compile= buffer visible, do *not* proceed to display a report. Partial data is worse than no data.
+- Run completes but no LCOV produced → error naming the expected path.
+
+*Post-flight classification:* three buckets, not two.
+- *Covered* — changed line in LCOV's covered-line set.
+- *Uncovered* — changed line in a tracked file but not covered.
+- *Not tracked* — changed file isn't in LCOV at all (test files, READMEs, config). Reported separately — don't conflate "coverage didn't look here" with "tests didn't exercise this code."
+
+*Happy-path degenerates:*
+- Zero changed lines in scope → "No changes in this scope; nothing to report."
+- All changed lines covered → "N of N changed lines covered. "
+
+** Keybindings
+
+*Global:*
+- =F7= → =cj/coverage-report= (prompts scope, shows report).
+- =C-u F7= → force re-run regardless of LCOV freshness.
+
+*In the report buffer* (compilation-mode derived, most inherited for free):
+- =RET= → jump to source under point.
+- =n= / =p= → next / previous uncovered line.
+- =g= → refresh (re-run + redisplay).
+- =q= → bury buffer.
+
+*Globally available via compilation-mode integration:*
+- =M-g n= / =M-g p= → =next-error= / =previous-error= on the last compilation buffer.
+- =C-x `= → visit next uncovered line without leaving the current buffer.
+
+The =F4=–=F7= developer block (compile+run, debug, test, coverage) gets its full rework in a separate todo ticket. The coverage work binds =F7= now because it's its final position.
+
+** Testing
+
+*Pure helpers, fully tested* (Normal / Boundary / Error for each):
+- =cj/--coverage-parse-lcov= — handcrafted LCOV fragments in temp files; empty, headers-only, spaces/unicode in filenames, malformed lines, missing =end_of_record=.
+- =cj/--coverage-changed-lines= — =cl-letf= over =shell-command-to-string= to return canned =git diff= output; single hunk, new-file hunk, deletion-only hunk, binary marker, no-diff case.
+- =cj/--coverage-intersect= — pure table-in / table-out; covered ⊇ changed, unknown files, nil/empty inputs.
+
+*Backend registry, structurally tested:*
+- =cj/coverage-backend-for-project ROOT= — synthetic temp project roots with marker files; assert correct backend. Registration-order test: two backends match, first-registered wins.
+
+*Not tested:*
+- =cj/coverage-report= interactive command — one smoke test with a prepared LCOV and a stubbed git-diff. No tests for the prompt UI or the compilation-buffer display.
+- The elisp backend's =:run= function — shells to =make coverage=; integration-test-shaped, low value, slow. Skipped by design.
+
+* Open Questions
+
+- [ ] Which tests should a coverage run actually execute? All of them (simple, slow for 265 files), or only the test files whose target modules changed (fast, but dependent-test discovery in Elisp is non-trivial)? Deferred until implementation.
+- [ ] Default behavior when LCOV is stale but not missing: prompt, or auto-rerun? Current design prompts. Revisit after first use.
+- [ ] Whether =cj/coverage-base-branch= should be a single value or a list of candidates (useful if you routinely stack PRs more than one level deep). Single value for v1.
+
+* Next Steps
+
+1. Replace the existing =[#C] Integrate undercover.el for test coverage= entry in =todo.org= with a sharper implementation ticket referencing this design.
+2. Begin implementation, starting with the pure helpers (TDD) and the elisp backend, then the =cj/coverage-report= command, then the =make coverage= Makefile target.
+3. Open questions above → individual =arch-decide= ADRs if they turn out to be load-bearing; otherwise resolve inline during implementation.
diff --git a/docs/design/dev-setup-project.org b/docs/design/dev-setup-project.org
new file mode 100644
index 00000000..280b015b
--- /dev/null
+++ b/docs/design/dev-setup-project.org
@@ -0,0 +1,158 @@
+#+TITLE: Design: cj/dev-setup-project
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-04-22
+
+* Status
+
+Draft. Not yet implemented.
+
+* Problem
+
+Adopting the F4 / F6 / F7 dev-block keybindings (compile+run, test, coverage) on a new project means configuring projectile's per-project compile/run/test commands plus the coverage backend. That's a few minutes of ceremony per project, and the polyglot Docker case (backend + frontend in subdirectories) needs per-subproject configuration that projectile's cache doesn't handle cleanly.
+
+=cj/dev-setup-project= is the interactive helper that removes that ceremony. It detects the project shape, proposes the right =.dir-locals.el= content for each subproject, optionally generates a starter Makefile when none exists, and writes everything in one reviewed step.
+
+* Non-Goals
+
+- Running the detected commands. The helper only writes configuration; you invoke F4 / F6 / F7 afterwards.
+- Managing Dockerfile changes, compose file edits, or container orchestration. Those stay hand-owned.
+- Replacing projectile's cache for simple single-language projects. If you're fine with projectile's prompt-and-cache, don't run the helper.
+- Supporting every possible project shape. The helper targets the shapes the user actually uses: pure Elisp, pure Go, pure Python, pure Node/TS, Docker Compose polyglot.
+
+* Approaches Considered
+
+** Recommended: detect + review buffer + user commits
+
+Interactive command opens a review buffer pre-populated with proposals. User edits inline. On =C-c C-c=, helper writes the files.
+
+Detection is three-tier: existing Makefile, existing package.json / pyproject.toml scripts, or fall back to generating a starter Makefile. Re-runs use the same buffer with status banners (UNCHANGED, WILL UPDATE, WILL CREATE) so nothing changes silently.
+
+*Pros:* Zero silent surprises. User sees exactly what's going to change. Reuses the same UX for initial setup and re-runs.
+
+*Cons:* More code than a "just write the files" approach. Review buffer mode is a small but non-trivial piece of UX.
+
+** Rejected: silent auto-detect and commit
+
+Helper inspects project, writes =.dir-locals.el= immediately with best-guess conventions, prints a summary. Zero friction on the easy cases. Wrong results on edge cases go unnoticed until you hit F4/F6 and they misfire. Not worth the friction savings.
+
+** Rejected: wizard (prompt each question in sequence)
+
+Helper asks "Test command: [default: make test] > " and so on. Explicit and safe, but slow and the series of minibuffer prompts is a worse fit than a single editable review buffer.
+
+** Rejected: hybrid (silent for obvious cases, wizard for polyglot)
+
+Two code paths to maintain. The review-buffer approach is already fast for obvious cases (one =C-c C-c= to accept the proposal) and correct for polyglot cases. No need for a second path.
+
+* Design
+
+** Detection
+
+Three tiers, checked in order.
+
+*** Tier 1: existing Makefile
+
+Parse Makefile for =.PHONY:= declarations and bare =^target:= lines. Collect the target names.
+
+Best-guess role mapping:
+- *compile* role: prefer =build=, =compile=, =install=
+- *run* role: prefer =run=, =start=, =dev=, =serve=
+- *test* role: prefer =test=, =tests=, =check=
+
+If multiple targets match (e.g., both =test= and =check=), pick the first match and list the others in the review buffer as "other available targets."
+
+*** Tier 2: existing package.json scripts or pyproject.toml sections
+
+- =package.json= with a =scripts= block: parse the block, same best-guess mapping (=dev= → run, =build= → compile, =test= → test). Command prefix is =npm run=.
+- =pyproject.toml= with =[tool.pytest]= or =[project.scripts]=: for v1, skip this — fall back to =pytest= as the test command if =pytest= is on PATH. More sophisticated parsing can come later.
+
+*** Tier 3: no build file found
+
+Propose a starter Makefile in the review buffer. User edits or declines.
+
+The starter Makefile adapts to the detected project type:
+
+- Elisp: =make compile=, =make test= wrapping =emacs --batch= invocations.
+- Go: =go build=, =go run=, =go test ./...=.
+- Python (non-Docker): =pip install -r requirements.txt=, =python -m <module>=, =pytest=.
+- Node/TS (non-Docker): =npm install=, =npm run dev=, =npm test=.
+- Docker Compose polyglot: =docker compose build=, calls to user-named external run script (prompted), =docker compose exec <service> <runner>= for tests per service.
+
+** Review Buffer
+
+Custom major mode derived from =emacs-lisp-mode= with two local bindings:
+
+- =C-c C-c= — parse the buffer, validate all blocks, write files, show summary.
+- =C-c C-k= — abort, write nothing.
+
+Block syntax: =;; ==== <path> ====[ <status>]== banner lines delimit each file's proposed content. Status banner is one of:
+
+- (unset, initial setup) — file will be created
+- =[UNCHANGED]= — current file matches proposal; skipped unless user edits
+- =[WILL UPDATE]= — current file differs; shown with both current and proposed for the user to pick
+- =[WILL CREATE]= — file doesn't exist yet; will be created
+
+=;; ==== .gitignore (append if missing) ===== is a special banner — lines under it are appended to =.gitignore= if not already present.
+
+=;; ==== Makefile ====[...]= is a special banner — only honored if no Makefile exists at the target path. On re-run with an existing Makefile, this banner is suppressed entirely.
+
+** Escape Hatch
+
+A =.dir-locals.el= containing =;;; cj/dev-setup-project: ignore= as the first line is skipped on re-run. Lets the user diverge intentionally without every re-run reverting.
+
+** Write Step
+
+On =C-c C-c=:
+
+1. Parse all blocks. Validate each is well-formed elisp (or well-formed Makefile / gitignore entries).
+2. If any block is malformed, show an error in the review buffer and do not write.
+3. For each WILL UPDATE / WILL CREATE block: write the file.
+4. For the gitignore block: append each line only if not present (idempotent).
+5. Clear projectile's per-project command cache for this project (so new commands take effect on next F4/F6).
+6. Print a summary: ="Wrote backend/.dir-locals.el, frontend/.dir-locals.el, appended 2 lines to .gitignore."=
+
+** Coverage Backend Forward References
+
+The helper writes =(cj/coverage-backend . python)=, =(cj/coverage-backend . typescript)=, etc. even when those backends don't exist yet (MVP coverage ships Elisp only). The binding silently does nothing until the backend lands; after that, it activates automatically. Simpler than leaving empty and coming back.
+
+** Example Flows
+
+*** Fresh setup on orchestration_dashboard_mvp (Tier 3, no Makefile)
+
+Review buffer proposes a Makefile (calling the user's existing =reset-dashboard.sh= as the =run= target) plus backend/ and frontend/ =.dir-locals.el= files plus gitignore updates. User edits the Makefile's =run= target to match their actual script path. =C-c C-c=. Four files written.
+
+*** Fresh setup on .emacs.d (Tier 1, rich Makefile)
+
+Review buffer shows target-to-role mapping derived from the existing 14-target Makefile (=make compile= → compile role, =make test= → test role; =run= role left nil since this is a config project). Single file written: =.dir-locals.el= at project root.
+
+*** Re-run after adding a new compose service
+
+The helper detects a new =worker= service in docker-compose.yml with a =./worker/= build context. Existing backend/ and frontend/ files show =[UNCHANGED]=. New =worker/.dir-locals.el= block shows =[WILL CREATE]=. =C-c C-c=. One file written.
+
+*** Re-run after renaming a Makefile target
+
+Makefile's =test-frontend= was renamed to =test-frontend-unit=. The helper detects the mismatch and shows frontend/.dir-locals.el as =[WILL UPDATE]= with current and proposed visible. User either accepts (the test command updates) or edits the buffer to keep =test-frontend=. Nothing silent.
+
+* Testing
+
+Pure helpers, fully tested per the project's Normal / Boundary / Error discipline:
+
+- =cj/--dev-setup-parse-makefile-targets FILE= — handcrafted Makefiles. Normal: two-target file with .PHONY. Boundary: tabs vs spaces, continuation lines, pattern-rule targets (skip them). Error: file missing, non-Makefile content.
+- =cj/--dev-setup-parse-package-json-scripts FILE= — synthetic package.json fixtures. Normal: valid scripts block. Boundary: no scripts block, empty scripts. Error: malformed JSON.
+- =cj/--dev-setup-detect-project-shape ROOT= — temp directories with combinations of marker files. Assert returned shape plist. Normal: each single-language case. Boundary: docker-compose polyglot with one subproject, with two subprojects, with a service that uses an external image (no subproject). Error: empty directory returns 'unknown.
+- =cj/--dev-setup-map-targets-to-roles TARGETS= — input list of target names, output role mapping. Normal: well-named project (build/run/test). Boundary: unusual names (start instead of run; check instead of test). Error: empty input returns empty mapping.
+- =cj/--dev-setup-review-buffer-parse CONTENTS= — the buffer-format parser. Normal: well-formed buffer with multiple blocks. Boundary: single block, block with empty body. Error: missing banner, malformed elisp inside a dir-locals block.
+
+Not tested (by design):
+- The interactive command =cj/dev-setup-project= itself — one smoke test that runs against a prepared temp project and asserts the expected files exist after =C-c C-c=.
+- The review-buffer major mode's keybindings.
+
+* Open Questions
+
+- [ ] Whether to also detect Cargo.toml (Rust), pom.xml (Java/Maven), etc. v1 targets Elisp, Go, Python, TS/JS. Rust/Java defer.
+- [ ] Whether =cj/dev-setup-project= should also offer to add a =make coverage= target when generating a Makefile. Probably yes — it's the natural partner to the coverage work.
+- [ ] Whether to support a project-wide config override file (=.cj-dev-setup.el= at project root) that pins choices regardless of what detection finds. Defer unless the detection-only path proves annoying.
+
+* Next Steps
+
+1. Implement after the F-key rework ticket ships.
+2. Open questions above → resolve inline or via =arch-decide= if they turn out to be load-bearing.