diff options
Diffstat (limited to '.ai/workflows/spec-review.org')
| -rw-r--r-- | .ai/workflows/spec-review.org | 113 |
1 files changed, 45 insertions, 68 deletions
diff --git a/.ai/workflows/spec-review.org b/.ai/workflows/spec-review.org index d956f00..0da8e65 100644 --- a/.ai/workflows/spec-review.org +++ b/.ai/workflows/spec-review.org @@ -5,9 +5,9 @@ * Overview -The spec-review workflow evaluates a feature/specification document before implementation and decides one thing: can an engineer implement it confidently, test it thoroughly, and ship behavior that matches the user's mental model? If yes, say so and stop. If no, write a review file next to the spec naming every blocking gap and the concrete change that closes it. +The spec-review workflow evaluates a feature/specification document before implementation and decides one thing: can an engineer implement it confidently, test it thoroughly, and ship behavior that matches the user's mental model? If yes, say so and stop. If no, record every blocking gap and the concrete change that closes it as findings in the spec's own =* Review findings= section. -This is the *reviewer* side of a pair. Its counterpart is the spec-response workflow, which the spec's author runs to fold a review back in. The contract between them is the review file: =<spec-basename>-review.org= (e.g. =docs/issue-query-spec.org= → =docs/issue-query-spec-review.org=). spec-review produces it; spec-response consumes it. +This is the *reviewer* side of a pair. Its counterpart is the spec-response workflow, which the spec's author runs to disposition the findings. The contract between them lives *in the spec*: a =* Review findings= section carrying one =TODO= task per finding, with a =[/]= cookie — the same shape the spec's =* Decisions= section already uses. spec-review writes the findings; spec-response completes them. No separate review file is written, so nothing dangles when a review is processed and the full review/response trail stays in the spec. The goal is not to prove the spec is clever. It is to leave the implementer with *fewer* hidden decisions, not more prose. @@ -28,7 +28,7 @@ A review is complete when: 1. *The implementation-readiness gate has been evaluated* and a rubric label assigned (=Ready= / =Ready with caveats= / =Not ready= / =Needs research=). 2. *If ready:* the user is told plainly ("This spec is implementation-ready. I have no further blocking review notes."), and the review stops — no churn for its own sake. -3. *If not ready:* a =<spec>-review.org= file is written next to the spec, in the standard structure, with every finding specific and actionable (current behavior named, risk explained, change recommended, blocking-or-not stated). +3. *If not ready:* findings are recorded in the spec's =* Review findings= section as =TODO= tasks (one per finding, =[/]= cookie on the heading), each specific and actionable (current behavior named, risk explained, change recommended, blocking-or-not stated). 4. *The spec's review history is updated* with who reviewed it, when, which iteration it was, what changed or was recommended, and why. 5. *Deferred work is logged* to =todo.org= (v1 = =[#B]=, vNext/someday = =[#D]=), not left only in chat. 6. *Implementation tasks are enumerated* — the spec's =Implementation phases= section is lifted into a drop-in =todo.org= block (one entry per phase plus a test-surface entry), or, if the spec has no phase decomposition, that gap is raised as a finding. @@ -50,6 +50,11 @@ Run it *early* — design review exists to catch viability problems and costly m Before Phase 1, verify the file under review ends with =-spec.org=. Every design, decision, or planning document under a project's =docs/= directory carries that suffix as its identifier. The =.org= extension alone is not enough because =docs/= holds non-spec org files too (tutorials, frozen inventories, reference material). +*Location expectation (docs-lifecycle convention).* Formal specs live in =docs/specs/=. Whether that's enforced depends on whether the project has run its one-time =spec-sort= retrofit: + +- =:LAST_SPEC_SORT:= present in =.ai/notes.org= Workflow State → the project has sorted; a =-spec.org= file outside =docs/specs/= fails this precondition. Surface it: "this spec sits outside docs/specs/ — move it (and update inbound links) before review." +- Marker absent → legacy locations (=docs/= root, =docs/design/=) stay reviewable; add one nudge line to the review output ("this project's docs pile has never been spec-sorted — say 'run spec-sort' to sort it") and proceed. No legacy spec is ever unreviewable during the transition. + If the file does not end with =-spec.org=, stop immediately and surface the mismatch: #+begin_example @@ -93,19 +98,19 @@ Mark the spec implementation-ready only if *all* of these hold: - The plan can be phased without shipping broken intermediate states, and phases are small enough to reach a clean stopping point in one focused work session. - External API assumptions are verified or explicitly listed as prerequisites. -If all true → tell the user it's ready and stop unless they ask for more. If any false → continue and write the review file. A "ready" at this phase is provisional; confirm it at Phase 3 after the code read. +If all true → tell the user it's ready and stop unless they ask for more. If any false → continue and record findings (Phase 5). A "ready" at this phase is provisional; confirm it at Phase 3 after the code read. ** Phase 2: Required reading order Never review a spec in isolation. 1. *Read the existing implementation first.* The code paths the spec would touch: public commands and entry points, internal helpers/boundaries, current data representation, persistence/write-back, async/sync, caching, error handling, existing tests, naming/style. Capture current-state facts with function names and file paths. Don't recommend designs that ignore how the package works today. -2. *Read related specs and task tracking.* Companion specs, relevant =todo.org= tasks, README/testing docs, prior review files. Record which tasks the spec absorbs, which stay separate, which decisions are already made, which are still open. +2. *Read related specs and task tracking.* Companion specs, relevant =todo.org= tasks, README/testing docs, prior reviews (in each spec's =* Review findings= and =Review and iteration history=). Record which tasks the spec absorbs, which stay separate, which decisions are already made, which are still open. 3. *Read the target spec end to end — twice.* First for its problem/behavior/phases/assumptions; second looking only for gaps. The second read asks: "What would an implementer still have to invent?" ** Phase 3: Re-run the gate (authoritative) -After reading code and spec, re-run the Phase 1 gate — this is the pass that counts, because now you can actually judge the items that needed the code: architecture fit, API verification, integration points. If now ready, don't manufacture churn. If not, write the review file. +After reading code and spec, re-run the Phase 1 gate — this is the pass that counts, because now you can actually judge the items that needed the code: architecture fit, API verification, integration points. If now ready, don't manufacture churn. If not, record findings (Phase 5). ** Phase 4: Evaluate across dimensions @@ -128,6 +133,8 @@ Work the spec against these. Each is a source of concrete findings, not a box to - *Performance & scale.* Expected counts (issues/comments/labels/teams/projects/views)? Server-side filtering where possible? Bounded, visible pagination? Cached name→ID lookups? Sync calls in the command path acceptable? Could a save hook or whole-file scan make N network calls? Rendering linear? Full-file rewrites avoided? Long-running operations async/cancellable/observable? Is concurrency/queueing/backpressure defined? Are high-output process filters throttled and cheap? Is progress/ETA exposed only when defensible, and are hung/stalled operations detectable and killable? Identify UI freezes, repeated network calls, unbounded pagination — without premature optimization. - *Security & privacy.* API keys safe? Debug logs leaking secrets or private issue text? Confirmations before mutating shared workspace objects? Personal vs shared distinguished? Local files holding sensitive descriptions/comments? Anything to redact from messages/logs? Any work-tracker integration may handle private company data. - *UX & accessibility.* Discoverable commands? Recoverable mistakes? Prompts ordered to the task? Safe, useful defaults? Informative-not-noisy status messages? Does the UI avoid implying unsupported actions are supported? Match the upstream product's permissions/concepts? Are customizations named in user language, with clear defaults and docstrings? For Emacs packages, command names, completion candidates, buffer layout, defcustom names, and message wording *are* the UX. +- *Operational-panel UI traps.* Applies when the spec covers a user-facing panel, dialog, or control surface; skip otherwise. Lists that mix saved, current, and generated items must name each item's source. Refresh or scan actions must not gate data that could be shown immediately. Add-forms must not ask the user to retype values the system already discovered. Destructive confirmations read in future tense before the action and verified-result tense after it. Diagnostics, performance, logging, and repair affordances are reviewed as one coherent flow before extra pages or buttons are added. A popup launched from a bar, tray, or tool surface should visually belong to that launcher. (Promoted from archsetup's Waybar network-panel review, 2026-06-30.) +- *Prototype process for non-trivial UI.* Applies when the deliverable is a real UI (a panel, a multi-control surface, an interacting visual layout — not a single dialog or CLI flag); skip otherwise. Verify the =claude-rules/ui-prototyping.md= process ran: category research is cited in Goals/Design, the final prototype is linked in the design section, a =Prototype iterations= subsection under the status heading lists every pass, and each UI design decision is backed by a prototype it was seen working in rather than asserted on the page. A non-trivial-UI spec with decisions but no prototype evidence is a =:blocking:= finding. - *Test strategy and coverage.* Characterization tests before behavior changes? Pure functions to unit-test? API responses needing fixtures? Command flows needing stubs? Regression tests for prior bugs? Boundary/error cases? What's covered elsewhere and shouldn't be re-tested? Which existing tests must change? How is coverage generated, summarized, and used to find untested/refactor-worthy code? Prefer tests that lock contracts: representation shape, query compilation, sync no-op, conflict refusal, pagination, dirty-buffer protection, log redaction, and long-running/slow-operation behavior via fakes rather than flaky live dependencies. - *Observability & operations.* How does a user see what the package is doing? Progress messages for long ops? Useful, safe debug logging? Are logs structured enough to isolate issues from a bug report? Are commands provided to inspect/clear caches, test connectivity, diagnose backends/tools, copy redacted debug info, or reproduce command invocations? How are terminal states discovered: completion, failure, partial success, stalled/hung, cancelled, cleanup-unverified, and "needs user action"? Does the product notify only when useful, avoid noisy success spam, and keep non-success states visible until acknowledged? For generated org files, headers should often carry source, filter/view name, refresh time, count, truncation state. - *Comparable-product sentiment.* When there are obvious adjacent products, research what users love and hate about them from official docs plus current community reports. Do not cargo-cult their feature set; translate findings into the spec's scope. For each loved behavior, say whether the spec provides it, intentionally omits it, or defers it. For each hated behavior, say whether the spec avoids, resolves, inherits, or accepts it. @@ -136,66 +143,41 @@ Work the spec against these. Each is a source of concrete findings, not a box to - *Development tooling.* Does the repo give contributors obvious commands for setup, fast tests, specific tests, compile, lint, coverage, cleanup, slow/manual tests, and release checks? Are optional/live tests gated by explicit environment variables? Is the Makefile/script surface consistent with sibling projects? - *Small enhancement radar.* Are there low-complexity, high-value affordances already provided by the platform that should be surfaced now or explicitly deferred? Examples: archive/compress commands in file managers, built-in history, previews, diagnostics, or doctor commands. Keep the hot path simple; capture the opportunity rather than accidentally losing it. -** Phase 5: Write the review file +** Phase 5: Record findings in the spec -Use this structure for =<spec-basename>-review.org= unless the spec calls for something different: +Findings live in the spec, not a sibling file. Add (or append to) a =* Review findings= section near the spec's =* Decisions= section, with a =[/]= cookie on the heading. Each finding is a =** TODO= task: the heading is the smallest noun phrase naming the gap; the body names current behavior, the risk, and the recommended change. Tag a blocking (high-priority) finding =:blocking:= — it holds the rubric at =Not ready= until dispositioned; leave non-blocking findings untagged. Findings accumulate across review rounds the way decisions do, and the responder completes each one in place (Phase 4 of spec-response), so the section becomes the full review/response trail. #+begin_src org -,#+TITLE: Review: <Spec Title> -,#+AUTHOR: <reviewer> -,#+DATE: <date> -,#+STARTUP: showall - -,* Scope reviewed -What code, tests, docs, and specs you read. - -,* Implementation-readiness -Whether the spec is ready. If not, summarize the blockers. - -,* Overall assessment -The short senior-engineering read: what's right, what's risky, what must be clarified. - -,* High-priority findings -Concrete headings. Each: why it matters and what to change. - -,* Medium-priority findings -Important improvements that shouldn't block all progress. - -,* UX observations -,* Architecture observations -,* Robustness and performance observations -,* Test strategy recommendations -Specific test cases, not generic "add tests". -,* Documentation and tooling recommendations -README/user/developer docs, Makefile/package scripts, coverage, debug tools, and customization surface. - -,* Suggested spec edits -Concrete edits to make the spec implementation-ready. - -,* Agreed decisions -Answers reached during review. Omit if none. - -,* Open questions -Only questions that truly block or materially affect implementation. - -,* vNext candidates -Deferred features to capture in task tracking. +,* Review findings [/] +,** TODO Comment edit-back is undefined :blocking: +The spec says fetched comments render as subheadings but doesn't define whether +editing one syncs back. Linear only lets users edit their own comments. V1 should +treat fetched comments as remote-owned display content and support only adding new +comments; editing own comments can be vNext. (blocking) +,** TODO Empty result and fetch error render identically +A failed fetch and a successful-but-empty fetch produce the same buffer, so the +user can't tell "no issues" from "the query broke." Define a distinct empty-state +message. (non-blocking) #+end_src +Where the old review-file sub-sections go now: the scope-reviewed and overall-assessment narrative goes in the =Review and iteration history= entry (Phase 6); suggested spec edits are the recommended-change line in each finding's body; agreed decisions flip the spec's own =* Decisions= tasks; open questions are =:blocking:= findings or open decisions; vNext candidates are logged to =todo.org= as =[#D]= (Phase 6). The Phase 4 review dimensions are where findings come *from* — not headings to reproduce in the spec. + ** Phase 6: Assign the rubric and update tracking Assign one label consistently: -- =Ready= — no blocking open questions; implementation can start. Requires no decision in the spec's =* Decisions= section to still be =TODO= (the =[/]= cookie reads complete; =SUPERSEDED= and =CANCELLED= count as resolved) — a decision still =TODO= holds the rubric at =Not ready=, or =Ready with caveats= if the author consciously accepts and records the risk. +- =Ready= — no blocking open questions; implementation can start. Requires both cookies complete: no decision in =* Decisions= and no =:blocking:= finding in =* Review findings= still =TODO= (the =[/]= cookies read complete; =SUPERSEDED=/=CANCELLED= and a completed or rejected finding count as resolved) — a still-=TODO= decision or =:blocking:= finding holds the rubric at =Not ready=, or =Ready with caveats= if the author consciously accepts and records the risk. A non-blocking finding left =TODO= is author's discretion and does not hold the rubric. - =Ready with caveats= — can start if the caveats are accepted and tracked. - =Not ready= — blocking ambiguity / missing decisions would force implementers to invent product behavior. - =Needs research= — external API/library/platform assumptions must be verified first. The most useful reviews move a spec from =Not ready= to =Ready with caveats= or =Ready= once decisions are captured. +*The =Ready= verdict flips the spec's lifecycle status.* spec-review owns the =DRAFT= → =READY= transition (docs-lifecycle convention): on assigning =Ready= (or =Ready with caveats= the author accepts), update the spec's top-level status heading keyword to =READY=, add a dated history line under it naming the review that passed, and set the Metadata =Status= mirror to =ready= — three lines, one file. Any other rubric label leaves the keyword where it stands (a re-review that finds new blockers on a =READY= spec demotes it back to =DRAFT= the same three-line way, with the reason in the history line). + Finding severity maps to blocking power: *high-priority findings block =Ready=* — they hold the rubric at =Not ready= (or =Ready with caveats= if the author accepts and tracks them) until dispositioned; *medium-priority findings are the author's discretion* and don't block. State the blocking status on each finding so the author running spec-response knows which ones gate the rubric. -Then update the spec's review history. Specs should carry a bottom section named =Review and iteration history= (or the nearest existing equivalent) that tracks each material author/reviewer pass. Add a concise entry for this review even when the spec is ready and no review file is written. +Then update the spec's review history. Specs should carry a bottom section named =Review and iteration history= (or the nearest existing equivalent) that tracks each material author/reviewer pass. Add a concise entry for this review even when the spec is ready and no findings are recorded. Each entry is an org subheading with a compound id followed by three body fields. @@ -207,27 +189,11 @@ Body fields: - *What changed or was recommended:* high-signal summary, not a duplicate of the whole review. - *Why:* the decision pressure or rationale that caused the contribution. -- *Artifacts:* links to the review file, response/disposition section, commits, task IDs, or source checks when useful. +- *Artifacts:* links to the relevant findings, commits, task IDs, or source checks when useful. If the spec has no such section, add it at the bottom. Keep the history short and cumulative; it is provenance for future readers, not a session transcript. -*Emit implementation tasks (drop-in for =todo.org=).* Read the spec's =Implementation phases= section and turn it into a paste-ready block in the review file, under a heading =Implementation tasks (drop-in for todo.org)=. One =** TODO= entry per phase, plus a final entry for the test surface. The point: the handoff to whoever implements is one paste, not a re-read of the spec, and a spec that can't be decomposed into phases fails this step, surfacing a shape problem before =Ready=. - -Per-phase entry, following =todo-format.md= (terse heading names the phase; body holds the one-line deliverable plus a pointer back to the spec; tags on the heading): - -#+begin_example -** TODO [#B] <phase name — smallest noun phrase> :feature: -<what this phase delivers, one line>. Spec: [[file:<spec path>]] (Implementation phases, phase N). -#+end_example - -Final test-surface entry, mirroring the spec's =Acceptance criteria= when present: - -#+begin_example -** TODO [#B] <feature> — test surface :test: -Unit: <...>. Integration: <...>. E2e / manual-verify: <acceptance criteria as checkable items>. Spec: [[file:<spec path>]] (Acceptance criteria). -#+end_example - -Priority and tags follow the deferred-work rule below. Emit the block in the review file; the author pastes it into =todo.org= during spec-response, or you log it directly when you're also closing the loop. If the spec has no =Implementation phases= section, don't invent one — that absence is the finding, and the step becomes the prompt to ask the author to add a phase decomposition before the spec can be =Ready=. +*Check the spec decomposes into phases.* A =Ready= spec needs an =Implementation phases= section an implementer can turn into one task per phase plus a test surface. Confirm it's present and decomposable — each phase small enough to reach a clean stopping point in one focused session, with no broken intermediate states. If it's missing or can't be phased, file that as a =:blocking:= finding; don't invent the phases. The phase-to-task breakdown itself is spec-response's job (its Phase 6 reads =Implementation phases= directly once the author confirms =Ready=); the reviewer only verifies the section exists and is sound. Then log deferred work to =todo.org=: v1 implementation = =[#B]= (unless urgent or speculative); vNext/someday = =[#D]=. Tag =:feature:= / =:bug:= / =:refactor:= / =:test:= / =:quick:= / =:solo:= only when accurate. Don't leave important deferred decisions only in chat. @@ -256,8 +222,11 @@ Every material comment should be tagged by force: blocking, should-fix, or optio ** Make feedback author-usable Review comments should be specific, neutral, and actionable: quote or name the spec behavior, explain the risk, recommend the smallest concrete change, and say how the author can verify the fix. Avoid personal language, rhetorical questions, vague "this needs work" comments, and comments that require the author to infer the desired edit. +** Keep review and response roles explicit +If the user asks for review plus "enhance the spec" in the same turn, produce the findings first. Make only low-risk provenance and tracking edits unless the user clearly wants the reviewer to respond too. Don't silently resolve product decisions on the author's behalf — a proposed default belongs in a finding until it's accepted, modified, or rejected. + ** Preserve iteration provenance -Future reviewers and implementers need to know not just the current decision, but how the spec got there: how many review/response loops happened, who contributed, what they changed or recommended, and why. Keep that record in the spec itself under =Review and iteration history= so the trail survives deleted review files, chat loss, and agent handoffs. +Future reviewers and implementers need to know not just the current decision, but how the spec got there: how many review/response loops happened, who contributed, what they changed or recommended, and why. Keep that record in the spec itself under =Review and iteration history= so the trail survives chat loss and agent handoffs. ** Be strict about ownership Especially for org-mode features: a user treats visible text as editable unless the representation says otherwise. Make generated-vs-editable explicit. @@ -265,6 +234,9 @@ Especially for org-mode features: a user treats visible text as editable unless ** Never depend on an unverified API shape If the spec assumes fields/mutations/enums, they're verified against current schema/docs/live responses, or listed as a research prerequisite. =Needs research= is a real, useful verdict. +** Source external-dependency checks in the finding +When a finding turns on a current external-dependency fact (release version, API capability, platform behavior, package availability, hosted-service terms), cite the checked source in the finding body. Stale dependency assumptions are common, and the next reviewer needs to tell "verified this pass" from "remembered from prior context." + ** Favor small pure cores and thin IO layers Push findings toward separable, unit-testable pure functions surrounded by thin command/transport layers. @@ -354,3 +326,8 @@ Sources: - *What:* Two refinements to the same-day decisions convention after Craig's review: the gate item and =Ready= rubric now read "no decision is still =TODO=" with =SUPERSEDED= and =CANCELLED= counting as resolved (spec-create's template defines them as done-class keywords via a =#+TODO:= header), and a spec still on the retired =State:= field model explicitly fails the gate item until converted — closing the vacuous-pass hole on old specs. - *Why:* Review of the freshly-landed convention flagged that TODO/DONE alone lost the old model's superseded state and that the gate as written would silently pass a spec with no decision tasks at all. Craig chose the two done-class keywords and the auto-added =#+TODO:= header (the in-file header is what makes custom keywords portable). - *Artifacts:* Paired spec-create.org edits (keyword scheme + template header) in the same commit. + +** 2026-06-21 Sun @ 23:16:06 -0400 — Claude Code (rulesets) — responder +- *What:* Moved findings from a sibling =<spec>-review.org= file into the spec itself. Findings are now =** TODO= tasks under a =* Review findings= section with a =[/]= cookie, mirroring =* Decisions=; =:blocking:= marks high-priority. Phase 5 records findings in the spec instead of writing a review file; the Phase 6 =Ready= rubric gates on both the decisions and the findings cookie; the implementation-task drop-in (which lived in the review file) is gone, leaving the reviewer to verify the spec decomposes into phases and spec-response to build the breakdown. Also added two reviewer-practice principles harvested from a home spec-review: keep review and response roles explicit, and source external-dependency checks in the finding. +- *Why:* The delete-the-review-file convention left the iteration-history =Artifacts= line dangling and dropped the verbatim review; keeping the file instead collided with spec-response's file discovery and its "no review files remain" done-condition. Craig's call: incorporate the review into the document, reusing the decisions machinery so the readiness signal is a cookie, not a file's presence or absence. The role-explicit and source-checking practices came in from the home finance-report spec via inbox handoffs. +- *Artifacts:* Paired spec-response.org edits in the same commit. Inbox handoffs =2026-06-20-2339-from-home-spec-response-readiness-gate-proposal.org=, =2026-06-21-0156-from-home-companion-to-tonight-s-spec-response.org=, and the home-edited =2026-06-21-0156-from-home-spec-review.org=. |