diff options
Diffstat (limited to 'docs/specs/2026-07-01-docs-lifecycle-spec.org')
| -rw-r--r-- | docs/specs/2026-07-01-docs-lifecycle-spec.org | 214 |
1 files changed, 214 insertions, 0 deletions
diff --git a/docs/specs/2026-07-01-docs-lifecycle-spec.org b/docs/specs/2026-07-01-docs-lifecycle-spec.org new file mode 100644 index 0000000..081abdd --- /dev/null +++ b/docs/specs/2026-07-01-docs-lifecycle-spec.org @@ -0,0 +1,214 @@ +#+TITLE: Docs Lifecycle — Spec +#+AUTHOR: Craig Jennings & Claude +#+DATE: 2026-07-01 +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* DRAFT Docs lifecycle +:PROPERTIES: +:ID: 80b0787b-4a60-4c82-8a16-b383d3e3c8f2 +:END: +- 2026-07-01 Wed @ 22:13:00 -0400 — drafted from the five decisions settled 2026-06-28 (todo.org "Spec storage location + lifecycle-status convention"). + +* Metadata +| Status | draft | +|----------+------------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+------------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+------------------------------------------------------------------| +| Date | 2026-07-01 | +|----------+------------------------------------------------------------------| +| Related | [[file:../design/2026-06-15-spec-storage-lifecycle-proposal.org][source proposal]]; todo.org "Spec storage location + | +| | lifecycle-status convention" | +|----------+------------------------------------------------------------------| + +* Summary + +Formal specs and working notes currently share one directory per project, and a spec's lifecycle state (drafted, in progress, shipped, dead) is invisible without opening the file. This spec adopts two coupled conventions — a location split (=docs/specs/= for formal specs, =docs/design/= for notes) and an authoritative in-file status carried by an org TODO keyword on a top-level status heading — plus =org-id= links for rename-safety, a general =docs-lifecycle= rule capturing the shape, and a one-time confirmed retrofit that sorts every project's existing pile. + +* Problem / Context + +.emacs.d triaged ~28 design docs and had to run a four-agent sweep reading every spec against the code to reconstruct which had shipped (6 implemented, 8 in progress, 12 not started, 1 superseded). Nothing in the filename, location, or file records the state, so the answer to "what's open?" degrades into "open every file and infer." rulesets has the same shape: 41 files in =docs/design/= of which only 3 carry a formal spec spine, plus two =-spec.org= files misfiled at the =docs/= root. The cost compounds with every doc added, and every project inherits the problem through the shared spec-create workflow. + +Two forces beyond triage cost: + +- *Links are load-bearing.* =todo.org= tasks, session archives, and sibling docs link specs by =file:= path. Any convention that renames or moves files on every status change (the filename-suffix approach) breaks those links repeatedly across a cross-linked, template-synced doc set. +- *The convention is worthless if legacy docs stay misfiled* (Craig, 2026-06-28). Template sync distributes rules and workflows but cannot perform a one-time per-project migration, so the design must include a reach mechanism that gets each project's existing pile sorted once. + +* Goals and Non-Goals + +** Goals +- A directory listing answers "which docs are specs, and what state is each in" without opening files. +- Status transitions cost one keyword edit — no rename, no link surgery. +- Cross-doc spec links survive moves and renames. +- The shape is captured once as a general rule (=docs-lifecycle=) so future artifact collections (brainstorm piles, recording queues) can reuse it. +- Every existing project's =docs/design/= pile gets sorted exactly once, with human confirmation on each classification. + +** Non-Goals +- No automation of status flips — the keyword is edited by whoever changes the state (spec-create, spec-review, spec-response, or a human), not by a watcher. +- No retroactive rewriting of session archives or git history that reference old paths; only live inbound links (=todo.org=, =notes.org=, docs) are updated by the retrofit. +- No new tracking database or index file — the files are the index. + +** Scope tiers +- v1: the location split, the status-heading convention, the org-id link standard, the =docs-lifecycle= rule, spec-create/spec-review/spec-response updates, the retrofit helper + startup nudge, and the rulesets pilot. +- Out of scope: applying the lifecycle shape to non-doc collections (the rule documents the pattern; adopting it elsewhere is per-collection work). +- vNext: an org-agenda custom view over =docs/specs/*.org= keyed on the status keywords (nice-to-have once the keywords exist; log to todo.org). + +* Design + +** The location split + +- =docs/specs/= — formal specs only. A *spec* is a doc proposing a buildable change that carries a =Decisions= section and =Implementation phases= (the spec-create spine). Filenames keep the existing =YYYY-MM-DD-<topic>-spec.org= shape — the =-spec.org= suffix stays because spec-review's Phase 0 precondition keys on it; only the *status* suffixes from the original proposal are dropped. +- =docs/design/= — everything else: brainstorms, inventories, proposals, research notes, frozen source material. Review findings live inside the spec they review (current spec-review behavior), so standalone review files are legacy notes and stay in =docs/design/=. + +** The status heading (the authoritative record) + +Each spec's first element after the file header is a single top-level *status heading* carrying the org TODO keyword: + +#+begin_example +,#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +,* DOING <spec short name> +:PROPERTIES: +:ID: <uuid> +:END: +- <dated one-line history entries, newest first> +#+end_example + +- *The keyword is authoritative.* The Metadata table's =Status= field mirrors it in lowercase for readers already in the table; on disagreement the heading wins. +- *Vocabulary:* =DRAFT= (being written) → =READY= (review passed, buildable) → =DOING= (implementation in progress) → =IMPLEMENTED= / =SUPERSEDED= / =CANCELLED= (terminal). This supersedes the old =TODO | DONE SUPERSEDED CANCELLED= header line; the retrofit updates it. +- *Glanceability without opening files:* one grep gives the full board — + + #+begin_src sh + rg -H '^\* (DRAFT|READY|DOING|IMPLEMENTED|SUPERSEDED|CANCELLED) ' docs/specs/ + #+end_src + + and because the keyword sits on a real org heading, an org-agenda view over =docs/specs/= works for free (the vNext item). +- *The heading body is the dated status history* — one line per transition (=YYYY-MM-DD Day @ HH:MM:SS -ZZZZ — <what changed, by whom>=), the record a filename could never carry. +- Why a dedicated status heading rather than restructuring each spec under one top-level heading: demoting every section in every existing spec is a large, link-hostile rewrite; a prepended heading is additive, retrofittable by script, and leaves the familiar flat section layout untouched. + +** Rename-safe links + +The status heading carries an =:ID:= UUID. Cross-doc references to a spec use =[[id:<uuid>]]= rather than =file:= paths, so the one move the retrofit performs (and any future move) can't orphan them. =file:= links remain fine for intra-doc anchors and for notes that never move. The KB's existing id-resolution recipe applies: =rg ':ID:[[:space:]]+<uuid>' docs/=. + +** The =docs-lifecycle= rule (the generalization) + +A new =claude-rules/docs-lifecycle.md= captures the reusable shape, with spec-create as the first instance: + +1. Separate formal artifacts from working notes by location. +2. Lifecycle state lives *in* the artifact, on a scannable, greppable carrier (an org keyword heading), with a dated history. +3. Links use rename-safe identifiers. +4. A growing collection earns this treatment when "which of these are live?" starts requiring a file-by-file read. + +** The retrofit (reach mechanism for existing piles) + +A synced helper, =.ai/scripts/spec-sort=, run once per project: + +1. *Classify* each =docs/design/*.org= (and stray =docs/*-spec.org=) by heuristic: a doc with a =Decisions= section plus =Implementation phases= or a =Metadata= table is a spec candidate; everything else is a note. The heuristic *proposes*; a human confirms every move (classification is a judgment call — Craig, 2026-06-28). +2. *Move* confirmed specs to =docs/specs/=, prepend the status heading (keyword proposed from the doc's current Status field or review history; confirmed by the human), assign an =:ID:=, and update the =#+TODO:= header line. +3. *Relink* inbound =file:= references (=todo.org=, =notes.org=, =.ai/=, other docs) to the new path — and to =id:= form where the target is a spec. +4. *Stamp* =:LAST_SPEC_SORT: YYYY-MM-DD= in =notes.org= Workflow State. + +A startup nudge (one line in Phase C) fires in projects that have a =docs/design/= but no =:LAST_SPEC_SORT:= marker, so each project runs the sort exactly once without template sync having to perform it. + +* Alternatives Considered + +** Filename status suffix (=-spec-doing.org=, =-spec-implemented.org=) +- Good, because the state is visible in a bare =ls= with no tooling. +- Bad, because every transition renames a file in a cross-linked, template-synced doc set — each rename is link surgery or a broken link, and the churn lands in git history and inbound =todo.org= links. +- Neutral, because the ls-visibility it buys is matched by the one-line =rg= over status headings. +- Rejected 2026-06-28 (Craig chose org-keyword over his earlier filename-suffix lean). + +** Status field in the Metadata table only (no keyword) +- Good, because the field already exists and needs no new structure. +- Bad, because a table cell is neither org-agenda-scannable nor reliably greppable across format drift, and it carries no dated history. +- Neutral, because the field stays anyway — as the in-table mirror. + +** Relink-helper instead of org-id (keep =file:= links, fix them on every move) +- Good, because readers see plain paths. +- Bad, because it makes every future move a tooling event, and one missed run silently breaks links — the failure mode is invisible until someone clicks. +- Neutral, because the retrofit needs relink logic once regardless; org-id just makes it a one-time need. + +* Decisions [5/5] + +All five were settled with Craig on 2026-06-28 (recorded in todo.org; migrated here per that note). + +** DONE Location split — adopt +- Context: specs and notes share one directory; telling them apart requires opening files. +- Decision: =docs/specs/= for formal specs (Decisions + phases spine); =docs/design/= for notes. Documented in spec-create and the docs-lifecycle rule. +- Consequences: easier — a listing answers "what's formal"; harder — one-time migration and link updates (the retrofit). + +** DONE Status mechanism — org keyword authoritative, no filename suffix +- Context: filename suffix vs org keyword; suffix wins =ls= visibility, keyword wins link stability and zero-rename transitions. +- Decision: the org TODO keyword on the spec's top status heading is authoritative, mirrored by the Metadata =Status= field. No status suffixes in filenames. +- Consequences: easier — a transition is one keyword edit and links never break; harder — glanceability needs the one-line =rg= (or the vNext agenda view) instead of bare =ls=. + +** DONE Link safety — org-id for cross-doc spec links +- Context: both the migration move and any future rename break =file:= links. +- Decision: specs carry =:ID:= UUIDs on the status heading; cross-doc references use =[[id:...]]=. +- Consequences: easier — moves are free; harder — following a link outside org needs the =rg ':ID:'= lookup. + +** DONE Generalize as a =docs-lifecycle= rule +- Context: the shape (in-artifact lifecycle state, formal-vs-notes split, rename-safe links) recurs for any processed-document collection. +- Decision: capture it in =claude-rules/docs-lifecycle.md= with spec-create as the first instance. +- Consequences: easier — the next collection reuses a decided pattern; harder — the rule must stay honest as the spec instance evolves. + +** DONE Retrofit existing files across ALL projects +- Context: template sync distributes conventions but cannot perform a per-project one-time migration; legacy piles would stay misfiled forever. +- Decision: ship a confirmed classify-move-relink helper (=spec-sort=) plus a startup nudge gated on =:LAST_SPEC_SORT:=; the helper proposes, a human confirms. Pilot on rulesets first. +- Consequences: easier — every project converges without manual archaeology; harder — the helper needs real relink logic and tests, and classification stays a judgment call. + +* Implementation phases + +** Phase 1 — Rule + template updates +Write =claude-rules/docs-lifecycle.md=. Update spec-create (emit into =docs/specs/=, new =#+TODO:= vocabulary, status heading with =:ID:= in the template, transition mechanics), spec-review (precondition path =docs/specs/=; flipping =DRAFT= → =READY= on a passing review updates the heading + history line), and spec-response (status flips it owns). Tree stays working: new specs land in the new shape; old specs are untouched until Phase 3. + +** Phase 2 — The =spec-sort= helper +Build =.ai/scripts/spec-sort= (classify → confirm → move + prepend status heading + assign =:ID:= → relink inbound references → stamp =:LAST_SPEC_SORT:=), with bats coverage for classification, moving, relinking, idempotence, and the confirm gate. Tree stays working: the script is callable but nothing invokes it yet. + +** Phase 3 — Pilot on rulesets +Run =spec-sort= against rulesets' own =docs/= (41 design files, 3 spec-spine candidates, 2 stray root specs). Fix what the pilot surfaces before any other project runs it. Tree stays working: moves are confirmed one by one, links updated in the same pass. + +** Phase 4 — Startup nudge + broadcast +Add the Phase C nudge (fires when =docs/design/= exists and =:LAST_SPEC_SORT:= is absent). Send .emacs.d a note that the convention is live and its ~28-doc pile is ready to sort. Tree stays working: the nudge is one read-only line per session until acted on. + +* Acceptance criteria +- [ ] =rg '^\* (DRAFT|READY|DOING|IMPLEMENTED|SUPERSEDED|CANCELLED) ' docs/specs/= lists every rulesets spec with its state, and the answer matches reality. +- [ ] A status transition on a spec changes exactly one keyword plus one history line — no rename, no link edits. +- [ ] Every doc remaining in rulesets =docs/design/= is a note (no Decisions + phases spine); both stray =docs/= root specs are re-homed. +- [ ] All inbound =todo.org= / =notes.org= / doc links to moved specs resolve after the pilot. +- [ ] spec-create emits new specs into =docs/specs/= in the new shape; spec-review refuses a spec outside it. +- [ ] A project with an unsorted =docs/design/= gets the startup nudge; one confirmed =spec-sort= run clears it via =:LAST_SPEC_SORT:=. + +* Readiness dimensions +- Data model & ownership: the spec file owns its state; the Metadata mirror is display-only. No external index to drift. +- Errors, empty states & failure: =spec-sort= on a project with no =docs/= is a silent no-op; an ambiguous classification is surfaced, never auto-moved; a relink pass that finds zero inbound links is normal. +- Security & privacy: N/A because the docs are already in-repo; no new exposure surface. +- Observability: the status grep is the dashboard; =spec-sort= prints every proposed move and every rewritten link. +- Performance & scale: N/A because collections are tens of files; everything is one-shot or grep-speed. +- Reuse & lost opportunities: reuses org TODO keywords, org-id, the existing scheme-header pattern of declared vocabularies, and spec-review's in-file findings convention. +- Architecture fit & weak points: weak point is the classification heuristic — mitigated by the confirm gate. The status heading is additive, so old readers of spec files see one extra heading and nothing breaks. +- Config surface: none new — one marker line (=:LAST_SPEC_SORT:=) in the existing Workflow State section. +- Documentation plan: the docs-lifecycle rule is the documentation; spec-create's template is the worked example. +- Dev tooling: =spec-sort= ships with bats tests under the existing glob-discovered suite. +- Rollout, compatibility & rollback: additive per project, one project at a time, rulesets first. Rollback of a sort is =git revert= of the pilot commit (moves + relinks are one commit). +- External APIs & deps: N/A — plain files, =rg=, =uuidgen=. + +* Risks, Rabbit Holes, and Drawbacks +- *Relink misses an inbound link shape* (org radio links, bare paths in scripts). Dodge: the pilot greps for the old path after moving and fails loudly on any residue. +- *Heuristic over-classifies notes as specs.* Dodge: the confirm gate is mandatory; the helper never moves unconfirmed. +- *Keyword vocabulary drift* between this spec, the rule, and spec-create's template. Dodge: the rule names the vocabulary once and the others link it. + +* Testing / Verification / Rollout +bats for =spec-sort= (classification, move, relink, idempotence, confirm gate, marker stamp). The pilot run on rulesets is the live verification; the post-move residue grep is the acceptance check. Rollout is per-project via the startup nudge, each run human-confirmed. + +* References / Appendix +- Source proposal: [[file:../design/2026-06-15-spec-storage-lifecycle-proposal.org]] (.emacs.d handoff, 2026-06-15). +- Decisions record: todo.org "Spec storage location + lifecycle-status convention" (settled 2026-06-28). +- This file is the convention's first resident: it lives in =docs/specs/=, carries the status heading + =:ID:=, and drops the status filename suffix. + +* Review and iteration history +** 2026-07-01 Wed @ 22:13:00 -0400 — Claude — author +- What: initial draft, written from the five pre-ratified decisions. +- Why: the queued-specs half of the 2026-06-30 session goal; decisions were settled 2026-06-28 and needed migration into a buildable spec. +- Artifacts: todo.org task "Spec storage location + lifecycle-status convention"; source proposal above. |