1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
|
#+TITLE: Docs Lifecycle — Spec
#+AUTHOR: Craig Jennings & Claude
#+DATE: 2026-07-01
#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED
* DRAFT Docs lifecycle
:PROPERTIES:
:ID: 80b0787b-4a60-4c82-8a16-b383d3e3c8f2
:END:
- 2026-07-01 Wed @ 22:13:00 -0400 — drafted from the five decisions settled 2026-06-28 (todo.org "Spec storage location + lifecycle-status convention").
* Metadata
| Status | draft |
|----------+------------------------------------------------------------------|
| Owner | Craig Jennings |
|----------+------------------------------------------------------------------|
| Reviewer | Craig Jennings |
|----------+------------------------------------------------------------------|
| Date | 2026-07-01 |
|----------+------------------------------------------------------------------|
| Related | [[file:../design/2026-06-15-spec-storage-lifecycle-proposal.org][source proposal]]; todo.org "Spec storage location + |
| | lifecycle-status convention" |
|----------+------------------------------------------------------------------|
* Summary
Formal specs and working notes currently share one directory per project, and a spec's lifecycle state (drafted, in progress, shipped, dead) is invisible without opening the file. This spec adopts two coupled conventions — a location split (=docs/specs/= for formal specs, =docs/design/= for notes) and an authoritative in-file status carried by an org TODO keyword on a top-level status heading — plus =org-id= links for rename-safety, a general =docs-lifecycle= rule capturing the shape, and a one-time confirmed retrofit that sorts every project's existing pile.
* Problem / Context
.emacs.d triaged ~28 design docs and had to run a four-agent sweep reading every spec against the code to reconstruct which had shipped (6 implemented, 8 in progress, 12 not started, 1 superseded). Nothing in the filename, location, or file records the state, so the answer to "what's open?" degrades into "open every file and infer." rulesets has the same shape: 41 files in =docs/design/= of which only 3 carry a formal spec spine, plus two =-spec.org= files misfiled at the =docs/= root. The cost compounds with every doc added, and every project inherits the problem through the shared spec-create workflow.
Two forces beyond triage cost:
- *Links are load-bearing.* =todo.org= tasks, session archives, and sibling docs link specs by =file:= path. Any convention that renames or moves files on every status change (the filename-suffix approach) breaks those links repeatedly across a cross-linked, template-synced doc set.
- *The convention is worthless if legacy docs stay misfiled* (Craig, 2026-06-28). Template sync distributes rules and workflows but cannot perform a one-time per-project migration, so the design must include a reach mechanism that gets each project's existing pile sorted once.
* Goals and Non-Goals
** Goals
- A directory listing answers "which docs are specs, and what state is each in" without opening files.
- Status transitions cost one keyword edit — no rename, no link surgery.
- Cross-doc spec links survive moves and renames.
- The shape is captured once as a general rule (=docs-lifecycle=) so future artifact collections (brainstorm piles, recording queues) can reuse it.
- Every existing project's =docs/design/= pile gets sorted exactly once, with human confirmation on each classification.
** Non-Goals
- No automation of status flips — the keyword is edited by whoever changes the state (spec-create, spec-review, spec-response, or a human), not by a watcher.
- No retroactive rewriting of session archives or git history that reference old paths; only live inbound links (=todo.org=, =notes.org=, docs) are updated by the retrofit.
- No new tracking database or index file — the files are the index.
** Scope tiers
- v1: the location split, the status-heading convention, the org-id link standard, the =docs-lifecycle= rule, spec-create/spec-review/spec-response updates, the retrofit helper + startup nudge, and the rulesets pilot.
- Out of scope: applying the lifecycle shape to non-doc collections (the rule documents the pattern; adopting it elsewhere is per-collection work).
- vNext: an org-agenda custom view over =docs/specs/*.org= keyed on the status keywords (nice-to-have once the keywords exist; log to todo.org).
* Design
** The location split
- =docs/specs/= — formal specs only. A *spec* is a doc proposing a buildable change that carries a =Decisions= section and =Implementation phases= (the spec-create spine). Filenames keep the existing =YYYY-MM-DD-<topic>-spec.org= shape — the =-spec.org= suffix stays because spec-review's Phase 0 precondition keys on it; only the *status* suffixes from the original proposal are dropped.
- =docs/design/= — everything else: brainstorms, inventories, proposals, research notes, frozen source material. Review findings live inside the spec they review (current spec-review behavior), so standalone review files are legacy notes and stay in =docs/design/=.
** The status heading (the authoritative record)
Each spec's first element after the file header is a single top-level *status heading* carrying the org TODO keyword:
#+begin_example
,#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED
,* DOING <spec short name>
:PROPERTIES:
:ID: <uuid>
:END:
- <dated one-line history entries, newest first>
#+end_example
- *The keyword is authoritative.* The Metadata table's =Status= field mirrors it in lowercase for readers already in the table; on disagreement the heading wins.
- *Vocabulary:* =DRAFT= (being written) → =READY= (review passed, buildable) → =DOING= (implementation in progress) → =IMPLEMENTED= / =SUPERSEDED= / =CANCELLED= (terminal). This supersedes the old =TODO | DONE SUPERSEDED CANCELLED= header line; the retrofit updates it.
- *Glanceability without opening files:* one grep gives the full board —
#+begin_src sh
rg -H '^\* (DRAFT|READY|DOING|IMPLEMENTED|SUPERSEDED|CANCELLED) ' docs/specs/
#+end_src
and because the keyword sits on a real org heading, an org-agenda view over =docs/specs/= works for free (the vNext item).
- *The heading body is the dated status history* — one line per transition (=YYYY-MM-DD Day @ HH:MM:SS -ZZZZ — <what changed, by whom>=), the record a filename could never carry.
- Why a dedicated status heading rather than restructuring each spec under one top-level heading: demoting every section in every existing spec is a large, link-hostile rewrite; a prepended heading is additive, retrofittable by script, and leaves the familiar flat section layout untouched.
** Rename-safe links
The status heading carries an =:ID:= UUID. Cross-doc references to a spec use =[[id:<uuid>]]= rather than =file:= paths, so the one move the retrofit performs (and any future move) can't orphan them. =file:= links remain fine for intra-doc anchors and for notes that never move. The KB's existing id-resolution recipe applies: =rg ':ID:[[:space:]]+<uuid>' docs/=.
** The =docs-lifecycle= rule (the generalization)
A new =claude-rules/docs-lifecycle.md= captures the reusable shape, with spec-create as the first instance:
1. Separate formal artifacts from working notes by location.
2. Lifecycle state lives *in* the artifact, on a scannable, greppable carrier (an org keyword heading), with a dated history.
3. Links use rename-safe identifiers.
4. A growing collection earns this treatment when "which of these are live?" starts requiring a file-by-file read.
** The retrofit (reach mechanism for existing piles)
A synced helper, =.ai/scripts/spec-sort=, run once per project:
1. *Classify* each =docs/design/*.org= (and stray =docs/*-spec.org=) by heuristic: a doc with a =Decisions= section plus =Implementation phases= or a =Metadata= table is a spec candidate; everything else is a note. The heuristic *proposes*; a human confirms every move (classification is a judgment call — Craig, 2026-06-28).
2. *Move* confirmed specs to =docs/specs/=, prepend the status heading (keyword proposed from the doc's current Status field or review history; confirmed by the human), assign an =:ID:=, and update the =#+TODO:= header line.
3. *Relink* inbound =file:= references (=todo.org=, =notes.org=, =.ai/=, other docs) to the new path — and to =id:= form where the target is a spec.
4. *Stamp* =:LAST_SPEC_SORT: YYYY-MM-DD= in =notes.org= Workflow State.
A startup nudge (one line in Phase C) fires in projects that have a =docs/design/= but no =:LAST_SPEC_SORT:= marker, so each project runs the sort exactly once without template sync having to perform it.
* Alternatives Considered
** Filename status suffix (=-spec-doing.org=, =-spec-implemented.org=)
- Good, because the state is visible in a bare =ls= with no tooling.
- Bad, because every transition renames a file in a cross-linked, template-synced doc set — each rename is link surgery or a broken link, and the churn lands in git history and inbound =todo.org= links.
- Neutral, because the ls-visibility it buys is matched by the one-line =rg= over status headings.
- Rejected 2026-06-28 (Craig chose org-keyword over his earlier filename-suffix lean).
** Status field in the Metadata table only (no keyword)
- Good, because the field already exists and needs no new structure.
- Bad, because a table cell is neither org-agenda-scannable nor reliably greppable across format drift, and it carries no dated history.
- Neutral, because the field stays anyway — as the in-table mirror.
** Relink-helper instead of org-id (keep =file:= links, fix them on every move)
- Good, because readers see plain paths.
- Bad, because it makes every future move a tooling event, and one missed run silently breaks links — the failure mode is invisible until someone clicks.
- Neutral, because the retrofit needs relink logic once regardless; org-id just makes it a one-time need.
* Decisions [5/5]
All five were settled with Craig on 2026-06-28 (recorded in todo.org; migrated here per that note).
** DONE Location split — adopt
- Context: specs and notes share one directory; telling them apart requires opening files.
- Decision: =docs/specs/= for formal specs (Decisions + phases spine); =docs/design/= for notes. Documented in spec-create and the docs-lifecycle rule.
- Consequences: easier — a listing answers "what's formal"; harder — one-time migration and link updates (the retrofit).
** DONE Status mechanism — org keyword authoritative, no filename suffix
- Context: filename suffix vs org keyword; suffix wins =ls= visibility, keyword wins link stability and zero-rename transitions.
- Decision: the org TODO keyword on the spec's top status heading is authoritative, mirrored by the Metadata =Status= field. No status suffixes in filenames.
- Consequences: easier — a transition is one keyword edit and links never break; harder — glanceability needs the one-line =rg= (or the vNext agenda view) instead of bare =ls=.
** DONE Link safety — org-id for cross-doc spec links
- Context: both the migration move and any future rename break =file:= links.
- Decision: specs carry =:ID:= UUIDs on the status heading; cross-doc references use =[[id:...]]=.
- Consequences: easier — moves are free; harder — following a link outside org needs the =rg ':ID:'= lookup.
** DONE Generalize as a =docs-lifecycle= rule
- Context: the shape (in-artifact lifecycle state, formal-vs-notes split, rename-safe links) recurs for any processed-document collection.
- Decision: capture it in =claude-rules/docs-lifecycle.md= with spec-create as the first instance.
- Consequences: easier — the next collection reuses a decided pattern; harder — the rule must stay honest as the spec instance evolves.
** DONE Retrofit existing files across ALL projects
- Context: template sync distributes conventions but cannot perform a per-project one-time migration; legacy piles would stay misfiled forever.
- Decision: ship a confirmed classify-move-relink helper (=spec-sort=) plus a startup nudge gated on =:LAST_SPEC_SORT:=; the helper proposes, a human confirms. Pilot on rulesets first.
- Consequences: easier — every project converges without manual archaeology; harder — the helper needs real relink logic and tests, and classification stays a judgment call.
* Implementation phases
** Phase 1 — Rule + template updates
Write =claude-rules/docs-lifecycle.md=. Update spec-create (emit into =docs/specs/=, new =#+TODO:= vocabulary, status heading with =:ID:= in the template, transition mechanics), spec-review (precondition path =docs/specs/=; flipping =DRAFT= → =READY= on a passing review updates the heading + history line), and spec-response (status flips it owns). Tree stays working: new specs land in the new shape; old specs are untouched until Phase 3.
** Phase 2 — The =spec-sort= helper
Build =.ai/scripts/spec-sort= (classify → confirm → move + prepend status heading + assign =:ID:= → relink inbound references → stamp =:LAST_SPEC_SORT:=), with bats coverage for classification, moving, relinking, idempotence, and the confirm gate. Tree stays working: the script is callable but nothing invokes it yet.
** Phase 3 — Pilot on rulesets
Run =spec-sort= against rulesets' own =docs/= (41 design files, 3 spec-spine candidates, 2 stray root specs). Fix what the pilot surfaces before any other project runs it. Tree stays working: moves are confirmed one by one, links updated in the same pass.
** Phase 4 — Startup nudge + broadcast
Add the Phase C nudge (fires when =docs/design/= exists and =:LAST_SPEC_SORT:= is absent). Send .emacs.d a note that the convention is live and its ~28-doc pile is ready to sort. Tree stays working: the nudge is one read-only line per session until acted on.
* Acceptance criteria
- [ ] =rg '^\* (DRAFT|READY|DOING|IMPLEMENTED|SUPERSEDED|CANCELLED) ' docs/specs/= lists every rulesets spec with its state, and the answer matches reality.
- [ ] A status transition on a spec changes exactly one keyword plus one history line — no rename, no link edits.
- [ ] Every doc remaining in rulesets =docs/design/= is a note (no Decisions + phases spine); both stray =docs/= root specs are re-homed.
- [ ] All inbound =todo.org= / =notes.org= / doc links to moved specs resolve after the pilot.
- [ ] spec-create emits new specs into =docs/specs/= in the new shape; spec-review refuses a spec outside it.
- [ ] A project with an unsorted =docs/design/= gets the startup nudge; one confirmed =spec-sort= run clears it via =:LAST_SPEC_SORT:=.
* Readiness dimensions
- Data model & ownership: the spec file owns its state; the Metadata mirror is display-only. No external index to drift.
- Errors, empty states & failure: =spec-sort= on a project with no =docs/= is a silent no-op; an ambiguous classification is surfaced, never auto-moved; a relink pass that finds zero inbound links is normal.
- Security & privacy: N/A because the docs are already in-repo; no new exposure surface.
- Observability: the status grep is the dashboard; =spec-sort= prints every proposed move and every rewritten link.
- Performance & scale: N/A because collections are tens of files; everything is one-shot or grep-speed.
- Reuse & lost opportunities: reuses org TODO keywords, org-id, the existing scheme-header pattern of declared vocabularies, and spec-review's in-file findings convention.
- Architecture fit & weak points: weak point is the classification heuristic — mitigated by the confirm gate. The status heading is additive, so old readers of spec files see one extra heading and nothing breaks.
- Config surface: none new — one marker line (=:LAST_SPEC_SORT:=) in the existing Workflow State section.
- Documentation plan: the docs-lifecycle rule is the documentation; spec-create's template is the worked example.
- Dev tooling: =spec-sort= ships with bats tests under the existing glob-discovered suite.
- Rollout, compatibility & rollback: additive per project, one project at a time, rulesets first. Rollback of a sort is =git revert= of the pilot commit (moves + relinks are one commit).
- External APIs & deps: N/A — plain files, =rg=, =uuidgen=.
* Risks, Rabbit Holes, and Drawbacks
- *Relink misses an inbound link shape* (org radio links, bare paths in scripts). Dodge: the pilot greps for the old path after moving and fails loudly on any residue.
- *Heuristic over-classifies notes as specs.* Dodge: the confirm gate is mandatory; the helper never moves unconfirmed.
- *Keyword vocabulary drift* between this spec, the rule, and spec-create's template. Dodge: the rule names the vocabulary once and the others link it.
* Testing / Verification / Rollout
bats for =spec-sort= (classification, move, relink, idempotence, confirm gate, marker stamp). The pilot run on rulesets is the live verification; the post-move residue grep is the acceptance check. Rollout is per-project via the startup nudge, each run human-confirmed.
* References / Appendix
- Source proposal: [[file:../design/2026-06-15-spec-storage-lifecycle-proposal.org]] (.emacs.d handoff, 2026-06-15).
- Decisions record: todo.org "Spec storage location + lifecycle-status convention" (settled 2026-06-28).
- This file is the convention's first resident: it lives in =docs/specs/=, carries the status heading + =:ID:=, and drops the status filename suffix.
* Review and iteration history
** 2026-07-01 Wed @ 22:13:00 -0400 — Claude — author
- What: initial draft, written from the five pre-ratified decisions.
- Why: the queued-specs half of the 2026-06-30 session goal; decisions were settled 2026-06-28 and needed migration into a buildable spec.
- Artifacts: todo.org task "Spec storage location + lifecycle-status convention"; source proposal above.
|
