diff options
Diffstat (limited to 'docs')
50 files changed, 5024 insertions, 189 deletions
diff --git a/docs/design/2026-05-28-generic-agent-runtime-spec.org b/docs/design/2026-05-28-generic-agent-runtime-spec.org index 01be6d4..7d7a549 100644 --- a/docs/design/2026-05-28-generic-agent-runtime-spec.org +++ b/docs/design/2026-05-28-generic-agent-runtime-spec.org @@ -3,6 +3,10 @@ #+DATE: 2026-05-28 #+STARTUP: showall +* Status note (2026-06-16) + +The cross-agent-comms subsystem this spec references as an existing substrate (=cross-agent-send= / =-recv= / =-watch= / =-status= / =-discover= / =-halt= / =-resume=, the =inbox/from-agents/= file-IPC protocol) was *removed* on 2026-06-16 as unused — every real cross-project handoff goes through =inbox-send= instead. Sections below that propose extending the cross-agent protocol (e.g. "Cross-agent updates", the =machine.project.agent-id= targeting) are historical: if this arc is revived, that layer would be rebuilt on =inbox-send=, not the deleted scripts. + * Introductory note Craig asked for a design pass on making =rulesets= generic rather than diff --git a/docs/design/2026-05-28-pattern-catalog-prompt-labels-and-defaults.org b/docs/design/2026-05-28-pattern-catalog-prompt-labels-and-defaults.org index cc2cb77..9ee29d6 100644 --- a/docs/design/2026-05-28-pattern-catalog-prompt-labels-and-defaults.org +++ b/docs/design/2026-05-28-pattern-catalog-prompt-labels-and-defaults.org @@ -28,7 +28,7 @@ A generic =pearl--with-sentinel SENTINEL CANDIDATES= helper lets each call site ** Why for the catalog -This is the "no hidden affordances" pattern from the earlier note ([[file:2026-05-28-0003-from-pearl-rulesets-followup-no-empty-input.org]]) sharpened with a second rule: *if the affordance is visible, its label has to match what picking it does*. Visibility without accuracy is its own problem. A label that says "none" when the behavior is "any" is no better than an invisible empty-input idiom — both leave the user holding the wrong model. +This is the "no hidden affordances" pattern from the earlier note (a processed pearl-rulesets inbox item, since removed) sharpened with a second rule: *if the affordance is visible, its label has to match what picking it does*. Visibility without accuracy is its own problem. A label that says "none" when the behavior is "any" is no better than an invisible empty-input idiom — both leave the user holding the wrong model. Catalog shape suggestion: this is the same principle as Pattern 3, in a follow-up form. Either a single entry that captures both halves (visible + accurate) or two cross-linked entries. diff --git a/docs/design/2026-06-02-flush-promotion.org b/docs/design/2026-06-02-flush-promotion.org index 9d9d8a3..ff4e579 100644 --- a/docs/design/2026-06-02-flush-promotion.org +++ b/docs/design/2026-06-02-flush-promotion.org @@ -1,5 +1,5 @@ #+TITLE: Flush Promotion — Handoff Bundle (from work) -#+AUTHOR: Craig Jennings & Claude +#+AUTHOR: Craig Jennings #+DATE: 2026-06-02 * Provenance diff --git a/docs/design/2026-06-02-pattern-catalog-spec.org b/docs/design/2026-06-02-pattern-catalog-spec.org index e74b8ae..6bb105a 100644 --- a/docs/design/2026-06-02-pattern-catalog-spec.org +++ b/docs/design/2026-06-02-pattern-catalog-spec.org @@ -1,5 +1,5 @@ #+TITLE: Cross-Project Pattern Catalog — Spec -#+AUTHOR: Craig Jennings & Claude +#+AUTHOR: Craig Jennings #+DATE: 2026-06-02 One-page spec for capturing reusable design patterns so they travel from one project to the next instead of being re-derived. Drafted for Craig's spec-review; the five open questions from the [[file:../../todo.org][todo.org]] task carry a recommended call each, marked DECISION. diff --git a/docs/design/2026-06-15-auto-triage-intake-spec.org b/docs/design/2026-06-15-auto-triage-intake-spec.org new file mode 100644 index 0000000..b41c99c --- /dev/null +++ b/docs/design/2026-06-15-auto-triage-intake-spec.org @@ -0,0 +1,47 @@ +#+TITLE: Proposed engine addition — Auto mode (auto triage-intake) +#+DATE: 2026-06-15 + +* What this adds + +A new *mode* of the triage-intake engine: *auto mode* (auto triage-intake). It's a self-running monitor for when Craig is away from the desk but wants tight awareness — a loop that runs the standard triage on a short interval, accumulates rather than mutating, and hands Craig a controlled checkpoint to commit the batch. + +Origin: 2026-06-15, the morning Craig had to clear his day for a family emergency and wanted the desk watched while he was in and out. He asked for 20-minute sweeps that summarize what's come in for him, with an explicit command to process and commit the batch. + +* The mode + +** Cadence +A loop (CronCreate / =/loop=) fires the triage on an interval — default *20 minutes*. Craig sets the interval. + +** Accumulate, don't mutate (the core difference from a normal run) +A normal triage run writes the sentinel, creates =:quick:reactive:= todos, takes mail actions on confirmation, and is a one-shot. An auto-mode *sweep* does none of the mutations: + +- Does NOT advance the sentinel — the scan window grows from the last *close* until the next close, so nothing is dropped between sweeps. +- Does NOT create todo.org tasks — accumulates them (in the session log) for the close. +- Does NOT take mail actions (trash / mark-read / star). +- Does NOT commit. +- DOES run the full plugin scan, DOES update an active daily-prep (Update mode) and re-open it on change, DOES report. + +** End-of-sweep output +Each sweep ends with two short sections: +1. *Deltas* — what changed since the last sweep (one line if nothing). +2. *Responses awaiting your acknowledgment* — every Slack reply, email, or message directed at the user that he hasn't explicitly acknowledged or had the agent answer. This is a *running list carried forward* across sweeps until the user acks each item or closes the triage. It exists because an away user's main need is "who's waiting to hear back from me," which a delta-only sweep loses the moment it scrolls past. + +** Close / stop commands (the checkpoint) +The mutations are gated behind two user commands: + +- *"close the triage"* — finish the mail + sentinel processing, add all accumulated todos (asking the user questions along the way), commit and push, then *keep looping* (next sweep on the normal interval). This is the "flush the batch and carry on" checkpoint. +- *"stop the triage"* — the same close processing, then *stop the loop* and revert to manual triage (run only when asked). + +* Why it's worth adding to the engine + +The standard engine is one-shot and mutating, which is right for an at-the-desk "what's new?" glance. It's wrong for unattended monitoring: running it every 20 minutes would spray reactive todos, advance the sentinel past unprocessed items, and commit noise without review. Auto mode separates the cheap, frequent *watching* from the deliberate, gated *committing* — and adds the away-user's missing primitive, the running unacknowledged-responses list. + +* Companion notes + +- The interval loop is the delivery mechanism (CronCreate session-only, or a durable schedule); the mode is the behavior. They compose. +- "Responses awaiting acknowledgment" tracking needs a small piece of state. In a session it can live in the session-context log; if the engine wants it durable across sessions, a tiny =.ai/triage-unacked.org= (or similar) is the natural home — flagged as a design choice for the rulesets side to decide. +- Deltas-only reporting (the 2026-06-11 ruling) and loud scan-failure surfacing both still apply inside each sweep. + +* Addendum (work, 2026-06-15 17:16) + +Add a third end-of-sweep output line: the current date/time/timezone, on its own final line, via =date "+%A %Y-%m-%d %H:%M:%S %Z (%z)"=. Reason: on an away day with frequent unattended sweeps, the per-sweep stamp shows how fresh each summary is at a glance. Sweep output sections become: (a) Deltas, (b) Responses awaiting acknowledgment, (c) the timestamp. Implemented 2026-06-15; the stamp prints on quiet sweeps too, as proof the loop ran. diff --git a/docs/design/2026-06-15-fix-speedrun-workflow-proposal.org b/docs/design/2026-06-15-fix-speedrun-workflow-proposal.org new file mode 100644 index 0000000..c1c7077 --- /dev/null +++ b/docs/design/2026-06-15-fix-speedrun-workflow-proposal.org @@ -0,0 +1,21 @@ +#+TITLE: Proposed reusable workflow: "fix speedrun" mode, available t +#+SOURCE: from .emacs.d +#+DATE: 2026-06-15 19:22:56 -0500 + +Proposed reusable workflow: "fix speedrun" mode, available to all coding projects. + +Origin: a 2026-06-15 .emacs.d theme-studio session. Craig batched a list of quick wins / small fixes and asked me to run them autonomously. + +The shape that worked: +- Entry: Craig names an ordered set of tasks (or points at a tagged subset) and says "fix speedrun" / "no approvals until done". +- Execution: work the set in order, no per-step approval gates. Each task still runs the full quality bar (TDD red->green, /review-code, /voice on the commit), and each is committed + pushed as its own logical commit when green ("always push this session" pairs naturally). +- Ambiguity handling: if a task turns out underspecified or already-satisfied, do not guess-implement — file a VERIFY noting why and move on. (This session: "raise max spans to 5" — every cap was already 8.) +- Exit / handoff: when the set is done, PAGE the user (proactive push notification) with the project name, the completed task(s), and a numbered list of remaining tasks. The user confirms ready + names the next project in one reply. + +Open questions for the canonical: +- Where the page fires (every task vs end-of-set) and via what (push notification). +- How it composes with the existing no-approvals + always-push session modes (is "fix speedrun" just a named preset of those plus the end page?). +- Whether it should auto-pull the task set from a tag/priority query rather than an explicit list. +- Guardrails: it should refuse to speedrun tasks that need design decisions or carry data-loss risk without a checkpoint (e.g. the unused-tile flag here was biased-safe deliberately). + +This is a rulesets-owned concept (cross-project), so sending it here rather than building it locally. No local stopgap made. diff --git a/docs/design/2026-06-15-spec-storage-lifecycle-proposal.org b/docs/design/2026-06-15-spec-storage-lifecycle-proposal.org new file mode 100644 index 0000000..7dbba72 --- /dev/null +++ b/docs/design/2026-06-15-spec-storage-lifecycle-proposal.org @@ -0,0 +1,44 @@ +#+TITLE: Proposal — spec storage location + lifecycle-status convention + +* What I'm proposing + +Two coupled documentation conventions, surfaced while triaging ~28 design docs in .emacs.d. Both belong in rulesets — the spec-create workflow (=.ai/workflows/spec-create.org=) and likely a new docs-lifecycle rule under =claude-rules/=. + +** A. Separate specs from working notes by location +Today .emacs.d/docs/design/ holds everything in one bucket: formal specs (goals/decisions/phases/acceptance) jumbled with brainstorms, inventories, reviews, and idea-lists. You can't tell specs from notes without opening each file. + +Proposal: formal specs live in =docs/specs/=; =docs/design/= keeps working notes, brainstorms, inventories, and reviews. A "spec" is a doc proposing a buildable change with a Decisions section and phases; everything else is a note. + +** B. Make a spec's lifecycle status glanceable +Specs carry no lifecycle status today, so which shipped, which are open, and which are dead is invisible. I had to run a four-agent sweep reading every spec against the code to reconstruct it (6 implemented, 8 in-progress, 12 not-started, 1 superseded). A status convention makes that a one-line scan. + +Two parts: +- Filename suffix (Craig's idea): =<topic>-spec.org= (draft / not-started) → =-spec-doing.org= → =-spec-implemented.org= → =-spec-superseded.org= / =-spec-cancelled.org=. Visible in =ls=, greppable by glob, impossible to forget. +- Authoritative Status field in the spec's Metadata table, on every spec (retrofit old ones). The filename is the index; the field is the record — it also carries a dated history line a filename can't hold. + +* Why + +Triage. A collection of specs with no status field and no location split degrades into "open it to find out." The cost compounds with every spec added. The two conventions together make the answer visible from a directory listing. + +* Recommendations / refinements + +- Pair filename suffix + Status field; the field is authoritative, the filename is the at-a-glance index. Don't let the filename be the only record (it's volatile and loses the why/when). +- Link safety is the real cost. Both the move to =docs/specs/= and every status rename break =[[file:...]]= links (e.g. todo.org → a spec's Related link). Mitigate one of two ways, and pick one as the standard: + - Switch cross-doc links to =org-id= (=[[id:...]]=), which survive moves and renames; or + - A helper that moves/renames + relinks inbound references + stamps the Status field + appends a dated history line, run on each transition — so a status change is one command, not manual link surgery. +- Vocabulary: draft (no suffix) / doing / implemented / superseded / cancelled. An in-progress-but-partial spec is "doing." + +* The bigger design choice rulesets should weigh + +Filename-encoded status is one option; the other is the org TODO keyword on the spec's top heading. These specs already carry =#+TODO: TODO | DONE SUPERSEDED CANCELLED=, so the top-heading keyword could be the status — scannable via a docs/specs org-agenda view, greppable, and it never breaks a link because the filename stays stable. The tradeoff is ls-visibility (filename wins) versus link-stability and zero-rename transitions (org-keyword wins). Worth deciding deliberately; my lean is filename suffix for the listing-level visibility Craig wants, paired with the Status field, with org-id links to neutralize the rename cost. + +* Generalization — a reusable pattern + +This is not spec-specific. The shape is: lifecycle state visible in the artifact's name (or location), an authoritative status record inside the artifact, rename-safe linking, and formal artifacts separated from working notes by location. Reach for it whenever triaging a growing collection of processed documents or media — brainstorms, inbox items, working-files, a transcription/recording queue, anything with a draft → in-progress → done/dropped lifecycle. Worth capturing as a general docs-lifecycle convention in claude-rules/, with the spec-create workflow as the first concrete instance. + +* Follow-up + +- Decide the canonical mechanism (filename suffix vs org-TODO-keyword) and the link-safety standard (org-id vs relink-helper). +- Update spec-create to emit into =docs/specs/= with a Status field and the chosen status mechanism documented. +- If you want the relink-helper, it's a natural =.ai/scripts/= addition that downstream projects get via the template sync. +- Send a note back if you want .emacs.d to pilot the chosen approach before it's generalized. diff --git a/docs/design/2026-06-16-inbox-zero-phase-e-proposal.diff b/docs/design/2026-06-16-inbox-zero-phase-e-proposal.diff new file mode 100644 index 0000000..e3d8ee8 --- /dev/null +++ b/docs/design/2026-06-16-inbox-zero-phase-e-proposal.diff @@ -0,0 +1,65 @@ +--- claude-templates/.ai/workflows/inbox-zero.org 2026-06-13 13:18:35.988799778 -0500 ++++ /proc/self/fd/12 2026-06-16 00:18:07.592944696 -0500 +@@ -1,6 +1,3 @@ +-#+TITLE: Inbox Zero Workflow +-#+AUTHOR: Craig Jennings & Claude +-#+DATE: 2026-06-13 + + * Overview + +@@ -19,7 +16,7 @@ + Reused from three callers so the steps live in one place: + - *Startup* (read-only nudge) — count the items, identify which appear related to this project, surface both numbers, offer processing as one of the startup options. Never auto-files. + - *Wrap-up* (Step 3 sub-step) — sweep items that belong here before the cleanup scripts, so imported tasks lint and ride the wrap commit. +-- *On demand* — "inbox zero", "empty the inbox", "process the roam inbox", "triage my roam inbox". ++- *On demand* — "inbox zero", "empty the inbox", "process the roam inbox", "triage my roam inbox". The on-demand caller (and the recurring loop that invokes it) also runs *Phase E* below: after routing the inbox, it works the project's actionable backlog, implementing eligible =:next:= / =:quick:+:solo:= tasks. Startup and wrap-up skip Phase E. + + Each project touches the roam inbox at least twice a session this way: once at startup, once at wrap-up. + +@@ -69,6 +66,46 @@ + + Report: moved (with their new priorities and tags), folded, dropped-as-done. Then the residue: foreign items (left for their owners, count only) and unowned items (count plus the headings that appear related to this project, for manual claim or prefix). Same "summarize what we kept" shape. + ++* Phase E — Execute actionable tagged tasks (autonomous; on-demand / loop caller only) ++ ++After routing the inbox, the on-demand and loop callers work the project's actionable backlog. *Startup (read-only) and wrap-up (winding down) skip this phase entirely* — it runs only when a human or the recurring loop invokes the workflow to make progress, never as a side effect of session bookkeeping. ++ ++** Eligibility gate ++ ++Scan =todo.org= (both items freshly filed in Phase B and the existing backlog). A task is a candidate when ALL hold: ++ ++1. Status is =TODO= — not =VERIFY=, =DOING=, =DONE=, or =CANCELLED=. =VERIFY= means "awaiting Craig's manual confirmation"; never auto-implement a VERIFY. Those are the manual-testing verifications this phase deliberately leaves alone. ++2. Tagged =:next:=, OR tagged BOTH =:quick:= AND =:solo:=. ++3. Implementable solo — no input or undecided judgment call from Craig. ++4. Estimated at roughly 30 minutes or less of focused work. ++ ++** Act-vs-file decision ++ ++For each candidate, after a quick scope read of the relevant code: ++ ++- *Clear, bounded, solo, ≤ ~30 min* → implement it now (below). ++- *Needs Craig's input, a decision, or design discussion* → do NOT implement. Leave it filed, add a one-line note on the task naming the input it needs, and surface it. ++- *An hour or more* → do NOT implement. Leave it filed and surface it as a larger task for a dedicated =/start-work= session. ++ ++When unsure which side a task falls on, file rather than implement. A wrong auto-implement costs more than a deferred task. ++ ++** Implementing a candidate ++ ++Per task, follow the project's commit discipline — the per-project waiver: no approval gate, but TDD + =/review-code= + =/voice personal= on every commit, no AI attribution: ++ ++1. Trace to root cause; write the failing test first (Red → Green → Refactor). ++2. Live-reload into the running daemon and verify per the emacs reload-and-verify loop. ++3. Close the task per =todo-format.md= (top-level → =DONE= + =CLOSED:=; sub-task → dated log rewrite). When the only residue is Craig's manual check, file a =VERIFY= child under "Manual testing and validation" and close the originating task (the codified manual-verification-handoff pattern). ++4. =/review-code --staged= → fix all Critical/Important → =/voice personal= on the message → commit individually. Push per the project's flow. ++ ++** Bounding the run ++ ++Default to one task per run: implement the highest-priority eligible candidate (=[#A]= before =[#B]= before =[#C]=), commit, then stop and let the next tick or the next on-demand invocation continue. A caller may work more than one in a run when the eligible tasks are small and clearly independent — but each gets its own test and its own commit, and the run stays reviewable. Never batch unrelated changes into one commit. ++ ++** Surface ++ ++Report what was implemented (task + commit), what was deferred and why (needs-input / too-large), and what stays filed. ++ + * Skip conditions + + - No =~/org/roam/inbox.org= → silent no-op. diff --git a/docs/design/2026-06-16-inbox-zero-phase-e-proposal.org b/docs/design/2026-06-16-inbox-zero-phase-e-proposal.org new file mode 100644 index 0000000..5fa7e12 --- /dev/null +++ b/docs/design/2026-06-16-inbox-zero-phase-e-proposal.org @@ -0,0 +1,137 @@ +#+TITLE: Inbox Zero Workflow +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-13 + +* Overview + +The roam global inbox (=~/org/roam/inbox.org=) is Craig's cross-project GTD capture: one shared file every project can see. This workflow routes each inbox item to the project that owns it. The current session claims only the items belonging to THIS project, files them into the project's =todo.org=, and removes them from the shared inbox. Everything it doesn't own stays. The aspiration is inbox zero: over time, every item lands in its owning project. + +This is NOT =process-inbox.org=. That workflow handles the project's own =inbox/= directory (handoffs from other projects, scripts, Craig). This one handles the single global =roam/inbox.org= and the cross-project routing a shared file creates. + +This is also distinct from the wrap-up inbox/transcript routing feature (which moves session-filed keepers between projects). This routes the shared roam capture file by ownership prefix. + +** Scope: single-destination (v1) + +This version routes each item to its one owning project, identified by an explicit =<project>:= heading prefix. The multi-project domain-aware mode, which would guess the owner of every unprefixed item and empty the whole inbox in one run, is deferred (see "Deferred: domain-aware routing" at the end). v1 claims only what's prefixed for the current project, surfaces the rest, and never guesses. + +** Three callers + +Reused from three callers so the steps live in one place: +- *Startup* (read-only nudge) — count the items, identify which appear related to this project, surface both numbers, offer processing as one of the startup options. Never auto-files. +- *Wrap-up* (Step 3 sub-step) — sweep items that belong here before the cleanup scripts, so imported tasks lint and ride the wrap commit. +- *On demand* — "inbox zero", "empty the inbox", "process the roam inbox", "triage my roam inbox". The on-demand caller (and the recurring loop that invokes it) also runs *Phase E* below: after routing the inbox, it works the project's actionable backlog, implementing eligible =:next:= / =:quick:+:solo:= tasks. Startup and wrap-up skip Phase E. + +Each project touches the roam inbox at least twice a session this way: once at startup, once at wrap-up. + +* The ownership rule (the coordination primitive) + +The inbox is shared, so the workflow must never let two projects fight over an item or let one project grab another's. Ownership is by explicit prefix: + +- =<project>: ...= heading → owned by that project. The current project claims only items prefixed with its own identifier. +- Prefixed for *another* project → leave untouched (cross-project boundary, =protocols.org=). +- *No prefix* → unowned. Never auto-claim. Surface as candidates a human can claim or prefix. + +The prefix partition is what makes concurrent triage across projects safe: each project only ever removes its own items, so two sessions editing the inbox touch disjoint lines. + +** Resolving this project's identifier (v1) + +Use the project root basename plus its common aliases (=.emacs.d= ↔ =emacs=, and the obvious ones: =rulesets=, =work=, =home=). A project may override the inferred set with an =:INBOX_PREFIX:= line in =notes.org='s *Workflow State* section when the basename is fragile (a dot in the name, an alias the inference misses). The explicit override is optional in v1; the durable multi-project resolution is part of the deferred domain-aware mode. + +* Phase A — Identify, count, and match + +1. Resolve the current project's identifier and aliases (above). +2. Read =~/org/roam/inbox.org=. If absent, silent no-op (the file lives only on machines with the roam clone). +3. Bucket every item under the inbox heading: + - *claimed* — prefixed for this project + - *foreign* — prefixed for another project → leave + - *unowned* — no project prefix +4. *Summarize the scan* (Craig's requirement, every scan): report the total item count in the inbox, then the count that appears related to this project. "Appears related" is the union of claimed items (exact prefix) and any unowned item whose topic plainly concerns this project's domain (a content judgment, surfaced as a candidate, never auto-claimed). Foreign-prefixed items are not "related" — they belong to their owner. +5. If both claimed and related-unowned are empty, report the total and stop (the common case for most wraps). + +* Phase B — File each claimed item into todo.org + +Apply =process-inbox.org='s discipline against the project's =todo.org=; don't reinvent it: + +1. *Status check first.* Already done, or already a task in =todo.org=? → drop it, or fold into the existing task (dated sub-entry per =todo-format.md=). Don't duplicate. +2. *Rewrite* to terse-heading + body per =todo-format.md=. +3. *Priority + tags from THIS project's scheme* — the legend at the top of its =todo.org=, tags from that scheme's allowed set only. The project expresses someday-maybe with =[#D]=; there's no special someday-maybe routing. +4. *File* under the project's Open Work section. + +* Phase C — Reconcile the shared inbox + +The roam inbox lives in a git repo (=~/org/roam=, auto-synced by the =roam-sync= timer). Edit it carefully: + +1. *Pull first* (=git -C ~/org/roam pull --ff-only=). If it can't fast-forward (dirty tree, divergence), surface and stop. Don't auto-stash, auto-merge, or force. Resolve before removing items. +2. *Remove only the claimed items.* Never touch foreign or unowned items. +3. *Commit the roam repo as its own commit* (separate from any project wrap commit): =chore(inbox): route <project> tasks to <project>/todo.org=. Push, or leave for the =roam-sync= timer. Surface a blocked push; don't force. + +* Phase D — Surface + +Report: moved (with their new priorities and tags), folded, dropped-as-done. Then the residue: foreign items (left for their owners, count only) and unowned items (count plus the headings that appear related to this project, for manual claim or prefix). Same "summarize what we kept" shape. + +* Phase E — Execute actionable tagged tasks (autonomous; on-demand / loop caller only) + +After routing the inbox, the on-demand and loop callers work the project's actionable backlog. *Startup (read-only) and wrap-up (winding down) skip this phase entirely* — it runs only when a human or the recurring loop invokes the workflow to make progress, never as a side effect of session bookkeeping. + +** Eligibility gate + +Scan =todo.org= (both items freshly filed in Phase B and the existing backlog). A task is a candidate when ALL hold: + +1. Status is =TODO= — not =VERIFY=, =DOING=, =DONE=, or =CANCELLED=. =VERIFY= means "awaiting Craig's manual confirmation"; never auto-implement a VERIFY. Those are the manual-testing verifications this phase deliberately leaves alone. +2. Tagged =:next:=, OR tagged BOTH =:quick:= AND =:solo:=. +3. Implementable solo — no input or undecided judgment call from Craig. +4. Estimated at roughly 30 minutes or less of focused work. + +** Act-vs-file decision + +For each candidate, after a quick scope read of the relevant code: + +- *Clear, bounded, solo, ≤ ~30 min* → implement it now (below). +- *Needs Craig's input, a decision, or design discussion* → do NOT implement. Leave it filed, add a one-line note on the task naming the input it needs, and surface it. +- *An hour or more* → do NOT implement. Leave it filed and surface it as a larger task for a dedicated =/start-work= session. + +When unsure which side a task falls on, file rather than implement. A wrong auto-implement costs more than a deferred task. + +** Implementing a candidate + +Per task, follow the project's commit discipline — the per-project waiver: no approval gate, but TDD + =/review-code= + =/voice personal= on every commit, no AI attribution: + +1. Trace to root cause; write the failing test first (Red → Green → Refactor). +2. Live-reload into the running daemon and verify per the emacs reload-and-verify loop. +3. Close the task per =todo-format.md= (top-level → =DONE= + =CLOSED:=; sub-task → dated log rewrite). When the only residue is Craig's manual check, file a =VERIFY= child under "Manual testing and validation" and close the originating task (the codified manual-verification-handoff pattern). +4. =/review-code --staged= → fix all Critical/Important → =/voice personal= on the message → commit individually. Push per the project's flow. + +** Bounding the run + +Default to one task per run: implement the highest-priority eligible candidate (=[#A]= before =[#B]= before =[#C]=), commit, then stop and let the next tick or the next on-demand invocation continue. A caller may work more than one in a run when the eligible tasks are small and clearly independent — but each gets its own test and its own commit, and the run stays reviewable. Never batch unrelated changes into one commit. + +** Surface + +Report what was implemented (task + commit), what was deferred and why (needs-input / too-large), and what stays filed. + +* Skip conditions + +- No =~/org/roam/inbox.org= → silent no-op. +- No claimed and no related-unowned items → report the total, stop. +- Roam pull blocked → surface, stop before editing. + +* Caller integration + +** Startup (read-only nudge) + +Phase A of =startup.org= reads =~/org/roam/inbox.org= and produces the scan summary; Phase C surfaces one line: "Roam inbox: N items total, M appear related to this project — say 'inbox zero' to file them." Offered as one of the priority options. Startup never auto-files; it counts and offers. + +** Wrap-up (Step 3 sub-step) + +A sub-step at the start of wrap-up Step 3 (before the cleanup scripts, so imported tasks get linted and ride the wrap commit) delegates here for the claimed set. Skip-fast when nothing matches. + +* Deferred: domain-aware routing (future work, multi-project) + +v1 handles the single-destination case via the prefix rule. The multi-project parts are deferred until the need is real: + +1. *Domain-aware empty-it-all mode.* If rulesets held a description of each project's domain, one run could guess the owner of every item (prefixed or not) and empty the whole inbox at once, delivering each item to its owning project's =inbox/= via =inbox-send= (where that project's =process-inbox= gate still decides whether to file it). This turns "inbox zero" from a per-project aspiration into a single command. Open: where the domain map lives (central registry vs each project's =notes.org=), how confident a guess must be before auto-routing vs surfacing, and whether a low-confidence item stays put. +2. *Explicit per-project =:INBOX_PREFIX:= as the durable resolver*, replacing basename inference. +3. *Unowned-item lifecycle* once domain-aware routing exists (no item stays unrouted indefinitely). +4. *Concurrent push contention* on the shared roam repo: the pull-before-edit + ff-only + surface-on-conflict floor may want a retry-once-after-pull. + +Take these up when the single-destination version is in use and the multi-project pain is concrete. diff --git a/docs/design/2026-06-16-inbox-zero-phase-e-sender-note.org b/docs/design/2026-06-16-inbox-zero-phase-e-sender-note.org new file mode 100644 index 0000000..08e7650 --- /dev/null +++ b/docs/design/2026-06-16-inbox-zero-phase-e-sender-note.org @@ -0,0 +1,27 @@ +#+TITLE: inbox-zero Phase E — autonomous execution of :next: / :quick:+:solo: tasks + +* What changed + +Added a new "Phase E — Execute actionable tagged tasks" to inbox-zero.org (edited locally in .emacs.d as a stopgap; the edited file is attached alongside this note). After routing the roam inbox, the on-demand and loop callers now scan todo.org and autonomously implement eligible tasks. + +Eligibility gate (all must hold): status TODO (not VERIFY/DOING/DONE/CANCELLED); tagged :next: OR both :quick: and :solo:; solo-doable without Craig's input; ~30 min or less. VERIFY is explicitly excluded — in this project VERIFY means "awaiting Craig's manual confirmation," i.e. the manual-testing verifications that must NOT be auto-implemented. Anything needing input or an hour-plus is filed and surfaced, never implemented. Each implementation follows the project commit discipline (TDD + /review-code + /voice personal + individual commit, no AI attribution). Startup (read-only) and wrap-up (winding down) skip Phase E. Default is one task per run, highest priority first. + +* Why + +Craig wants the 30-minute inbox-zero loop to also burn down the small actionable backlog autonomously, not just route capture. The existing tag convention (:next:, :quick:+:solo:) already marks exactly the solo-doable work, so Phase E acts on it under a conservative act-vs-file gate. Companion change: .emacs.d is scheduling inbox-zero on a 30-minute loop with Phase E enabled. + +* Requesting considerations for a general (cross-project) improvement + +The local edit is a stopgap; the durable, canonical version is yours to shape. Open questions for the canonical: + +1. Commit autonomy. Phase E as written assumes .emacs.d's per-project waiver (no per-commit approval gate). Most projects do NOT have that waiver. The canonical Phase E should probably gate on whether autonomous commits are allowed, defaulting to file-only (no auto-commit, surface a diff) unless a project opts in. + +2. Tag resolution. The eligibility tags and the per-project priority/tag scheme vary. Phase E hardcodes :next: / :quick:+:solo:. Better: read the project's todo.org priority/tag scheme header to resolve which tags mean "autonomous-safe." + +3. Do-not-implement markers. "Manual testing verification" is encoded here as status VERIFY (faithful to todo-format.md). Other projects may use VERIFY differently or not at all. Worth a canonical definition of the "do-not-auto-implement" marker set. + +4. Bounding and kill switch. A loop firing every 30 min that implements + commits unattended is powerful. Consider a per-run cap (or a token budget) and an explicit kill switch beyond "default one task per run." + +5. Seam. Whether autonomous execution belongs inside inbox-zero at all, or in its own workflow (e.g. work-the-backlog.org) that the loop chains after inbox-zero. Coupling capture-routing with autonomous-implementation may be the wrong seam — separating them keeps inbox-zero's three existing callers (startup, wrap-up, on-demand) clean. + +Sent from .emacs.d (helper id n/a — primary session). diff --git a/docs/design/2026-06-17-flashcard-multitag-note.md b/docs/design/2026-06-17-flashcard-multitag-note.md new file mode 100644 index 0000000..cfe960d --- /dev/null +++ b/docs/design/2026-06-17-flashcard-multitag-note.md @@ -0,0 +1,28 @@ +# Flashcard tooling: multi-tag headings + --tag-filter (for curated subset decks) + +Source: work project, 2026-06-17. Two synced scripts edited locally as a stopgap; please reconcile into the rulesets canonical so the next sync doesn't revert them. + +## What changed and why + +Craig wanted a curated "DeepSat Fundamentals" Anki deck: the 100 most fundamental cards out of the 465-card deepsat.org org-drill deck, marked with a second org tag (`:fundamental:`) so they stay findable/grep-able in the source. + +The blocker: both `flashcard-to-anki.py` and `flashcard-stats.py` keyed cards on a heading ending in exactly ` :drill:` (`CARD_RE = ^\*\*\s+(.+?)\s+:drill:\s*$`). Adding any second org tag turns the heading into `... :fundamental:drill:`, which that regex does not match — so the 100 tagged cards would silently drop from the full-deck apkg and be undercounted by stats. The "passing gate skips your file" failure mode. + +## flashcard-to-anki.py + +- `CARD_RE` now matches a trailing org tag block (`^\*\*\s+(.*?)\s+(:[A-Za-z0-9_@#%:]+:)\s*$`); a heading is a card when `drill` is among its tags. Other org tags ride along as Anki tags next to the section tag. +- Card body is now bounded by any L1/L2 heading (`HEADING_RE = ^\*{1,2}\s`) instead of only `* ` or a drill heading. +- New `--tag-filter <tag>`: emit only cards carrying that org tag (e.g. `--tag-filter fundamental` → the 100-card subset). +- New `--guid-salt <s>`: salt note GUIDs so a derived subset deck gets its own GUID space. Without it, the subset's notes share fronts with the full deck, Anki dedupes on GUID, and the subset deck imports empty. Default (no salt) is unchanged — `guid_for(front)` — so the existing deepsat deck's GUIDs and SRS state are untouched. + +Generation used: `flashcard-to-anki.py deepsat.org --tag-filter fundamental --deck "DeepSat Fundamentals" --guid-salt fundamentals`. + +## flashcard-stats.py + +- Same `CARD_RE` broadening + `HEADING_RE` body bound, and the card guard now checks `drill` membership in the tag block. Verified: full deck still counts 465 after 100 cards were multi-tagged. + +## Companion files to reconcile + +- Both rulesets copies: `~/code/rulesets/.ai/scripts/` and `~/code/rulesets/claude-templates/.ai/scripts/` (the synced source). +- `claude-templates/.ai/scripts/tests/flashcard-sync.bats` — worth adding a multi-tag case (a `:foo:drill:` heading still parses; `--tag-filter foo` returns only those) so this doesn't regress. +- Regression checked locally: full deck parses to 465 with and without the change; `--tag-filter fundamental` returns exactly 100. diff --git a/docs/design/2026-06-17-flashcard-multitag-stats.py b/docs/design/2026-06-17-flashcard-multitag-stats.py new file mode 100755 index 0000000..3c984e7 --- /dev/null +++ b/docs/design/2026-06-17-flashcard-multitag-stats.py @@ -0,0 +1,332 @@ +#!/usr/bin/env python3 +"""Inventory + authoring-quality checks for an org-drill deck source file. + +Reports counts and flags two tiers of issue. + +Blocking WARNs (exit 1): +- PROPERTIES drawer count not matching card count +- Cards missing :ID: (risks SRS-state loss across rewrites) +- `*** Answer` sub-headers (should be 0 per flashcard-review.org) +- Non-prompt headings (topic-as-heading not yet rewritten) +- #+TITLE missing, or carrying source-tool jargon ("org-drill") +- Answer leakage: a card whose question echoes most of its own answer + (Source: citation lines and created-date lines are excluded from the + overlap, and range/category cards that recall numbers are exempted) +- Duplicate / near-duplicate fronts (interference between confusable cards) + +Non-blocking NOTEs (exit unaffected): +- Overloaded backs (long answer — candidate to split into atomic cards) +- List-shaped backs (enumeration — candidate to split or use overlapping cloze) +- Binary yes/no prompts (low retrieval effort — candidate to reformulate) + +Exits 0 when no blocking warnings are present, 1 otherwise, 2 on bad usage. +Use as a gate before regenerating the Anki deck or running flashcard-sync. + +The fuzzy checks (leakage, duplicate, overloaded) are tuned by the LEAKAGE_* +and BACK_WORD_LIMIT constants below; loosen them if a real deck trips false +positives. + +Usage: + flashcard-stats.py <file.org> +""" +from __future__ import annotations + +import re +import sys +from pathlib import Path + +# A level-2 card heading carries a trailing org tag block that includes +# `drill`. The block may hold more than one tag (e.g. ":fundamental:drill:"), +# so match the whole block and check membership rather than pinning :drill: +# as the literal last tag. +CARD_RE = re.compile(r"^\*\*\s+(.+?)\s+(:[A-Za-z0-9_@#%:]+:)\s*$") +HEADING_RE = re.compile(r"^\*{1,2}\s") +ANSWER_RE = re.compile(r"^\*\*\*\s+Answer\b") +PROP_START_RE = re.compile(r"^\s*:PROPERTIES:\s*$") +PROP_END_RE = re.compile(r"^\s*:END:\s*$") +ID_RE = re.compile(r"^\s*:ID:\s+(\S+)\s*$") +TITLE_RE = re.compile(r"^#\+TITLE:\s*(.+?)\s*$", re.IGNORECASE) +SOURCE_TOOL_RE = re.compile(r"\borg[-\s]?drill\b", re.IGNORECASE) +PLANNING_RE = re.compile(r"^\s*(SCHEDULED|DEADLINE|CLOSED):\s") +SOURCE_LINE_RE = re.compile(r"^\s*source:\s", re.IGNORECASE) +CREATED_LINE_RE = re.compile(r"^\s*:?created:?\s", re.IGNORECASE) +RANGE_RE = re.compile(r"\d[^\n]*[-–—]\s*\d") +THRESHOLD_RE = re.compile(r"[<>≤≥]\s*\d") +BULLET_RE = re.compile(r"^\s*([-+*]|\d+[.)])\s+") +BINARY_LEAD_RE = re.compile( + r"^\s*(is|are|was|were|does|do|did|can|could|should|would|will|has|have|had)\b", + re.IGNORECASE, +) + +# A heading qualifies as "prompt form" if it contains `?` or starts with one of +# these imperative verbs (directive prompts like "Spell these out" and +# "Introduce yourself" are valid even without `?`). +IMPERATIVE_VERBS = frozenset({ + "spell", "describe", "explain", "name", "list", "give", + "show", "tell", "define", "compare", "identify", "outline", + "introduce", "walk", "state", "recite", "recall", "summarize", +}) + +# Function words ignored when comparing a question against its answer. +STOPWORDS = frozenset({ + "the", "a", "an", "is", "are", "was", "were", "of", "to", "in", "on", + "for", "and", "or", "with", "what", "who", "whom", "when", "where", "why", + "how", "which", "does", "do", "did", "tell", "me", "about", "their", "this", + "that", "it", "as", "at", "by", "be", "your", "you", "they", "them", +}) + +# Tuning knobs for the fuzzy checks. +LEAKAGE_RATIO = 0.8 # share of a question's content words echoed in its answer +LEAKAGE_MIN_WORDS = 3 # ignore very short questions, where overlap is noise +BACK_WORD_LIMIT = 60 # words on a card back before it's flagged as overloaded + + +def is_prompt_form(heading: str) -> bool: + """True if the heading reads as a question or imperative prompt.""" + if "?" in heading: + return True + first_word = heading.split(None, 1)[0].lower().rstrip(":,;") + return first_word in IMPERATIVE_VERBS + + +def content_words(text: str) -> set[str]: + """Lowercased alphanumeric tokens of length >= 3, minus stopwords.""" + return {w for w in re.findall(r"[a-z0-9]+", text.lower()) + if len(w) >= 3 and w not in STOPWORDS} + + +def leakage_ratio(heading: str, body: str) -> float: + """Fraction of the question's content words that reappear in the answer. + + A high ratio means the answer is largely restated in the question, so the + card can be answered by recognition rather than recall. Returns 0.0 for a + question with fewer than LEAKAGE_MIN_WORDS content words, where overlap is + just noise. + """ + hw = content_words(heading) + if len(hw) < LEAKAGE_MIN_WORDS: + return 0.0 + return len(hw & content_words(body)) / len(hw) + + +def prose_body(body: str) -> str: + """Body with Source: citation and created-date lines removed. + + Those lines are metadata, not the answer. A Source line's URL slug often + repeats the question's words, and a created date is bookkeeping — neither + should count toward answer-leakage overlap. + """ + return "\n".join( + ln for ln in body.splitlines() + if not SOURCE_LINE_RE.match(ln) and not CREATED_LINE_RE.match(ln) + ) + + +def has_distinct_numeric_recall(heading: str, body: str) -> bool: + """True if the answer carries numeric ranges/thresholds the question lacks. + + A range/category card ("What are the HbA1c ranges across normal, + prediabetes, and diabetes?") echoes its categories in the answer, but the + recalled content is the numbers, which the question doesn't give away — so + high word overlap isn't leakage. + """ + body_nums = bool(RANGE_RE.search(body) or THRESHOLD_RE.search(body)) + head_nums = bool(RANGE_RE.search(heading) or THRESHOLD_RE.search(heading)) + return body_nums and not head_nums + + +def is_leaky(heading: str, body: str) -> bool: + """True if a card leaks its answer, after excluding citation lines and + numeric-recall (range/category) cards.""" + prose = prose_body(body) + if leakage_ratio(heading, prose) < LEAKAGE_RATIO: + return False + return not has_distinct_numeric_recall(heading, prose) + + +def normalize_heading(heading: str) -> str: + """Collapse a heading to a comparison key (lowercase, alnum + single spaces).""" + return re.sub(r"\s+", " ", re.sub(r"[^a-z0-9 ]", " ", heading.lower())).strip() + + +def is_binary_prompt(heading: str) -> bool: + """True for yes/no or 'A or B' prompts, which need little retrieval effort.""" + if BINARY_LEAD_RE.match(heading): + return True + return bool(re.search(r"\bor\b", heading, re.IGNORECASE)) and heading.rstrip().endswith("?") + + +def back_word_count(body: str) -> int: + return len(body.split()) + + +def is_list_back(body: str) -> bool: + """True if the answer body is mostly an org list (an enumeration card).""" + lines = [ln for ln in body.splitlines() if ln.strip()] + if len(lines) < 2: + return False + bullets = sum(1 for ln in lines if BULLET_RE.match(ln)) + return bullets >= 2 and bullets * 2 >= len(lines) + + +def parse_cards(lines: list[str]) -> tuple[list[dict], int]: + """Parse :drill: cards from org lines. + + Returns (cards, prop_count). Each card is a dict with heading, has_id, + has_answer, and body (the answer text with PROPERTIES drawers, planning + lines, and `*** Answer` headers removed, approximating the rendered back). + """ + cards: list[dict] = [] + prop_count = 0 + i = 0 + n = len(lines) + while i < n: + m = CARD_RE.match(lines[i]) + if not m or "drill" not in [t for t in m.group(2).split(":") if t]: + i += 1 + continue + heading = m.group(1).strip() + i += 1 + has_id = False + has_answer = False + in_drawer = False + body_lines: list[str] = [] + while i < n: + line = lines[i] + if HEADING_RE.match(line): + break + if PROP_START_RE.match(line): + prop_count += 1 + in_drawer = True + elif in_drawer and PROP_END_RE.match(line): + in_drawer = False + elif in_drawer: + if ID_RE.match(line): + has_id = True + elif ANSWER_RE.match(line): + has_answer = True + elif PLANNING_RE.match(line): + pass + else: + body_lines.append(line) + i += 1 + cards.append({ + "heading": heading, + "has_id": has_id, + "has_answer": has_answer, + "body": "\n".join(body_lines).strip(), + }) + return cards, prop_count + + +def find_duplicate_fronts(cards: list[dict]) -> list[tuple[str, str]]: + """Return (first, dup) heading pairs that normalize to the same key.""" + seen: dict[str, str] = {} + dups: list[tuple[str, str]] = [] + for c in cards: + key = normalize_heading(c["heading"]) + if not key: + continue + if key in seen: + dups.append((seen[key], c["heading"])) + else: + seen[key] = c["heading"] + return dups + + +def main() -> int: + if len(sys.argv) != 2: + print(f"usage: {sys.argv[0]} <file.org>", file=sys.stderr) + return 2 + + path = Path(sys.argv[1]).expanduser().resolve() + if not path.is_file(): + print(f"error: {path} not found", file=sys.stderr) + return 2 + + lines = path.read_text(encoding="utf-8").splitlines() + + title: str | None = None + for line in lines[:20]: + m = TITLE_RE.match(line) + if m: + title = m.group(1).strip() + break + + cards, prop_count = parse_cards(lines) + + no_id = [c["heading"] for c in cards if not c["has_id"]] + not_prompt = [c["heading"] for c in cards if not is_prompt_form(c["heading"])] + answer_count = sum(1 for c in cards if c["has_answer"]) + leaky = [c["heading"] for c in cards if is_leaky(c["heading"], c["body"])] + dups = find_duplicate_fronts(cards) + overloaded = [c["heading"] for c in cards if back_word_count(c["body"]) > BACK_WORD_LIMIT] + listy = [c["heading"] for c in cards if is_list_back(c["body"])] + binary = [c["heading"] for c in cards if is_binary_prompt(c["heading"])] + + print(f"{path.name} — drill deck stats") + print() + print(f"Deck title: {title if title else '(no #+TITLE)'}") + print(f"Cards: {len(cards)}") + drawer_status = "match" if prop_count == len(cards) else f"mismatch (expected {len(cards)})" + print(f"PROPERTIES drawers: {prop_count} ({drawer_status})") + print(f"*** Answer sub-headers: {answer_count} ({'clean' if answer_count == 0 else 'workflow violation'})") + print(f"Cards missing :ID:: {len(no_id)}") + print(f"Cards with non-prompt heading: {len(not_prompt)}") + print(f"Cards with possible answer leakage: {len(leaky)}") + print(f"Duplicate / near-duplicate fronts: {len(dups)}") + print() + + warnings = 0 + + def emit_list(items: list[str]) -> None: + for h in items[:5]: + print(f" - {h}") + if len(items) > 5: + print(f" - ... and {len(items) - 5} more") + + def warn(msg: str, items: list[str] | None = None) -> None: + nonlocal warnings + warnings += 1 + print(f"WARN: {msg}") + if items: + emit_list(items) + + def note(msg: str, items: list[str] | None = None) -> None: + print(f"NOTE: {msg}") + if items: + emit_list(items) + + if title is None: + warn("no #+TITLE: line found; deck name will fall back to the file basename") + elif SOURCE_TOOL_RE.search(title): + warn(f"#+TITLE contains source-tool jargon ('{title}'); the deck name shows in Anki — drop 'Org-Drill' for a name that reads well on the consumption side") + if answer_count: + warn(f"{answer_count} cards have *** Answer sub-headers (drop per flashcard-review.org)") + if prop_count != len(cards): + warn(f"PROPERTIES count {prop_count} does not match card count {len(cards)}") + if no_id: + warn(f"{len(no_id)} cards missing :ID:; losing identity risks SRS-state loss across rewrites", no_id) + if not_prompt: + warn(f"{len(not_prompt)} cards have non-prompt headings (no '?' and no imperative-verb start); likely topic-as-heading not yet rewritten", not_prompt) + if leaky: + warn(f"{len(leaky)} cards may leak their answer (question echoes >= {int(LEAKAGE_RATIO * 100)}% of its own answer's key words); reformulate so the answer is recalled, not recognized", leaky) + if dups: + warn(f"{len(dups)} duplicate / near-duplicate fronts (interference between confusable cards); disambiguate or merge", + [f"{a} == {b}" for a, b in dups]) + + if overloaded: + note(f"{len(overloaded)} cards have a long answer (> {BACK_WORD_LIMIT} words); candidates to split into atomic cards", overloaded) + if listy: + note(f"{len(listy)} cards have a list-shaped answer; enumeration cards recall poorly — candidates to split or use overlapping cloze", listy) + if binary: + note(f"{len(binary)} cards are binary (yes/no or 'A or B'); low retrieval effort — candidates to reformulate open-ended", binary) + + if warnings == 0: + print("clean (with non-blocking notes above)" if (overloaded or listy or binary) else "clean") + return 0 + return 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/docs/design/2026-06-17-flashcard-multitag-to-anki.py b/docs/design/2026-06-17-flashcard-multitag-to-anki.py new file mode 100755 index 0000000..3764acf --- /dev/null +++ b/docs/design/2026-06-17-flashcard-multitag-to-anki.py @@ -0,0 +1,294 @@ +#!/usr/bin/env -S uv run --script +# /// script +# requires-python = ">=3.11" +# dependencies = [ +# "genanki>=0.13", +# ] +# /// +"""Convert an org-drill file into an Anki .apkg deck. + +Parses org-drill structure: + - Top-level "* Section" headings become tags on every card under them. + - Each "** Card name :drill:" entry becomes a card. Front = heading + text (sans the org tag block). Back = entry body with newlines + converted to <br>. + +A card heading may carry more than one org tag (e.g. +"** Question :fundamental:drill:"). Any heading whose trailing tag block +includes `drill` is a card; the other org tags ride along as Anki tags +next to the section tag. Pass --tag-filter <tag> to emit only the cards +carrying that org tag (e.g. a curated "fundamentals" subset). + +Deck name defaults to the input basename, case preserved. Deck and model +IDs are derived from the deck name via stable hash so re-importing the +same deck updates existing cards instead of duplicating them. + +Note GUIDs default to a hash of the card front, so re-running against the +same source preserves SRS state. A derived subset deck (one built with +--tag-filter) should pass --guid-salt so its notes get a distinct GUID +space and Anki treats it as a separate deck rather than merging its cards +into a full deck that shares the same fronts. + +Output defaults to ~/sync/phone/anki/<input-basename>.apkg. The .apkg is +a mobile-Anki artifact the phone picks up from its sync dir, so it lands +there rather than next to the org source. + +Usage: + flashcard-to-anki.py <input.org> + flashcard-to-anki.py <input.org> --deck "My Deck Name" + flashcard-to-anki.py <input.org> --output /path/to/deck.apkg + flashcard-to-anki.py <input.org> --tag-filter fundamental \ + --deck "DeepSat Fundamentals" --guid-salt fundamentals + +Requires genanki, which uv resolves automatically via the PEP 723 +script metadata above. No venv or system install needed. +""" +from __future__ import annotations + +import argparse +import hashlib +import re +import sys +from pathlib import Path + +import genanki + +# 32-bit integer space genanki accepts. Start above the conventional +# "user model" floor so collisions with hand-written decks stay +# unlikely. +ID_BASE = 1_500_000_000 +ID_RANGE = 500_000_000 + + +def stable_id(name: str, salt: str) -> int: + """Derive a deterministic 32-bit id from `name` and a `salt`. + + Same (name, salt) pair always returns the same id, so re-running + against the same source produces a stable deck/model id pair and + Anki imports update existing cards in place rather than duplicating. + """ + h = hashlib.sha256(f"{salt}:{name}".encode()).hexdigest() + return ID_BASE + (int(h[:8], 16) % ID_RANGE) + + +def make_model(deck_name: str) -> genanki.Model: + return genanki.Model( + stable_id(deck_name, "model"), + f"{deck_name} (Craig)", + fields=[{"name": "Front"}, {"name": "Back"}], + templates=[ + { + "name": "Card 1", + "qfmt": "{{Front}}", + "afmt": '{{FrontSide}}<hr id="answer">{{Back}}', + } + ], + css=( + ".card { font-family: sans-serif; font-size: 18px; " + "color: #222; background: #fafafa; line-height: 1.45; }\n" + "hr#answer { margin: 14px 0; }\n" + ), + ) + + +def section_to_tag(title: str) -> str: + return re.sub(r"[^a-z0-9]+", "-", title.lower()).strip("-") + + +def escape_html(s: str) -> str: + return ( + s.replace("&", "&") + .replace("<", "<") + .replace(">", ">") + ) + + +def strip_org_metadata(body_lines: list[str]) -> list[str]: + """Drop :PROPERTIES: drawers, planning lines, and created-date lines. + + Org-drill needs these in the source file (SRS state lives in the + PROPERTIES drawer; SCHEDULED carries the next-review date), but they + are noise on the back of an Anki card. A created/added date never + belongs on a card, so a stray "Created:" or ":CREATED:" body line is + dropped too. + """ + cleaned: list[str] = [] + in_drawer = False + planning_re = re.compile(r"^\s*(SCHEDULED|DEADLINE|CLOSED):\s") + created_re = re.compile(r"^\s*:?created:?\s", re.IGNORECASE) + drawer_start_re = re.compile(r"^\s*:PROPERTIES:\s*$") + drawer_end_re = re.compile(r"^\s*:END:\s*$") + for line in body_lines: + if in_drawer: + if drawer_end_re.match(line): + in_drawer = False + continue + if drawer_start_re.match(line): + in_drawer = True + continue + if planning_re.match(line) or created_re.match(line): + continue + cleaned.append(line) + return cleaned + + +# A level-2 heading carrying a trailing org tag block. Group 1 is the +# front text, group 2 the colon-delimited tag block (e.g. ":fundamental:drill:"). +CARD_RE = re.compile(r"^\*\*\s+(.+?)\s+(:[A-Za-z0-9_@#%:]+:)\s*$") +# Any level-1 or level-2 heading — used to bound a card's body. +HEADING_RE = re.compile(r"^\*{1,2}\s") +SECTION_RE = re.compile(r"^\*\s+(.+?)\s*$") + + +def parse( + org_text: str, tag_filter: str | None = None +) -> list[tuple[str, str, list[str]]]: + """Return [(front, back_html, anki_tags), ...] for every :drill: card. + + A card is any level-2 heading whose trailing org tag block includes + `drill`. Additional org tags become Anki tags alongside the section + tag. When `tag_filter` is set, only cards carrying that org tag are + returned. + """ + cards: list[tuple[str, str, list[str]]] = [] + current_section: str | None = None + + lines = org_text.splitlines() + i = 0 + while i < len(lines): + line = lines[i] + + sec = SECTION_RE.match(line) + if sec: + current_section = sec.group(1).strip() + i += 1 + continue + + m = CARD_RE.match(line) + tags = [t for t in m.group(2).split(":") if t] if m else [] + if m and "drill" in tags: + front = m.group(1).strip() + body_lines: list[str] = [] + i += 1 + while i < len(lines): + nxt = lines[i] + if HEADING_RE.match(nxt): + break + body_lines.append(nxt) + i += 1 + body_lines = strip_org_metadata(body_lines) + while body_lines and not body_lines[0].strip(): + body_lines.pop(0) + while body_lines and not body_lines[-1].strip(): + body_lines.pop() + back_html = "<br>".join(escape_html(ln) for ln in body_lines) + + org_tags = [t for t in tags if t != "drill"] + if tag_filter and tag_filter not in org_tags: + continue + anki_tags: list[str] = [] + if current_section: + anki_tags.append(section_to_tag(current_section)) + anki_tags.extend(org_tags) + if not anki_tags: + anki_tags = ["drill"] + cards.append((front, back_html, anki_tags)) + continue + + i += 1 + + return cards + + +def build( + cards: list[tuple[str, str, list[str]]], + deck_name: str, + guid_salt: str | None = None, +) -> genanki.Deck: + deck = genanki.Deck(stable_id(deck_name, "deck"), deck_name) + model = make_model(deck_name) + for front, back, tags in cards: + guid = ( + genanki.guid_for(guid_salt, front) + if guid_salt + else genanki.guid_for(front) + ) + note = genanki.Note( + model=model, + fields=[front, back], + tags=tags, + guid=guid, + ) + deck.add_note(note) + return deck + + +def default_deck_name(input_path: Path) -> str: + return input_path.stem + + +def default_output_path(input_path: Path) -> Path: + anki_dir = Path.home() / "sync" / "phone" / "anki" + return anki_dir / f"{input_path.stem}.apkg" + + +def main() -> int: + parser = argparse.ArgumentParser( + description="Convert an org-drill file into an Anki .apkg deck.", + ) + parser.add_argument( + "input", + type=Path, + help="Path to the org-drill source file.", + ) + parser.add_argument( + "--deck", + help="Deck name. Defaults to the input basename.", + ) + parser.add_argument( + "--output", + type=Path, + help="Output .apkg path. Defaults to " + "~/sync/phone/anki/<input-basename>.apkg.", + ) + parser.add_argument( + "--tag-filter", + help="Emit only cards carrying this org tag (e.g. 'fundamental').", + ) + parser.add_argument( + "--guid-salt", + help="Salt note GUIDs with this string so a derived subset deck " + "gets its own GUID space and Anki keeps it separate from a " + "full deck sharing the same card fronts.", + ) + args = parser.parse_args() + + input_path: Path = args.input.expanduser().resolve() + if not input_path.is_file(): + print(f"error: {input_path} not found", file=sys.stderr) + return 1 + + org_text = input_path.read_text(encoding="utf-8") + deck_name = args.deck or default_deck_name(input_path) + output_path: Path = (args.output or default_output_path(input_path)).expanduser().resolve() + output_path.parent.mkdir(parents=True, exist_ok=True) + + cards = parse(org_text, tag_filter=args.tag_filter) + if not cards: + if args.tag_filter: + print( + f"error: no :drill: cards tagged :{args.tag_filter}: in {input_path}", + file=sys.stderr, + ) + else: + print(f"error: no :drill: cards found in {input_path}", file=sys.stderr) + return 1 + + deck = build(cards, deck_name, guid_salt=args.guid_salt) + genanki.Package(deck).write_to_file(str(output_path)) + print(f"wrote {output_path} ({len(cards)} cards, deck '{deck_name}')") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/docs/design/2026-06-17-ntfy-agent-comms-proposal.org b/docs/design/2026-06-17-ntfy-agent-comms-proposal.org new file mode 100644 index 0000000..0961b47 --- /dev/null +++ b/docs/design/2026-06-17-ntfy-agent-comms-proposal.org @@ -0,0 +1,89 @@ +#+TITLE: Proposal — Promote the ntfy phone channel into a general agent-comms tool +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-17 + +* Why this is in rulesets' inbox + +The home project built a working, private phone-notification channel for Craig on 2026-06-17 (self-hosted ntfy over Tailscale). Craig wants rulesets to consider promoting it from a one-way notification system into a *general two-way communication tool* between him and his agents — and, critically, to move it off pure polling toward event-driven delivery (an inbound message can trigger an action or notify an agent, not just sit in a queue waiting to be polled). + +This is a proposal, not a change to anything rulesets owns. It documents exactly what exists, what ntfy makes possible, and the open design decisions rulesets would own. It also relates directly to the cross-agent-comms scripts that were retired from the templates in this same session — ntfy may be the transport layer that effort was missing. + +* Part 1 — What exists now (as-built, verified) + +- *Server:* ntfy in Docker on =ratio= at =~/docker/ntfy/= (=compose.yml= + =server.yml=, =data/= volume, =restart=unless-stopped=, healthcheck on =/v1/health=). Listens container :80, published to =127.0.0.1:2586=. +- *Tailnet exposure:* =tailscale serve --bg --http=80 http://127.0.0.1:2586= → reachable at =http://ratio.tailf3bb8c.ts.net= (tailnet only, no public exposure). Disable with =tailscale serve --http=80 off=. +- *Transport security:* plain HTTP, but every byte rides inside the WireGuard mesh (Tailscale), so it is encrypted end to end. The Tailscale account does not support TLS certs, and TLS would be redundant on the tailnet anyway. If ever exposed publicly, TLS + stronger auth become mandatory. +- *Auth:* =auth-default-access: deny-all=. User =cj= has read-write on topics =claude= and =infra=. Anonymous is denied — verified 403 on both publish AND subscribe without a token. Token =tk_…= never expires. App login is username =cj= + a short password. +- *Phone:* Pixel 6, ntfy F-Droid build (no Firebase / Google Play Services), WebSocket instant delivery. Already on the tailnet. +- *Publisher wrapper:* =~/.local/bin/phone-notify= (on ratio only). Reads =~/.config/phone-notify/config= (chmod 600: URL, token, topic). Supports =-t/--title=, =-p/--priority=, =-T/--tags=, =--topic=, =--click=, =--url=. +- *Verified two-way:* agent → phone push lands instantly; phone → publish to the topic lands on the server and is readable by the agent (Craig sent "It did, in fact, land." from the app and the agent polled and saw it). + +* Part 2 — The ntfy building blocks rulesets can use + +** Publish (agent → phone), already wired +- =curl -H "Authorization: Bearer <token>" -d "msg" <url>/<topic>= or =phone-notify=. +- Rich features available, unused so far: =Priority= (1-5), =Tags= (emoji/keywords), =Click= (URL opened on tap), =Actions= (tappable buttons — =view= a URL, =http= fire a request, =broadcast= an Android intent), =Attach= (files/images), =Markdown=, scheduled/delayed delivery (=At:= / =Delay:= header), and email/call forwarding. + +** Read (agent ← phone) +- One-shot poll, all cached: =GET /<topic>/json?poll=1= (needs the token). +- Only-new since a point: =?since=<id|timestamp|duration>= (e.g. =?since=5m= or =?since=<last-seen-id>=). This is the basis of a =phone-recv= helper that prints only messages newer than the last one seen. +- Cache window is 12h (=cache-duration= in server.yml), so on-demand polling never misses a recent message. + +** Subscribe with side effects (the event-driven primitive) +- =ntfy subscribe <topic> '<command>'= holds a persistent connection (WebSocket / JSON stream) and runs =<command>= for *every* inbound message, with fields exposed as environment variables (=$message=, =$title=, =$topic=, =$tags=, =$priority=, etc.). +- This is the answer to "not all polling": a long-running subscriber reacts the instant a message arrives. + +* Part 3 — Making it event-driven (Craig's core ask) + +Three tiers, increasing capability and difficulty: + +** Tier A — Subscriber daemon routes inbound (clean, doable now) +A systemd *user* service on ratio (always-on): +#+begin_src +ntfy subscribe --since=<last> claude /usr/local/bin/ntfy-inbound-handler +#+end_src +=ntfy-inbound-handler= classifies the message and routes it: +- Append to a watched queue (a project =inbox/= or a dedicated comms file) → the next agent session picks it up at a task boundary (already in protocols: inbox check at task boundaries). +- Fire desktop =notify= so a human at a screen sees it immediately. +- Tag-based dispatch: =#task= → file as a TODO; =#infra= → infra queue; etc. + +This gets us instant reaction with zero polling, and it degrades gracefully — if nothing is listening, the message still sits in the queue. + +** Tier B — Inbound spawns a *new* agent session +The handler invokes the =ai= launcher (or a scheduled/cron Claude run) to process the message autonomously. An inbound phone text becomes an agent action — "remind me to X" from anywhere, "what's the status of Y", "approve the pending commit". This is where it stops being a notifier and becomes a remote control for the agent fleet. Ties into the harness cron/schedule features and the retired cross-agent-comms intent. + +** Tier C — Notify / interrupt a *live* agent session (hardest, harness-dependent) +A turn-based session has no native external interrupt. Honest options to explore: +- The session runs a background subscriber/poll loop; the harness re-invokes the agent when backgrounded work emits or completes (the background-Bash + Monitor + ScheduleWakeup / =/loop= dynamic-pacing mechanisms). +- A =/loop= that polls the topic every N seconds (still polling, but bounded and cheap). +- Whatever the harness exposes for inbound push into a live session (e.g. a RemoteTrigger / inbound-PushNotification path) — needs experimentation. + +Recommendation: ship Tier A first (high value, low risk), prototype Tier B, treat Tier C as research. + +* Part 4 — The general-comms vision (beyond notifications) + +- *Channels as topics:* =claude= (agent ↔ Craig), =infra= (server/health/backup alerts — the DEGRADED-pool class), per-project topics, a cross-agent topic. +- *Bidirectional chat:* Craig texts his agent from anywhere over Tailscale; the agent replies. Effectively private, self-hosted "SMS with your agent." +- *Approval buttons:* the publish =Actions= feature can render Approve / Reject buttons on the phone. For the commits.md approval gates (commit message, PR body, PR review) when Craig is away from the desk, a tapped button fires a webhook the handler turns into "proceed." This is a concrete, high-value use. +- *Attachments:* agent sends a generated screenshot/report to the phone; Craig sends a photo to the agent. + +* Part 5 — What rulesets would own / decide + +1. *Canonical tooling:* promote =phone-notify= (send) and add =phone-recv= (check-since) as rulesets bin scripts, synced to all machines via dotfiles/templates. Today =phone-notify= lives only on ratio. +2. *Config + secret convention:* where the server URL + token live per machine (=~/.config/phone-notify/config= chmod 600 today), and whether the token should be a rulesets-managed GPG-encrypted secret distributed via dotfiles. +3. *The subscriber daemon:* a reference =ntfy-inbound-handler= + a systemd user-unit template, plus the routing convention (tags → destinations). +4. *Protocol conventions:* topic naming, a message format/tag vocabulary for routing, and how inbound maps to the existing =inbox/= and (retired) cross-agent-comms protocols. +5. *Harness integration:* how — if at all — to wake or notify a live/new agent session on inbound. The Tier C research. +6. *Relationship to cross-agent-comms:* decide whether ntfy is the transport that replaces the just-retired scripts, and whether agent↔agent messaging rides the same server (a dedicated topic) or stays separate. + +* Part 6 — Open questions + +- Multi-machine token distribution (per-machine config vs encrypted-in-dotfiles). +- Daemon placement: one always-on subscriber on ratio vs per-machine subscribers. +- Inbound integration with the existing inbox + the retired cross-agent protocols. +- Live-session interrupt feasibility (entirely harness-dependent — needs a spike). +- Whether agent↔agent comms and agent↔Craig comms share a server or are isolated. + +* Companion artifact + +The full as-built runbook (concrete values, server.yml, the verification checklist, the security model) lives in the home project at =working/phone-notifications/spec.org=. This proposal is the forward-looking half; that file is the operational record of what was deployed. diff --git a/docs/design/2026-06-18-triage-intake-phone-push-note.org b/docs/design/2026-06-18-triage-intake-phone-push-note.org new file mode 100644 index 0000000..2f6502b --- /dev/null +++ b/docs/design/2026-06-18-triage-intake-phone-push-note.org @@ -0,0 +1,11 @@ +#+TITLE: WORKFLOW UPDATE — triage-intake.org auto mode gains a phone +#+SOURCE: from work +#+DATE: 2026-06-18 15:15:47 -0500 + +WORKFLOW UPDATE — triage-intake.org auto mode gains a phone (ntfy) delivery step. Supersedes/consolidates the earlier 2026-06-18-1512 send. + +WHAT we did: added a new subsection 'Push each sweep to Craig's phone (ntfy) — the primary delivery' under 'Trigger and delivery' in the auto-mode section. It makes phone-notify (the self-hosted ntfy channel over Tailscale) the primary delivery for every auto-mode sweep, pushing the fuller End-of-sweep output (per-source deltas + open-PR/Linear state + the awaiting-ack list + a one-line verdict + a timestamp; SCAN FAILED banner if any source failed), and polling phone-recv each sweep for Craig's replies. Falls back to inline if phone-notify is absent. + +WHY: auto mode exists precisely for when Craig is away from the desk (out of office for a while, on vacation — which is the case right now, traveling 6/17-24). An inline-only report is useless if he is not at the screen; the whole point is reaching his phone. We ran it live this way all day and it worked, so we codified the delivery rather than re-deriving it each time. Craig confirmed this is his standard pattern: he starts auto-triage before leaving the office / on vacation. + +Companion: the reference_phone_notify_ntfy_channel harness memory documents the channel + the high-bar caveat. A separate task is producing standalone ntfy setup instructions (install + start/stop the service) so the channel can be brought up on other machines. diff --git a/docs/design/2026-06-18-triage-intake-phone-push-workflow.org b/docs/design/2026-06-18-triage-intake-phone-push-workflow.org new file mode 100644 index 0000000..a0bb416 --- /dev/null +++ b/docs/design/2026-06-18-triage-intake-phone-push-workflow.org @@ -0,0 +1,427 @@ +#+TITLE: Triage Intake Workflow (Engine) +#+AUTHOR: Craig Jennings +#+DATE: 2026-05-01 + +* Summary + + +Lightweight, between-meetings sweep across whatever sources are plugged in — email, calendar, chat, open PRs, ticketing. Classifies what came in since the last check (Action / FYI / Noise-keep / Noise-trash), produces a single synthesized summary, and offers to execute the routine actions (trash, mark-read, star, respond, merge, attachment fetch). + +Think of it as the ER intake queue: every new message, invite, and PR notification is a "patient" walking through the door. This workflow is the triage nurse looking at the queue and telling Craig what needs attention now, what's just FYI, and what can be cleared. + +*This file is the engine.* It carries no sources of its own. Every source it scans comes from a *source plugin* — a =triage-intake.<source>.org= file the engine loads at Phase 0. The engine is source-agnostic and project-agnostic; the project- and account-specific knowledge lives entirely in the plugins. To add a source, drop a plugin file. To change one, edit its plugin. Never wire a source into this file. + +Distinct from =daily-prep.org=: +- *daily-prep* — heavier, once daily, builds the day's plan + standup brief + meeting prep + time blocks. +- *triage-intake* — fast, repeatable, just answers "what's new since last check?" + + +Quick contract — what it does: fans out across source plugins, classifies every item into Action / FYI / Noise-keep / Noise-trash, synthesizes one deduped summary, writes each Action item to =todo.org= as a =:quick:reactive:= task, and executes star/mark-read/trash on confirmation. + +** When to Use This Workflow + +Trigger phrases: + +- "Run a triage-intake" +- "Triage intake" +- "What's new" / "What's new since I last checked" +- "Do a sweep" / "Do a triage sweep" +- "Check email, calendar, and PRs" + +Typical timing: + +- Between meetings (1-2 minute glance) +- After a long focused-work block +- Before context-switching to a new task +- When ambient anxiety about "did I miss something?" creeps in + +Do *not* use when running daily-prep — daily-prep already does this as Phase 3. + + +* Execution + +** Phase 0 — Load source plugins (MANDATORY — do not skip) + +The engine has no sources baked in. It discovers them by globbing *two* directories, and you MUST glob *both*: + +#+begin_src bash +ls .ai/workflows/triage-intake.*.org .ai/project-workflows/triage-intake.*.org 2>/dev/null +#+end_src + +- =.ai/workflows/triage-intake.*.org= — *general* source plugins, template-synced (personal Gmail, personal calendar, cmail/Proton, Telegram, personal GitHub PRs). +- =.ai/project-workflows/triage-intake.*.org= — *PROJECT-SPECIFIC* source plugins, never synced, owned by this project (e.g. a work project's Linear, work Gmail, work Slack, enterprise-GitHub PRs). + +⚠ *THE #1 FAILURE MODE — read this twice.* Globbing only =.ai/workflows/= and silently missing every project plugin. If you skip =.ai/project-workflows/=, the sweep runs with *half its sources* and Craig never learns what it dropped — the omission is invisible, because a missing source looks identical to a quiet source in the output. There is no error, no empty block, no warning. The sweep just lies by omission. *Glob both directories. Always.* + +The glob exclude is automatic: =triage-intake.*.org= matches the plugins but not this engine file (=triage-intake.org= has no second dot-segment), so the engine never loads itself. + +After globbing, for each plugin file: +1. Read it. +2. Evaluate its =ENABLED= precondition. If false, *announce the skip with its reason* ("skipping linear — mcp__linear not present") and move on. +3. The surviving set is the source list for Phases A-D. + +*Announce the loaded set before scanning* so the omission can't hide: + +#+begin_example +Loaded 5 source plugins: + general: personal-gmail, personal-calendar, cmail, github-prs + project: deepsat-gmail + skipped: linear (mcp__linear not present) +#+end_example + +If the project directory glob returns nothing, say so explicitly ("no project plugins in .ai/project-workflows/") rather than staying silent — silence is indistinguishable from forgetting to look. + + +** Approach: Phases A → D + +*** Phase A: Fan-out (one parallel batch) + +Issue every enabled source's =Scan= command in a single message, with the anchor substituted in each source's declared format. They have no dependencies and benefit from running concurrently. + +Per-source subagent escalation: if a source's scan is expected to return more than its =SUBAGENT_OVER= count (e.g. personal Gmail after a multi-day gap), dispatch a subagent for that source. The subagent applies Phase B classification and returns the synthesized buckets, not the raw item list. + +*** Phase B: Classify per source (shared four-bucket model) + +Every item lands in one bucket. Plugins refine these with source-specific bias and noise patterns in their =Classify= section; they do not redefine the buckets. + +- *Action* — needs Craig to do something: an explicit ask, a decision needed, blocked-on-Craig, a mergeable PR, an invite needing a response, a deadline inside 48h. +- *FYI* — substantive context worth seeing, but no action owed. +- *Noise-keep* — low value but worth retaining (audit trail, receipts). +- *Noise-trash* — safe to discard: newsletters, marketing, social digests, bot pings, redundant aggregator digests, wrong-recipient mail, past-event artifacts. + +Per-source bias (a work email account leans keep for audit value; a personal account leans trash on high noise volume) lives in each plugin's =Classify= section. Read it from there; don't re-derive it here. + +*** Phase C: Synthesize a single summary + +One markdown summary surfaced inline to Craig. Order: + +0. *Scan failures — first, loud, always.* Any loaded source whose scan failed, hung, was killed, or was skipped for an operational reason renders at the very top of the summary, before Top signals: + + #+begin_example + ⚠ SCAN FAILED: <source> — <reason, one line> — <what's now unknown> + #+end_example + + A failed scan is never folded into "quiet." Quiet means the scan ran and found nothing; a failure means the sweep is blind on that channel, and the reader must know which. The same applies to a precondition skip the user hasn't standing-approved (e.g. a messaging client that needs a temporary server spin-up): run the lifecycle or report the failure — don't silently narrow the sweep. + +1. *Top signals to act on* — bullet list of 3-7 items, ordered by urgency, *Action only*. Each bullet links to the source (permalink, thread URL, PR number). +2. *Per-source breakdown* — one short section per loaded source *that has changes*, in =ORDER=, using that plugin's =Render= shape: Action items detailed, FYI items as a short list, Noise as a tally only ("Noise: 12 trash candidates, 4 keep, 0 starred"). +3. *Suggested actions* — explicit list of state changes Craig could take this run (trash these N messages, mark-read these M, star this Action item, respond to this invite, merge PRs #X and #Y, etc.). This line stays whenever there are queued actions, regardless of how quiet the sweep was. + +*Deltas only.* The summary reports what *changed* since the anchor: a new invite, a new/moved/cancelled calendar event, a new message needing attention. A source with no changes gets no block — no "Calendar — quiet", no "PRs — nothing new" roll-call. A sweep where nothing changed anywhere renders as a single line: + +#+begin_example +17:39 sweep: no changes +#+end_example + +(Craig, 2026-06-11: "we only need to report if anything's changed when we do triage intake. did someone send me a new invite? did christine throw something on my calendar that wasn't there earlier? did someone cancel a meeting?") + +Scan failures are the standing exception: a failed or skipped scan always renders loudly per point 0 above and is never folded into the no-change line — "no changes" is a claim about channels the sweep could actually see. + +Format target: scannable in 30 seconds, full read in 2 minutes. Don't pad. + +**** Sub-step: write each Action item into =todo.org= as its own =:quick:= task + +After surfacing the summary inline, append every Action item — regardless of source — to =todo.org= as its own top-level =** TODO= heading carrying the =:quick:= tag plus =:reactive:= and any relevant person/entity tag. + +Each Action item is one task. Don't group items by source under =** Email Response=, =** PR Review=, etc. sub-headings. Each response is its own filterable task so Craig can re-prioritize, =SCHEDULE:= / =DEADLINE:=, or tag individually. + +Format: + +#+begin_example +*** TODO [#B] Merge PR #42 on archsetup (approved, CI green) — [[https://github.com/<user>/archsetup/pull/42][PR #42]] :quick:reactive: +*** TODO [#B] Respond to the 2pm reschedule invite from Dana :quick:reactive: +*** TODO [#B] Reply to the contract-terms email thread :quick:reactive: +#+end_example + +Rules: + +- Heading is plain prose. Lead with the verb (Read / Re-review / Reply / Respond / Address / Merge / Schedule). +- Priority: default =[#B]= for fresh reactive items. Bump to =[#A]= only if blocking someone or a deadline lands inside 7 days. +- Tags: always =:quick:= + =:reactive:=. Add person/entity tags when the dependency is sharp. +- Link the source in the heading when it has a URL (GitHub PR, mail thread, chat permalink). Use org's =[[url][label]]= form so the heading stays clickable in Emacs. +- *Record the source locator in the task body* so a reply can be routed back to where the request came from — the channel + thread id for chat, the repo + PR number, the message id for mail. The general rule: a reply goes back to the *origin* of the request, not a fixed notification channel. (Project plugins may add stricter routing rules in their own files.) +- Placement: append at end of =* Work Open Work= (just before =* Work Incubate=) unless the project's =todo.org= has a designated triage section near the top (=* Triage= or =* Inbox=). + +This sub-step makes triage-intake's findings *persist* in =todo.org= instead of evaporating after the inline summary. + +*** Phase D: Execute actions on confirmation + +Wait for Craig's go-ahead before running any state changes. Default to single-confirmation for the whole batch ("yes" → run everything proposed). Craig may also pick a subset ("trash personal but hold the work account") or hand back a different plan ("trash all but star the expense thread and queue PR merges for after lunch"). + +Each action dispatches to the owning source plugin's =Actions= verb (trash, mark-read, star, respond, merge, comment, attachment-fetch). The engine doesn't hardcode action commands — it reads them from the loaded plugins. Read each plugin's =Actions= section for the exact command. + +After actions complete, write the Phase A capture into the sentinel's *content* (see "Capture the Phase A timestamp"): =echo "$PHASE_A_TS $(date -d "@$PHASE_A_TS" '+%Y-%m-%d %H:%M:%S %z')" > .ai/last-triage-intake=. Do not use plain =touch= (writes mtime to /now/ and strands items posted between Phase A and end of run) and do not use =touch -d "@$PHASE_A_TS"= (correct timestamp but mtime is per-machine — won't survive a fresh clone or cross-machine sync). + +*Do not close the workflow yet.* See Exit Criteria below. + +*** Exit Criteria + +The workflow stays open until Craig has *explicitly* either: + +1. *Confirmed* that the executed actions are sufficient and nothing more is needed this round, or +2. *Handed back a different plan* (e.g., "actually hold the PR merges, address #131 first"). + +A successful Phase D run is *not* an exit signal. After the action batch returns, surface what shipped and wait. Don't volunteer "done" or "all set" — those are exit-claim phrases that pre-empt Craig's call. Use a status report ("17 actions succeeded, sentinel written at 12:19") and stop. + +If Craig has been silent for a while after Phase D and the surface looks closed-out, *ask*: "Anything else on this triage, or are we good to close out?" Don't auto-terminate. + +This rule prevents the failure mode where the workflow self-declares done and the next exchange has to relitigate what state things are in. + + +* Auto mode (unattended monitoring) + +Auto mode is a self-running variant of the engine for when Craig is away from the desk but wants tight awareness — a loop that runs the standard sweep on a short interval, *accumulates* findings rather than mutating state, and hands Craig a gated checkpoint to commit the batch. It composes two things: the *delivery* (a =/loop= in the live session) and the *behavior* (accumulate-don't-mutate sweeps with a checkpoint). The one-shot run above is unchanged; auto mode is an additional way to run the same Phase 0 / A-D engine. + +** Trigger and delivery + +- "auto triage" / "auto triage-intake" / "watch the desk" / "monitor the triage" — start auto mode. +- Default interval *20 minutes*; Craig sets it. + +Auto mode runs as a =/loop= in the *live session*, not a detached cron job: + +#+begin_src +/loop 20m run an auto-mode triage-intake sweep per triage-intake.org +#+end_src + +Running in the live session means MCP auth (Slack, Gmail, Linear) is inherited from the session — the headless-auth wall that blocks a detached cron run does not apply. A durable cross-session schedule is out of scope here; that belongs to the morning-ops orchestrator, which can later invoke auto mode's accumulate behavior as its triage limb. The close/stop commands below require a live session by design. + +*** Push each sweep to Craig's phone (ntfy) — the primary delivery + +Auto mode exists for when Craig is away from the desk (out of office for a while, on vacation), so the report's primary delivery is a push to his phone, not an inline message he won't be looking at. After every sweep, send the End-of-sweep output to his phone via =phone-notify= (the self-hosted ntfy channel over Tailscale; see the phone-notify reference memory for usage and the high-bar caveat). Push on *every* sweep, including a quiet "no changes" one — the timestamp line is the proof the loop ran. + +The pushed summary is the *fuller* shape, not a terse one-liner: per-source deltas (Slack / work-email / Linear / PRs / calendar / Telegram, noise tallies included), the current open-PR + Linear state, the awaiting-acknowledgment list, a one-line verdict on whether anything needs Craig, and the timestamp. Lead with a ⚠ SCAN FAILED banner if any source failed. + +Poll =phone-recv= at the top of each sweep for anything Craig sends back (delivery is not pushed to the agent); act on his requests and reply via =phone-notify=. Note that =phone-recv= echoes the agent's own outgoing pushes back, so only treat a message as inbound from Craig when it is not one of the sweep summaries. + +If =phone-notify= isn't installed on the host (it lives on ratio for now), fall back to inline delivery and say so once. + +** Preconditions and Close-out + +Auto mode borrows the inbox-monitor gates (=monitor-inbox.org=): do not start on a dirty worktree or a red test suite — a close's batch commit would otherwise sweep up unrelated changes — and leave the tree clean and green when the loop stops. Surface a blocker with inline numbered options per =interaction.md= and wait. + +** A sweep: accumulate, don't mutate + +Each sweep runs Phase 0 (load *both* plugin dirs — the loud requirement still holds) and Phases A-D's scan / classify / synthesize, but performs *none* of the normal run's mutations: + +- Does NOT advance the sentinel. The scan window grows from the last *close* until the next close: every sweep scans from the existing sentinel up to now, so nothing between sweeps is dropped. +- Does NOT write =todo.org= Action tasks — accumulates them for the close. +- Does NOT take mail actions (trash / mark-read / star). +- Does NOT commit. +- DOES update an active daily-prep in Update mode and re-open it on change (per =daily-prep.org=). +- DOES report, deltas-only, with loud scan-failure banners (Phase C rules unchanged). + +** End-of-sweep output — three sections + +1. *Deltas* — what changed since the *previous sweep* (the standard Phase C summary scoped to the inter-sweep delta; one line if nothing: "HH:MM sweep: no changes"). +2. *Responses awaiting your acknowledgment* — every Slack reply, email, or message directed at Craig that he hasn't acknowledged or had the agent answer. A *running list carried forward across sweeps* until Craig acks each item or closes the triage. An away user's first need is "who's waiting to hear back from me," which a delta-only sweep loses the moment it scrolls past. +3. *Timestamp* — the current date, time, and timezone on the sweep's own final line, so an away reader sees how fresh the summary is without computing it. Print it on *every* sweep, including a quiet "no changes" one — on a quiet sweep the stamp is the proof the loop ran. Generate it with: + + #+begin_src bash + date "+%A %Y-%m-%d %H:%M:%S %Z (%z)" + #+end_src + +** The unacked list — durable state + +The awaiting-acknowledgment list lives in =.ai/triage-intake-unacked.org=, so it survives a session crash, a =/clear=, or a restart — the away-from-desk case auto mode exists for. It's project-local state, tracked the same way as the sentinel (=.ai/last-triage-intake=), created on first need. + +Shape — one =** = heading per awaiting item: + +#+begin_example +#+TITLE: Triage Intake — Responses Awaiting Acknowledgment +# Maintained by triage-intake auto mode. One heading per item; acked items are removed. + +** Dana — 2pm reschedule invite +:PROPERTIES: +:SOURCE: personal-calendar +:LOCATOR: <event id or thread url — the dedupe key> +:SINCE: 2026-06-15 10:42 +:END: +She's waiting on a yes/no to the move. +#+end_example + +- *Add* — a sweep appends any new directed-at-Craig response not already listed, deduped on =LOCATOR=. +- *Carry forward* — every sweep re-renders the full list in its second section, whether or not it changed this sweep. +- *Ack* — "ack <item>" (e.g. "ack the Dana thread") removes that heading; "ack all" clears the list. +- *Close* — a close empties the list as part of processing (each item is either actioned or filed). + +** Close and stop — the checkpoint + +The mutations are gated behind two commands: + +- *"close the triage"* — run the full close: take the accumulated mail actions, add the accumulated Action items to =todo.org= as =:quick:reactive:= tasks (asking Craig the questions a normal Phase C/D would), empty the unacked list, then *advance the sentinel* — capture the close run's Phase A timestamp, do the mutations, write that timestamp to =.ai/last-triage-intake= exactly as a normal run does (per "Capture the Phase A timestamp") — and commit + push the batch. Then *keep looping* (next sweep on the normal interval). This is the "flush the batch and carry on" checkpoint. +- *"stop the triage"* — the same close processing, then *stop the loop* and revert to manual (on-demand) triage. + +A close is the only point auto mode advances the sentinel or commits. Between closes the engine state is untouched — that is what makes a 20-minute sweep cheap and non-destructive, and it preserves the engine invariant: the sentinel still means "everything before this timestamp has been scanned," it just advances once per close instead of once per run. + +** Why a separate mode + +The standard engine is one-shot and mutating — right for an at-the-desk "what's new?" glance, wrong for unattended polling: run every 20 minutes it would advance the sentinel past unprocessed items, spray reactive todos, take mail actions, and commit noise without review. Auto mode separates the cheap, frequent *watching* from the deliberate, gated *committing*, and adds the away-user's missing primitive — the running unacked-responses list. + + +* Reference + +** Source Plugin Contract + +A source plugin is a file named =triage-intake.<source>.org=. The first dot after =triage-intake= is the engine/plugin boundary; the segment after it is the source id. Hyphens stay *inside* a segment (=triage-intake.personal-gmail.org= is engine =triage-intake=, source =personal-gmail=). Deeper dots (=triage-intake.<source>.<sub>.org=) are reserved for sub-adapters — YAGNI for now, but the namespace accommodates them at no cost. + +A plugin file declares exactly one source through a fixed shape: + +*Property drawer* on the top-level =* Source:= heading: +- =ORDER= — integer. Output ordering in the per-source breakdown (lower = earlier). +- =ENABLED= — the precondition the engine evaluates before loading the source. The source is skipped — *with an announced reason* — when it's false. Forms: =always=, a shell test (=command -v gh && gh auth status=), or =mcp <server> present=. +- =ANCHOR= — the cutoff format this source consumes: =epoch=, =iso8601=, =day=, or =none= (state-based source with no since-window — e.g. live IMAP unread, or open-PR state). The engine computes the anchor once and substitutes it in the requested format. +- =SUBAGENT_OVER= — integer. If the scan is expected to return more than this many items, dispatch a subagent for the source so its raw output stays out of main context. The subagent applies Phase B and returns buckets only, not the raw list. + +*Body sections:* +- =** Scan= — the command(s) that fetch new/unread items since =<anchor>=, emitting raw items. +- =** Classify= — the source's per-bucket bias and noise patterns. *Deltas* from the engine's shared four-bucket model below, not a re-derivation. +- =** Render= — the source's block in the Phase C summary. "Omit if empty." +- =** Actions= — the executable state-changes, one verb per line: =verb :: command template (parameterized by item id)=. + +Template: + +#+begin_example + +** Source: <id> +:PROPERTIES: +:ORDER: <n> +:ENABLED: <precondition> +:ANCHOR: epoch | iso8601 | day | none +:SUBAGENT_OVER: <n> +:END: + +*** Scan +<command(s) that fetch new/unread items since <anchor>> + +*** Classify +<bias + noise patterns; deltas from the shared four-bucket model> + +*** Render +"<Source label> — N <unit>" block; omit if empty. + +*** Actions +- <verb> :: <command, parameterized by <id>> +#+end_example + + +** Anchor: Since When? + +The workflow needs a "scan since" timestamp. Resolution order: + +1. *Sentinel file content:* first whitespace-delimited token in =.ai/last-triage-intake= is the Phase A scan-kickoff epoch from the most recent successful run (see "Capture the Phase A timestamp" below). Most accurate. +2. *Sentinel file mtime* (back-compat): if the file exists but is empty, read its mtime — that's the older mtime-based convention that pre-dates the content-based change. Still accurate on the machine that wrote it. +3. *Most recent prep doc:* if no sentinel content or readable mtime, anchor on the latest =daily-prep/YYYY-MM-DD-daily-prep.org= mtime. +4. *Most recent session file:* if none of the above, anchor on the most recent =.ai/sessions/= file's mtime. +5. *Session start:* fall back to the current session's start time. Last resort. + +The engine computes the anchor *once* and exposes it in every format a plugin might request (=epoch=, =iso8601=, =day=). Each plugin's =ANCHOR= field says which it consumes; the engine substitutes that form into the plugin's =<anchor>= placeholder. Sources with =ANCHOR: none= are state-based (live unread, open-PR state) and get no cutoff substituted — they report current state, and Phase B uses the anchor only to flag what's *new since* last check. + +*** Capture the Phase A timestamp + +Just before issuing the Phase A batch, capture the current epoch seconds: + +#+begin_src bash +PHASE_A_TS=$(date +%s) +#+end_src + +Hold this value through Phases B, C, and D. At end of run, *write* the captured timestamp into the sentinel's content (not its mtime): + +#+begin_src bash +echo "$PHASE_A_TS $(date -d "@$PHASE_A_TS" '+%Y-%m-%d %H:%M:%S %z')" > .ai/last-triage-intake +#+end_src + +The file ends up with a single line like =1778683109 2026-05-13 09:38:29 -0500= — epoch first (machine-readable, parsed by reading the first token), human-readable timestamp second. + +*Why content, not mtime:* the sentinel is checked into git. Git tracks content, not mtime, so an mtime-based sentinel is per-machine: one machine's anchor stays on that machine; a fresh clone gets the file but the mtime is whenever the clone happened, not the actual triage time. Writing the epoch as content means the anchor travels with the repo and stays accurate after a fetch + pull on any machine. + +*Why Phase A and not end-of-run:* Phase A runs at one moment, but Phases B-D may take 5-30 minutes. Items posted to any source /during/ Phases B-D land between the Phase A scan time and the eventual end-of-run time. If the sentinel were set to the end-of-run time, those items would silently fall through the cracks: the next triage's Phase A would skip the gap window and never see them. Anchoring the sentinel to Phase A's scan time guarantees the next run's window starts where this run's window ended, with zero gap. + +*** Reading the sentinel + +When the workflow needs the anchor at the start of a new run: + +#+begin_src bash +# Content-first, mtime-fallback. +ANCHOR_EPOCH=$(awk 'NR==1 {print $1; exit}' .ai/last-triage-intake 2>/dev/null) +if [ -z "$ANCHOR_EPOCH" ] && [ -f .ai/last-triage-intake ]; then + ANCHOR_EPOCH=$(stat -c %Y .ai/last-triage-intake) +fi +#+end_src + +If both fail, fall through to the resolution order above (prep doc → session file → session start). + + +** Output Template + +The summary follows this shape (deltas only: a source with no changes gets no block; when *nothing* changed anywhere, the whole summary collapses to the one-line form below — plus any scan-failure banners and the suggested-actions line if actions are queued): + +#+begin_example +17:39 sweep: no changes +#+end_example + +When there are changes, render one block per changed source in =ORDER=, using each plugin's =Render= shape: + +#+begin_example +**Anchor:** <previous run timestamp> → now (<elapsed> elapsed) +**Loaded:** <general plugins> + <project plugins> (skipped: <disabled, with reason>) + +**Top signals to act on:** +1. <terse Action description with link> +2. ... + +<one block per loaded source, in ORDER — see each plugin's Render> + +**Suggested actions:** +- Trash N noise items +- Mark-read M keep items +- Respond to <invite> +- Merge PRs #X and #Y +- ... +#+end_example + +Order matters: top-signals first because that's what Craig reads in 30 seconds between meetings. Per-source detail second. Suggested actions last because they require a decision. + + +** Common Mistakes + +1. *Globbing only =.ai/workflows/= and missing the project plugins.* The single most damaging failure mode — the sweep runs with half its sources and the omission is invisible (a missing source looks identical to a quiet one). Phase 0 globs *both* =.ai/workflows/triage-intake.*.org= and =.ai/project-workflows/triage-intake.*.org=, every run, and announces the loaded set. +2. *Running Phase A sequentially.* Send every enabled source's scan in one message — the whole point is parallelism. +3. *Wiring a source into the engine.* Sources live in plugin files, never here. If you find yourself editing this file to add an account, repo, or channel, stop — write or edit a =triage-intake.<source>.org= plugin instead. +4. *Executing actions without explicit confirmation.* Phase D runs only after Craig says "yes" or picks a subset. +5. *Forgetting to set the sentinel at the end.* Without it, the next run re-scans the same window. +6. *Using mtime instead of content for the sentinel.* Plain =touch= writes /now/ to mtime, stranding items posted between Phase A and end of run. =touch -d "@$PHASE_A_TS"= fixes the time but mtime is per-machine — git tracks content, not metadata, so the anchor doesn't survive a clone or cross-machine sync. Always write the epoch into the file's *content*. +7. *Running this alongside daily-prep.* Daily-prep already does this as Phase 3 — don't duplicate. +8. *Mixing Action and FYI in the top-signals list.* Top signals = Action only. FYI lives in the per-source detail. +9. *Reporting a failed or skipped scan as a quiet source.* A hung receive, a dead daemon, or a skipped spin-up looks identical to "no new messages" in the output unless it's flagged. The 2026-06-10 sweep shipped with Signal silently missing because the scan hung on an account lock. Failures lead the summary, in their own banner line. +10. *Rendering a per-source quiet roll-call.* "Calendar — quiet" / "PRs — nothing new" lines on every silent source bury the one change that matters and pad a no-change sweep into a report. Deltas only: changed sources get blocks, unchanged sources get nothing, and an all-quiet sweep is one line (Craig's 2026-06-11 ruling in Phase C). + + +* History / Design Notes + +** Living Document + +Update the engine as the orchestration pattern evolves; update a plugin as its source evolves. Source-specific learnings belong in the plugin's own file, not here. + +*** Updates and Learnings + +**** 2026-06-15: Auto mode (unattended monitoring) +Added a self-running mode for when Craig is away but wants tight awareness — a =/loop= in the live session running accumulate-don't-mutate sweeps with "close the triage" / "stop the triage" as the gated checkpoint. Born the morning Craig cleared his day for a family emergency and wanted the desk watched while in and out. Design decisions (work-project proposal, ratified by Craig 2026-06-15): the unacked-responses list is durable in =.ai/triage-intake-unacked.org= (survives a crash/clear, the away-from-desk case it exists for); the sentinel advances only at close, preserving the scanned-before invariant; delivery is an in-session loop so MCP auth is inherited (a detached cron schedule belongs to the morning-ops orchestrator, not here, because of the headless-auth wall); it stays a mode of this engine, distinct from but reusable by that orchestrator. Same-day addendum (work, 2026-06-15): each sweep ends with a date/time/timezone stamp on its own final line (printed on quiet sweeps too, as proof the loop ran) so an away reader gauges freshness at a glance. + +**** 2026-05-01: Initial creation +Extracted from daily-prep's Phase 3 pattern as a standalone, lightweight, between-meetings sweep. + +**** 2026-05-07: Anchor the sentinel to Phase A scan time, not run-end time +Gap-window bug: a run had Phase A fire at 13:35 and the sentinel set at 15:04, so an item posted at 14:20 would be skipped by the next run (the sentinel claimed everything before 15:04 was scanned when Phase A only reached 13:35). Fix: capture =PHASE_A_TS= just before Phase A, hold it through B-D, write it to the sentinel at end of run. The sentinel means "everything before this timestamp has been scanned," the only invariant that prevents items falling through the cracks. + +**** 2026-05-13: Move the sentinel from mtime to content (cross-machine survivability) +The sentinel is checked into git, but git tracks content, not mtime — so an mtime anchor is per-machine. Fix: write the captured epoch into the sentinel's content (=EPOCH ISO-8601=), read with =awk 'NR==1 {print $1}'=, mtime as back-compat fallback. + +**** 2026-06-11: Deltas-only reporting (Phase C + Output Template + Common Mistake 10) +Craig, via the work project's same-day handoff: "we only need to report if anything's changed when we do triage intake." Sweep summaries report deltas only — a new invite, a new/moved/cancelled event, a new message needing attention. Unchanged sources get no block (the "Calendar — quiet" roll-call is retired), and an all-quiet sweep renders as a single "HH:MM sweep: no changes" line. Failures keep their loud banner (never folded into the no-change line) and the suggested-actions line stays when actions are queued. Same ruling: the telegram plugin's dev-community group traffic is dropped from reports entirely unless Craig asks (see that plugin's 2026-06-11 note). + +**** 2026-06-10: Loud failure surfacing (Phase C item 0 + Common Mistake 9) +Craig: "highlight any failures in daily triage loudly. I get important communication from all these channels." Trigger: the 2026-06-10 sweep shipped with Signal silently missing — a standalone receive hung on the account lock while the signel daemon owned it, and the failure looked identical to a quiet source. Failures now lead the summary in a ⚠ SCAN FAILED banner; the telegram plugin's failure path points at this rule. + +**** 2026-05-26: Refactor into engine + source plugins +Split the monolithic workflow into a source-agnostic engine (this file) and per-source plugins named =triage-intake.<source>.org=. The engine carries the anchor/sentinel logic, the four-bucket model, the Phase A-D orchestration, the todo.org persistence convention, and the exit criteria. Each source's scan/classify/render/action knowledge moved to its own plugin. General plugins (personal-gmail, personal-calendar, cmail, github-prs) live in =.ai/workflows/= and are template-synced; project-specific plugins (a work project's Linear, work Gmail, work Slack, enterprise PRs) live in the project's =.ai/project-workflows/= and are never synced. Phase 0 globs *both* directories — the loud requirement, because missing the project dir silently halves the sweep. Naming convention: first dot is the engine/plugin boundary, deeper dots reserved for sub-adapters. This removed all DeepSat/Linear specifics from the engine; they become work-project plugins. + diff --git a/docs/design/2026-06-21-anki-titlefix-proposal.org b/docs/design/2026-06-21-anki-titlefix-proposal.org new file mode 100644 index 0000000..08b8c13 --- /dev/null +++ b/docs/design/2026-06-21-anki-titlefix-proposal.org @@ -0,0 +1,57 @@ +#+TITLE: Proposal — flashcard-to-anki.py deck name should come from #+TITLE + +From: home session, 2026-06-21. Two attached files are the edited +canonical scripts (flashcard-to-anki.py + its test). Applied locally in +home as a stopgap; this is the durable proposal for the rulesets +canonical. Please reconcile and re-sync. + +* The bug (longstanding) + +flashcard-to-anki.py's default_deck_name returned input_path.stem (the +filename), so every deck generated through flashcard-sync (which passes no +--deck) was named after the file, e.g. "personal-drill" / "health-drill" +/ "kit", not the curated #+TITLE. + +flashcard-review.org already documents the intended behavior: "The +#+TITLE line drives ... the Anki deck name on the phone" and "derives the +Anki deck ID from the deck name." The script never matched the doc. +deepsat only looked correct because its first run used an explicit +--deck "DeepSat Flashcards". + +* The fix + +default_deck_name(input_path, org_text) now scans for a #+TITLE: line +(case-insensitive, surrounding whitespace trimmed) and returns it; falls +back to input_path.stem when there's no non-empty #+TITLE. main() passes +the already-read org_text. Help text + module docstring updated. + +TDD: the two old deck-name tests asserted the buggy basename behavior — +rewrote them. New tests cover title-driven naming, trimming, +case-insensitive #+title, basename fallback (no title), and basename +fallback (blank title). Full file: 29 pass. + +No companion script changes needed: flashcard-sync passes no --deck so it +picks up the new default automatically, and flashcard-stats.py already +reads #+TITLE. flashcard-review.org needs no change (the script now +matches what it already says). + +* Migration caveat (worth a line in the doc if you want) + +Deck ID derives from the deck name, so this fix changes the ID for any +deck previously generated without --deck. On next import those land as +new decks; the old basename-named decks keep their review history and +must be deleted by hand. The workflow's existing "Stable-ID caveat" +already covers the mechanics. In home this affected personal-drill, +health-drill, kit (regenerated this session as Personal / Health / KIT, +with titles also stripped of "Flashcards"/"Drill" per Craig). deepsat is +unaffected (already title-named). + +* Related idea (separate, not in these files) — apkg → org-drill converter + +deepsat-fundamentals.apkg (100-card DeepSat subset, made once with +--deck "DeepSat Fundamentals") has no saved .org source anywhere. Craig +wants an apkg → org-drill converter — the inverse of flashcard-to-anki.py +— to recover orphaned decks and pull phone-authored cards back into the +org source-of-truth. Flagging as a candidate rulesets tool alongside the +flashcard-* family; deepsat-fundamentals is the concrete first use case. +Not built yet; raising for the backlog. diff --git a/docs/design/2026-06-21-apkg-to-orgdrill-buildreq.org b/docs/design/2026-06-21-apkg-to-orgdrill-buildreq.org new file mode 100644 index 0000000..37a866f --- /dev/null +++ b/docs/design/2026-06-21-apkg-to-orgdrill-buildreq.org @@ -0,0 +1,68 @@ +#+TITLE: Build request — apkg → org-drill converter (inverse of flashcard-to-anki.py) + +From: home session, 2026-06-21. Craig wants this built (backlogged, not +urgent). Standalone build request — the earlier anki-title-fix-proposal +only mentioned it in passing; this is the real ask. + +* Why + +The flashcard pipeline is one-directional (org-drill → apkg). Decks +authored or curated on the phone, and orphaned apkgs whose .org source +was never saved, can't get back into the org source-of-truth. Concrete +case: deepsat-fundamentals.apkg — a 100-card DeepSat subset generated +once with --deck "DeepSat Fundamentals" — has no .org source anywhere on +ratio, velox, or in work git history. The converter recovers it and makes +phone → org round-tripping possible. + +* What — contract (inverse of flashcard-to-anki.py) + +Input: an Anki =.apkg= (a zip containing collection.anki2 / .anki21 +sqlite, plus a media blob). +Output: an org-drill =.org= file in the house canonical shape that +flashcard-stats.py / flashcard-to-anki.py already agree on. + +Mapping (mirror flashcard-to-anki.py's parse/build): +- Deck name (from the apkg) → =#+TITLE:=. +- Each note → =** <Front> :drill:= with the Back as the body. +- Card tag → top-level =* Section= grouping (inverse of section_to_tag; + cards sharing a tag collect under one section; the slug won't round-trip + to the exact original section title, so this is best-effort — emit the + tag as the section heading and let a human retitle). +- Back HTML → org: convert =<br>= back to newlines; unescape + =&/</>=; strip the =<hr id="answer">= the card template adds + (the Back field itself shouldn't contain it, but guard anyway). +- Generate a fresh =:ID:= UUID per card in a =:PROPERTIES:= drawer so the + output is immediately org-drill-valid and round-trips back through + flashcard-to-anki.py. (Note: GUIDs in flashcard-to-anki.py are derived + from the front text, not the :ID:, so a regenerated apkg still matches + existing phone cards by front — call that out in the docstring.) + +Edge cases to cover in tests (Normal/Boundary/Error): +- Multiple decks in one apkg (emit one file per deck, or error asking for + a deck filter — pick one and document it). +- Notes with multiple fields / non-basic note types (the pipeline only + models Front/Back — skip or warn on others, don't silently drop). +- HTML entities, embedded =<br>=, and any =Source:= footer surviving + round-trip. +- Empty back; media references (flag, since org side has no media path). +- collection.anki2 vs .anki21 schema differences. + +* Where it lives + +Rulesets-owned, beside the flashcard-* family +(=claude-templates/.ai/scripts/=): suggest =anki-to-flashcard.py= (or +=apkg-to-orgdrill.py= — your naming call). Add tests under +=scripts/tests/=. A new file can't be built downstream — home/.ai/scripts/ +is wiped to match the template by the startup =--delete= rsync — so this +has to be built in the rulesets canonical. PEP 723 uv-run script like its +sibling; genanki isn't needed for reading (stdlib =zipfile= + =sqlite3= +suffice), so it has no runtime deps. + +* Acceptance + +Round-trip test: take a known org-drill source, run it through +flashcard-to-anki.py, run the result back through this converter, and +assert the cards (front/back/section) match the original (modulo +regenerated :ID:s and best-effort section titles). Plus: run it on the +real deepsat-fundamentals.apkg and hand the recovered .org back so its +source can be filed (work project). diff --git a/docs/design/2026-06-21-flashcard-stats-refutation-proposal.org b/docs/design/2026-06-21-flashcard-stats-refutation-proposal.org new file mode 100644 index 0000000..bbbe175 --- /dev/null +++ b/docs/design/2026-06-21-flashcard-stats-refutation-proposal.org @@ -0,0 +1,57 @@ +#+TITLE: Proposal — flashcard-stats.py refutation / claim-prompt mode + +From: home session, 2026-06-21. Backlog, not urgent. Relates to the +refutation-drill deck being built in the home project. + +* Problem + +A new card family doesn't fit the linter: the *refutation / claim-prompt* +card. Its heading is a bare false claim ("The earth is flat.") and its +body is the rebuttal. This is a legit org-drill simple card (org-drill is +happy), but flashcard-stats.py — built for Q&A decks — trips two BLOCKING +checks on every such card, both false positives: + +- *non-prompt heading*: a declarative claim has no '?' and no + imperative verb, so it reads as "topic-as-heading not yet rewritten". + But for this family the declarative claim IS the intended prompt. +- *answer leakage*: the claim's words necessarily reappear in the + refutation, so front/back overlap is high. But the answer (the rebuttal) + is not given away by the claim — there's no actual leakage. + +Concrete: the home refutation-drill.org (6 cards) reports 6 non-prompt +headings + 1 leakage WARN, so flashcard-sync's gate blocks it entirely. +The deck currently has to be generated with the flashcard-to-anki.py +override, losing the safety net. + +* Proposed fix + +A per-deck opt-in marker that switches the two checks off for that file +only. Two options (your call): + +1. A file-level keyword: =#+DECK_KIND: refutation= near the top. When + present, flashcard-stats skips the non-prompt-heading check and the + answer-leakage check for the whole file (keeps the others: + missing-:ID:, *** Answer sub-headers, duplicate fronts, the + non-blocking NOTEs). +2. A per-card tag: cards tagged =:claim:= (alongside =:drill:=) are + exempted from those two checks individually. + +Option 1 is simpler and matches how this deck works (the whole file is +one family). Option 2 is finer-grained if a deck ever mixes families. + +Either way: document the new card family in flashcard-review.org (a +"Refutation / claim-prompt cards" subsection under Canonical Card Shape — +heading is the bare claim, body is snap-response + backups + named-fallacy ++ restate, Source footer), and note that flashcard-sync then works +normally on these decks. + +* Affected files +- =flashcard-stats.py= — the check skip + (option 1) keyword parse / (option 2) tag check. +- =flashcard-review.org= — document the family + the marker. +- =flashcard-to-anki.py= / =flashcard-sync= — no change needed (they don't gate on heading form). +- Tests: add cases for a refutation-marked file passing despite declarative headings + claim/answer overlap. + +* Companion context +The home deck's card format and the org-drill-fine / Anki-linter-fights +finding are written up in home:refutation-drill-sources.org (Tooling +note). The override command is documented there too. diff --git a/docs/design/2026-06-21-host-identity-guard-proposal.org b/docs/design/2026-06-21-host-identity-guard-proposal.org new file mode 100644 index 0000000..f389825 --- /dev/null +++ b/docs/design/2026-06-21-host-identity-guard-proposal.org @@ -0,0 +1,54 @@ +#+TITLE: From archsetup — hardcoded machine identity in CLAUDE.md (consider fleet-wide) +#+DATE: 2026-06-21 + +* What we did + +Built a Super+F Dirvish popup in the archsetup/dotfiles + .emacs.d projects, +modeled on the existing Super+Shift+N org-capture popup (launcher script names an +emacsclient frame, Hyprland window rules float it, an Emacs command runs in the +frame and q closes it). Cross-project: dotfiles half committed from archsetup, +Emacs half handed off to .emacs.d's inbox. + +* The bug it surfaced + +While stowing on this machine, =make stow hyprland= pulled the *velox* host tier, +and =uname -n= returned =velox=. But archsetup's CLAUDE.md asserted, as a fixed +fact, "This machine is **ratio**." It was simply wrong on velox — a stale +identity baked into a per-project doc that travels to every machine via git. + +I'd been reasoning from that line all session (e.g. "the touchpad-auto reminder +is velox-only, and we're on ratio, so skip it") — exactly backwards. A hardcoded +"this machine is X" in a synced/tracked project file is a latent trap on any +multi-machine setup: the file is identical on every host, so the claim is false +on every host but one. + +* The fix (this project) + +Replaced the fixed identity with a runtime instruction. The attached CLAUDE.md +now reads, in the Notes section: + + Never assume which machine this is — always run =uname -n= to find the hostname + (the =hostname= binary is absent, so =uname -n= is the source of truth; + =uname -r= is the kernel release, not the host). The fleet is ratio + (workstation) and velox (laptop), both Hyprland (Wayland)... + +(Craig initially said =uname -r=; that's the kernel release. =uname -n= is the +nodename/hostname, which is what the stow host-tier logic already keys on.) + +* Why this is a rulesets concern + +This isn't an archsetup-only quirk. Any project whose CLAUDE.md / notes get +synced or cloned across machines can hardcode environment identity — current +host, current OS, "the laptop", an IP, a display name — and be wrong everywhere +the doc lands but the origin. rulesets governs how every project's CLAUDE.md and +rules are shaped, so it's the right layer to consider a general guard: + +- A rule (claude-rules) along the lines of: don't assert mutable + environment/host identity as a fixed fact in a tracked/synced project file; + derive it at runtime (=uname -n= for host, etc.) and name the command. +- Possibly a startup or codify-time lint that flags "this machine is <name>" / + "the current host is" style claims in CLAUDE.md. + +Sending the edited CLAUDE.md (attached separately) plus this note so the rulesets +session can decide whether to codify the broader pattern. Proposal, not a +directive — your value gate applies. diff --git a/docs/design/2026-06-22-inbox-zero-capture-hardening.org b/docs/design/2026-06-22-inbox-zero-capture-hardening.org new file mode 100644 index 0000000..69acf94 --- /dev/null +++ b/docs/design/2026-06-22-inbox-zero-capture-hardening.org @@ -0,0 +1,39 @@ +#+TITLE: inbox-zero Phase D wedges live org-capture sessions on the roam inbox + +* The bug + +Phase D of =inbox-zero.org= removes claimed items from =~/org/roam/inbox.org= by editing the file on disk (Edit / sed / Write). That collides with any live org-capture session Craig has open against the same file. + +org-capture works through an *indirect buffer* cloned from the target file. When the inbox-zero disk write lands and Emacs reverts the main =inbox.org= buffer underneath, the indirect capture buffers are left pointing at stale state and wedge — they can no longer finalize cleanly with =C-c C-c=. The visible symptom is org-capture failing, and one or more orphaned =CAPTURE-*inbox.org= / =CAPTURE-N-inbox.org= buffers piling up as Craig retries. + +Hit live on 2026-06-22 during a home-session inbox-zero: I filed three home items, wrote =inbox.org= on disk, and Craig's open capture wedged, leaving two orphaned =CAPTURE-inbox.org= buffers. No data was lost that time (the orphaned buffers held only existing file content, not a freshly-typed item), but that was luck — had he typed an item into the capture before it wedged, finalizing the stale buffer afterward would have written it back against the post-edit file and could have clobbered the routing or a foreign item. + +* Why it's worth fixing in the canonical + +=inbox-zero.org= is a rulesets-owned synced workflow that runs in every project (startup nudge, wrap-up sub-step, on demand), and the roam inbox is the single shared file all of them edit. Craig edits that same file live in Emacs and captures into it constantly. So the collision window recurs in every project, every session — not a home-only quirk. A local fix in home gets reverted by the next template sync, so the durable fix has to land in rulesets. + +* Proposed fix (recommended: guard before the disk write) + +Add a pre-edit guard to Phase D, before removing any claimed items: + +1. If Emacs is reachable (=emacsclient -e t= succeeds), check for live capture buffers targeting the roam inbox: + + #+begin_src bash + emacsclient -e '(mapcar #(quote buffer-name) + (seq-filter (lambda (b) (string-match-p "CAPTURE.*inbox" (buffer-name b))) + (buffer-list)))' 2>/dev/null + #+end_src + +2. If any =CAPTURE-*inbox*= buffer exists, *stop before editing* and surface it: "You have a live org-capture session open against the roam inbox — finalize (=C-c C-c=) or abort (=C-c C-k=) it before I route items, otherwise the edit will wedge the capture." Resume Phase D only once it's clear. This mirrors the existing pull-before-edit / surface-and-stop discipline already in Phase D. + +3. Independently, when Emacs has =inbox.org= open and *unmodified* (the common case, no live capture), the disk edit is benign — Emacs reverts a clean buffer without complaint. Optionally trigger an explicit =revert-buffer= via emacsclient afterward so the buffer is immediately consistent rather than lazily on next focus. + +* Alternative (heavier): do the removal through Emacs when it's running + +Instead of editing on disk, when Emacs is reachable, perform the claimed-item removal inside the running daemon (find the buffer, delete the items, save), and fall back to the disk edit only when Emacs isn't running. This keeps Emacs's buffer authoritative and sidesteps the disk/buffer divergence entirely. It's more code and more failure surface for arbitrary item removal, so I'd lean on the guard above unless you want the stronger guarantee. + +* Note for whoever builds it + +The =emacs.md= rule already covers "don't make Craig restart Emacs; push changes into the running daemon." This is the same principle one layer out: don't edit a file *on disk* that the running daemon is actively editing/capturing into. Worth a line in =emacs.md= too, or at least a cross-reference from inbox-zero Phase D. + +Origin: home, 2026-06-22. diff --git a/docs/design/2026-06-23-install-lang-claude-md-gap.org b/docs/design/2026-06-23-install-lang-claude-md-gap.org new file mode 100644 index 0000000..cf16256 --- /dev/null +++ b/docs/design/2026-06-23-install-lang-claude-md-gap.org @@ -0,0 +1,31 @@ +#+TITLE: install-lang CLAUDE.md gap — non-elisp projects get a wrong or missing CLAUDE.md +#+DATE: 2026-06-23 + +Surfaced while running the archangel .ai/ conversion you sent (the 2026-06-20 handoff). archangel is a bash project — 437 =.sh= files; the only =.el=/=.py= in the tree are under =work/x86_64/airootfs/= (archiso staging, not source). Per your handoff I installed both elisp and python bundles. The result exposed two coupled issues that block CLAUDE.md consistency across projects. + +* The two findings + +1. *Only the elisp bundle ships a CLAUDE.md template.* =languages/elisp/CLAUDE.md= exists; =languages/python/=, =go/=, =typescript/= ship none. =install-lang.sh= guards on =[ -f "$SRC/CLAUDE.md" ]=, so a bundle without a template silently contributes nothing — no line printed, no file seeded. + +2. *No shell/bash bundle exists* (only elisp, go, python, typescript). archangel and archsetup are bash projects with no bundle that fits. + +* The consequence + +- Install python (or go/ts) alone → project gets *no* CLAUDE.md. +- Install elisp + anything → project gets the elisp stub, whose first line is "Elisp project." Because install-lang seeds CLAUDE.md only on first install and never overwrites without FORCE=1, install order doesn't matter — the elisp template is the only one available, so it always wins. +- Net: archangel, a bash project, ended up with a CLAUDE.md headed "Elisp project." An inaccurate CLAUDE.md is worse than none — it mislabels the project for every future session. + +* Proposals (rulesets' call) + +1. *Add a shell/bash language bundle.* This is the real gap for archangel/archsetup and any other shell-heavy project. +2. *Give every bundle its own CLAUDE.md template*, or ship a language-neutral default so install-lang always seeds an accurate (or at least non-misleading) header. A stub that says "<LANG> project — customize this" is only safe when the bundle actually matches the language. +3. *Consider the multi-bundle case* — when a project installs more than one bundle, the CLAUDE.md "Project" line shouldn't hardcode a single language picked by which template happened to exist. + +* Companion files to reconcile + +- =scripts/install-lang.sh= — the seed-on-first-install / no-overwrite logic (sections 3 and 3b) is correct; the gap is the missing templates and missing bash bundle, not this logic. +- =languages/elisp/CLAUDE.md= — the only template today; pattern to replicate per language. + +* What archangel did locally (stopgap) + +Installed both bundles as you asked; the generic =.claude/rules/= and gitignore hygiene are the real gain there. I flagged the elisp-stub mismatch to Craig and offered to hand-write archangel's CLAUDE.md as a bash ISO-build project. That local fix doesn't address the cross-project pattern — hence this note. diff --git a/docs/design/2026-06-23-wrap-teardown-shutdown-proposal.org b/docs/design/2026-06-23-wrap-teardown-shutdown-proposal.org new file mode 100644 index 0000000..a47aa2d --- /dev/null +++ b/docs/design/2026-06-23-wrap-teardown-shutdown-proposal.org @@ -0,0 +1,124 @@ +#+TITLE: Proposal — wrap-it-up teardown + "wrap it up and shutdown" variant + +* Source + +Raised by Craig in a home-project session, 2026-06-23, after talking the +design through. Two related additions to =wrap-it-up.org=. Both touch the +Claude-session lifecycle (workflow + hook + the =ai-term= buffer/tmux pair), +so they're rulesets — with one companion piece that has to live in +=.emacs.d/modules/ai-term.el= (flagged below). Originally floated as an +archsetup task; archsetup owns the Hyprland/waybar layer, not the +Claude-session lifecycle, so it was re-routed here. + +* Architecture this depends on (so the design is grounded) + +- =ai-term.el= (=.emacs.d=) is the in-Emacs launcher: a vertical-split vterm + buffer running a tmux session named =aiv-<project-basename>= (prefix + =aiv-=). Layering: =claude= process → tmux session =aiv-<proj>= → Emacs + vterm buffer. +- Killing the tmux session takes the =claude= process with it, so "quit + Claude Code" is a *consequence* of killing =aiv-<proj>=, not a separate + step. +- Hooks already exist under =~/.claude/hooks/= (e.g. =session-clear-resume.sh=, + =precompact-priorities.sh=) — the teardown trigger fits that pattern. +- =sudo= is =NOPASSWD: ALL= on Craig's machines, so =sudo shutdown now= runs + unattended. + +* Item 1 — wrap-up also removes the buffer, quits Claude, removes the tmux session + +Recommend: yes, with one structural rule — the wrap-up runs *inside* the +things it tears down, so teardown is self-terminating and must be the last, +decoupled action, or the valediction may not flush before the session dies. + +Design: +1. *Teardown lives in =ai-term.el=* (companion, see below): one function + =cj/ai-term-quit= that kills the =aiv-<proj>= tmux session (takes =claude= + with it), kills the vterm buffer, and restores the saved window geometry — + =ai-term.el= already owns the buffer↔session pair and the geometry logic. +2. *Trigger from a Stop / SessionEnd hook, not inline.* Wrap-up does all its + git/archive work, delivers the valediction, then drops a sentinel (flag + file, e.g. =/tmp/ai-wrap-teardown-<session>=). The hook fires when Claude + finishes, sees the sentinel, and runs =cj/ai-term-quit= via =emacsclient=. + Decoupling guarantees the valediction lands before the session dies. +3. *Gate on commit+push verified* — never tear down before the session record + is pushed (wrap-up's existing Step 4 / validation checklist already + enforces push; teardown is strictly after it). +4. *Phrase split — teardown IS the default* (Craig's decision 2026-06-23). + Bare "wrap it up" does the full wrap AND removes the buffer/session/quits — + that's his typical case. The non-destructive variant gets the explicit + qualifier: "wrap it up with summary" summarizes + commits + pushes + + archives but keeps the buffer (no teardown), so the summary stays readable. + So: "wrap it up" → teardown; "wrap it up with summary" → no teardown; + "wrap it up and shutdown" → wrap + poweroff (supersedes teardown, Item 2). + +* Item 2 — "wrap it up and shutdown": 10-count then =sudo shutdown now= + +Recommend: yes, but the safety gate is load-bearing and the countdown has a +rendering gotcha. + +Design: +1. *"Only ai-term left" = hard blocking precondition*, evaluated BEFORE the + countdown. Count live sessions (=tmux ls | grep '^aiv-'= or + =pgrep -fc claude=). If more than this one is alive, ABORT the shutdown, + list what's running, and fall back to a normal wrap. Never power the box + off out from under another active Claude session. This is the most + important part of the item. +2. *The live countdown can't run through Claude's tool output.* The Bash tool + buffers stdout until the command returns, so a =for i in $(seq 10 -1 1); + sleep 1= prints all ten at once at the end, not one per second. It has to + run detached or in Emacs: + - tty writer: =for i in $(seq 10 -1 1); do printf '\rShutting down in %2d…' + "$i" > /dev/tty; sleep 1; done; sudo shutdown now= (backgrounded), or + - an Emacs =run-at-time= timer printing 10→1 in the echo area, then + =(shell-command "sudo shutdown now")=. +3. *Make it abort-able* (Ctrl-C / keypress cancels). A 10-second countdown's + whole purpose is a last-chance window; a non-cancellable one is just a + delay. +4. *Sequencing.* "...and shutdown" supersedes Item 1's teardown — if the box + is powering off, killing the buffer/session first is moot. Wrap (commit + + push + archive) → session-count gate → countdown → =shutdown=. + +Packaging: a small rulesets bin script (e.g. =ai-wrap-shutdown=) doing the +gate → abort-able countdown → shutdown, invoked by the workflow after the wrap +commit/push. Countdown either in that script (tty) or handed to Emacs. + +* Companion — required change in =.emacs.d/modules/ai-term.el= + +Item 1's teardown function =cj/ai-term-quit= must live in =ai-term.el= (it +owns =aiv-<proj>= session naming, the vterm buffer, and geometry restore). +rulesets owns the workflow + hook + bin script that *call* it; =.emacs.d= owns +the function itself. Spec for the =.emacs.d= side: + +- =cj/ai-term-quit (&optional project)= — resolve the =aiv-<basename>= session + for the current/!named project, =tmux kill-session= it, =kill-buffer= the + associated vterm buffer, restore saved geometry. Idempotent / no-op if the + session or buffer is already gone. Callable from =emacsclient -e= so the + Stop hook can invoke it headlessly. +- (Optional) a count helper =cj/ai-term-live-count= so the Item-2 gate can ask + Emacs how many ai-term sessions are live, as an alternative to =tmux ls= / + =pgrep=. + +When rulesets builds the workflow/hook side, route this companion to +=.emacs.d= (inbox-send) so the two land together. + +* Open decisions for Craig + +- Phrase set: DECIDED (2026-06-23) — "wrap it up" tears down (default); + "wrap it up with summary" wraps without teardown; "wrap it up and shutdown" + is the poweroff variant. Remaining nuance: confirm the exact non-destructive + qualifier wording is "with summary" (vs e.g. "and summarize"). +- Countdown home: tty-writer bin script vs Emacs timer. (Emacs timer reads + cleaner inside the vterm and is trivially abort-able.) +- Session-count mechanism for the gate: =tmux ls=, =pgrep claude=, or + =cj/ai-term-live-count=. + +* Verify + +- Item 1: bare "wrap it up" → valediction renders fully, THEN buffer + + =aiv-<proj>= session + claude all gone, geometry restored; "wrap it up with + summary" → wrap completes but the buffer stays intact (no teardown). +- Item 2 gate: with a second =aiv-*= session alive, "wrap it up and shutdown" + refuses, lists the other session, and does a normal wrap (no poweroff). +- Item 2 happy path: sole session → 10→1 renders one-per-second, is + cancellable, then =shutdown= fires. +- Teardown never runs before commit+push is verified. diff --git a/docs/design/2026-06-27-bug-priority-matrix-cover-note.org b/docs/design/2026-06-27-bug-priority-matrix-cover-note.org new file mode 100644 index 0000000..d6398e0 --- /dev/null +++ b/docs/design/2026-06-27-bug-priority-matrix-cover-note.org @@ -0,0 +1,5 @@ +#+TITLE: Cover note for the bug-priority-matrix-proposal.md just sent +#+SOURCE: from emacs-wttrin +#+DATE: 2026-06-27 23:57:30 -0400 + +Cover note for the bug-priority-matrix-proposal.md just sent. WHAT: a Severity × Frequency bug-priority matrix (P1-P4) distilled from wttrin's priority scheme today. It derives a bug's priority from two facts (severity, frequency) instead of opinion, maps each cell to a release vehicle, and — for projects running the todo-format [#A]-[#D] scheme — has bugs derive their letter from the matrix while features keep their roadmap judgment. Includes a special-category rule (privacy/security/safety graded on severity alone) and a tie-in that makes a 'no open [#A]' release gate fact-based. WHY SEND: this is a generic defect-management pattern, not wttrin-specific, so it belongs upstream to distribute with any code-centric project (a claude-rule and/or a todo.org priority-scheme template snippet — your call on placement and whether it folds into todo-format.md or stands alone). TWO ASKS: (1) consider a companion NON-CODING matrix for work-product/ops/decision defects — the same frequency×severity shape likely generalizes beyond bugs; (2) treat this as a LIVING DOCUMENT — adjust bands and wording as projects use it until it's as mature as the coding matrix. SOURCING: the matrix is a standard industry pattern; I adapted a generic version and the wttrin worked-example. Nothing work-confidential is copied — please keep the upstream artifact generic and uncited to any private reference. wttrin keeps its local copy in todo.org as the stopgap until you distribute the canonical. diff --git a/docs/design/2026-06-27-bug-priority-matrix-proposal.md b/docs/design/2026-06-27-bug-priority-matrix-proposal.md new file mode 100644 index 0000000..4b49769 --- /dev/null +++ b/docs/design/2026-06-27-bug-priority-matrix-proposal.md @@ -0,0 +1,70 @@ +# Bug Priority — Severity × Frequency Matrix (proposed rule for code projects) + +Applies to: code-centric projects (a `:bug:`-tracking `todo.org`, an issue tracker, or any defect backlog). + +Status: proposed. Distilled from wttrin's priority scheme on 2026-06-27. Generic defect-management pattern — adapt the bands per project. Treat as a living document; refine the wording and bands as projects use it, the way wttrin's matrix will mature in use. + +## The problem + +Without a systematic scheme, bugs get prioritized by who's loudest, who has the most authority, or who's most frustrated. Most bugs drift to "medium," and no one can say what medium means. Whether a bug is "high priority" stays a matter of opinion, so the release decision becomes a judgment call every time. + +## The approach — derive priority from two facts + +Priority should fall out of two relatively objective factors, not an argument: + +1. Severity — how bad is it when the bug occurs? (Service down / data loss / security or privacy leak at one end; cosmetic nit at the other.) +2. Frequency — how often will a user hit it? (Every user every time at one end; rare edge case at the other.) + +Put severity on one axis and frequency on the other. Each cell maps to a priority level; each level maps to a release vehicle. + +``` +| Frequency / Severity | Critical | Major | Minor | Cosmetic | +|------------------------+----------+-------+-------+----------| +| Every user, every time | P1 | P1 | P2 | P3 | +| Most users, frequently | P1 | P2 | P3 | P4 | +| Some users, sometimes | P2 | P3 | P3 | P4 | +| Rare edge case | P2 | P3 | P4 | P4 | +``` + +Response per level: + +- P1 / Critical — fix in the current release or patch. Blocks a release. +- P2 / High — next patch release. +- P3 / Medium — next major release. +- P4 / Low — backlog, fix when convenient. + +(A P0 / showstopper tier — all hands, emergency release — sits above P1 for projects that need it.) + +## Adapt the bands per project + +The axis labels are generic; each project defines what Critical/Major/Minor/Cosmetic and the frequency rows mean in its own terms. wttrin's worked example: + +- Critical: no weather at all on the common path (crash, hard error), wrong weather shown as if correct (silent data error), or a privacy/security leak. +- Major: a feature broken but with a workaround. +- Minor: degraded but usable. +- Cosmetic: a pure visual nit. +- Frequency rows map to default-config-every-time → common-action → feature/config-specific → unusual-input/environment. + +## Special category — severity alone + +Some defects don't correlate with frequency: privacy/security leaks, compliance violations, safety issues. Grade these on severity alone — one occurrence with the right consequences is a showstopper no matter how rarely it'd be noticed. (wttrin's worked case: a home address leaked into public git history — Critical on severity alone, scrubbed 2026-06-26.) + +## Integrating with a project's `[#A]`–`[#D]` priority scheme + +Projects that already run a letter-priority scheme (the rulesets `todo-format.md` convention) keep it for features — deciding feature X is more urgent than feature Y is a legitimate roadmap judgment. Bugs are different: their letter should be derived, not argued. + +Rule: a `:bug:`'s letter comes from the matrix, not a choice. + +- P1, P2 → `[#A]` (release-gating) +- P3 → `[#C]` (scheduled fix, has a workaround) +- P4 → `[#D]` (backlog) + +(The exact letter mapping is a per-project knob; this is wttrin's.) Features keep their `[#A]`–`[#D]` judgment as a roadmap call; only bugs derive their letter from the matrix. + +## Tie-in to release criteria + +When a project gates a release on a soak or a "no open [#A]" rule, the matrix makes that gate fact-based: "no open [#A] bugs" becomes "no open P1/P2," which the matrix produces. A severity-graded soak follows naturally — a P1/P2/P3 found in the window resets the clock and must be fixed; a P4 logs to backlog without resetting. + +## The payoff + +Once each priority maps to a release vehicle, you've pre-decided most issues and only discuss the genuine edge cases. The release decision stops being a soul-search and becomes arithmetic. When there's disagreement on a bug's priority, the matrix wins. diff --git a/docs/design/2026-06-29-green-baseline-proposal.org b/docs/design/2026-06-29-green-baseline-proposal.org new file mode 100644 index 0000000..47de18d --- /dev/null +++ b/docs/design/2026-06-29-green-baseline-proposal.org @@ -0,0 +1,72 @@ +#+TITLE: Proposal: ensure a green test run before starting work +#+AUTHOR: Craig Jennings (via .emacs.d session) +#+DATE: 2026-06-29 + +* Why + +While running a multi-task refactor speedrun in =.emacs.d=, the very first full +=make test= surfaced a failing test (=test-system-cmd-restart-emacs-no-service-aborts=) +that turned out to be *pre-existing* -- it failed on clean HEAD, unrelated to +the work. It had nothing to do with the task; it just happened to fail on this +machine (a native-comp mock that bypasses =symbol-function= redefinition, real +check passing because the box has =emacs.service=). + +Two costs landed because the red was already there when work began: + +1. Every later "did I break this?" suite run carried a known failure, so the + green bar became "only that one fails" instead of a clean pass I could read + at a glance. Easy to let a *new* regression hide behind the familiar red. +2. The work assumed the tree was in a known-good state. It wasn't, and nothing + in the workflow forced that assumption to be checked first. + +The fix is cheap and general: run the suite *before* starting work, and clear +(or explicitly triage) any failure before the work begins. A green start +confirms we're in the known-good place we think we are, and any issue is fixed +before it can be confused with our own changes. + +* Proposed change 1 -- claude-rules/verification.md + +Add a section (suggested placement: right after =## The Rule=, before +=## What Fresh Means=). Proposed text, ready to paste: + +#+begin_example +## Green Baseline Before Starting Work + +Run the test suite before you start work, not only before you finish. A clean run at the start confirms the tree is in the known-good state you assume it is, so the baseline you build on and measure your changes against is actually green. + +If the suite is red before you touch anything, fix or explicitly triage the failure first. A pre-existing failure left in place poisons every later "did I break this?" check: you can't separate your own regressions from the noise, and the end-of-work run stops being readable as pass/fail at a glance. Work that assumes a known-good base may also be built on a broken assumption you never saw. + +When a pre-existing failure genuinely can't be fixed before the work begins (out of scope, or it needs a decision), record it as a tracked task with the diagnosis and carry its name forward. The green bar for the rest of the work is then explicitly "only this known failure remains," not a silent tolerance for red. + +This is the start-of-work counterpart to the Before Committing gate below: one confirms the ground is solid before you build, the other confirms you didn't crack it. +#+end_example + +* Proposed change 2 -- start-work skill, Pre-work phase + +The start-work skill already has a Pre-work phase (eligibility, fetch-and-reconcile +against base, source-code check that the problem still exists). Add a green-baseline +step to that phase: + +- Run the project's test suite before claiming the work. +- If it's fully green, proceed. +- If it's red, fix the failure first, or (when out of scope / needs a decision) + file a tracked task with the diagnosis and carry its name forward as the only + tolerated failure for this work. +- Surface the baseline result so "we started from green" is on the record. + +This makes the verification.md principle operational at the exact moment it +matters -- the start of a task -- the same way the Verify phase and the +Review-and-Publish flow operationalize the end-of-work gates. + +* Note on why this came as a proposal, not a direct edit + +=.emacs.d='s =.claude/rules/*.md= are symlinks into =~/code/rulesets/claude-rules/=, +so editing =verification.md= from the downstream session would modify the +rulesets canonical directly. Per the cross-project rule, downstream sessions +send rulesets the proposed change rather than editing its canonical in place. +Hence this note instead of a local stopgap edit. The start-work skill isn't +installed on this machine to edit anyway. + +The test that surfaced all this is already fixed in =.emacs.d= (commit on main: +mock =executable-find= at the boundary instead of the helper). The durable +process change is the part that belongs here. diff --git a/docs/design/2026-06-29-lint-org-structural-checkers-proposal.org b/docs/design/2026-06-29-lint-org-structural-checkers-proposal.org new file mode 100644 index 0000000..c464aca --- /dev/null +++ b/docs/design/2026-06-29-lint-org-structural-checkers-proposal.org @@ -0,0 +1,55 @@ +#+TITLE: lint-org.el — four structural heading checkers org-lint doesn't cover + +* What changed (from .emacs.d, 2026-06-29) + +Added four custom judgment checkers to =lint-org.el=, following the existing +=lo--check-tables= / =lo--check-level2-dated-headers= pattern (custom scans run +after the org-lint pass, emitting judgment items, never auto-fixed): + +- =indented-heading= — a line of whitespace + stars + space OUTSIDE any block. + org parses a heading only at column 0, so leading whitespace silently demotes + a would-be heading to body text: the task vanishes from the agenda and never + archives. The worst defect class (an invisible task) and entirely silent + today. Skips indented stars inside =#+begin_/#+end_= blocks (legit content). +- =empty-heading= — a line of bare stars with no title. +- =malformed-priority-cookie= — a =[#x]=-shaped token org rejected (lowercase, + multi-char, non-letter) left stranded where a cookie would be. Checks only the + first cookie token per heading; skips verbatim-wrapped =[#D]= in dated-log + titles. +- =level2-done-without-closed= — a level-2 DONE/CANCELLED with no CLOSED line. + Directly supports the todo-cleanup aging step (sent separately today): an + undated completed task gets force-archived immediately, so flagging it lets + the human add CLOSED first. + +Two attached files (edited canonical candidates): =lint-org.el=, +=tests/test-lint-org.el=. + +* Why + +org-lint validates links, drawers, blocks, and babel — but NOT heading +well-formedness. On Craig's .emacs.d todo.org a missing org-bullet in the live +buffer prompted the question "is the file structurally okay?", and org-lint +(even unfiltered, all checkers) reported nothing actionable. These four close +the gap. They are general (any org file), not project-specific. + +* Design notes for the canonical + +- All four are regex-based, NOT org-element/keyword-based, so they don't depend + on which TODO keywords the batch Emacs happens to recognize (lint-org.el does + not set =org-todo-keywords=). The =level2-done-without-closed= done set is a + defconst =lo-done-keywords= (DONE/CANCELLED) for easy extension. +- *Gotcha worth carrying in the canonical:* =case-fold-search= defaults to t, so + a naive =[A-Z]= cookie check accepts =[#a]= as valid and =\(DONE\|CANCELLED\)= + matches the title words "done"/"cancelled". Both letter-sensitive checkers + bind =case-fold-search nil=. (Caught by a failing test before it shipped.) +- Wired into =lo-process-file= after =lo--check-level2-dated-headers=. Judgment + output already flows through the existing report + followups-file machinery. +- 8 new ERT tests (good-input-silent + bad-input-flagged for each, plus + block-skip and verbatim-skip boundary cases). 44/44 green. Zero false + positives on a real 5600-line todo.org. + +* Note + +=make task-sorted= in .emacs.d now runs =lint-org.el todo.org= after the +archive, so these checkers also gate the task-hygiene target. Makefiles aren't +template-synced; that wiring is project-local (noted for context). diff --git a/docs/design/2026-06-29-todo-cleanup-aging-proposal.org b/docs/design/2026-06-29-todo-cleanup-aging-proposal.org new file mode 100644 index 0000000..5a18990 --- /dev/null +++ b/docs/design/2026-06-29-todo-cleanup-aging-proposal.org @@ -0,0 +1,64 @@ +#+TITLE: todo-cleanup.el — add Resolved-section file-aging to --archive-done + +* What changed (from .emacs.d, 2026-06-29) + +Extended =todo-cleanup.el='s =--archive-done= mode (the =make task-sorted= +target) with a SECOND step, run after the existing Open Work -> Resolved move: + +- *Age the Resolved section.* Level-2 DONE/CANCELLED subtrees whose CLOSED date + is older than =tc-archive-retain-days= (default 7) — AND any with no parseable + CLOSED date — move out of the in-file Resolved section to =tc-archive-file= + (default =archive/task-archive.org= beside the todo file). Only tasks closed + within the last week stay in todo.org itself. + +Two files are attached (the edited canonical candidates): +- =todo-cleanup.el= +- =tests/test-todo-cleanup.el= + +* Why + +Craig's .emacs.d todo.org had grown to 768KB / 9616 lines, ~44% of it a +243-task in-file "Resolved" section. The existing =--archive-done= only moved +closures Open Work -> Resolved (same file), so the file grew without bound. The +new step keeps only the last week of closed tasks in the file and sheds the rest +to a git-tracked archive sibling. After this run: 207 aged out, todo.org +9616 -> 5625 lines. + +* Design notes for the canonical + +- New defvars: =tc-archive-retain-days= (7; nil disables the step, preserving + legacy in-file-only behavior), =tc-archive-reference-date= ((YEAR MONTH DAY), + nil=real today — mockable for deterministic tests), =tc-archive-file= (nil => + =archive/task-archive.org= beside the todo file). +- Policy: KEEP iff CLOSED date present AND within the window (cutoff inclusive). + Older OR undated => archive. The undated->archive call is deliberate ("keep + the last week and that's it"); an earlier undated->keep version left 14 legacy + undated tasks behind and read as two weeks. +- The aging step honors =--check= (previews + reports, writes nothing). +- Report: an additive "N aged subtree(s) moved to task-archive.org" line, only + when N>0, so the existing real-mode-no-op silence tests are unaffected. +- Archive file scaffold: =#+TITLE: Task Archive= / =#+FILETAGS: :archive:= / + =* Resolved (archived)=; aged subtrees append as level-2 children; created on + first use, appended to thereafter (one scaffold, never duplicated). +- Tests: =tc-test--reset= now sets the aging knobs OFF (retain nil) so the + existing in-file-move + sync tests are untouched by the wall clock; a new + =tc-test--age= harness re-enables them with a fixed reference date and a temp + archive file. 6 new tests (old+undated move, cutoff-inclusive stay, disabled, + idempotent, check-no-write, straggler pipeline, append-preserves). 34/34 green. + +* Cross-project consideration for your value gate + +Default is ON (retain 7) for ALL consuming projects once this syncs. A project's +first =task-sorted= after the sync will shed everything in its Resolved section +older than a week to a new =archive/task-archive.org=. That's the intended +feature, but flag it — projects with a large historical Resolved section will see +a big first-run move (git-tracked, recoverable). Adjust the default or gate it if +you'd rather it be opt-in per project. + +* Companion (project-local, NOT synced) + +.emacs.d's Makefile =task-sorted= target now also runs =lint-org.el todo.org= +after the archive, as a structural-safety pass (org-lint catches links/drawers/ +blocks; we separately verified heading-level structure by hand). Makefiles aren't +template-synced, so this is per-project — noting it in case the pattern is worth +documenting alongside the tool. diff --git a/docs/design/2026-06-30-daily-drivers-tailscale-correction.org b/docs/design/2026-06-30-daily-drivers-tailscale-correction.org new file mode 100644 index 0000000..9e2bb52 --- /dev/null +++ b/docs/design/2026-06-30-daily-drivers-tailscale-correction.org @@ -0,0 +1,9 @@ +#+TITLE: Correction to claude-rules/daily-drivers.md: it states 'the +#+SOURCE: from .emacs.d +#+DATE: 2026-06-30 13:20:30 -0400 + +Correction to claude-rules/daily-drivers.md: it states 'the agent can't reach the other machine; the point is to surface...'. That assumption is now demonstrably false. On 2026-06-30, from velox, I drove ratio directly over tailscale ssh — git fetch + reset --hard to repair ratio's .emacs.d after a history rewrite, plus scp'd a file across. Both daily drivers are on the same tailnet and reachable from each other. + +Gotcha worth capturing in the rule: the BARE hostname does not resolve (ssh ratio -> 'Could not resolve hostname'), which makes it look unreachable. The tailscale IP (e.g. 100.71.182.1) and the MagicDNS name (ratio.tailf3bb8c.ts.net) DO resolve and connect. First connection from a given address fails host-key verification under BatchMode; -o StrictHostKeyChecking=accept-new clears it. 'tailscale status' lists every node's IP + online state. + +Suggested rule change: reframe daily-drivers.md from 'can't reach, so surface it' to 'CAN reach over tailscale ssh — so the agent can directly sync/verify/repair the other daily driver, not just flag it'. Keep the flag-it guidance as the fallback for when tailscale is actually down. Add the bare-hostname-doesn't-resolve / use-tailscale-IP-or-MagicDNS gotcha. uname -n still tells you which machine you're on. diff --git a/docs/design/2026-07-02-auto-flush-mechanism-note.org b/docs/design/2026-07-02-auto-flush-mechanism-note.org new file mode 100644 index 0000000..fbe06ae --- /dev/null +++ b/docs/design/2026-07-02-auto-flush-mechanism-note.org @@ -0,0 +1,20 @@ +#+TITLE: AUTO-FLUSH capability — proven live in the archsetup session +#+SOURCE: from archsetup +#+DATE: 2026-07-02 01:26:20 -0400 + +AUTO-FLUSH capability — proven live in the archsetup session 2026-07-02, Craig asks that it be promoted to all projects and recommended as part of the no-approvals speedrun to keep sessions sharp. + +Problem: /clear is a user-only keystroke, so long autonomous sessions either bloat or hit arbitrary auto-compaction. Craig can't always be around to type it. + +Mechanism (companion script: self-inject.sh, sent separately to this inbox): +1. At a clean task boundary, the agent refreshes .ai/session-context.org exactly as the flush skill does (checkpoint with Active Goal / Decisions / Next Steps). +2. It derives its own tmux pane: match pane_pid from 'tmux list-panes -a' against its process ancestry (the ai launcher runs every agent session inside tmux, so this holds everywhere). +3. It arms the injection VIA THE TMUX SERVER — tmux run-shell -b "sleep 25; tmux send-keys -t %N -l '/clear'; tmux send-keys -t %N Enter; sleep 15; tmux send-keys -t %N -l 'go — auto-flush resume: read .ai/session-context.org and continue per Next Steps'; tmux send-keys -t %N Enter" — and immediately ends its turn so the prompt is idle when the keys land. +4. /clear fires the SessionStart hook (which already points a fresh context at notes.org + session-context.org), and the injected resume line starts the next turn. Zero human keystrokes. + +Gotchas learned the hard way: +- A detached child (setsid/nohup/&) of a tool call DIES when the tool call ends; only tmux run-shell -b (server-owned) survives the turn boundary. +- Under run-shell the process is a child of the tmux server, so ancestry-based pane detection can't run there — derive the pane first from the agent's shell, pass it explicitly. +- Collision: if the user is typing when the keys fire, the injection merges into their input (a real /clear became '/clearto' mid-word). Fine for unattended sessions; warn the user to keep hands off the armed window if present. + +Suggested integration: an 'auto' mode on the flush skill (checkpoint, then self-inject instead of prompting the user), plus a line in the no-approvals speedrun workflow to auto-flush at clean boundaries when context grows heavy. The script could live in claude-templates' .ai/scripts/ so every project gets it on sync. diff --git a/docs/design/2026-07-13-runtime-portability-inventories.org b/docs/design/2026-07-13-runtime-portability-inventories.org new file mode 100644 index 0000000..b9e8244 --- /dev/null +++ b/docs/design/2026-07-13-runtime-portability-inventories.org @@ -0,0 +1,84 @@ +#+TITLE: Runtime Portability — Hook, MCP, and Memory Inventories +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-13 + +Decision-prep for the generic-agent-runtime arc (spec: [[file:2026-05-28-generic-agent-runtime-spec.org]]). Three of the eight gap child tasks are inventories; this doc is their deliverable. Each section ends with a verdict and any decision left for Craig. + +* Hook parity inventory + +What's wired today (global =.claude/settings.json=, symlinked to =~/.claude/settings.json=): + +- PreToolUse AskUserQuestion hard-deny — blocks the popup-choice tool, forcing inline numbered options per =interaction.md=. Exists because prose alone failed. +- PreCompact =precompact-priorities.sh= — saves priorities before context compaction. +- SessionStart =session-title.sh= (cosmetic title) and =session-clear-resume.sh= (resume-after-/clear plumbing). +- Stop =ai-wrap-teardown.sh= — sentinel-gated session teardown. + +Shipped but unwired (reference copies in =hooks/= + =settings-snippet.json=): =git-commit-confirm.py=, =gh-pr-create-confirm.py=, =destructive-bash-confirm.py=. Not active under Claude either, so they impose no parity requirement today. + +Language bundles wire PostToolUse validators per edit (=validate-bash.sh=, =validate-el.sh=, =validate-go.sh=, plus python/typescript equivalents) — and every bundle also ships the same validation as a =githooks/pre-commit=. + +Per-hook portability verdicts: + +- AskUserQuestion deny: moot off Claude. The popup tool is Claude Code's; Codex-style and local harnesses have no equivalent tool to deny. No port needed. +- PostToolUse validators: covered off Claude. The bundles' git pre-commit hooks run the same validators, so enforcement survives at commit time instead of edit time. Acceptable downgrade, no work needed. +- SessionStart clear-resume: not portable as-is; /clear semantics are Claude's. This is the same surface as the "session plumbing per runtime" child task — resolve it there, not as a hook port. +- session-title: cosmetic, skip on other runtimes. +- PreCompact priority-save: genuine gap. Codex-style harnesses compact without a hook point. Downgrade is prose (the session-context write triggers in protocols.org already say "save before compaction"), which is weaker but is also the pre-2026 Claude behavior. Accept prose, or shorten sessions on those runtimes. +- Stop wrap-teardown: partial gap. Codex has a turn-completion notify command that could run the same sentinel-gated script; local harnesses vary. Port the script trigger where a hook point exists; where none does, teardown becomes a manual step in the wrap phrase. + +Verdict: only PreCompact and Stop carry real porting work, and both have acceptable degraded modes. Nothing here blocks a pilot. + +* MCP portability check + +Locally-configured servers (user scope, =~/.claude.json=): linear, notion, figma, slack-deepsat, google-calendar, google-docs-personal, google-docs-work, drawio, google-keep. No project-scope servers anywhere. All are ordinary MCP server definitions, portable to any MCP-speaking harness (Codex CLI reads them from its config; most local harnesses that matter speak MCP now). Port = translate the server blocks + carry the auth material. Mechanical. + +claude.ai-managed connectors (session-injected, not in local config): Gmail, and the claude.ai variants of Calendar/Drive. These do not travel. Overlap analysis: calendar and docs are already covered by the locally-configured servers; Gmail's workflow-preferred path is already =cmail-action= (local CLI), so the managed Gmail connector is convenience, not dependency. + +Named gap: signal-mcp. protocols.org "Paging Craig" names it as the only supported away-from-desk page path, and it is not in any local config — it lives claude.ai-side. Off Claude there is currently no page channel at all. This lands squarely on the open Signal-pager task ([#C], todo.org): the signal-cli runbook that task produces would be the runtime-neutral paging path. Recommend noting runtime-portability as an added motivation on that task. + +Verdict: portable except paging; paging's fix is already a filed task. + +* Memory story for non-Claude agents + +Claude auto-memory (=~/.claude/projects/<enc>/memory/=, MEMORY.md index) is harness-owned: written by Claude's memory tooling, loaded by its session start. A non-Claude agent should neither read nor write it. + +The designed cross-agent store already exists: the org-roam KB (=knowledge-base.md= — reading is plain rg over files, writing is one node per fact, both runtime-neutral). Project state lives in file artifacts every agent reads anyway: todo.org, notes.org, session anchors, docs/. + +One wording gap: knowledge-base.md's "Capture, then promote" section names harness memory as the capture layer. For a non-Claude agent the capture layer is the session log itself. Recommended one-sentence addition there: an agent without harness memory captures into its session log and promotes from it at wrap-up. Shared-asset edit, needs approval. + +Verdict: no build. One approved sentence in knowledge-base.md closes it. + +* Skill and command parity + +What rulesets ships: 11 skills (SKILL.md, model-invokable — add-tests, debug, five-whys, flush, frontend-design, pairwise-tests, playwright-js, playwright-py, review-code, root-cause-trace, voice) and 18 commands (plain .md prompts under =.claude/commands/=, user-invoked). Every body is markdown instructions; nothing executable lives in the registration layer. + +The load-bearing observation: a =/name= reference is just a pointer to a file on disk. Any harness that reads files can execute all 29 artifacts through one resolution rule in its bootstrap entry file: "a skill or command reference (=/voice=, =/review-code=, =/brainstorm=) resolves to =~/.claude/skills/<name>/SKILL.md= or =~/.claude/commands/<name>.md= — read the file and follow it." That single sentence, emitted by the instruction-bootstrap install target, makes the whole library portable with zero per-skill work. commits.md's existing "/voice unavailable — walk the patterns inline" fallback generalizes the same way. + +What that rule does not carry: + +- Auto-invocation. Claude Code triggers skills from their descriptions; other harnesses won't. The high-value auto-triggers (voice and review-code inside the publish flow) don't actually need it — commits.md invokes them by name at fixed flow points, and the resolution rule covers a by-name invocation. The convenience triggers (debug, frontend-design firing on topic match) degrade to on-request. Acceptable. +- Native slash registration. Codex-style harnesses have their own custom-prompt mechanism; the 18 commands could also be registered there for ergonomics (an optional install nicety, not a requirement — the resolution rule already works). +- flush. Its mechanics are Claude Code's (/clear, the self-inject resume hook). Explicitly not ported; the session-plumbing child owns that surface. +- Harness built-ins (code-review ultra, plan mode, artifacts). Not rulesets' to port; flows that name them need per-runtime alternatives or graceful absence, same as today when they're unavailable. + +Verdict: no per-skill porting matrix needed. One resolution sentence in the bootstrap entry file (instruction-bootstrap child), optional native registration as an install nicety, flush explicitly excluded. + +* Session plumbing per runtime + +Four pieces, examined: + +- =session-context-path= is already runtime-aware by design — the =AI_AGENT_ID= shape carries a runtime segment (=host.project.runtime.epoch=) and the resolver is pure bash. Nothing to do. +- The anchor cycle (suspend entry, startup's interrupted-session recovery, wrap-up's archive rename) is plain files driven by workflow prose. Any agent that reads protocols.org executes it identically. Nothing to do. +- =self-inject.sh= is harness-agnostic: it types arbitrary strings into a tmux pane. The Claude-bound part is only the *payload* — "/clear" plus a resume line that leans on the SessionStart hook. A Codex auto-flush is a payload variant: the clear command differs (confirm the exact one when wiring) and the second injected line must itself carry the resume instruction ("read .ai/session-context.org and resume") since no hook fires. Arguably simpler than the Claude path. +- =session-clear-resume.sh= (the SessionStart hook) is Claude-only, and the payload variant above makes it unnecessary off Claude. + +Verdict: no build. The flush *skill* stays Claude-official; a codex auto-flush is a documented payload variant to wire the day a codex session actually wants it, not before. + +* Decisions for Craig + +All four approved by Craig, 2026-07-13: + +1. PreCompact downgrade off Claude: prose-only "save before compaction" accepted. +2. Stop-teardown off Claude: port via Codex notify where available, manual teardown elsewhere. +3. Runtime-portability motivation note added to the Signal-pager task. +4. knowledge-base.md capture-layer sentence added (non-Claude runtimes capture into the session log and promote from there). diff --git a/docs/design/2026-07-14-sentry-workflow-proposal.org b/docs/design/2026-07-14-sentry-workflow-proposal.org new file mode 100644 index 0000000..d29ceb8 --- /dev/null +++ b/docs/design/2026-07-14-sentry-workflow-proposal.org @@ -0,0 +1,150 @@ +#+TITLE: Sentry Workflow — Proposal for rulesets +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-13 + +* Status +Proposal from the work project to the rulesets session, for implementation as a +general (shared-layer) workflow: =.ai/workflows/sentry.org= plus a small +supporting script or two under =.ai/scripts/=. + +* Problem + +An agent session left running overnight can keep a project clean and healthy on +a cadence, but the current pieces don't coordinate. Two gaps: + +1. *No agent-vs-agent collision guard on shared public files.* =capture-guard= + only detects Emacs org-capture buffers (the human case) on one target file, + and it's advisory, not a lock. There is no flock anywhere in =.ai/scripts/=. + Multiple sessions (several projects, sometimes both daily drivers) can + read-modify-write =~/org/roam/inbox.org= and the =agents/= KB nodes at once. + The only coordination is the git pull/commit/push discipline in + knowledge-base.md, which is optimistic — a pull/push race forks a + =*sync-conflict*= file, and the KB search already globs those out, so we know + the forks happen and get dropped. + +2. *No single serialized runner.* Firing inbox-zero and triage-intake as + separate hourly crons means they can overlap each other and other agents. + +Sentry is one supervisor loop that runs a fixed sequence of hygiene passes +serially, holds locks so it never collides with itself or other agents, and +checkpoints aggressively for crash recovery. + +* What sentry is + +A single workflow, driven by one interactive =/loop= (one cron), that on each +fire walks an ordered list of passes inside a single turn. It replaces the +per-task crons. Design invariants: + +- *Serial.* Passes run one at a time, in order. They never overlap each other. +- *Single-runner lock.* Sentry takes a project-local flock + (=.ai/.sentry.lock=) at entry. If a prior fire is still running, this fire + skips instead of doubling up. +- *Shared-file lock.* Any pass that writes a public roam file first acquires a + roam-write lock (flock on =~/org/roam/.roam-write.lock=) held across the whole + read-modify-write-commit-push cycle, so it serializes against other agents on + the same host. =capture-guard= still runs underneath for the human case. (The + roam-write lock is new; it's the general guard the dropped KB-guard note would + have proposed, folded in here.) +- *Non-destructive by default.* Read-only and idempotent-filing passes run + fully. Anything destructive or judgment-heavy (triage trash/mark-read, task + regrades, spec flips) queues its proposal into the digest for morning + approval, never fires unattended. +- *Idempotent + isolated.* Every pass is safe to repeat. A pass that errors + doesn't abort the rest; its failure is a loud banner in the digest. + +* Crash-recovery spine (applies to every pass) + +Crash recovery has been a live consideration. Two enforced steps wrap each pass: + +1. *Session-context between passes.* Before the next pass starts, the current + pass appends a dated entry to =.ai/session-context.org= recording what it + did. A crash mid-sequence leaves a readable "got this far" trail, so the next + session resumes without re-deriving state. +2. *Local commit, no push, on any disk change.* Every pass that writes to disk + gets its own =chore(sentry): <pass> — <what changed>= commit on the current + branch, unpushed. A crash loses at most one pass. Craig reviews the stack in + the morning and pushes deliberately. Conventional messages, no AI + attribution. (Open decision: commit on the current branch vs a dedicated + =sentry/<date>= branch — see Open decisions.) + +The unit per pass is therefore: do the work → log to session-context → commit. +That triplet is the recovery boundary. + +* Passes (v1 — all ten, plus the first-run KB promotion) + +Ordered so read-only/pull passes precede writes, and the heaviest external +sweep sits mid-run: + +1. *Roam pull* — =git -C ~/org/roam pull --ff-only= so later reads are fresh. + Read-only. +2. *Inbox zero* — inbox.org roam mode. Route roam-inbox items this project + owns. Acquires the roam-write lock for the inbox edit. +3. *Triage intake* — triage-intake.org. External-accounts sweep, classify, + file Action tasks. Destructive actions (trash/mark-read) queue for morning; + =mbsync -a= and the sentinel advance run. +4. *Todo cleanup* — clean-todo.org + todo-cleanup.el + lint-org.el. Convert + level-3 done subtasks to dated logs, archive done subtrees, flag lint. +5. *Task audit* — task-audit.org. Re-check =:solo:=/=:quick:= tags, + priority-scheme conformance, bug severity×frequency grades, stale =DOING= + specs with closed parents, blocked/blocker reciprocity. Regrades queue for + morning. +6. *Working-files hygiene* — flag =working/<slug>/= dirs whose task is DONE but + not filed to =assets/=. Report only; filing is a morning decision. +7. *Spec status board* — the docs-lifecycle grep. Flag =DOING= specs with + closed parents, or specs stuck in =DRAFT=. +8. *Link integrity* — lint-org broken =file:= links across todo/notes/prep. +9. *Git health* — uncommitted drift, unpushed commits, stale local branches, + whether main is behind origin. +10. *Prep + symlink freshness* — verify tomorrow's prep and both symlinks + resolve. +11. *KB lesson promotion (first run of the session, and thereafter as new + lessons accrue)* — promote recent durable lessons out of the fast capture + layer. CLASSIFICATION-GATED: + - Personal project → write to the roam KB as =agents/= nodes (pull, lock, + write, commit, push), per knowledge-base.md. + - Work / denylisted project → NEVER touch roam. Promote to the project's own + durable store (e.g. =deepsat/knowledge.org=) instead, and emit the + one-line refusal note from the knowledge-base.md refusal contract so + nothing is lost silently. + +* Digest output + +One consolidated, timestamped summary per fire: run time first, then per-pass +one-liners (what it found / did), the morning-approval queue (proposed +trash/mark-read, regrades, files-to-move), and any failures as loud banners at +the top. Delta-only where a pass supports it (triage): a pass with nothing to +report is one line. + +* Cadence + +One hourly sentry to start (passes short-circuit fast when there's nothing to +do). A slower evening interval (every 2–3 hours) is a config knob if hourly +proves noisy. + +* Open decisions for rulesets + +1. *Commit target* — per-pass commits on the current branch (simplest, matches + "commit but don't push") vs a dedicated =sentry/<date>= branch (keeps the + working branch clean, costs a branch dance). Proposal leans current branch. +2. *Roam-write lock scope* — host-local flock only (solves same-host + multi-agent, the common case) vs also hardening roam-sync to surface + cross-host conflicts loudly. Proposal: ship the flock now, note the + cross-host limit, treat roam-sync conflict-surfacing as a separate follow-up. +3. *Interval default* — hourly vs evening-only. Proposal: hourly, config knob. +4. *Which passes gate on a green tree* — e.g. skip todo-cleanup commits if the + project suite is red, matching inbox monitor-mode's clean-tree gate. + +* Cover note from sender (work, 2026-07-14 00:02) + +Intro for the sentry-proposal.org file just sent. This is a NEW general workflow proposal (.ai/workflows/sentry.org + supporting scripts under .ai/scripts/), not an edit to an existing synced file. + +What it is: one serialized supervisor loop that runs a project's evening hygiene passes on a cadence, with locks so it never collides with itself or with other agents. It replaces firing inbox-zero + triage-intake as separate crons. + +Why now: an agent left running overnight can keep a project clean, but the current pieces don't coordinate. capture-guard only handles the Emacs-capture case on one file and holds no lock; there is no flock anywhere in .ai/scripts/; and separate crons let inbox-zero and triage overlap each other and other agents on shared roam files, which already fork *sync-conflict* files that the KB search globs out. + +Three design points worth your judgment: +1. Crash-recovery spine on every pass (crash recovery has been a live concern): enforce a session-context.org update BETWEEN passes, and a local commit (no push) after any disk change, so a crash loses at most one pass. +2. A new roam-write flock (~/org/roam/.roam-write.lock) as the shared-file guard. This absorbs a separate 'guard all public KB files' note we considered and dropped. The lock is the better home for that concern than a bespoke per-file guard. +3. The KB lesson-promotion pass is classification-gated: personal projects promote to roam agents/ nodes; a denylisted work project promotes to its own local store (deepsat/knowledge.org) and never touches roam, per knowledge-base.md's refusal contract. + +Companion files to reconcile: knowledge-base.md (the gated promotion + refusal contract), inbox.org (roam mode + the new lock), triage-intake.org (runs as a sentry pass; keep its own triggers), capture-guard (becomes one layer under the roam-write lock, not the whole guard), and clean-todo.org / task-audit.org / lint-org.el (invoked as passes). Open decisions for you are listed at the bottom of the proposal file: commit target, lock scope, interval default, and which passes gate on a green tree. diff --git a/docs/design/2026-07-15-subproject-pattern-proposal.org b/docs/design/2026-07-15-subproject-pattern-proposal.org new file mode 100644 index 0000000..61ade18 --- /dev/null +++ b/docs/design/2026-07-15-subproject-pattern-proposal.org @@ -0,0 +1,22 @@ +#+TITLE: Proposal: promote the "subproject" pattern into the claude-r +#+SOURCE: from home +#+DATE: 2026-07-15 23:24:07 -0500 + +Proposal: promote the "subproject" pattern into the claude-rules layer. + +WHAT WE DID (in home, 2026-07-15) +We named and built a pattern for what happens when a standalone project is consolidated into another project. home absorbed nine former projects on 2026-06-11 (danneel, finances, jr-estate, kit, health, documents, clipper, elibrary, philosophy). We now call each folded-in unit a "subproject": a former standalone project that lives as a subdirectory, shares the parent's .ai/ scope (one session, one toolchain), but keeps its own files, assets, and history self-contained under its subdir. + +The pattern has four parts: +1. Vocabulary — parent project, subproject, consolidation (the process), subproject archiving. Terminology so we can talk precisely about where a subproject's documents/assets live and archive them cleanly. +2. Subproject brief — a read-first orientation doc at <subproject>/<subproject>-brief.org (situation, current state, reading list, and for deep ones cast/timeline/key-facts-with-verify-flags). The parent's notes.org holds a thin index + a read-first rule + pointers to urgent items only, never a copy of subproject substance ("one fact, one home"). +3. Lifecycle criteria — when to CREATE a subproject (consolidate: domain fit, low independent cadence, shared-scope safety re: privacy/remote posture, no independent collaborators), and when/how to ARCHIVE one cleanly (terminal state -> move the self-contained subdir to <parent>/archive/, reconcile pointers/links/tags, manifest note). +4. Instrumentation — a use/miss log, metrics weighted toward Craig-caught errors over self-report, review folded into task-audit, and a falsifiable kill criterion. + +WHY IT'S GENERAL (not just a home thing) +Consolidating a project into another drops what the former project's own startup used to auto-load (its full depth), so sessions start cold and re-read files. That failure recurs whenever ANY project folds into another. The pattern also gives consolidation and archiving a named, repeatable shape, and the self-contained-subdir rule is what makes archiving a move-plus-pointer-reconcile instead of an excavation. + +THE PROPOSAL +Promote this to a rulesets claude-rule (e.g. claude-rules/subprojects.md): the vocabulary, the brief convention, the parent-vs-subproject content criterion, and the create/archive criteria. Any parent/hub project would then inherit it. home's .ai/subprojects.org (attached next) becomes the local instance/index under the general rule. It's a companion to working-files.md (in-progress artifact layout) and docs-lifecycle.md (formal-doc lifecycle) — same family, different scope: this one governs whole-subproject orientation and lifecycle. + +Attached: the home instance .ai/subprojects.org as the reference draft to adapt. Apply your value gate; this is a proposal, not a synced-file edit. diff --git a/docs/design/2026-07-15-subprojects-convention-home-instance.org b/docs/design/2026-07-15-subprojects-convention-home-instance.org new file mode 100644 index 0000000..b031a41 --- /dev/null +++ b/docs/design/2026-07-15-subprojects-convention-home-instance.org @@ -0,0 +1,282 @@ +#+TITLE: Subprojects — Convention +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-15 + +# The reference spec for the subproject pattern: what a subproject is, when to +# create one (consolidation), how to keep it legible (the brief + content +# split), and how to archive one cleanly. Not read every session — the +# always-on parts (the subproject index + the read-first rule) live in +# notes.org and point here. Read this when consolidating a project, creating +# or maintaining a subproject brief, or archiving a subproject. + +* Vocabulary (say it the same way every time) + +The terminology is the point — it's what lets us talk precisely about where a +subproject's documents and assets live, and archive them cleanly. + +- *Parent project* (a.k.a. hub) — a top-level project that owns an =.ai/= + scope: its own protocols, session cadence, memory, inbox, todo. home, work, + rulesets, .emacs.d are parent projects. Reached by launching an agent + session in its directory. +- *Subproject* — a formerly-standalone project that has been *consolidated* + into a parent and now lives as a subdirectory of it (=danneel/=, + =finances/=, =kit/= …). A subproject shares the parent's =.ai/= tooling and + session — it is not a separate agent scope — but keeps its own working + files, assets, notes, and history self-contained under its one subdirectory. +- *Consolidation* — the process of turning a standalone project into a + subproject of a parent. (home's nine subprojects were consolidated on + 2026-06-11.) The inverse — promoting a subproject back to a standalone + project — is rare but allowed; it's just consolidation run backward. +- *Subproject archiving* — retiring a completed or abandoned subproject + cleanly. Because a subproject is self-contained under one subdirectory, + archiving is a move plus a pointer reconcile, not an excavation. +- *Subproject brief* — the read-first orientation doc for one subproject, at + =<subproject>/<subproject>-brief.org=. + +The cross-project boundary rule (=claude-rules/cross-project.md=) governs +*parent* projects, which are separate =.ai/= scopes. It does *not* apply +between a parent and its subprojects — they share one scope. + +* Why the pattern exists + +As a standalone project, each of these units auto-loaded its own notes.org — +its full depth — at every startup, so recall felt instant. Consolidated into a +parent, a subproject is one compressed section competing with the others, and +the parent's startup surfaces hub-wide state, not any one subproject's depth. +So a session starts cold on a given subproject and re-reads several files on +demand. This convention restores fast, accurate orientation without +re-bloating what loads every session — and gives consolidation and archiving a +named, repeatable shape. + +* Document & asset location (what makes clean archiving possible) + +Everything for a subproject lives under =<subproject>/= — its =notes.org=, +=assets/=, working files, event log, briefs. Links *within* a subproject are +relative to the subproject dir, so the whole tree can move without breaking +them. The parent's own top-level =assets/= is for parent-level material only, +never a subproject's. This self-containment is exactly what lets archiving be +a single =mv= plus a pointer reconcile. Keep it strict: a subproject's asset +in the parent's shared =assets/= is a future broken link. + +* The two layers (parent vs subproject) + +- *Always-on layer* — the parent's notes.org. Read at every startup, so every + line is a tax paid whether or not it's relevant today. Holds only what must + be seen session-independently. +- *On-demand layer* — the subproject brief + the subproject's notes.org. + Loaded only when working that subproject. Holds the depth. + +The whole design is: sort content by *when it's needed*, keep the always-on +layer thin, put the depth one open away. + +* Creating a subproject (consolidation criteria) + +Consolidate a standalone project into a parent as a subproject when *all* hold: + +1. *Domain fit* — the project is a facet of a domain the parent already owns + (danneel is home life-admin; finances is home). +2. *Low independent cadence* — it no longer needs its own session rhythm. + You'd rather reach it from the parent than launch it standalone, and its + standalone startup/sync/session overhead now outweighs the isolation. +3. *Shared-scope safety* — it does not need a git/remote/visibility posture + distinct from the parent. A public code project, a team repo, or anything + with its own CI or gitignore-privacy posture *stays standalone* — folding + it would break the parent's privacy model (see + =claude-rules/git-hosting-privacy-model= and the gitignore-vs-track + decision). Personal/documentation units fold; code-with-a-public-remote + does not. +4. *No independent collaborators* depend on it as a standalone deliverable. + +Keep it standalone when any of those fail: active independent development, its +own team or remote, a distinct visibility/security posture, or activity high +enough that competing in the parent's hub would bury it. + +*The consolidation act itself* (mechanics, mirroring home's 2026-06-11 fold): +move the project tree under the parent as =<parent>/<subproject>/=; import its +tasks into the parent's todo with an area =:tag:= and a =:MIGRATED_FROM:= +property; fold its live reminders/decisions into the parent's notes as +pointers (see the content split); build the subproject brief; record the fold +in a consolidation manifest (=docs/consolidation-manifest-<subproject>.org=). +Retire the source to =~/projects/.retired/<name>= once parity is verified. + +* The subproject brief + +** Path (fixed — discovery is mechanical) + +=<subproject>/<subproject>-brief.org= (=danneel/danneel-brief.org=, +=finances/finances-brief.org=). The subproject name is the directory name, so +the brief name follows for free. One predictable name means the read-first +rule never has to hunt; put descriptive richness *inside*, not in the filename. + +** Spine (scales from light to deep) + +Required in every brief, even a fifteen-line one: +- *One-paragraph situation* — what this subproject is, in plain language. +- *Current state / open threads* — the live stuff. The only high-churn + section; everything else is stable. +- *Reading list* — the subproject's deeper files and what each holds. The + brief *points into* notes.org and the rest; it does not copy them. + +Add only where the subproject warrants it (danneel, jr-estate, finances earn +all of these; elibrary or documents may earn none): +- Cast / contacts. +- Timeline (for matters with history). +- Key facts / numbers, with explicit *verify* flags anywhere the underlying + record disagrees with itself. A brief that asserts a wrong number is worse + than no brief — flag, don't guess. + +** Read-first rule (the behavior) + +When a session's work targets a subproject — a file under =<subproject>/=, its +=:tag:=, or Craig naming the topic — open =<subproject>/<subproject>-brief.org= +*first*, before touching anything else in it. If the subproject has no brief +yet, creating it is the first step of the work (lazy backfill), and the same +pass reconciles that subproject's parent-vs-subproject content. + +* Parent-vs-subproject content criterion + +The test for any piece of content: *if next session I'm working a different +subproject, do I still need to see this?* Three bins: + +1. *Yes, and it's a fact or policy* → hub-level → the parent's notes.org. + Machine names, mail policy, calendar access, the hub map, the subproject + index. +2. *Yes, but only because it's urgent* — a deadline or blocking status that + would be missed if it only lived where I might not look this week → the + parent holds a *one-line pointer* (item + date + "see the subproject + brief"); the substance stays in the subproject. +3. *No* → the subproject. Working depth: contacts, contract facts, case + theory, evidence, document index, and any reminder that isn't time-critical. + +One hard rule over both axes: *one fact, one home.* The parent never copies +subproject substance — it points. The parent holds the trigger; the subproject +holds the truth. Every parent↔subproject duplicate is a second surface that +goes stale. + +* Archiving a subproject (archiving criteria + clean process) + +Archive a subproject when it reaches a *terminal* state: +- *Complete* — the work is done and closed (a settlement signed, a probate + closed, a trip finished): no live threads, no open tasks, nothing + time-sensitive in the parent still pointing at it; or +- *Abandoned / superseded* — dropped, or replaced by other work. + +Clean archive process (self-containment is what makes each step a one-liner): +1. Close or settle its tasks in the parent's todo (DONE / CANCELLED), per the + todo completion rules. +2. Move the whole subproject tree to the parent's archive: + =<parent>/archive/<subproject>/= (keep it intact — archiving is reversible, + not deletion). +3. Update inbound =file:= links that pointed into it (grep first, per the + keep-links-current rule). +4. Move or retire its brief with it, and drop its row from the active + subproject index (or mark it =archived= there). +5. Strip its pointers/reminders from the parent's always-on layer, and retire + its =:tag:= from active use. +6. Record the archive in a manifest note + (=docs/archive-manifest-<subproject>.org=): why, when, where it moved, + what links were repointed. Mirrors the consolidation manifest. + +The =:LAST_UPDATED:= on a brief plus "no open tasks under the tag" is the +signal a subproject is a candidate for archiving — surfaced at the review +cadence, never auto-archived. + +* Freshness + +- *Wrap-up hook*: at wrap-up, for each subproject touched this session, refresh + its brief's Current-state section and bump its =:LAST_UPDATED:= date. Stable + sections rarely change, so this is cheap. +- *Staleness nudge*: a brief whose =:LAST_UPDATED:= is old while its subproject + was recently active is worth surfacing at startup. + +* Self-improvement loop + +Every real use emits a signal, and the signal is *captured*, not silently +acted on: +- Brief answers the question → *hit*. +- Brief is missing something, stale, or wrong → *miss*, logged in + subprojects-log.org (a home-side artifact; no rulesets file). +- Each miss drives *two* updates: *local* (fix that brief) and, if the same + kind of miss recurs across subprojects, *structural* (fix this spine/spec, + not just the one file). The structural half is the actual self-improvement — + grow-and-refine from real use rather than rewriting from scratch (the ACE + idea behind the =codify= skill). + +* Metrics + +Split by who can trust them. + +*Process signals (I observe, cheaply):* +- *Orientation cost* — files/tool-calls before my first correct action in a + subproject. Target: ~1 (the brief), occasionally a second deep dive. +- *Brief hit rate* — fraction of subproject questions answered from the brief + alone. Should climb. +- *Freshness gap* — days between =:LAST_UPDATED:= and last subproject activity. + Target ~0. + +*Outcome signals (Craig is ground truth — weight these over the above):* +- *Craig-caught errors* — the real quality bar. Target: trend to zero. If + briefs work, Craig stops catching me. +- *Felt speed* — did dropping into the subproject feel fast and right. + +*Honesty caveat.* Most process signals are self-observed, and self-report is +biased toward "hit." So the external signals (Craig-caught errors, felt speed) +outrank mine, and the log is auditable so Craig can spot-check my self-grades. +The instrumentation must stay lighter than the briefs, or it rots like an +unmaintained brief. Small N, no clean A/B — this is directional evidence, not +proof. + +* Review cadence + +Fold a "subprojects health" check into the existing task-audit cadence: read +the log, look at the hit-rate / correction / freshness trends, flag any +subproject that's gone terminal (archive candidate), and check the +parent-vs-subproject criterion — anywhere the parent holds a copy where it +should hold a pointer. Decide continue / adjust / kill for the pattern itself. + +* Kill criteria (falsifiable) + +After ~6-8 subproject-sessions across ≥3 subprojects, drop or rework the +pattern if: +- the correction rate isn't falling, or Craig is still catching errors; or +- the freshness gap keeps reopening despite the wrap-up hook; or +- maintenance costs more time than the orientation it saves. + +Success is the mirror: orientation down to ~1 read, Craig catching ~nothing, +briefs current, and I'm right fast in any subproject cold. + +* Rollout + +- Define the convention (this file) + the index scaffold in notes.org. Done + 2026-07-15. +- *Lazy backfill*: the next session that touches a subproject with no brief + creates the brief as its first step — cost paid exactly when it's used. +- *Reconcile in the same pass*: building a subproject's brief includes + reconciling its parent-vs-subproject content — push substance down, leave + pointers up, drop dead reminders. +- *Promotion*: this pattern is home-instanced but general. Consolidation into + subprojects will recur whenever any project folds into another, so this is a + candidate to promote into the rulesets rules layer (proposed to rulesets + 2026-07-15). + +* Adoption status + +| Subproject | Brief exists | Reconciled | Notes | +|------------+--------------+------------+---------------------------------------| +| danneel | yes | yes | First worked instance, 2026-07-15. | +|------------+--------------+------------+---------------------------------------| +| clipper | no | no | Lazy backfill on next touch. | +|------------+--------------+------------+---------------------------------------| +| documents | no | no | Lazy backfill on next touch. | +|------------+--------------+------------+---------------------------------------| +| elibrary | no | no | No notes.org yet; lightest subproject.| +|------------+--------------+------------+---------------------------------------| +| finances | no | no | Deep; earns the full spine. | +|------------+--------------+------------+---------------------------------------| +| health | no | no | Has its own notes.org. | +|------------+--------------+------------+---------------------------------------| +| jr-estate | no | no | Deep; earns the full spine. | +|------------+--------------+------------+---------------------------------------| +| kit | no | no | Lazy backfill on next touch. | +|------------+--------------+------------+---------------------------------------| +| philosophy | no | no | Lazy backfill on next touch. | diff --git a/docs/design/2026-07-16-polyglot-bundle-collision.txt b/docs/design/2026-07-16-polyglot-bundle-collision.txt new file mode 100644 index 0000000..b7e1257 --- /dev/null +++ b/docs/design/2026-07-16-polyglot-bundle-collision.txt @@ -0,0 +1,30 @@ +Proposal: language bundles collide on coverage-makefile.txt in a polyglot project, and the second install loses silently. + +Context: scaffolded a new project (clock-panel) on 2026-07-16 with both the python and typescript bundles installed into the same project. + +What happened. The second install printed: + + [skip] coverage-makefile.txt already exists (use FORCE=1 to overwrite) + +Both bundles ship a file at that same path. Python installed first and won, so the project ended up with Python's coverage fragment only. The TypeScript one (c8, Istanbul json-summary) never landed. The line reads like routine idempotence, not a dropped deliverable, so it's easy to skim past. I only caught it by checking which targets were actually in the file. + +The workaround is bad. FORCE=1 gets the TypeScript fragment but overwrites the Python one, so you can't have both without renaming by hand between installs. FORCE=1 also re-seeds CLAUDE.md, which is fine on a fresh project and destructive on a customized one. What I did in clock-panel: install python, mv coverage-makefile.txt coverage-makefile-python.txt, install typescript FORCE=1, mv coverage-makefile.txt coverage-makefile-typescript.txt. Both fragments now coexist under language-suffixed names. + +The deeper problem, which the filename collision only hints at: both fragments define targets with the SAME names. + + coverage: + coverage-summary: + +So even with both files present, a polyglot project can't copy both into one Makefile. It gets duplicate targets. The fragments assume they're the only language in the project. Renaming the files doesn't fix that, it just makes both sets of instructions visible while leaving the conflict for whoever pastes them. + +Options, in the order I'd weigh them: + +Suffix the shipped filename per bundle (coverage-makefile-python.txt, coverage-makefile-typescript.txt) so installs never collide. Cheap, and it makes the multi-bundle case work by default. It leaves the target-name conflict. + +Namespace the targets too (coverage-python / coverage-typescript, with a coverage target that runs both). Fixes the real problem for polyglot projects. Bigger change, and it makes single-language projects type a longer target name unless there's an alias. + +At minimum, make the skip line loud when the skipped file came from a DIFFERENT bundle than the one that wrote it. A skip that means "already yours" and a skip that means "another language's file is here and yours is being dropped" currently look identical. + +Worth deciding whether polyglot projects are a supported case at all. If they are, the bundles need a collision story. If they aren't, install-lang could say so when it detects a second bundle going into a project that already has one, rather than half-installing. + +Also flagged separately in a note from home today: the same install-lang run is where I'd look for other per-bundle files that could collide on a shared filename. I only checked coverage-makefile.txt. diff --git a/docs/design/2026-07-17-dated-log-planning-line-strip-proposal.md b/docs/design/2026-07-17-dated-log-planning-line-strip-proposal.md new file mode 100644 index 0000000..2448600 --- /dev/null +++ b/docs/design/2026-07-17-dated-log-planning-line-strip-proposal.md @@ -0,0 +1,15 @@ +Proposal: two linked gaps that let a closed sub-task keep polluting the org agenda. + +WHAT HAPPENED (home, 2026-07-17) +Six completed sub-tasks under a DONE parent (a finished trip) had each been closed correctly to the dated event-log form per todo-format.org's depth-based completion rule — keyword, priority, and tags dropped, heading rewritten to "YYYY-MM-DD Day @ HH:MM:SS -ZZZZ <past-tense>". But every one kept its old active SCHEDULED: <date> planning line. A dated-log heading has no TODO keyword, and org-agenda renders ANY headline carrying an active SCHEDULED, so all six showed on the daily agenda as weeks-overdue ("Sched.31x") long after the work and the parent were done. They're invisible to a keyword scan (no TODO) and survive archive-done (parent isn't level-2 CANCELLED), so nothing catches them. + +ROOT CAUSE +The completion rewrite strips keyword/priority/tags but nothing strips the planning line, and an interactive org close (org-log-done) only stamps CLOSED — it never removes a pre-existing SCHEDULED/DEADLINE. So the stale timestamp lingers and pins the entry to the agenda permanently. + +TWO FIXES, both rulesets-owned: + +1. todo-format.org — the sub-task completion rule (the "*** and deeper — rewrite to a dated event-log entry" section). Add an explicit step: remove any SCHEDULED:/DEADLINE: planning line when rewriting to the dated form. The completion time already lives in the heading; an active planning date on a historical log entry is always wrong. The VERIFY dated-completion path and todo-cleanup.el --convert-subtasks want the same treatment (the batch converter pulls the timestamp from CLOSED and keeps heading text — it should also drop the planning line). + +2. lint-org.el — add a checker (proposed name: dated-log-heading-active-timestamp). Flag any dated-log heading (a heading matching the YYYY-MM-DD ... @ ... form, no TODO keyword) that still carries an active <...> SCHEDULED or DEADLINE. This is the mechanical backstop for #1, the same way subtask-done-not-dated backstops the depth rule. It would have caught all six here. + +Local fix already applied in home (stripped the six lines by hand); this is about the durable convention + lint so the next instance is prevented and caught. No home-specific detail needs to travel — the gap is general to any project using the dated-log completion form. diff --git a/docs/design/2026-07-17-todo-cleanup-dated-seal-proposal.md b/docs/design/2026-07-17-todo-cleanup-dated-seal-proposal.md new file mode 100644 index 0000000..61d457c --- /dev/null +++ b/docs/design/2026-07-17-todo-cleanup-dated-seal-proposal.md @@ -0,0 +1,38 @@ +Proposal: change todo-cleanup.el's --archive-done aging to a dated-seal model + +Origin: work project, 2026-07-17. Reconciling a stale rotation convention with the tool's actual behavior. Craig ratified the design in-session. + +## The problem + +`--archive-done` today does two things: move level-2 DONE/CANCELLED out of Open Work into Resolved, then age Resolved by moving subtrees older than `tc-archive-retain-days` (default 7) into one rolling file `tc-archive-file` (default archive/task-archive.org). + +Two mismatches with the intended convention: + +1. Retention is 7 days; the convention wants the last month kept inline in Resolved. +2. One rolling file forever; the convention wants periodic dated files so todo.org's archive is browsable by seal date. + +An earlier attempt to name files by calendar quarter (resolved-YYYY-QN.org) hit a seam: with a one-month retention window, a task closed in the last month of a quarter isn't archived until after the quarter boundary, so a quarter-named file can't hold it without mislabeling. + +## The design (ratified) + +Redefine the sealed file by a predicate that's always true of its contents, and date it by the seal run rather than by calendar quarter: + +- A level-2 DONE/CANCELLED subtree is archived when its CLOSED date is older than one month, OR its CLOSED date can't be parsed (archive regardless — a keyword-complete task with no readable close date is cruft, not live work). +- The working (open) file keeps the existing name task-archive.org. A seal renames it to resolved-YYYY-MM-DD.org (the seal date) and starts a fresh working file. +- The dated file means "everything sealed as of that date," not a quarter — so a late-quarter close archived after a boundary is never mislabeled. Cadence (quarterly) becomes independent of correctness; slip is harmless. + +## Concrete changes to todo-cleanup.el + +1. `tc-archive-retain-days` default 7 → 31 (or expose it; the value should reflect "one month"). Reconcile the test at test-todo-cleanup.el:362 that asserts 7-day aging. +2. Aging predicate: archive when `closed < now - retain-days` OR `closed` is unparseable. Today an unparseable-CLOSED subtree is already moved out in aging per the docstring ("those with no parseable CLOSED date are moved out") — keep that, and make it explicit in the contract. +3. Add a seal step (new flag, e.g. `--seal`, or fold into a boundary check): when invoked, rename the current `tc-archive-file` (task-archive.org) → `resolved-<today>.org` beside it and let the next aging recreate a fresh working file. Whether the seal fires automatically at a quarter boundary or stays a manual flag is your call — the manual quarterly rotation task covers it until then. +4. Tests: update the file-aging tests (test-todo-cleanup.el:362, :378, :491, :513, :518) for the new retain default and the seal/rename behavior. The gitignore-inheritance logic (tc--ensure-archive-gitignored) applies unchanged to both the working and sealed names. + +## Companion state already applied in the work project (for reference, not to sync) + +- todo.org grew a `* Work Resolved` section; 30 level-2 DONE/CANCELLED moved there. +- archive/task-archive.org renamed to archive/resolved-2026-07-17.org (grandfathered H1 seal), inbound link updated. +- archive/README.org rewritten to the dated-seal contract above. +- The rotation task rewritten to the new procedure. + +The rulesets change is only the todo-cleanup.el behavior + its tests. The README lives per-project (this one is already updated); if rulesets ships a canonical archive README template, align it to the dated-seal contract too. diff --git a/docs/design/2026-07-18-colloquialisms-and-the-list-proposal.md b/docs/design/2026-07-18-colloquialisms-and-the-list-proposal.md new file mode 100644 index 0000000..7254447 --- /dev/null +++ b/docs/design/2026-07-18-colloquialisms-and-the-list-proposal.md @@ -0,0 +1,22 @@ +#+TITLE: PROPOSAL: a "Colloquialisms and Expansions" convention + the +#+SOURCE: from home +#+DATE: 2026-07-18 18:04:42 -0500 + +PROPOSAL: a "Colloquialisms and Expansions" convention + the "the list" before-close-queue norm. Craig recommends other projects adopt both. Home has implemented it locally as the reference; sending it up so rulesets can decide whether to make it a shared norm. + +THE NORM: "the list" = a before-close FIFO queue +When Craig says "put X on the list" / "add X to the list", X is appended to a Before-Close Queue: a FIFO queue of tasks and actions to finish before the session closes (wrap-up). Work oldest-first. Process the queue at wrap-up before teardown, surfacing anything unfinished rather than dropping it. Session-scoped: resets when the session anchor is archived at wrap. Anything that must outlive the session is a todo.org task instead. + +Home implementation (reference): +- The norm is documented in home .ai/notes.org under a new "* Colloquialisms and Expansions" section (notes.org is read every startup, so the agent honors it without a workflow change). +- The queue itself lives in the session anchor (session-context.org) under a "* Before-Close Queue" heading, seeded empty. + +THE BROADER IDEA: a "Colloquialisms and Expansions" shorthand dictionary +A per-project (or shared) reference mapping Craig shorthand phrases to their expansion, so the agent applies them without asking. Seed entries Craig named: +1. "the list" -> append to the Before-Close Queue (above). +2. "tell <project> <message>" -> drop the message in that project inbox via inbox-send (python3 .ai/scripts/inbox-send.py <project> --text "..."), the sanctioned cross-project handoff, never a direct write. + +WHY RULESETS SHOULD CARE +Both are cross-project by nature. "tell <project>" already rides inbox-send, which every project has. The before-close queue is a session-lifecycle concept that wrap-it-up owns, and wrap-it-up is a rulesets-owned synced workflow. If this becomes a shared norm, the durable wiring is: (a) a colloquialisms reference shipped in the template (or a protocols.org section), and (b) a wrap-it-up step that processes the Before-Close Queue before teardown. Home cannot wire (b) durably from downstream because wrap-it-up.org is synced, so it is stubbed via the notes.org norm for now and needs the canonical change to stick. + +RECOMMENDATION FROM CRAIG: roll this shorthand out to the other projects. diff --git a/docs/design/2026-07-20-signal-pager-runbook.org b/docs/design/2026-07-20-signal-pager-runbook.org new file mode 100644 index 0000000..f31ed18 --- /dev/null +++ b/docs/design/2026-07-20-signal-pager-runbook.org @@ -0,0 +1,198 @@ +#+TITLE: Signal Pager Runbook +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-20 + +The operational reference for the agent pager — how a page reaches Craig's +phone, how his replies come back, how the account stays healthy, and the +signal-cli setup behind it. This is the Signal successor to the retired ntfy +runbook. Canonical home is rulesets because the pager is cross-machine tooling. + +* What the pager is + +One Signal identity, =+15045173983=, registered in *velox's* signal-cli +(account file 465310, velox is the primary device). It is a dedicated pager +number, not Craig's personal Signal. Pages go *from* that identity *to* Craig's +own Signal account, which fires a normal mobile push on his phone. + +As of 2026-07-20 the identity spans two devices: velox (primary) and ratio +(linked device "ratio-pager"). Any machine holding the account sends directly; +a machine that doesn't relays to velox over the tailnet. + +Two constants the tooling depends on: + +- Pager account: =+15045173983= (primary on velox, linked on ratio). +- Recipient: Craig's Signal account UUID =b1b5601e-6126-47f8-afaa-0a59f5188fde=. + His phone *number* reads as unregistered in Signal's directory — always + target the UUID, never the number. + +velox is the laptop that travels with Craig, so the pager account rides with +him; ratio holds it too, so a page still lands when velox is down. + +* Choosing a channel + +Two trigger words, two channels, and both work from any agent runtime (nothing +here is Claude-specific). protocols.org "Reaching Craig" is the short version +pointed at every project; this runbook is the full one for the Signal side. + +- *"page me"* — desktop notification, stays up until dismissed: + + #+begin_src bash + notify info "Title" "Message" --persist + #+end_src + +- *"text me"* — the phone, over Signal: + + #+begin_src bash + agent-text "Message for Craig's phone" + #+end_src + +- *"text and page me"* — both. The default when a run can't tell whether he's + away: the desktop one is free and the phone one reaches him if he is. + +* Sending a text + +=agent-text= (shipped at =claude-templates/bin/agent-text=, installed to +=~/.local/bin= by =make -C ~/code/rulesets install=) is the interface. It hides +the machine topology by checking whether the account is registered in the +local signal-cli: + +- If the account is local (velox's primary or a linked device like ratio), it + sends directly — no velox dependency. +- Otherwise it ssh-relays the send to velox over the tailnet. +- On failure (velox down or unreachable from a non-linked machine) it prints the + desktop fallback line and exits non-zero, so a caller can tell the page did not + land. + +The raw command it runs, for reference or a manual send from velox: + +#+begin_src bash +signal-cli -a +15045173983 send -m "your message" b1b5601e-6126-47f8-afaa-0a59f5188fde +#+end_src + +From another machine, the same send relayed over the tailnet: + +#+begin_src bash +ssh velox.tailf3bb8c.ts.net \ + "signal-cli -a +15045173983 send -m 'your message' b1b5601e-6126-47f8-afaa-0a59f5188fde" +#+end_src + +Prefer =agent-text= over the raw command — it hardens the message for the remote +shell and handles the fallback. Reach for the raw form only when debugging. + +* Reading replies + +Craig replies to a page straight from Signal on his phone. The reply is a normal +data message *to* the pager account, so it is waiting in the pager's inbound +queue until something receives it. + +Drain the queue and read what is there: + +#+begin_src bash +# On velox: +signal-cli -a +15045173983 receive --timeout 10 +# From another machine: +ssh velox.tailf3bb8c.ts.net "signal-cli -a +15045173983 receive --timeout 10" +#+end_src + +=receive= prints every queued envelope and exits 0 once the queue drains or the +timeout elapses. A text reply from Craig arrives as an envelope from his UUID +carrying a =Body:= line — that line is the reply text. Most envelopes are +delivery/read receipts and typing indicators (no =Body:=); the reply you want is +the data message with body text. For a script that waits on a reply, add +=--send-read-receipts= so his phone shows the page was read, and parse stdout for +the =Body:= line on an envelope from =b1b5601e-…=. + +Note: =receive= is destructive — it consumes the queue. Whatever drains the +queue (an on-demand read, or the warm-keeping timer below) is what sees the +reply, and it is seen once. An agent that pages and then waits for an answer +should do its own =receive= rather than race the timer. + +* Keeping the account warm (receive timer) + +Signal expects a registered account to receive regularly. Left alone, the pager +account drifts stale — signal-cli warns "Messages have been last received N days +ago" (observed at 47 days on 2026-07-20 before a manual drain reset it). A stale +account is a reliability risk on the one channel that reaches Craig when he is +away. + +The fix mirrors roam-sync: a systemd user timer that drains the queue on a +cadence, keeping the account warm and, as a bonus, picking up async replies. With +the account linked on both machines, each device wants its own regular receive, +so the timer runs on *both* velox and ratio (the shared =common= dotfiles +package, same home as roam-sync). + +- Script: =scripts/signal-receive.sh= (rulesets, so both machines get it on + =git pull=). It no-ops cleanly on a machine that lacks the account. +- Units: =scripts/signal-receive.service= + =.timer= (reference copies under + =scripts/systemd/=; the stowed copies live in =common/.config/systemd/user/= + of the dotfiles repo, so both machines get them). +- Cadence: every 15 minutes (=OnUnitActiveSec=15min=), matching roam-sync. + +Enable on each machine (one-time, per daily-drivers.md's one-time-setup class): + +#+begin_src bash +# After the dotfiles + rulesets pull, on each daily driver: +systemctl --user daemon-reload +systemctl --user enable --now signal-receive.timer +systemctl --user status signal-receive.service # confirm a clean receive +#+end_src + +* signal-cli setup notes + +- *Version:* signal-cli 0.14.5 on velox (2026-07-20). +- *Accounts:* velox's signal-cli holds the pager identity =+15045173983= as the + registered primary (account file 465310). ratio's signal-cli holds two + accounts: Craig's personal number =+15103169357= (its own primary, + note-to-self only — no phone push) and the pager identity as a *linked device* + (Device 2, "ratio-pager", linked 2026-07-20). Both accounts coexist; target + the pager with =-a +15045173983=. A future daily driver joins the same way. +- *signal-mcp:* on velox, Claude sessions may also expose a =signal-mcp= tool + (=send_message_to_user=, same pager identity) configured in velox's global + =~/.claude.json=. It works there but is invisible from any other machine and + from non-Claude runtimes, so =agent-text= is the portable habit. The old + =page-signal= shell script was removed 2026-06-12 — do not resurrect it. +- *Linking a device:* to add a second signal-cli as a linked device of the pager + account (see the open decision below), provision it from the new machine and + approve the link from the account holder: + + #+begin_src bash + # On the new machine — prints a tsdevice:/ URI (render as QR to approve): + signal-cli link -n "ratio-pager" + # Approve from velox (the primary device): + signal-cli -a +15045173983 addDevice --uri "tsdevice:/?uuid=…" + #+end_src + +* Decision — linked device (2026-07-20) + +The topology question — ssh-relay only vs. registering daily drivers as linked +devices — was decided in favor of linked devices. ssh-relay only was simpler +(one identity, one receive point) but had a single point of failure: a page +failed when velox was down or off the tailnet. + +Registering ratio as a linked device removes that: ratio sends directly, so a +page lands even when velox is down. The costs, both paid: linked-device +provisioning per machine (the =link= / =addDevice= handshake above), and each +device wanting its own regular =receive= — so the warm-keeping timer moved from a +velox-only home to the shared =common= package, running on both. + +Adding another daily driver later is the same handshake plus a dotfiles stow; +the timer and =agent-text= already generalize to "any machine holding the +account." + +* History + +- 2026-07-04 — home retired ntfy (self-hosted on ratio) and tore it down, + switching agent paging to Signal. Handoff to rulesets to document and own. +- 2026-07-13 — reconciled to one pager identity on velox; =agent-page= shipped + (direct on velox, ssh-relay elsewhere, desktop fallback); protocols.org "Paging + Craig" rewritten around the two channels. +- 2026-07-20 — this runbook; receive-timer script + units added; a manual drain + cleared the 47-day staleness live; ratio linked as a device of the pager + account and its direct send verified; the tool (still named =agent-page= that + morning) generalized to send directly from any machine holding the account; + receive timer moved to the shared =common= package and enabled on both machines. +- 2026-07-20 (later) — notification vocabulary split: "page me" is the desktop + channel, "text me" is Signal, "text and page me" is both. The tool was renamed + =agent-page= → =agent-text= to match, with a deprecated =agent-page= shim + delegating to it. protocols.org section renamed "Paging Craig" → "Reaching + Craig". diff --git a/docs/design/task-review.org b/docs/design/task-review.org index 6c6dac7..c9ae023 100644 --- a/docs/design/task-review.org +++ b/docs/design/task-review.org @@ -1,5 +1,5 @@ #+TITLE: Design: Daily Task-Review Habit -#+AUTHOR: Craig Jennings & Claude +#+AUTHOR: Craig Jennings #+DATE: 2026-05-16 #+OPTIONS: toc:nil num:nil diff --git a/docs/design/wrapup-routing-spec.org b/docs/design/wrapup-routing-spec.org deleted file mode 100644 index 0091806..0000000 --- a/docs/design/wrapup-routing-spec.org +++ /dev/null @@ -1,181 +0,0 @@ -#+TITLE: Wrap-Up Inbox/Transcript Routing — Spec -#+AUTHOR: Craig Jennings -#+DATE: 2026-06-13 -#+TODO: TODO | DONE SUPERSEDED CANCELLED - -* Metadata -| Status | ready for review | -|----------+-----------------------------------------------------| -| Owner | Craig Jennings | -|----------+-----------------------------------------------------| -| Reviewer | Codex (spec-review) | -|----------+-----------------------------------------------------| -| Related | [[file:../../todo.org][todo.org: wrap-up routing task]] · [[file:2026-06-13-wrapup-inbox-transcript-routing-proposal.org][archsetup proposal]] | -|----------+-----------------------------------------------------| - -* Summary - -At wrap-up, an inbox handoff that belongs to another project has nowhere to go but the current project's =todo.org= or a deferral. This adds an optional routing step to =wrap-it-up.org=: surface the items that belong elsewhere, recommend a destination project for each, and move the whole batch there on one confirmation. A parallel step files meeting-transcript recordings into the right project's =assets/=. - -* Problem / Context - -=process-inbox.org= dispositions each handoff as act / fold / file / reject, and "file as TODO" lands the task in the *current* project's =todo.org=. When the real home is a different project, the choices today are: file it locally and let it rot in the wrong tracker, hand-edit two projects' =todo.org= files, or defer it and carry the debt to next session. - -The wrap-up's existing Step 3 "Inbox sanity check" only counts unprocessed items and blocks the wrap until they clear. It answers "is the inbox clean?" — it doesn't route anything. - -Meeting transcripts have the same homelessness: a recording dropped during a session belongs in some project's =assets/=, but nothing moves it there at wrap. - -The friction is small per-item but recurring, and the manual cross-project edit is error-prone (two files, two repos, easy to leave one half-done). - -* Goals and Non-Goals - -** Goals -- At wrap-up, surface inbox items (and transcripts) whose home is a different project, with a recommended destination each. -- Move the whole batch on one confirmation ("go with recommendations") or leave it entirely ("skip"). No per-item triage. -- Move a task/event item as a proper org task into the destination's "Open Work" section per =todo-format.md=; move a transcript as a flat-filed artifact per =working-files.md=. -- Keep the move atomic and visible (it shows in the destination's next git diff, with a provenance note). -- Discover any project with a =todo.org= as a candidate destination, not only =.ai/= projects. - -** Non-Goals -- Not a wrap gate. A skip is a clean, complete wrap. -- Not per-item triage. The interaction is batch-level: go or skip. -- Not a replacement for =process-inbox.org='s value gate. Routing assumes the item is already an accepted keeper. -- Not a confidence-free auto-mover. A low-confidence destination recommendation says so, and the batch "go" stays trustworthy because the surfaced list is reviewable before the keystroke. - -** Scope tiers -- v1: task/event routing to a destination project's =todo.org=. The interaction, the recommendation engine, the atomic move helper, the widened project discovery. -- Out of scope: per-item destination editing, an interactive correction loop, moving items that aren't accepted keepers. -- vNext: meeting-transcript filing (gated on the unresolved source-location decision and the file-vs-file+extract question — see Decisions). - -* Design - -** User-facing (the wrap interaction) - -The router is a new sub-step of =wrap-it-up.org='s Step 3, running after the existing inbox sanity check. Its input is filed keepers, not raw inbox files (decision: Reading B): tasks =process-inbox= accepted and filed into the local =todo.org= this session whose inferred home is a different project. When the router finds such a keeper, it surfaces it in a list, one line each: the task, the recommended destination project, and a confidence marker when the inference is weak. Then two options, batch-level: - -1. Go with the recommendations — apply every recommended move. -2. Skip — leave the whole batch in place. A skip is a clean wrap. - -That is the entire interaction. No per-item walk. The surfaced list is the review surface; the single keystroke is trustworthy because the list was reviewable and low-confidence recommendations flagged themselves. - -A move of a task/event relocates it into the destination project's "Open Work" section as a proper org task (terse heading, body for detail, tags on the heading line, per =todo-format.md=), and removes it from the source. A skipped or unroutable item stays where it is; the existing sanity check still governs whether the wrap is clean. - -** Implementer (the mechanics) - -*Candidate set (what the router considers).* Reading B means the router does not scan the whole local backlog — it would otherwise suggest moving legitimate local tasks every wrap. The candidate set is keepers =process-inbox= filed this session whose inferred home differs from the current project. How those are marked is an implementation detail for Phase 3/4: either =process-inbox= tags a cross-project-candidate keeper at file time, or the router infers from a =CREATED= stamp dated this session plus content. The reviewer should pin which; the design constraint is "session-filed inbox keepers only, never the standing backlog." - -*Destination discovery.* Widen the project-discovery filter from "directory with a =.ai/protocols.org= marker" (what =inbox-send.py= and the =ai= launcher use) to "directory with a =todo.org= containing a level-1 'Open Work' heading." A plain code repo Craig keeps a =todo.org= in is a valid destination; an =.ai/= directory is not required. - -*Destination anchor.* Reuse =todo-cleanup.el='s existing matcher: =tc--find-section= locates the unique level-1 heading containing "Open Work" (case-insensitive) and returns =nil= / ='multiple= when absent or ambiguous. A destination whose =todo.org= lacks a clean Open Work heading is surfaced and skipped, never guessed at. - -*The move helper.* A small tool inserts a task subtree under a named project's "Open Work" heading and removes the source atomically — extend =todo-cleanup.el= (it already owns the section matcher and the subtree-move logic for =--archive-done=) or add a sibling =.ai/scripts= tool. Hand-editing across two repos is the error-prone path this replaces. - -*Recommendation engine.* Infer the destination from the item's content — project names, file paths, topic words — matched against the discovered project list. Conservative by design: a weak match is labeled low-confidence so "go" stays a safe single keystroke. The engine is the interesting, uncertain part; it earns the spec. - -*Cross-project write discipline.* Moving an item into project X's =todo.org= writes into X's scope (=cross-project.md=). The batch "go" authorizes it, but the move stays visible (X's next git diff) and leaves a one-line provenance note on the moved task naming the source project. - -* Alternatives Considered - -** Per-item triage instead of batch go/skip -- Good, because it gives precise control over each destination. -- Bad, because it taxes the common case (a batch that's all-correct, or all-stay) with a walk. Craig explicitly asked for two options, not a triage loop. -- Neutral, because per-item correction could return as a vNext refinement if batch-only proves too blunt. - -** Fold the router into the existing Inbox sanity check step -- Good, because one inbox step is simpler than two. -- Bad, because the sanity check *gates* the wrap (blocks until clean) and the router is *optional* (skip is clean). Merging a blocking check with an optional action muddies both. -- Neutral, because the two share discovery code while staying separate steps. (Resolved: D1 keeps them separate, with the router acting on filed keepers rather than inbox files.) - -** Reuse process-inbox's "file as TODO" with a destination argument -- Good, because it avoids a second mechanism. -- Bad, because =process-inbox= runs per-item mid-session against the local project; the router runs at wrap, batch-level, cross-project. Different cadence, different scope. -- Neutral, because both ultimately call the same atomic move helper — the helper is the shared primitive, the two callers stay distinct. - -* Decisions [6/6] - -** DONE Reuse the Open Work matcher for destination anchoring -- Context: the move needs a reliable insertion point in the destination =todo.org=; guessing risks corrupting another project's file. -- Decision: We will reuse =todo-cleanup.el='s =tc--find-section "open work"= matcher, which already handles the unique / missing / ambiguous cases, and skip+surface any destination without a clean Open Work heading. -- Consequences: easier — no new parser, consistent with =--archive-done=. Harder — destinations must carry the "Open Work" heading convention, so a project with a differently-named section is silently unroutable until it conforms. - -** DONE Move atomically through a helper, never hand-edit two repos -- Context: a move touches two files in two repos; a half-done move loses or duplicates a task. -- Decision: We will route every move through one helper (extend =todo-cleanup.el= or a sibling =.ai/scripts= tool) that inserts under the destination's Open Work heading and removes the source as one operation. -- Consequences: easier — no partial-move corruption, one place to test. Harder — a new helper to build and cover with tests before the router can ship. - -** DONE Cross-project writes stay visible and carry provenance -- Context: writing into another project's =todo.org= crosses the =cross-project.md= scope boundary. -- Decision: We will treat the batch "go" as the authorization, leave the move visible in the destination's git diff, and stamp a one-line provenance note (source project + date) on each moved task. -- Consequences: easier — the boundary rule is honored without a per-move prompt. Harder — the destination's next session sees an externally-authored task it didn't file, so the provenance note is load-bearing, not decorative. - -** DONE Separate router step, operating on filed keepers (Reading B) -- Context: the sanity check gates the wrap on inbox/ contents; the router is optional. The deeper question was the router's input — raw inbox files (Reading A, which overlaps the sanity check) or already-filed keepers that belong elsewhere (Reading B, a todo-routing concern). -- Decision: We will keep the router a separate optional sub-step after the sanity check, and its input is Reading B: accepted keepers process-inbox filed into the local =todo.org= whose inferred home is another project. The sanity check stays a pure inbox gate; the router is a todo-routing action that shares only the destination-discovery code. -- Consequences: easier — each step has one job, the gate can't be muddied by an optional action, and the router never competes with the inbox gate over the same files. Harder — the candidate set (which local tasks the router considers) needs a marking mechanism (see the Implementer "candidate set" note); Reading A's "dispose raw inbox files at wrap" convenience is given up. - -** DONE Transcript routing deferred to vNext -- Context: transcripts file as artifacts, not tasks, and a meeting usually produces both a recording to keep and action items to track. Two unknowns block it: where recordings accumulate (a recordings inbox, a downloads dir, wherever the meeting tooling drops them), and whether filing should also extract action items into the destination's =todo.org=. -- Decision: We will defer transcript routing to vNext. Both the source-location dependency and the file-only-vs-extract-action-items question are deferred with it, to be settled when the vNext work is specced. v1 ships task routing only. -- Consequences: easier — v1 isn't blocked on the unresolved source location. Harder — until vNext, a meeting recording still has no automatic home; only its action items (if filed as tasks) route through v1. - -** DONE Keep defer-and-stage and the router as distinct policies -- Context: the 2026-06-12 Skeptical Review added a defer-and-stage path in =process-inbox.org= that files a =[#B]= VERIFY for shared-asset proposals parked for review. That also turns an inbox item into a =todo.org= task — overlapping surface with this router. -- Decision: We will keep them distinct. Defer-and-stage parks a proposal-under-review locally as a VERIFY; the router moves an accepted keeper to its home project as a TODO. They differ on review status (proposal vs accepted) and destination (local vs cross-project), and share only the atomic move helper, not the policy. Reading B makes the split clean: the router acts on accepted keepers, never on proposals under review. -- Consequences: easier — two clear, non-competing policies on one shared primitive. Harder — the workflow prose must name the boundary so a future reader doesn't collapse them and reintroduce the ambiguity. - -* Implementation phases - -** Phase 1 — Widened project discovery -A discovery function returning every project with a =todo.org= that has a clean Open Work heading, reusing =tc--find-section=. Unit-tested against fixtures: =.ai/= project, plain-code-repo-with-todo, todo-without-Open-Work (excluded), ambiguous-Open-Work (excluded). Leaves the tree working — nothing calls it yet. - -** Phase 2 — Atomic cross-project move helper -Extend =todo-cleanup.el= (or sibling tool) with a "move this subtree into project X's Open Work" operation that inserts at the destination and removes the source as one step, stamping the provenance line. ERT coverage: successful move, missing-destination-heading refusal, source-removal-on-success, no-partial-move-on-failure. - -** Phase 3 — Recommendation engine + candidate-set marking -Infer destination from item content against the discovered list, with a confidence label. Pure function over (item, project-list) → (destination, confidence). Unit-tested: strong match (project named in item), weak match (topic-only → low-confidence), no match (stays put). Also settle the candidate-set marking (tag at file time vs CREATED-this-session inference) so the router considers only session-filed inbox keepers, never the standing backlog. - -** Phase 4 — Wrap-up step wiring -Add the router sub-step to =wrap-it-up.org= Step 3: surface the batch, the two options, apply-on-go via the Phase 2 helper. Per the D1/D5 decisions once settled. Sync the =.ai/= mirror. - -** Phase 5 — Transcript routing (vNext, gated on the transcript decision) -Only after the transcript-scope decision resolves. File a recording into the destination =assets/= per =working-files.md=, batch go/skip mirroring the task router. - -* Acceptance criteria -- [ ] At wrap, an inbox item naming another project is surfaced with that project as the recommended destination. -- [ ] "Go" moves every recommended item into its destination's Open Work section as a valid org task with a provenance line, and removes it from the source. -- [ ] "Skip" leaves every item in place and the wrap completes cleanly. -- [ ] A destination =todo.org= without a clean Open Work heading is surfaced and skipped, never corrupted. -- [ ] A low-confidence recommendation is visibly labeled in the surfaced list. -- [ ] A plain code repo with a =todo.org= (no =.ai/=) is a valid destination. -- [ ] A failed move leaves both source and destination unchanged (no partial move). - -* Readiness dimensions -- Data model & ownership: items are org subtrees; the destination owns the moved task after the move (provenance note records origin). N/A for remote/cached state — all local files. -- Errors, empty states & failure: missing/ambiguous Open Work heading → skip+surface; failed move → atomic no-op; empty routable set → router stays silent (no prompt). -- Security & privacy: N/A — local org files, no credentials or external services. -- Observability: the move shows in the destination's git diff plus the provenance line; the surfaced batch list is the pre-move view. -- Performance & scale: bounded by inbox size (single digits) and project count (tens); no hot path. -- Reuse & lost opportunities: reuses =tc--find-section= and todo-cleanup's subtree-move; widens existing discovery rather than adding a parallel one. -- Architecture fit & weak points: the recommendation engine is the weak point (a wrong-confident destination is the worst failure) — mitigated by the confidence label and reviewable batch list. -- Config surface: possibly a discovery-root list (defaults to =~/projects/=, =~/code/=, matching =inbox-send.py=). Name it if it needs to be user-visible. -- Documentation plan: =wrap-it-up.org= step prose; a note in =cross-project.md= that the router is a sanctioned cross-project write path. -- Dev tooling: ERT for the elisp helper + discovery; the existing =make test= picks up new test files by glob. -- Rollout, compatibility & rollback: additive workflow step; rollback is removing the sub-step. No persisted-data migration. -- External APIs & deps: none. - -* Risks, Rabbit Holes, and Drawbacks -- *Recommendation accuracy is the rabbit hole.* A confidently-wrong destination silently files a task in the wrong project. Dodge: keep the engine conservative, label low confidence, and keep the batch list reviewable before the keystroke. Don't chase a clever inference model in v1. -- *Two inbox-touching steps* (sanity check + router) risk reading as redundant. Dodge: the D1 decision states the gate-vs-optional split in the workflow prose. -- *Scope creep into transcripts* before the source-location question is answered would stall v1. Dodge: transcripts are explicitly vNext behind decision D4. - -* Review and iteration history - -** 2026-06-13 Sat @ 01:23:13 -0500 — Claude Code (rulesets) — author -- What: initial draft. Problem, goals/scope tiers, two-altitude design, alternatives, six decisions (three DONE from grounding, three TODO for Craig), five implementation phases, acceptance criteria, readiness dimensions, risks. -- Why: the archsetup 2026-06-13 handoff cleared the spec bar in inbox triage and was filed spec-bound rather than applied. This draft turns the proposal into a reviewable design with the open questions isolated as decision tasks. -- Artifacts: proposal source at =docs/design/2026-06-13-wrapup-inbox-transcript-routing-proposal.org=; grounded against =wrap-it-up.org= Step 3, =todo-cleanup.el= =tc--find-section=, and =inbox-send.py= discovery. - -** 2026-06-13 Sat @ 01:36:28 -0500 — Craig Jennings + Claude Code (rulesets) — author -- What: resolved all three open decisions. The router's input is Reading B (filed keepers that belong elsewhere, not raw inbox files), so D1 keeps it a separate sub-step from the inbox gate and D5 keeps it distinct from the defer-and-stage router; D4 defers transcript routing to vNext. Reworked the design (input definition, a candidate-set note bounding the router to session-filed keepers) and Phase 3 to match. Cookie now [6/6]; Status moved to ready-for-review. -- Why: Craig chose Reading B after the A-vs-B input ambiguity surfaced as the root under D1 and D5. Reading B keeps the inbox gate, the router, and defer-and-stage each simple instead of entangling three mechanisms. -- Artifacts: this spec; the candidate-set marking mechanism is the one detail flagged for spec-review to pin. diff --git a/docs/specs/2026-06-16-autonomous-batch-execution-spec.org b/docs/specs/2026-06-16-autonomous-batch-execution-spec.org new file mode 100644 index 0000000..a42adc3 --- /dev/null +++ b/docs/specs/2026-06-16-autonomous-batch-execution-spec.org @@ -0,0 +1,393 @@ +#+TITLE: Autonomous-Batch Task Execution — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-16 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Autonomous-Batch Task Execution — Spec +:PROPERTIES: +:ID: 90f623cd-fdbe-4f5c-b63d-b2f84d9151cf +:END: +- 2026-07-02 Thu @ 05:26:07 -0400 — DOING → IMPLEMENTED: all six phases built (work-the-backlog.org, both callers, the waiver gate, checklist/Q&A/page mechanics, metrics record, KB synthesis) and the live trial validated — run c726f526, 3/3 tasks as reviewed commits with the pre-flight Q&A, page, and metrics all exercised. Craig confirmed and granted :LOOP_MAY_COMMIT:. +- 2026-07-02 Thu @ 00:44:59 -0400 — READY → DOING: spec-response decomposition ran — the speedrun build parent in todo.org carries the :SPEC_ID: binding, one task per phase (1-6) plus the live-trial validation and the flip-to-IMPLEMENTED task. Phase 0 had already landed 2026-07-01. +- 2026-07-02 Thu @ 00:17:01 -0400 — retrofitted by spec-sort; status set to READY (evidence-based, human-confirmed) + +* Metadata +| Status | implemented | +|----------+--------------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+--------------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+--------------------------------------------------------------------| +| Date | 2026-06-16 | +|----------+--------------------------------------------------------------------| +| Related | [[file:../design/2026-06-16-inbox-zero-phase-e-proposal.org][Phase E proposal]]; [[file:../design/2026-06-15-fix-speedrun-workflow-proposal.org][speedrun proposal]] | +|----------+--------------------------------------------------------------------| + +* Summary + +Two proposals arrived within a day of each other describing the same capability: have Claude work a batch of small, well-marked tasks autonomously, with a full quality bar per task and no per-step approval gate. The inbox-zero "Phase E" proposal drives it from a tag/priority query on a recurring loop; the "speedrun" proposal drives it from an explicit ordered list a human dictates in-session. This spec reconciles both into one feature: a single dedicated workflow, =work-the-backlog.org=, that holds the task-execution logic, with two thin callers feeding it. It also designs the instrumentation that measures whether the autonomy is actually paying off. + +* Problem / Context + +Craig has a standing backlog of small, solo-doable fixes across several projects, already marked with a tag convention (=:next:=, =:quick:+:solo:=). Doing them by hand one at a time is the bottleneck — the context-switch and the per-commit approval ceremony dominate the actual work. He wants Claude to burn these down unattended: on a recurring loop for the routed inbox case, and on demand when he batches a named list and says "speedrun, no approvals until done." The speedrun is the away-from-desk / working-on-something-else mode, so it must be able to take on larger tasks too — not only sub-30-minute ones — or it forces him to stay at the desk for anything non-trivial. + +Two separate proposals tried to answer this: + +- *Phase E* (in =inbox-zero.org=, edited in =.emacs.d= as a stopgap) bolted autonomous execution onto the inbox-zero workflow's on-demand and loop callers. The sender flagged the seam as the open question: coupling capture-routing with autonomous-implementation pollutes inbox-zero's three existing callers (startup, wrap-up, on-demand), two of which must never execute anything. +- *speedrun* (a =.emacs.d= theme-studio session that worked well) is the same execution loop driven by an explicit ordered task set, with end-of-set paging and always-push. + +They overlap almost entirely. The execution loop — eligibility gate, act-vs-file decision, per-task quality bar, bounded run — is identical. Only the *input* differs (tag query vs explicit list) and the *session mode* differs (loop default vs no-approvals + always-push + page). Building them as two features would duplicate the execution logic and let the two copies drift. The forces: keep inbox-zero's callers clean, share one execution loop, and make the autonomy safe enough to run unattended on a 30-minute timer without Craig watching. + +A second, explicit ask from Craig: instrument this so its effectiveness is measurable. "Gather data on this and create some org-roam articles we can look at later." Autonomous execution that silently makes bad commits is worse than no autonomy; the only way to know which it is, is to measure tasks completed vs deferred vs reverted, and human corrections in the following session, over time. + +* Goals and Non-Goals + +** Goals +- One workflow, =work-the-backlog.org=, owns the task-execution loop. Both input shapes (tag query, explicit list) and both session modes feed it. +- inbox-zero's three existing callers stay clean: the loop caller chains into =work-the-backlog= *after* routing; startup and wrap-up never touch it. +- The *no-approvals speedrun* is a thin named preset, not a second implementation: autonomous-commit + always-push + end-of-set page, fed an explicit ordered list, with all approvals front-loaded into a single pre-flight step (below) so the run itself is uninterrupted. +- Eligibility is decided by *crisp, checkable criteria*, not adjectives: a mechanical tag/status gate (=:solo:= + status =TODO=), then a per-task defer checklist whose keystone is "can I write the failing test from the task text without inventing a requirement?" Task *size* is explicitly not a gate — a large task is decomposed into per-logical-commit chunks, not deferred. +- The autonomy tags (=:solo:=, =:quick:=) carry hard definitions in =todo-format.md= and are applied + enforced as a mandatory step in the task-review and task-audit workflows, so the run-time gate trusts the author's tag instead of re-deriving it. +- Commit autonomy defaults to file-only (surface a diff, no auto-commit). A project opts into autonomous commit+push explicitly via its per-project waiver. +- Hard guardrails: refuse any task carrying data-loss / irreversible / external-state risk without a checkpoint; gather any one-or-two quick decisions a task needs *up front* (speedrun) rather than guessing; file a =VERIFY= for anything underspecified or needing design deliberation; a per-run cap / kill switch beyond "one task per run." +- A lightweight per-run metrics log plus a periodic synthesis step that writes org-roam KB articles summarizing the trend. + +** Non-Goals +- *Not* a replacement for =/start-work=. Tasks needing deliberation or design stay with =/start-work= and its approval gates. This feature only touches the marked, solo set — regardless of size. +- *Not* a new tag convention. It reads the project's own priority/tag scheme header; it never invents or hardcodes tags across projects. +- *Not* an inbox-routing change. =inbox-zero.org= keeps its A-D phases. The Phase E text added in =.emacs.d= as a stopgap is *removed* and its logic moves here. +- *Not* a multi-project orchestrator. One run works one project's backlog. Cross-project handoff stays with =inbox-send= and the paging reply. +- *Not* a credential-handling or external-API feature. Tasks that touch secrets or external mutations are out of the eligible set by the guardrail. + +** Scope tiers +- *v1:* =work-the-backlog.org=; crisp =:solo:= / =:quick:= definitions in =todo-format.md= plus their mandatory application in task-review and task-audit; the eligibility gate (=:solo:= + status =TODO=, read against the project's scheme header); the act-vs-file *defer checklist* (test-writability keystone, enumerated data-loss list, already-satisfied, design-deliberation); the no-approvals speedrun's pre-flight decision-gathering step; file-only commit default with per-project opt-in; the loop caller wiring and inbox-zero Phase E removal; the speedrun preset with end-of-set =notify --persist= page; the per-run metrics log (structured JSONL). +- *Out of scope:* a token-budget kill switch (cap is a task count in v1); cross-project batch runs; a dashboard or live UI over the metrics. +- *vNext (log to todo.org):* the periodic org-roam synthesis step if it doesn't make v1; a token/cost budget alongside the task-count cap (more pressing now that task size is uncapped — a single large task can run long in the unattended loop); auto-detection of "human corrected my autonomous commit" from the next session's diff. + +* Design + +** Overview + +The architecture is one execution workflow with two callers and one preset, plus an instrumentation sidecar. + +#+begin_example + inbox-zero loop caller ──(after Phase D routing)──┐ + ├──▶ work-the-backlog.org ──▶ metrics log (JSONL) + no-approvals speedrun ──(explicit ordered list)──┘ │ + = pre-flight Q&A + autonomous-commit + push + page ▼ + periodic synthesis ──▶ org-roam KB articles +#+end_example + +=work-the-backlog.org= is the only place the execution loop lives. It takes a *task set* (however assembled) and a *session mode* (which gates commit autonomy and paging), and works the set under a fixed safety contract. The two callers differ only in how they build the task set and which session mode they pass. + +This is the seam the Phase E sender asked for: separating capture-routing (inbox-zero) from autonomous-implementation (work-the-backlog) keeps inbox-zero's startup and wrap-up callers — which must never execute anything — untouched. The loop caller is the only one of inbox-zero's callers that chains forward into execution, and it does so as an explicit second step after routing completes, not as a phase buried inside inbox-zero. + +** The execution loop (two-altitude: caller's view) + +A caller hands =work-the-backlog= three things: + +1. *A task set* — either an explicit ordered list of task headings (speedrun), or the result of a tag/priority query against =todo.org= (the loop). The workflow does not care which; it receives an ordered list of candidate tasks. +2. *A session mode* — =file-only= (default) or =autonomous-commit= (requires the project's per-project waiver), and a paging flag. +3. *A run cap* — the maximum number of tasks to complete this run. + +It returns: per-task outcome (implemented+committed / implemented+diff-surfaced / deferred-VERIFY / dropped-by-craig / skipped-ineligible), and a metrics record per task. + +** The execution loop (implementer's view) + +For the task set, in order, until the run cap is hit: + +1. *Eligibility gate* (below). Ineligible → record =skipped-ineligible=, next task. +2. *Scope read* of the relevant code. Cheap; just enough to run the defer checklist. +3. *Defer checklist* (below). Any hit → record the deferral reason (or, under the speedrun preset, route the quick-question gap to the pre-flight Q&A), next task. +4. *Implement* under the project's commit discipline: TDD red→green→refactor, then =/review-code --staged=, fix all Critical/Important, then close the task per =todo-format.md=. Decompose into as many logical commits as the change needs — size is not capped. +5. *Commit autonomy branch:* + - =file-only= → surface the diff, do *not* commit. Record =implemented-diff-surfaced=. + - =autonomous-commit= → =/voice personal= on the message, commit individually, push per the project's flow. Record =implemented-committed=. +6. *Record metrics* for the task (the JSONL append, below). +7. Decrement the cap. At zero, stop. + +After the set: if the paging flag is set, fire the end-of-set page (below). Surface the run summary. + +** Eligibility gate (mechanical — no judgment) + +A task is autonomous-safe when *both* hold. This layer is a lookup, not a judgment; all the judgment lives in the defer checklist below. + +1. *Status is =TODO=* — never =VERIFY=, =DOING=, =DONE=, or =CANCELLED=. =VERIFY= is the "awaiting Craig's manual confirmation" marker; auto-implementing one defeats the manual check it represents. The do-not-implement set is safe-by-omission: anything not plainly =TODO= (plus any project-declared "hold" marker) is out. +2. *Tagged =:solo:=* — the autonomy tag, resolved against the project's priority/tag scheme header (not hardcoded). =:solo:= carries a hard definition (see Tag definitions, below): the task is completable without Craig's involvement beyond at most one or two quick decisions answerable up front, with no design deliberation. A project whose scheme declares a different autonomous-safe tag set overrides the default. Priority / =:next:= drive *ordering* within the eligible set, not eligibility. + +Task *size* is deliberately absent from this gate. The old "≤ ~30 minutes / one logical commit" criterion is removed: a large but well-specified, decision-free task is in scope and is decomposed into per-logical-commit chunks during implementation. Size never sends a task to =/start-work=; only *deliberation* or *risk* does (the checklist below). This is what makes the speedrun usable as an away-from-desk mode rather than a sub-30-minute-only mode. + +*** Tag definitions (land in =todo-format.md=, enforced in task-review + task-audit) + +- *=:solo:= — autonomy.* The task can be completed without Craig's involvement, except for at most one or two quick decisions that can be stated and answered before the run starts. No open design question, no "weigh these approaches," no waiting on Craig mid-task. This is the eligibility tag. +- *=:quick:= — effort hint only.* A small, fast task. Informational for batching and estimating a run's duration; *not* an eligibility gate (size no longer gates). + +Both tags are applied at task creation and *re-checked as a mandatory step* in the task-review and task-audit workflows, so the run-time gate can trust the author's tag rather than re-derive autonomy and effort from the task body. A task-review or task-audit that skips the =:solo:= / =:quick:= assessment is incomplete. + +** Act-vs-file decision (the defer checklist) + +After the scope read, run each eligible candidate through the checklist below. Each item is a concrete, answerable question, not an adjective. *Any* hit — or any "unsure" — sends the task to defer (or, for a quick-decision gap under the speedrun preset, to the pre-flight Q&A). Only a task that clears every item is implemented. + +1. *Test-writability (the keystone).* Can I write the failing test from the task text — plus any decisions gathered up front — without inventing a requirement? *No / unsure* → underspecified. Under the speedrun preset, if the gap is one or two quick answerable questions, route it to the pre-flight Q&A; otherwise file a =VERIFY= noting what's missing. Under the unattended loop, file the =VERIFY= (no one to ask). This replaces the old "clear / bounded / underspecified" adjectives with an action that fails loudly: if the red test isn't writable, the task isn't ready. +2. *Data-loss / irreversible / external operation.* Does implementing it require any of: =rm= of non-scratch data, =git reset --hard= / force-push, =DROP= / =DELETE= / =TRUNCATE=, file truncate/overwrite of persisted content, a schema or data migration, any external or shared-state mutation, any credential touch? *Yes* → do NOT implement; file a =VERIFY= naming the risk. This is the hard safety gate; an upfront answer never overrides it without an explicit checkpoint. Replaces the vague "data-loss risk" with an enumerated, greppable set. +3. *Already-satisfied.* Does the scope read show the desired end-state already holds? *Yes* → file a =VERIFY= noting it (the "raise max spans to 5 — every cap was already 8" case) and move on. Don't make a no-op change. +4. *Design deliberation.* Does the task carry an unresolved design question, a "weigh these approaches" with real tradeoffs, or a TBD that isn't a quick factual answer? *Yes* → under the speedrun preset, if it collapses to one or two quick questions, route to pre-flight Q&A; otherwise file and surface as a =/start-work= candidate. Under the loop, file. The discriminator is now *quick-answerable question* vs *deliberation* — not task size. + +A task that clears 1–4 is implemented under the project's commit discipline, decomposed into as many logical commits as the change needs. When genuinely unsure which side a task falls on, defer — a wrong auto-implement costs a revert *and* the next-session correction the metrics are designed to catch. + +** Pre-flight decision gathering (the no-approvals speedrun's only interaction) + +The speedrun preset front-loads every approval into one step before the run, so the run itself is uninterrupted — that is what "no approvals" means. It is *not* "no input ever"; it is "all input first, then hands-off." + +When Craig kicks off a speedrun over an explicit list: + +1. *Gather* the named task set. +2. *Scope-read and classify* each task against the eligibility gate + defer checklist: ready (clears the checklist), needs-quick-decisions (one or two upfront-answerable questions — checklist item 1 or 4), or drop (data-loss / irreversible, or design deliberation that isn't a quick question). +3. *Order* the list (priority, then the author's ordering / =:next:=). +4. *Intro the work* — present the ordered plan: what will run, what was dropped and why, and the batched questions for the needs-quick-decisions tasks. +5. *Craig answers each question, or says "skip this"* → a skipped task is removed from the run (recorded =dropped-by-craig=); an answered task has the answer recorded so implementation works from the decision, not a guess. +6. *Run the finalized list autonomously* — no further approvals until done. +7. *End-of-set page* with completed + remaining + skipped. + +The unattended *loop* caller has no human at kickoff, so it cannot gather decisions: there, a needs-quick-decisions task simply defers (files its note) like any other checklist hit. The pre-flight Q&A is a speedrun-preset capability, not a loop one. + +** Session modes and the no-approvals speedrun preset + +Two orthogonal session-mode dimensions feed the loop: + +- *Commit autonomy:* =file-only= (default) or =autonomous-commit=. =autonomous-commit= is honored only when the project carries the per-project waiver (=.emacs.d= and =rulesets= have it; most projects do not). Absent the waiver, a request for =autonomous-commit= degrades to =file-only= and says so. +- *Paging:* on or off. End-of-set only. + +The *no-approvals speedrun* is the named preset = =autonomous-commit= + always-push + paging-on, fed an *explicit ordered list*, run after the pre-flight decision-gathering step above. It is not a separate code path; it is a label for that combination of mode flags plus the explicit-list input, with the pre-flight Q&A as its only interactive moment. The loop caller, by contrast, runs =file-only= (unless the project has the waiver and opts the loop into commits) with paging off, fed the *tag query*, with no pre-flight step. + +** Bounding the run and the kill switch + +Default cap: one task per run for the loop caller — implement the highest-priority eligible candidate (=[#A]= before =[#B]= before =[#C]=), record, then stop and let the next tick continue. The speedrun preset works the whole explicit list in order (the human bounded it by naming it), still one commit per logical change. + +The kill switch is a hard per-run task cap passed by the caller, independent of "one per run": even the speedrun stops at the cap and pages with the remainder listed. A loop that fires every 30 minutes and commits unattended needs a ceiling that a runaway can't exceed. With task size now uncapped, the count cap no longer bounds *cost* — a single large task can run long — so a token/cost budget is the most pressing vNext addition. + +** End-of-set paging + +When the set is done (or the cap is hit), if paging is on, fire one page — end-of-set only, never per-task: + +#+begin_src sh +notify alarm "Page" "<project>: <N> done, <M> remaining — <one-line summary>" --persist +#+end_src + +=--persist= keeps it on screen until dismissed (the page-me convention). The message carries the project name, the completed count, and the remaining count, so Craig can reply confirming ready + naming the next project in one turn. The page-signal wrapper removed 2026-06-12 is reconciled to =notify= here — there is no separate page-signal call. + +* Alternatives Considered + +** Fold execution into inbox-zero (the Phase E stopgap shape) +- Good, because it's the smallest diff — the loop caller already runs inbox-zero, so execution is "one more phase." +- Bad, because it couples capture-routing with implementation. inbox-zero has three callers; startup and wrap-up must never execute. A Phase E inside inbox-zero forces both to carry a "skip Phase E" caveat and risks a future caller running it by accident. +- Neutral, because the eligibility-gate and defer-checklist text is identical either way — only its *home* differs. + +** Two separate features (keep Phase E and speedrun distinct) +- Good, because each proposal ships as written with no reconciliation work. +- Bad, because the execution loop is duplicated in two places and will drift; a guardrail tightened in one won't reach the other. Two ways to do autonomous execution is two things to audit. +- Neutral, because the input and session-mode differences are real — but they're thin caller-level differences, not a reason to fork the engine. + +** Keep the task-size gate (defer anything over ~30 minutes) +- Good, because it bounds per-task cost and blast radius with a single number. +- Bad, because it defeats the away-from-desk use case — anything non-trivial bounces back to Craig, so he can't actually leave. Size correlates poorly with risk; a large mechanical refactor is safer than a tiny change to persisted state. +- Neutral, because the things size was a proxy for (risk, cost) are covered directly — risk by the data-loss checklist, cost by the run cap (and the vNext token budget). The defer checklist's deliberation item, not size, is what routes genuine =/start-work= tasks out. + +** Autonomous-commit as the default +- Good, because it's faster end-to-end with no diff to review. +- Bad, because most projects lack the per-project waiver, and an unattended loop committing to a project that never opted in is exactly the failure the file-only default prevents. The blast radius of a bad autonomous commit is a revert plus lost trust in the loop. +- Neutral, because the projects that *do* want it (=.emacs.d=, =rulesets=) opt in explicitly, so the capability is available where it's wanted without being the default everywhere. + +* Decisions [8/8] + +** DONE Eligibility tag set and where it's read +- Owner / by-when: Craig / spec-review +- Context: Projects' priority/tag schemes vary, and the =todo-format.md= scheme header is the declared per-project source of truth. Task size is no longer a gate, so eligibility rests on the autonomy tag, not an effort cap. +- Decision: Eligibility = status =TODO= AND the =:solo:= autonomy tag, resolved against the project's scheme header (a project may declare a different autonomous-safe set). Priority / =:next:= drive ordering, not eligibility. =:quick:= is an effort hint, never a gate. +- Consequences: easier — one workflow works across projects with different vocab, and the gate is a pure lookup; harder — a project with no/malformed scheme header needs a fallback, and the default (=:solo:=) must be defined precisely enough that two projects agree. + +** DONE Crisp =:solo:= / =:quick:= definitions, enforced in task-review + task-audit +- Owner / by-when: Craig / spec-review +- Context: The run-time gate is only as crisp as the tags. Today =:quick:= / =:solo:= are listed in the scheme header with no hard definition, and nothing enforces that tasks get assessed for them. +- Decision: Define =:solo:= (completable without Craig beyond at most one-or-two upfront-answerable quick decisions; no design deliberation) and =:quick:= (small/fast effort hint only) in =todo-format.md=, and make assessing both a *mandatory step* in the task-review and task-audit workflows. A review/audit that skips the assessment is incomplete. +- Consequences: easier — authoring-time judgment by the human who knows the answer, and the run-time gate trusts the tag; harder — task-review and task-audit grow a required step, and existing untagged tasks need a back-fill pass. + +** DONE The do-not-auto-implement marker set +- Owner / by-when: Craig / spec-review +- Context: =VERIFY= means "awaiting Craig's manual confirmation"; other projects may use markers differently. +- Decision: Do-not-implement = any status that is not =TODO=, plus any project-declared "hold" marker. Safe-by-omission: exclude anything not plainly =TODO=. +- Consequences: easier — portable, and manual-check tasks can't auto-run; harder — richer per-project overrides need marker semantics in the scheme header, which most lack, so the default must stay conservative. + +** DONE Pre-flight decision gathering for the speedrun preset +- Owner / by-when: Craig / spec-review +- Context: Forcing every decision-needing task to defer wastes the away-from-desk use case — many tasks need only one or two quick answers Craig could give at kickoff. The speedrun is interactive at its start but must be hands-off after. +- Decision: The speedrun preset gathers + orders the set, intros the work, and batches all needed quick decisions into one pre-flight Q&A; Craig answers or says "skip this" (drops the task); the run then proceeds with zero further approvals. The unattended loop has no kickoff human, so it defers decision-needing tasks instead. +- Consequences: easier — "no approvals" becomes "all approvals first," which fits working-while-away, and larger / lightly-underspecified tasks become runnable; harder — the classifier must reliably split quick-question vs real-deliberation, and the recorded answers must reach the implementer so it works from the decision, not a guess. + +** DONE Commit-autonomy opt-in mechanism +- Owner / by-when: Craig / spec-review +- Context: =file-only= is the default; =.emacs.d= and =rulesets= have a per-project waiver allowing autonomous commits. Where does the workflow *read* that a project has opted in? +- Decision: Read the opt-in from the project's existing per-project waiver location (=notes.org= Workflow State or =CLAUDE.md=), not a new config file. Two flags: "has commit waiver" and "loop may commit" can differ. +- Consequences: easier — no new config surface, reuses the existing waiver concept; harder — the waiver location/format must be pinned for deterministic detection, and "waiver yes, loop-commit no" needs the two-flag split. + +** DONE Run-cap default and the kill switch shape +- Owner / by-when: Craig / spec-review +- Context: The loop default is one task per run; the speedrun works an explicit list. Both need a hard ceiling. Task size is now uncapped, so a single task can be large. +- Decision: The caller passes a hard per-run task cap (loop default 1; speedrun = length of the explicit list, capped at a ceiling); stop + page with the remainder when the cap is hit. v1 caps by task count, not token budget. +- Consequences: easier — a simple caller-controlled integer with a bounded task count; harder — a count cap doesn't bound *cost*, and with size uncapped a single large task can run long, so a token budget is vNext and more pressing than before. + +** DONE Metrics log location and format +- Owner / by-when: Craig / spec-review +- Context: Per-run metrics must land somewhere structured and queryable, per-project, and survive across sessions for the synthesis step to read. +- Decision: Append one JSONL record per task to a per-project log at =.ai/metrics/work-the-backlog.jsonl=, git-tracked, with the synthesis step reading the union across projects. +- Consequences: easier — append-only JSONL is trivial to write and =jq=-queryable, and per-project keeps it local to the work; harder — a git-tracked log adds commit churn, and "union across projects" needs the synthesis step to know where every log lives. + +** DONE Synthesis cadence and trigger +- Owner / by-when: Craig / spec-review +- Context: Craig wants periodic org-roam articles summarizing the data. What triggers synthesis, and how often? +- Decision: Run synthesis on an explicit trigger ("synthesize backlog metrics") and optionally a weekly scheduled run, writing one KB node per synthesis under =~/org/roam/agents/= per the knowledge-base rule. +- Consequences: easier — an explicit trigger means no surprise writes, and the KB rule already governs node shape; harder — a weekly run needs a scheduler entry, and the personal-only write-classification must gate it so work-project metrics never land in the KB. + +* Implementation phases + +** Phase 0 — Tag definitions + task-review/audit enforcement +Add the hard =:solo:= / =:quick:= definitions to =todo-format.md=, and add the mandatory tag-assessment step to the task-review and task-audit workflows. Independent of the workflow build; lands first so the eligibility gate has crisp tags to read and existing tasks start getting assessed. Tree stays working: these are rule + workflow prose additions. + +** Phase 1 — Extract the execution loop into work-the-backlog.org +Write =work-the-backlog.org= holding the eligibility gate, defer checklist, per-task quality bar, and run-cap logic — taking a task set + session mode + cap as input. Remove the stopgap "Phase E" text from =inbox-zero.org= (restore it to its A-D shape) in the same change so there's one home, not two. Tree stays working: inbox-zero reverts to routing-only, and the new workflow is callable but not yet wired to the loop. + +** Phase 2 — Wire the two callers +Add the loop caller's chain step (after inbox-zero Phase D, invoke work-the-backlog with the tag query + file-only + cap 1) and the no-approvals speedrun preset (pre-flight decision-gathering → explicit list + autonomous-commit + always-push + paging-on). Both go through the same workflow; only the speedrun runs the pre-flight Q&A. Tree stays working: each caller is independently testable. + +** Phase 3 — File-only vs autonomous-commit gate +Implement the commit-autonomy branch: read the per-project waiver, degrade =autonomous-commit= to =file-only= when absent, surface the degrade. Tree stays working: default file-only behavior is the safe path even before the waiver-read lands. + +** Phase 4 — The defer checklist, pre-flight Q&A, and the page +Implement the act-vs-file defer checklist (test-writability keystone, enumerated data-loss list, already-satisfied, design-deliberation), the speedrun pre-flight decision-gathering (gather → classify → order → intro → batch-ask → skip/answer), the =VERIFY=-on-ambiguity filing, and the end-of-set =notify alarm ... --persist= page. Tree stays working: the checklist only ever *reduces* what runs, and the pre-flight step only runs under the speedrun preset. + +** Phase 5 — Metrics log +Append the per-task JSONL record at each task outcome. Tree stays working: logging is a side effect that doesn't alter execution. + +** Phase 6 — Synthesis to org-roam +Write the synthesis step: read the JSONL union, compute the per-run and trend metrics (below), write a KB node under =~/org/roam/agents/= per the knowledge-base rule, personal-projects-only classification enforced. Tree stays working: synthesis is read-only over the logs plus a KB write. + +* Acceptance criteria +- [ ] =work-the-backlog.org= exists and is the only home for the execution loop; =inbox-zero.org= is back to its A-D routing-only shape with no Phase E. +- [ ] The loop caller chains into work-the-backlog after routing; startup and wrap-up never invoke it. +- [ ] The no-approvals speedrun runs as the preset (pre-flight Q&A → autonomous-commit + always-push + end-page) over an explicit ordered list, one commit per logical change. +- [ ] =:solo:= and =:quick:= carry hard definitions in =todo-format.md=, and task-review + task-audit both refuse to complete without assessing them. +- [ ] Eligibility = status =TODO= AND =:solo:=, read from the project's scheme header, not hardcoded; a =VERIFY= / =DOING= / =DONE= / =CANCELLED= task is skipped by the gate. +- [ ] Task size never sends a task to =/start-work=; a large but =:solo:=, well-specified task runs and is decomposed into per-logical-commit chunks. +- [ ] The defer checklist fires correctly: a task whose red test isn't writable (and isn't a quick-question gap), one carrying an enumerated data-loss operation, an already-satisfied one, and one needing design deliberation are each deferred (or routed to pre-flight Q&A under the speedrun), not implemented. +- [ ] Under the speedrun preset, a task needing one or two quick decisions is surfaced in the pre-flight Q&A; "skip this" drops it, an answer is recorded and used; the run then proceeds with no further approvals. +- [ ] Under the unattended loop, a decision-needing task defers (no pre-flight Q&A). +- [ ] In a project without the commit waiver, an =autonomous-commit= request degrades to file-only and says so; no commit is made. +- [ ] The run stops at the per-run cap and pages with the remaining tasks listed. +- [ ] Each task outcome appends one JSONL record to =.ai/metrics/work-the-backlog.jsonl=. +- [ ] The synthesis step reads the logs and writes a KB node under =~/org/roam/agents/=; it refuses to write for work-classified projects. + +* Effectiveness measurement + +This section answers Craig's explicit ask: measure whether autonomous-batch execution is actually effective, and build the "gather data → org-roam articles" loop. + +** What "effective" means here + +The autonomy is effective if it completes real work that *stays* completed — i.e. tasks land green and the next session doesn't have to undo or fix them. The two failure modes to catch are (1) the loop defers everything (over-cautious, no value delivered) and (2) the loop implements badly (commits that get reverted or hand-corrected next session). Both are measurable. + +** Per-run metrics (the JSONL record) + +One record per task, appended to =.ai/metrics/work-the-backlog.jsonl= at each task outcome: + +| Field | Meaning | +|-------------------+--------------------------------------------------------------------| +| =ts= | ISO timestamp of the task outcome | +|-------------------+--------------------------------------------------------------------| +| =run_id= | UUID shared by all tasks in one run | +|-------------------+--------------------------------------------------------------------| +| =project= | project basename | +|-------------------+--------------------------------------------------------------------| +| =caller= | =loop= or =speedrun= | +|-------------------+--------------------------------------------------------------------| +| =task= | task heading (slug) | +|-------------------+--------------------------------------------------------------------| +| =outcome= | implemented-committed / implemented-diff / deferred-verify / | +| | skipped-ineligible / dropped-by-craig (skipped at pre-flight) | +|-------------------+--------------------------------------------------------------------| +| =defer_reason= | underspecified / data-loss / already-satisfied / needs-deliberation | +|-------------------+--------------------------------------------------------------------| +| =upfront_decision=| true if a pre-flight answer was recorded and used for this task | +|-------------------+--------------------------------------------------------------------| +| =wall_clock_s= | seconds from task start to outcome | +|-------------------+--------------------------------------------------------------------| +| =commit_sha= | for committed tasks; empty otherwise | +|-------------------+--------------------------------------------------------------------| +| =review_findings= | count of /review-code Critical+Important findings on this task | +|-------------------+--------------------------------------------------------------------| + +Per-run rollups computed at synthesis (not stored per record): tasks attempted, completed, VERIFY-deferred, dropped-by-craig, reverted; wall-clock total; commits landed; review findings per commit. + +** The corrections signal (the key metric) + +The hardest and most valuable metric is *human corrections in the following session* — did Craig revert or hand-fix an autonomous commit? v1 captures the cheap proxy: at synthesis, for each =commit_sha=, check whether a later commit touching the same files reverted it or carries a "fix"/"revert" of that change within N days. A clean run is one where the autonomous commits survive untouched. (Auto-detecting "this later commit corrected that autonomous one" precisely is a vNext refinement; the proxy — reverted-or-touched-soon-after — is good enough to flag a problem run for human review.) + +** Where the data lands + +Per-project git-tracked JSONL at =.ai/metrics/work-the-backlog.jsonl=. Append-only, =jq=-queryable, survives across sessions and machines via the normal project sync. Git-tracked so the history is auditable and the synthesis step can read it from any clone. + +** The synthesis loop (gather → article) + +On the "synthesize backlog metrics" trigger (and optionally a weekly scheduled run): + +1. Read the JSONL union across the personal projects the synthesizer can see. +2. Compute the rollups and the trend: completion rate over time, defer-reason distribution, review-findings-per-commit trend, and the corrections-signal flag count. +3. Write one org-roam KB node under =~/org/roam/agents/YYYYMMDDHHMMSS-backlog-metrics-<window>.org= per the knowledge-base rule — filetags =:agent:metrics:=, a concise title, the rollup table, the trend narrative, and =[[id:...]]= links to prior synthesis nodes so the series is traceable. +4. Enforce the KB write-classification: *personal projects only*. A work-classified project's metrics never write to the KB — they stay in that project's own =.ai/metrics/= log and the synthesizer reports the refusal per the KB refusal contract. + +The KB node is the artifact Craig reviews later — "are the autonomous runs completing more and getting corrected less over the last month?" reads off the trend table without re-querying raw logs. + +* Readiness dimensions + +- *Data model & ownership:* The task set is read from =todo.org= (project-owned, user-authored). The metrics JSONL is generated, append-only, git-tracked, project-owned. KB nodes are agent-generated under =~/org/roam/agents/= (never overwriting Craig's hand-authored nodes — link only). No editable region is co-owned. +- *Errors, empty states & failure:* Empty task set → report "nothing eligible" and stop. Malformed scheme header → fall back to the default tag reading and surface the fallback. A task that fails mid-implementation → leave the tree working (don't commit a broken state), record the failure outcome, surface it, continue to the next task. No silent data loss: the data-loss guardrail refuses irreversible tasks outright. +- *Security & privacy:* Tasks touching credentials or external mutations are excluded by the data-loss / external-state checklist item. The KB write is personal-projects-only; work metrics never leave the project. No secrets in the JSONL (task slugs and SHAs only). +- *Observability:* The end-of-set page surfaces the run outcome. The per-task surface (implemented / deferred + reason / dropped / skipped) is the live progress view. The metrics log + KB synthesis is the long-run observability. A bad run is isolable from the JSONL (which task, which outcome, which review findings). +- *Performance & scale:* Expected counts are small — a handful of tasks per run, one run per 30-min tick. No bottleneck at this scale. The cap bounds the worst case on task count; with size uncapped, a single large task is the cost outlier the vNext token budget addresses. Synthesis over months of JSONL is still a small file (one record per task). +- *Reuse & lost opportunities:* Reuses =todo-format.md= for task close + the tag definitions, =/review-code= and =/voice personal= for the quality bar, =notify= for paging, the knowledge-base rule for KB writes, the per-project waiver for commit-autonomy, and task-review / task-audit for tag enforcement. No new config file (the opt-in rides the existing waiver). The execution loop is the one new shared asset. +- *Architecture fit & weak points:* Integration points — inbox-zero loop caller (chain after Phase D), the per-project waiver location, =todo.org= scheme header, task-review / task-audit, =~/org/roam/agents/=. Weak point: the commit-autonomy gate depends on deterministically reading the waiver; mitigated by defaulting to file-only when the read is ambiguous (fail safe, not open). Second weak point: a 30-min loop committing unattended with uncapped task size; mitigated by the hard count cap and file-only default, with the token budget as the vNext backstop. +- *Config surface:* Per-project — commit-autonomy opt-in (via existing waiver), optional loop-commit flag, optional autonomous-safe tag override in the scheme header. Per-call — task set, session mode, run cap. Defaults: file-only, paging-off (loop) / paging-on (speedrun), cap 1 (loop). +- *Documentation plan:* The workflow file itself is the user/operator doc (matches inbox-zero.org's self-documenting style). The =.emacs.d= stopgap note and the speedrun proposal are superseded by this spec; no separate migration doc needed beyond removing the Phase E text. +- *Dev tooling:* N/A for new build targets — the workflows are prose, exercised by invocation. The metrics JSONL is =jq=-inspectable by hand; a tiny rollup helper may be added under =.ai/scripts/= if the synthesis prose proves to need it (decided at Phase 6, not a v1 prerequisite). +- *Rollout, compatibility & rollback:* Rollout is removing Phase E from inbox-zero and adding work-the-backlog — both prose changes, instantly reversible. Compatibility: inbox-zero's three callers are unchanged except the loop caller gaining a forward chain. Rollback: delete work-the-backlog and the loop chain step; inbox-zero is already back to A-D. The file-only default means the worst pre-rollback state is surfaced diffs, not committed changes. +- *External APIs & deps:* =notify alarm "Page" "<msg>" --persist= verified against =/home/cjennings/.local/bin/notify= and the page-me workflow. =~/org/roam/= KB write path and node shape verified against the knowledge-base rule. No external API calls. + +* Risks, Rabbit Holes, and Drawbacks + +- *The corrections signal is a proxy, not ground truth.* "A later commit touched the same files" over-counts (legitimate follow-up work) and under-counts (a correction in a different file). It's a flag for human review, not a verdict. Don't rabbit-hole on making it precise in v1 — the proxy plus a human glance is the design. +- *Waiver detection drift.* If the per-project waiver location moves or its format changes, the commit-autonomy gate could mis-read. Mitigation: fail safe to file-only. Pin the waiver format in the Phase 3 decision before building. +- *Unattended-commit blast radius.* The headline risk. Mitigated four ways: file-only default, the hard cap, the data-loss checklist item, and the metrics loop (which makes a bad run visible after the fact even if the first three let something through). With task size uncapped, the cost dimension of this risk grows — the vNext token budget is the planned fifth layer. +- *Scope creep into /start-work territory.* Size is intentionally no longer the brake. The brake is the defer checklist's design-deliberation item plus the "when unsure, defer" rule — keep item 4 strict so genuine deliberation-class tasks still route out even when they're tagged =:solo:= by mistake. +- *Pre-flight classifier error.* The speedrun's gather step has to split quick-answerable-question from real-deliberation. Misclassifying a deliberation task as a quick question puts a half-baked decision into an autonomous run. Mitigation: when the question isn't answerable in one or two lines, treat it as deliberation and drop it from the run, not as a pre-flight question. + +* Testing / Verification / Rollout + +Verification is by invocation against a project's real =todo.org=: run the loop caller in file-only mode and confirm it surfaces diffs without committing; run the speedrun against a small explicit list in a waiver-carrying project and confirm the pre-flight Q&A fires, "skip this" drops a task, an answer is recorded and used, then one commit per logical change + the end page; plant a =VERIFY=-status task, a data-loss task, an already-satisfied task, and a large-but-=:solo:= task and confirm the first three are skipped/refused while the large one runs and decomposes; confirm the JSONL grows one record per task; run synthesis and confirm a KB node lands (personal project) or is refused (work project). Rollout is the Phase 0-6 sequence, each leaving the tree working; the file-only default makes early phases safe to ship before the commit and paging phases land. + +* References / Appendix + +- [[file:../design/2026-06-16-inbox-zero-phase-e-proposal.org][Phase E proposal (inbox-zero stopgap)]] and [[file:../design/2026-06-16-inbox-zero-phase-e-sender-note.org][its sender note with the 5 open questions]]. +- [[file:../design/2026-06-15-fix-speedrun-workflow-proposal.org][speedrun proposal]] (file retains its original on-disk name pending a rename pass). +- [[file:../../.ai/workflows/inbox.org][inbox.org (canonical A-D; was inbox-zero.org)]] — the routing workflow this feature decouples from. +- =~/code/rulesets/claude-rules/knowledge-base.md= — the org-roam write contract the synthesis step follows. + +* Review and iteration history +** 2026-06-16 Tue — author +- What: initial draft reconciling the Phase E and fix-speedrun proposals into one work-the-backlog.org feature, plus the effectiveness-measurement instrumentation. +- Why: two overlapping proposals arrived within a day; building them separately would duplicate the execution loop and let it drift. Craig also asked explicitly for measurement + org-roam synthesis. +- Artifacts: this spec; the two source proposals under docs/design/ (the Phase E proposal, diff, and sender note now filed there). +** 2026-06-28 Sun — revision (Craig) +- What: removed the task-size gate (size no longer defers; large tasks decompose into per-commit chunks); recast the act-vs-file rule as a crisp four-item defer checklist keyed on test-writability; added crisp =:solo:= / =:quick:= definitions destined for =todo-format.md= and made their assessment mandatory in task-review + task-audit; added the speedrun's pre-flight decision-gathering step (batch the quick questions up front, "skip this" drops a task, then run hands-off); renamed "fix speedrun" → "no-approvals speedrun" in prose. Status stays draft pending ratification of the revised decisions. +- Why: the original criteria were adjectives, not checkable; the size gate forced Craig to stay at his desk for anything non-trivial, defeating the away-from-desk use case; and decision-needing tasks were over-deferred when many need only a quick upfront answer. +** 2026-06-29 Mon — ratified +- What: Craig ratified all eight revised decisions; Status → ready. Implementation-ready across Phase 0 (tag definitions + task-review/audit enforcement) through Phase 6 (synthesis). +- Why: the crisp defer checklist and the pre-flight-Q&A design resolved the "criteria too soft" and "size shouldn't gate" concerns that held the spec in draft. diff --git a/docs/specs/2026-06-16-encourage-kb-contribution-spec.org b/docs/specs/2026-06-16-encourage-kb-contribution-spec.org new file mode 100644 index 0000000..3d08ed5 --- /dev/null +++ b/docs/specs/2026-06-16-encourage-kb-contribution-spec.org @@ -0,0 +1,206 @@ +#+TITLE: Encourage Org-Roam KB Contribution Across Workflows — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-16 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* READY Encourage Org-Roam KB Contribution Across Workflows — Spec +:PROPERTIES: +:ID: f67f5f45-5aa1-4a5a-8704-d636e4e16f75 +:END: +- 2026-07-02 Thu @ 00:17:01 -0400 — retrofitted by spec-sort; status set to READY (evidence-based, human-confirmed) + +* Metadata +| Status | ready | +|----------+------------------------------------------------| +| Owner | Craig Jennings | +|----------+------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+------------------------------------------------| +| Date | 2026-06-16 | +|----------+------------------------------------------------| +| Related | [[file:../../todo.org][rulesets todo.org]] | +|----------+------------------------------------------------| + +* Summary + +The org-roam KB already exists (=knowledge-base.md=: =~/org/roam/agents/=, =:agent:= filetag, capture-then-promote, personal-vs-work write boundary), but nothing in the daily workflow loop encourages agents to use it. The wrap-up's =KB: promoted N / consulted yes-no= receipt is the only touchpoint, and it fires at the very end when the session's learnings have already faded. This feature wires four light prompts into the synced template workflows — startup, triage-intake, inbox-zero, wrap-it-up — plus one curated best-practices node in the KB, so contributing durable knowledge becomes a habit the workflows nudge rather than a rule agents forget. + +* Problem / Context + +The KB rule is sound but passive. An agent reads =knowledge-base.md= once at rule-load and then never gets reminded to consult or contribute, so the KB stays nearly empty and never reaches the critical mass where consulting it pays off. The compounding asset Craig wants — a cross-project store that gets more valuable as it grows — needs a contribution habit, and habits in this system come from workflow prompts, not from a rule sitting in the background. + +Three gaps: + +1. *No quality guidance.* =knowledge-base.md= says what goes in (durable facts) and where (=agents/= nodes), but not /how/ to write a good node — atomic, descriptively titled, linked. An agent following the rule literally can still produce a junk drawer of vague, unlinked notes that no future agent can find or trust. +2. *No mid-session capture prompts.* Triage-intake and inbox-zero both surface durable signal (a recurring pattern across messages, a reference pointer worth keeping) and then drop it. Nothing tells the agent "that was worth a node." +3. *The only contribution prompt is too late.* Wrap-up's KB promotion check runs in Step 1, after the session, when the agent is reconstructing learnings from the log rather than capturing them while fresh. + +* Goals and Non-Goals + +** Goals +- Curate a best-practices node in the KB that teaches agents how to write good nodes, drawing on established note-taking guidance. +- Link that node from startup with a light, one-line encouragement to contribute through the session. +- Add a short end-of-flow KB reminder to triage-intake and inbox-zero. +- Add an early wrap-up prompt that asks what the agent learned worth remembering, feeding the existing =KB: promoted N= receipt. +- Keep every prompt light and non-blocking — encouragement, never a gate. + +** Non-Goals +- *Not* changing =knowledge-base.md='s write boundary, schema, or the work/personal classification. The feature builds on that rule unchanged. +- *Not* adding a blocking gate anywhere. No workflow stalls or fails because a node wasn't written. +- *Not* automating node creation. The agent decides what's durable; the prompts only ask the question. +- *Not* a second receipt or metric. Wrap-up's =KB: promoted N / consulted yes-no= line stays the single instrumentation point. +- *Not* touching the wrap-up's existing Step 1 KB-promotion sub-section's schema — the new early prompt /feeds/ it, it doesn't replace it. + +** Scope tiers +- v1: the four workflow edits + the one curated best-practices node. All synced templates, so the edits propagate to every project on next startup. +- Out of scope: a contribution-rate dashboard, per-project KB stats, auto-suggesting nodes from session content. +- vNext: a "consult the KB before this task" prompt in start-work / spec-create (deferred — log to todo.org). + +* Design + +The feature is four small prompt insertions plus one authored artifact. The design work is mostly about /placement/ and /wording/: these are synced templates, so a prompt that reads as nagging gets paid forward to every project on every run. The governing constraint is "light enough that an agent welcomes it, specific enough that it actually fires." + +** The best-practices node (the artifact) + +The node lives at =~/org/roam/agents/<timestamp>-agent-kb-best-practices.org=, authored by hand (not agent-generated), with the standard =:agent:reference:= filetags so it's a first-class KB node agents can find by the same =rg= the rule already documents. It is the one node startup links to, and the substance the workflow prompts point at instead of re-explaining note-taking inline. + +Its content is curated from the established note-taking literature — Sönke Ahrens' systematization of Luhmann's Zettelkasten, Andy Matuschak's evergreen-notes practice, and the org-roam community's own guidance — distilled to the handful of principles that matter for an /agent/ writing /durable facts/, not a human building a thinking environment. Proposed outline: + +1. *Why the KB exists* — one paragraph: a cross-project, cross-machine asset that compounds. Consulting it saves re-deriving; contributing to it pays the next agent forward. +2. *One idea per node (atomicity).* Each node holds a single durable fact. Atomicity is what makes a note linkable and findable — a node about three things links cleanly to none of them. (Ahrens; zettelkasten.de atomicity guide.) +3. *Descriptive, declarative titles.* The title states the claim, not the topic: "SSH auth routes through gpg-agent with a separate cache TTL" beats "SSH notes." A title you can read as a standalone statement is one a future agent can scan and trust without opening the node. (Matuschak evergreen notes; org-roam community practice.) +4. *Link liberally.* Use =[[id:...]]= to connect a new node to related ones; the value is in the network, not the isolated note. Link to Craig's hand-authored nodes, never edit them. (Matuschak "densely linked"; the linking principle.) +5. *Capture, then promote.* Harness memory is the fast capture layer; the KB is for facts that cleared the durability bar. Don't promote everything — promote what transfers. (Mirrors =knowledge-base.md='s capture-then-promote.) +6. *What goes in / what stays out.* Restate the rule's inclusion bar tersely (durable, cross-project, the why behind a decision, environment gotchas, reference pointers) and the exclusion bar (session state, task state, high-churn facts, secrets, anything the repo already records). +7. *The write boundary.* One line pointing at =knowledge-base.md=: personal projects only, work and unknown projects never write — with the refusal contract. The node /defers/ to the rule here rather than restating the denylist, so there's one source of truth for the boundary. +8. *Sources.* The citations below, as a reference footer. + +Two-altitude note: for a /reading/ agent the node is "how do I tell a good node from a bad one before I trust it?"; for a /writing/ agent it's "what shape should this fact take before I commit it?" The outline serves both — principles 2-4 are the writing checklist, 6-7 are the reading/eligibility filter. + +** The four workflow prompts (placement + wording) + +Each is the minimum that fires reliably without nagging. Exact insertion points and proposed copy are in Implementation phases below; the design rationale per prompt: + +- *Startup (link + light encouragement).* Startup already reads =notes.org= and surfaces nudges in Phase C. The KB encouragement rides there as one line, not a new phase — it points at the best-practices node and frames the session's contribution as welcome, not required. It fires once per session at the top, setting the frame; the other three prompts collect on it. +- *Triage-intake (end-of-flow reminder).* Placed at the very end of Phase D / Exit Criteria, after actions ship — the moment the agent has just seen a sweep's worth of signal and might recognize a durable pattern. One line, conditional in spirit ("if anything here was durable…"), never a blocking step before close-out. +- *Inbox-zero (end-of-flow reminder).* Same shape, placed in Phase D (Surface) after the moved/folded/dropped report — the agent has just triaged a batch and may have spotted a reference pointer worth keeping. +- *Wrap-up (early prompt feeding the existing receipt).* Placed at the /start/ of Step 1, before the Summary is finalized, while the session is fresh — "what did you learn worth remembering, for yourself or a future agent?" The answer flows into the existing Step 1 KB-promotion sub-section and its =KB: promoted N / consulted yes-no= receipt. The early prompt and the existing check are one pipeline: the prompt captures while fresh, the existing sub-section does the promotion and writes the receipt. No second receipt. + +** How the early wrap-up prompt feeds the existing receipt + +The existing wrap-up Step 1 already has a "KB promotion check" sub-section that asks the promotion question and writes =KB: promoted N / consulted yes-no=. The new early prompt is not a second check — it's a /relocation of the asking/ to the top of Step 1 so the question lands while the session is fresh rather than after the Summary is reconstructed. The existing sub-section keeps ownership of the actual promotion (writing the =agents/= nodes per schema) and the receipt line. Concretely: the early prompt asks and collects candidate facts into the session's working notes; the existing sub-section consumes those candidates, writes the nodes, and emits the one receipt. This avoids duplication by making the early prompt a /capture/ step and the existing check the /commit + receipt/ step of the same pipeline. + +* Alternatives Considered + +** A blocking gate ("you must write ≥1 node to wrap up") +- Good, because it would guarantee contributions and grow the KB fast. +- Bad, because it manufactures junk — agents would write a throwaway node to clear the gate, polluting exactly the asset the feature is meant to grow. It also fights the "light, non-nagging" constraint head-on. +- Neutral, because the receipt already gives visibility into contribution rate without forcing it. + +** Inlining the best-practices guidance into each workflow prompt +- Good, because the guidance is right there at the point of use; no indirection. +- Bad, because it's four copies of the same note-taking advice in four synced templates — duplication that drifts, and four times the prompt length, which reads as nagging. One linked node keeps each prompt to one line. +- Neutral, because a one-node-plus-links shape is exactly what the best-practices node /teaches/, so the design eats its own dogfood. + +** Putting the encouragement only in =knowledge-base.md= (no workflow edits) +- Good, because it's the least change — one rule edit, no template churn. +- Bad, because that's the status quo that produced the problem: a rule read once at load and then forgotten. Habits in this system come from workflow prompts, not background rules. +- Neutral, because the rule still carries the authoritative boundary; the workflow prompts are the habit layer on top. + +* Decisions [6/6] + +** DONE Where exactly does the startup link land — Phase A read, Phase C nudge, or notes.org? +- Owner / by-when: Craig / before implementation +- Context: Startup has three candidate homes for the KB encouragement: a Phase A parallel read of the best-practices node (costs context every session), a Phase C surfaced nudge (one line, conditional, consistent with the existing roam-inbox and task-review nudges), or a static line in each project's =notes.org= Active Reminders (per-project, not synced, drifts). The Phase C nudge matches the established nudge pattern and costs nothing when there's nothing to say. +- Decision: We will add the encouragement as a one-line Phase C nudge in startup.org, pointing at the best-practices node by its KB path, surfaced once near the other Phase C nudges. +- Consequences: easier — consistent with existing nudge mechanics, synced to every project, no per-session read cost; harder — one more line competing for attention in the Phase C surface, so the wording has to earn its place and stay terse. + +** DONE Is the startup nudge unconditional, or gated on the KB clone being present? +- Owner / by-when: Craig / before implementation +- Context: =~/org/roam/= isn't on every machine. The existing roam-inbox nudge already guards on the clone's presence ([ -f ~/org/roam/inbox.org ]). An unconditional KB nudge would fire on machines where the agent can't act on it. +- Decision: We will gate the startup nudge on the roam clone being present, reusing the existing presence check, so the encouragement only appears where the agent can act on it. +- Consequences: easier — no dead nudge on KB-less machines, mirrors the roam-inbox guard; harder — one more conditional in Phase C, and a machine without the clone gets no encouragement at all (acceptable — it can't contribute there anyway). + +** DONE Does the early wrap-up prompt stop and ask Craig, or self-answer silently? +- Owner / by-when: Craig / before implementation +- Context: Wrap-up is meant to be quick — Craig already authorized the wrap, and the existing KB-promotion check self-answers (the agent decides what's durable; work projects skip the write). An early prompt that /stops and asks Craig/ "what did you learn?" would add an interactive turn to a flow designed not to have them. But a purely silent self-answer risks the agent skipping the reflection. +- Decision: We will have the agent self-answer the early prompt — reflect on session learnings and stage candidate facts — without stopping to ask Craig, matching the wrap-up's no-extra-turns design; the candidates flow into the existing promotion check which writes the nodes and receipt. +- Consequences: easier — preserves wrap-up cadence, no new interactive gate, one pipeline from reflect to receipt; harder — relies on the agent actually reflecting rather than rubber-stamping "nothing learned," which the receipt makes visible over time but doesn't enforce. + +** DONE Do triage-intake and inbox-zero reminders fire every run, or only when the run surfaced something durable? +- Owner / by-when: Craig / before implementation +- Context: Both workflows run frequently (triage-intake between meetings, inbox-zero twice a session). A reminder on /every/ run is the textbook nag-fatigue failure — a line the agent learns to skip. A reminder gated on "this run surfaced a pattern / reference pointer worth keeping" fires rarely and stays meaningful, but requires the agent to make that judgment, which is softer than a mechanical condition. +- Decision: We will make both reminders conditional in spirit — a single line phrased as "if anything here was durable, write it to the KB" that the agent acts on only when the run actually surfaced something, rather than an unconditional step; an all-quiet triage sweep or an empty inbox-zero run emits no KB line. +- Consequences: easier — the reminder stays rare and credible, never pads a no-change sweep, fits triage-intake's deltas-only discipline; harder — "durable-looking" is an agent judgment with no mechanical check, so the reminder's effectiveness rides on the best-practices node teaching that judgment well. + +** DONE Best-practices node: agent-authored once, or hand-authored by Craig? +- Owner / by-when: Craig / before implementation +- Context: =knowledge-base.md= says agents never edit Craig's hand-authored nodes. The best-practices node is /about/ how agents write nodes — if an agent authors it, future agents may treat it as fair game to edit; if Craig hand-authors it, it's protected and stable but he writes it. Given it's a foundational reference the whole feature points at, stability matters. +- Decision: We will have Craig hand-author the best-practices node from the outline in this spec, so it's a protected, stable reference; the spec supplies the full drafted content for him to review and commit. +- Consequences: easier — the node is stable and protected from agent edits, one authoritative reference; harder — Craig writes (or reviews-and-commits) it rather than delegating, and updates to it are his call, not an agent's. + +** DONE Read side: how does startup surface lessons to consult, not just encourage contribution? +- Owner / by-when: Craig / ratified 2026-06-20 +- Context: The original spec only strengthened the /write/ side — startup encourages contributing (D1) but never surfaces existing KB lessons to /read/. The wrap-up receipt data shows "consulted no" across recent sessions: agents don't reach for the KB because nothing brings it to their attention at the moment work starts. =knowledge-base.md='s "search the KB first" is reactive and read-once-at-rule-load. A proactive surfacing at startup is the missing counterpart to D1. The cost constraint is the same one D1 dodged: a full Phase A read of matching nodes would spend context every session. +- Decision: We will add a second startup Phase C nudge (alongside D1's contribute-link, gated on the same roam-clone presence check) that surfaces KB lessons relevant to the current project — a count plus the nodes' declarative /titles only/ (no full-node read), capped at ~5. Relevance is matched cheaply on the project basename and obvious topic words against node titles/filetags/paths, with a most-recent fallback when nothing matches. The agent opens a node on demand. Titles are declarative by the best-practices node's own rule, so a title alone tells the agent whether to open it. +- Consequences: easier — closes the "consulted no" half with near-zero context cost (titles only), reuses the Phase C nudge pattern and the roam guard, and the consult and contribute nudges sit together as one KB surface; harder — relevance matching is a heuristic that can miss or mis-surface, and it adds a second KB line to Phase C, so both must stay terse to avoid nudge fatigue. If the receipt shows consults rising but the surfaced titles are noise, tighten the match. + +* Implementation phases + +** Phase 1 — Author the best-practices node +Write =~/org/roam/agents/<timestamp>-agent-kb-best-practices.org= from the outline in Design, with a generated =:ID:=, =#+title:=, =:filetags: :agent:reference:=, the eight content sections, =[[id:...]]= links to any existing related =:agent:= nodes, and the sources footer. Commit + push the roam repo per =knowledge-base.md='s session discipline. Leaves the KB with one new reference node and nothing else touched. + +** Phase 2 — Wire the startup encouragement (contribute + consult) +Add two one-line Phase C nudges to =claude-templates/.ai/workflows/startup.org= (canonical side), both gated on the roam-clone presence check: (1) D1's contribute-link pointing at the best-practices node by path, and (2) D6's consult-surface listing project-relevant KB node titles (count + titles only, capped ~5, project-basename match with recent fallback). A Phase A read counts =:agent:= nodes cheaply so Phase C only does the title surfacing when there's something to show. Run =scripts/sync-check.sh --fix=, commit both canonical + mirror. Propagates to every project on next startup. + +** Phase 3 — Wire the three remaining prompts +Add the end-of-flow KB reminder to =triage-intake.org= (end of Phase D / Exit Criteria) and =inbox-zero.org= (Phase D Surface), and the early KB prompt to =wrap-it-up.org= (top of Step 1, feeding the existing promotion check). All on the canonical side, then sync-check + commit. Each edit is one short block; the tree stays working after each. + +** Phase 4 — Verify propagation + receipt linkage +Confirm the four edits survive a startup sync into a test project, the wrap-up early prompt's output reaches the existing =KB: promoted N / consulted yes-no= receipt (no duplicate receipt), and the best-practices node is reachable by the =rg= the rule documents. + +* Acceptance criteria +- [ ] Best-practices node exists at =~/org/roam/agents/= with =:agent:reference:= tags, is found by =rg '#\+filetags:.*:agent:' ~/org/roam/=, and cites its sources. +- [ ] Startup surfaces a single KB-contribution line in Phase C, gated on the roam clone, pointing at the node — and stays silent when the clone is absent. +- [ ] Startup also surfaces a KB-consult line in Phase C (D6): project-relevant node titles (count + titles only, capped ~5), gated on the clone, silent when nothing matches and the clone is absent. +- [ ] Triage-intake and inbox-zero each emit one KB reminder line only when the run surfaced something durable; an all-quiet run emits none. +- [ ] Wrap-up asks the "what did you learn?" reflection early in Step 1, and its candidates feed the existing promotion check — producing exactly one =KB: promoted N / consulted yes-no= receipt, not two. +- [ ] No workflow blocks, stalls, or fails because a node wasn't written. +- [ ] All four workflow edits are on the canonical =claude-templates/.ai/= side, mirror synced, sync-check clean. + +* Readiness dimensions +- Data model & ownership: KB nodes are agent-written under =agents/=; the best-practices node is Craig-authored and protected. No new persisted state beyond the one node and the four template edits. Wrap-up receipt ownership unchanged. +- Errors, empty states & failure: roam clone absent → all KB prompts silently no-op (reuse existing presence guards). Work/unknown project → write boundary in =knowledge-base.md= still refuses with its contract; prompts fire but the agent declines to write per the rule. No silent data loss — nothing is deleted. +- Security & privacy: no secrets in nodes (rule's exclusion bar). Work-confidential facts never written (the boundary). The best-practices node is reference-only, no sensitive content. +- Observability: the existing =KB: promoted N / consulted yes-no= receipt is the single metric; grepping session archives for =KB:= answers "are agents using this?" No new instrumentation added. +- Performance & scale: four one-line prompts; negligible. The startup nudge is a Phase C surface line, not a Phase A read, so no per-session context cost from loading the node. +- Reuse & lost opportunities: reuses the existing Phase C nudge pattern, the roam-clone presence guard, the wrap-up promotion check + receipt, and =knowledge-base.md='s boundary. Nothing reinvented. +- Architecture fit & weak points: the four workflows are synced templates; canonical-vs-mirror edit discipline applies (CLAUDE.md). Weak point — nag fatigue if the reminders fire unconditionally; mitigated by the conditional-in-spirit decision. Weak point — the reminders rely on agent judgment ("durable-looking"); mitigated by the best-practices node teaching that judgment. +- Config surface: none. No new knobs; the prompts are unconditional copy gated only on the existing roam-clone check. +- Documentation plan: the best-practices node /is/ the user-facing doc. =knowledge-base.md= stays the authoritative rule; this feature adds no new rule file. No migration doc needed. +- Dev tooling: =scripts/sync-check.sh --fix= keeps canonical + mirror aligned (enforced by =githooks/pre-commit=). =make test= covers the repo's existing gates; no new test target needed for prose-only workflow edits. +- Rollout, compatibility & rollback: edits propagate via the startup rsync to every project on next session — no migration. Rollback is reverting the four template edits + deleting the node; nothing persisted depends on them. Fully reversible. +- External APIs & deps: none — no API calls, no new dependencies. The only external surface is the =~/org/roam/= git repo, already in use by the rule. + +* Risks, Rabbit Holes, and Drawbacks +- *Nag fatigue* — the central risk. Four prompts across four frequently-run workflows can train agents to skip them. Dodge: one line each, conditional in spirit, the startup line gated, the triage/inbox reminders firing only on real signal. If the receipt shows agents tuning them out, cut the lowest-value prompt rather than adding more. +- *Junk-node accumulation* — encouraging contribution without a quality bar grows a junk drawer. Dodge: the best-practices node /is/ the quality bar, and the exclusion list keeps high-churn / session-state facts out. Craig prunes at will (the rule already grants this). +- *Receipt double-counting* — if the early wrap-up prompt writes its own receipt, the metric breaks. Dodge: the early prompt is explicitly a capture step feeding the existing check; only the existing sub-section emits the receipt. Acceptance criterion guards this. + +* References / Appendix +Sources for the best-practices node's curated content: +- Sönke Ahrens, /How to Take Smart Notes/ — atomicity, own-words, linking: [[https://www.soenkeahrens.de/en/takesmartnotes][soenkeahrens.de]]; principle of atomicity: [[https://zettelkasten.de/atomicity/guide/][zettelkasten.de atomicity guide]]. +- Andy Matuschak, /Evergreen notes/ — concept-oriented, densely linked, write for yourself: [[https://notes.andymatuschak.org/Evergreen_notes_should_be_concept-oriented][notes.andymatuschak.org]]. +- Org-roam community practice — declarative titles, atomic nodes, capture-then-refine: [[https://www.orgroam.com/manual.html][Org-roam manual]]; [[https://lucidmanager.org/productivity/taking-notes-with-emacs-org-mode-and-org-roam/][lucidmanager.org org-roam guide]]. +- Existing rule this builds on: =~/code/rulesets/claude-rules/knowledge-base.md=. + +* Review and iteration history +** 2026-06-16 Tue — author +- What: initial draft. +- Why: Craig wants the org-roam KB to compound into a cross-project asset; needs the workflow wiring + curated best-practices node speced before building. +- Artifacts: this spec; four target workflows (startup, triage-intake, inbox-zero, wrap-it-up); =knowledge-base.md=. +** 2026-06-20 Sat — ratified + read-side added +- What: ratified all five original decisions; added decision D6 (read-side startup consult-nudge) and threaded it through Design, Phase 2, and acceptance. Status draft → approved. +- Why: receipt data showed the write-only design left "consulted no" across recent sessions. Craig asked for the reverse of contribution — surfacing relevant lessons to read at startup. D6 is that counterpart. +- Artifacts: this spec; startup.org (now two Phase C nudges); the lint level-2-dated-header checker tracked separately. diff --git a/docs/specs/2026-07-01-docs-lifecycle-spec.org b/docs/specs/2026-07-01-docs-lifecycle-spec.org new file mode 100644 index 0000000..91c1603 --- /dev/null +++ b/docs/specs/2026-07-01-docs-lifecycle-spec.org @@ -0,0 +1,361 @@ +#+TITLE: Docs Lifecycle — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-01 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Docs lifecycle +:PROPERTIES: +:ID: 80b0787b-4a60-4c82-8a16-b383d3e3c8f2 +:END: +- 2026-07-04 Sat @ 11:46:31 -0500 — DOING → IMPLEMENTED: all four build phases shipped (docs-lifecycle rule + spec-workflow updates, spec-sort helper + 30-test bats suite, rulesets pilot sorting the pile and standing up the status board, startup nudge gated on :LAST_SPEC_SORT:) plus the follow-up file:→id: link conversion. Remaining manual validation (startup nudge fires/clears, moved-spec links click through in Emacs) tracked as its own task; a failed check promotes to a bug. +- 2026-07-01 Wed @ 23:34:15 -0400 — READY → DOING: spec-response decomposition ran — build parent in todo.org carries the :SPEC_ID: binding, one task per phase plus the flip-to-IMPLEMENTED task and the manual-testing child. First live exercise of the transition-ownership table. +- 2026-07-01 Wed @ 23:22:50 -0400 — DRAFT → READY: Codex re-review found all fourteen review findings closed and no remaining blocking implementation-readiness gaps. +- 2026-07-01 Wed @ 22:54:41 -0400 — verify pass on the second responder round: all five fixes held, findings 1-9 unregressed, verdict ready; three minor nits folded in (scoped id-link criterion, untracked-copy cleanup in the recovery recipe, two stale prose spots). Stays DRAFT pending the reviewers' flip. +- 2026-07-01 Wed @ 22:46:52 -0400 — second responder pass: all five re-review findings fixed (fourteen of fourteen closed); stays DRAFT — the READY flip belongs to the reviewers this round. +- 2026-07-01 Wed @ 22:41:33 -0400 — READY → DRAFT: Codex re-review found five new blocking implementation-readiness gaps after the response pass. +- 2026-07-01 Wed @ 22:41:21 -0400 — DRAFT → READY: dual independent review (Codex + fresh-context Claude agent, both initially Not ready), all nine findings fixed, verify pass by the original reviewer returned ready; flip authorized by Craig. +- 2026-07-01 Wed @ 22:13:00 -0400 — drafted from the five decisions settled 2026-06-28 (todo.org "Spec storage location + lifecycle-status convention"). + +* Metadata +| Status | implemented | +|----------+------------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+------------------------------------------------------------------| +| Reviewer | Craig Jennings | +|----------+------------------------------------------------------------------| +| Date | 2026-07-01 | +|----------+------------------------------------------------------------------| +| Related | [[file:../design/2026-06-15-spec-storage-lifecycle-proposal.org][source proposal]]; todo.org "Spec storage location + | +| | lifecycle-status convention" | +|----------+------------------------------------------------------------------| + +* Summary + +Formal specs and working notes currently share one directory per project, and a spec's lifecycle state (drafted, in progress, shipped, dead) is invisible without opening the file. This spec adopts two coupled conventions — a location split (=docs/specs/= for formal specs, =docs/design/= for notes) and an authoritative in-file status carried by an org TODO keyword on a top-level status heading — plus =org-id= links for rename-safety, a general =docs-lifecycle= rule capturing the shape, and a one-time confirmed retrofit that sorts every project's existing pile. + +* Problem / Context + +.emacs.d triaged ~28 design docs and had to run a four-agent sweep reading every spec against the code to reconstruct which had shipped (6 implemented, 8 in progress, 12 not started, 1 superseded). Nothing in the filename, location, or file records the state, so the answer to "what's open?" degrades into "open every file and infer." rulesets has the same shape: 41 files in =docs/design/= of which only 3 carry a formal spec spine, plus two =-spec.org= files misfiled at the =docs/= root. The cost compounds with every doc added, and every project inherits the problem through the shared spec-create workflow. + +Two forces beyond triage cost: + +- *Links are load-bearing.* =todo.org= tasks, session archives, and sibling docs link specs by =file:= path. Any convention that renames or moves files on every status change (the filename-suffix approach) breaks those links repeatedly across a cross-linked, template-synced doc set. +- *The convention is worthless if legacy docs stay misfiled* (Craig, 2026-06-28). Template sync distributes rules and workflows but cannot perform a one-time per-project migration, so the design must include a reach mechanism that gets each project's existing pile sorted once. + +* Goals and Non-Goals + +** Goals +- A directory listing answers "which docs are specs, and what state is each in" without opening files. +- Status transitions cost one small in-file edit (keyword + history line + Metadata mirror) — no rename, no link surgery. +- Cross-doc spec links survive moves and renames. +- The shape is captured once as a general rule (=docs-lifecycle=) so future artifact collections (brainstorm piles, recording queues) can reuse it. +- Every existing project's =docs/design/= pile gets sorted exactly once, with human confirmation on each classification. + +** Non-Goals +- No automation of status flips — the keyword is edited by whoever changes the state (spec-create, spec-review, spec-response, or a human), not by a watcher. +- No retroactive rewriting of session archives or git history that reference old paths; only live inbound links (=todo.org=, =notes.org=, docs) are updated by the retrofit. +- No new tracking database or index file — the files are the index. + +** Scope tiers +- v1: the location split, the status-heading convention, the org-id link standard, the =docs-lifecycle= rule, spec-create/spec-review/spec-response updates, the retrofit helper + startup nudge, and the rulesets pilot. +- Out of scope: applying the lifecycle shape to non-doc collections (the rule documents the pattern; adopting it elsewhere is per-collection work). +- vNext: an org-agenda custom view over =docs/specs/*.org= keyed on the status keywords (nice-to-have once the keywords exist; log to todo.org). + +* Design + +** The location split + +- =docs/specs/= — formal specs only. A *spec* is a doc proposing a buildable change that carries a =Decisions= section and =Implementation phases= (the spec-create spine). Filenames keep the existing =YYYY-MM-DD-<topic>-spec.org= shape — the =-spec.org= suffix stays because spec-review's Phase 0 precondition keys on it; only the *status* suffixes from the original proposal are dropped. +- =docs/design/= — everything else: brainstorms, inventories, proposals, research notes, frozen source material. Review findings live inside the spec they review (current spec-review behavior), so standalone review files are legacy notes and stay in =docs/design/=. + +** The status heading (the authoritative record) + +Each spec's first element after the file header is a single top-level *status heading* carrying the org TODO keyword: + +#+begin_example +,#+TODO: TODO | DONE +,#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +,* DOING <spec short name> +:PROPERTIES: +:ID: <uuid> +:END: +- <dated one-line history entries, newest first> +#+end_example + +- *The keyword is authoritative.* The Metadata table's =Status= field mirrors it in lowercase for readers already in the table, and a status transition updates keyword + history line + mirror in the same edit; on disagreement the heading wins. +- *Two keyword sequences, no collisions.* The lifecycle sequence *joins* — never replaces — the =TODO | DONE= sequence that the =* Decisions= and =* Review findings= task machinery depends on. The two sequences share no keyword (the old header's =SUPERSEDED CANCELLED= done-states migrate to the lifecycle sequence; a legacy =CANCELLED= decision heading still parses as a done-state there, so =[/]= cookies stay mechanically correct). The retrofit rewrites each legacy header to carry both lines. +- *Vocabulary:* =DRAFT= (being written) → =READY= (review passed, buildable) → =DOING= (implementation in progress) → =IMPLEMENTED= / =SUPERSEDED= / =CANCELLED= (terminal). +- *Transition ownership — every flip has a named owner:* + - =DRAFT= — spec-create stamps it at authoring time. + - =DRAFT= → =READY= — spec-review, on a passing gate (keyword + history line + mirror in the review pass). + - =READY= → =DOING= — spec-response, when it decomposes the phases into build tasks. *The decomposition writes the spec-to-task binding:* the =todo.org= parent task it creates (or updates) carries a =:SPEC_ID:= property holding the spec's status-heading UUID. That property is the durable join between the spec and its build work. + - =DOING= → =IMPLEMENTED= — the session that completes the final implementation phase. To make that a tracked obligation rather than a memory, spec-response's phase-to-task breakdown *always emits a final task*: "flip the spec to IMPLEMENTED + history line," as a child of the bound parent. Safety net: task-audit's reconcile pass runs one query — for each =docs/specs/*.org= whose keyword is =DOING=, find the =todo.org= task with the matching =:SPEC_ID:=; flag the spec when that parent is =DONE=/=CANCELLED=, archived, or missing. Checking the *parent's* keyword (not "are all child tasks closed") sidesteps both the flip-task chicken-and-egg (the parent only closes after the flip task ran) and =--convert-subtasks= rewriting completed children into dated entries (dated children never affect the parent's keyword). This is the mechanism whose absence produced the .emacs.d six-shipped-specs-with-no-record failure; "a human remembers" is explicitly not the design. + - =SUPERSEDED= / =CANCELLED= — whoever makes the call, with the reason in the history line. +- *Glanceability without opening files:* one grep gives the full board — + + #+begin_src sh + rg -H '^\* (DRAFT|READY|DOING|IMPLEMENTED|SUPERSEDED|CANCELLED) ' docs/specs/ + #+end_src + + and because the keyword sits on a real org heading, an org-agenda view over =docs/specs/= works for free (the vNext item). +- *The heading body is the dated status history* — one line per transition (=YYYY-MM-DD Day @ HH:MM:SS -ZZZZ — <what changed, by whom>=), the record a filename could never carry. +- Why a dedicated status heading rather than restructuring each spec under one top-level heading: demoting every section in every existing spec is a large, link-hostile rewrite; a prepended heading is additive, retrofittable by script, and leaves the familiar flat section layout untouched. + +** Rename-safe links + +The status heading carries an =:ID:= UUID, assigned at authoring time (and by the retrofit for legacy specs). The target state is that cross-doc references to a spec use =[[id:<uuid>]]= rather than =file:= paths, so any future move can't orphan them. =file:= links remain fine for intra-doc anchors and for notes that never move. The KB's existing id-resolution recipe applies: =rg ':ID:[[:space:]]+<uuid>' docs/=. + +*Staged conversion — ids assigned now, links converted only when clickable.* =org-id-locations= only indexes agenda files and files org has visited, so a fresh =:ID:= in =docs/specs/= won't resolve on click in a live Emacs until the id index learns about project docs. =org-id-extra-files= is not a glob mechanism — it's a literal file list, only consulted under =org-id-track-globally= — so "point it at the globs" is not executable as written. The sequencing is therefore: + +1. *Pilot and retrofit rewrite =file:= links only* (path recomputation per the relink contract). Every link stays clickable throughout; no conversion window exists. +2. *:ID: properties are still assigned* during the sort — harmless, and they make the later conversion mechanical. +3. *Link conversion to =id:= is a separate follow-up pass*, gated on .emacs.d landing an executable id-index mechanism: enumerate each project's =docs/specs/*.org= into =org-id-extra-files= as real file names (a small function globbing at startup, with =org-id-track-globally= t), or a periodic =org-id-update-id-locations= over that enumeration — verified by clicking a known id link. The Phase 4 note to .emacs.d carries this ask; the =rg= recipe is the fallback for non-Emacs consumers either way. + +** The =docs-lifecycle= rule (the generalization) + +A new =claude-rules/docs-lifecycle.md= captures the reusable shape, with spec-create as the first instance: + +1. Separate formal artifacts from working notes by location. +2. Lifecycle state lives *in* the artifact, on a scannable, greppable carrier (an org keyword heading), with a dated history. +3. Links use rename-safe identifiers. +4. A growing collection earns this treatment when "which of these are live?" starts requiring a file-by-file read. + +** The retrofit (reach mechanism for existing piles) + +A synced helper, =spec-sort=, run once per project. *Canonical placement:* like every synced asset, the helper and all workflow edits land in rulesets' canonical tree first — =claude-templates/.ai/scripts/spec-sort= with its bats tests in =claude-templates/.ai/scripts/tests/= (the glob-discovered suite), workflow changes in =claude-templates/.ai/workflows/= — then =scripts/sync-check.sh --fix= propagates the committed =.ai/= mirror and both sides commit together. A mirror-only edit is reverted by the next sync; nothing in this feature is exempt from that contract. Downstream projects receive everything through the normal startup rsync. The run itself, per project: + +1. *Classify* each =docs/**/*.org= outside =docs/specs/= by one predicate: a doc carrying *both* a =Decisions= heading *and* an =Implementation phases= heading is a spec candidate; everything else is a note. (A =Metadata= table alone does not qualify — real counter-case: =docs/design/task-review.org= has a Metadata table and no spine, and is a note.) The heuristic *proposes*; a human confirms every move (classification is a judgment call — Craig, 2026-06-28). +2. *Move* confirmed specs to =docs/specs/=, *renaming to carry the =-spec.org= suffix* when the file lacks it (spec-review's Phase 0 precondition requires it — a retrofitted spec must be reviewable in its new home). Prepend the status heading, assign an =:ID:=, and rewrite the keyword header to the two-sequence form above. *The proposed keyword is evidence-based, not laundered:* the doc's own Status field is one signal among several, because stale Status fields are exactly what caused the original .emacs.d sweep. For each candidate the helper shows an evidence panel — the current Status value, the decision/finding cookie states, the state and heading of any =todo.org= task that links or binds to the doc, the most recent history/review entries, and (where cheap) whether artifacts the phases name actually exist — and proposes the keyword the evidence supports. When the evidence is inconclusive, the default is the most conservative *non-terminal* state it supports (never a terminal one). =IMPLEMENTED= / =SUPERSEDED= / =CANCELLED= are never applied without an explicit human-stated reason, recorded in the status-history line. +3. *Relink* under an explicit contract: + - *Rewritten roots (project-owned):* =todo.org=, =.ai/notes.org=, =docs/**=, =.ai/project-workflows/=, =.ai/project-scripts/=. The rewrite recomputes each link's relative path from the linking file's directory to the new location. *All rewrites stay =file:= links* — conversion to =[[id:...]]= is the separate follow-up pass gated on the Emacs id-index mechanism (see Rename-safe links), never part of a sort run. + - *Reported, never rewritten:* =.ai/sessions/= archives (frozen history), git history, and synced template paths (=.ai/workflows/=, =.ai/scripts/=, =.ai/protocols.org=) — a downstream edit there is reverted by the next template sync, so the report names the canonical rulesets file that needs the edit instead. + - *Supported link shapes:* org =[[file:...]]= links, relative or project-root-anchored, with or without a description. Bare-path mentions in prose or scripts are *reported for manual handling*, never rewritten. + - *Safety:* dry-run report is the default; =--apply= writes, under a fail-safe contract sized to the fact that one run mutates filenames, links, headers, and =.ai/notes.org= together: + - *Clean-worktree preflight.* =--apply= refuses on a dirty git tree (=git status --porcelain= non-empty) unless =--allow-dirty= is passed, which prints exactly what recovery loses. A clean tree is what makes recovery trivially safe. + - *Validate, then write.* The full move + relink plan — every source, destination, and link edit — is computed and validated first (every link parses, every target is unambiguous, every destination path is free), written to a plan file for inspection, and only then executed from that recorded plan. Ambiguous cases (two candidates sharing a basename, an unparseable link) block validation: listed, untouched, non-zero exit until each is resolved or explicitly waived. + - *Failure mid-apply is not a shrug.* Any write failure or a failed post-apply residue grep stops the run, names what was and wasn't applied (from the plan), and prints the recovery recipe — =git restore= over the plan's touched paths *plus* deletion of the plan's newly-created destination paths (=git restore= reverts tracked edits but doesn't remove untracked copies the move created). Safe by construction because preflight required a clean tree; the project is never silently left half-migrated. + - After a successful apply, the residue grep for each old path across the rewritten roots must return zero or =spec-sort= exits non-zero naming the residue. +4. *Stamp* =:LAST_SPEC_SORT: YYYY-MM-DD= in =.ai/notes.org='s =* Workflow State= section — the same surface as =:LAST_AUDIT:= and =:LAST_INBOX_PROCESS:=, created idempotently (append the section if the file lacks it) exactly as task-audit already does. + +*The startup nudge — concrete contract.* Phase A's parallel batch gains one read-only probe: + +#+begin_src bash +{ [ -d docs/design ] || [ -n "$(find docs -maxdepth 1 -name '*-spec.org' -print -quit 2>/dev/null)" ]; } \ + && ! grep -qs ':LAST_SPEC_SORT:' .ai/notes.org \ + && echo "spec-sort: unsorted docs present" || true +#+end_src + +(Phase 4 refined the stray-root check from =compgen= to =find=: =compgen= is bash-only and zsh aborts on an unmatched glob, so the original snippet false-negatived on stray root specs under zsh.) + +(The probe also fires on stray =docs/*-spec.org= root files, so a project whose only misfiled specs sit at the =docs/= root still gets nudged.) + +Phase C surfaces one line when the probe printed ("this project's docs pile has never been spec-sorted — say 'run spec-sort' to sort it") and stays silent otherwise. Projects with nothing to sort — no =docs/design/= and no stray root specs — never see it; a stamped marker permanently clears it. + +* Alternatives Considered + +** Filename status suffix (=-spec-doing.org=, =-spec-implemented.org=) +- Good, because the state is visible in a bare =ls= with no tooling. +- Bad, because every transition renames a file in a cross-linked, template-synced doc set — each rename is link surgery or a broken link, and the churn lands in git history and inbound =todo.org= links. +- Neutral, because the ls-visibility it buys is matched by the one-line =rg= over status headings. +- Rejected 2026-06-28 (Craig chose org-keyword over his earlier filename-suffix lean). + +** Status field in the Metadata table only (no keyword) +- Good, because the field already exists and needs no new structure. +- Bad, because a table cell is neither org-agenda-scannable nor reliably greppable across format drift, and it carries no dated history. +- Neutral, because the field stays anyway — as the in-table mirror. + +** Relink-helper instead of org-id (keep =file:= links, fix them on every move) +- Good, because readers see plain paths. +- Bad, because it makes every future move a tooling event, and one missed run silently breaks links — the failure mode is invisible until someone clicks. +- Neutral, because the retrofit needs relink logic once regardless; org-id just makes it a one-time need. + +* Decisions [5/5] + +All five were settled with Craig on 2026-06-28 (recorded in todo.org; migrated here per that note). + +** DONE Location split — adopt +- Context: specs and notes share one directory; telling them apart requires opening files. +- Decision: =docs/specs/= for formal specs (Decisions + phases spine); =docs/design/= for notes. Documented in spec-create and the docs-lifecycle rule. +- Consequences: easier — a listing answers "what's formal"; harder — one-time migration and link updates (the retrofit). + +** DONE Status mechanism — org keyword authoritative, no filename suffix +- Context: filename suffix vs org keyword; suffix wins =ls= visibility, keyword wins link stability and zero-rename transitions. +- Decision: the org TODO keyword on the spec's top status heading is authoritative, mirrored by the Metadata =Status= field. No status suffixes in filenames. +- Consequences: easier — a transition is one keyword edit and links never break; harder — glanceability needs the one-line =rg= (or the vNext agenda view) instead of bare =ls=. +- (Refined in review, 2026-07-01: "one keyword edit" became "three lines in one file" — keyword + history line + Metadata mirror. The ratified decision stands; see Review findings.) + +** DONE Link safety — org-id for cross-doc spec links +- Context: both the migration move and any future rename break =file:= links. +- Decision: specs carry =:ID:= UUIDs on the status heading; cross-doc references use =[[id:...]]=. +- Consequences: easier — moves are free; harder — following a link outside org needs the =rg ':ID:'= lookup. +- (Refined in review, 2026-07-01: the decision stands; the *sequencing* is staged — IDs are assigned at sort time, but link conversion to =id:= waits for the executable Emacs id-index mechanism, so no window exists where converted links don't click. See Review findings.) + +** DONE Generalize as a =docs-lifecycle= rule +- Context: the shape (in-artifact lifecycle state, formal-vs-notes split, rename-safe links) recurs for any processed-document collection. +- Decision: capture it in =claude-rules/docs-lifecycle.md= with spec-create as the first instance. +- Consequences: easier — the next collection reuses a decided pattern; harder — the rule must stay honest as the spec instance evolves. + +** DONE Retrofit existing files across ALL projects +- Context: template sync distributes conventions but cannot perform a per-project one-time migration; legacy piles would stay misfiled forever. +- Decision: ship a confirmed classify-move-relink helper (=spec-sort=) plus a startup nudge gated on =:LAST_SPEC_SORT:=; the helper proposes, a human confirms. Pilot on rulesets first. +- Consequences: easier — every project converges without manual archaeology; harder — the helper needs real relink logic and tests, and classification stays a judgment call. + +* Review findings [14/14] +:PROPERTIES: +:ID: cc77a7f6-e4c3-488a-ac3b-e739420a5c2b +:END: + +Two independent reviews (Codex, 2026-07-01 22:22; a fresh-context Claude agent, 2026-07-01 22:25) converged on =Not ready= with the same worst finding. All nine findings were dispositioned accept and fixed in the responder pass below; each carries its response. + +** DONE Org TODO vocabulary drops decision and finding task states :blocking: +(Codex; the Claude reviewer found the same, adding that keywords must be unique across sequences so a naive two-line fix collides on =SUPERSEDED=/=CANCELLED=.) The spec's example header replaced the file-level keyword vocabulary, so =TODO=/=DONE= stopped being task states and the =[/]= cookies that gate readiness went vacuous — this file itself was the first casualty. +Response: the scheme is now two collision-free sequences — =TODO | DONE= for decisions/findings, =DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED= for lifecycle (the old header's =SUPERSEDED CANCELLED= done-states migrate to the lifecycle sequence, and a legacy =CANCELLED= decision still parses as a done-state, so cookies stay correct). This file's own header now carries both lines; the Design section documents the two-sequence rule and the retrofit rewrites legacy headers to it. New acceptance criterion: cookies must compute by org, not hand counting. + +** DONE Relink behavior is too vague for a safe migration :blocking: +(Codex; the Claude reviewer independently flagged the synced-.ai/ slice — a downstream rewrite there is reverted by the next template sync, e.g. =startup.org:154='s reference to a spec candidate.) The retrofit named no scan scope, link-shape list, rewrite rule, residue policy, or dry-run format — the implementer would have had to invent the migration's data-safety contract. +Response: the retrofit section now carries the explicit contract: rewritten roots (=todo.org=, =.ai/notes.org=, =docs/**=, project-owned =.ai/= dirs), reported-never-rewritten surfaces (=.ai/sessions/=, git history, synced template paths — with the canonical rulesets file named in the report), supported link shapes (org =file:= links; bare paths report-only), relative-path recomputation, dry-run default with =--apply=, post-apply residue grep gating exit status, and refuse-loudly on ambiguity. + +** DONE Sort marker and startup nudge do not name the actual state surface :blocking: +(Codex; the Claude reviewer rated the same gap minor — Codex's version was sharper: startup reads =.ai/notes.org=, not a root =notes.org=, and Workflow State may not exist.) +Response: the marker is pinned to =.ai/notes.org='s =* Workflow State= (the =:LAST_AUDIT:= / =:LAST_INBOX_PROCESS:= surface), created idempotently as task-audit already does; the Design section now spells the Phase A probe command, its exact fire condition, and the Phase C one-liner. + +** DONE Phase order can strand legacy specs behind the new review precondition :blocking: +(Codex; the Claude reviewer found the same at medium severity.) Hardening spec-review's path precondition in Phase 1 while piles stay unsorted until Phases 3-4 would make every legacy spec unreviewable in the gap. +Response: Phase 1 now carries the compatibility rule — legacy =-spec.org= locations stay reviewable (with a "run spec-sort" nudge) until the project stamps =:LAST_SPEC_SORT:=; the precondition hardens only after. Acceptance criterion 5 updated to match. + +** DONE No owner for the DOING → IMPLEMENTED flip :blocking: +(Claude reviewer.) spec-create owns =DRAFT= and spec-review owns =DRAFT= → =READY=, but implementation finishes outside the spec trio, and "a human edits it" is the exact mechanism whose failure produced this spec (.emacs.d's six shipped-but-unmarked specs). +Response: the Design section now has a transition-ownership table naming an owner for every flip. =READY= → =DOING= belongs to spec-response; =DOING= → =IMPLEMENTED= is a tracked obligation — spec-response's phase-to-task breakdown always emits a final "flip the spec" task — with task-audit's reconcile pass as the safety net (flag any =DOING= spec whose implementation tasks are all closed). Phase 1 includes both workflow edits. + +** DONE Classification heuristic is precedence-ambiguous +(Claude reviewer.) "Decisions plus phases or Metadata table" reads two ways, and =docs/design/task-review.org= (Metadata table, no spine) classifies differently under each. +Response: one predicate now — spec candidate iff the doc carries *both* a =Decisions= heading *and* an =Implementation phases= heading; a Metadata table alone does not qualify. The task-review.org counter-case is cited in the retrofit step. + +** DONE spec-sort never renames moved files to the -spec.org suffix +(Claude reviewer.) spec-review's Phase 0 hard-requires the suffix, so a retrofitted legacy spec without it would be unreviewable in its new home. +Response: retrofit step 2 now renames moved files to carry =-spec.org= when they lack it; the relink pass covers the rename like any move. Acceptance criterion 3 checks the suffix on the re-homed root specs. + +** DONE Clicked id: links won't resolve in Craig's Emacs +(Claude reviewer.) =org-id-locations= indexes only agenda and visited files, so fresh =:ID:=s in =docs/specs/= are invisible-until-clicked broken — the convention would trade visible link breakage for invisible breakage. +Response: named as an explicit .emacs.d-side prerequisite in the Rename-safe-links section (=org-id-extra-files= over =docs/specs/= globs, or periodic =org-id-update-id-locations=), carried in the Phase 4 note to .emacs.d, with the =rg= recipe as the interim fallback. + +** DONE Acceptance criterion 2 contradicts the Metadata Status mirror +(Claude reviewer.) "Exactly one keyword edit" was irreconcilable with the mandated mirror update. +Response: a transition is now defined everywhere as three lines in one file — keyword, history line, mirror — still no rename and no link edits. Goals, Design, and criterion 2 all say the same thing. + +** DONE Synced helper placement ignores the canonical/mirror split :blocking: +The spec says to build =.ai/scripts/spec-sort= and update =.ai/workflows/= behavior, but rulesets' current contract is that =claude-templates/.ai/= is canonical and the repo-root =.ai/= tree is only the committed mirror kept honest by =scripts/sync-check.sh=. =CLAUDE.md= explicitly warns that mirror-only edits get silently reverted by the next sync, and =make test= runs the mirror-side tests only after the canonical copy has been synced. V1 should say every shared workflow/script edit lands in =claude-templates/.ai/{workflows,scripts}/= first, then =scripts/sync-check.sh --fix= updates the mirror; =spec-sort= tests should be placed in the synced script-test tree and the acceptance criteria should include =sync-check= / workflow-integrity where relevant. (blocking) +Response: the retrofit section now opens with the canonical-placement contract (helper + tests in =claude-templates/.ai/scripts{,/tests}/=, workflow edits canonical-side, =sync-check --fix= propagates, both sides commit together); Phases 1 and 2 name it per artifact; new acceptance criterion requires =sync-check= to exit clean after the build commits. + +** DONE Task-audit safety net has no spec-to-task binding :blocking: +The spec says task-audit flags a =DOING= spec whose implementation tasks are all closed, but current =task-audit.org= audits open =todo.org= tasks and has no model for scanning =docs/specs/=, finding a spec's implementation tasks, or deciding "all closed" after =todo-cleanup.el --convert-subtasks= rewrites completed child tasks into dated entries. The added final "flip to IMPLEMENTED" task also means there may always be one open task, so a naive "all tasks closed" check never fires. V1 should define the binding spec-response writes into =todo.org= (for example a parent task property or stable link to the spec ID), the exact audit query, how converted dated entries count, and whether the final flip task is excluded from or satisfies the reconciliation rule. (blocking) +Response: spec-response's decomposition now stamps a =:SPEC_ID:= property (the spec's status-heading UUID) on the build parent task — the durable binding. The audit query is defined: for each =DOING= spec, find the task with matching =:SPEC_ID:=; flag when that parent is closed, archived, or missing. Checking the parent's keyword (not "all children closed") dissolves both the flip-task chicken-and-egg and the dated-entry conversion concern. New acceptance criterion exercises the flag. + +** DONE spec-sort apply path can leave a half-migrated tree :blocking: +The retrofit contract has dry-run by default and a post-apply residue grep, but it does not say what happens when =--apply= has moved files and then a relink, parse, or residue check fails. Because the operation mutates filenames, links, headers, IDs, and =.ai/notes.org= together, a partial failure can strand the project in the exact mixed state the tool is meant to prevent. V1 should require a clean-worktree preflight (or an explicit dirty-tree refusal/override), validate the full move/relink plan before the first write, write from a single recorded plan, and define recovery behavior for every failed apply: no files moved, automatic rollback, or a printed =git restore= / =git revert= recovery recipe that is safe for uncommitted local edits. (blocking) +Response: the relink contract's safety block now specifies the fail-safe apply: clean-worktree preflight (refuse on dirty, explicit =--allow-dirty= override that prints what recovery loses), full plan computed + validated + written to a plan file before the first write, execution from the recorded plan, and mid-apply failure stopping with a named applied/not-applied breakdown plus the =git restore= recovery recipe — safe by construction because preflight required a clean tree. Bats covers the preflight and the forced-failure recovery output (Phase 2, plus a new acceptance criterion). + +** DONE org-id Emacs prerequisite is not executable as written :blocking: +The spec says the .emacs.d-side fix can be =org-id-extra-files= over =docs/specs/= globs, but Emacs' own docstring says =org-id-extra-files= is a list of additional files and is only relevant when =org-id-track-globally= is set; it does not establish that project glob strings will be expanded or that every project root will be discovered. The rollout also converts links during the rulesets pilot before the Phase 4 note asks .emacs.d to make clicked =id:= links resolvable. V1 should either keep =file:= links until the Emacs support has landed, or specify the executable Emacs-side implementation precisely: how project =docs/specs/*.org= files are enumerated into =org-id-extra-files= or fed to =org-id-update-id-locations=, when it runs, how it is tested, and how rollout avoids a window where converted links do not click through. (blocking) +Response: took the fork that removes the window entirely — the pilot and every sort run rewrite =file:= links only; =:ID:= properties are still assigned (harmless, enables later mechanics); conversion to =id:= is a separate follow-up pass gated on .emacs.d landing an executable id-index mechanism, now specified concretely (enumerate =docs/specs/*.org= into =org-id-extra-files= as real file names under =org-id-track-globally=, or feed the enumeration to =org-id-update-id-locations=; verified by clicking a known link). Decision 3 carries a sequencing-refinement note; a new acceptance criterion asserts zero =id:= links exist after the pilot. + +** DONE Status confirmation can still encode stale reality :blocking: +The retrofit proposes lifecycle status from a doc's current =Status= field or review history, then asks a human to confirm. Those are the same stale/incomplete signals that caused the original .emacs.d sweep: shipped specs and dead specs were only knowable by reading code/tasks against the spec. If =spec-sort= only confirms a guessed keyword, the pilot can produce a clean-looking board whose state is still wrong. V1 should define status-confirmation evidence: for each spec candidate, what sources the helper shows (current Status, decision/finding cookies, linked =todo.org= parent state, recent history, matching implementation files/tests), what default is allowed when evidence is inconclusive, and that =IMPLEMENTED= / =SUPERSEDED= / =CANCELLED= require an explicit reason in the status history line. (blocking) +Response: retrofit step 2 now defines the evidence panel the helper shows per candidate (Status value, cookie states, bound/linking =todo.org= task state, recent history entries, cheap existence checks on phase-named artifacts) with the keyword proposed from the evidence, not the Status field alone. Inconclusive evidence defaults to the most conservative non-terminal state; =IMPLEMENTED= / =SUPERSEDED= / =CANCELLED= always require an explicit human-stated reason recorded in the history line. + +* Implementation phases + +** Phase 1 — Rule + template updates +Write =claude-rules/docs-lifecycle.md=. Update spec-create (emit into =docs/specs/=, the two-sequence keyword header, status heading with =:ID:= in the template, transition mechanics), spec-review (path expectation with the compatibility rule below; flipping =DRAFT= → =READY= on a passing review updates keyword + history + mirror), spec-response (owns =READY= → =DOING=; its decomposition stamps the =:SPEC_ID:= binding on the build parent and always emits the final "flip to IMPLEMENTED" task), and task-audit (one reconcile bullet running the =:SPEC_ID:= query: a =DOING= spec whose bound parent is closed, archived, or missing gets flagged). All four are synced assets: edits land in =claude-templates/.ai/= (and =claude-rules/=), the mirror follows via =sync-check --fix=, both commit together. *Compatibility rule:* spec-review keeps accepting legacy =-spec.org= locations (=docs/= root, =docs/design/=) until the project's =:LAST_SPEC_SORT:= is stamped, nudging "run spec-sort" when it meets one; only after the stamp does the =docs/specs/= precondition harden. No legacy spec is ever unreviewable during the transition. Tree stays working: new specs land in the new shape; old specs remain reviewable until their project sorts. + +** Phase 2 — The =spec-sort= helper +Build =claude-templates/.ai/scripts/spec-sort= (classify → evidence-based confirm → plan + validate → move + rename + prepend status heading + assign =:ID:= → relink =file:= references → stamp =:LAST_SPEC_SORT:=), with bats coverage in =claude-templates/.ai/scripts/tests/= (glob-discovered by =make test=) for classification, the evidence/confirm gate, plan validation, moving + renaming, relinking, the clean-worktree preflight, mid-apply failure recovery output, idempotence, and the marker stamp. Mirror synced via =sync-check --fix= in the same commit. Tree stays working: the script is callable but nothing invokes it yet. + +** Phase 3 — Pilot on rulesets +Run =spec-sort= against rulesets' own =docs/= (41 design files, 3 spec-spine candidates, 2 stray root specs). Fix what the pilot surfaces before any other project runs it. Tree stays working: moves are confirmed one by one, links updated in the same pass. + +** Phase 4 — Startup nudge + broadcast +Add the Phase A probe + Phase C nudge line (the concrete contract in the Design retrofit section). Send .emacs.d a note that the convention is live, its ~28-doc pile is ready to sort, and the id-index mechanism is its side of the staged link conversion: enumerate each project's =docs/specs/*.org= into =org-id-extra-files= as real file names (with =org-id-track-globally= t) or feed that enumeration to a periodic =org-id-update-id-locations=, verified by clicking a known id link. The =id:= link-conversion pass across projects runs only after that lands — it is follow-up work, not part of v1's sort runs. Tree stays working: the nudge is one read-only line per session until acted on; every link is a working =file:= link until conversion day. + +* Acceptance criteria +- [ ] =rg '^\* (DRAFT|READY|DOING|IMPLEMENTED|SUPERSEDED|CANCELLED) ' docs/specs/= lists every rulesets spec with its state, and the answer matches reality. +- [ ] A status transition on a spec changes exactly three lines in one file — the keyword, a history line, and the Metadata mirror — with no rename and no link edits. +- [ ] Every doc remaining in rulesets =docs/design/= is a note (lacks the Decisions + Implementation-phases spine); both stray =docs/= root specs are re-homed and carry the =-spec.org= suffix. +- [ ] All inbound links in the rewritten roots resolve after the pilot, and the post-apply residue grep returns zero. +- [ ] The spec's own decision/finding =[/]= cookies compute correctly under the two-sequence keyword header (org, not hand counting). +- [ ] spec-create emits new specs into =docs/specs/= in the new shape; spec-review accepts legacy locations until =:LAST_SPEC_SORT:= is stamped and refuses them after. +- [ ] Every helper/workflow artifact of this feature lives canonical-side (=claude-templates/.ai/=, =claude-rules/=) with the mirror in sync — =scripts/sync-check.sh= exits clean after the build commits. +- [ ] A =DOING= spec whose =:SPEC_ID:=-bound parent task is closed or missing is flagged by task-audit's reconcile pass (exercised in the pilot or a fixture). +- [ ] =spec-sort --apply= on a dirty worktree refuses (absent the override); a forced mid-apply failure in the bats suite yields the named-recovery output, not a half-migrated tree. +- [ ] After the pilot, no link the sort *rewrote* uses =[[id:...]]= form and no rewritten root gained a new =id:= link targeting a spec (conversion is the gated follow-up); every rewritten link is a resolving =file:= link. The check scopes to actual rewritten and spec-target links — literal prose mentions of the id syntax (which already exist in =todo.org= and older specs) don't count, so a naive whole-file grep is the wrong implementation. +- [ ] A project with an unsorted =docs/design/= gets the startup nudge; one confirmed =spec-sort= run clears it via =:LAST_SPEC_SORT:=. + +* Readiness dimensions +- Data model & ownership: the spec file owns its state; the Metadata mirror is display-only. No external index to drift. +- Errors, empty states & failure: =spec-sort= on a project with no =docs/= is a silent no-op; an ambiguous classification is surfaced, never auto-moved; a relink pass that finds zero inbound links is normal. +- Security & privacy: N/A because the docs are already in-repo; no new exposure surface. +- Observability: the status grep is the dashboard; =spec-sort= prints every proposed move and every rewritten link. +- Performance & scale: N/A because collections are tens of files; everything is one-shot or grep-speed. +- Reuse & lost opportunities: reuses org TODO keywords, org-id, the existing scheme-header pattern of declared vocabularies, and spec-review's in-file findings convention. +- Architecture fit & weak points: weak point is the classification heuristic — mitigated by the confirm gate. The status heading is additive, so old readers of spec files see one extra heading and nothing breaks. +- Config surface: none new — one marker line (=:LAST_SPEC_SORT:=) in the existing Workflow State section. +- Documentation plan: the docs-lifecycle rule is the documentation; spec-create's template is the worked example. +- Dev tooling: =spec-sort= ships with bats tests under the existing glob-discovered suite. +- Rollout, compatibility & rollback: additive per project, one project at a time, rulesets first. Rollback of a sort is =git revert= of the pilot commit (moves + relinks are one commit). +- External APIs & deps: N/A — plain files, =rg=, =uuidgen=. + +* Risks, Rabbit Holes, and Drawbacks +- *Relink misses an inbound link shape* (org radio links, bare paths in scripts). Dodge: the pilot greps for the old path after moving and fails loudly on any residue. +- *Heuristic over-classifies notes as specs.* Dodge: the confirm gate is mandatory; the helper never moves unconfirmed. +- *Keyword vocabulary drift* between this spec, the rule, and spec-create's template. Dodge: the rule names the vocabulary once and the others link it. + +* Testing / Verification / Rollout +bats for =spec-sort= (classification, the evidence/confirm gate, plan validation, move + rename, relink, the clean-worktree preflight, forced mid-apply failure recovery output, idempotence, marker stamp). The pilot run on rulesets is the live verification; the post-move residue grep is the acceptance check. Rollout is per-project via the startup nudge, each run human-confirmed. + +* References / Appendix +- Source proposal: [[file:../design/2026-06-15-spec-storage-lifecycle-proposal.org]] (.emacs.d handoff, 2026-06-15). +- Decisions record: todo.org "Spec storage location + lifecycle-status convention" (settled 2026-06-28). +- This file is the convention's first resident: it lives in =docs/specs/=, carries the status heading + =:ID:=, and drops the status filename suffix. + +* Review and iteration history +** 2026-07-01 Wed @ 22:13:00 -0400 — Claude — author +- What: initial draft, written from the five pre-ratified decisions. +- Why: the queued-specs half of the 2026-06-30 session goal; decisions were settled 2026-06-28 and needed migration into a buildable spec. +- Artifacts: todo.org task "Spec storage location + lifecycle-status convention"; source proposal above. + +** 2026-07-01 Wed @ 22:22:34 -0400 — Codex — reviewer +- What changed or was recommended: rubric =Not ready=. Four blocking findings were added: preserve Org task keywords while adding lifecycle status, make =spec-sort= relinking executable and failure-safe, define the actual =.ai/notes.org= marker/startup-nudge contract, and avoid stranding legacy specs behind a stricter path precondition before retrofit. +- Why: current rulesets workflows still depend on =TODO= / =DONE= decision and finding tasks, startup state lives in =.ai/notes.org=, and the repo still contains formal specs outside =docs/specs/= until the migration runs. +- Artifacts: Review findings section; current-state checks against =.ai/workflows/spec-create.org=, =.ai/workflows/spec-review.org=, =.ai/workflows/startup.org=, =scripts/sync-check.sh=, and =todo.org=. + +** 2026-07-01 Wed @ 22:25:00 -0400 — Claude (fresh-context agent) — reviewer +- What: rubric =Not ready=. Independently found Codex's keyword-vocabulary blocker (adding the cross-sequence uniqueness wrinkle) and the stranded-legacy-specs and marker-surface gaps, plus five findings of its own: no owner for the =DOING= → =IMPLEMENTED= flip (blocking), the precedence-ambiguous classification heuristic, the missing =-spec.org= rename in spec-sort, org-id click-resolution in a live Emacs, and the criterion-2/mirror contradiction. +- Why: fresh-eyes adversarial pass requested by Craig after his own read found nothing; the two reviews converging on the same worst bug from independent context is the confidence signal. +- Artifacts: Review findings section (findings 5-9); spot-checks against real repo files (=docs/design/task-review.org=, the two stray root specs, =startup.org:154=). + +** 2026-07-01 Wed @ 22:46:52 -0400 — Claude — second responder pass +- What: fixed all five of Codex's re-review findings in place (fourteen of fourteen closed): canonical-placement contract for every synced artifact (+ sync-check acceptance criterion), the =:SPEC_ID:= spec-to-task binding with the parent-keyword audit query (dissolving the flip-task chicken-and-egg), the fail-safe =--apply= contract (clean-tree preflight, validate-then-write from a recorded plan, named recovery), staged id-link conversion (pilot rewrites =file:= links only; =id:= conversion gated on the concrete .emacs.d id-index mechanism — the fork Craig approved), and evidence-based status confirmation (evidence panel, conservative non-terminal default, reasons required for terminal states). Status stays DRAFT; the READY flip belongs to the reviewers this round. +- Why: Craig approved fixing all five ("1", 2026-07-01), including the keep-file:-links-through-pilot fork. +- Artifacts: per-finding responses inline; the fixed Design/phase/criteria sections. + +** 2026-07-01 Wed @ 23:22:50 -0400 — Codex — reviewer +- What changed or was recommended: rubric =Ready=. No new blocking findings. The second responder pass closed all five Codex re-review blockers without regressing the first nine findings, and the spec now gives implementers concrete contracts for canonical synced assets, =:SPEC_ID:= task binding, fail-safe =spec-sort --apply= behavior, staged id-link conversion, evidence-based status confirmation, phase sequencing, and test coverage. +- Why: the current spec can be implemented and tested without hidden product decisions; remaining vNext work is separately tracked. +- Artifacts: status heading flipped to =READY=; =* Decisions= [5/5]; =* Review findings= [14/14]; Emacs batch cookie check. + +** 2026-07-01 Wed @ 22:41:21 -0400 — Claude (fresh-context agent) — verify pass; Claude — READY flip +- What: the original reviewer re-read the fixed spec against its own nine findings: all held, none regressed, verdict ready. It re-ran the classification predicate live (exactly 5 candidates; task-review.org excluded) and confirmed org computes the cookies. Two non-blocking minors folded in before the flip: a refinement note under Decision 2 (whose frozen body still said "one keyword edit") and a wider nudge probe that also fires on stray =docs/*-spec.org= root files. Status flipped DRAFT → READY. +- Why: Craig authorized the flip contingent on the verify pass clearing; it did. +- Artifacts: the status heading's history line; verify-pass report in the session record. + +** 2026-07-01 Wed @ 22:41:33 -0400 — Codex — reviewer +- What changed or was recommended: rubric =Not ready=. Five new blocking findings were added after the response pass: make shared workflow/script edits obey the =claude-templates/.ai/= canonical plus =.ai/= mirror contract; define how task-audit binds a =DOING= spec to its implementation tasks; make =spec-sort --apply= failure-safe; turn the org-id Emacs prerequisite into an executable rollout step; and require status confirmation to be evidence-based rather than a rubber-stamp of stale fields. +- Why: the response fixed the original keyword/relink/precondition issues but introduced new integration points in synced template assets, task-audit, Emacs id resolution, and migration safety that are not yet buildable from the spec. +- Artifacts: Review findings section; checks against =CLAUDE.md=, =scripts/sync-check.sh=, =.ai/workflows/task-audit.org=, =.ai/workflows/startup.org=, =.ai/notes.org=, current =docs/= inventory, and Emacs batch/docstring checks for Org TODO cookies and =org-id-extra-files=. + +** 2026-07-01 Wed @ 22:30:06 -0400 — Claude — responder +- What: merged both reviews into one findings ledger (nine findings, all dispositioned accept) and fixed all nine in place: two-sequence keyword header (applied to this file itself), transition-ownership table with the tracked flip-to-IMPLEMENTED task, single classification predicate, the -spec.org rename step, the full relink data-safety contract, the =.ai/notes.org= marker + Phase A/C startup contract, the legacy-location compatibility rule, the org-id Emacs prerequisite, and the three-line transition definition. Acceptance criteria updated to match. +- Why: Craig approved fixing all nine ("1", 2026-07-01); none touched the five ratified decisions. +- Artifacts: Review findings section (responses inline per finding); the fixed sections themselves. diff --git a/docs/specs/2026-07-14-sentry-workflow-spec.org b/docs/specs/2026-07-14-sentry-workflow-spec.org new file mode 100644 index 0000000..307d0be --- /dev/null +++ b/docs/specs/2026-07-14-sentry-workflow-spec.org @@ -0,0 +1,278 @@ +#+TITLE: Sentry Workflow — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-14 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED sentry workflow +:PROPERTIES: +:ID: f6c51f27-d7a2-4b63-9ff9-5ba005a66dfb +:END: +- 2026-07-19 Sun @ 05:04:00 -0500 — IMPLEMENTED: all four phases built, committed, and pushed (agent-lock a8b6cf4, engine ccc9c26, companions c6383e9). Full suite green throughout. The overnight live trial is handed to Craig as a structured manual-testing task; its findings file as follow-up tasks, not a build gate. +- 2026-07-19 Sun @ 04:35:57 -0500 — DOING: build started. Decomposed into the four implementation phases as todo.org build tasks under the sentry parent (SPEC_ID-bound); running in no-approvals + auto-flush mode. +- 2026-07-19 Sun @ 04:35:57 -0500 — READY: Craig completed his deep read and approved the spec for build. Gate passed. +- 2026-07-14 Tue @ 02:03:28 -0500 — all 12 review findings dispositioned live with Craig and folded into the design; decisions now 10/10; still DRAFT pending Craig's deep read. +- 2026-07-14 Tue @ 00:52:03 -0500 — drafted, all nine design decisions resolved live with Craig during the authoring session. + +* Metadata +| Status | implemented | +|----------+------------------------------------------| +| Owner | Craig Jennings | +|----------+------------------------------------------| +| Reviewer | (spec-review, next session) | +|----------+------------------------------------------| +| Related | [[file:../../todo.org][todo.org — sentry build task]] | +|----------+------------------------------------------| + +* Summary + +Sentry is one supervisor workflow that runs a project's hygiene passes serially on a cadence, holds locks so it never collides with itself or other agents, commits its work to a host-suffixed side branch for morning review, and checkpoints between passes so a crash loses at most one pass. It replaces the idea of firing inbox-zero and triage-intake as separate uncoordinated crons. + +* Problem / Context + +An agent session left running overnight can keep a project clean, but the pieces don't coordinate. There is no lock anywhere in .ai/scripts/ (verified 2026-07-14): capture-guard covers only the Emacs org-capture case on one file, and it's advisory. Multiple sessions (several projects, sometimes both daily drivers) can read-modify-write ~/org/roam/inbox.org and the agents/ KB nodes at once. The only coordination is git optimism, and we know it loses: the KB search already globs out *sync-conflict* files because pull/push races fork them. + +Separate crons compound it: inbox-zero and triage-intake fired independently can overlap each other and any interactive session on the same shared files. + +Proposal origin: the work project, 2026-07-13 (preserved at [[file:../design/2026-07-14-sentry-workflow-proposal.org][docs/design/2026-07-14-sentry-workflow-proposal.org]]). This spec supersedes the proposal where they differ: the skeptical review found four conflicts with existing discipline, each resolved as a Decision below. + +* Goals and Non-Goals + +** Goals +- One serialized runner per project: passes never overlap each other, and a fire that finds a prior fire still running skips. +- A shared-file lock that actually serializes same-host agents writing public roam files. +- Crash-lossless to one-pass granularity: session-context entry plus a commit after every disk-writing pass. +- Main stays clean: overnight commits land on a host-suffixed sentry branch so cross-machine pulls and the template sync keep working. +- Nothing destructive fires unattended: trash/mark-read, regrades, and file moves queue for morning approval. +- Portable across projects with zero configuration: passes detect their targets and skip where not applicable. + +** Non-Goals +- No report-only or degraded pass mode. A pass runs fully or it doesn't run (Craig's explicit direction). The morning-approval queue is a permanent division of labor, not a degraded mode. +- No cross-host locking. The lock is host-local; cross-host races stay with roam-sync's abort-loudly rebase. +- No unattended /schedule contract. Sentry runs inside a live session Craig starts, driven by /loop, the same boundary inbox.org draws for auto inbox zero. +- Sentry never runs git against the roam repo. roam-sync remains that repo's only committer. + +** Scope tiers +- v1: the sentry.org engine (ten mechanical passes, entry gates, branch mechanics, digest, the stop-sentry operation), the agent-lock helper script with bats tests, the roam-write lock adopted by every roam write path, companion-file reconciliations (knowledge-base.md write recipe, inbox.org core §5, roam-sync.sh header, triage-intake.org note, wrap-it-up.org active-sentry guard), INDEX.org entry. +- Out of scope: changes to roam-sync.sh behavior; changes to the wrap-up teardown feature. +- vNext (logged to todo.org): cross-host roam conflict surfacing; the KB lesson-promotion pass (blocked on the lesson-detection heuristic task); a fully-unattended /schedule variant if the interactive shape proves too narrow. + +* Design + +** For Craig (the user altitude) + +You type the trigger in a project session ("run sentry", or the /loop line it expands to). Sentry first checks its entry ticket: the project's notes.org must carry :COMMIT_AUTONOMY: yes, or sentry declines to start and names the marker. Then the entry gates run while you're still at the terminal: a dirty tracked tree stops and describes what's dirty, file by file, and offers numbered options (finish the job, stash it, or name changes to roll back); a red test suite stops the same way with an offer to investigate. The loop starts only once both are answered. + +Launching is a handoff — the launch contract: sentry owns the repo until the morning merge. Working in that repo mid-night means saying "stop sentry" first (it cancels the loop and walks the branch and queue disposition), and open Emacs buffers on project files want a revert after launch and again after the merge. "Wrap it up" while sentry is active refuses and points at "stop sentry". + +Each fire (hourly by default) walks the pass list serially and appends one timestamped digest to the session log: failures as banners on top, then one line per pass, then the morning-approval queue. A pass with nothing to do is one line. Anything destructive or judgment-shaped waits in that queue for you. + +In the morning you review the sentry/<date>-<host> branch (a stack of small unpushed chore(sentry) commits), squash-merge it into your working branch, delete it, and answer the approval queue. If a night's branch is still unmerged when sentry next starts, that fire skips and says so. + +** For the implementer + +*The engine.* A synced template workflow, .ai/workflows/sentry.org, driven by /loop <interval> (default hourly). Each fire acquires the single-runner lock, verifies branch state, walks the pass list, writes the digest, commits the spine residue, and releases the lock. The runner re-touches the lock between passes (the heartbeat), so a live run's lock is never older than one pass. Every pass follows one contract: probe, work, session-context entry, then a commit when the pass wrote to disk. The probe asks whether the pass's target exists in this project; when it doesn't, the pass is one skip line and nothing more. The session-context entry precedes the commit so a crash between them still leaves the trail. + +*Branch mechanics.* At loop start (after the entry gates and a fetch-and-ff-only reconcile of the project branch — a diverged branch joins the interactive gate), sentry creates sentry/YYYY-MM-DD-<host> from HEAD and checks it out; the host suffix prevents a same-date collision between the daily drivers. Commits are conventional, one per writing pass: chore(sentry): <pass> — <what changed>. Nothing pushes. At entry, an existing unmerged sentry/* branch means skip-and-note. No second branch stacks on the first. Morning teardown (review, squash-merge, delete) is Craig's, documented in the workflow, never automated. + +*Locks.* One helper script, .ai/scripts/agent-lock, serves both locks. flock is unusable here: every Bash call an agent makes is its own short-lived shell, so a flock dies with the call that acquired it. Instead: atomic mkdir as acquire (create-and-win or fail-and-lose), a metadata file inside recording PID, hostname, and ISO timestamp, and age-based staleness reclaim so a crashed run's lock expires instead of wedging every later fire. Subcommands: acquire <name> [--ttl], release <name>, status <name>. Lock homes: /run/user/<uid>/agent-locks/<name>/ for both locks — the helper owns the path scheme, callers pass names — with ~/.cache/agent-locks/ as the fallback where no runtime dir exists. tmpfs makes a lock host-local by construction, keeps it out of every repo (a lock inside ~/org/roam would ride roam-sync's git add -A to the other machine as a phantom hold), and clears it on reboot. Contention is a bounded wait (~30s), then defer-and-note. With the heartbeat keeping a live lock young, TTLs size to the longest single pass; every reclaim surfaces in the digest. + +*Roam writes.* A pass touching a public roam file acquires the roam-write lock, runs capture-guard --wait (the human-capture layer stays underneath), edits the working tree, triggers roam-sync with systemctl --user start roam-sync.service, and releases the lock. Sentry never runs a git write against ~/org/roam, and pass 1's ff-only pull is its only git read of that repo. The 06-24 one-git-owner rule holds. The lock spans only edit-plus-trigger; roam-sync itself is serialized by systemd (a oneshot unit never runs concurrently with itself). + +*Unattended safety.* With no one at the terminal, any unsafe state (unexpected dirty tree, red suite, lost lock, unmerged prior branch) makes the affected scope (the pass, or the whole fire, whichever the state poisons) skip with one digest line. The next fire retries; if the state persists to morning, the interactive entry gate handles it with Craig present. Skips are never silent and never partial: no pass runs in a reduced form. Two refinements keep the machinery honest: the dirty check excludes the spine set (session-context files, path resolved via session-context-path) so sentry's own bookkeeping can't trip it, and after the second consecutive fire skipped on an unmerged prior branch, sentry sends one persistent desktop notification naming the project and branch, then repeats at most daily — a multi-day stall never stays silent. + +*Pass list (v1), in order, each with its detection probe:* + +1. Roam pull — git -C ~/org/roam pull --ff-only, skipped when the tree is dirty (roam-sync owns that case) or the clone is absent. Read-only; the one narrow exception to "don't touch roam git," pull-only and ff-only, so later reads are fresh. +2. Inbox zero — inbox.org roam mode, run under the engine's no-approvals contract (quick+solo+agreed items execute; shared-asset proposals park; everything lands in the morning queue). Probe: roam clone or project inbox/ exists. +3. Triage intake — triage-intake.org. Probe: triage plugins present. Destructive actions queue. +4. Todo cleanup — clean-todo.org mechanics. Probe: root todo.org. +5. Task audit — task-audit.org. Probe: root todo.org. Regrades queue. +6. Working-files hygiene — flag working/<slug>/ dirs whose task is closed. Probe: working/ exists. Filing queues. +7. Spec status board — the docs-lifecycle grep. Probe: docs/specs/ exists. +8. Link integrity — broken file: links in the project's org files. Probe: lint-org.el present. +9. Git health — drift, unpushed commits, stale branches, main-behind-origin. Probe: .git. +10. Prep + symlink freshness. Probe: the prep dir / symlinks exist (work and home only, in practice). +(KB lesson promotion, the proposal's eleventh pass, is deferred to vNext — see the KB finding and the filed lesson-detection-heuristic task. v1 ships the ten mechanical passes above.) + +*Digest.* Appended per fire to the session-context.org Session Log (which the spine already writes), so it survives a crash, rides the session archive, and is on screen in the running session. The morning-approval queue accumulates under one heading in the same file — each item carries what, why, and the exact command or edit that fires on approval. + +* Alternatives Considered + +** Separate crons per task (status quo direction) +- Good, because each piece stays independently simple. +- Bad, because nothing serializes them: the observed sync-conflict forks are this cost, already paid. +- Bad, because N crons means N session lifecycles to manage instead of one. + +** flock for both locks (as proposed) +- Good, because it's the standard tool and kernel-enforced. +- Bad, because it binds to a living process; agent tool calls are short-lived shells and /loop turns share none, so the lock evaporates on return. Disqualifying. + +** Per-pass commits on the current branch (as proposed) +- Good, because commits sit where the edits apply; no morning merge step. +- Bad, because unpushed commits on main on two daily drivers diverge main by morning, the exact state startup's fast-forward refuses and the template-sync guard blocks on. Disqualifying with two machines. + +** Degraded report-only mode for unsafe states +- Good, because some findings still land when committing is unsafe. +- Bad, because it blurs the contract: a pass that half-ran reads as having run. Craig rejected it outright; skip-and-note is the replacement. + +** Per-project pass manifest instead of detection +- Good, because explicit control over what runs where. +- Bad, because it's one more thing to maintain and forget; detection activates a pass the day its target appears. Rejected for v1. + +* Decisions [10/10] + +** DONE Commit target: host-suffixed sentry branch +- Context: overnight commits must survive two daily drivers that both pull main; unpushed commits on main diverge it across machines, breaking startup fast-forward and the template-sync guard. +- Decision: We will commit each writing pass to sentry/YYYY-MM-DD-<host>, created from HEAD at loop start, never pushed. Morning flow: review, squash-merge, delete. An unmerged prior sentry branch at entry skips the fire. +- Consequences: easier — main's ref stays clean for cross-machine sync; a bad night is discarded by deleting one branch. Harder — a daily merge step; the working tree sits on a non-main branch overnight. + +** DONE Roam-write lock scope: host-local only +- Context: observed conflicts are same-host multi-agent; cross-host races go through roam-sync's rebase, which aborts loudly. +- Decision: We will ship the host-local lock and file cross-host conflict-surfacing as a separate task. +- Consequences: easier — small, testable v1. Harder — a true cross-host race still forks; we accept the rarity. + +** DONE Interval default: hourly, with a knob +- Context: passes short-circuit in seconds when idle; the real cost is digest noise, not compute. +- Decision: We will default to hourly and expose the interval as the /loop argument. +- Consequences: easier — projects stay clean through the night. Harder — a capture-heavy evening produces several small commits; the knob is the escape hatch. + +** DONE Entry gates are interactive, not silent +- Context: Craig types the sentry command in, so the first fire runs with him at the terminal. +- Decision: We will stop at entry on a dirty tracked tree (describe what's dirty; offer finish-the-job / stash / rollback-named-changes) and on a red suite (describe failures; offer to investigate). The loop starts only after both are answered. Untracked files never block. +- Consequences: easier — no guessing about his in-progress work. Harder — sentry can't start unattended from a dirty state; that's the point. + +** DONE No report-only mode; unsafe unattended states skip +- Context: a degraded pass mode blurs whether a pass ran; Craig rejected it explicitly. +- Decision: We will make every pass run fully or not at all. Unattended unsafe states (dirty tree, red suite, lost lock, unmerged prior branch) skip the poisoned scope with one digest line; the next fire retries. The morning-approval queue for destructive/judgment actions stays — it's a permanent contract, not degradation. +- Consequences: easier — a pass line in the digest means it fully ran. Harder — a persistent unsafe state means zero hygiene until morning; accepted. + +** DONE Roam git discipline: roam-sync stays the only committer +- Context: the proposal had sentry commit-and-push roam under the lock; the 06-24 fix made roam-sync the repo's single git owner because the tree is chronically dirty from live captures. +- Decision: We will wrap only edit-plus-trigger in the roam-write lock; sentry never runs git against ~/org/roam (pass 1's ff-only pull is the sole, read-only exception). +- Consequences: easier — one git owner, no mid-rebase states from agents. Harder — an edit lands remotely only when roam-sync fires; the manual trigger closes most of that gap. + +** DONE Lock mechanics: mkdir-atomic helper with staleness reclaim +- Context: flock can't span tool calls (short-lived shells, no shared process across /loop turns). +- Decision: We will build .ai/scripts/agent-lock — atomic mkdir acquire, PID/host/timestamp metadata, age-based reclaim, acquire/release/status subcommands, bats-tested — and use it for both locks. +- Consequences: easier — locks survive between calls and self-clear after crashes. Harder — TTL tuning; a reclaim during a genuinely slow pass is possible, so the helper surfaces every reclaim rather than reclaiming silently. + +** DONE Autonomy gate: :COMMIT_AUTONOMY: yes is the entry ticket +- Context: commits.md gates commits on approval; sentry commits unattended, so it needs standing, per-project authorization. +- Decision: We will have sentry decline to start in any project whose notes.org lacks :COMMIT_AUTONOMY: yes, naming the marker. +- Consequences: easier — running sentry somewhere is a deliberate grant; no half-running mode to reason about. Harder — read-only passes don't run in ungranted projects either; that's acceptable, since launch is one marker away. + +** DONE Pass portability: detection over configuration +- Context: several proposed passes are work/home-specific; run verbatim elsewhere they're vacuous or error. +- Decision: We will open every pass with a cheap existence probe and skip with one digest line when the target is absent. +- Consequences: easier — zero config, passes self-activate when targets appear. Harder — an intentionally-unwanted pass needs a vNext exclusion marker if that ever becomes real. + +** DONE Suite policy: entry run plus conditional fire-end run, no per-pass runs +- Context: the verification discipline requires a full suite run before every commit, but sentry commits per pass, hourly, mostly touching org files the suite doesn't exercise; a per-commit run would turn a seconds-long fire into minutes, all night. +- Decision: We will run the suite once at entry (the green baseline the gates require) and again at fire-end only when a pass modified files outside the org/spine set. No per-pass runs. The deviation is justified by the unpushed branch and the morning review gating everything before push. +- Consequences: easier — idle fires stay cheap and the rare code-touching pass is still caught before its commits age. Harder — a suite break introduced by an org-only edit (possible via fixtures) surfaces at morning review rather than at the offending commit. + +* Review findings [12/12] + +** DONE Overnight working-tree ownership is undefined +Resolved 2026-07-14 (Craig): launch contract with in-place checkout. Launching sentry hands the repo to sentry until the morning merge; reclaiming it mid-night means stopping the loop first. The entry gate (clean tree, Craig present) fronts the handoff, the unattended dirty-skip backstops anything that slips, and the workflow documents the Emacs buffer-revert caveat at launch and after the morning merge. Worktree isolation was rejected because untracked inbox drops exist only in the main tree, which blinds the inbox pass; plumbing commits were rejected as fragile. + +** DONE Roam-write lock guards nothing unless every roam writer acquires it +Resolved 2026-07-14 (Craig): Phase 3 strengthened from notes to mandatory write-path changes. inbox.org core §5 gains acquire/release around the Phase D edit; knowledge-base.md's write recipe gains the same around its write block. Both degrade gracefully when the helper isn't installed (proceed unlocked — today's behavior); an interactive caller finding the lock busy does a bounded wait (~30s) then surfaces to the user rather than proceeding unlocked. + +** DONE Roam lock location gets committed by roam-sync +Resolved 2026-07-14 (Craig): both locks live under /run/user/<uid>/agent-locks/<name>/, with ~/.cache/agent-locks/ as the fallback where no runtime dir exists. The agent-lock helper owns the path scheme; callers pass names, never paths. tmpfs gives host-locality by construction and reboot self-clearing for free. + +** DONE Sentry's own spine writes trip its dirty-tree skip +Resolved 2026-07-14 (Craig), all three parts: the unsafe-state dirty check excludes the spine set (session-context.org / session-context.d/), each fire ends with one digest commit that sweeps accumulated spine writes so read-only fires still leave a clean tree, and the spine path resolves through .ai/scripts/session-context-path so a concurrent agent's anchor is never clobbered. + +** DONE Pass 8 depends on a known data-corrupting bug +Resolved 2026-07-14, mid-review: the lint-org fix landed in 951b6fc — block-type-aware scanning in both helpers, CLI report-only by default with writes behind --fix, and the true corruption path (wrap-org-table's load-time dispatch firing on lint-org's require) guarded to entry-script-only. Pass 8 runs against the fixed linter; the prerequisite is satisfied and no Implementation-phases dependency is needed. + +** DONE Unattended inbox-pass semantics are undefined +Resolved 2026-07-14 (Craig): sentry runs inbox processing under inbox.org's no-approvals contract verbatim — quick+solo+agreed items execute, shared-asset and convention proposals park (prepared diff, VERIFY task, sender reply). Everything the pass did lands in the morning queue, and anything the engine would ask interactively defers to the queue instead of blocking the fire. + +** DONE Per-pass commits skip the pre-commit suite run — undecided convention conflict +Resolved 2026-07-14 (Craig): recorded as a Decision (see Decisions — suite policy). Entry run establishes the green baseline; no per-pass runs; a fire-end suite run fires only when a pass modified files outside the org/spine set, so the rare code-touching pass is caught before its commits age overnight. Justification recorded with both halves of the consequences. + +** DONE KB promotion pass has no defined lesson source +Resolved 2026-07-14 (Craig): pass 11 is cut from v1 and deferred to vNext. KB promotion stays a wrap-up concern — the editorial moment where Craig answers the promotion prompt. An unattended judgment pass writing to the shared KB waits until sentry has quiet weeks behind it AND a designed detection heuristic; the heuristic design is filed in todo.org ([#D] "KB lesson-detection heuristic", which blocks re-adding the pass). + +** DONE Lock contention and reclaim mechanics are half-specified +Resolved 2026-07-14 (Craig): both mechanics adopted. Contention: bounded wait (~30s, capture-guard's --wait shape), then defer — the pass skips with a digest line and the next fire retries. Reclaim: heartbeat refresh — the runner re-touches the lock timestamp between passes, so a live run's lock is never older than one pass and the TTL sizes to the longest single pass (minutes). Every reclaim surfaces in the digest. + +** DONE Wrap-up while the loop is live is undefined +Resolved 2026-07-14 (Craig): wrap-up refuses while sentry is active. It detects the live single-runner lock and stops with "sentry is active — say 'stop sentry' first." The stop-sentry operation is defined in sentry.org and owns the shutdown: cancel the loop, walk the branch disposition (squash-merge now or leave named), walk-or-carry the approval queue. Wrap-up itself gains only the one guard; the shutdown logic lives with sentry. + +** DONE Entry should reconcile the project branch before branching +Resolved 2026-07-14 (Craig): adopted. After the entry gates pass, sentry runs the same fetch-and-ff-only reconcile startup uses, then creates the branch. A diverged branch joins the interactive entry gate (Craig is present at launch) rather than being auto-resolved. + +** DONE Multi-day unmerged branch stalls hygiene silently +Resolved 2026-07-14 (Craig): adopted. After the second consecutive fire skipped for the unmerged-branch reason, sentry sends one persistent desktop notification naming the project and branch ("sentry stalled: <branch> unmerged — merge or delete to resume"), then repeats at most daily. Persistent notify matches the paging convention: it stays on screen until dismissed. + +* Implementation phases + +** Phase 1 — agent-lock helper +.ai/scripts/agent-lock (canonical: claude-templates/.ai/scripts/) with bats tests: acquire/release/status, contention, staleness reclaim, metadata. Tree stays working; nothing calls it yet. + +** Phase 2 — sentry.org engine +The workflow file: entry ticket, interactive gates, branch mechanics, pass runner with the probe→work→log→commit contract, digest + approval queue, skip semantics. INDEX.org entry. Mirror synced. + +** Phase 3 — companion reconciliations +knowledge-base.md write recipe (acquire/release the roam-write lock around the write block; edit-plus-trigger replaces inline pull/commit/push); inbox.org core §5 (acquire/release around the Phase D edit, capture-guard staying as the human layer underneath; graceful degradation when the helper is absent); roam-sync.sh header comment; triage-intake.org note (runs as a sentry pass; own triggers kept); wrap-it-up.org active-sentry guard (refuse and point at "stop sentry"). + +** Phase 4 — verification +make test green; a live trial night on rulesets (ratio): entry gates exercised, one fire observed end to end, morning branch review performed; follow-up tasks filed from what the trial surfaces. + +* Acceptance criteria +- [ ] agent-lock: two concurrent acquires produce exactly one winner; release frees; a stale lock (aged past TTL) is reclaimed with a surfaced note; a busy lock is a bounded wait then defer; heartbeat refresh keeps a live lock young; locks live under the runtime dir (cache fallback), never inside a repo; bats suite green. +- [ ] Sentry declines to start without :COMMIT_AUTONOMY: yes, naming the marker. +- [ ] Entry on a dirty tracked tree stops with the file-by-file description and the three options; untracked files don't trigger it. +- [ ] A fire on a clean project produces one digest with every pass either run or skipped-with-reason; no other output shape exists. +- [ ] Writing passes commit individually to sentry/<date>-<host>; main's ref is untouched; nothing is pushed. +- [ ] A roam-writing pass acquires the roam-write lock, passes capture-guard, edits, triggers roam-sync, releases, and never runs git write commands against ~/org/roam. +- [ ] An unmerged prior sentry branch at entry causes a skip-and-note, not a second branch. +- [ ] Destructive/judgment actions appear only in the approval queue, never executed unattended. +- [ ] On a project missing a pass target (no todo.org, no triage plugins), the pass skips with one line and the rest run. +- [ ] Entry reconciles the project branch ff-only before creating the sentry branch; a diverged branch stops at the interactive gate. +- [ ] Sentry's own spine writes never trip the dirty-tree skip (a full fire of read-only passes ends with a clean tree via the digest commit). +- [ ] "Wrap it up" during an active loop refuses and names "stop sentry"; "stop sentry" cancels the loop and walks the branch and queue disposition. +- [ ] The second consecutive unmerged-branch skip produces one persistent desktop notification; repeats are at most daily. +- [ ] The inbox pass parks a shared-asset proposal (VERIFY + prepared diff + sender reply) rather than applying it, and the morning queue lists everything the pass did. + +* Readiness dimensions +- Data model & ownership: locks own their directories under /run/user/<uid>/agent-locks/ (tmpfs, outside every repo); digest and queue live in session-context.org (session-owned, path via session-context-path); commits on the sentry branch are Craig-authored per commits.md. +- Errors, empty states & failure: every skip names its reason in the digest; a failed pass banners and the rest continue; the digest surfaces lock-reclaim events. No silent outcomes. +- Security & privacy: no credentials touched; KB promotion honors the knowledge-base.md work-project refusal contract; commit messages follow the no-tooling-enumeration rule where applicable. +- Observability: the digest is the observability surface (per-fire, timestamped, failures on top); session-context.org persists it across crashes. +- Performance & scale: idle fire is seconds (probes + short-circuits); heaviest pass is triage (network); hourly cadence assumed fine; the knob covers it. +- Reuse & lost opportunities: reuses inbox.org, triage-intake.org, clean-todo, task-audit, lint-org.el, capture-guard, roam-sync wholesale; the only new code is agent-lock and the engine file. +- Architecture fit & weak points: engine+passes mirrors the engine+plugin convention; weak points are lock TTL tuning (reclaim vs slow pass) and /loop session lifetime (a killed session ends sentry; the lock's TTL clears the residue). +- Config surface: /loop interval; :COMMIT_AUTONOMY: marker; lock TTLs as script flags with defaults. Nothing else. +- Documentation plan: the workflow file is the doc (When to Use + the morning-review section); INDEX.org entry; no README change needed. +- Dev tooling: bats for agent-lock via the existing glob-discovered suites; make test covers it with no Makefile edit. +- Rollout, compatibility & rollback: additive, new files plus comment-level reconciliations; no consuming project changes behavior until Craig launches sentry there. Rollback is deleting the branch and not launching. +- External APIs & deps: none beyond tools already in use (git, systemctl, bats). N/A for schema verification. + +* Risks, Rabbit Holes, and Drawbacks +- The heartbeat bounds the TTL question to a single pass, but a pass that legitimately outruns its TTL (a slow triage sweep on a bad network) can still be reclaimed under itself. Start generous, surface every reclaim, tune from the digest. +- /loop lifetime: sentry dies with its session (harness restart, machine sleep). Acceptable for v1: the morning state is a readable branch plus session-context, and nothing corrupts. +- Digest fatigue: hourly fires on an active evening could stack noise. The interval knob and the one-line-when-idle rule are the mitigations; revisit after a week of use. +- The sentry branch holds todo.org/notes.org edits overnight; Craig editing those files on main before the morning merge makes the squash-merge conflict. That conflict is small, self-inflicted, and resolvable by hand; the morning-review section documents it. + +* Review and iteration history + +** 2026-07-14 Tue @ 02:03:28 -0500 — Claude Code (rulesets) — responder +- What: all 12 findings walked with Craig one by one and dispositioned; each completed in place and folded into the body. Load-bearing calls: launch contract with in-place checkout (worktrees rejected — untracked inbox drops are invisible there); every roam writer adopts the lock; locks relocate to the XDG runtime dir; spine-set exclusion + fire-end digest commit; inbox pass runs the no-approvals contract; suite policy recorded as Decision 10; KB promotion pass cut to vNext (heuristic task filed); heartbeat + bounded-wait lock mechanics; wrap-up refuses during a live loop ("stop sentry" owns shutdown); entry ff-only reconcile; stall notify after two skips. Mid-walk, Craig redirected the pass-8 finding into fixing the lint-org bug immediately — landed as 951b6fc, which also found and closed the true 2026-07-09 corruption path (wrap-org-table's load-time dispatch firing on lint-org's require). +- Why: Craig chose to resolve every open question together rather than batch them into a written response round. +- Artifacts: findings section above (12/12 done); todo.org tasks (KB heuristic [#D], bug task closed); commit 951b6fc. + +** 2026-07-14 Tue @ 01:15:00 -0500 — Claude Code (rulesets) — reviewer +- What: adversarial weakness review at Craig's direction — batches of candidate weaknesses checked against the spec, looped until a round produced fewer than five substantive new ones. Round 1: 20 candidates, 11 answered or mitigated by the spec, 9 became findings. Round 2: 7 candidates, 3 answered, 4 folded into findings. Round 3 produced fewer than five substantive candidates; loop terminated. Result: 10 blocking + 2 non-blocking findings recorded above. Rubric: Not ready until the blocking findings are dispositioned. Status stays DRAFT for Craig's deeper review. +- Why: Craig asked for the weakness loop before his own deep review, in place of a standard first-pass spec-review. +- Artifacts: findings grounded in code reads — roam-sync.sh:28 (git add -A commits anything inside the roam repo), .ai/scripts/session-context-path (AI_AGENT_ID scoping), todo.org [#B] "Org-table helpers corrupt example blocks" (lint-org mutate-on-lint), inbox.org core §2 no-approvals park path, verification.md pre-commit suite rule. + +** 2026-07-14 Tue @ 00:52:03 -0500 — Craig Jennings — author +- What: initial draft. +- Why: work project's sentry proposal accepted after skeptical review; nine design decisions resolved live in-session, recorded under Decisions. +- Artifacts: [[file:../design/2026-07-14-sentry-workflow-proposal.org][origin proposal]]; todo.org sentry build task. diff --git a/docs/specs/2026-07-20-silent-until-signal-monitors-spec.org b/docs/specs/2026-07-20-silent-until-signal-monitors-spec.org new file mode 100644 index 0000000..c15d10f --- /dev/null +++ b/docs/specs/2026-07-20-silent-until-signal-monitors-spec.org @@ -0,0 +1,126 @@ +#+TITLE: Silent-Until-Signal Monitor Loops — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-20 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED silent-until-signal monitor loops +:PROPERTIES: +:ID: af592bd6-d3e6-47e2-8804-2a287b4d9303 +:END: +- 2026-07-20 Mon @ 13:34:53 -0500 — READY (Craig approved after his read) → building Phases 2-5 immediately → IMPLEMENTED. Phase 2 (auto triage-intake heartbeat), Phase 3 (auto inbox-zero heartbeat), Phase 4 (shared-policy home: the spec is the single definition, each workflow states its own heartbeat and references it), Phase 5 (manual-testing checklist). All phases shipped. +- 2026-07-20 Mon @ 12:30:00 -0500 — Phase 1 (sentry quiet-fire heartbeat) implemented ahead of the full READY gate, at Craig's direction — a quiet sentry fire now collapses to =sentry at HH:MM: nothing=, and the "no silent skip" discipline is reconciled (the heartbeat is the explicit "nothing" record). Phases 2-5 (triage, inbox, shared-policy home, verification) still pending Craig's deep read. Spec stays DRAFT. +- 2026-07-20 Mon @ 12:14:20 -0500 — drafted during the morning session. Design decisions resolved live with Craig off the .emacs.d proposal and the sentry live-trial evidence. DRAFT pending his deep read. + +* Metadata +| Status | draft | +|----------+--------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+--------------------------------------------------------------| +| Reviewer | (spec-review, next session) | +|----------+--------------------------------------------------------------| +| Related | [[file:../../todo.org::*Silent-until-signal for in-session monitor loops][todo.org — silent-until-signal task]] | +|----------+--------------------------------------------------------------| + +* Summary + +An in-session monitor loop (sentry, auto triage-intake, auto inbox-zero) fires the model once per interval. Today each fire narrates a full turn even when nothing changed, so a long session fills with walls of "quiet fire" / "no new items" output. The fix is a policy, not a mechanism: every monitor fire does cheap detection first, and on an empty check collapses to a single labelled heartbeat line ("sentry at 03:20: nothing") and stops. Only a fire that finds a genuinely new item spends a full surface-and-judge turn. Detection stays in-session, so the policy applies uniformly to file-based and MCP-auth loops alike. + +* Problem / Context + +In-session cron/loop monitors surface a visible model turn on every fire. When nothing changed, that turn is pure noise — a per-pass sentry digest full of SKIP lines, or a triage/inbox "nothing new" report. Over a night or a long session the signal (the one fire that found something) drowns in the empty ticks. The rulesets sentry live trial (2026-07-20) demonstrated it directly: fires 1-2 did real work, fires 3-8 were near-identical walls of no-op lines. + +Origin: a Craig-approved proposal from .emacs.d (2026-07-20), captured off an auto inbox-zero session filling with empty-check noise. + +** Why not an external watcher (the rejected shape) + +The proposal's first instinct was a shell-level watcher (systemd timer or backgrounded loop) doing detection outside the model, producing zero output on an empty check and dropping a handoff into inbox/ only on a real item — so the model runs zero turns when nothing changed. That is truly-zero-idle, but it has a disqualifying cost: it moves detection out of the session, and auto triage-intake's sources (Gmail, Slack, Linear) are reachable only through the session's inherited MCP auth. triage-intake.org is explicit that it runs in the live session precisely because "the headless-auth wall that blocks a detached cron run does not apply." A detached watcher cannot scan those sources at all. The external-watcher shape would therefore split the three targets into two incompatible cases (file-detectable vs MCP-auth) and still leave triage unsolved. + +The reframing (Craig, 2026-07-20): the noise is a *policy* problem — when to spend a full model turn — not a missing piece of infrastructure. Keeping detection in-session and making the empty fire cheap solves the actual complaint without the watcher, without per-machine daemon setup, and without breaking MCP auth. It applies to all three loops uniformly. + +* Goals and Non-Goals + +** Goals +- An empty monitor fire produces exactly one labelled heartbeat line, not a full narrated turn: =<workflow> at HH:MM: nothing=. +- A fire that detects a genuinely new item does the full surface-and-judge turn unchanged. +- One uniform policy across sentry, auto triage-intake, and auto inbox-zero — no per-loop special-casing. +- Detection stays in-session so MCP-auth loops (triage) get the same treatment as file-based loops. +- Each loop reuses the seen-state it already keeps; no new watcher-owned seen-list. + +** Non-Goals +- No external watcher, systemd timer, or Monitor-tool daemon. Detection is the loop body's own cheap check. +- No truly-zero-idle (no model turn at all on empty). That needs an external watcher and is the shape rejected above; if it is ever wanted for the file-based loops only, it is a separate vNext, logged not built. +- No change to what a loop does when it *does* find something — the surface/judge/act behaviour is untouched. +- No change to the one-shot (non-loop) invocations of these workflows. + +* Design + +** The policy + +Every in-session monitor fire runs in two steps: + +1. *Detect (cheap, silent).* Run the loop's existing detection against its seen-state — sentry's pass probes and branch/digest state, triage's sentinel scan, the inbox monitor's disposition check. This is Bash/tool work, not narration. +2. *Branch on the result.* + - *Nothing new* → emit one line, =<workflow> at HH:MM: nothing=, and end the fire. No digest, no per-pass lines, no report. + - *Something new* → the full existing turn: surface, judge, act/queue, and its normal richer output. + +The heartbeat is the whole output of an empty fire. Its format is fixed: the workflow's short name, =at=, =HH:MM= (local, from =date=), then =: = and the result word (=nothing= for an empty check). Examples: =sentry at 03:20: nothing=, =triage intake at 03:30: nothing=, =inbox zero at 03:30: nothing=. + +** Per-workflow application + +- *Sentry.* A fire whose passes all probe-skip or no-op collapses to =sentry at HH:MM: nothing=. No per-pass digest block is written for a quiet fire. A fire that runs or queues anything writes its full digest as today. (This supersedes the trial's behaviour, where fires 3-8 each wrote a full no-op digest.) +- *Auto triage-intake.* An auto-mode sweep that finds nothing across its enabled sources collapses to =triage intake at HH:MM: nothing=. Detection stays in-session, so the MCP sources are scanned normally; only the output on empty changes. +- *Auto inbox-zero.* A roam-mode cycle that finds no new inbox items collapses to =inbox zero at HH:MM: nothing=. + +** Seen-state (no new artifact) + +Because detection stays in-session, each loop keeps using the state it already maintains to know what is "new": triage's =.ai/last-triage-intake= sentinel, sentry's branch + digest, the inbox monitor's per-cycle disposition. The proposal's "watcher-owned seen-list replacing the in-anchor Dispositioned list" is dropped — it was an artifact of the external-watcher shape, which is not being built. + +** The accepted consequence + +An in-session loop still fires the model once per interval; the harness invokes it each time. So "silent" means the empty fire is *cheap* (one detection pass plus one heartbeat line), not *absent*. This is the deliberate trade for uniformity and for keeping the MCP auth that makes triage possible. The heartbeat also doubles as a liveness pulse: a visible "still running, nothing to do" beats silence that is indistinguishable from a stalled loop. + +* Decisions + +** DONE Policy, not mechanism — detection stays in-session +CLOSED: [2026-07-20 Mon] +Craig reframed the proposal: silent-until-signal is a policy about when to spend a full model turn, not a new watcher. Keeping detection in-session dissolves the MCP-auth split (triage's sources need session auth) and needs no per-machine daemon. + +** DONE Empty fire → one labelled heartbeat line (not fully silent) +CLOSED: [2026-07-20 Mon] +Format =<workflow> at HH:MM: nothing=. A visible pulse is worth one line so a running loop is distinguishable from a stalled one; full silence was the rejected alternative. + +** DONE Applies to sentry, auto triage-intake, and auto inbox-zero uniformly +CLOSED: [2026-07-20 Mon] +The two MCP-auth loops are first-class targets, not just sentry, precisely because detection stays in-session. + +** DONE No external watcher; no truly-zero-idle in v1 +CLOSED: [2026-07-20 Mon] +The systemd-timer / Monitor-tool / backgrounded-shell shapes are out. Truly-zero-idle (no turn on empty) is the only thing they'd buy, it only works for file-based loops, and it breaks triage. Logged as a possible file-only vNext, not built. + +** DONE Reuse each loop's existing seen-state +CLOSED: [2026-07-20 Mon] +No new watcher-owned seen-list; the sentinel / branch-digest / disposition each loop already keeps is the detection state. + +* Implementation Phases + +** Phase 1 — sentry quiet-fire heartbeat — DONE 2026-07-20 +Edit =.ai/workflows/sentry.org= (canonical + mirror): a fire whose passes all probe-skip or no-op writes =sentry at HH:MM: nothing= instead of a full per-pass digest; a fire that runs or queues anything writes the full digest unchanged. Update the digest section and the Common Mistakes "silent skip" note to draw the quiet-fire-vs-working-fire line. Run sync-check. (Separable and the highest-value piece — do first.) + +Shipped 2026-07-20: five edits to sentry.org — Pass Runner step 3 (per-pass lines collapse on a quiet fire), Fire-end step 2 (the heartbeat-vs-digest decision + heartbeat commit variant), the digest section (working-block vs quiet-heartbeat), Unattended-safety (quiet fire is not a silent skip), Common Mistakes #7 (the carve-out). Canonical + mirror synced, lint clean. + +** Phase 2 — auto triage-intake heartbeat +Edit =.ai/workflows/triage-intake.org= Auto mode: an empty sweep collapses to =triage intake at HH:MM: nothing=. Preserve in-session detection and the accumulate-don't-mutate contract. Run sync-check. + +** Phase 3 — auto inbox-zero heartbeat +Edit =.ai/workflows/inbox.org= Auto inbox zero mode: an empty cycle collapses to =inbox zero at HH:MM: nothing=. Run sync-check. + +** Phase 4 — state the shared policy once +Factor the policy statement into one place both loops and sentry point at (a short section in =inbox.org= monitor-mode core, or a =claude-rules/= note if it reads as cross-cutting), so the three workflows reference one definition rather than restating it. Decide the home during build. + +** Phase 5 — verification +Workflow prose, no bats surface. Verify by exercising: a sentry quiet fire prints one heartbeat line and writes no digest; a working fire still writes its full digest and commits. For the two MCP loops, confirm an empty sweep prints the heartbeat and a sweep with a planted item still does the full turn. Add a manual-testing checklist entry per =verification.md= where a live check is the only proof. + +* Prototype / UI + +Not applicable — no UI surface; the deliverable is the one-line heartbeat format. diff --git a/docs/specs/2026-07-20-triage-source-activation-spec.org b/docs/specs/2026-07-20-triage-source-activation-spec.org new file mode 100644 index 0000000..dce1ec1 --- /dev/null +++ b/docs/specs/2026-07-20-triage-source-activation-spec.org @@ -0,0 +1,127 @@ +#+TITLE: Triage Source Activation — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-07-20 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED triage source activation +:PROPERTIES: +:ID: af73ef0b-cd1d-46f1-9e1d-62695733a4de +:END: +- 2026-07-20 Mon @ 13:42:32 -0500 — READY (Craig approved after his read, both open decisions resolved via cj comments: declaration format "good. approved", interactive gate "all of them") → built and IMPLEMENTED same session. Phase 0 activation gate in triage-intake.org, sentry pass-3 probe, engine-intro documentation (the template notes.org has no Workflow State block, so the declaration is documented in triage-intake.org rather than there — deviation from the drafted Phase 3), migration handoffs to home + work, a manual-testing entry. Canonical + mirror synced. +- 2026-07-20 Mon @ 08:43:23 -0500 — drafted during the morning sentry review. The activation model converged live with Craig off the sentry live-trial finding. DRAFT pending his deep read. + +* Metadata +| Status | draft | +|----------+--------------------------------------------------------------| +| Owner | Craig Jennings | +|----------+--------------------------------------------------------------| +| Reviewer | (spec-review, next session) | +|----------+--------------------------------------------------------------| +| Related | [[file:../../todo.org::*Triage source activation][todo.org — Triage source activation task]] | +|----------+--------------------------------------------------------------| + +* Summary + +triage-intake should pull only the sources a project has chosen to pull. Today it runs every plugin it can discover, and the general (personal-account) plugins are template-synced into every project, so every project looks like it wants to triage Craig's personal Gmail, cmail, calendar, and Telegram. The fix is one activation layer: a general plugin runs only when the project names it in a =:TRIAGE_SOURCES:= declaration; a project-specific plugin stays active by its presence, which is already a deliberate per-project act. Sentry's pass-3 probe then reads the same signal. + +* Problem / Context + +The 8-fire sentry live trial (rulesets, 2026-07-20) surfaced this. Sentry's pass 3 probe is "triage source plugins present for this project," and triage-intake discovers sources by globbing =.ai/workflows/triage-intake.*.org= (general, template-synced) plus =.ai/project-workflows/triage-intake.*.org= (project-specific, never synced). Because the general plugins sync into every project, "plugins present" is true everywhere, so the triage pass self-activates in every project — including rulesets, which is not a triage target. + +The harm is two-layered, and sentry's existing safety rule only catches one layer. Sentry's pass-3 line says "destructive actions queue; they never fire unattended," which would hold back the trash/mark-read/star hygiene. But triage-intake also reads the accounts and files each Action item to the local =todo.org= as a =:quick:reactive:= task. Reading and filing are not destructive, so nothing queues them. Under sentry, triage would still authenticate to Craig's personal inboxes overnight and file his personal action items into whatever project the fire runs in. Wrong scope, and it touched real accounts to get there. + +The same over-pull exists interactively: running triage-intake by hand in rulesets today would pull personal Gmail too. The trial only made the unattended case visible. + +Root cause: the model conflates *presence* with *activation*. A plugin has two existing gates — it must be globbed (presence) and pass its =ENABLED= precondition (capability: "is the gmail MCP reachable?"). Neither answers the question the trial exposed: *should this project pull this source?* For the general plugins, presence comes from sync and capability is true on Craig's machine everywhere, so both gates pass in every project. + +Asymmetry that shapes the fix: project-specific plugins do not leak. They live in =.ai/project-workflows/=, are never synced, and exist only where someone deliberately dropped one. A project that polls an RSS feed puts an =triage-intake.rss-*.org= plugin there, and it runs in that project and nowhere else. Only the general synced plugins leak. So the missing activation layer only needs to gate the general plugins. + +* Goals and Non-Goals + +** Goals +- Per-project source selection: a project pulls exactly the sources it declares, nothing more. +- Off-limits by default: a general source that a project has not named is never pulled there. +- Project-specific sources keep working by presence — dropping the plugin is the declaration (Craig's RSS-feed case works unchanged). +- One fix covers both paths: the activation layer lives in triage-intake, so interactive and unattended (sentry) runs both respect it. +- Sentry's pass-3 probe reads the same activation signal rather than mere plugin presence. + +** Non-Goals +- No change to the per-plugin =ENABLED= capability check — it stays the "can I reach this source" gate. +- No change to the four-bucket classification, the digest shape, or the close behavior. +- No auto-migration across projects. Projects that pull general sources today declare them via handoff; the change never edits another project's config unattended. +- No new plugin discovery mechanism — the two-directory glob stays. + +* Design + +** Two plugin classes, one new activation layer + +- *Project-specific plugins* (=.ai/project-workflows/triage-intake.*.org=): active by presence. The plugin exists only because the project author put it there, which is itself the per-project declaration. No further gate. This is where a project's own sources live — an RSS feed, a work Linear, a work Slack. +- *General plugins* (=.ai/workflows/triage-intake.*.org=, template-synced — personal Gmail, cmail, calendar, Telegram, GitHub PRs): active only when the project names the source in its =:TRIAGE_SOURCES:= declaration. Present-but-undeclared means available-not-active: the plugin is on disk, but this project does not pull it. + +** The declaration + +A line in the project's =.ai/notes.org= Workflow State block, alongside =:COMMIT_AUTONOMY:= and =:LAST_AUDIT:=: + +: :TRIAGE_SOURCES: personal-gmail cmail + +Space-separated source names matching general-plugin basenames (=personal-gmail=, =cmail=, =personal-calendar=, =telegram=, =github-prs=). Absent or empty means no general sources are active for this project. Project-specific plugins are unaffected by this line — they run regardless, because presence is their declaration. + +** triage-intake Phase 0 change + +Phase 0 keeps globbing both directories. The loaded-set computation changes: for each *general* plugin, additionally require its basename to appear in =:TRIAGE_SOURCES:=; skip it with an announced reason otherwise ("skipping personal-gmail — not in :TRIAGE_SOURCES:"). Project-specific plugins skip this check. The =ENABLED= capability check still runs on the survivors. The announce-loaded-set block already exists and gains an "inactive (undeclared)" line so the omission stays visible rather than silent — the same anti-silence discipline Phase 0 already enforces. + +** Sentry pass-3 probe change + +The probe changes from "triage source plugins present" to "the project has at least one active triage source" — any project-specific plugin present, or a non-empty =:TRIAGE_SOURCES:= intersecting the general plugins on disk. rulesets, declaring nothing and owning no project-specific plugin, probe-skips cleanly. + +** Migration + +Projects that pull general sources today (home, and possibly work) each add a =:TRIAGE_SOURCES:= line, or their triage goes quiet. This is a per-project handoff, not an automated sweep — the change can't safely guess each project's intended source set. Projects that only ever ran project-specific plugins need no migration. + +* Decisions + +** DONE Activation gates general plugins only; project-specific stay active-by-presence +CLOSED: [2026-07-20 Mon] +Converged live with Craig. Project-specific plugins are already per-project (never synced), so they need no gate; only the general synced plugins leak, so only they need a declaration. + +** DONE The activation layer lives in triage-intake Phase 0, not only the sentry probe +CLOSED: [2026-07-20 Mon] +Placing it in the engine fixes the interactive over-pull too. Sentry inherits the signal rather than reimplementing it. + +** DONE Off-limits by default +CLOSED: [2026-07-20 Mon] +An undeclared general source is never pulled. Opt-in is explicit; there is no "pull everything discovered" default. + +** DONE Migration is per-project handoffs, never an unattended edit of another project's config +CLOSED: [2026-07-20 Mon] +The change can't guess a project's intended source set, and cross-project auto-edits violate the boundary rule. + +** DONE Declaration format — =:TRIAGE_SOURCES:= space-separated basenames in notes.org Workflow State +CLOSED: [2026-07-20 Mon] +Approved by Craig (2026-07-20). =:TRIAGE_SOURCES: personal-gmail cmail= in notes.org Workflow State, mirroring =:COMMIT_AUTONOMY:=. An empty-but-present marker and an absent one are treated identically — both mean "no general sources active" — so there's no need to distinguish them. A project-specific-only project carries no marker; its absence is correct, since those plugins activate by presence. + +** DONE Interactive triage adopts the same gate as unattended +CLOSED: [2026-07-20 Mon] +Craig: "all of them" (2026-07-20). The activation gate lives in triage-intake Phase 0 and applies to every path — interactive and unattended alike — so running triage by hand in a project also respects its =:TRIAGE_SOURCES:= declaration. One activation layer, no per-path special-casing. + +* Implementation Phases + +** Phase 1 — triage-intake Phase 0 activation rule +Edit =.ai/workflows/triage-intake.org= (canonical =claude-templates/.ai/workflows/= + mirror): add the general-vs-project-specific activation rule to Phase 0, the =:TRIAGE_SOURCES:= read, and the "inactive (undeclared)" announce line. Run sync-check. + +** Phase 2 — sentry pass-3 probe +Edit =.ai/workflows/sentry.org= (canonical + mirror): change the pass-3 probe to "any active triage source present" and note the activation source. Run sync-check. + +** Phase 3 — document the declaration +Document =:TRIAGE_SOURCES:= in the notes.org Workflow State reference (the template =claude-templates/.ai/notes.org= Workflow State block) so new projects see it. Cross-reference from triage-intake. + +** Phase 4 — migration handoffs +=inbox-send= home (and work, if it pulls general sources) the =:TRIAGE_SOURCES:= line each should add, with the reason. No auto-edit. + +** Phase 5 — verification +triage-intake is a workflow (prose), not a script, so there's no bats surface for the activation rule directly. Verify by a scripted check that a project with no declaration loads zero general plugins (a small fixture around the Phase-0 glob-and-filter logic if it's extracted, or a manual-testing checklist entry otherwise). Confirm rulesets probe-skips triage under sentry, and home still pulls its declared sources. + +* Prototype / UI + +Not applicable — no UI surface. diff --git a/docs/agent-knowledge-base-spec.org b/docs/specs/agent-knowledge-base-spec.org index 78ff9bd..d5c0ce7 100644 --- a/docs/agent-knowledge-base-spec.org +++ b/docs/specs/agent-knowledge-base-spec.org @@ -1,12 +1,20 @@ #+TITLE: Agent Knowledge Base on Org-roam — Spec -#+AUTHOR: Craig Jennings & Claude +#+AUTHOR: Craig Jennings #+DATE: 2026-06-10 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Agent Knowledge Base on Org-roam — Spec +:PROPERTIES: +:ID: 08a5ec99-9e1e-40e4-8241-e8a41e9de49f +:END: +- 2026-07-02 Thu @ 00:17:01 -0400 — retrofitted by spec-sort; status set to IMPLEMENTED (reason: v1 (Phases 0-4) shipped 2026-06-10 on Craig's go; KB live at ~/org/roam with the knowledge-base rule installed machine-wide) * Metadata -| Status | implemented — v1 (Phases 0-4) shipped 2026-06-10 on Craig's go; manual validation + other-machine clones outstanding (todo.org) | +| Status | implemented | | Owner | Craig Jennings | | Reviewer | Craig Jennings; Codex (2026-06-10) | -| Related | [[file:../todo.org][todo.org — "Check that memories are sync'd across machines via git"]] | +| Related | [[file:../../todo.org][todo.org — "Check that memories are sync'd across machines via git"]] | This spec supersedes the 2026-06-05 draft (formerly docs/design/2026-06-05-org-roam-knowledge-base-spec.org, removed; content in git history), folding in Craig's 2026-06-10 ratification answers and restructuring to the spec-create format. @@ -308,4 +316,4 @@ Modified recommendations from the 2026-06-10 Codex review, with reasons. Everyth ** 2026-06-10 Wed @ 17:31:10 -0500 — Codex — reviewer - What changed or was recommended: re-ran the spec-review workflow after the caveat resolution. Rubric: ready. No new blocking or medium-priority findings; no review file written. Confirmed the implementation phases and test-surface tasks are already represented under the existing parent task in todo.org. - Why: the prior blockers are dispositioned, the work-root denylist is confirmed, the pointer-rule install path matches the current Makefile RULES glob, and v1's manual/agent-runnable verification surface is explicit. -- Artifacts: this file; [[file:../todo.org][todo.org]] parent task "Check that memories are sync'd across machines via git". +- Artifacts: this file; [[file:../../todo.org][todo.org]] parent task "Check that memories are sync'd across machines via git". diff --git a/docs/specs/inbox-workflow-consolidation-spec.org b/docs/specs/inbox-workflow-consolidation-spec.org new file mode 100644 index 0000000..24ce30a --- /dev/null +++ b/docs/specs/inbox-workflow-consolidation-spec.org @@ -0,0 +1,199 @@ +#+TITLE: Inbox Workflow Consolidation — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-23 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* READY Inbox Workflow Consolidation — Spec +:PROPERTIES: +:ID: a7fe2a10-dfa8-4ba3-a11a-e7b1288b7573 +:END: +- 2026-07-02 Thu @ 00:17:01 -0400 — retrofitted by spec-sort; status set to READY (evidence-based, human-confirmed) + +* Metadata +| Status | ready | +|----------+-------------------------------------------------------------| +| Owner | Craig | +|----------+-------------------------------------------------------------| +| Reviewer | Craig | +|----------+-------------------------------------------------------------| +| Related | [[file:../../todo.org][Consolidate inbox/triage workflows + scheduled inbox check]] | +|----------+-------------------------------------------------------------| + +* Summary + +Four inbox-named workflows (=inbox-zero=, =process-inbox=, =monitor-inbox=, plus the startup/wrap-up nudges) circle the same disposition logic across three different surfaces. This spec consolidates them into one =inbox= engine with explicit modes, keeps =triage-intake= (external accounts) and =no-approvals= (session mode) separate, and adds an interactive recurring roam check (=auto inbox zero=). The fully-unattended cron pass is named but deferred to vNext. + +* Problem / Context + +"Too many inbox related workflows" (Craig, roam capture 2026-06-23). The word "inbox" is overloaded onto three genuinely different surfaces, and the workflows that serve them have grown to circle the same logic: + +- *Project-local =inbox/= dir* (handoffs from other projects/scripts/Craig) → =process-inbox.org= owns the value gate and disposition; =monitor-inbox.org= is a thin cadence layer on top of it ("loop process-inbox every 15 min + act-vs-file + reply discipline"). +- *Global roam inbox* (=~/org/roam/inbox.org=, GTD capture) → =inbox-zero.org=, whose Phase A already *calls* =process-inbox= for the local dir before doing the roam-routing part. +- *External accounts* (email / calendar / PRs) → =triage-intake.org= + six source plugins. + +So a reader (or a non-Claude agent) facing "deal with my inbox" has to know which of four files to invoke, and the shared concepts — the three-question value gate, the skeptical review, the implement/fold/file/defer/reject disposition, the reply-to-sender discipline, the capture-guard before a roam write, the priority-scheme check before filing — are spread across and cross-referenced between them. The duplication is real (=monitor-inbox= and =inbox-zero= both lean on =process-inbox='s machinery) and the count is the symptom Craig named. + +A second gap surfaced in the same capture: there's no documented way to run a *recurring* inbox check, and Craig wants a keyword trigger for it. v1 answers this with an interactive in-session loop (=auto inbox zero=); a fully-unattended cron pass that fires while Craig is away is a larger contract (mutation safety, surfacing-when-away, cross-run state) and is deferred. + +* Goals and Non-Goals + +** Goals +- One engine is the single entry point for the inbox surfaces, with the shared value-gate / disposition / reply / capture-guard / priority-scheme logic living in exactly one place. +- Mode selection is unambiguous from the trigger phrase and the caller (startup, wrap-up, on-demand). +- Every existing trigger phrase still works, routing to the right mode — no relearning. +- A documented interactive recurring check (=auto inbox zero=, =/loop=-based). The fully-unattended cron pass (=/schedule=) is vNext, not v1. +- INDEX.org, protocols.org, and the startup/wrap-up callers reconciled to the new shape with no dangling references. +- No behavior regression: the value gate, disposition rules, capture-guard, and reply discipline behave exactly as today. + +** Non-Goals +- *Not* merging =triage-intake.org=. External-account triage ("what's new across my email/cal/PRs") is a different domain from "my inbox dirs"; keeping it distinct is correct, not redundancy. +- *Not* merging =no-approvals.org=. It's a session mode, not an inbox workflow (it's referenced by the monitor cadence, not part of it). +- *Not* changing value-gate semantics or disposition rules. This is a structural merge, behavior-preserving. +- *Not* the domain-aware whole-roam-inbox routing (still deferred, unchanged). +- *Not* the agent-neutral language sweep over these files — that is the parked half of the agent-source task and runs *after* this merge, over fewer files. +- *Not* renaming =CLAUDE.md=, =.claude/=, or other structural paths. + +** Scope tiers +- v1: merge =process-inbox= + =monitor-inbox= + =inbox-zero= into one =inbox.org= engine with =process= / =monitor= / =roam= modes; preserve all trigger phrases; reconcile INDEX + protocols + startup + wrap-up; add the interactive =auto inbox zero= recurring check (=/loop=). +- Out of scope: =triage-intake= merge, =no-approvals= merge, domain-aware roam routing, the agent-neutrality sweep. +- vNext (log to todo.org): the fully-unattended =/schedule= cron pass — needs its own contract (read-only vs may-mutate =todo.org= / =~/org/roam/inbox.org=, how a find surfaces when Craig is away, how dedup state survives across runs, auth/session constraints); a later umbrella unifying =triage-intake='s "what's new" with the inbox engine; the agent-neutrality pass over the consolidated =inbox.org=. + +* Design + +The consolidation produces one engine file, =inbox.org=, structured as a shared core plus three thin modes. The core holds every concept that today is duplicated or cross-referenced: the three-question value gate, the skeptical review (with the cross-project battery for shared-asset proposals), the disposition ladder (implement-now / fold / file / defer / reject-by-source / park), the reply-to-sender discipline, the capture-guard before any roam-inbox disk write, and the priority-scheme check before filing. A mode is a short front section that says which surface it reads, how it enters and exits, and which core steps it runs. + +*Two altitudes.* + +For the *user*: the trigger phrase picks the mode, and the phrases are unchanged. "process inbox" / "handle the inbox" → process mode (the local =inbox/= dir). "monitor the inbox" / "watch the inbox" → monitor mode (process mode on a loop, with the act-vs-file and reply discipline and the clean-tree/green-suite gates). "inbox zero" / "process the roam inbox" → roam mode (route the global roam inbox by =<project>:= prefix, sweep empties, capture-guard the write). Startup calls process mode for the local dir and the read-only roam nudge; wrap-up calls process mode then the roam sweep. + +For the *implementer*: =inbox.org= is one file. The core sections are written once. Each mode is a section that references core steps by name rather than restating them ("run the value gate (core §X) on each item", "guard and reconcile the roam write (core §Y)"). The old three files are deleted; their content is absorbed, not copied. The =triage-intake= engine and its plugins are untouched and keep their own namespace. + +*Routing and callers.* protocols.org's terminology section and the startup workflow's INDEX-driven routing both key off trigger phrases, so the phrase→mode map is the contract. Each caller that today names =process-inbox.org= / =monitor-inbox.org= / =inbox-zero.org= (startup Phase C, wrap-up Step 3, protocols, INDEX) is repointed at =inbox.org= and the relevant mode. INDEX gets one entry for =inbox.org= listing every trigger phrase, grouped by mode. + +*Auto inbox zero (the scheduled mode).* The trigger phrase =auto inbox zero= starts a recurring roam-mode pass. On invocation the engine *asks Craig for the interval* (e.g. 30 min, 2 hours), then drives the loop with =/loop <interval>= running roam mode. It's in-session and interactive by design — each cycle reports, and a find waits for Craig's go before any work happens. Per cycle: + +- *Nothing found* → no inbox summary. A single acknowledgement line: ran at =HH:MM=, nothing found. Nothing else. +- *Items found* → summarize the found items, file them as tasks, and *append them to a displayed queue* (the harness task list, =TaskCreate=) so the queue accumulates across cycles. Then ask: "run this batch next?" If Craig says yes, the engine launches into implementing the found items (each through the normal disposition + verify flow); if no, they stay queued for a later go. Subsequent cycles add only newly-found items to the same displayed queue, never re-surfacing what's already there. + +The acknowledge-only-on-empty rule keeps a quiet inbox quiet — no noise when there's nothing to do — while a find is always surfaced and gated on Craig's yes. =auto inbox zero= is the interactive =/loop= shape because its execute step waits for a yes, so it is inherently in-session. + +A fully-unattended =/schedule= cron pass (firing while Craig is away) is a different contract and is *vNext, not v1*: it can't wait for a yes, so it has to decide up front whether it may mutate =todo.org= and the roam inbox or stays read-only, how a find reaches Craig asynchronously, how dedup state persists between runs that don't share a session, and what session/auth context a cron run carries. v1 ships only the interactive loop; the unattended contract is logged to =todo.org= for its own design pass. + +* Alternatives Considered + +** Option A — One engine with modes (chosen) +- Good, because it cuts four inbox-named files to one and puts the shared logic in a single authoritative place, which is exactly the "too many" complaint. +- Good, because every trigger phrase can re-home to a mode with no user relearning. +- Bad, because =inbox.org= becomes a larger file with internal mode branching. +- Neutral, because =triage-intake= and =no-approvals= stay separate either way. + +** Option B — Keep three files, extract a shared include +- Good, because the diffs are smaller and the per-surface entry points stay familiar. +- Bad, because it does not reduce the file count — Craig's actual complaint is the number of files, and this keeps three plus adds an include. +- Neutral, because the dedup of logic happens, just without the count reduction. + +** Option C — Merge only process-inbox + monitor-inbox, leave inbox-zero +- Good, because it fixes the tightest, least-ambiguous redundancy (monitor is literally a loop over process) at the lowest risk. +- Bad, because roam vs local stays two files; the consolidation is partial (4→3, not 4→2). +- Neutral, because it could be a first phase of Option A rather than a competing end state. + +** Option D — Do nothing, just document which file is which +- Good, because zero risk to load-bearing synced workflows. +- Bad, because it doesn't reduce the count at all; the complaint stands. + +* Decisions [4/4] + +** DONE Engine shape — one file with modes vs partial merge +- Context: Option A (one =inbox.org=, 4→1) maximally addresses "too many" but is the biggest single change to load-bearing synced files. Option C (merge the process/monitor pair only, 4→3) is lower-risk and could be A's first phase. +- Decision: We will build Option A — one =inbox.org= engine with =process= / =monitor= / =roam= modes. (Craig, 2026-06-23.) +- Consequences: easier discovery and one home for the logic; harder single-file size and a bigger, higher-blast-radius diff, mitigated by the shared-core + thin-mode structure and the plugin-namespace escape hatch for a mode that wants depth. + +** DONE Trigger-phrase routing — preserve all existing phrases +- Context: protocols + startup route by phrase; users have these in muscle memory. +- Decision: We will keep every existing trigger phrase, re-homing each to its mode on the one engine, adding only the new =auto inbox zero= phrase. (Craig, 2026-06-23 — accepted as recommended.) +- Consequences: easier — no relearning, no broken muscle memory; harder — the engine must document a longer phrase→mode table and guard against collisions. + +** DONE triage-intake stays separate +- Context: external-account triage is a different surface; folding it in would re-bloat the engine. +- Decision: We will leave =triage-intake.org= and its plugins untouched, out of this consolidation. (Craig, 2026-06-23 — accepted as recommended.) +- Consequences: easier — smaller, coherent inbox engine; harder — two "what's arriving" entry points remain (inbox engine vs triage-intake), documented so the boundary is clear. + +** DONE Scheduled-check mechanism + behavior + keyword +- Context: Craig wants a recurring inbox check with a keyword, an interactive find-then-execute flow, and a running queue. +- Decision: The trigger phrase is =auto inbox zero=. On invocation it asks Craig for the interval, then runs roam-mode on =/loop <interval>=. Empty cycle → one acknowledgement line (ran at HH:MM, nothing found), no inbox summary. Find → summarize, file as tasks, append to the displayed task queue (=TaskCreate=), and ask "run this batch next?"; on yes, implement the found items; subsequent cycles append only new finds to the same queue. =/schedule= stays available for a fully-unattended pass. (Craig, 2026-06-23.) +- Consequences: easier — a quiet inbox stays quiet, a find is always gated on a yes, and the queue is one accumulating view; harder — the loop must dedup against already-queued items so it doesn't re-surface them, and the in-session =/loop= shape means the unattended case still needs =/schedule=. + +* Review findings [2/2] + +** DONE Fully unattended scheduled behavior is not specified :blocking: +Disposition: accepted via the narrow option. v1 ships only the interactive =auto inbox zero= (=/loop=); the fully-unattended =/schedule= pass is deferred to vNext with its open contract questions named (read-only vs may-mutate, surface-when-away, cross-run dedup state, auth/session). Folded into Summary, Goals, Problem/Context, Scope tiers (vNext), and the Design "Auto inbox zero" subsection; the vNext contract is logged to todo.org. This sequences the unattended pass rather than dropping it, preserving Decision 4's intent. +The Summary and Goals promise a scheduled unattended inbox check with trigger keywords, but the concrete =auto inbox zero= design is intentionally interactive: it asks for an interval, runs =/loop <interval>= in the live session, and waits for Craig before executing found work. The only unattended behavior is the sentence that =/schedule= remains available for a fully-unattended cloud-cron pass. That leaves an implementer to invent the actual scheduled contract: trigger phrase(s), whether the pass is read-only or may mutate =todo.org= / =~/org/roam/inbox.org=, how findings are surfaced when Craig is away, how dedup state survives across runs, and what auth/session constraints apply. Add a distinct =/schedule= subsection and acceptance criteria for the fully unattended mode, or narrow the Summary/Goals to say v1 ships only the interactive =/loop= mode and log the unattended cron shape as vNext. (blocking) + +** DONE Stale-reference verification relies on a checker that does not check workflow links +Phase 2 and the Risks section rely on the workflow-integrity / INDEX-drift check as the backstop for missed references to =process-inbox.org=, =monitor-inbox.org=, and =inbox-zero.org=. Current =scripts/workflow-integrity.py= checks INDEX coverage, script references, plugin parentage, orientation sections, and duplicate trigger phrases; it does not validate arbitrary =[[file:...org]]= workflow links or prose references in workflows/protocols/rules. That means a deleted-workflow link in =startup.org=, =wrap-it-up.org=, or =protocols.org= can survive the named checker. Keep the grep requirement, but make it an explicit acceptance item with the exact scope: at minimum =rg 'process-inbox|monitor-inbox|inbox-zero' claude-templates/.ai .ai claude-rules= after caller rewrites, allowing only intentional historical/spec/todo mentions. Optionally extend =workflow-integrity.py= to validate local workflow links, but do not imply it already catches this class. (non-blocking) + +Disposition: accepted. Added the exact grep as an acceptance item and a Phase 2 step, reworded the Risks "missed caller reference" dodge and the Dev-tooling readiness line so the integrity checker is no longer implied to validate workflow links, and noted the optional =workflow-integrity.py= extension as not-required-for-v1. + +* Implementation phases + +** Phase 1 — Author the inbox engine +Write =inbox.org= (canonical =claude-templates/.ai/workflows/=): the shared core (value gate, skeptical review, disposition ladder, reply discipline, capture-guard, priority-scheme check) plus the three mode sections, absorbing the content of the three source files. No caller changes and no deletions yet — the tree still works with the old files in place, the new engine sits alongside for review. + +** Phase 2 — Reconcile callers and retire the old files +Repoint INDEX.org (one =inbox.org= entry, phrases grouped by mode), protocols.org terminology, startup.org Phase C, and wrap-it-up.org Step 3 at =inbox.org= + mode. Delete =process-inbox.org=, =monitor-inbox.org=, =inbox-zero.org=. Grep for stale references — =rg 'process-inbox|monitor-inbox|inbox-zero' claude-templates/.ai .ai claude-rules= — and clear every live caller (the integrity checker covers INDEX coverage and trigger duplication, not workflow links). Run the workflow-integrity / INDEX-drift check. Sync the mirror. + +** Phase 3 — Auto inbox zero + scheduled check +Add the =auto inbox zero= mode to =inbox.org=: ask-for-interval, =/loop <interval>= over roam mode, the empty-cycle acknowledgement, and the find → summarize → file → queue → ask-to-execute flow with cross-cycle dedup against the displayed queue. Document the =/schedule= recipe for the fully-unattended pass alongside it. + +** Phase 4 — Verify +Trigger-phrase coverage (every old phrase resolves to a mode), startup + wrap-up dry-run against the new engine, capture-guard still gates the roam write, INDEX drift clean, mirror in sync. + +* Acceptance criteria +- [ ] Every trigger phrase that today routes to =process-inbox= / =monitor-inbox= / =inbox-zero= resolves to a mode of =inbox.org=. +- [ ] The three old workflow files are deleted and absent from INDEX; the integrity check reports no orphan or stale entry. +- [ ] After caller rewrites, =rg 'process-inbox|monitor-inbox|inbox-zero' claude-templates/.ai .ai claude-rules= returns only intentional historical / spec / todo mentions — no live caller reference to a deleted file. (The integrity checker validates INDEX coverage, not arbitrary workflow links, so this grep is the real backstop.) +- [ ] Startup still processes the local inbox and produces the read-only roam nudge; wrap-up still sweeps the project's roam items. +- [ ] The capture-guard runs before any roam-inbox disk write in the consolidated engine. +- [ ] The value gate, disposition ladder, and reply-to-sender discipline are present once and unchanged in behavior. +- [ ] =auto inbox zero= asks for an interval, then runs roam mode on =/loop <interval>=. +- [ ] An empty auto cycle emits only a timestamped acknowledgement (ran at HH:MM, nothing found) — no inbox summary. +- [ ] A find summarizes the items, files them as tasks, appends them to the displayed queue, and asks before executing; "yes" runs the batch; later cycles append only newly-found items, never re-surfacing queued ones. +- [ ] Canonical and mirror copies are in sync (=sync-check.sh=). + +* Readiness dimensions +- Data model & ownership: the engine reads two files it doesn't own (project =inbox/= dir contents, =~/org/roam/inbox.org=) and writes =todo.org= + the roam file. Ownership unchanged from today; the merge moves no data. +- Errors, empty states & failure: empty inbox → report and stop (preserved per surface); roam pull blocked or dirty → surface and stop, never auto-stash (preserved); live org-capture on the roam file → capture-guard blocks the write (preserved). +- Security & privacy: N/A because no credentials or sensitive data; the engine moves task text between local files. +- Observability: the user sees which mode ran and its disposition summary; INDEX drift check surfaces a mis-wired routing. +- Performance & scale: N/A because the inputs are small text files triaged by hand-scale counts. +- Reuse & lost opportunities: the whole point — the shared core is written once instead of three times; =triage-intake='s plugin pattern is intentionally not reused here (different surface). +- Architecture fit & weak points: integration points are INDEX.org, protocols.org terminology, startup Phase C, wrap-up Step 3. Weak point: a missed caller reference to an old filename breaks routing — mitigated by the explicit stale-reference grep (acceptance item), since the integrity check covers INDEX coverage, not workflow links. +- Config surface: trigger phrases (the phrase→mode table) and the scheduled-check keyword set + cron expression. +- Documentation plan: the engine file is the doc; INDEX entry updated; protocols terminology updated. No separate user doc needed. +- Dev tooling: the workflow-integrity check + startup INDEX-drift check cover INDEX coverage and trigger-phrase duplication; they do *not* validate workflow file links, so the stale-reference grep (acceptance item) is a manual step. Optionally extend =workflow-integrity.py= to validate local =[[file:...org]]= workflow links — not required for v1. +- Rollout, compatibility & rollback: the merge lands via the template sync; =rsync --delete= removes the three retired files from every consuming project on its next startup, and the new =inbox.org= arrives the same pass. Rollback = git revert of the rulesets commit, then the next sync restores the old files. Trigger-phrase preservation is the compatibility guarantee. +- External APIs & deps: N/A because no external API; =/schedule= and =/loop= are harness features, not deps. + +* Risks, Rabbit Holes, and Drawbacks +- *Missed caller reference.* A lingering mention of =process-inbox.org= / =monitor-inbox.org= / =inbox-zero.org= in a workflow, protocol, skill, or the INDEX would break routing after the files are deleted. Dodge: =rg 'process-inbox|monitor-inbox|inbox-zero' claude-templates/.ai .ai claude-rules= and clear every live caller before deleting. The workflow-integrity checker validates INDEX coverage and trigger-phrase duplication, *not* arbitrary =[[file:...]]= links, so the grep — not the checker — is the real backstop here. +- *Single-file sprawl.* One engine with three modes risks becoming the wall-of-text the workflows were split to avoid. Dodge: the shared-core + thin-mode structure and the terseness pass; if a mode wants real depth, it can become a =inbox.<mode>.org= plugin under the engine namespace (the same pattern =triage-intake= uses) rather than bloating the core. +- *Sequencing with the agent-neutrality sweep.* If the neutrality sweep runs first, it edits three files about to be deleted. Dodge: this consolidation lands first by construction (it's why the sweep was parked). + +* Review and iteration history +** 2026-06-23 Tue @ 21:51:51 -0400 — Claude — author +- What: initial draft. +- Why: Craig chose to spec the inbox-workflow consolidation before building a load-bearing 3-to-1 merge of synced workflows. +- Artifacts: docs/inbox-workflow-consolidation-spec.org; todo.org "Consolidate inbox/triage workflows + scheduled inbox check". +** 2026-06-23 Tue @ 22:05:00 -0400 — Craig — decision-maker +- What: resolved all four decisions; added the =auto inbox zero= scheduled mode (ask-for-interval, empty-cycle acknowledgement only, find → summarize → file → displayed queue → ask-to-execute, cross-cycle dedup). Status → ready for review. +- Why: chose Option A (4→1 engine) and specified the recurring-check behavior in full. +- Artifacts: Decisions [4/4]; Design "Auto inbox zero" subsection. +** 2026-06-23 Tue @ 22:15:58 -0400 — Codex — reviewer +- What: spec-review pass rated the spec =Not ready= and added two findings: the fully unattended =/schedule= behavior is not specified, and stale-reference verification leans on a checker that does not validate workflow links. +- Why: the current design is strong enough for the consolidation and interactive =/loop= mode, but the stated scheduled/unattended goal would force implementers to invent behavior before shipping. +- Artifacts: Review findings [0/2]. +** 2026-06-23 Tue @ 22:28:00 -0400 — Claude — responder +- What: both findings accepted and folded. Finding 1 (blocking) resolved by narrowing v1 to the interactive =auto inbox zero= (=/loop=) and deferring the fully-unattended =/schedule= contract to vNext with its open questions named — Summary, Goals, Problem/Context, Scope tiers, and Design updated. Finding 2 resolved by adding the exact stale-reference grep as an acceptance item + Phase 2 step and dropping the over-claim that the integrity checker validates workflow links. Findings [2/2], Decisions [4/4]; scope narrowed (not expanded), so no readiness-rubric rerun needed. Status → Ready. +- Why: the scheduled/unattended promise outran what Craig actually specced (he detailed the interactive loop); sequencing the cron pass to vNext keeps v1 honest. The checker genuinely doesn't catch stale workflow links. +- Artifacts: Decisions [4/4]; Review findings [2/2]; vNext task to be logged for the unattended cron contract. diff --git a/docs/specs/wrapup-routing-spec.org b/docs/specs/wrapup-routing-spec.org new file mode 100644 index 0000000..07d9ec6 --- /dev/null +++ b/docs/specs/wrapup-routing-spec.org @@ -0,0 +1,226 @@ +#+TITLE: Wrap-Up Inbox/Transcript Routing — Spec +#+AUTHOR: Craig Jennings +#+DATE: 2026-06-13 +#+TODO: TODO | DONE +#+TODO: DRAFT READY DOING | IMPLEMENTED SUPERSEDED CANCELLED + +* IMPLEMENTED Wrap-Up Inbox/Transcript Routing — Spec +:PROPERTIES: +:ID: 00b47414-2213-4a99-be35-48ceb266fc08 +:END: +- 2026-07-04 Sat @ 11:49:59 -0500 — DOING → IMPLEMENTED: the task-routing build shipped and is green — route_recommend.py (destination discovery + recommendation, 13 unit tests), the :ROUTE_CANDIDATE: marker wired into inbox process mode, the wrap-it-up router sub-step, and the route-batch delivery helper (9 bats tests). Manual cross-project end-to-end validation tracked as its own task; the transcript-filing half stays a deferred vNext. +- 2026-07-02 Thu @ 00:17:01 -0400 — retrofitted by spec-sort; status set to DOING (evidence-based, human-confirmed) + +* Metadata +| Status | implemented | +|----------+-----------------------------------------------------| +| Owner | Craig Jennings | +|----------+-----------------------------------------------------| +| Reviewer | Codex (spec-review) | +|----------+-----------------------------------------------------| +| Related | [[file:../../todo.org][todo.org: wrap-up routing task]] · [[file:../design/2026-06-13-wrapup-inbox-transcript-routing-proposal.org][archsetup proposal]] | +|----------+-----------------------------------------------------| + +* Summary + +At wrap-up, an inbox handoff that belongs to another project, once accepted and filed locally, has no clean home in the current project's =todo.org=. This adds an optional routing step to =wrap-it-up.org=: surface the filed keepers whose home is elsewhere, recommend a destination for each, and on one confirmation deliver each to that project's =inbox/= via =inbox-send= (one handoff per task), removing it from the local =todo.org=. The destination's own next session files it through =process-inbox=, applying that project's value gate, priority scheme, and =todo-format.md=. A parallel step (vNext) files meeting-transcript recordings into the right project's =assets/=. + +* Problem / Context + +=process-inbox.org= dispositions each handoff as act / fold / file / reject, and "file as TODO" lands the task in the *current* project's =todo.org=. When the real home is a different project, the choices today are: file it locally and let it rot in the wrong tracker, hand-edit two projects' =todo.org= files, or defer it and carry the debt to next session. + +The wrap-up's existing Step 3 "Inbox sanity check" only counts unprocessed items and blocks the wrap until they clear. It answers "is the inbox clean?" — it doesn't route anything. + +Meeting transcripts have the same homelessness: a recording dropped during a session belongs in some project's =assets/=, but nothing moves it there at wrap. + +The friction is small per-item but recurring, and the manual cross-project edit is error-prone (two files, two repos, easy to leave one half-done). + +* Goals and Non-Goals + +** Goals +- At wrap-up, surface filed keepers whose home is a different project, with a recommended destination each. +- Route the whole batch on one confirmation ("go with recommendations") or leave it entirely ("skip"). No per-item triage. +- Deliver each routable keeper to the destination's =inbox/= via =inbox-send=, one handoff per task, and remove the keeper from the local =todo.org= on send. The destination files it through its own =process-inbox=. +- Provenance is automatic: =inbox-send= stamps the source project and date on every handoff (the =from-<source>= filename and =#+SOURCE:= line). The delivery shows in the destination inbox; the removal shows in the source's git diff. +- The destination set is any project with an =inbox/= — reuse =inbox-send='s existing discovery. + +** Non-Goals +- Not a wrap gate. A skip is a clean, complete wrap. +- Not per-item triage. The interaction is batch-level: go or skip. +- Not a replacement for =process-inbox.org='s value gate. Routing assumes the item is already an accepted keeper. +- Not a confidence-free auto-mover. A low-confidence destination recommendation says so, and the batch "go" stays trustworthy because the surfaced list is reviewable before the keystroke. + +** Scope tiers +- v1: task/event routing by =inbox-send= delivery to the destination's =inbox/=. The interaction, the recommendation engine, the candidate-set marker stamped at file time, reusing =inbox-send='s discovery and delivery. +- Out of scope: per-item destination editing, an interactive correction loop, moving items that aren't accepted keepers, a new cross-repo =todo.org= move primitive (the superseded direct-move design). +- vNext: meeting-transcript filing (gated on the unresolved source-location decision and the file-vs-file+extract question — see Decisions). + +* Design + +** User-facing (the wrap interaction) + +The router is a new sub-step of =wrap-it-up.org='s Step 3, running after the existing inbox sanity check. Its input is filed keepers, not raw inbox files (decision: Reading B): tasks =process-inbox= accepted and filed into the local =todo.org= this session whose inferred home is a different project. When the router finds such a keeper, it surfaces it in a list, one line each: the task, the recommended destination project, and a confidence marker when the inference is weak. Then two options, batch-level: + +1. Go with the recommendations — route every recommended item (inbox-send to the destination + local removal). +2. Skip — leave the whole batch in place. A skip is a clean wrap. + +That is the entire interaction. No per-item walk. The surfaced list is the review surface; the single keystroke is trustworthy because the list was reviewable and low-confidence recommendations flagged themselves. + +On "go", each routable keeper is delivered to its recommended destination's =inbox/= via =inbox-send= (one handoff per task) and removed from the local =todo.org=; the destination's own next session files it through =process-inbox=. A skipped or no-match item stays where it is; the existing sanity check still governs whether the wrap is clean. + +** Implementer (the mechanics) + +*Candidate set (what the router considers).* Reading B means the router does not scan the whole local backlog — it would otherwise suggest moving legitimate local tasks every wrap. The candidate set is keepers =process-inbox= filed this session whose inferred home differs from the current project, identified by a marker stamped at file time (decision D8): =process-inbox='s "file as TODO" step stamps =:ROUTE_CANDIDATE: <inferred-project>= on any keeper whose inferred home is not the current project. At wrap, the router's candidate set is exactly the local tasks carrying that property — never the standing backlog. + +*Destination discovery.* Reuse =inbox-send.py='s existing =discover_projects= (a project is a directory with =.ai/= AND =inbox/=). The destination must have an =inbox/= to receive a handoff, so that is the natural destination set — no new discovery code. A project with a =todo.org= but no =inbox/= cannot receive an inbox handoff and must be bootstrapped first; in practice every active project has an =inbox/=. + +*Delivery.* For each candidate, on "go": (1) =inbox-send <destination> --file= a one-task handoff into the destination's =inbox/= (one file per task, so the destination's =process-inbox= dispositions it as a single item), then (2) remove the keeper from the local =todo.org=. Step 1 is a cross-project write, but it uses the =cross-project.md=-sanctioned path (dropping a file in another project's inbox needs no confirmation); step 2 is a single-file edit in the current project's own =todo.org=, which the wrap is already committing. No new cross-repo move primitive, no foreign =todo.org= edit. + +*Provenance and filing.* =inbox-send= stamps the source and date automatically (=from-<source>= filename + =#+SOURCE:= line), so the destination's session knows where the item came from. That session files it through its own =process-inbox= — value gate, priority scheme, =todo-format.md= — so the task lands per the destination's conventions rather than as an externally-authored insertion. + +*Recovery (mis-route).* If the recommendation engine picks a wrong destination, the receiving session rejects it via =process-inbox='s reject-from-another-project flow (write a response, =inbox-send= it back to the source named in the provenance, delete the local copy). The task returns to the source project's inbox; nothing is lost or corrupted. This is why removing the source on send is safe — the reject path is the undo. + +*Recommendation engine.* Infer the destination from the item's content — project names, file paths, topic words — matched against the discovered project list, with a confidence tier: *strong* = a destination project's name or path appears literally in the item; *weak* = topic-word overlap only; *none* = no match, the item stays put and is never surfaced as a route. "Go" routes strong and weak items (weak visibly labeled); a no-match item is left in place. Pure function =(item, project-list) → (destination, confidence)=, unit-tested directly. The engine is the interesting, uncertain part; it earns the spec. + +* Alternatives Considered + +** Per-item triage instead of batch go/skip +- Good, because it gives precise control over each destination. +- Bad, because it taxes the common case (a batch that's all-correct, or all-stay) with a walk. Craig explicitly asked for two options, not a triage loop. +- Neutral, because per-item correction could return as a vNext refinement if batch-only proves too blunt. + +** Fold the router into the existing Inbox sanity check step +- Good, because one inbox step is simpler than two. +- Bad, because the sanity check *gates* the wrap (blocks until clean) and the router is *optional* (skip is clean). Merging a blocking check with an optional action muddies both. +- Neutral, because the two share discovery code while staying separate steps. (Resolved: D1 keeps them separate, with the router acting on filed keepers rather than inbox files.) + +** Reuse process-inbox's "file as TODO" with a destination argument +- Good, because it avoids a second mechanism. +- Bad, because =process-inbox= runs per-item mid-session against the local project; the router runs at wrap, batch-level, cross-project. Different cadence, different scope. +- Neutral, because both ultimately call the same atomic move helper — the helper is the shared primitive, the two callers stay distinct. + +* Decisions [9/9] + +** DONE Reuse the Open Work matcher for destination anchoring +- Context: the move needs a reliable insertion point in the destination =todo.org=; guessing risks corrupting another project's file. +- Decision: We will reuse =todo-cleanup.el='s =tc--find-section "open work"= matcher, which already handles the unique / missing / ambiguous cases, and skip+surface any destination without a clean Open Work heading. +- Consequences: easier — no new parser, consistent with =--archive-done=. Harder — destinations must carry the "Open Work" heading convention, so a project with a differently-named section is silently unroutable until it conforms. + +** SUPERSEDED Move atomically through a helper, never hand-edit two repos +Superseded 2026-06-21 by "Deliver via inbox-send" below. The original plan built a new atomic helper to insert a subtree into a foreign =todo.org= and remove the source. The inbox-route delivers the keeper to the destination's inbox instead, so no cross-repo move primitive is built. +- Context: a move touches two files in two repos; a half-done move loses or duplicates a task. +- Decision (superseded): route every move through one helper that inserts under the destination's Open Work heading and removes the source as one operation. + +** SUPERSEDED Cross-project writes stay visible and carry provenance +Superseded 2026-06-21 by "Deliver via inbox-send" below. =inbox-send= already stamps provenance (=from-<source>= filename + =#+SOURCE:= line), so the hand-stamped note is unnecessary; the destination files the item through its own gate rather than receiving an externally-authored insertion. +- Context: writing into another project's =todo.org= crosses the =cross-project.md= scope boundary. +- Decision (superseded): treat the batch "go" as authorization, leave the move visible in the destination's git diff, and stamp a one-line provenance note on each moved task. + +** DONE Separate router step, operating on filed keepers (Reading B) +- Context: the sanity check gates the wrap on inbox/ contents; the router is optional. The deeper question was the router's input — raw inbox files (Reading A, which overlaps the sanity check) or already-filed keepers that belong elsewhere (Reading B, a todo-routing concern). +- Decision: We will keep the router a separate optional sub-step after the sanity check, and its input is Reading B: accepted keepers process-inbox filed into the local =todo.org= whose inferred home is another project. The sanity check stays a pure inbox gate; the router is a todo-routing action that shares only the destination-discovery code. +- Consequences: easier — each step has one job, the gate can't be muddied by an optional action, and the router never competes with the inbox gate over the same files. Harder — the candidate set (which local tasks the router considers) needs a marking mechanism (see the Implementer "candidate set" note); Reading A's "dispose raw inbox files at wrap" convenience is given up. + +** DONE Transcript routing deferred to vNext +- Context: transcripts file as artifacts, not tasks, and a meeting usually produces both a recording to keep and action items to track. Two unknowns block it: where recordings accumulate (a recordings inbox, a downloads dir, wherever the meeting tooling drops them), and whether filing should also extract action items into the destination's =todo.org=. +- Decision: We will defer transcript routing to vNext. Both the source-location dependency and the file-only-vs-extract-action-items question are deferred with it, to be settled when the vNext work is specced. v1 ships task routing only. +- Consequences: easier — v1 isn't blocked on the unresolved source location. Harder — until vNext, a meeting recording still has no automatic home; only its action items (if filed as tasks) route through v1. + +** DONE Keep defer-and-stage and the router as distinct policies +- Context: the 2026-06-12 Skeptical Review added a defer-and-stage path in =process-inbox.org= that files a =[#B]= VERIFY for shared-asset proposals parked for review. That also turns an inbox item into a =todo.org= task — overlapping surface with this router. +- Decision: We will keep them distinct. Defer-and-stage parks a proposal-under-review locally as a VERIFY; the router moves an accepted keeper to its home project as a TODO. They differ on review status (proposal vs accepted) and destination (local vs cross-project), and share only the atomic move helper, not the policy. Reading B makes the split clean: the router acts on accepted keepers, never on proposals under review. +- Consequences: easier — two clear, non-competing policies on one shared primitive. Harder — the workflow prose must name the boundary so a future reader doesn't collapse them and reintroduce the ambiguity. + +** DONE Deliver via inbox-send to the destination's inbox, not a direct todo.org move (supersedes D2/D3) +- Owner / by-when: Craig / ratified 2026-06-21 (spec-response) +- Context: D2/D3 built a new atomic helper that edits a foreign =todo.org= and removes the source, with a hand-stamped provenance note. =inbox-send= + =process-inbox= already do cross-project delivery: inbox-send writes the handoff with =from-<source>= provenance, and the destination's process-inbox files it through that project's own gate. =cross-project.md= names the inbox as the sanctioned cross-scope write path. A verified precondition reversed the old assumption — some projects have =inbox/= but no =todo.org=, so direct-move's discovery silently drops keepers headed there while inbox-route delivers. +- Decision: We will route each keeper by =inbox-send= into the destination's =inbox/= (one handoff per task) and let the destination's own =process-inbox= file it; we will not edit the destination's =todo.org= directly. D2 (atomic move helper) and D3 (hand-stamped provenance) are superseded — the helper isn't built, and provenance is inbox-send's by construction. +- Consequences: easier — no new cross-repo write primitive, no foreign-tracker corruption risk, provenance and per-project filing for free, graceful when the destination lacks a =todo.org=. Harder — filing is deferred to the destination's next session (self-resolving, since startup auto-runs =process-inbox= on a non-empty inbox), and a project never opened accumulates a visible inbox backlog rather than a silent foreign insertion. + +** DONE Candidate-set marking: tag :ROUTE_CANDIDATE: at process-inbox file time (Option A) +- Owner / by-when: Craig / ratified 2026-06-21 (spec-response) +- Context: the router must consider only this-session-filed inbox keepers whose home is elsewhere, never the standing backlog. Two options: tag at file time (process-inbox stamps a marker) or infer from a =CREATED=-this-session stamp + content. =process-inbox= does not stamp =:CREATED:= today, so the inference option would need that paired edit anyway, removing its only advantage. +- Decision: We will tag at file time. =process-inbox='s "file as TODO" step stamps =:ROUTE_CANDIDATE: <inferred-project>= on any keeper whose inferred home differs from the current project; the router's candidate set is the local tasks carrying it. +- Consequences: easier — precise (zero standing-backlog false positives), the inference happens once where context is richest, and the marker doubles as the router's "go" trigger. Harder — a paired edit to =process-inbox.org= Phase D ships coupled with the router. + +** DONE Source removal is a local todo.org edit on send; recovery via the reject flow +- Owner / by-when: Craig / ratified 2026-06-21 (spec-response) +- Context: the review left source-handling vague ("leave the source until the destination confirms by filing"), but there is no confirmation callback, so leaving it duplicates the task once the destination files. The keeper was filed into the *current* project this session and doesn't belong there. +- Decision: On "go" we will remove the routed keeper from the *current* project's =todo.org= (a local single-file edit, not a cross-repo write) right after the =inbox-send=. If the destination rejects the handoff, =process-inbox='s reject-from-another-project flow returns it to the source's inbox, so the removal is reversible. +- Consequences: easier — no duplication, the only deletion is from a file we own and are already committing, the reject path is the undo. Harder — a brief window exists where the task lives only as an in-flight inbox handoff (between send and the destination's filing); acceptable because the handoff file is durable and the reject path recovers a mis-route. + +* Implementation phases + +** Phase 1 — Destination discovery (reuse inbox-send) +Reuse =inbox-send.py='s =discover_projects= (a directory with =.ai/= AND =inbox/=) as the destination set — no new discovery code. Confirm the destination universe: if a real destination has a =todo.org= but no =.ai/+inbox/=, name it and bootstrap its inbox; otherwise the existing filter already covers it. Leaves the tree working. + +** Phase 2 — Candidate-set marking in process-inbox +Extend =process-inbox.org='s "file as TODO" step (Phase D) to stamp =:ROUTE_CANDIDATE: <inferred-project>= on any keeper whose inferred home differs from the current project (decision D8). Sync the =.ai/= mirror. This is the paired workflow edit that lets the wrap-up router find candidates without scanning the standing backlog. (Replaces the superseded atomic-move helper.) + +** Phase 3 — Recommendation engine +Infer destination from item content against the discovered list, with a confidence tier. Pure function =(item, project-list) → (destination, confidence)=. Unit-tested: strong match (destination project named or path present literally → high) , weak match (topic-word overlap only → low, still routed but labeled), no match (stays put, never surfaced), two-project tie (lowest-confidence / tie-break), empty project list (all stay put). The engine is shared by process-inbox's file-time marker (Phase 2) and the wrap-up router (Phase 4), so it lives where both can call it. + +** Phase 4 — Wrap-up step wiring +Add the optional router sub-step to =wrap-it-up.org= Step 3, after the inbox sanity check: surface the candidate batch (one line each: task, destination, delivery mode, confidence), the two options (go / skip). On "go", for each candidate, =inbox-send= a one-task handoff to the destination's =inbox/= and remove the keeper from the local =todo.org=. Empty candidate set = zero interaction (silent). Name the gate-vs-optional split in the prose (the sanity check gates; the router is optional). Sync the =.ai/= mirror. + +** Phase 5 — Transcript routing (vNext, gated on the transcript decision) +Only after the transcript-scope decision resolves. File a recording into the destination =assets/= per =working-files.md=, batch go/skip mirroring the task router. + +* Acceptance criteria +- [ ] At wrap, a filed keeper naming another project is surfaced with that project as the recommended destination. +- [ ] "Go" delivers every recommended item as a one-task =from-<source>= handoff into its destination's =inbox/= and removes it from the local =todo.org=. +- [ ] "Skip" leaves every item in place and the wrap completes cleanly. +- [ ] An empty candidate set produces zero interaction (no prompt, no "0 items" line). +- [ ] A weak (low-confidence) recommendation is visibly labeled in the surfaced list; a no-match item is never surfaced as a route. +- [ ] A candidate whose destination has an =inbox/= but no =todo.org= still delivers (degrades gracefully). +- [ ] A mis-routed handoff is recoverable via =process-inbox='s reject-from-another-project flow, returning it to the source's inbox. +- [ ] The router considers only =:ROUTE_CANDIDATE:=-tagged keepers, never the standing backlog. + +* Readiness dimensions +- Data model & ownership: items are org subtrees; the destination owns the moved task after the move (provenance note records origin). N/A for remote/cached state — all local files. +- Errors, empty states & failure: missing/ambiguous Open Work heading → skip+surface; failed move → atomic no-op; empty routable set → router stays silent (no prompt). +- Security & privacy: N/A — local org files, no credentials or external services. +- Observability: the move shows in the destination's git diff plus the provenance line; the surfaced batch list is the pre-move view. +- Performance & scale: bounded by inbox size (single digits) and project count (tens); no hot path. +- Reuse & lost opportunities: reuses =tc--find-section= and todo-cleanup's subtree-move; widens existing discovery rather than adding a parallel one. +- Architecture fit & weak points: the recommendation engine is the weak point (a wrong-confident destination is the worst failure) — mitigated by the confidence label and reviewable batch list. +- Config surface: possibly a discovery-root list (defaults to =~/projects/=, =~/code/=, matching =inbox-send.py=). Name it if it needs to be user-visible. +- Documentation plan: =wrap-it-up.org= step prose; a note in =cross-project.md= that the router is a sanctioned cross-project write path. +- Dev tooling: ERT for the elisp helper + discovery; the existing =make test= picks up new test files by glob. +- Rollout, compatibility & rollback: additive workflow step; rollback is removing the sub-step. No persisted-data migration. +- External APIs & deps: none. + +* Risks, Rabbit Holes, and Drawbacks +- *Recommendation accuracy is the rabbit hole.* A confidently-wrong destination silently files a task in the wrong project. Dodge: keep the engine conservative, label low confidence, and keep the batch list reviewable before the keystroke. Don't chase a clever inference model in v1. +- *Two inbox-touching steps* (sanity check + router) risk reading as redundant. Dodge: the D1 decision states the gate-vs-optional split in the workflow prose. +- *Scope creep into transcripts* before the source-location question is answered would stall v1. Dodge: transcripts are explicitly vNext behind decision D4. + +* Review dispositions + +Everything in the 2026-06-21 review was accepted, with one modify: + +- *Modified — H1 source-handling.* The review proposed leaving the source keeper in place "until the destination confirms by filing." There is no confirmation callback, so leaving it would duplicate the task once the destination files. Resolved instead (decision D9) to remove the keeper from the *local* =todo.org= on send — a single-file edit in the project we already own and are committing, with =process-inbox='s reject flow as the undo for a mis-route. Keeps the no-foreign-write safety win without the duplication. + +Everything else accepted as written: H1 (inbox-route supersedes direct-move; D2/D3 superseded), H1a (one handoff per task), H1b (reuse =inbox-send= discovery; Phase 1), H2 (tag at file time; D8), M1 (confidence tiers defined in Phase 3 + acceptance), M2 (empty-set silence; acceptance), M3 (paired =process-inbox= edit; Phase 2), M4 (=cross-project.md= note adjusted to "the router uses the sanctioned inbox path"). + +* Review and iteration history + +** 2026-06-13 Sat @ 01:23:13 -0500 — Claude Code (rulesets) — author +- What: initial draft. Problem, goals/scope tiers, two-altitude design, alternatives, six decisions (three DONE from grounding, three TODO for Craig), five implementation phases, acceptance criteria, readiness dimensions, risks. +- Why: the archsetup 2026-06-13 handoff cleared the spec bar in inbox triage and was filed spec-bound rather than applied. This draft turns the proposal into a reviewable design with the open questions isolated as decision tasks. +- Artifacts: proposal source at =docs/design/2026-06-13-wrapup-inbox-transcript-routing-proposal.org=; grounded against =wrap-it-up.org= Step 3, =todo-cleanup.el= =tc--find-section=, and =inbox-send.py= discovery. + +** 2026-06-13 Sat @ 01:36:28 -0500 — Craig Jennings + Claude Code (rulesets) — author +- What: resolved all three open decisions. The router's input is Reading B (filed keepers that belong elsewhere, not raw inbox files), so D1 keeps it a separate sub-step from the inbox gate and D5 keeps it distinct from the defer-and-stage router; D4 defers transcript routing to vNext. Reworked the design (input definition, a candidate-set note bounding the router to session-filed keepers) and Phase 3 to match. Cookie now [6/6]; Status moved to ready-for-review. +- Why: Craig chose Reading B after the A-vs-B input ambiguity surfaced as the root under D1 and D5. Reading B keeps the inbox gate, the router, and defer-and-stage each simple instead of entangling three mechanisms. +- Artifacts: this spec; the candidate-set marking mechanism is the one detail flagged for spec-review to pin. + +** 2026-06-21 Sun @ 01:58:41 -0400 — Claude Code (rulesets) — reviewer +- What: spec-review pass. Rubric *Not ready*, two blocking findings. H1: the inbox-route alternative (inbox-send each routable keeper to the destination's inbox/, let its own process-inbox file it) supersedes the direct-move design — reshape D2, drop Phase 2 and D3's provenance burden. H2: pin the candidate-set marking to Option A (tag =:ROUTE_CANDIDATE:= at process-inbox file time). Four medium findings (M1 confidence tiers, M2 empty-set silence, M3 paired process-inbox edit phase, M4 cross-project.md note). Full review + drop-in implementation tasks in the review file. +- Why: Craig challenged D2 directly (why edit a foreign todo.org rather than use the sanctioned inbox-send path). The review confirmed it: inbox-send already emits the exact provenance D3 reinvents, process-inbox already files per-item with the destination's own gate, cross-project.md sanctions the inbox path, and a verified precondition reverses the spec's assumption — chime and yt-sync have inbox/ but no todo.org, so direct-move silently drops keepers headed there while inbox-route degrades gracefully. +- Artifacts: the review file (since folded into this spec). Next: spec-response to disposition H1/H2 (recommend accept both), which moves the rubric to Ready. + +** 2026-06-21 Sun @ 02:06:37 -0400 — Craig Jennings + Claude Code (rulesets) — responder +- What: folded the spec-review in. Accepted H1 (inbox-route) and H2 (tag at file time); superseded D2 and D3; added D7 (deliver via =inbox-send=), D8 (=:ROUTE_CANDIDATE:= marker at file time), D9 (local source removal + reject-flow recovery). Rewrote Summary, Goals, Design mechanics, Implementation phases (dropped the atomic-move helper — Phase 2 is now the =process-inbox= marker edit), and Acceptance criteria for the inbox-route. One modify (D9) refines H1's vague source-handling. Cookie [9/9]; Status → Ready. +- Why: Craig's inbox-route challenge held up under review — it reuses the sanctioned cross-project path, gets provenance and per-project filing for free, and degrades gracefully where direct-move drops the task. D9 closes the duplication gap the review left open. +- Artifacts: review file deleted on this pass. Next: Phase 6 implementation-task breakdown into =todo.org= on the author's go. |