diff options
| author | Craig Jennings <c@cjennings.net> | 2026-07-19 19:52:13 -0500 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-07-19 19:52:13 -0500 |
| commit | 2e190486c84b28d313ba0a2e096d86bec99d54d6 (patch) | |
| tree | e3a959b013e4b8e8138edd38d20c2202224dcaa6 | |
| parent | 179c495fa7fcb2c2dcf7bdb81b1459afaa0e5075 (diff) | |
| download | rulesets-2e190486c84b28d313ba0a2e096d86bec99d54d6.tar.gz rulesets-2e190486c84b28d313ba0a2e096d86bec99d54d6.zip | |
docs(todo-format): add the measurable-acceptance pattern for open-ended tasks
A task phrased as an absence ("find bugs until none remain," "refactor until nothing worthwhile is left," "clean it up") can't be :solo:, because it fails the verifiable-by-the-agent gate. Absence isn't falsifiable, so "done" is a judgment call, which is what :solo: forbids. This kept surfacing (the ai-launcher-hardening task is the live case) with no written recipe for fixing it.
todo-format.md gains a subsection, right after the :solo:/:quick: definitions, on converting a fuzzy goal into a measurable one: bound the surface (enumerate the units, that's the done-set), net the behavior with characterization tests, disposition every finding (fixed / filed / declined-with-reason, never "looked and it's fine"), and gate on an objective floor (static-analysis clean, suite green, coverage of the enumerated surface). The qualifying answer becomes a dispositioned report, not a claim of perfection, and the honest limit stays honest: "no visible bugs" means "every enumerated path passes its characterization set and clears the audit." Once a task carries those criteria it can carry :solo:.
work-the-backlog's keystone defer item now names open-ended goals as a specific, recognizable failure of test-writability, so a mistagged :solo: task with no writable acceptance test is deferred and routed to get criteria rather than guessed at.
start-work has the same Justify/Approach gate and wants the same pointer, but its skill lives outside rulesets, so that edit is a follow-up in its home.
| -rw-r--r-- | .ai/workflows/work-the-backlog.org | 2 | ||||
| -rw-r--r-- | claude-rules/todo-format.md | 42 | ||||
| -rw-r--r-- | claude-templates/.ai/workflows/work-the-backlog.org | 2 |
3 files changed, 44 insertions, 2 deletions
diff --git a/.ai/workflows/work-the-backlog.org b/.ai/workflows/work-the-backlog.org index 090841d..ab24d3b 100644 --- a/.ai/workflows/work-the-backlog.org +++ b/.ai/workflows/work-the-backlog.org @@ -80,7 +80,7 @@ Task *size* is deliberately absent from this gate. A large but well-specified, d After the scope read, run each eligible candidate through the checklist. Each item is a concrete, answerable question, not an adjective. *Any* hit — or any "unsure" — defers the task. Only a task that clears every item is implemented. -1. *Test-writability (the keystone).* Can I write the failing test from the task text — plus any decisions gathered up front — without inventing a requirement? *No / unsure* → underspecified. Under the speedrun preset, if the gap is one or two quick answerable questions, route it to the pre-flight Q&A; otherwise file a =VERIFY= noting what's missing. Under the unattended loop, file the =VERIFY= (no one to ask). +1. *Test-writability (the keystone).* Can I write the failing test from the task text — plus any decisions gathered up front — without inventing a requirement? *No / unsure* → underspecified. Under the speedrun preset, if the gap is one or two quick answerable questions, route it to the pre-flight Q&A; otherwise file a =VERIFY= noting what's missing. Under the unattended loop, file the =VERIFY= (no one to ask). *Open-ended goals are a specific, recognizable failure of this item:* a task phrased as an absence ("find bugs until none remain," "refactor until nothing worthwhile is left," "clean it up") has no writable acceptance test and so isn't really =:solo:=, even when tagged. Don't guess a stopping point — defer it and note that it needs measurable acceptance criteria (bound the surface, characterization net, dispositioned findings, objective floor — see =todo-format.md='s "Making an open-ended task measurable"). Once those are in the task body, it becomes runnable. 2. *Data-loss / irreversible / external operation.* Does implementing it require any of: =rm= of non-scratch data, =git reset --hard= / force-push, =DROP= / =DELETE= / =TRUNCATE=, file truncate/overwrite of persisted content, a schema or data migration, any external or shared-state mutation, any credential touch? *Yes* → do NOT implement; file a =VERIFY= naming the risk. This is the hard safety gate; an upfront answer never overrides it without an explicit checkpoint. 3. *Already-satisfied.* Does the scope read show the desired end-state already holds? *Yes* → file a =VERIFY= noting it and move on. Don't make a no-op change. 4. *Design deliberation.* Does the task carry an unresolved design question, a "weigh these approaches" with real tradeoffs, or a TBD that isn't a quick factual answer? *Yes* → under the speedrun preset, if it collapses to one or two quick questions, route to the pre-flight Q&A; otherwise file and surface as a =/start-work= candidate. Under the loop, file. The discriminator is *quick-answerable question* vs *deliberation* — never task size. diff --git a/claude-rules/todo-format.md b/claude-rules/todo-format.md index 09b39cf..a4ffc34 100644 --- a/claude-rules/todo-format.md +++ b/claude-rules/todo-format.md @@ -64,6 +64,48 @@ step** in the task-review and task-audit workflows, so the run-time gate can trust the tag. A review or audit that skips the `:solo:`/`:quick:` assessment is incomplete. +### Making an open-ended task measurable (so it can be `:solo:`) + +A task phrased as the *absence* of something — "find bugs until none are +visible," "refactor until no worthwhile opportunities remain," "clean this +up until it's good" — cannot be `:solo:`, because it fails the +*verifiable-by-the-agent* gate. Absence isn't falsifiable: an agent can +always look once more, so "done" is a judgment call, which is exactly what +`:solo:` forbids. The fix is not to drop the task but to give it an +objective completion criterion. Four moves convert a fuzzy goal into a +measurable one: + +1. **Bound the surface.** Enumerate the concrete units the task covers (the + N functions, the M files, the named code paths). The done-set is that + list, not the platonic set of all possible defects. Every claim is made + against the list, so "covered" is checkable where "found everything" + isn't. +2. **Net the behavior.** Bring the surface under characterization tests + (Normal/Boundary/Error per unit — see `testing.md`) before changing + anything. This is the objective floor: writing a characterization test is + mechanical (record what the code does, not what it should), so it scales + across the surface, and it doubles as the safety net that makes any + later refactor falsifiable. +3. **Disposition every finding.** Run the relevant audits (a fixed + footgun/OWASP checklist, `/refactor`, `/review-code`) and give **every** + finding a verdict: fixed, filed as its own task, or declined with a + one-line reason. "Looked and it's fine" is not a disposition. The + measurement is zero undispositioned findings, not zero findings. +4. **Gate on an objective floor.** Static analysis clean (linter, + type-checker, `shellcheck`), the test suite green before and after, and + coverage of the enumerated surface (a per-unit test checklist, or a real + coverage number where the tooling exists). + +The **qualifying answer** is then a dispositioned report — surface split +covered/uncovered, tests before → after, static-analysis result, the audit +matrix fully dispositioned, all green — not a claim of perfection. The +honest limit stays honest ("no visible bugs" means "every enumerated path +passes its characterization set and clears the audit," never "zero bugs +exist"), but the criterion is now falsifiable, which is what lets the task +carry `:solo:`. Write these criteria into the task body at creation or +review time; a task that can't be given them stays non-`:solo:` until it +can. + ### Bug priority from severity × frequency (mandatory where a codebase exists) Some projects carry a codebase — source the project maintains under version diff --git a/claude-templates/.ai/workflows/work-the-backlog.org b/claude-templates/.ai/workflows/work-the-backlog.org index 090841d..ab24d3b 100644 --- a/claude-templates/.ai/workflows/work-the-backlog.org +++ b/claude-templates/.ai/workflows/work-the-backlog.org @@ -80,7 +80,7 @@ Task *size* is deliberately absent from this gate. A large but well-specified, d After the scope read, run each eligible candidate through the checklist. Each item is a concrete, answerable question, not an adjective. *Any* hit — or any "unsure" — defers the task. Only a task that clears every item is implemented. -1. *Test-writability (the keystone).* Can I write the failing test from the task text — plus any decisions gathered up front — without inventing a requirement? *No / unsure* → underspecified. Under the speedrun preset, if the gap is one or two quick answerable questions, route it to the pre-flight Q&A; otherwise file a =VERIFY= noting what's missing. Under the unattended loop, file the =VERIFY= (no one to ask). +1. *Test-writability (the keystone).* Can I write the failing test from the task text — plus any decisions gathered up front — without inventing a requirement? *No / unsure* → underspecified. Under the speedrun preset, if the gap is one or two quick answerable questions, route it to the pre-flight Q&A; otherwise file a =VERIFY= noting what's missing. Under the unattended loop, file the =VERIFY= (no one to ask). *Open-ended goals are a specific, recognizable failure of this item:* a task phrased as an absence ("find bugs until none remain," "refactor until nothing worthwhile is left," "clean it up") has no writable acceptance test and so isn't really =:solo:=, even when tagged. Don't guess a stopping point — defer it and note that it needs measurable acceptance criteria (bound the surface, characterization net, dispositioned findings, objective floor — see =todo-format.md='s "Making an open-ended task measurable"). Once those are in the task body, it becomes runnable. 2. *Data-loss / irreversible / external operation.* Does implementing it require any of: =rm= of non-scratch data, =git reset --hard= / force-push, =DROP= / =DELETE= / =TRUNCATE=, file truncate/overwrite of persisted content, a schema or data migration, any external or shared-state mutation, any credential touch? *Yes* → do NOT implement; file a =VERIFY= naming the risk. This is the hard safety gate; an upfront answer never overrides it without an explicit checkpoint. 3. *Already-satisfied.* Does the scope read show the desired end-state already holds? *Yes* → file a =VERIFY= noting it and move on. Don't make a no-op change. 4. *Design deliberation.* Does the task carry an unresolved design question, a "weigh these approaches" with real tradeoffs, or a TBD that isn't a quick factual answer? *Yes* → under the speedrun preset, if it collapses to one or two quick questions, route to the pre-flight Q&A; otherwise file and surface as a =/start-work= candidate. Under the loop, file. The discriminator is *quick-answerable question* vs *deliberation* — never task size. |
