diff options
Diffstat (limited to '.ai')
| -rw-r--r-- | .ai/metrics/work-the-backlog.jsonl | 6 | ||||
| -rw-r--r-- | .ai/notes.org | 3 | ||||
| -rw-r--r-- | .ai/protocols.org | 7 | ||||
| -rwxr-xr-x | .ai/scripts/cj-remove-block.py | 81 | ||||
| -rw-r--r-- | .ai/scripts/lint-org.el | 53 | ||||
| -rw-r--r-- | .ai/scripts/route_recommend.py | 9 | ||||
| -rw-r--r-- | .ai/scripts/tests/test-lint-org.el | 108 | ||||
| -rw-r--r-- | .ai/scripts/tests/test-todo-cleanup.el | 103 | ||||
| -rw-r--r-- | .ai/scripts/tests/test_cj_remove_block.py | 167 | ||||
| -rw-r--r-- | .ai/scripts/tests/test_route_recommend.py | 28 | ||||
| -rw-r--r-- | .ai/scripts/todo-cleanup.el | 34 | ||||
| -rw-r--r-- | .ai/sessions/2026-07-24-18-05-sentry-implement-pass-review-and-hook-fixes.org (renamed from .ai/session-context.org) | 174 | ||||
| -rw-r--r-- | .ai/sessions/2026-07-25-09-24-invalid-block-filter-push-task-review.org | 83 | ||||
| -rw-r--r-- | .ai/sessions/2026-07-25-15-34-clean-wrap-and-inbox-safe-sync.org | 85 | ||||
| -rw-r--r-- | .ai/workflows/startup.org | 27 | ||||
| -rw-r--r-- | .ai/workflows/work-the-backlog.org | 2 | ||||
| -rw-r--r-- | .ai/workflows/wrap-it-up.org | 58 |
17 files changed, 994 insertions, 34 deletions
diff --git a/.ai/metrics/work-the-backlog.jsonl b/.ai/metrics/work-the-backlog.jsonl index f7bb44b..88c6764 100644 --- a/.ai/metrics/work-the-backlog.jsonl +++ b/.ai/metrics/work-the-backlog.jsonl @@ -7,3 +7,9 @@ {"ts":"2026-07-23T23:57:59-05:00","run_id":"711400a5-c6bc-4684-b928-6b818d0d33f2","project":"rulesets","caller":"speedrun","task":"inbox-send two smaller defects","outcome":"implemented-committed","upfront_decision":false,"commit_sha":"a053e9d","review_findings":0} {"ts":"2026-07-24T00:02:14-05:00","run_id":"711400a5-c6bc-4684-b928-6b818d0d33f2","project":"rulesets","caller":"speedrun","task":"lint-org todo-format checkers fire on specs","outcome":"implemented-committed","upfront_decision":true,"commit_sha":"c38bab9","review_findings":0} {"ts":"2026-07-24T00:02:49-05:00","run_id":"711400a5-c6bc-4684-b928-6b818d0d33f2","project":"rulesets","caller":"speedrun","task":"notes.org template four lint flags","outcome":"already-satisfied","defer_reason":"already-satisfied","upfront_decision":false,"commit_sha":"","review_findings":0} +{"ts":"2026-07-24T01:41:14-05:00","run_id":"sentry-fire2-1784875274","project":"rulesets","caller":"loop","task":"cj-remove-block over-deletion + unsafe write","outcome":"implemented-committed","upfront_decision":false,"commit_sha":"17f5d48","review_findings":0} +{"ts":"2026-07-24T01:43:52-05:00","run_id":"sentry-fire2","project":"rulesets","caller":"loop","task":"route_recommend duplicate-name tier downgrade","outcome":"implemented-committed","upfront_decision":false,"commit_sha":"1b0f284","review_findings":0} +{"ts":"2026-07-24T02:38:48-05:00","run_id":"sentry-fire3","project":"rulesets","caller":"loop","task":"audit.bats flaky teardown","outcome":"implemented-committed","upfront_decision":false,"commit_sha":"7f45d4b","review_findings":0} +{"ts":"2026-07-24T03:39:16-05:00","run_id":"sentry-fire4","project":"rulesets","caller":"loop","task":"todo-cleanup missing backup","outcome":"implemented-committed","upfront_decision":false,"commit_sha":"0686784","review_findings":0} +{"ts":"2026-07-24T04:36:09-05:00","run_id":"sentry-fire5","project":"rulesets","caller":"loop","task":"attachment filename sanitization","outcome":"deferred-verify","defer_reason":"needs-deliberation","upfront_decision":false,"commit_sha":"","review_findings":0} +{"ts":"2026-07-24T07:38:47-05:00","run_id":"sentry-fire8","project":"rulesets","caller":"loop","task":"bin/ lint coverage gap","outcome":"implemented-committed","upfront_decision":false,"commit_sha":"f91feef","review_findings":1} diff --git a/.ai/notes.org b/.ai/notes.org index 5ac74f0..447c33c 100644 --- a/.ai/notes.org +++ b/.ai/notes.org @@ -78,10 +78,11 @@ Format: :COMMIT_AUTONOMY: yes :LOOP_MAY_COMMIT: yes +:SENTRY_MAY_IMPLEMENT: yes :LAST_SPEC_SORT: 2026-07-02 Markers maintained by workflows to record when they last ran. Read by other workflows that gate their behavior on freshness. :LAST_AUDIT: 2026-07-20 (open set current — this session's shipped work (working/temp, triage-source-activation, silent-until-signal, suspend detach) closed as it went; sentry cluster consolidated (merged the /schedule tasks, added cross-host-coordination); nothing shipped-but-open per git reconcile. Live finding: the Polyglot + Subprojects scouting tasks are SCHEDULED 2026-07-20 and due.) -:LAST_INBOX_PROCESS: 2026-07-20 (morning sentry review + build session: triage migration confirmations from home + work processed; earlier 2026-07-18 pass folded in) +:LAST_INBOX_PROCESS: 2026-07-25 (consolidated home + work Claude-to-Codex MCP registry proposals into one [#B] parked spec decision; memory auditor split from the registry work) Format: one =:MARKER: YYYY-MM-DD= line per workflow. Workflows overwrite their own marker on completion. diff --git a/.ai/protocols.org b/.ai/protocols.org index 3cd02ef..2ea041b 100644 --- a/.ai/protocols.org +++ b/.ai/protocols.org @@ -84,7 +84,7 @@ Do NOT estimate, guess, or rely on memory. Just run the command. It takes one se Every session pulls rulesets first, then the local project repo. Rulesets carries the canonical behavioral rules and =.ai/= templates (the old =claude-templates= repo is folded in as a subtree at =rulesets/claude-templates/=); the project pull lands commits pushed from other machines or teammates since the last session. -Resolve any dirty-tree or merge issue at each step before moving on. Both pulls run as =git pull --ff-only= (or =git merge --ff-only= against the fetched upstream), so anything non-trivial — non-fast-forward history, dirty working tree, diverged branches — aborts. Surface the state and stop. Never auto-stash, auto-merge, or auto-rebase; the user resolves the conflict before further work. +Resolve any sync-blocking tree or merge issue at each step before moving on. The shared =git-worktree-gate sync-safe= policy permits untracked deliveries beneath =inbox/= so receiving a handoff never prevents another project from refreshing rulesets; every staged or tracked change, dirty submodule, Git operation in progress, or untracked path outside =inbox/= blocks. Both pulls run as =git pull --ff-only= (or =git merge --ff-only= against the fetched upstream), so non-fast-forward history and diverged branches also abort. Surface the state and stop. Never auto-stash, auto-merge, or auto-rebase; the user resolves the conflict before further work. Mechanics live in =startup.org= Phase A.0. The rule lives here because it governs the very first action of every session: load the freshest behavioral rules and templates before anything else runs. @@ -567,12 +567,13 @@ When monitoring a long-running process (rsync, large downloads, builds, VM tests ** "Wrap it up" / "That's a wrap" / "Let's call it a wrap" -When Craig says any of these phrases (or variations), execute the wrap-up workflow: [[file:workflows/wrap-it-up.org][wrap-it-up.org]]. Four steps: +When Craig says any of these phrases (or variations), execute the wrap-up workflow: [[file:workflows/wrap-it-up.org][wrap-it-up.org]]. Five load-bearing steps: 1. *Finalize the Summary* in =.ai/session-context.org= (populate the 5 subsections from the Session Log) 2. *Rename* =.ai/session-context.org= → =.ai/sessions/YYYY-MM-DD-HH-MM-description.org= 3. *Git commit + push* to all remotes (see Git Commit Requirements) -4. *Valediction* — brief, warm, specific closing +4. *Certify the clean tree* with =git-worktree-gate certify=. Any remaining staged, unstaged, untracked, submodule, or in-progress-operation state blocks wrap entirely; report each path and the exact decision needed. There is no dirty-file deferral. +5. *Valediction* — brief, warm, specific closing, reachable only after certification The absence of =.ai/session-context.org= after wrap-up is the signal that the session ended cleanly. If the file is still there at the next session start, the previous session was interrupted. diff --git a/.ai/scripts/cj-remove-block.py b/.ai/scripts/cj-remove-block.py index 71c7b3d..d5137a3 100755 --- a/.ai/scripts/cj-remove-block.py +++ b/.ai/scripts/cj-remove-block.py @@ -16,8 +16,12 @@ Companion to the /respond-to-cj-comments skill and to cj-scan.py. from __future__ import annotations import argparse +import os import re +import shutil import sys +import tempfile +from datetime import datetime from pathlib import Path SRC_OPEN_RE = re.compile(r"^\s*#\+begin_src\s+cj:", re.IGNORECASE) @@ -57,12 +61,83 @@ def looks_like_cj_range(lines: list[str], start: int, end: int) -> tuple[bool, s f"Line {end} does not look like a #+end_src closing fence " f"(got: {last[:60]!r})" ) + + # The range must hold exactly ONE block. Checking only the first and last + # lines let a drifted range run from one block's opener to a *later* block's + # closer: validation passed and the removal silently deleted everything + # between, prose and headings included. Drift is the case this check exists + # for, so it has to look inside the range, not just at its ends. + for offset, line in enumerate(lines[start:end - 1], start=start + 1): + if SRC_CLOSE_RE.match(line): + return False, ( + f"Range {start}..{end} covers more than one cj block — " + f"a #+end_src appears at line {offset}, before the range ends. " + f"Re-scan for current line numbers; removing this range would " + f"delete everything between the two blocks." + ) + if SRC_OPEN_RE.match(line): + return False, ( + f"Range {start}..{end} covers more than one cj block — " + f"a second #+begin_src cj: appears at line {offset}. " + f"Re-scan for current line numbers." + ) return True, "" +def _backup(path: Path) -> Path: + """Copy path to /tmp before mutating it, mirroring lint-org.el's convention. + + These are Craig's org files. lint-org.el, the other tool that rewrites them, + leaves a /tmp copy before touching anything; this matches it so a bad edit is + always recoverable without reaching for git (which only reaches the last + commit, losing intra-session work). + """ + stamp = datetime.now().strftime("%Y%m%d-%H%M%S") + base = Path(tempfile.gettempdir()) / f"{path.name}.before-cj-remove.{stamp}" + # Never overwrite an earlier backup. The skill removes several annotations + # in quick succession, so a second-resolution stamp collides and the later + # copy would replace the earlier one with already-mutated content — losing + # the pre-session original the backup exists to preserve. + dest = base + n = 2 + while dest.exists(): + dest = base.with_name(f"{base.name}-{n}") + n += 1 + shutil.copy2(path, dest) + return dest + + +def _atomic_write(path: Path, text: str) -> None: + """Write text to path via a temp sibling and os.replace. + + A bare write_text truncates the target on open, so a mid-write failure left + the org file truncated with no complete copy on disk. Writing a temp sibling + and renaming means the file is either its old content or its new content, + never a partial. + """ + # Follow a symlink to the file it names. os.replace would otherwise swap the + # symlink itself for a regular file, leaving the real target holding the old + # content — the edit silently goes nowhere. Resolving also puts the temp + # sibling on the same filesystem as the real file, which os.replace needs. + path = path.resolve() + fd, tmp = tempfile.mkstemp(dir=path.parent, prefix=f".{path.name}.", suffix=".tmp") + os.close(fd) + tmp_path = Path(tmp) + # Carry the original's permissions across. mkstemp creates 0600, and + # defaulting to the umask instead widened a deliberately-restricted file + # (a 0600 org file came back 0644). + shutil.copymode(path, tmp_path) + try: + tmp_path.write_text(text, encoding="utf-8") + os.replace(tmp_path, path) + except BaseException: + tmp_path.unlink(missing_ok=True) + raise + + def remove_range(path: Path, start: int, end: int) -> None: """Read path, validate range looks like cj content, remove the range, write back.""" - text = path.read_text() + text = path.read_text(encoding="utf-8") had_trailing_newline = text.endswith("\n") lines = text.splitlines(keepends=False) @@ -77,7 +152,9 @@ def remove_range(path: Path, start: int, end: int) -> None: new_text += "\n" elif not new_lines and had_trailing_newline: new_text = "" - path.write_text(new_text) + + _backup(path) + _atomic_write(path, new_text) def main() -> int: diff --git a/.ai/scripts/lint-org.el b/.ai/scripts/lint-org.el index 928f83b..33dc52f 100644 --- a/.ai/scripts/lint-org.el +++ b/.ai/scripts/lint-org.el @@ -306,6 +306,52 @@ Craig-specific annotation marker rather than Babel src-block syntax." (lo--goto-line line) (looking-at-p "^[ \t]*#\\+begin_src[ \t]+cj:"))) +(defvar-local lo--matched-blocks-cache nil + "Cons of (TICK . REGIONS) memoizing `lo--matched-block-regions'. +TICK is the `buffer-chars-modified-tick' the regions were computed at, so a +fix applied mid-pass invalidates them.") + +(defun lo--matched-block-regions () + "Return ((BEGIN-LINE . END-LINE) ...) for every correctly paired block. +Scans lines directly rather than asking org, because org's own parser is what +mis-reads these blocks: a heading-shaped line inside a verbatim body reads as a +structural break and loses the open block. The scan applies org's real rule — +once a block is open, only its own `#+end_TYPE' closes it, so a nested +`#+begin_' or a foreign `#+end_' in the body is just text." + (let ((tick (buffer-chars-modified-tick))) + (if (eql (car lo--matched-blocks-cache) tick) + (cdr lo--matched-blocks-cache) + (let ((case-fold-search t) + (regions nil) (open-type nil) (open-line nil) (line 0)) + (save-excursion + (goto-char (point-min)) + (while (not (eobp)) + (setq line (1+ line)) + (let ((text (buffer-substring-no-properties + (line-beginning-position) (line-end-position)))) + (cond + (open-type + (when (string-match + (format "\\`[ \t]*#\\+end_%s[ \t]*\\'" + (regexp-quote open-type)) + text) + (push (cons open-line line) regions) + (setq open-type nil open-line nil))) + ((string-match "\\`[ \t]*#\\+begin_\\([^ \t\n]+\\)" text) + (setq open-type (match-string 1 text) + open-line line)))) + (forward-line 1))) + (setq lo--matched-blocks-cache (cons tick (nreverse regions))) + (cdr lo--matched-blocks-cache))))) + +(defun lo--in-matched-block-p (line) + "Non-nil when LINE sits within a correctly paired block, delimiters included. +org-lint reports `invalid-block' at the delimiter lines themselves, so the +range has to be inclusive for the suppression to reach them." + (cl-some (lambda (region) + (and (>= line (car region)) (<= line (cdr region)))) + (lo--matched-block-regions))) + (defun lo--handle-item (item) (let ((name (lo--checker-name item)) (line (lo--line item)) @@ -318,6 +364,13 @@ Craig-specific annotation marker rather than Babel src-block syntax." wrong-header-argument)) (lo--cj-comment-block-opener-p line)) nil) + ;; `invalid-block' on a block that is in fact correctly paired — the + ;; checker is org-lint's own, so this filters its output rather than + ;; fixing a local checker. A genuinely unterminated block isn't in any + ;; matched region, so it still reports. + ((and (eq name 'invalid-block) + (lo--in-matched-block-p line)) + nil) ((eq name 'item-number) (lo--apply-or-preview name line msg #'lo-fix-item-number)) ((eq name 'missing-language-in-src-block) diff --git a/.ai/scripts/route_recommend.py b/.ai/scripts/route_recommend.py index 7b36405..12ab132 100644 --- a/.ai/scripts/route_recommend.py +++ b/.ai/scripts/route_recommend.py @@ -71,6 +71,15 @@ def recommend(item: str, projects: list[str]) -> tuple[str | None, str]: if not projects: return (None, "none") + # Collapse identical names first. Projects are addressed by bare basename, so + # two projects sharing one across roots (~/code/notes, ~/projects/notes) arrive + # twice; both literal-match, and the tie test below then read that as ambiguity + # and downgraded a correct strong match to weak. Deduping here rather than in + # discover_destination_names protects every caller of the pure core, not just + # the CLI path. Order-preserving, and it collapses only identical names — two + # *different* projects matching is real ambiguity and still downgrades. + projects = list(dict.fromkeys(projects)) + item_lower = item.lower() item_tokens = _tokens(item) diff --git a/.ai/scripts/tests/test-lint-org.el b/.ai/scripts/tests/test-lint-org.el index d398c4c..ceee209 100644 --- a/.ai/scripts/tests/test-lint-org.el +++ b/.ai/scripts/tests/test-lint-org.el @@ -193,6 +193,65 @@ real suspicious-language warning here #+end_src ") +;; invalid-block, false-positive case — a correctly paired example block whose +;; body holds a heading-shaped line. org's parser reads the `** ' inside the +;; verbatim body as a structural break, loses the open block, and flags BOTH +;; delimiters as "Possible incomplete block". +(defconst lo-test--verbatim-heading-block "\ +* Heading + +#+begin_example +** Feature Name or Topic +Body line. +#+end_example + +Trailing prose. +") + +;; invalid-block, literal-delimiter case — a paired src block whose body holds +;; a literal `#+end_example' plus a heading-shaped line. Only `#+end_src' +;; closes a src block, so all three findings here are false. +(defconst lo-test--literal-end-in-src "\ +* Heading + +#+begin_src text +#+end_example +** heading shaped +#+end_src +") + +;; invalid-block, uppercase-delimiter case — org accepts #+BEGIN_/#+END_ in +;; either case, and the pre-fix script flagged both delimiters here too. +(defconst lo-test--uppercase-verbatim-block "\ +* Heading + +#+BEGIN_EXAMPLE +** heading shaped +#+END_EXAMPLE +") + +;; invalid-block, genuine case — a block that really is never closed. The +;; suppression must not reach this one. +(defconst lo-test--unterminated-block "\ +* Heading + +#+begin_example +truly unterminated block body +") + +;; A genuinely unterminated block *after* a correctly paired one — verifies the +;; suppression is scoped per block rather than per file. +(defconst lo-test--paired-then-unterminated "\ +* Heading + +#+begin_example +** heading shaped +#+end_example + +#+begin_example +never closed +") + ;; Mixed fixture — each category once. (defconst lo-test--mixed "\ * Mixed @@ -392,6 +451,55 @@ suspicious-language judgment." (should (= 1 suspicious)))) ;;; --------------------------------------------------------------------------- +;;; invalid-block — false positives on correctly paired verbatim blocks + +(ert-deftest lo-verbatim-heading-block-emits-no-invalid-block () + "Normal: a paired example block containing a heading-shaped body line emits +no invalid-block judgment. Both delimiters are flagged by org-lint because the +parser treats the `** ' inside the verbatim body as a structural break." + (let* ((out (lo-test--run lo-test--verbatim-heading-block)) + (res (plist-get out :result)) + (judgments (lo-test--judgments (plist-get out :issues)))) + ;; File untouched, no fixes applied — suppression only, never a rewrite. + (should (equal lo-test--verbatim-heading-block res)) + (should (= 0 (plist-get out :fixes))) + (should-not (member 'invalid-block (lo-test--checkers judgments))))) + +(ert-deftest lo-literal-end-delimiter-in-src-emits-no-invalid-block () + "Boundary: a paired src block whose body holds a literal `#+end_example' and +a heading-shaped line emits no invalid-block judgment. Only `#+end_src' closes +a src block, so the interior delimiter is body text." + (let* ((out (lo-test--run lo-test--literal-end-in-src)) + (judgments (lo-test--judgments (plist-get out :issues)))) + (should-not (member 'invalid-block (lo-test--checkers judgments))))) + +(ert-deftest lo-uppercase-verbatim-block-emits-no-invalid-block () + "Boundary: block delimiters are case-insensitive in org, so an uppercase +`#+BEGIN_EXAMPLE' pair is suppressed the same as a lowercase one." + (let* ((out (lo-test--run lo-test--uppercase-verbatim-block)) + (judgments (lo-test--judgments (plist-get out :issues)))) + (should-not (member 'invalid-block (lo-test--checkers judgments))))) + +(ert-deftest lo-unterminated-block-still-emits-invalid-block () + "Error: a block that is never closed still emits its invalid-block judgment. +This is the finding the checker exists for — the suppression must not mask it." + (let* ((out (lo-test--run lo-test--unterminated-block)) + (judgments (lo-test--judgments (plist-get out :issues)))) + (should (member 'invalid-block (lo-test--checkers judgments))))) + +(ert-deftest lo-invalid-block-suppression-is-scoped-per-block () + "Boundary: a paired block and an unterminated block in the same file — the +paired one is suppressed and the unterminated one still reports. Exactly one +invalid-block judgment, and it points at the unterminated opener (line 7)." + (let* ((out (lo-test--run lo-test--paired-then-unterminated)) + (judgments (lo-test--judgments (plist-get out :issues))) + (invalid (cl-remove-if-not + (lambda (i) (eq (plist-get i :checker) 'invalid-block)) + judgments))) + (should (= 1 (length invalid))) + (should (= 7 (plist-get (car invalid) :line))))) + +;;; --------------------------------------------------------------------------- ;;; --check mode (ert-deftest lo-check-mode-does-not-modify-file () diff --git a/.ai/scripts/tests/test-todo-cleanup.el b/.ai/scripts/tests/test-todo-cleanup.el index 838913f..a92d238 100644 --- a/.ai/scripts/tests/test-todo-cleanup.el +++ b/.ai/scripts/tests/test-todo-cleanup.el @@ -516,6 +516,12 @@ gitignore todo.org, then run `--archive-done' aging with the DEFAULT archive pat .gitignore contents or nil), :archive-ignored (whether git ignores the archive), :archive-exists." (let* ((root (make-temp-file "tc-git-" t)) + ;; Private backup dir: this helper writes a file literally named + ;; todo.org and runs a real (non-check) pass, so without this its + ;; backup lands in the shared temp dir under the exact production + ;; name and is indistinguishable from a real one. + (temporary-file-directory + (file-name-as-directory (make-temp-file "tc-git-bk-" t))) (todo (expand-file-name "todo.org" root)) (archive (expand-file-name "archive/task-archive.org" root)) (gi (expand-file-name ".gitignore" root))) @@ -536,7 +542,8 @@ gitignore todo.org, then run `--archive-done' aging with the DEFAULT archive pat :archive-ignored (eq 0 (call-process "git" nil nil nil "check-ignore" "-q" archive)) :archive-exists (file-readable-p archive))) - (delete-directory root t)))) + (delete-directory root t) + (delete-directory temporary-file-directory t)))) (ert-deftest tc-age-self-protect-gitignores-archive-when-todo-ignored () "When the todo file is gitignored, the aged-out archive is added to .gitignore @@ -1073,3 +1080,97 @@ line) is left untouched — the strip stops at the first non-planning line." (provide 'test-todo-cleanup) ;;; test-todo-cleanup.el ends here + +;;; --------------------------------------------------------------------------- +;;; Backup before mutating (parity with lint-org.el / wrap-org-table.el) +;; +;; todo-cleanup rewrites todo.org in place and left no copy behind, while both +;; sibling org-mutators back up to /tmp first. It is also the one that runs most +;; often (every wrap, every sentry fire). Emacs's own backup does not fire under +;; --batch -q, so there was genuinely no undo short of git. + +(ert-deftest tc-backup-written-before-a-real-mutation () + "A real (non-check) run leaves a copy holding the pre-edit content. + +`temporary-file-directory' is rebound to a private dir for the duration: the +backup name derives from the *file's* basename, and the real todo.org shares +that basename, so a live sentry run writing /tmp/todo.org.before-todo-cleanup.* +would otherwise be indistinguishable from this test's own artifact. The first +version of this test globbed the shared /tmp and passed only until a real run +created one (2026-07-24)." + (let* ((dir (make-temp-file "tc-backup-" t)) + (bdir (file-name-as-directory (make-temp-file "tc-bk-" t))) + (file (expand-file-name "todo.org" dir)) + (before "* P Open Work\n** TODO [#B] parent\n*** DONE a subtask\nCLOSED: [2026-07-01 Tue]\n")) + (unwind-protect + (progn + (with-temp-file file (insert before)) + (let ((tc-check-only nil) + (tc-convert-subtasks t) + (temporary-file-directory bdir)) + (tc-process-file file)) + (let ((backups (file-expand-wildcards + (concat bdir "todo.org.before-todo-cleanup.*")))) + (should backups) + (should (string-match-p + "a subtask" + (with-temp-buffer (insert-file-contents (car backups)) + (buffer-string)))))) + (delete-directory dir t) + (delete-directory bdir t)))) + +(ert-deftest tc-no-backup-in-check-mode () + "--check writes nothing, so it must not leave a backup either. +Uses a private `temporary-file-directory' for the same isolation reason." + (let* ((dir (make-temp-file "tc-backup-" t)) + (bdir (file-name-as-directory (make-temp-file "tc-bk-" t))) + (file (expand-file-name "todo.org" dir))) + (unwind-protect + (progn + (with-temp-file file + (insert "* P Open Work\n** TODO [#B] parent\n*** DONE sub\nCLOSED: [2026-07-01 Tue]\n")) + (let ((tc-check-only t) + (tc-convert-subtasks t) + (temporary-file-directory bdir)) + (tc-process-file file)) + (should-not (file-expand-wildcards + (concat bdir "todo.org.before-todo-cleanup.*")))) + (delete-directory dir t) + (delete-directory bdir t)))) + +(ert-deftest tc-backup-never-overwrites-an-earlier-one () + "Two invocations in the same second must not collapse to one backup. + +open-tasks.org runs --convert-subtasks then --archive-done back to back, each +a sub-second batch run. With a second-resolution stamp and copy-file's +OK-IF-ALREADY-EXISTS, the second invocation overwrote the first's backup with +already-mutated content, so the true pre-session original was unrecoverable — +the exact state the backup exists to preserve (found 2026-07-24 in review)." + (let* ((dir (make-temp-file "tc-collide-" t)) + (bdir (file-name-as-directory (make-temp-file "tc-cbk-" t))) + (file (expand-file-name "todo.org" dir)) + (original (concat "* P Open Work\n** TODO [#B] parent\n*** DONE sub\n" + "CLOSED: [2026-07-01 Tue]\n" + "* P Resolved\n** DONE [#C] old\nCLOSED: [2025-01-01 Wed]\n"))) + (unwind-protect + (progn + (with-temp-file file (insert original)) + ;; Two back-to-back invocations, as the shipped workflow does. + (let ((temporary-file-directory bdir)) + (let ((tc-check-only nil) (tc-convert-subtasks t)) + (tc-process-file file)) + (let ((tc-check-only nil) (tc-convert-subtasks nil) (tc-archive-done t) + (tc-archive-retain-days nil)) + (tc-process-file file))) + (let ((backups (file-expand-wildcards + (concat bdir "todo.org.before-todo-cleanup.*")))) + ;; Both invocations kept their own backup. + (should (= (length backups) 2)) + ;; And one of them still holds the true original. + (should (cl-some (lambda (b) + (string= original + (with-temp-buffer (insert-file-contents b) + (buffer-string)))) + backups)))) + (delete-directory dir t) + (delete-directory bdir t)))) diff --git a/.ai/scripts/tests/test_cj_remove_block.py b/.ai/scripts/tests/test_cj_remove_block.py index 2c8dade..3cdee46 100644 --- a/.ai/scripts/tests/test_cj_remove_block.py +++ b/.ai/scripts/tests/test_cj_remove_block.py @@ -14,6 +14,34 @@ import pytest SCRIPT = Path(__file__).parent.parent / "cj-remove-block.py" +@pytest.fixture(autouse=True) +def isolated_tmpdir(tmp_path, monkeypatch): + """Give every test in this module a private TMPDIR. + + The script backs up to the system temp dir under a name derived from the + edited file's BASENAME. The real todo.org shares that basename, so any test + operating on a fixture named todo.org writes something indistinguishable + from a production backup — and an earlier version of this file globbed the + shared /tmp and unlinked every match, so a routine `make test` destroyed + Craig's real backups (found in review, 2026-07-24). + + Isolating at module scope rather than per-test is deliberate: the same bug + was fixed once in the elisp sibling and left here, so relying on each new + test to remember is exactly how it recurred. Autouse makes it structural. + """ + d = tmp_path / "_tmpdir" + d.mkdir() + # TMPDIR covers subprocess invocations of the script. + monkeypatch.setenv("TMPDIR", str(d)) + # tempfile.gettempdir() caches its answer on first call, so a test that + # loads the module in-process would keep writing to the real /tmp no matter + # what TMPDIR says. Override the cache too — this is the gap that made the + # env-var-only version still leak one backup per suite run. + import tempfile as _tempfile + monkeypatch.setattr(_tempfile, "tempdir", str(d)) + return d + + @pytest.fixture def run_remove(tmp_path): """Write content to a temp org file, run cj-remove-block, return new contents.""" @@ -155,3 +183,142 @@ class TestCjRemoveBlockSafety: err, post_content = run_remove_expecting_failure(original, start=4, end=2) assert err.returncode != 0 assert post_content == original + + +class TestMultiBlockRangeRefused: + """The validation exists to catch a drifted range, but it only checked the + first and last lines of that range. A span from one block's opening fence to + a LATER block's closing fence passed, and the removal silently deleted every + line between — real prose, headings, whole tasks — with a zero exit. Drift is + the skill's normal operating mode (respond-to-cj-comments edits the file as it + processes, and a file under cj review usually holds several blocks), so this + is the exact scenario the check was written for. Reproduced 2026-07-24.""" + + TWO_BLOCKS = ( + "* Alpha\n" + "#+begin_src cj:\n" + "note A\n" + "#+end_src\n" + "KEEP THIS LINE\n" + "* Beta\n" + "#+begin_src cj:\n" + "note B\n" + "#+end_src\n" + ) + + def test_range_spanning_two_blocks_is_refused(self, run_remove_expecting_failure): + # Lines 2..9: block one's opener through block two's closer. + err, content = run_remove_expecting_failure(self.TWO_BLOCKS, 2, 9) + assert err.returncode == 1 + assert "KEEP THIS LINE" in content, "content between the blocks was destroyed" + assert "* Beta" in content, "a heading between the blocks was destroyed" + + def test_refusal_names_the_reason(self, run_remove_expecting_failure): + err, _ = run_remove_expecting_failure(self.TWO_BLOCKS, 2, 9) + assert "more than one" in err.stderr.decode().lower() + + def test_a_correct_single_block_range_still_removes(self, run_remove): + # The fix must not over-tighten: the legitimate range still works. + out = run_remove(self.TWO_BLOCKS, 2, 4) + assert "note A" not in out + assert "KEEP THIS LINE" in out + assert "note B" in out, "the second block must be untouched" + + def test_a_nested_end_src_inside_the_range_is_refused(self, run_remove_expecting_failure): + # Any #+end_src before the final line means the range covers >1 block. + content = ( + "#+begin_src cj:\n" + "a\n" + "#+end_src\n" + "middle\n" + "#+begin_src cj:\n" + "b\n" + "#+end_src\n" + ) + err, after = run_remove_expecting_failure(content, 1, 7) + assert err.returncode == 1 + assert "middle" in after + + +class TestSafeMutation: + """The script rewrites Craig's org files (todo.org, notes.org). It wrote with + a bare write_text, which truncates the target on open, and took no backup — + so a mid-write failure left the file truncated with no copy to recover from. + lint-org.el, the other tool that mutates these files, backs up to a temp dir + first. Match that, and make the write atomic. + + Every test here redirects TMPDIR to a private directory. The backup name + derives from the file's basename, and the real todo.org shares it, so a test + globbing the shared temp dir cannot tell its own artifact from a genuine + backup — and an earlier version of this class globbed /tmp and unlinked every + match, so a routine `make test` destroyed real backups (found in review, + 2026-07-24). Never glob or delete across the shared temp dir.""" + + ONE_BLOCK = "* T\n#+begin_src cj:\nnote\n#+end_src\nkeep\n" + + def test_a_backup_is_written_before_mutating(self, tmp_path): + import subprocess, glob, os + bdir = tmp_path / "bk" + bdir.mkdir() + f = tmp_path / "todo.org" + f.write_text(self.ONE_BLOCK) + subprocess.run( + ["python3", str(SCRIPT), "--file", str(f), "--start", "2", "--end", "4"], + check=True, capture_output=True, + env={**os.environ, "TMPDIR": str(bdir)}, + ) + backups = glob.glob(str(bdir / "todo.org.before-cj-remove.*")) + assert backups, "no backup was written before mutating the org file" + assert "note" in Path(max(backups)).read_text() + + def test_no_partial_file_when_the_write_fails(self, tmp_path, monkeypatch): + import importlib.util + spec = importlib.util.spec_from_file_location("crb", SCRIPT) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + bdir = tmp_path / "bk" + bdir.mkdir() + monkeypatch.setenv("TMPDIR", str(bdir)) + f = tmp_path / "todo.org" + f.write_text(self.ONE_BLOCK) + def boom(*a, **k): + raise OSError("disk full") + monkeypatch.setattr(mod.os, "replace", boom) + with pytest.raises(OSError): + mod.remove_range(f, 2, 4) + # The original survives intact — no truncation, no partial. + assert f.read_text() == self.ONE_BLOCK + + +class TestBackupNeverOverwrites: + """Same defect class as todo-cleanup's, and more reachable here: the + respond-to-cj-comments skill removes several annotations in quick + succession, so a second-resolution stamp collides and the later backup + overwrote the earlier one with already-mutated content.""" + + TWO_BLOCKS = ( + "* A\n#+begin_src cj:\nfirst\n#+end_src\n" + "* B\n#+begin_src cj:\nsecond\n#+end_src\n" + ) + + def test_consecutive_removals_each_keep_a_backup(self, tmp_path, monkeypatch): + import subprocess, glob + bdir = tmp_path / "bk" + bdir.mkdir() + monkeypatch.setenv("TMPDIR", str(bdir)) + f = tmp_path / "todo.org" + f.write_text(self.TWO_BLOCKS) + original = f.read_text() + # Remove the second block, then the first — back to back, same second. + subprocess.run(["python3", str(SCRIPT), "--file", str(f), + "--start", "6", "--end", "8"], + check=True, capture_output=True, + env={**__import__("os").environ, "TMPDIR": str(bdir)}) + subprocess.run(["python3", str(SCRIPT), "--file", str(f), + "--start", "2", "--end", "4"], + check=True, capture_output=True, + env={**__import__("os").environ, "TMPDIR": str(bdir)}) + backups = glob.glob(str(bdir / "todo.org.before-cj-remove.*")) + assert len(backups) == 2, f"expected 2 backups, got {len(backups)}" + contents = [Path(b).read_text() for b in backups] + assert original in contents, "no backup holds the true original" diff --git a/.ai/scripts/tests/test_route_recommend.py b/.ai/scripts/tests/test_route_recommend.py index acc4755..2ec900a 100644 --- a/.ai/scripts/tests/test_route_recommend.py +++ b/.ai/scripts/tests/test_route_recommend.py @@ -122,3 +122,31 @@ def test_cli_exclude_drops_current_project(tmp_path): r = _run(["--exclude", "foo"], roots=[tmp_path / "projects"], item="fix the foo widget") assert r.returncode == 0 assert r.stdout.strip() == "none" + + +# ---------------------------------------------------------------------- +# Duplicate candidate names +# +# Projects are collapsed to bare basenames, so two projects sharing a basename +# across roots (~/code/notes and ~/projects/notes) appear twice in the candidate +# list. Both literal-match, recommend read len(strong) > 1 as an ambiguous tie, +# and a correct strong match was downgraded to weak. Latent when discovered +# 2026-07-24 (27 projects, 27 distinct basenames) but real. +# ---------------------------------------------------------------------- + +def test_duplicate_candidate_name_keeps_strong_confidence(): + assert rr.recommend("fix the notes thing", ["notes", "other"]) == ("notes", "strong") + # The same name twice must not read as a tie. + assert rr.recommend("fix the notes thing", ["notes", "notes", "other"]) == ("notes", "strong") + + +def test_genuine_ambiguity_still_downgrades(): + # Two DIFFERENT projects both matching is a real tie and stays weak — the + # dedupe must collapse identical names only, never real ambiguity. + dest, conf = rr.recommend("notes and other both", ["notes", "other"]) + assert conf == "weak" + + +def test_duplicates_do_not_change_the_chosen_destination(): + dest, _ = rr.recommend("fix the notes thing", ["notes", "notes"]) + assert dest == "notes" diff --git a/.ai/scripts/todo-cleanup.el b/.ai/scripts/todo-cleanup.el index c4a87de..cb333e2 100644 --- a/.ai/scripts/todo-cleanup.el +++ b/.ai/scripts/todo-cleanup.el @@ -100,6 +100,12 @@ ;; --check-child-priority is the report-only alias for --sync-child-priority ;; --check. +;; Before any modification a backup is copied to +;; /tmp/<basename>.before-todo-cleanup.<YYYYMMDD-HHMMSS> +;; matching lint-org.el and wrap-org-table.el. Skipped under --check, which +;; writes nothing. +;; + (require 'org) (require 'cl-lib) (require 'calendar) @@ -832,9 +838,37 @@ event-log entry, pulling the timestamp from its CLOSED cookie. Honors ;;; --------------------------------------------------------------------------- ;;; Driver + reporting +(defun tc--backup (file) + "Copy FILE to /tmp before any modification. Skipped in --check mode. + +Matches `lint-org.el' and `wrap-org-table.el', the other tools that rewrite +these org files. todo-cleanup runs the most often of the three (every wrap, +every sentry fire), and Emacs's own backup does not fire under --batch -q, so +without this a mechanical rewrite has no undo short of git — which recovers +only to the last commit and loses intra-session work." + (let* ((base (format "%s%s.before-todo-cleanup.%s" + temporary-file-directory + (file-name-nondirectory file) + (format-time-string "%Y%m%d-%H%M%S"))) + (backup base) + (n 2)) + ;; Never overwrite an earlier backup. A second-resolution stamp collides + ;; when two invocations run back to back, which the shipped workflow does + ;; (open-tasks.org runs --convert-subtasks then --archive-done, each a + ;; sub-second batch run). Overwriting there replaces the true pre-session + ;; original with already-mutated content — losing exactly what the backup + ;; exists to preserve. Suffix instead, so every invocation keeps its own. + (while (file-exists-p backup) + (setq backup (format "%s-%d" base n)) + (setq n (1+ n))) + (copy-file file backup nil) + backup)) + (defun tc-process-file (file) (setq tc-current-file (file-name-nondirectory file)) (setq tc-current-dir (file-name-directory (expand-file-name file))) + (unless tc-check-only + (tc--backup file)) (with-current-buffer (find-file-noselect file) (org-mode) (cond diff --git a/.ai/session-context.org b/.ai/sessions/2026-07-24-18-05-sentry-implement-pass-review-and-hook-fixes.org index e3f539c..6c60a8f 100644 --- a/.ai/session-context.org +++ b/.ai/sessions/2026-07-24-18-05-sentry-implement-pass-review-and-hook-fixes.org @@ -6,16 +6,37 @@ ** Active Goal -Sentry armed for the night on =sentry/2026-07-23-ratio=, hourly. Morning teardown is Craig's. +A long session (2026-07-23 into 2026-07-24) spanning several arcs: applied two Craig-ordered sentry amendments, ran a no-approvals speedrun over four solo tasks, shipped the sentry implement-pass feature itself, ran eight overnight sentry fires that found and fixed real bugs, then subjected the whole night's output to an adversarial review that found defects in the fixes, repaired those, absorbed a repo-wide fail-open security fix from .emacs.d, and processed the inbox to zero. Ended with the branch merged to main (unpushed by Craig's choice) and the session wrapped. ** Decisions +- Sentry gains an opt-in solo-implementation pass (pass 12, gated on =:SENTRY_MAY_IMPLEMENT:=, separate from =:COMMIT_AUTONOMY:=) plus refactor-finding in pass 11. Craig's direction, after the discussion that the branch already contains blast radius and a skeptical premise-first review is the fact-checker that makes fixing-on-a-branch safe. Shipped to main. +- Reviews must fact-check the *premise* (reproduce the bug) before judging the diff, not just check the diff is clean. Craig's correction; saved as harness memory =feedback-reviews-verify-premise=. Across the night the premise check killed roughly one wrong hypothesis per real bug. +- Craig chose NOT to push main at wrap. The hook fail-open fix therefore stays undelivered to consuming projects until he pushes. Flagged and reaffirmed. + ** Data Collected / Findings +- The dominant defect class across the session, five instances over two projects: a quality gate that enumerates its inputs instead of discovering them, so a new input is silently skipped and the green check reads as covered. Promoted to a KB node this wrap. +- The secret-scan pre-commit hook failed open on any git error, in ALL FIVE language bundles (not just the elisp one .emacs.d reported). Two of the five were hooks I wrote the day before by copying bash — I propagated the defect. Fixed across all eleven sites; graded [#A]. +- The adversarial review round found four real defects in my overnight fixes (mode-widening and symlink-clobbering in cj-remove-block's atomic write, a same-second backup collision in two tools, a test that deleted real backups from shared /tmp) plus one I'd left: the cj-block range check still can't prove it's deleting the block that was scanned. All repaired except the last, which needs a CLI-contract decision and is filed [#B]. +- I repeated my own worst mistake pattern three times: shipping a change whose correctness depended on shared /tmp state (the backup tests), and twice concluding causation from a single-sample measurement (the audit flake A/B, the /tmp-copy comparison). The re-run/isolation habit caught each. + ** Files Modified +- Merged to main (267d1de): cj-remove-block range guard + atomic write, todo-cleanup backup, route_recommend dedupe, audit.bats flake fix, lint.sh bin/ coverage, plus the review-round repairs. +- Hook fail-open fix (f0c1bc4): all five bundles' pre-commit, the elisp validate-el cap removal, the cross-bundle test now discovering variants, two adopted .emacs.d bats suites. +- Earlier: sentry.org pass 11/12 + marker (pushed), the speedrun's four fixes, the two .dotfiles amendments, voice #47, four approved parked proposals. +- KB: =agents/20260724180443-enumerate-vs-discover-gate-failure.org=. + ** Next Steps +- Push main (5+ commits ahead, all local). The hook security fix is the load-bearing one. +- The cj-block wrong-block design question: content assertion vs re-scan vs bottom-up removal. Filed [#B] at the top of todo.org. +- Seven parked VERIFYs await Craig, [#A] account-binding guard from home first, then the telegram down-is-launch fix and its engine sibling. +- Optional: ~1600 backup files accumulated in /tmp from the night's runs (harmless, cleared on reboot). + +KB: promoted 1 / consulted no + * Session Log ** 02:34 — Startup @@ -173,6 +194,157 @@ Filed [#A], SCHEDULED today — the only [#A] of the night. The fix is find-not- Two fires' worth of evidence now says bundle-to-bundle propagation is where this repo leaks changes. +** 00:51 — Fire 1 of the 2026-07-24 run (working) — first fire with pass 12 live + +Lock acquired, branch =sentry/2026-07-24-ratio= verified, tree clean outside the spine. Digest: + +- P1 roam pull — ran, already up to date. +- P2 inbox zero — ran, no-op. Inbox at zero (the question-capture proposal was parked during entry). +- P3 triage intake — skipped: no active source surviving the mail/messenger exclusion. +- P4 todo cleanup — ran with real work. =--archive-done= moved three completed speedrun tasks out of Open Work into Resolved; =--convert-subtasks= normalized the tree. Committed. +- P5 task audit — mechanical subset. Staleness 13, back to the pre-speedrun baseline now that the four solo tasks closed. +- P6 working-files hygiene — ran. All =working/= dirs still have open backing VERIFY tasks (the parked proposals). No orphans. +- P7 spec status board — ran. Two READY specs, nothing stuck. +- P8 link integrity — ran. The docs-lifecycle spec still shows its two known prose false positives (=file:→id:= and =keep-file:-links-through-pilot=, bare =file:= tokens org parses as bracketless links). Correctly unaffected by tonight's spec-scoping, since =link-to-local-file= is org-lint's own checker, not a todo-format one. No new findings. +- P9 git health — main level with origin, sentry branch correctly has no upstream, no stale branches. +- P10 prep freshness — skipped: no =daily-prep/=. +- P11 bug and refactor finding — ran, first fire under the widened pass. Rotating area: the cross-project routing scripts (=route_recommend.py=, =broadcast.py=), untouched by the previous six areas. One verified latent bug filed [#D] (below). Refactor note not worth a task: =recommend='s two weak-tier branches collapse to a single =_tiebreak= call, since =_tiebreak= on a one-element list returns that element. Two lines, no behavior change, so it's a digest line rather than backlog noise. +- P12 solo-task implementation — *active this run* (=:SENTRY_MAY_IMPLEMENT: yes=) but a correct no-op: zero eligible tasks. All four open =:solo:= TODOs were completed in tonight's speedrun, so the ready bucket is empty. Nothing to implement, nothing deferred. + +*** The finding + +=route_recommend='s =discover_destination_names= collapses projects to bare basenames, so two projects sharing a basename across roots would both literal-match and read as an ambiguous tie, downgrading a correct strong match to weak. Reproduced by direct probe. Latent rather than live: 27 projects, 27 distinct basenames today. The destination stays right, only the tier is wrong, so the cost is an extra routing prompt. Filed [#D] with the order-preserving dedupe as the fix. + +** 01:33 — Fire 2 (working) — pass 12's first real implementations + +Digest: + +- P1 roam pull — ran, already up to date. +- P2 inbox zero — ran. Project inbox at zero. The roam inbox dropped 9 → 5 (another session filed some); all 5 remaining belong to archsetup or work, none rulesets-claimed, so roam mode is correctly a no-op here. +- P3 triage intake — skipped: no active source. +- P4 todo cleanup — ran, all three checks clean (fire 1 did the archiving). +- P5 task audit — mechanical subset. Staleness 13, flat. +- P6 working-files hygiene — ran. All =working/= dirs still backed by open VERIFYs. +- P7 spec status board — ran. Two READY specs, nothing stuck. +- P8 link integrity — ran. Only the docs-lifecycle spec's two known prose false positives. +- P9 git health — main level with origin, sentry branch correctly upstream-less. +- P10 prep freshness — skipped: no =daily-prep/=. +- P11 bug and refactor finding — rotating area: the cj-comment tooling (=cj-scan.py=, =cj-remove-block.py=). Two findings filed, one of them serious. +- P12 solo-task implementation — *two tasks implemented and committed to the branch* (17f5d48, 1b0f284). Both premise-checked before a line was written. + +*** The serious find: cj-remove-block destroys content + +=looks_like_cj_range= validated only the first and last lines of a range. A span from one cj block's opener to a *later* block's closer passed, and the removal then deleted everything between — prose, headings, whole tasks — silently, exit 0. That is exactly the failure the check exists to prevent, and drift is its normal case, since =respond-to-cj-comments= edits the file while processing and a file under cj review usually holds several blocks. Reproduced on a two-block fixture that lost a heading and two content lines. Filed [#B], then implemented in pass 12 after an independent re-verification on a different fixture shape. + +Same file, second defect fixed alongside: =remove_range= rewrote the org file with a bare =write_text= (truncates on open) and took no backup, so a mid-write failure would leave =todo.org= truncated with nothing to recover from. =lint-org.el= already backs these files up to =/tmp= before mutating; cj-remove-block now matches that and writes atomically. + +*** The measurement lesson, third time tonight + +A full-suite run went red on =audit.bats= test 4. I stashed my changes, saw it pass clean, and had a one-sample A/B pointing straight at my own diff. Re-ran three times with the changes restored and it passed every time. The failure is an intermittent teardown flake (=rm -rf= racing something still writing into a fixture =.git/objects=), not my change, and I nearly filed the wrong cause off a single sample. Filed [#C] with the git-background-gc theory explicitly labelled a lead rather than a verified cause. + +Committed only on a genuinely green re-run, not on the red with a hand-wave. + +** 02:33 — Fire 3 (working) — the flake's cause traced, and my own lead disproved + +Digest: + +- P1 roam pull — ran, already up to date. +- P2 inbox zero — ran, no-op. Project inbox at zero; roam holds 5, none rulesets-claimed. +- P3 triage intake — skipped: no active source. +- P4 todo cleanup — ran. =--archive-done= moved fire 2's two completed tasks to Resolved. Committed. +- P5 task audit — mechanical subset. Staleness 13, flat. +- P6 working-files hygiene — ran, all dirs backed. +- P7 spec status board — ran, two READY, nothing stuck. +- P8 link integrity — ran, only the known docs-lifecycle prose false positives. +- P9 git health — main level with origin, branch upstream-less, nothing stale. +- P10 prep freshness — skipped. +- P11 bug and refactor finding — no new findings. The fire's whole investigative budget went to confirming fire 2's filed flake, which is the honest place for it; a hunt that finds nothing new is a result. +- P12 solo-task implementation — one task implemented and committed (7f45d4b). + +*** Confirming the cause before fixing, and disproving my own lead + +Fire 2 filed the =audit.bats= flaky teardown with a stated theory: =gc.auto='s loose-object threshold. Pass 12's premise check went after that theory rather than the fix, and killed it — a fixture holds five objects against a default threshold of 6700, so that mechanism cannot fire. + +The real cause came from a =GIT_TRACE= run: =git commit= spawns =git maintenance run --auto --quiet --detach= on git 2.55. The commit returns while the detached process is still writing a pack, and teardown's =rm -rf= races it. Every failed run had left a =tmp_pack_*= behind, which is the thread that led there. + +Fix: =maintenance.auto false= plus =gc.auto 0= in the fixture, killing the background writer rather than retrying the delete (a retry loop hides a live process instead of removing it). Validated over 20 consecutive clean runs against a ~1-in-8 baseline, and recorded honestly in the task that 20 clean runs alone would be ~7% likely by luck — the trace is the evidence, the runs confirm. + +This is the second night running where the premise check changed the outcome. Fire 2 it stopped a wrong causation call; here it stopped me implementing a fix for a mechanism that was never operating. Both times the cost of checking was minutes and the cost of not checking would have been a plausible, wrong, committed change. + +** 03:33 — Fire 4 (working) — two hypotheses killed, one real gap closed + +Digest: + +- P1 roam pull — ran, up to date. P2 inbox zero — no-op, both surfaces clean. P3 triage — skipped, no active source. +- P4 todo cleanup — ran. Archived fire 3's completed task to Resolved. Committed. +- P5 task audit — mechanical subset. Staleness 13, flat all night. +- P6 working-files — all dirs backed. P7 spec board — two READY, nothing stuck. P8 link integrity — only the known prose false positives. P9 git health — clean. P10 — skipped, no prep dir. +- P11 bug and refactor finding — rotating area: the org-file mutators, chosen because =cj-remove-block= yielded a serious find in that same class last fire. One gap filed. +- P12 solo-task implementation — one task implemented and committed (0686784). + +*** What the area review actually found, and what it disproved + +The lead was that =todo-cleanup.el= might lose data. It rewrites =todo.org=, creates archive files, and moves subtrees *between* files, which is the shape that bit =cj-remove-block=. Two specific hypotheses, both tested, both wrong: + +- *"A mid-move failure loses a subtree from both files."* No. The order is delete-from-buffer, write-archive, save-todo.org-last, so an archive-write failure aborts before the save. Verified by making the archive directory unwritable: exit 255, =todo.org= byte-identical, content intact. +- *"Errors are swallowed on the mutation path."* No. The only =ignore-errors= in the file wrap =call-process "git"=, never a write. + +What survived was narrower and real: todo-cleanup mutates with *no backup*, while both sibling mutators (=lint-org.el=, =wrap-org-table.el=) copy to =/tmp= first, and =cj-remove-block= joined them last fire. It is also the one that runs most often. Confirmed empirically that Emacs's own backup does not fire under =--batch -q=, so there was genuinely no undo short of git. Filed [#C], then implemented in P12. + +Also checked my own change for regression rather than assuming: a missing input file exits 255 and creates nothing, identical to the pre-change version tested from git. + +*** Running tally on the premise habit + +Three fires, three times it changed the outcome. Fire 2 it stopped a wrong causation call. Fire 3 it disproved my own filed =gc.auto= theory before I could implement against it. Here it killed two data-loss hypotheses before they became tasks, leaving only the gap that was actually there. The pattern is consistent: the cheap check keeps a plausible story from becoming a committed change. + +** 04:33 — Fire 5 (working) — the first real defer, and last fire's fix proving itself + +Digest: + +- P1 roam pull — ran, up to date. P2 inbox zero — no-op, both surfaces clean. P3 triage — skipped, no active source. +- P4 todo cleanup — ran, archived fire 4's completed task. *Confirmed last fire's backup fix working live*: the real =--archive-done= run left =/tmp/todo.org.before-todo-cleanup.20260724-043331=. Dogfooded within an hour of shipping. +- P5 task audit — staleness 13, flat all night. P6 working-files — all backed. P7 spec board — two READY. P8 link integrity — only the known prose false positives. P9 git health — clean. P10 — skipped. +- P11 bug and refactor finding — rotating area: the attachment/email handlers, chosen because they parse genuinely untrusted input, unlike every internal-tooling area covered so far. One finding filed. +- P12 solo-task implementation — *deferred*, and correctly. First defer of the run. + +*** The find: attachment filenames are partly sanitized, in two different ways + +Both writers derive on-disk names from the =filename= an email declares, and both sanitize incompletely, covering *different* gaps. =eml-view= cleans the name but interpolates the extension raw. =gmail-fetch='s =safe_filename= handles path separators and leading =..= and nothing else. Probed both with the same adversarial set: a =; rm -rf ~= extension survives in both, a literal newline survives in both, a 300-character extension produces a 314-character filename in both. + +Two things I checked so this doesn't get over-graded later. Not RCE — files are written through Python =open=, never a shell. Not path traversal — =splitext= only returns an extension when the last dot follows the last separator, so =ext= can never hold a slash, and the traversal case is neutralized in both scripts. The genuine harms are narrower: a newline in a filename breaks downstream tooling that reads the directory as a line-delimited list, and an unbounded extension blows the 255-byte limit so a crafted attachment aborts extraction. Graded [#C] on that honest read rather than the scarier one. + +I also corrected my own framing mid-investigation. I first read this as the familiar "one sibling hardened, the other not" pattern from the last two fires. It isn't — =safe_filename= is narrower than it looks, and the two scripts are *differently* incomplete. Neither handles newlines or length. + +*** Why pass 12 deferred instead of implementing + +The fix itself is clear, but where the shared sanitizer lives is a design call: a shared helper module (clean, but a new synced template file plus =importlib= gymnastics for kebab-named scripts), duplicate it in both (self-contained, but drift — the exact defect class that produced three separate findings tonight), or patch each in place (smallest diff, permanent divergence). That is deliberation, not a quick factual question, so checklist item 4 fires and the unattended loop defers. Filed the VERIFY with the three options and my lean, and left the task *un-=:solo:=-tagged* so a later run doesn't pick it up and guess. + +This is the checklist discriminating rather than rubber-stamping. Four fires implemented; this one correctly didn't. + +sentry at 05:33: nothing (bug hunt swept =scripts/*.py=; two candidate gaps both disproved — =workflow-integrity.py= *is* gated, its bats runs the real checker against the real canonical tree under =make test=, and =update-skills.py= is an on-demand maintenance command rather than a gate. Recorded so neither gets re-investigated.) + +sentry at 06:33: nothing (bug hunt swept =wrap-org-table.el=, the third org-file mutator and the one with prior history — its load-time dispatch caused the 2026-07-09 corruption. Clean on every probe: the entry-script guard correctly refuses to dispatch when lint-org merely =require='s it, it backs up before writing like its siblings, and it is block-aware — a table inside =#+begin_example= stayed verbatim while a real over-budget table wrapped onto continuation rows with rules. Second consecutive quiet hunt.) + +** 07:33 — Fire 8 (working) — a time bomb I planted four fires ago went off + +Digest: P1-P10 all no-op (roam current, both inboxes clean, todo cleanup clean, staleness 13, working dirs backed, two READY specs, only the known prose link false positives, git clean). P11 found a real coverage gap. P12 implemented it, and the suite caught a regression of my own making. + +*** The find: the only ungated shell in the repo + +=scripts/lint.sh= sweeps =scripts/*.sh=, the language hooks, and the language githooks. It never touched =claude-templates/bin/= — zero references. Those four scripts (=ai=, =agent-text=, =agent-page=, =install-ai=) are the ones =make install= symlinks onto PATH, which makes them the *most* exposed shell in the repo and left them the only shell with no gate over it. All four are clean today, so the guard is a no-op by design; it exists so a future regression can't pass silently. The =ai= launcher was hardened to 42 tests recently and nothing enforced that going forward. + +The test pins *coverage*, not cleanliness: it plants a broken file in each swept location and asserts lint complains, so a location that stops being swept fails the suite rather than passing quietly. + +Surfaced a bigger question I did *not* answer overnight: rulesets ships shellcheck enforcement to consuming projects (the bash bundle's pre-commit, =validate-bash.sh=) and runs none on itself. Filed as a VERIFY, because turning it on would surface the false positives dispositioned earlier this session and choosing between fixing them or adding disable-directives is a preference, not a fact. + +*** The regression: my own test, detonating on schedule + +The full suite went red on the two backup tests I added in fire 4. Not a flake — a time bomb. They globbed =/tmp/todo.org.before-todo-cleanup.*=, but the backup name derives from the file's *basename*, and the real =todo.org= shares it. So a live sentry run's genuine backup was indistinguishable from the test's own artifact, and the check-mode test (which asserts *no* backup exists) failed the moment fire 5's real archive pass created one. + +They passed when written only because no real backup existed yet. Three now sit in =/tmp=. Both tests rebind =temporary-file-directory= to a private dir, and I verified they pass with the real backups present rather than by clearing them. + +Worth naming plainly: I shipped a test whose correctness depended on the state of a shared directory that the code under test writes to in production. That is the "no shared mutable state" rule in =testing.md=, and I broke it while fixing a different durability bug. The suite caught it two fires later, which is the argument for running the full suite every fire rather than only the touched file. + * Sentry approval queue (2026-07-23) Five items. Each names what, why, and the exact edit. Items 2 through 5 are also filed as =VERIFY [#B]= tasks in =todo.org= with prepared diffs under =working/=, so they survive this anchor being archived — say "approve the parked <topic>" for any of them. diff --git a/.ai/sessions/2026-07-25-09-24-invalid-block-filter-push-task-review.org b/.ai/sessions/2026-07-25-09-24-invalid-block-filter-push-task-review.org new file mode 100644 index 0000000..d0814f9 --- /dev/null +++ b/.ai/sessions/2026-07-25-09-24-invalid-block-filter-push-task-review.org @@ -0,0 +1,83 @@ +#+TITLE: Session Context — 2026-07-24 +#+AUTHOR: Craig Jennings + +* Summary + +** Active Goal + +An evening of backlog clearing in the order Craig picked: push the delivery-blocking commits, fix the lint-org =invalid-block= false positive that home had just unblocked, then run a task-review cycle. All three finished. + +** Decisions + +- The =invalid-block= fix is a filter on org-lint's output, not a local checker edit. Home settled the open question and I re-verified both halves before acting: the string appears nowhere in =lint-org.el=, and =org-lint--checkers= enumerates it in batch Emacs. Same resolution as the earlier =link-to-local-file= episode. +- Left the =,**= comma-escape in =claude-templates/.ai/notes.org= in place. The task said the fix "lets that escape be reverted," but the escape is correct org for a literal =**= inside a verbatim block, so reverting trades correctness for nothing once the finding is suppressed. Reasoning recorded in the closed task rather than acting on the permission silently. +- Task review: all seven in the batch kept as-is, no new =:quick:= or =:solo:= tags, confirmed by Craig in one pass. +- Work's sentry loop was left alone. Craig said "sentry stop" here, but the running loop belongs to the work project (branch =sentry/2026-07-25-ratio=, firing hourly into =aiv-work:1.1=), and the stop procedure's later steps — lock release, branch disposition, approval queue — need that session's context. Surfaced rather than half-executed. + +** Data Collected / Findings + +- =invalid-block= is org-lint's own checker. Reproduced the false positive three ways: a paired example block with a heading-shaped body line flags both delimiters, a paired src block holding a literal =#+end_example= flags three lines, and a genuinely unterminated block flags once and must keep doing so. +- Verified the fix against home's live fixture in both directions: its =.ai/notes.org= produced exactly the two reported findings (lines 386 and 398) under the pre-change script and zero under the new one, file untouched. +- The uppercase-delimiter path (=#+BEGIN_EXAMPLE=) was handled but untested. Confirmed it was a real trigger — 2 findings before, 0 after — before adding the boundary test. +- Part of the task-review staleness count measures work that can't be delegated rather than work nobody read. This batch was the deliberation-heavy tail, where every task needs a decision from Craig mid-stream, which is why the speedruns kept stepping past them. +- The KB orphan task cites a 2026-07-01 snapshot of 53 agent nodes. Startup counted 104, so the KB has doubled and the snapshot is worth even less than the task body assumed. +- Five =[#D]= tasks carry no =:LAST_REVIEWED:= at all. The staleness script excludes =[#D]= but =lint-org= flags them, so the two tools disagree permanently. Left alone; needs a decision about which is right. + +** Files Modified + +- =claude-templates/.ai/scripts/lint-org.el= (+ mirror) — =lo--matched-block-regions= pairs blocks by line scan under org's real rule, memoized on the buffer modification tick; =lo--handle-item= drops an =invalid-block= finding inside a paired region. +- =claude-templates/.ai/scripts/tests/test-lint-org.el= (+ mirror) — five tests: heading-in-example, literal =#+end_example= in src, uppercase delimiters, unterminated block still reports, and one file with both proving per-block scoping. +- =todo.org= — closed the =invalid-block= task with its verification record, stamped seven review dates, inserted a missing properties drawer. +- KB: =agents/20260725093500-parser-cannot-verify-its-own-misreading.org=. + +** Next Steps + +- Work's sentry is still running on ratio and untouched. Stopping it properly means saying "sentry stop" in the work session (pane =aiv-work:1.1=), which handles the branch disposition and the overnight approval queue. +- Home is waiting on this push to re-run its =invalid-block= fixture. +- Next review batch starts with the agent-source improvements and flashcard tooling tasks. +- Seven parked VERIFYs still await Craig, the =[#A]= account-binding guard from home first. +- The sentry spec still wants Craig's deep read before the READY flip. + +KB: promoted 1 / consulted no + +* Session Log + +** Startup + +Startup ran clean: rulesets already current, =make install= had nothing new to link, project repo up to date with origin (5 commits ahead, unpushed — carried over from the prior session by Craig's choice). =.ai/= synced from templates with no churn. No crash anchor. Task staleness reported 13 top-level tasks unreviewed for >7 days. + +** Inbox — home's answer on invalid-block + +One pending handoff: home answering the open question left in the =[#C] lint-org invalid-block false-positives= task — is =invalid-block= lint-org.el's own checker or org-lint's? Home says org-lint's, and I re-verified both halves rather than taking it: =grep invalid-block= over =claude-templates/.ai/scripts/lint-org.el= returns nothing, and =org-lint--checkers= enumerates =invalid-block= in batch Emacs alongside =link-to-local-file=. So the fix is a filter on org-lint's output, matching the =link-to-local-file= episode. + +Folded into the existing task as a dated sub-entry rather than filing anything new — the task was already filed and only needed its open question closed. Also recorded home's regression fixture: its own =.ai/notes.org= PENDING DECISIONS block (lines 386-398) is left unescaped on purpose and trips both delimiters, so the filter should take those two findings to zero without touching the file. Replied to home confirming, and asked them to keep the fixture unescaped pending a ping when the filter lands. Inbox back to zero. + +** Pushed main + +Craig picked the evening's order: push, then the invalid-block fix, then a task review. Pre-push reconcile showed ahead-only by 5, no divergence, so =git push origin main= went out and verified at 0/0. The secret-scan fail-open fix is now delivered to consuming projects. + +** invalid-block filter + +Built test-first. Reproduced the failure three ways before writing anything: a paired example block with a heading-shaped body line flags both delimiters; a paired src block holding a literal =#+end_example= flags three lines; a genuinely unterminated block flags once and must keep doing so. Wrote four ERT tests covering those plus a mixed file, watched three fail for the right reason, then implemented. + +The fix is =lo--matched-block-regions=: a line scan that pairs blocks under org's real rule (once open, only the block's own =#+end_TYPE= closes it), memoized on the buffer's modification tick. =lo--handle-item= then drops an =invalid-block= finding falling inside a matched region, delimiters included, since org-lint reports at the delimiters. Line-scanning instead of asking org is the whole point — org's parser is the thing that's confused. + +One test failed after the implementation on an off-by-one in my own expectation (the unterminated opener is line 7, not 8); the code was right and I corrected the test. Verified against home's live fixture both ways: two findings under the pre-change mirror copy, zero under the new canonical, home's file untouched. Synced canonical → mirror, full =make test= green at exit 0. + +Review caught two things I fixed rather than filed: the uppercase-delimiter path was handled but untested (confirmed it was a real trigger — 2 findings before, 0 after — then added the boundary test), and the cache-tick comparison used =eq= where =eql= is strictly correct. Verdict Approve, committed 8822b0d after the voice pass. Pinged home that the filter landed and the fixture can be re-run. + +Left the =,**= comma-escape in the notes template alone. The task said the fix "lets that escape be reverted," but the escape is correct org for a literal =**= in a verbatim block, so reverting buys nothing now that the finding is suppressed. Recorded the reasoning in the closed task rather than acting on the permission silently. + +** Task review + +Batch of 7 from the staleness script, oldest first. Every one came back Keep with no new =:quick:= or =:solo:= tag, and Craig confirmed the batch in one pass. Backed todo.org up to /tmp first (matching the mutator convention) and stamped by exact line number rather than a global replace, since ten other tasks carried the same 2026-07-13 date and only six were in the batch. The Sentry vNext task had no properties drawer at all, so it got one. + +Two observations worth keeping. The uniform Keep-with-no-tags result isn't the review going soft: this batch is the deliberation-heavy tail, where every task needs a decision from Craig somewhere in the middle, which is exactly why the speedruns kept stepping past them and why they aged. So part of the staleness count is measuring work that can't be handed off rather than work nobody read. And the KB orphan task cites a 2026-07-01 snapshot of 53 agent nodes; tonight's startup counted 104, so the KB has doubled and the snapshot is worth even less than the task body already assumed. + +** Inbox — home's acknowledgment + +Home replied confirming both messages landed, the fixture stays unescaped, and they'll pick the filter up after the rulesets push and their next clean startup sync. A pure FYI asking nothing, so it skipped the skeptical review and got no reply (acking an ack loops). Deleted; inbox back to zero. It does corroborate that the unpushed commit is the only thing standing between home and the fix. + +** Task review (cont.) + +Staleness went 13 → 6. Lint flags five more tasks missing =:LAST_REVIEWED:= entirely (lines 327, 336, 434, 593, 602) — all =[#D]=, which the staleness script excludes but the checker doesn't. Pre-existing, not touched. diff --git a/.ai/sessions/2026-07-25-15-34-clean-wrap-and-inbox-safe-sync.org b/.ai/sessions/2026-07-25-15-34-clean-wrap-and-inbox-safe-sync.org new file mode 100644 index 0000000..737b1c5 --- /dev/null +++ b/.ai/sessions/2026-07-25-15-34-clean-wrap-and-inbox-safe-sync.org @@ -0,0 +1,85 @@ +#+TITLE: Session Context — 2026-07-25 +#+AUTHOR: Craig Jennings + +* Summary + +** Active Goal + +Guarantee that rulesets can only report a successful wrap with a completely clean Git worktree, while allowing other projects to refresh rulesets when its only residue is untracked inbox deliveries. Implemented, reviewed, fully tested, and prepared for a strict self-hosted wrap. + +** Decisions + +- One executable, =git-worktree-gate=, owns both repository-state policies. =strict= means no staged, unstaged, untracked, dirty-submodule, or in-progress-operation state; =sync-safe= permits only untracked paths beneath =inbox/=. +- Cleanup failure is not a degraded wrap. It leaves the session open and must report each path, its Git state, why it cannot be resolved safely, and the decision Craig needs. Dirty-file deferrals, valediction, and teardown are forbidden in that state. +- Final wrap verification is a HEAD-bound certificate stored in the Git directory and freshly rechecked inside the existing teardown hook. Integrating the check avoids the concurrent-hook race documented by Codex. +- Another project's structured Edit/Write call may not resolve through an installed symlink into rulesets. The runtime hook denies it and points the sender to =inbox-send rulesets=. +- The two MCP-registry handoffs were consolidated into one parked =[#B]= specification decision. The memory auditor remains separate; no machine-owned MCP configuration was promoted into canonical rulesets. + +** Data Collected / Findings + +- The prior rulesets archive completed at 09:24; the inherited three tracked changes were written at 10:27. The repository was dirtied after wrap through a later write path, which is why wrap certification alone needed the symlink-aware boundary guard. +- The previous wrap prose contradicted itself: clean Git state was an exit criterion, but the leftover and inbox sections allowed explicit deferral. The teardown hook checked only the sentinel. +- Adversarial review found a fail-open process-substitution edge: a low-level =git status= failure could appear as an empty stream. The gate now captures status and its exit code in Git-directory temporary files and blocks on failure. +- Current Codex hooks support Stop blocking with =continue=false= and run matching commands concurrently. A user-level =hooks.json= is installed; a new Codex session must complete the normal hook review/trust step. +- Full =make test= passed twice on the implementation. The final run includes 436 core Python tests, 72 hook tests, language suites, ERT suites, and all Bats suites. Focused additions cover inbox-only pulls, staged/unstaged/untracked/submodule/operation states, status failure, certificate/HEAD drift, Claude and Codex Stop outputs, installation, and realpath-based write denial. +- The wrap roam sweep found Craig's 120-column table question. It was already enforced by =org-tables.md=, =lint-org='s =org-table-standard= judgment, and =wrap-org-table.el=; the live lint pass flagged the existing over-wide table. Removed the duplicate capture and synced roam. + +** Files Modified + +- =claude-templates/bin/git-worktree-gate= — shared strict/sync-safe classifier plus certificate/verify modes. +- Startup protocol/workflow mirrors and =claude-templates/bin/ai= — inbox-only state remains visible but no longer blocks fast-forward refresh. +- Wrap protocol/workflow mirrors and =hooks/ai-wrap-teardown.sh= — no deferral escape, actionable hard blocker, final certificate, fresh teardown verification. +- =hooks/rulesets-write-boundary.py=, Claude/Codex hook configuration, cross-project rule, Makefile, and hook documentation — prevent structured writes through installed symlinks and install the enforcement on both runtimes. +- Bats and pytest suites — repository-state, launcher, teardown, installer, and cross-project boundary regressions. +- =todo.org= and workflow state — parked the consolidated MCP registry spec decision and recorded inbox processing. +- Three inherited backlog files — retain the post-09:24 definition that speedrunnable means =:solo:=. + +** Next Steps + +- Start a new Codex session and review/trust the new user-level hooks when =/hooks= prompts; Claude already reads the linked hook configuration. +- Say "spec the MCP registry sync" when ready to design the separate host-level registry reconciler. +- The existing =inbox/lint-followups.org= pipeline retains its 14 current judgment items, including the over-wide table and older missing review stamps/links; they do not represent uncommitted work after this wrap. + +KB: promoted 0 / consulted no + +* Session Log + +** Startup and clean-worktree investigation + +Ran the required startup workflow. The canonical rulesets pull and template sync were blocked by three tracked modifications; two untracked inbox handoffs were also pending. Read the project and global behavioral rules, recent session archive, wrap workflow, relevant hook and launcher code, and the current Codex MCP and hook documentation needed to evaluate the inbox proposals. + +Investigated Craig's requirement that rulesets finish with an absolutely clean worktree while still allowing downstream projects to sync when rulesets has received inbox deliveries. The current wrap workflow states a clean exit criterion but later permits explicitly deferred dirty files, and its teardown hook checks only the wrap sentinel rather than Git state. The startup shell's tracked-change check already ignores untracked inbox files, but the general launcher dirty check does not distinguish inbox deliveries from other untracked residue. + +The three tracked files now dirty were written at 10:27, after the latest archived rulesets session wrapped at 09:24. That establishes a post-wrap write path: a strict wrap gate can guarantee the state at completion, but preventing later contamination also needs a realpath-aware cross-project write guard because globally installed rules and workflows are symlinks into this repository. + +The proposed design is one shared repository-state classifier with two policies. Strict wrap requires no staged, unstaged, or untracked entries and no dirty submodules. Inbox-safe sync permits an otherwise clean tracked/index state with untracked entries only below =inbox/=; those deliveries do not block pull or template sync. The strict check should run after the final push and again inside the ordered teardown hook, tied to the checked HEAD, while the inbox-safe policy should drive both startup and the =ai= launcher. Tests should cover every Git state, unusual path names, inbox-only sync, hook behavior, and the canonical/template mirrors. + +Two pending inbox handoffs both propose a shared Claude-to-Codex MCP registry mirror. They pass the value gate but overlap. The recommendation is to consolidate them into one =[#B]= specification with Codex-only entries preserved, atomic and redacted updates, dependency and health checks, both-machine verification, and the Claude-memory audit split into a separate task. No inbox disposition or project implementation has been applied. + +Completed the investigation plan without changing product code. The worktree proposal is ready for Craig's approval; implementation, tests, and inbox filing remain deliberately pending. + +** Clean-wrap invariant clarified + +Craig confirmed that cleanup failure must prevent wrap-up entirely. The agent must keep the session open and report exactly what remains in the Git worktree, why it could not resolve each item safely, and the specific action or decision Craig needs to supply. A warning, deferred-file exception, valediction, archived-as-complete status, or teardown is not an acceptable substitute for a clean tree. + +** Clean-wrap enforcement implemented + +Craig approved implementation and asked for a full wrap when it is done. Added =git-worktree-gate= as the single policy executable: strict mode rejects every staged, unstaged, untracked, dirty-submodule, and in-progress-operation state; sync-safe mode permits only untracked =inbox/= deliveries. Certificate and verify modes bind the final clean check to HEAD inside the Git directory. + +Wired sync-safe behavior into both startup workflow copies and the =ai= launcher. The picker labels inbox-only state distinctly and now fast-forwards a behind repository with inbox deliveries present while refusing other untracked residue. + +Made wrap cleanup fail closed: removed every dirty-file and inbox deferral escape, added the post-push clean certificate as a hard prerequisite to valediction, and required an exact path/state/needed-decision report when cleanup cannot finish. The existing teardown hook now freshly verifies the certificate and HEAD before consuming either sentinel, emits the runtime-appropriate Claude or Codex Stop blocker, and leaves the sentinel/session intact on failure. Added global Codex hook configuration and installed its symlink; Codex will require its normal hook review/trust on a new session. + +Added =rulesets-write-boundary.py= and configured Claude and Codex Edit/Write hooks. It resolves targets through symlinks and denies another project's write when the real path lands in rulesets, directing the proposal through =inbox-send rulesets=. The cross-project rule now states the installed-symlink case explicitly. + +Focused verification is green: 10 state-gate Bats cases, 13 teardown-hook cases including dirty/changed-HEAD/missing-certificate and both runtime outputs, 36 launcher cases including inbox-only pull, 5 installer cases, and 5 Python write-boundary cases. + +** Inbox — MCP registry proposals consolidated + +Processed the two pending 2026-07-25 handoffs from work and home. They were duplicate evidence for a host-level Claude-to-Codex MCP registry reconciler, not project-level implementation requests. Filed one =[#B]= parked specification decision in =todo.org= with preservation, redaction, atomicity, transport, health-check, two-machine, malformed-input, and token-rotation gates; split the memory auditor into separate future work. Deleted both inbound handoffs, stamped =:LAST_INBOX_PROCESS:=, sent acknowledgements to both source projects, and verified zero pending project handoffs. + +** Review, full verification, and wrap cleanup + +The first full =make test= run passed. Adversarial self-review then found the Git-status process-substitution fail-open and a path-resolution fail-open in the cross-project hook; fixed both, added status-error and sequencer regressions, aligned =protocols.org= with the new policies, and reran the complete suite successfully. + +Executed wrap cleanup: no sentry lock, todo hygiene/convert/archive/priority passes made no changes, lint-org applied zero mechanical changes and refreshed the 14-item judgment pipeline, project inbox remained at zero, route-batch had no candidates, and 30-day staleness was zero. The shared roam inbox held one rulesets capture about 120-column tables; verified the enforcement already exists in three layers, removed the duplicate, and pushed the roam update with the repository's sync helper. diff --git a/.ai/workflows/startup.org b/.ai/workflows/startup.org index 929d482..1dff1db 100644 --- a/.ai/workflows/startup.org +++ b/.ai/workflows/startup.org @@ -29,10 +29,16 @@ Inside a rulesets session, the project-repo refresh below covers this — the ru #+begin_src bash rs="$HOME/code/rulesets" if [ -d "$rs/.git" ]; then - if (cd "$rs" && git diff --quiet --ignore-submodules HEAD -- 2>/dev/null); then + gate="$rs/claude-templates/bin/git-worktree-gate" + if [ -x "$gate" ] && "$gate" sync-safe "$rs"; then + (cd "$rs" && git pull --ff-only origin main 2>&1) | tail -3 + elif [ ! -x "$gate" ] \ + && (cd "$rs" && git diff --quiet --ignore-submodules HEAD -- 2>/dev/null); then + # Bootstrap fallback for a checkout old enough not to have the shared + # gate yet. The pull that follows installs it for subsequent starts. (cd "$rs" && git pull --ff-only origin main 2>&1) | tail -3 else - echo "rulesets: dirty working tree — using as-is, skipping pull" + echo "rulesets: changes beyond untracked inbox deliveries — using as-is, skipping pull" fi else echo "rulesets: not a git checkout — skipping" @@ -40,11 +46,11 @@ fi #+end_src Behavior: -- *Clean working tree* → fast-forward pull. =git pull --ff-only= refuses any merge or rebase, so the operation is either a no-op (already current) or a clean advance. -- *Dirty working tree* → skip the pull. Don't auto-stash and don't auto-merge — those would either lose work or invite conflicts at the worst possible moment (session start). +- *Clean working tree, or untracked deliveries only beneath =inbox/=* → fast-forward pull. =git pull --ff-only= refuses any merge or rebase, so the operation is either a no-op (already current) or a clean advance. Inbox files are queue input, not source-tree work, and do not block other projects from receiving rulesets updates. +- *Any staged or tracked change, dirty submodule, Git operation in progress, or untracked file outside =inbox/=* → skip the pull. Don't auto-stash and don't auto-merge — those would either lose work or invite conflicts at the worst possible moment (session start). - *Non-fast-forward history* → =--ff-only= aborts with an error. Surface that to the user; the rsync still proceeds against the working tree as-is. -*Template-freshness policy (applies to every dirty-check in the synced workflows).* "Dirty" means *tracked modifications only*. Untracked and gitignored files — an inbox drop, a file left in the tree to read, scratch output — never block a template pull, a fast-forward, or a monitoring gate. Projects were falling behind on templates because somebody sent them a task; that's the failure this policy closes. The checks here already comply (=git diff --quiet HEAD= sees only tracked changes; the ff gate uses =--untracked-files=no=), and any dirty-check added to a synced workflow follows the same rule. One deliberate exception: the rsync WIP-guard below counts untracked files *within rulesets' own synced source paths*, because an untracked half-written template is exactly the WIP it exists to hold back — that guard is about rulesets' outbound content, not the consuming project's local state. +*Template-freshness policy (applies to every dirty-check in the synced workflows).* The shared =git-worktree-gate sync-safe= policy is the source of truth: untracked files beneath =inbox/= and gitignored files do not block a pull, fast-forward, or monitoring gate; every other staged, tracked, untracked, submodule, or in-progress-operation state does. Projects must not fall behind merely because somebody sent them a task, but an arbitrary scratch file is not silently treated as safe. One deliberate exception remains: the rsync WIP-guard below is narrower than the repository gate and counts untracked files within rulesets' own synced source paths, because an untracked half-written template is exactly the WIP it exists to hold back. *** Install rulesets symlinks into ~/.claude (idempotent) @@ -74,8 +80,11 @@ if [ -d .git ]; then current=$(git symbolic-ref --short HEAD 2>/dev/null) dirty=0 - if ! git diff --quiet --ignore-submodules HEAD -- 2>/dev/null \ - || [ -n "$(git status --porcelain --untracked-files=no)" ]; then + gate="$HOME/code/rulesets/claude-templates/bin/git-worktree-gate" + if [ -x "$gate" ]; then + "$gate" sync-safe "$PWD" >/dev/null 2>&1 || dirty=1 + elif ! git diff --quiet --ignore-submodules HEAD -- 2>/dev/null \ + || [ -n "$(git status --porcelain --untracked-files=no)" ]; then dirty=1 fi @@ -107,8 +116,8 @@ fi #+end_src Behavior, per branch: -- *Behind only, current branch, clean tree* → =git merge --ff-only= advances HEAD. -- *Behind only, current branch, dirty tree* → fetched but not advanced. Surface so Craig can ff manually after dealing with the dirty state. +- *Behind only, current branch, sync-safe tree* → =git merge --ff-only= advances HEAD. An untracked =inbox/= delivery is sync-safe. +- *Behind only, current branch, sync-blocking tree* → fetched but not advanced. Surface so Craig can ff manually after dealing with the reported state. - *Behind only, non-checkout branch* → =git fetch . upstream:branch= advances the ref without touching the working tree. - *Diverged* (ahead and behind) → leave alone. Surface for Craig to resolve. Don't auto-rebase or auto-merge. - *Ahead only* or *up to date* → silent no-op. diff --git a/.ai/workflows/work-the-backlog.org b/.ai/workflows/work-the-backlog.org index 3933749..a0b24a8 100644 --- a/.ai/workflows/work-the-backlog.org +++ b/.ai/workflows/work-the-backlog.org @@ -70,6 +70,8 @@ A task is autonomous-safe when *both* hold. This layer is a lookup, not a judgme 1. *Status is =TODO=* — never =VERIFY=, =DOING=, =DONE=, or =CANCELLED=. =VERIFY= marks "awaiting Craig's input"; auto-implementing one defeats the check it represents. The do-not-implement set is safe-by-omission: anything not plainly =TODO= (plus any project-declared "hold" marker) is out. 2. *Tagged =:solo:=* — the autonomy tag, resolved against the project's priority/tag scheme header in =todo.org= (never hardcoded). =:solo:= carries the hard definition in =todo-format.md=: completable and verifiable without Craig beyond at most one or two quick decisions answerable up front, no design deliberation. A project whose scheme declares a different autonomous-safe tag set overrides the default. +Terminology: *speedrunnable means tagged =:solo:=*. It does not mean =:quick:= or require =:quick:solo:=. The =TODO= status check above is the execution-state gate over that speedrunnable set. + Priority and =:next:= drive *ordering* within the eligible set, not eligibility ([#A] before [#B] before [#C], then the author's ordering). =:quick:= is an effort hint for batching and duration estimates — never a gate. Task *size* is deliberately absent from this gate. A large but well-specified, decision-free task is in scope and gets decomposed into per-logical-commit chunks during implementation. Size never sends a task away; only *deliberation* or *risk* does (the checklist below). diff --git a/.ai/workflows/wrap-it-up.org b/.ai/workflows/wrap-it-up.org index 103e225..7d55a55 100644 --- a/.ai/workflows/wrap-it-up.org +++ b/.ai/workflows/wrap-it-up.org @@ -24,7 +24,7 @@ The wrap-up is complete when: 2. *File is archived.* =.ai/session-context.org= has been renamed to =.ai/sessions/YYYY-MM-DD-HH-MM-description.org=. The old path no longer exists. 3. *todo.org is clean.* Cleanup script ran. Any auto-fixes are staged for the wrap-up commit. Orphan planning lines surfaced for manual fix if there are any. 4. *Linear board is honest* (skip if project doesn't use Linear). Any Dev-Review ticket whose PR has merged was moved to Done or PM Acceptance per the classification rule. -5. *Git state is clean.* All changes committed + pushed to all remotes. Working tree clean. +5. *Git state is certified clean.* All changes are committed + pushed to all remotes, =git-worktree-gate certify= succeeded at the current HEAD, and the working tree has no staged, unstaged, untracked, submodule, or in-progress-operation state. 6. *Valediction delivered.* Brief, warm closing with key accomplishments and reminders, ending with =session wrapped.= on its own line as the signoff marker. The absence of =.ai/session-context.org= is the signal that the last session wrapped up cleanly. Its presence at session start means the previous session was interrupted. @@ -273,7 +273,7 @@ For an interactive walk of the judgments mid-day, run =/lint-org todo.org=. *** Inbox sanity check (surface unprocessed handoffs) -If the project has an =inbox/= directory, verify it holds nothing but =.gitkeep=, =lint-followups.org= (the lint-org pipeline file the next morning's daily-prep consumes), and any explicitly-deferred =PROCESSED-*= files before the wrap completes. An inbox that arrived at session start with handoffs from other projects, or that received handoffs mid-session, needs =inbox.org= process mode to run and apply its value-gate dispositions. Wrapping with a dirty inbox silently defers the work to next session and accumulates handoff debt that the sender can't see. +If the project has an =inbox/= directory, verify it holds nothing but =.gitkeep=, =lint-followups.org= (the lint-org pipeline file the next morning's daily-prep consumes), and =PROCESSED-*= files before the wrap completes. An inbox that arrived at session start with handoffs from other projects, or that received handoffs mid-session, needs =inbox.org= process mode to run and apply its value-gate dispositions. Wrapping with an unprocessed inbox silently defers the work to next session and accumulates handoff debt that the sender can't see. #+begin_src bash unprocessed=$(find inbox -maxdepth 1 -type f \ @@ -282,7 +282,7 @@ unprocessed=$(find inbox -maxdepth 1 -type f \ ! -name 'PROCESSED-*' \ 2>/dev/null | wc -l) if [ "$unprocessed" -gt 0 ]; then - echo "wrap-up: inbox/ has $unprocessed unprocessed item(s). Run inbox.org process mode before wrapping, or explicitly defer each item with a one-line reason in the valediction." + echo "wrap-up blocked: inbox/ has $unprocessed unprocessed item(s). Run inbox.org process mode before wrapping." find inbox -maxdepth 1 -type f \ ! -name '.gitkeep' \ ! -name 'lint-followups.org' \ @@ -291,7 +291,7 @@ if [ "$unprocessed" -gt 0 ]; then fi #+end_src -If the count is zero or the project has no =inbox/= directory, the check is a silent no-op. If non-zero, the wrap is incomplete by default. The user resolves each item (process now, defer with reason in the valediction, or delete with rationale) before the validation checklist passes. +If the count is zero or the project has no =inbox/= directory, the check is a silent no-op. If non-zero, the wrap is blocked. Process each item through its value-gate disposition, or delete it only when that workflow's rationale authorizes deletion, before continuing. The check exempts =lint-followups.org= explicitly because lint-org runs earlier in the same wrap-up workflow and writes its judgment items to that file in =inbox/= by design. The file is a pipeline artifact for the next morning's =daily-prep=, not a handoff that needs the value gate. @@ -501,17 +501,17 @@ Behavior: git status --short #+end_src -*Default policy: end every session with an empty =git status=.* The wrap is incomplete while anything remains dirty. There is no "leave it alone" default — every leftover gets an active resolution. The only way for a file to stay dirty across the wrap is the user explicitly saying "defer this one, leave it dirty." Surface each leftover with a concrete recommendation; the user has to actively opt out for the dirt to persist. +*Hard policy: end every session with an empty =git status=.* The wrap is incomplete while anything remains dirty. There is no deferral exception and no "wrapped with known changes" state: unresolved dirt means the session remains open and wrap-up does not occur. This inverts the older "intentional carryover" default, which let pre-existing dirty state accumulate across sessions silently. Carryover that lives for days or weeks is almost always one of: a forgotten commit from a prior wrap, a stale change that should be discarded, or genuine in-flight work that needs an explicit stash/branch home. None of those should default to "leave it dirty." **** Three kinds of leftover -| Pattern | What it is | Recommended action (apply unless user defers) | +| Pattern | What it is | Recommended action | |---+---+---| | Generated, runtime, or lock files that no human edits — e.g., =.claude/scheduled_tasks.lock=, =.pytest_cache/=, build outputs, IDE state, editor swap files | *Runtime artifact* — created by tooling or the harness, not by the user, and shouldn't be tracked | Add the matching pattern to =.gitignore= (project-level, not =~/.gitignore_global=). For tracked files, =git rm --cached <path>=. Stage =.gitignore= and any =rm --cached= changes in *one* follow-up commit (=chore: gitignore X=), push. Re-run =git status= to confirm clean. | | Modified or created during the session but not staged into the wrap-up commit | *Forgotten change* — real session work that should have been in the wrap commit but missed it | Stage and create a follow-up commit. Don't =--amend= the wrap-up commit once pushed (diverging history without a clear win). Push the follow-up to all remotes. | -| Was dirty at session start and still dirty at session end — work this session deliberately didn't touch | *Pre-existing dirt that needs a decision* — could be a missed commit from a prior wrap, stale abandoned work, or real in-flight work without a home | Investigate (show diff + check the originating session). Recommend one of: (a) commit now if the work is complete, (b) stash with a descriptive message if it's genuine WIP, (c) =git checkout -- <path>= / =git clean -f <path>= if stale and unwanted, (d) move to a feature branch if it's longer-running, (e) user explicitly defers and accepts the dirt. Do not silently leave dirty. | +| Was dirty at session start and still dirty at session end — work this session deliberately didn't touch | *Pre-existing dirt that needs a decision* — could be a missed commit from a prior wrap, stale abandoned work, or real in-flight work without a home | Investigate (show diff + check the originating session). Recommend one of: (a) commit now if the work is complete, (b) stash with a descriptive message if it's genuine WIP, (c) =git checkout -- <path>= / =git clean -f <path>= if stale and unwanted, or (d) move to a feature branch if it's longer-running. Do not silently leave dirty. | **** Per-file flow @@ -519,18 +519,40 @@ For each leftover line in =git status --short=: 1. Identify which of the three kinds above it matches. 2. State what the file is (one line) and the recommended action. -3. Apply the action unless the user explicitly defers. -4. Re-run =git status --short= after each follow-up commit until empty (or until every remaining line is an explicit user-deferred entry). +3. Apply the action when it is safe and authorized. +4. Re-run =git status --short= after each follow-up commit until empty. The pre-existing-dirt case (third row) is the one this rule most cares about. Treat each pre-existing-dirty file as a question that must get an answer this session, not as "carryover that's fine to inherit." A file that was dirty for a week before this session probably isn't going to get cleaner by waiting another week. Look at the diff, check the originating session's notes, and recommend a real resolution. -**** When the user defers +**** When cleanup cannot be completed -If the user does say "leave this one dirty for now" after seeing the recommendation, that is fine — log the deferral in the valediction so the next session knows it was an explicit choice, not a miss. Format: "Deferred (per Craig's decision today): =path/to/file= — <one-line reason>". Without that note, the next session can't distinguish "we agreed to defer" from "we forgot again." +Stop the wrap. Do not deliver the valediction, print =session wrapped.=, drop a teardown/shutdown sentinel, or describe the session as complete. Report: + +1. Every remaining path and its exact Git state. +2. What the file is and why the agent cannot safely resolve it alone. +3. The concrete action or decision Craig needs to provide to make the tree clean. + +An explicit decision to keep a file dirty changes the outcome from "wrapping" to "leaving the session interrupted." It never satisfies this workflow. + +*** Final clean-tree certificate — hard gate + +After all commits are pushed and every leftover appears resolved, run the shared gate: + +#+begin_src bash +gate="$(command -v git-worktree-gate 2>/dev/null || true)" +[ -n "$gate" ] || gate="$HOME/code/rulesets/claude-templates/bin/git-worktree-gate" +if [ ! -x "$gate" ]; then + echo "wrap blocked: git-worktree-gate is unavailable; install rulesets tooling and retry" + exit 1 +fi +"$gate" certify "$PWD" +#+end_src + +The certificate lives inside the Git directory, so it does not dirty the worktree. It records the exact verified HEAD. A non-zero result is a hard stop governed by "When cleanup cannot be completed" above. Step 5 is unreachable until certification succeeds. ** Step 5: Valediction -Brief, warm closing. 3-4 sentences max. +Only after the final clean-tree certificate succeeds, deliver a brief, warm closing. 3-4 sentences max. Include: - What was accomplished (specific, not generic) @@ -567,7 +589,7 @@ Do nothing. The buffer, the =aiv-<project>= tmux session, and =claude= all stay *** Teardown mode (default) -Confirm commit + push succeeded (Exit Criteria 5 — never tear down over unpushed work), then drop the sentinel: +Confirm commit + push and the final clean-tree certificate succeeded (Exit Criteria 5 — never tear down over unpushed or dirty work), then drop the sentinel: #+begin_src bash touch "/tmp/ai-wrap-teardown-$(basename "$PWD")" @@ -605,7 +627,8 @@ If =emacsclient= isn't resolvable or the daemon is down, the gate can't run — 7. *Leaving =.ai/session-context.org= in place* — its presence means "interrupted session", confuses next startup 8. *Long preachy valediction* — brief beats thorough 9. *Leaving runtime/generated files dirty without gitignoring them* — pollutes every future =git status= and erodes trust in "working tree clean" as a signal. Fix =.gitignore= during the wrap, not later. -10. *Treating "was dirty at session start, still dirty now" as fine by default* — that's how a forgotten commit from two sessions ago turns into "carryover" for two weeks. Every pre-existing dirty file needs an active resolution recommendation this session. Deferral is allowed only with an explicit user choice, logged in the valediction. +10. *Treating "was dirty at session start, still dirty now" as fine* — that's how a forgotten commit from two sessions ago turns into "carryover" for two weeks. Every pre-existing dirty file must be resolved or the wrap remains blocked. +11. *Calling a blocked cleanup a wrap* — if the strict gate fails, report the paths and needed decisions; do not valedict, certify completion, or tear down. * Validation Checklist @@ -618,19 +641,20 @@ Before considering wrap-up complete: - [ ] =todo-cleanup.el= ran — hygiene pass + =--convert-subtasks= + =--archive-done= + =--sync-child-priority= (if =todo.org= exists at project root) - [ ] =lint-org.el= ran on =todo.org= — mechanical fixes applied, judgments appended to follow-ups file (if =todo.org= exists) - [ ] Any orphan-planning-line warnings reviewed (fix or accept) -- [ ] Inbox carries nothing but expected pipeline artifacts (=.gitkeep=, =lint-followups.org=, =PROCESSED-*= prefixes), OR each remaining handoff has an explicit deferral logged in the valediction +- [ ] Inbox carries nothing but expected committed or ignored pipeline artifacts (=.gitkeep=, =lint-followups.org=, =PROCESSED-*= prefixes); any untracked inbox delivery was processed before wrap - [ ] Linear Dev-Review sweep ran; any merged-PR tickets moved to Done or PM Acceptance (skip if project doesn't use Linear) - [ ] Template-sync churn committed as its own =chore: sync .ai tooling from templates= (consuming projects only; skipped in rulesets), or surfaced if a synced path didn't match canonical -- [ ] After wrap-up commit + push, =git status --short= is empty OR every remaining line has an explicit user-deferred decision logged in the valediction +- [ ] After wrap-up commit + push, =git-worktree-gate certify "$PWD"= succeeded at the current HEAD - [ ] Each leftover was investigated and the user saw a concrete resolution recommendation - [ ] Runtime artifacts added to =.gitignore=, follow-up commit pushed, =git status= re-verified - [ ] Forgotten changes committed in a follow-up and pushed -- [ ] Pre-existing dirty files resolved (committed / stashed / discarded / moved to a feature branch) or explicitly deferred with a one-line reason in the valediction +- [ ] Pre-existing dirty files resolved (committed / stashed / discarded / moved to a feature branch); otherwise wrap stopped with an actionable blocker report - [ ] Current branch pushed to ALL remotes (verified with =git remote -v=) - [ ] All other local branches with a tracking upstream pushed to their remote - [ ] Any untracked-upstream branches surfaced for manual =git push -u= - [ ] Step 6 teardown matches the trigger phrase: no-teardown leaves the buffer; teardown drops only =/tmp/ai-wrap-teardown-<project>=; shutdown gates on =cj/ai-term-live-count= = 1 and drops only =/tmp/ai-wrap-shutdown-<project>= - [ ] No teardown/shutdown sentinel was dropped before commit + push was verified +- [ ] The teardown hook can re-verify the clean-tree certificate before consuming a sentinel - [ ] Shutdown aborted (fell back to normal wrap, logged in the valediction) when another =aiv-*= session was live or the gate couldn't run - [ ] Commit message follows format (no =session:=, no Claude attribution) - [ ] Valediction delivered (brief, specific, warm) |