aboutsummaryrefslogtreecommitdiff
path: root/todo.org
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2026-05-24 03:54:09 -0500
committerCraig Jennings <c@cjennings.net>2026-05-24 03:54:09 -0500
commit6ebd0314c880c4f9fc15936618ad412cef6ea309 (patch)
tree3d7125c2f43d4e17076aa854561c48314ff60e5f /todo.org
parent8270a4e94e3bdd7cb52a93f9f4b74c1ae6ad6c4f (diff)
downloaddotemacs-6ebd0314c880c4f9fc15936618ad412cef6ea309.tar.gz
dotemacs-6ebd0314c880c4f9fc15936618ad412cef6ea309.zip
docs(ai-kb): fold in review 6 and resolve the build-time decisions
The latest design review was a UX and performance pass, and I folded its findings into the spec and the implementation tasks. The important one: human Emacs edits now use the same write path as agent writes. An ai-kb minor mode runs index, full lint, and commit under flock on after-save, so a hand edit can't quietly skip the safety gate. The rest: the generated index.org is now invisible to backlink and orphan logic (excluded from the scan, referenced as plain text rather than id-links), a required :SUMMARY: property feeds the index and query without inference, query gains lexical ranking with recency only as a tie-break, the switch installs a full org-roam profile rather than a two-variable swap, and the browsing surface (dashboard, find, search, show, backlinks, map) is named. I also answered the six build-time decisions: concrete raw and curation limits, performance budgets for the perf fixtures, the lexical scoring weights, org-roam-graph as the first map implementation, the after-save failure UX (the save always lands, the commit is gated, and a failure shows without trapping the buffer), and the after-save recursion guard. The numeric limits and budgets are starting points to calibrate. The rest are firm. Step 1 stays buildable.
Diffstat (limited to 'todo.org')
-rw-r--r--todo.org28
1 files changed, 17 insertions, 11 deletions
diff --git a/todo.org b/todo.org
index ef9eb9a0..ee1527ef 100644
--- a/todo.org
+++ b/todo.org
@@ -42,16 +42,16 @@ Tags are additive. For example, a small wrong-behavior fix can be
=:feature:refactor:=.
* Emacs Open Work
** PROJECT [#B] Implement ai-kb :feature:ai-kb:
-Build v1 of the AI knowledge base per [[file:docs/design/ai-kb.org][docs/design/ai-kb.org]] (Ready; five reviews incorporated, all decisions resolved 2026-05-24). Step 1 splits into 1a (the safe write path — minimum usable) and 1b (retrieval, maintenance, push), since =remember= depends on =index=+=lint= and the adapter depends on =remember=. Step 2 is the Emacs browsing layer. Step 3 and the LLM-Wiki layer are vNext. Children are ordered by build sequence; the server bootstrap is the prerequisite.
+Build v1 of the AI knowledge base per [[file:docs/design/ai-kb.org][docs/design/ai-kb.org]] (Ready; six reviews incorporated, all decisions resolved 2026-05-24). Step 1 splits into 1a (the safe write path — minimum usable) and 1b (retrieval, maintenance, push), since =remember= depends on =index=+=lint= and the adapter depends on =remember=. Step 2 is the Emacs layer: a full org-roam profile on switch, the human-edit safety model (same write path as the agent), and the browsing surface. Step 3 and the LLM-Wiki layer are vNext. Children are ordered by build sequence; the server bootstrap is the prerequisite.
*** TODO [#B] ai-kb bare repo on cjennings.net :ai-kb:
Prerequisite, one-time server bootstrap (not doable by the local script): =sudo git init --bare /var/git/ai-kb.git= + chown on cjennings.net. Leave the github-mirror hook OFF — this repo is private. Required before every per-machine clone.
*** TODO [#B] ai-kb store + contract + seed :ai-kb:
-Step 1a. Clone =git@cjennings.net:ai-kb.git= to =~/.local/share/ai-kb=. Author =AGENT_CONTRACT.org= (canonical repo-resident contract: node format, write protocol, operations, routing) and seed =index.org= + a README/index node with a generated =:ID:=. Node format per spec — provenance (=:CREATED_BY:/:CONFIDENCE:/:VISIBILITY:/:SOURCE:/:STATUS:=), =:PROJECTS:= slugs, type filetags, relation labels. Define the durable external-pointer format as *ID-first*: =ai-kb: <Title> (<UUID>)=, resolved by ID with title fallback (filenames can change in curation).
+Step 1a. Clone =git@cjennings.net:ai-kb.git= to =~/.local/share/ai-kb=. Author =AGENT_CONTRACT.org= (canonical repo-resident contract: node format, write protocol, operations, routing) and seed =index.org= + a README/index node with a generated =:ID:=. Node format per spec — a *required* one-line =:SUMMARY:= (the index/query read it straight, no inference/LLM), provenance (=:CREATED_BY:/:CONFIDENCE:/:VISIBILITY:/:SOURCE:/:STATUS:=), =:PROJECTS:= slugs, type filetags, relation labels. Define the durable external-pointer format as *ID-first*: =ai-kb: <Title> (<UUID>)=, resolved by ID with title fallback (filenames can change in curation).
*** TODO [#B] ai-kb CLI 1a: index, lint, remember, doctor :ai-kb:
-Step 1a. Shell wrapper calling =emacs --batch= for org work. =index= regenerates =index.org= from node properties (never hand-maintained). =lint= = org-lint fatal checks + duplicate IDs + broken id-links (excl =raw/=) + missing required props + bad project slugs + stale/incomplete index + credential scan of nodes *and* =raw/=. =remember= = the write protocol: fetch + =pull --ff-only= (abort on diverge/dirty), write, regenerate index, then run the *full =ai-kb lint=* over the change as the commit gate (not just node org-lint — this is the safety boundary), commit locally, =flock=; no push. =doctor= = health + push-state report (repo, private remote, CLI on PATH, adapter linked, db buildable, no secrets, "ahead N"/"push failed"/"diverged").
+Step 1a. Shell wrapper calling Emacs for org work — =emacsclient= when a daemon is up, =emacs --batch= fallback, lint+index in *one* invocation per =remember=. =index= regenerates =index.org= from node properties incl. =:SUMMARY:= (never hand-maintained); the index references nodes as plain =Title (UUID)= text, never =[[id:]]= links, and is excluded from the scan so it can't manufacture backlinks or hide orphans. =lint= = org-lint fatal checks + duplicate IDs + broken id-links (excl =raw/= + index) + missing required props (incl =:SUMMARY:=) + bad project slugs + stale/incomplete index + credential scan of nodes *and* =raw/= text files (binaries skipped). =remember= = the write protocol: fetch + =pull --ff-only= (abort on diverge/dirty), write, regenerate index, then run the *full =ai-kb lint=* over the change as the commit gate (not just node org-lint — this is the safety boundary), commit locally, =flock=; no push. =doctor= / =status= = health + push-state + raw-dir-size report (repo, private remote, CLI on PATH, =graphviz= if the map needs it, adapter linked, db buildable, no secrets, "ahead N"/"push failed"/"diverged"); =status= is the fast non-diagnostic mode for the dashboard/nudge.
*** TODO [#B] claude-rules/ai-kb.md adapter :ai-kb:
Step 1a. Global L1 rule in rulesets pointing at the repo-resident =AGENT_CONTRACT.org=: path, routing (T1/T2/T3 tiers; per-project =MEMORY.md= shrinks to ID-first pointers into ai-kb), proactive + contradiction rules, concrete "read the index first" triggers, link-grep recipes, "use =ai-kb remember=, never bypass =ai-kb lint=", one-line nudge on unpushed commits / recorded push rejection. =make install= symlinks it into =~/.claude/rules/=.
@@ -60,29 +60,35 @@ Step 1a. Global L1 rule in rulesets pointing at the repo-resident =AGENT_CONTRAC
Step 1a (core; the timer-install line is added with 1b). Idempotent =scripts/setup-ai-kb.sh=: clone (or init+add-remote on first machine), seed, install the CLI on PATH, =ai-kb index=, =ai-kb doctor=. =make ai-kb-init= wraps it. The one-time server bootstrap stays a separate documented step.
*** TODO [#B] ai-kb Step-1a tests :ai-kb:tests:
-Write-path: a write with the remote unreachable still commits locally and does not error; =flock= serializes concurrent =remember=; each org-lint *fatal* check (malformed drawer, missing/dup =:ID:=, invalid required property, missing =#+title:=, unparseable org) rejects the commit, a style warning does not; =remember= aborts the commit when the *full* lint fails (stale index, broken link, secret in =raw/=). Index: regen from a fixture produces expected entries; an out-of-band node appears only after regen. Link recipes: backlink (excl =raw/=) + forward correct. Provisioning (bats): idempotent, valid =:ID:=, =doctor= passes.
+Write-path: a write with the remote unreachable still commits locally and does not error; =flock= serializes concurrent =remember=; each org-lint *fatal* check (malformed drawer, missing/dup =:ID:=, invalid required property, missing =#+title:=, unparseable org) rejects the commit, a style warning does not; a node missing =:SUMMARY:= fails lint; =remember= aborts the commit when the *full* lint fails (stale index, broken link, secret in a node or =raw/= text file); the credential scan skips binaries. Index: regen from a fixture produces expected entries; an out-of-band node appears only after regen; a node referenced only by =index.org= still reports as an orphan (the index is not a backlink source). Link recipes: backlink (excl =raw/= + index) + forward correct. Provisioning (bats): idempotent, valid =:ID:= + =:SUMMARY:=, =doctor= passes.
*** TODO [#B] ai-kb CLI 1b: query, curate, sync :ai-kb:
-Step 1b. =query <context>= with a *testable contract*: plain-text default + =--json=; fields title/ID/summary/projects/status/updated/path; searches index rows + title/tags/properties/body; default max-results + most-recently-updated ordering; =raw/= paths only as source references; exit codes for no-match / invalid KB / lint-index failure. =curate --dry-run= (four buckets; destructive ops human-only). =sync= (=org-roam-db-sync= against ai-kb).
+Step 1b. =query <context>= with a *testable contract*: plain-text default + =--json=; fields title/ID/summary/projects/status/updated/path + *match reason*; searches index rows + title/tags/properties/body; ranks by lexical score — sum of each matched field's weight, counted once per field: title 100, tag/project/status 50 each, summary 20, body 5; no term-frequency weighting in v1 — with most-recently-updated (=:UPDATED:=) only as the *tie-break* on equal scores (recency alone buries stable old preferences); default max-results; =raw/= paths only as source references; exit codes for no-match / invalid KB / lint-index failure. =show <id-or-title>= (resolve ID-first, print the node) and =backlinks <id>= (excl =raw/= + index) as the inspection primitives the Emacs commands wrap. =curate --dry-run= (four buckets; also flags orphan =raw/= captures and any =raw/= file over 256 KB; destructive ops human-only). =sync= (=org-roam-db-sync= against ai-kb) only when the db is missing/stale or forced.
*** TODO [#B] ai-kb push timer + failure observability :ai-kb:
Step 1b. =ai-kb-push.timer= + =ai-kb-push.service= =systemd --user= units: push only if ahead, ~15 min; installed + =enable --now= by the setup script (add this line to =setup-ai-kb.sh=). A failed push is logged to a state file (=$XDG_STATE_HOME/ai-kb=), never fatal; surfaced by =ai-kb doctor= and the adapter's startup nudge.
*** TODO [#B] ai-kb-curate workflow in rulesets :ai-kb:
-Step 1b. =~/code/rulesets/.ai/workflows/ai-kb-curate.org= — human-gated curation: the four buckets, node-count trigger, =:LAST_CURATED:= rotation, pointer-integrity (merge/supersede changes the canonical ID, so grep inbound =[[id:]]= + =MEMORY.md= =ai-kb: ... (UUID)= refs and repoint before deleting). Surfaced by =ai-kb doctor= + session startup when due.
+Step 1b. =~/code/rulesets/.ai/workflows/ai-kb-curate.org= — human-gated curation: the four buckets, node-count trigger (nudge at 150 nodes, re-fire every +50), =:LAST_CURATED:= rotation, pointer-integrity (merge/supersede changes the canonical ID, so grep inbound =[[id:]]= + =MEMORY.md= =ai-kb: ... (UUID)= refs and repoint before deleting). Surfaced by =ai-kb doctor= + session startup when due.
*** TODO [#B] ai-kb Step-1b tests :ai-kb:tests:
-=query --json= returns the specified fields/ordering/exit-codes on a fixture KB and =raw/= appears only as a source ref; a simulated push failure is recorded to the state file and surfaced by =ai-kb doctor=.
+=query --json= returns the specified fields (incl. match reason)/exit-codes on a fixture KB and =raw/= appears only as a source ref; a title match outranks a body-only match with recency only breaking ties (an old preference is not buried under a newer body-only hit); a simulated push failure is recorded to the state file and surfaced by =ai-kb doctor= / =status=. Performance (=:perf= tag): 100- and 1,000-node fixtures keep =index=/=query=/=lint=/=remember= under a stated time budget (catches an accidental per-check Emacs startup or an O(n²) scan).
-*** TODO [#B] Emacs: org-roam ai-kb switch + guard :ai-kb:
+*** TODO [#B] Emacs: org-roam ai-kb profile + switch :ai-kb:
Step 2.
-=org-roam-config.el=: ai-kb dir constant + =org-roam-ai.db=; =cj/org-roam-switch-to-ai-kb= / =cj/org-roam-switch-to-personal= with the guard contract — rescope the completed-task→daily hook, sync on entry, restore both vars + the hook exactly on exit, re-assert personal state at startup (abnormal-exit safety). =cj/ai-kb-db-sync= helper; =org-roam-file-exclude-regexp= excludes =raw/=.
+=org-roam-config.el=: =cj/org-roam-switch-to-ai-kb= / =cj/org-roam-switch-to-personal= install a full org-roam *profile*, not a two-variable swap — dir + =org-roam-ai.db= + =org-roam-file-exclude-regexp= (=raw/= + =index*.org=), and dailies, capture templates, topic/project/recipe find wrappers, and the agenda/refile + completed-task→daily hooks all rescoped or neutralized so ai-kb nodes never leak into personal journals/agenda. Restore everything exactly on exit; re-assert personal state at startup (abnormal-exit safety). =cj/ai-kb-db-sync= syncs only when the db is missing/stale or forced, with a status indicator.
+
+*** TODO [#B] Emacs: ai-kb edit safety (same write path) :ai-kb:
+Step 2. An =ai-kb= minor mode whose =after-save-hook= runs the agent's post-write sequence under =flock= — =ai-kb index=, full =ai-kb lint=, commit, push-state update — so a human Emacs edit can't bypass index/lint/commit. One write path for both agent and human. Failure UX: the save always writes to disk and the buffer stays editable (never read-only/blocked); on lint failure it does *not* commit, pops findings to a =*ai-kb-lint*= buffer (no focus steal), and shows the uncommitted-failing state in the modeline + dashboard — Craig fixes and re-saves, a clean save commits. Recursion guard, two layers: the mode's activation predicate excludes =index*.org= + =raw/=, and the pipeline binds a re-entrancy flag (=cj/ai-kb--in-pipeline=) the hook early-returns on; index regen prefers =write-region= over =save-buffer=.
+
+*** TODO [#B] Emacs: ai-kb browsing surface :ai-kb:
+Step 2. =cj/ai-kb-dashboard= (status banner: active KB, node count, unpushed commits, push-failure state, curation due, last index/sync), =cj/ai-kb-find-node= (=org-roam-node-find= in the ai-kb profile), =cj/ai-kb-search= (=ai-kb query= or scoped =consult-ripgrep=), =cj/ai-kb-show-node= (resolve ID-first, open), =cj/ai-kb-backlinks= (excl =raw/= + index), =cj/ai-kb-map= (built-in =org-roam-graph= *first* — the profile's exclude regexp already keeps =raw/= + index out of the db, so the graph inherits the right scope; custom DOT export only if project/tag/status filtering proves necessary; =graphviz= dep). Simple wrappers over the CLI primitives where possible.
*** TODO [#B] Emacs: ai-kb keybindings + which-key :ai-kb:
-Bind the switch + sync commands under the =C-c n= roam prefix (e.g. =C-c n a= → ai-kb, =C-c n A= → personal), avoiding the dense existing set; which-key labels.
+Bind the switch + sync + browsing commands under the =C-c n= roam prefix (e.g. =C-c n a= → ai-kb, =C-c n A= → personal, a small transient for the browsing commands), avoiding the dense existing set; which-key labels.
*** TODO [#B] Emacs: ai-kb Step-2 ERT tests :ai-kb:tests:
-Switch sets the ai-kb dir + db; switch-back restores the personal values exactly; the completed-task hook does not fire into ai-kb while switched; startup re-asserts personal state after a simulated abnormal exit.
+Profile: switch installs the ai-kb dir + db + exclude regexp and switch-back restores personal *exactly* — completed-task hook, agenda/refile finalize hook, dailies, and capture templates all untouched by ai-kb while switched; startup re-asserts personal state after a simulated abnormal exit. Edit path: a save in an ai-kb buffer runs index+lint+commit (a bad save surfaces the lint failure rather than committing). Sync runs only when stale.
** TODO [#C] Manually verify cj/org-finalize-task journal copy :test:
Confirm the live behavior the unit tests mock out. In a real Emacs (org-roam loaded), run =C-; O d= on a level-3 sub-task and on a level-2 task. Expect the sub-task to flip to a dated entry, the level-2 to keep its keyword and gain a date-only CLOSED line, and in both cases a copy to land in today's daily under "Completed Tasks".