#+TITLE: Drill Deck Review Workflow #+AUTHOR: Craig Jennings & Claude #+DATE: 2026-05-30 * Overview Take an org-drill flashcard file and bring it into the canonical shape — every card a question that doesn't give the answer away, every fact current — then regenerate the Anki =.apkg= and drop it where the phone can sync it. The workflow has three substantive passes (question-form audit, content-accuracy audit, source rewrite) followed by a mechanical regenerate-and-place step. Content review is dispatched to a subagent because it's bounded research across project source-of-truth files; the structural rewrite stays in the main thread because it touches the SRS state we don't want to lose. Three helper scripts (=drill-deck-stats.py=, =drill-deck-diff-ids.py=, =drill-deck-sync=) automate the inventory, the safety check, and the regenerate-and-place. * When to Use This Workflow Trigger phrases: - "Review the drill deck" - "Update the drill deck" - "Refresh the Anki cards" - "Let's run the drill-deck-review workflow" Typical timing: - After a wave of personnel changes (titles, roles, employment status) - After a major milestone (a demo ships, a contract closes, a submission goes in) - When org-drill review surfaces a card with stale or wrong content - When the Anki deck on the phone hasn't been regenerated in weeks * Inputs - *Source file*: the org-drill file. Common locations: - =deepsat.org= at the work project root (symlinked from =~/sync/org/drill/=) - =health-drill.org= in the health project - Any =:drill:= deck under =~/sync/org/drill/= - *Source-of-truth docs for content accuracy*: project-specific. Typical set: - Project-root =knowledge.org=, =status.org=, =notes.org= - =todo.org= for the freshest signal on people / partnerships / projects - =deepsat/assets/= (or equivalent) for meeting transcripts when a specific fact needs confirmation - *Output location*: =~/sync/phone/anki/.apkg= (the phone-sync target). Both =drill-to-anki.py= and the =drill-deck-sync= wrapper default there. * Canonical Card Shape ** Deck title (=#+TITLE:= line) The =#+TITLE:= line at the top of the source file drives two surfaces: the org-drill display in Emacs and the Anki deck name on the phone. Pick a title that reads well in Anki — drop tool-name jargon like "Org-Drill" / "Drill" that's meaningful in Emacs but noise on the consumption side. Good: =DeepSat Flashcards=, =Health Flashcards=, =Philosophy Flashcards=. Bad: =DeepSat Org-Drill Flashcards=, =DeepSat Drill Deck=. =drill-deck-stats.py= flags any title containing =org-drill= (case-insensitive, hyphenated or spaced) as a workflow violation. *Stable-ID caveat.* =drill-to-anki.py= derives the Anki deck ID from the deck name. Changing =#+TITLE:= changes the deck ID, so the next import lands as a new deck rather than updating the existing one. Two consequences worth flagging: - Any review history accumulated in Anki under the old deck name stays attached to the old deck — it doesn't migrate. - On rename, delete the old deck from Anki to avoid having two decks with similar content. For most decks (especially on first deployment), this is a one-time event. The rename is cheap to do early. ** Heading (the question) Every card heading is a question that doesn't reveal the answer. Not the topic name, not the acronym, not the person's name — a question that tests recall. Three card families have different question shapes: *** Acronym / concept cards "What does X stand for and what is it?" or "What is X and why does it matter?" Promote the question that was already in the body up to the heading. Before: : ** AFRL :drill: : What does AFRL stand for and what is it? : *** Answer : Air Force Research Laboratory. ... After: : ** What does AFRL stand for and what is it? :drill: : Air Force Research Laboratory. ... *** Person cards Format: "Who is X? Tell me about their Y." where X is a role descriptor that doesn't name the person, and Y is whatever the answer body covers (background, role, limitations, scope). The answer body opens by naming the person, then continues. Before: : ** Vrezh Mikayelyan :drill: : Who is Vrezh? What are his key limitations? : *** Answer : Developer (also called "Reg"). Armenia-based ... After: : ** Who is DeepSat's Armenia-based developer? Tell me about his background and limitations. :drill: : Vrezh Mikayelyan. Armenia-based, full-time as of April 2026. Worked with Hayk at Bazoomq on Armenia's first satellite ... Note: pick a role descriptor that genuinely identifies one person. If multiple people share the role description, add a single distinguishing detail (e.g., "the one who works evenings", "the Vineti alum"). Don't pile on parentheticals. *** Talking-points and directive cards Already in prompt form ("Introduce Yourself", "Spell out these orbital regime acronyms", "What is DeepSat?"). Leave the heading alone. Still strip the =*** Answer= sub-header and audit the body content for staleness. The =drill-deck-stats.py= helper recognizes both =?=-form and imperative-verb form as valid prompts (verbs like Spell, Describe, Explain, Name, List, Give, Show, Tell, Define, Compare, Identify, Outline, Introduce, Walk, State, Recite, Recall, Summarize). ** Body (the answer) - *No =*** Answer= sub-header.* The body /is/ the answer; the heading /is/ the question. The sub-header was a workaround for topic-as-heading cards. - *Body opens by naming the topic.* "Air Force Research Laboratory. Air Force's R&D arm." or "Vrezh Mikayelyan. Armenia-based, full-time as of ..." The Anki back shows this directly under the front question; restating the topic makes the back read as a complete answer. - *PROPERTIES drawer stays.* Org-drill needs the =:ID:=, =:DRILL_LAST_INTERVAL:=, =:DRILL_EASE:= etc. for SRS state. The Anki output strips it (see the script change). - *=SCHEDULED:= / =DEADLINE:= planning lines stay.* Same reason. The Anki output strips them. * Approach: Phases ** Phase A: Question-form + title audit (per card and per file) Run =drill-deck-stats.py= on the source first to get the structural inventory: #+begin_src bash .ai/scripts/drill-deck-stats.py #+end_src The script reports the deck title from =#+TITLE:= (and flags it if it contains source-tool jargon like "Org-Drill"), card count, PROPERTIES-drawer count, =*** Answer= sub-header count, cards missing =:ID:=, and cards whose heading is neither =?=-form nor an imperative-verb prompt. Each surfaced card is a candidate for the rewrite, plus the title itself if flagged. For each candidate, propose the new heading in advance so Phase C is mechanical. For person cards, the proposal is the role descriptor + topical anchor pair. For acronym/concept cards, the proposal is the existing body question promoted to the heading. ** Phase B: Content-accuracy audit (subagent) Dispatch a subagent with this contract (adapt the source-of-truth list per project): #+begin_example Audit the answer bodies of every :drill: card in against and surface every fact that is stale, wrong, or out-of-date. Don't rewrite the cards; report only items that need to change. Output shape per finding: CARD: CURRENT: UPDATE: CONFIDENCE: high / medium / low Categories to look for: - Personnel: title changes, employment-status changes, departures, new joiners - Partner status: contracts that closed or fell through, partnerships that advanced or stalled, named individuals changing roles - Project facts: milestone shifts, submission states, exercise / demo dates - External contacts: title or affiliation changes - Company facts: head count, funding, customer status Skip cards where you find no staleness. Cap at 2,000 words. #+end_example Include any user-supplied seed fixes in the dispatch (e.g., "Vrezh is now full-time, drop the 'Reg' diarization error"). The subagent folds them into its report so they land in the same disposition table. Output of Phase B: a structured per-card list of content updates with confidence levels. High-confidence findings get baked in during Phase C. Medium-confidence findings are reviewed inline before baking. Low-confidence findings are surfaced but skipped unless the user calls them in. ** Phase C: Source rewrite Take Phase A's question-rewrite plan and Phase B's content-update list, apply them to the source file. Preserve every card's =:PROPERTIES:= drawer (especially =:ID:=) and =SCHEDULED:= line verbatim — those carry SRS state that must survive the rewrite. Rewrite shape per card: #+begin_example ** :drill: [SCHEDULED line if present] :PROPERTIES: [ID + DRILL_* lines unchanged] :END: #+end_example Drop the =*** Answer= sub-header entirely. The body that was under =*** Answer= becomes the body of the card. If the original body had a question above =*** Answer= (the pre-rewrite norm), drop that question — the new heading carries it. For the file as a whole, use a single =Write= rather than per-card =Edit= calls. One pass through the source, one write back. Per-card edits multiply tool calls by N and risk drift. ** Phase D: Regenerate the Anki deck Use the =drill-deck-sync= wrapper — it runs the stats check, optionally the ID-preservation check, then regenerates the apkg and places it at =~/sync/phone/anki/=: #+begin_src bash .ai/scripts/drill-deck-sync --diff-against #+end_src The =--diff-against= flag is recommended on any rewrite where you want to confirm zero card IDs disappeared (zero SRS-state loss). The "previous version" is typically the file as it was before this run; grab it from git with =git show HEAD~1: > /tmp/-prerewrite.org=. Skip =--diff-against= on a first run when there's no previous version to compare against. If the stats check or ID-preservation check fails, the wrapper exits non-zero and the apkg is not written. Fix the warnings, then re-run. To bypass the safety gates (rare, only when you know what you're doing), call =drill-to-anki.py= directly: #+begin_src bash .ai/scripts/drill-to-anki.py --output ~/sync/phone/anki/.apkg #+end_src The script writes the =.apkg= with stable deck/model IDs derived from the deck name, so re-importing into Anki updates existing cards rather than duplicating them. ** Phase E: Verify The =drill-deck-sync= wrapper covers the structural verify automatically (stats + diff-ids if =--diff-against= was passed). After it succeeds, do a quick visual spot-check: - Confirm the apkg size matches expectations. Significant changes are expected on a big rewrite; a wildly smaller file may mean the parser dropped cards. - Open the source in Emacs (or =head -100=) and confirm a few cards visually: question heading, no =*** Answer=, PROPERTIES preserved, body opens with topic name. - If you keep an org-drill session open, revert the buffer to pick up the rewrite. For ad-hoc verification on either side of a rewrite, run the individual scripts: #+begin_src bash .ai/scripts/drill-deck-stats.py .ai/scripts/drill-deck-diff-ids.py #+end_src ** Phase F: Commit Two clusters: - *Source rewrite*: the org file (e.g., =deepsat.org=). Commit subject: =chore(drill): restructure cards to question-form headings + content refresh=. Body lists the content-update categories (Vrezh full-time, DCVC passed, etc.) and notes that =*** Answer= sub-headers were dropped. - *Workflow / script changes* (if any): if this run prompted updates to =drill-deck-review.org= or the helper scripts in the rulesets repo, commit those separately with =chore(workflows):= or =chore(scripts):= subjects. Push both. The =.apkg= itself lives under =~/sync/phone/= which is outside the repo — no commit needed there; Syncthing (or whatever sync mechanism) handles propagation. * Helper Scripts Three scripts under =.ai/scripts/= (canonical lives in =rulesets/claude-templates/.ai/scripts/=): ** =drill-to-anki.py= The core converter. Reads an org-drill source file, emits a stable-ID Anki =.apkg=. Strips =:PROPERTIES:= drawers and =SCHEDULED:= / =DEADLINE:= / =CLOSED:= planning lines from card bodies before rendering. Front = heading text without =:drill:=. Back = cleaned body, HTML-escaped, joined with =
=. Deck and model IDs derived from the deck name + a salt, so re-imports update existing cards rather than duplicating. ** =drill-deck-stats.py= Inventory + workflow-violation warnings for a single deck source. Counts cards, PROPERTIES drawers, =*** Answer= sub-headers, cards missing =:ID:=, and cards whose heading is neither =?=-form nor an imperative-verb prompt. Exits 0 when clean, 1 when warnings present, so it gates =drill-deck-sync=. Imperative-verb allowlist: Spell, Describe, Explain, Name, List, Give, Show, Tell, Define, Compare, Identify, Outline, Introduce, Walk, State, Recite, Recall, Summarize. ** =drill-deck-diff-ids.py= SRS-state preservation check between two versions of a deck. Extracts every =:ID:= from each, reports IDs that disappeared (lost SRS state — worst-case bug) or appeared (new cards). Exits 0 when clean, 1 when any disappeared/appeared. ** =drill-deck-sync= (bash wrapper) Single command for the canonical "rewrote the deck, now ship it" step. Runs =drill-deck-stats=, optionally =drill-deck-diff-ids= (with =--diff-against=), then =drill-to-anki= writing to =~/sync/phone/anki/.apkg=. Exits non-zero if any gate fails; the apkg is not written when a gate fails. Usage: #+begin_src bash drill-deck-sync drill-deck-sync --diff-against #+end_src * Anki Script Behavior The =drill-to-anki.py= script has these contracts that this workflow depends on: 1. *Strips =:PROPERTIES:= drawers* from the card body before rendering. Org-drill needs them in source; Anki cards shouldn't show them. 2. *Strips =SCHEDULED:= / =DEADLINE:= / =CLOSED:= planning lines* from the card body. Same reason. 3. *Does NOT strip =*** Answer= sub-headers.* If the source still has them, the Anki cards will show them. This workflow's Phase C removes them at the source. =drill-deck-stats.py= flags any remaining as a workflow violation. 4. *Front of each Anki card* = the heading text without the =:drill:= tag. 5. *Back of each Anki card* = the cleaned body (after #1 and #2), joined with =
= and HTML-escaped. 6. *Stable IDs* derived from the deck name + a salt, so re-importing the same deck name updates cards rather than duplicating. If you find the script doing something else, update the script before regenerating. Don't work around a script bug in the source rewrite — the next deck will hit the same problem. * Output Path Convention - Default in =drill-to-anki.py=: =~/sync/phone/anki/.apkg=. - Default in =drill-deck-sync=: =~/sync/phone/anki/.apkg= (same target; the wrapper passes =--output= explicitly). =~/sync/org/drill/= holds the org sources and their symlinks; =~/sync/phone/anki/= holds the =.apkg= the phone consumes. Both tools write the =.apkg= to the phone dir by default, so a deck lands where Anki picks it up without an =--output= override. * Common Mistakes 1. *Per-card =Edit= calls instead of one =Write=.* Multiplies tool calls and risks drift between cards. Read once, rewrite in memory, write once. 2. *Dropping the PROPERTIES drawer in source.* Org-drill stores SRS state there; losing it resets every card's review history. =drill-deck-diff-ids.py= is the safety net. 3. *Rewriting person headings to include the name.* "Who is Vrezh Mikayelyan?" gives away the answer. The whole point is to test name recall from a role description. 4. *Forgetting to strip =*** Answer= sub-headers.* The Anki output will show them as visible card content. =drill-deck-stats.py= catches this. 5. *Skipping the content-accuracy pass.* The structural rewrite alone leaves stale facts in place. The drill cards become a memorization tool for the wrong information. 6. *Treating subagent output as gospel.* Medium- and low-confidence findings need human review before baking. The subagent surfaces; the main thread decides. 7. *Running =drill-deck-sync= without =--diff-against=.* The stats check still runs, but the SRS-state preservation check doesn't. On a rewrite of any size, pass =--diff-against /tmp/-prerewrite.org= (grab from git first). * Living Document Update this workflow as patterns emerge. Specifically: - New card family beyond acronym / person / talking-point → document the heading shape for it. - New source-of-truth doc beyond the standard set → add to Phase B's dispatch contract. - Script behavior changes → mirror them in the "Anki Script Behavior" section. - New imperative-verb prompt forms → add the verb to =drill-deck-stats.py=' s allowlist. ** Updates and Learnings *** 2026-05-30: First run Built against =deepsat.org= after Craig flagged that the existing apkg surfaced PROPERTIES drawers + =*** Answer= headers on the back of every card, and that the person-card content (Vrezh in particular) had drifted. The Phase B subagent surfaced 8 high-confidence content updates plus several medium-confidence enrichments. Validated by running the rewrite and regenerating =deepsat.apkg= to =~/sync/phone/anki/=. *** 2026-05-30: Helper scripts added (same day) After the first run, scripted the safety-net checks into three helpers: =drill-deck-stats.py= (inventory + warnings), =drill-deck-diff-ids.py= (SRS-state preservation between versions), and =drill-deck-sync= (single-command wrapper). Stats check on the deepsat rewrite flushed a heuristic bug — directive prompts ("Spell out these orbital regime acronyms", "Introduce Yourself") were flagged as non-question. Fix: =drill-deck-stats.py= now accepts =?=-form OR imperative-verb-start (Spell, Describe, Explain, ..., Recall) as valid prompt forms. *** 2026-05-30: Title-audit added (same day) Craig noticed the Anki deck name still showed as "DeepSat Org-Drill Flashcards" because the source =#+TITLE:= leaks tool-name jargon into Anki. Added a "Deck title" subsection under Canonical Card Shape, expanded Phase A to audit the title, and extended =drill-deck-stats.py= to flag any title matching =org[-\s]?drill= (case-insensitive). Stable-ID caveat documented: renaming the deck changes the Anki deck ID, so the next import lands as a new deck and the old one needs deleting from Anki.