aboutsummaryrefslogtreecommitdiff
path: root/docs/design
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2026-05-28 02:09:11 -0500
committerCraig Jennings <c@cjennings.net>2026-05-28 02:09:11 -0500
commit636c18341c1f9131bfabdd547cd60797d844a601 (patch)
treeb6c5db6ba8fc9d4f8de33a16334fc533c34ab76c /docs/design
parente373ba9e6dc1b288946ce57e3f29bf5372126dee (diff)
downloadorg-drill-636c18341c1f9131bfabdd547cd60797d844a601.tar.gz
org-drill-636c18341c1f9131bfabdd547cd60797d844a601.zip
docs: relocate v0 design specs to docs/design/
I moved both v0 design specs out of working/ and into docs/design/. That's the conventional permanent home for project documentation, where engineers will look during implementation. working/ is meant for transient in-progress artifacts that file away once the work ships, and these specs are long-lived design docs that don't fit that contract. Files moved: - working/stats-dashboard/stats-dashboard.org → docs/design/stats-dashboard.org - working/fsrs-spec/fsrs-spec.org → docs/design/fsrs-spec.org The git rename detection picked both up, so file history follows the move. I also dropped the stale /docs entry from .gitignore. The Makefile doesn't write to docs/ and nothing else references it as a build output, so the ignore was inherited cruft that would have silently dropped any tracked file under docs/. I updated path references in seven spots: three docstring/comment refs in org-drill.el, one in tests/test-org-drill-session-record.el (the Commentary block), and three inside the specs themselves. Two refs in fsrs-spec.org now point at the correct location for its defcustom docstring and option description. One in stats-dashboard.org's References section points at the sister spec. Full make test-unit green. eask compile clean.
Diffstat (limited to 'docs/design')
-rw-r--r--docs/design/fsrs-spec.org354
-rw-r--r--docs/design/stats-dashboard.org419
2 files changed, 773 insertions, 0 deletions
diff --git a/docs/design/fsrs-spec.org b/docs/design/fsrs-spec.org
new file mode 100644
index 0000000..e2e308a
--- /dev/null
+++ b/docs/design/fsrs-spec.org
@@ -0,0 +1,354 @@
+#+TITLE: FSRS scheduler for org-drill — v0 design spec
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-05-27
+
+* Status
+
+Draft v0 — pending decisions marked =DECIDE:= inline. The companion todo
+entry is =todo.org=:125 [#B] "FSRS (Free Spaced Repetition Scheduler)
+Algorithm". No implementation has started.
+
+* Context and motivation
+
+org-drill ships three SuperMemo-family schedulers — SM2, SM5, and Simple8.
+They share an interval/repeats/EF/failures/meanq/total-repeats state model
+inherited from the SuperMemo 2 paper and refined since. The card-state
+struct refactor (#147) bundles those fields into =org-drill-card-state= and
+makes plugging a new scheduler into the =cl-case= dispatch a one-arm change
+in =org-drill-smart-reschedule= and =org-drill-hypothetical-next-review-date=.
+
+FSRS (Free Spaced Repetition Scheduler) is the algorithm Anki has migrated
+to since 2024. It uses the DSR memory model (Difficulty, Stability,
+Retrievability) and a 17-to-21-parameter formula family fit on hundreds of
+millions of real reviews. Published evaluations show materially better
+retention than SM* at the same review load. Adding it as a fourth option
+to =org-drill-spaced-repetition-algorithm= lets a user opt in without
+disturbing existing SM-family decks.
+
+* Goals
+
+- A fourth value =fsrs= for =org-drill-spaced-repetition-algorithm=.
+- The =fsrs= arm of the =cl-case= in =smart-reschedule= and
+ =hypothetical-next-review-date= calls =org-drill-determine-next-interval-fsrs=
+ with the same shape the SM-family arms now use (a state value + the
+ rating quality + algorithm-specific extras).
+- Per-card FSRS state persisted to org properties. Existing SM-style
+ properties stay untouched on cards that haven't been switched.
+- Test coverage at parity with the SM* test files: Normal / Boundary /
+ Error per public function, plus reference-vector tests checking the
+ algorithm output against the canonical FSRS implementation.
+
+* Non-goals (v1)
+
+- *No parameter optimizer.* Anki's FSRS shines because of parameter
+ fitting on each user's review history. v1 ships fixed default
+ parameters only; the optimizer becomes its own todo and ticket. Once
+ the algorithm is in the tree, plugging an optimizer in front of it is
+ additive.
+- *No retro-migration heuristic.* A card switched from SM* to FSRS
+ cold-starts on its next review — FSRS treats it as a fresh first review.
+ Anki has heuristics that estimate initial =D= and =S= from prior review
+ history; we explicitly defer that as a follow-on.
+- *No bulk rescheduling pass.* Switching algorithms does not retroactively
+ rewrite SCHEDULED dates on existing cards. Cards keep their current
+ SCHEDULED until their next review.
+
+* Version pin
+
+=DECIDE:= FSRS-4.5 (recommended) vs FSRS-6.x.
+
+The current stable upstream is FSRS-6.3.1 (March 2026) with 21
+parameters. FSRS-4.5 is the last release whose generic defaults provide
+meaningful retention improvement over SM-family scheduling without
+optimization. The FSRS community's own guidance — quoting Expertium's
+algorithm explainer, March 2026 — is that "generic v6 defaults offer
+minimal improvement over v4.5" because v6's gains come from optimizing
+=w[17]..w[20]= on a user's data.
+
+Since v1 ships without an optimizer (above), the pragmatic pin is
+*FSRS-4.5*: 17 parameters, well-documented defaults, the most-published
+test vectors, and the version with the largest delta over SM* in the
+no-optimizer regime. Upgrading to v6 becomes a separate ticket alongside
+the optimizer one — by then there'd be a real reason to take v6's
+additional surface.
+
+Reference parameters (FSRS-4.5 defaults, source: =py-fsrs= up to v4 and
+the original =fsrs4anki= release notes):
+
+#+begin_src elisp
+;; w[0..16], FSRS-4.5
+(0.4 0.6 2.4 5.8 4.93 0.94 0.86 0.01 1.49 0.14 0.94 2.18 0.05 0.34 1.26 0.29 2.61)
+#+end_src
+
+(=DECIDE:= confirm this exact tuple before code lands. The 4.5 defaults
+were tuned twice during 4.x's life; pinning the final 4.5-stable tuple is
+a one-line lookup against the =py-fsrs= 4.x release tag, deferred to
+implementation.)
+
+* Algorithm shape (v4.5)
+
+The four-rating scale FSRS uses internally:
+
+| Rating | Name | Meaning |
+|--------+-------+------------------------|
+| 1 | Again | failed recall |
+| 2 | Hard | recalled with struggle |
+| 3 | Good | recalled normally |
+| 4 | Easy | trivial recall |
+
+org-drill uses a 0–5 quality scale. The mapping respects
+=org-drill-failure-quality= (default 2):
+
+| org-drill quality | FSRS rating |
+|-------------------+-------------|
+| 0, 1, 2 | Again (1) |
+| 3 | Hard (2) |
+| 4 | Good (3) |
+| 5 | Easy (4) |
+
+=DECIDE:= mapping table. Above is the natural reading of
+=org-drill-failure-quality=, but org-drill historically lets users
+remap session keys, and the mapping should respect that or be its
+own defcustom. Simplest: pin the mapping above for v1 and add
+=org-drill-fsrs-quality-mapping= as a future enhancement.
+
+** State model
+
+Per-card persisted state:
+
+| Property | Type | Meaning |
+|-----------------------+--------+--------------------------------------|
+| =DRILL_FSRS_STABILITY= | float | =S=, the memory-stability estimate |
+| =DRILL_FSRS_DIFFICULTY= | float | =D=, the difficulty estimate (1..10) |
+| =DRILL_FSRS_REVIEWS= | int | review count under FSRS (≥ 0) |
+| =DRILL_FSRS_LAPSES= | int | failures (rating=Again) under FSRS |
+| =DRILL_LAST_REVIEWED= | time | reused from the existing SM property |
+
+The last-reviewed timestamp is already written by the SM path
+(=org-drill-store-item-data= writes it via =org-set-property= elsewhere in
+the flow); FSRS reads the same property.
+
+A virgin FSRS card has all four =DRILL_FSRS_*= properties absent; the
+scheduler treats absence as "first review."
+
+** Update equations (v4.5 form)
+
+First review (no prior FSRS state) with rating R ∈ {1..4}:
+
+#+begin_src elisp
+S0 = w[R-1] ; initial stability per rating
+D0 = clamp (- w[4] (* w[5] (- R 2))) 1 10
+#+end_src
+
+Subsequent review at time =elapsed-days= since last review, with current
+stability =S=, difficulty =D=, and rating =R=:
+
+#+begin_src elisp
+;; Retrievability when the card is shown
+R-retrieval = (expt (1+ (/ elapsed-days (* 9 S))) -1)
+
+;; Difficulty update (linear with mean reversion toward w[4])
+D-delta = (* w[6] (- (- R 3))) ; positive on Again, neutral on Good
+D-new-raw = (+ D D-delta)
+D-new = (+ (* w[7] w[4]) (* (- 1 w[7]) D-new-raw)) ; mean reversion
+D-new = (clamp D-new 1 10)
+
+;; Stability update — success (R ≥ 2)
+S-new = S * (1 +
+ (exp w[8]) *
+ ((11 - D) / 9) *
+ (S ^ (-w[9])) *
+ ((exp ((1 - R-retrieval) * w[10])) - 1) *
+ (R == Hard ? w[15] : 1) *
+ (R == Easy ? w[16] : 1))
+
+;; Stability update — lapse (R = 1, Again)
+S-new = w[11] *
+ D^(-w[12]) *
+ ((S + 1)^w[13] - 1) *
+ exp((1 - R-retrieval) * w[14])
+#+end_src
+
+Next interval given desired retention =DR= and new stability =S=:
+
+#+begin_src elisp
+I = (* 9 S (- (expt DR -1) 1)) ; days, rounded down (floor)
+#+end_src
+
+Default desired retention is *0.9* (a common Anki default). Pinned as
+=org-drill-fsrs-desired-retention= defcustom.
+
+=DECIDE:= the exact form of the v4.5 update equations above is paraphrased
+from Expertium's algorithm walkthrough (March 2026, v6-oriented) and the
+=py-fsrs= reference implementation. Implementation will cross-check the
+exact constants and grouping against a tagged =py-fsrs= v4.x release
+before writing the function body.
+
+** Default parameters (org-drill defcustoms)
+
+#+begin_src elisp
+(defcustom org-drill-fsrs-parameters
+ '(0.4 0.6 2.4 5.8 4.93 0.94 0.86 0.01 1.49 0.14 0.94 2.18 0.05 0.34 1.26 0.29 2.61)
+ "17-tuple FSRS-4.5 default parameters. See =docs/design/fsrs-spec.org=.")
+
+(defcustom org-drill-fsrs-desired-retention 0.9
+ "Target retention for FSRS scheduling. Higher = shorter intervals.")
+#+end_src
+
+* Integration points
+
+** Public function
+
+#+begin_src elisp
+(defun org-drill-determine-next-interval-fsrs (state quality)
+ "Return next-interval (in days) plus updated FSRS state for STATE after
+QUALITY (0-5). STATE here is a 5-tuple (S D reviews lapses last-reviewed),
+not an `org-drill-card-state' — FSRS stores its own state and treats the SM
+slots as opaque.")
+#+end_src
+
+The shape =(state quality)= matches the post-#147 scheduler signature
+convention. =state= for FSRS is a *separate* struct =org-drill-fsrs-state=
+(or a plist — =DECIDE:= which), populated from the =DRILL_FSRS_*=
+properties at the call site.
+
+=DECIDE:= the return shape. Options:
+
+1. Same shape as the SM* schedulers (a list with =next-interval=
+ first plus updated state). Consistent but the trailing slots are
+ irrelevant to FSRS.
+2. A small =org-drill-fsrs-result= struct (=next-interval=,
+ =new-stability=, =new-difficulty=, =new-reviews=, =new-lapses=).
+ Cleaner for FSRS but introduces a return-shape special case in the
+ =cl-case= caller.
+
+Recommended: option 2. =cl-case= already special-cases =new-ofmatrix=
+for SM5; one more is fine and the FSRS state shape is genuinely different.
+
+** Caller integration
+
+In =org-drill-smart-reschedule=:
+
+#+begin_src elisp
+;; sketch — exact destructure shape TBD per the return-shape DECIDE
+(fsrs
+ (let ((fsrs-state (org-drill-get-fsrs-state)))
+ (org-drill-fsrs-apply-result
+ (org-drill-determine-next-interval-fsrs fsrs-state quality))))
+#+end_src
+
+Two new helpers parallel the existing item-data round-trip:
+
+- =org-drill-get-fsrs-state= — reads the =DRILL_FSRS_*= properties, returns
+ the state struct or nil for a virgin card.
+- =org-drill-store-fsrs-state= (or =org-drill-fsrs-apply-result=) — writes
+ the updated state back. The =SCHEDULED= timestamp is set via
+ =org-schedule= using the returned =next-interval=, the same way
+ =smart-reschedule= already does for SM*.
+
+Same in =hypothetical-next-review-date= (read-only, no store).
+
+** Defcustom entry
+
+=org-drill-spaced-repetition-algorithm= grows a fourth =:option= line:
+
+#+begin_src elisp
+- FSRS :: Free Spaced Repetition Scheduler, the DSR-based algorithm
+ used in Anki since 2024. See `docs/design/fsrs-spec.org` for the algorithm
+ and the state model.
+#+end_src
+
+* Backward compatibility
+
+A card last reviewed under SM* has =DRILL_LAST_INTERVAL=,
+=DRILL_REPEATS_SINCE_FAIL=, etc., but no =DRILL_FSRS_*=. On the first
+FSRS review after a switch:
+
+- The FSRS scheduler sees absent =DRILL_FSRS_*= and treats this as a
+ virgin first review (initial =S₀=, =D₀= from the rating).
+- The SM-style properties are left in place. If the user switches back,
+ the SM scheduler reads them as before.
+- The =DRILL_FSRS_*= properties are written from this point forward.
+ Cards drilled under both algorithms accumulate both property sets.
+
+The cold-start tradeoff is a known one-time loss of the SM-derived
+estimate of card difficulty. Anki's migration heuristic estimates
+initial =D= and =S= from the SM2 ease factor and interval; pulling that in
+is its own ticket (=todo:= "FSRS migration heuristic from SM history").
+
+=DECIDE:= whether to also surface a one-line "first FSRS review on this
+card — cold-starting" message during the prompt for the first FSRS review.
+Probably not — users don't need to know.
+
+* Test strategy
+
+Three categories per public function, matching the existing scheduler
+test files:
+
+1. =tests/test-org-drill-determine-next-interval-fsrs.el= — Normal /
+ Boundary / Error coverage of =org-drill-determine-next-interval-fsrs=.
+ *Reference-vector tests* check =(state, rating) -> (next-interval,
+ new-state)= against pre-computed outputs from =py-fsrs= at a tagged
+ v4.x release. About a dozen vectors covering the four ratings × first
+ review / nth review / lapse / long-elapsed-time, plus boundary ones
+ (=elapsed-days=0=, =D= and =S= at clamp limits).
+
+2. =tests/test-org-drill-fsrs-state-roundtrip.el= — get-fsrs-state /
+ store-fsrs-state round-trip parity, mirroring the SM
+ item-data-roundtrip test.
+
+3. =tests/test-org-drill-fsrs-integration.el= — end-to-end through
+ =smart-reschedule= with =org-drill-spaced-repetition-algorithm= set to
+ =fsrs=, covering: virgin card first review, returning card under
+ FSRS, switched-from-SM card cold-starts cleanly, both algorithms can
+ coexist on the same buffer.
+
+Mapping helpers (org-drill quality 0–5 → FSRS rating 1–4) and the
+defcustoms get their own small tests.
+
+The reference vectors get committed alongside the tests as a
+machine-checkable expected-output table, generated once from =py-fsrs=
+and pinned. The generation script lives at =tests/fixtures/fsrs-vectors.py=
+so re-generation is reproducible.
+
+* Effort estimate
+
+Multi-day, plausibly spanning sessions:
+
+- Algorithm function + helpers: 1 day.
+- State round-trip + integration in =smart-reschedule=/=hypothetical=: 0.5
+ day.
+- Test scaffolding + reference vectors + Normal/Boundary/Error: 1 day.
+- Documentation (manual entry, README option list, defcustom docstrings,
+ this spec ratified): 0.5 day.
+
+Realistic: a session to implement-and-test the algorithm + state
+round-trip with reference-vector tests; a follow-up session for the
+integration + docs. No optimizer.
+
+* Open decisions index
+
+Pinned in one place for the implementation gate:
+
+- =DECIDE:= version pin (FSRS-4.5 vs newer). Recommended: 4.5.
+- =DECIDE:= the exact FSRS-4.5 default-parameter tuple. One-line lookup
+ against the =py-fsrs= 4.x final release.
+- =DECIDE:= quality-mapping table (0–5 → 1–4). Recommended: above.
+- =DECIDE:= scheduler return shape — list (SM-shape) or
+ =org-drill-fsrs-result= struct. Recommended: struct.
+- =DECIDE:= whether to surface a "first FSRS review on this card"
+ message. Recommended: no.
+- =DECIDE:= the exact form of the update equations after cross-checking
+ =py-fsrs= 4.x source.
+
+* References
+
+- [[https://github.com/open-spaced-repetition/py-fsrs][py-fsrs]] — the canonical Python reference implementation. Tagged
+ v4.x releases pin FSRS-4.5; v6.x is the current line.
+- [[https://github.com/open-spaced-repetition/fsrs4anki][fsrs4anki]] — the Anki integration; its wiki has user-facing notes on
+ algorithm history and migration.
+- [[https://expertium.github.io/Algorithm.html][Expertium: A technical explanation of FSRS]] (March 2026, v6-oriented)
+ — the most accessible English-language walkthrough of the formulas
+ and parameter roles.
+- [[https://help.remnote.com/en/articles/9124137-the-fsrs-spaced-repetition-algorithm][RemNote: The FSRS Spaced Repetition Algorithm]] — third-party
+ implementer's overview, useful as a sanity check on terminology.
diff --git a/docs/design/stats-dashboard.org b/docs/design/stats-dashboard.org
new file mode 100644
index 0000000..6bb18d9
--- /dev/null
+++ b/docs/design/stats-dashboard.org
@@ -0,0 +1,419 @@
+#+TITLE: Comprehensive Statistics Dashboard for org-drill — v0 design spec
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-05-27
+
+* Status
+
+*Ratified 2026-05-27* — all 10 open decisions accepted as recommended.
+The companion todo entry is =todo.org=:20 [#A] "Comprehensive
+Statistics Dashboard". No implementation has started; the spec is now
+the implementation gate.
+
+See =Ratified decisions= at the bottom of this file for the locked
+choice on each question; inline =Decided:= blocks repeat the choice
+next to the section it affects.
+
+* Context and motivation
+
+org-drill's end-of-session report is =org-drill-final-report=
+(=org-drill.el=:3128) — a minibuffer prompt with the pass percentage,
+qualities histogram, and the new/mature/failed counts for the *single
+just-completed session*. Nothing about that report persists. Once the
+user presses a key to dismiss it, the only retained learning data is
+what each card carries in its own =DRILL_*= properties.
+
+The currently-retained per-card data is real and usable:
+
+| Property | Source |
+|----------+--------|
+| =DRILL_LAST_INTERVAL= | scheduler output, updated every review |
+| =DRILL_REPEATS_SINCE_FAIL= | review counter resetting on a lapse |
+| =DRILL_TOTAL_REPEATS= | lifetime review count for the card |
+| =DRILL_FAILURE_COUNT= | lifetime lapses |
+| =DRILL_AVERAGE_QUALITY= | running mean of the 0–5 quality scale |
+| =DRILL_EASE= | SM-family easiness factor |
+| =DRILL_LAST_QUALITY= | quality at the most recent review |
+| =DRILL_LAST_REVIEWED= | inactive timestamp of the most recent review |
+| =DATE_ADDED= | inactive timestamp of card creation |
+
+What's missing is the *temporal axis*: there's no record that the user
+reviewed N cards on a given day, or that the pass percentage trended
+upward over the last month, or that a particular card has been failed
+three times in the last two weeks. Those questions need a per-review
+or per-session event log that survives Emacs restarts.
+
+This spec sketches a dashboard that combines (a) per-card aggregates
+read live from existing properties (no new storage) with (b) a new
+persisted session/review history (org-persist, separate file) that
+gives the time-series view.
+
+* Goals
+
+- New command =org-drill-statistics= opens an interactive dashboard
+ buffer summarizing review history across all known drill files.
+- Per-card aggregates read live from the existing =DRILL_*= properties
+ — no migration, no new per-card properties.
+- A new persisted session log records, at the end of every session,
+ what landed (date, files scoped, cards reviewed, qualities histogram,
+ duration, pass percentage). Survives Emacs restarts.
+- Dashboard renders trends, distributions, and "needs attention" lists
+ by reading the session log + the per-card properties.
+- CSV export for users who want to bring the data into a spreadsheet
+ or a notebook.
+- No-op for users who don't open the dashboard — collection cost on
+ session end is bounded (single =persist-save= write), and the
+ dashboard command is lazy-loaded.
+
+* Non-goals (v1)
+
+- *No graphical charting.* The dashboard renders in plain org-mode
+ text — tables, unicode-block sparklines, and minibuffer summaries.
+ A Vega/SVG/PNG output stays out of v1; users who want charts use
+ CSV export.
+- *No per-review event log.* The persisted record is per-session, not
+ per-card-review. A full per-review log is more expensive to write,
+ more expensive to read, and crosses the line into "we are now a
+ learning-analytics platform." Per-card lifetime aggregates already
+ cover most card-level questions.
+- *No multi-machine sync.* The persist file is local; users who want
+ cross-machine continuity sync it themselves (org-roam-style,
+ Syncthing, whatever). Defining a sync protocol is out of scope.
+- *No retroactive backfill.* Sessions before this feature lands have
+ no history; the dashboard starts from the first session after the
+ upgrade. Per-card properties carry their own retroactive baseline.
+- *No forecasting.* "How many cards will be due tomorrow" is one of
+ the dashboard's panels (cheap — sum over SCHEDULED dates), but
+ predictive forecasting of retention or workload is FSRS territory
+ and stays there.
+
+* Data model
+
+** Live (no new storage)
+
+Read at dashboard-open time, no persistence:
+
+- Per-card aggregates by walking =org-map-entries= over the user's
+ configured =org-drill-scope= or an explicit scope argument. For each
+ drill entry, the existing accessors (=org-drill-entry-total-repeats=,
+ =-failure-count=, =-average-quality=, =-last-reviewed=, =-last-interval=,
+ =-ease=, =-last-quality=, =-days-since-creation=) yield the per-card
+ view without touching disk beyond reading the file.
+- Card status counts (new / young-mature / old-mature / overdue /
+ lapsed / due-tomorrow) by reusing =org-drill-entry-status= over the
+ scope. This is what the session opener already does in =org-drill=
+ for the initial counts.
+
+** Persisted (new storage)
+
+A single new persisted variable, the session log:
+
+#+begin_src elisp
+(persist-defvar org-drill-session-log nil
+ "List of completed-session records, newest first.
+
+Each entry is an `org-drill-session-record' struct.")
+#+end_src
+
+=org-drill-session-record= is a =cl-defstruct=:
+
+| Slot | Type | Meaning |
+|------+------+---------|
+| =start-time= | float | =float-time= at session start |
+| =end-time= | float | =float-time= at session end |
+| =scope= | symbol or list | =org-drill-scope= value at session start |
+| =algorithm= | symbol | =org-drill-spaced-repetition-algorithm= at start |
+| =qualities= | vector of int | every quality 0–5 entered, in order |
+| =pass-percent= | int | qualities > =org-drill-failure-quality=, / total |
+| =new-count= | int | size of =(oref session new-entries)= at end |
+| =mature-count= | int | =young-mature= + =old-mature= entries at end |
+| =failed-count= | int | =failed-entries= at end |
+| =cram-mode= | bool | =cram-mode= at session start |
+
+A single record is small (a few hundred bytes). At one session per
+day, the log holds a year of history in well under 100 KB. At three
+sessions per day for ten years, still under 4 MB. No pruning needed
+for v1.
+
+=Decided:= persistence shape = =persist-defvar=. Integrates with the
+=persist= dependency already in =Package-Requires=, lives at
+=~/.emacs.d/persist/org-drill-session-log= by default, mirrors the
+=org-drill-sm5-optimal-factor-matrix= precedent including its
+=condition-case= wrapper for corrupt-load recovery. Plain elisp file
+and org-mode log file declined — the former is what =persist-defvar=
+already is under the hood, the latter trades structured I/O for
+parse-and-rewrite failure modes.
+
+=Decided:= corrupted-load recovery = log a single warning, start with
+a fresh empty log, rename the corrupt file to =.corrupt-YYYY-MM-DD=
+before next save so it isn't overwritten. Mirrors the SM5-matrix
+path verbatim. History loss is bounded; the precedent makes this
+consistent.
+
+** Out-of-scope storage
+
+Explicit non-storage to keep scope tight:
+
+- No per-review event log (a row per card-rating).
+- No per-card review history (a list of past timestamps + qualities
+ per card). =DRILL_AVERAGE_QUALITY= and =DRILL_LAST_REVIEWED= cover
+ the dashboard's needs; full per-card history is what FSRS would
+ need, and FSRS owns that question.
+- No per-deck (per-file) aggregate cache. Walking org files on
+ dashboard open is fast enough for the file counts org-drill users
+ actually have; if a user has 10 000 files this becomes an issue —
+ cross that bridge if it shows up.
+
+* UI shell
+
+The dashboard is a single command =org-drill-statistics= that opens a
+read-only org-mode buffer named =*Org Drill Statistics*= with the
+sections below. =Decided:= keymap = =q= bury, =g= refresh, =e=
+export-csv, =s= scope, =r= range, =a= algorithm-filter, =RET= follow
+the card link at point. None conflict with read-only org-mode
+bindings.
+
+** Section 1 — Overview
+
+A four-column summary table:
+
+#+begin_example
+| Total cards | New | Mature | Lapsed |
+|-------------+-----+--------+--------|
+| 412 | 18 | 367 | 27 |
+#+end_example
+
+Plus a one-line "last session" recap reading the most recent record
+from =org-drill-session-log= (date, duration, cards reviewed, pass %).
+
+** Section 2 — Trends
+
+Two unicode-block sparklines:
+
+- Reviews per day (last 90 days). X axis = day, Y axis = card count.
+- Pass rate per day (last 90 days). Same X axis, Y axis = 0..100.
+
+Range and bar count are defcustoms. Default 90 days = roughly a
+quarter, fits on one line at 1 cell/day.
+
+Below the sparklines, a small table of weekly aggregates for the last
+12 weeks (reviews, pass %, average duration).
+
+=Decided:= sparkline character set = quadrant blocks (▁▂▃▄▅▆▇█),
+8 levels. Emacs renders them fine by default; users on a font without
+them are rare enough to handle by documentation rather than a runtime
+fallback.
+
+** Section 3 — Distribution
+
+Quality histogram across all sessions in the log (or scoped — see the
+range filter below). A horizontal bar per quality 0..5 with the
+absolute count and the percentage.
+
+** Section 4 — Needs attention
+
+Three tables:
+
+- *Leech candidates* — cards with =DRILL_FAILURE_COUNT= ≥
+ =org-drill-leech-failure-threshold= and =DRILL_AVERAGE_QUALITY= below
+ =org-drill-statistics-leech-quality-threshold= (=Decided:= default
+ *2.5*, below the Hard boundary of 3). Link to card.
+- *Long-overdue* — cards with =DRILL_LAST_REVIEWED= ≥
+ =org-drill-lapse-threshold-days= ago. Sorted most-overdue first.
+- *Forgotten new* — cards with =DATE_ADDED= ≥ 14 days ago but
+ =DRILL_TOTAL_REPEATS= = 0 (or absent). Useful for catching cards
+ that never got into rotation.
+
+Each table caps at =org-drill-stats-attention-row-limit= rows
+(defcustom, default 10) with a "+N more" footer.
+
+** Section 5 — Forecast
+
+A one-line table for the next 7 days:
+
+#+begin_example
+| Today | +1 | +2 | +3 | +4 | +5 | +6 |
+|-------+----+----+----+----+----+----+
+| 34 | 18 | 22 | 9 | 41 | 12 | 6 |
+#+end_example
+
+Computed from SCHEDULED timestamps across the scope. No prediction —
+just count of cards already scheduled for each day.
+
+** Range filter
+
+A line at the top of the buffer carrying the active filters:
+
+#+begin_example
+Scope: file (~/notes/drill.org) Range: last 90d Algorithm: simple8
+#+end_example
+
+Filters are interactive at the buffer header (=s= cycles scope, =r=
+cycles range, =a= filters algorithm). Defaults: =org-drill-scope=, "last
+90d", "all algorithms".
+
+=Decided:= single buffer-wide filter for v1. Per-section filters
+multiply the UI surface and most users want the same window across
+sections.
+
+* Export
+
+=M-x org-drill-statistics-export-csv= writes one CSV per requested
+view to a user-chosen directory:
+
+- =sessions.csv= — one row per session record in the log. Columns
+ match the struct slots.
+- =cards.csv= — one row per drill entry in the active scope, with
+ every =DRILL_*= property plus the computed status.
+- =daily.csv= — one row per day in the active range, with reviews,
+ passes, fails, pass-percent, duration-minutes.
+
+=Decided:= column delimiter = =,= (CSV) with proper quoting via
+=csv-mode='s writer if available, else a hand-rolled =csv-quote=
+helper. Users who want TSV can run a one-line sed pipe.
+
+* Performance
+
+The expensive paths and their bounds:
+
+| Path | Cost | Mitigation |
+|------+------+------------|
+| Session log save | one =persist-save= per session | already wrapped in =condition-case=; cost is a few KB write |
+| Scope walk on dashboard open | =org-map-entries= over scope | same cost as a session open; cached for the dashboard's lifetime, refreshes on =g= |
+| CSV export | one walk + one file write | one-off; user-triggered |
+| Sparkline rendering | bucket the in-memory log by day | log is bounded; bucket-by-day is linear in log length |
+
+The dashboard does *not* run at session open or close — only when the
+user invokes =M-x org-drill-statistics=. Session-end pays one =persist-save=
+write. Idle Emacs pays nothing.
+
+=Decided:= sync dashboard open for v1. =org-map-entries= over a
+typical scope is well under a second. Async refresh
+(=run-with-idle-timer=, status line in the buffer) is a follow-on
+ticket if anyone reports >2 s on a large scope.
+
+* Integration points
+
+** New code
+
+- =org-drill-session-record= struct (=cl-defstruct=).
+- =org-drill-session-log= persistent variable.
+- =org-drill-record-session= — called once from the end-of-session
+ finalizer (=org-drill-finalize-session= or wherever
+ =org-drill-final-report= currently sits) when the session was not
+ aborted. Appends a record, =persist-save='s the log.
+- =org-drill-statistics= — interactive command, opens the dashboard.
+- =org-drill-statistics-mode= — minor-mode-like keymap on top of an
+ org-mode buffer (=q g e s r a=).
+- =org-drill-statistics--render-*= — one helper per section (overview,
+ trends, distribution, attention, forecast).
+- =org-drill-statistics-export-csv= — interactive command.
+
+** Existing code touched
+
+- =org-drill-final-report= grows a single call to =org-drill-record-session=
+ before the read-char-exclusive dismisses the report (so the record
+ lands even if the user dismisses immediately).
+- A defcustom group =org-drill-statistics= added as a sibling group
+ next to =org-drill-session= (=Decided:= sibling, not nested — the
+ dashboard isn't session-state and a separate group keeps Customize's
+ tree readable).
+
+** Aborted-session handling
+
+A session ended via =C-g= or quit doesn't reach =org-drill-final-report=.
+=Decided:= record nothing for these — an =unwind-protect= path to
+salvage partial sessions is deferred. The "abort discards" semantics
+matches =org-drill-on-timeout-action= =discard-current= already, and a
+partial record's pass percentage misrepresents what the user
+experienced.
+
+* Defcustoms (proposed list)
+
+#+begin_src elisp
+(defcustom org-drill-statistics-trend-days 90
+ "Number of days the trends section spans.")
+
+(defcustom org-drill-statistics-forecast-days 7
+ "Number of days ahead the forecast section spans.")
+
+(defcustom org-drill-statistics-attention-row-limit 10
+ "Maximum rows in each `Needs attention' table.")
+
+(defcustom org-drill-statistics-leech-quality-threshold 2.5
+ "Cards with `DRILL_AVERAGE_QUALITY' below this and at least
+`org-drill-leech-failure-threshold' failures appear in `Leech candidates'.")
+
+(defcustom org-drill-statistics-export-directory
+ (expand-file-name "org-drill-stats/" user-emacs-directory)
+ "Default directory for CSV exports.")
+#+end_src
+
+* Test strategy
+
+Three test files, matching the existing file-per-area convention:
+
+1. =tests/test-org-drill-session-record.el= — struct construction,
+ round-trip through =persist-save=/=persist-load=, log appending,
+ newest-first ordering, corrupted-load recovery.
+2. =tests/test-org-drill-statistics-aggregates.el= — given a fixture
+ session log + a fixture org file with known =DRILL_*= properties,
+ each of the dashboard's section-render helpers produces the
+ expected table. Aggregation math (pass %, weekly buckets,
+ sparkline buckets) is tested here with deterministic input.
+3. =tests/test-org-drill-statistics-integration.el= — end-to-end:
+ simulate three completed sessions via =org-drill-record-session=
+ against a fixture file, open the dashboard, assert the buffer
+ contents and the export-CSV output.
+
+Normal / Boundary / Error per public function on the helpers.
+Boundary cases worth pinning: empty log (no sessions yet), single
+session, range filter that selects zero days, scope that contains
+zero drill entries, the day-bucket histogram on the day-boundary
+edge (a session that crosses midnight).
+
+* Effort estimate
+
+Multi-day, plausibly spanning sessions:
+
+- Session record + persist round-trip + recording hook: 0.5 day.
+- Dashboard renderer (5 sections) + minor-mode keymap + range filter:
+ 1.5 days. Each section is a small helper; the time goes to layout
+ polish and the sparkline math.
+- CSV export: 0.5 day.
+- Test coverage at parity (the three files above, ~40 tests): 1 day.
+- Documentation (manual entry, README option list, defcustom
+ docstrings, this spec ratified): 0.5 day.
+
+Realistic: a session for the persist + recording layer with full
+tests, a session for the dashboard renderer, a follow-up session for
+the export + docs + polish.
+
+* Ratified decisions (2026-05-27)
+
+All 10 open questions resolved as recommended. Implementation can
+proceed against this spec.
+
+| # | Question | Resolution |
+|---+----------+------------|
+| 1 | Persistence shape | =persist-defvar=, mirroring the SM5 matrix |
+| 2 | Corrupted-load recovery | warn, fresh-start, rename to =.corrupt-YYYY-MM-DD= |
+| 3 | Sparkline character set | quadrant blocks (▁▂▃▄▅▆▇█) |
+| 4 | Filter scope | single buffer-wide filter |
+| 5 | CSV delimiter | =,= with proper quoting |
+| 6 | Dashboard-open mode | sync |
+| 7 | Aborted-session recording | record nothing; =unwind-protect= deferred |
+| 8 | Dashboard keymap | =q g e s r a RET= |
+| 9 | Leech-quality threshold default | 2.5 |
+| 10 | Defcustom group placement | sibling group =org-drill-statistics= |
+
+* References
+
+- =org-drill.el= around line 729 — the =org-drill-session= EIEIO class.
+ Source of the per-session in-memory state that becomes a record.
+- =org-drill.el= around line 3128 — =org-drill-final-report=. Hook
+ site for =org-drill-record-session=.
+- =org-drill.el= around line 540 — the existing =persist-defvar= use
+ for the SM5 matrix. Template for the session-log persist + the
+ =condition-case= wrapper for corrupt-load recovery.
+- =docs/design/fsrs-spec.org= — sister v0 spec, same DECIDE-marker
+ convention.