path: root/docs/interactive-grouping-spec.org

#+TITLE: pearl — Interactive Grouping Spec
#+AUTHOR: Craig Jennings
#+DATE: 2026-06-06
#+STARTUP: showall

* Status

*Ready — review rounds 1-2 (Codex) incorporated (Craig, 2026-06-06).* Companion to [[file:issue-sort-order-spec.org][issue-sort-order-spec.org]]: that doc added "sort the view you're looking at" without hand-editing a source; this one adds "group the view you're looking at." The two are siblings — both change how the active buffer is laid out, both persist on the source so a refresh reproduces them, and both must move existing subtrees rather than re-render from parsed data so unsaved edits survive.

Round 1 closed four blocking gaps: the missing implementation-phase plan, the =:client-group= resolver contract across every grouping consumer, the false "cycle is in the buffer" assumption (cycle isn't rendered into the drawer, so it's deferred from v1), and undefined malformed-subtree placement. Round 2 (rubric =Ready with caveats=, no HP) cleaned up a stale "all five dimensions" sentence and added the explicit test-surface task. Dispositions are in Review dispositions below.

* Problem

Grouping is server-driven only. When a Linear Custom View is configured with an =issueGrouping=, pearl reads it, stamps =:group= on the source, and renders issues under level-2 group headings (issues at level 3). But to group a buffer that has no grouping — My Open Issues, a filter, or a Linear view with grouping unset — the user has to open Linear's web UI, set the grouping there, and re-fetch. There's no "group this by status" command for the view in front of you.

* Current state

- =pearl--group-issues= partitions normalized issues into ordered groups by a grouping string; =pearl--view-group-label= maps each issue to its bucket ("No project" etc. for an unset dimension). Supported dimensions: =workflowState=, =project=, =assignee=, =priority=, =cycle= (single-valued; label grouping and sub-grouping are deferred).
- =pearl--build-org-content= renders the grouped shape: group headings at level 2, issues at level 3, group headings carry no =LINEAR-ID= so save/merge skip them like the help header. An ungrouped source renders the flat level-2 list.
- The merge-refresh path is grouping-aware: =pearl--merge-append-grouped= places a new issue under its group section, =pearl--merge-issues-into-buffer= takes a grouping argument, and the reorder/merge preserve heading level. Every consumer reads =(plist-get source :group)= directly today: =pearl--build-org-content=, the dirty-merge branch of =pearl--update-org-from-issues=, =pearl--merge-query-result=, and =pearl--merge-issues-into-buffer= / =pearl--merge-append-grouped=. The =:client-group= resolver (below) must replace each of those reads, or refresh will redraw the server grouping and ignore the user's interactive choice.
- =:group= lives on the source plist and persists in =#+LINEAR-SOURCE=, so a refresh reproduces the grouping. It is only ever set from the view's Linear preference today — never interactively.
- The drawer data contract is the constraint on which dimensions v1 can offer. =pearl--format-issue-as-org-entry= (pearl.el:2882) renders =LINEAR-STATE-NAME=, =LINEAR-PROJECT-NAME=, =LINEAR-ASSIGNEE-NAME=, and =LINEAR-PRIORITY= into the drawer — so status, project, assignee, and priority are all recoverable from buffer text. It does *not* render cycle. The normalized issue carries =:cycle=, but only at render time; once an issue is on disk its cycle is gone. So an offline client-side regroup can group by the first four dimensions but not by cycle. Cycle is deferred from v1 (see Review dispositions HP3); a separate task adds cycle drawer fields and then re-enables cycle grouping.
- =pearl-set-sort='s client reorder (=pearl--reorder-issue-subtrees=) moves whole subtrees byte-for-byte at a fixed level. It refuses on a grouped buffer (returns =grouped=), because it walks the parent's direct level-2 children and a grouped buffer has group headings there instead.

* Proposed design

** The command

=pearl-set-grouping= (interactive, in the active file): =completing-read= the dimension (=status= / =project= / =assignee= / =priority= / =none=), coerce the display string to the Linear =issueGrouping= value the render path already speaks, regroup the buffer, persist on the source. =none= ungroups back to the flat level-2 list. =cycle= is deferred from v1 — it isn't in the drawer (see Current state). =pearl--check-grouping= refuses an unknown dimension before any buffer change or header write, mirroring =pearl--check-sort-order=; a legacy =:group= / =:client-group= value pearl doesn't section renders flat, matching the current render helpers. Transient/keymap placement deferred to a follow-up, same as sort shipped M-x-only in v1.

** Client-side only — group what you're looking at

All v1 dimensions (status, project, assignee, priority) are recoverable from the drawer fields already in the buffer, so grouping is a pure client-side re-layout: no refetch, instant, works offline. This is simpler than sort's split (sort needs the server for =updated=/=created=); grouping has no server branch in v1. (Cycle would need a refetch or a representation change, which is why it's deferred — see Current state.)

** Regroup in place: move subtrees, shift their level, never reconstruct (the crux)

Regrouping changes an issue's outline level — flat (level 2) becomes grouped (level 3 under a level-2 group heading), and =none= reverses it. The reorder must preserve unsaved edits the same way sort does, so it moves existing subtree text rather than re-rendering from parsed data. The added wrinkle over sort is the level shift. The command:

1. Captures each issue subtree's region text byte-for-byte (description, comments, drawers, unsaved edits intact), keyed by =LINEAR-ID=.
2. Computes each issue's group label from the *buffer* data, not a refetched plist: reads the relevant drawer field (=LINEAR-STATE-NAME= / =LINEAR-PROJECT-NAME= / =LINEAR-ASSIGNEE-NAME= / =LINEAR-PRIORITY=) for the chosen dimension and maps it to a bucket label the same way =pearl--view-group-label= does (a small buffer-reading label helper, or reparse a minimal plist per subtree and reuse =pearl--view-group-label=). Computes the target level: 3 when grouping, 2 when =none=.
3. Shifts every heading line within each captured subtree by the level delta — the issue heading and its comment children all move by the same +1 (group) or -1 (ungroup). Only the leading stars change; body, drawers, and comment text are untouched, so edits survive.
4. Emits the level-2 group headings in group order and lays each group's issue subtrees beneath, then replaces the old issue block in one delete+insert.
5. Re-folds afterward (=pearl--restore-page-visibility=) — the same fix b589445 applied to sort, since this rewrites the issue block too.
6. A subtree with no =LINEAR-ID= (a malformed or non-issue subtree) is never dropped and gets a defined, stable position: kept after all issue groups when grouping, and after all issue subtrees when ungrouping, preserving the non-issue subtrees' original relative order (mirroring sort's "sorted last, stable").

A test edits a description and a comment locally, groups by status, and asserts the exact edited text is intact under the right group heading at the right level.

** Persistence, the effective-grouping resolver, and every consumer

Mirror the sort model, with one resolver every grouping consumer routes through. Interactive grouping is recorded under =:client-group= for *all* source types (uniform key — simpler command code, one resolver), via =pearl--source-with-grouping= (analogous to =pearl--source-with-sort=). A view keeps its own Linear grouping in =:group=; a filter leaves =:group= nil. =none= clears =:client-group= so a view falls back to its server =:group= on the next refresh, and a filter falls back to flat.

=pearl--effective-grouping= (analogous to =pearl--effective-sort-order=) returns =:client-group= when present, else =:group=. This is the contract's load-bearing piece: every place that reads =(plist-get source :group)= today must read the effective value instead, or refresh redraws the server grouping and the interactive choice silently vanishes. The call sites to convert:

- =pearl--build-org-content= (full render).
- the dirty-merge branch of =pearl--update-org-from-issues=.
- =pearl--merge-query-result= (refresh).
- =pearl--merge-issues-into-buffer= / =pearl--merge-append-grouped= (new-issue placement).

=#+LINEAR-SOURCE= persists both keys where applicable: a view's =:group= stays recorded alongside any =:client-group= override, so clearing the override restores the view's own grouping without a re-fetch.

** Interaction with sort

=pearl--group-issues= keeps each group's issues in input order, so a client sort already applied to the buffer survives inside every section. And once grouping is interactive, sorting within groups (the filed [#C] "interactive sort in a grouped view", currently refused) becomes the natural next step — out of scope here, noted so the two features stay aligned.

** Outcome messages

"Grouped current buffer by status" / "Ungrouped current buffer." On a buffer with no issues, "No issues to group in this buffer." (parallel to sort's messages).

* v1 decisions (this feature)

1. Command: =pearl-set-grouping=, =completing-read= dimension incl. =none=. M-x only; transient/keymap deferred.
2. Client-side only — no refetch; group labels come from the buffer drawer fields.
3. Regroup moves subtrees byte-for-byte with a per-subtree heading-level shift; edits preserved; re-fold after.
4. Persist interactively-chosen grouping under =:client-group= for all source types (uniform key); =pearl--effective-grouping= prefers it over a view's server =:group=; =none= clears it. Every current =:group= consumer routes through the resolver.
5. Group order: first-appearance, matching the existing grouped render. Natural per-dimension order is vNext (MP1).
6. Supported dimensions: status / project / assignee / priority. Cycle deferred (not in the drawer — HP3); label and sub-grouping deferred.
7. =pearl--check-grouping= refuses unknown interactive input before any change; a legacy unsectioned =:group= / =:client-group= renders flat (MP3).
8. Malformed / non-issue subtrees kept stable and last — after all groups when grouping, after all issues when flat (HP4).

* Files touched

- =pearl.el= — =pearl-set-grouping= command; =pearl--grouping-choices= + display→value coercion; =pearl--check-grouping= validator; =pearl--source-with-grouping= + =pearl--effective-grouping= resolver; the regroup-in-place helper (subtree capture + buffer-read label + level shift + group emit + malformed-last), parallel to =pearl--reorder-issue-subtrees=; convert the four =:group= consumers to the resolver; header persistence; re-fold call. No GraphQL/network code — v1 touches rendered buffer text plus the source header only.
- =tests/test-pearl-grouping.el= (new or extend) — partition already covered; add resolver precedence, =pearl--source-with-grouping= persistence/clear, regroup-in-place (level shift, edit preservation, ungroup round-trip, malformed placement both directions), and the refresh paths consuming effective grouping.
- =README.org= — a "Grouping the current view" subsection alongside "Sorting the current view".

* Test plan

- Resolver: =pearl--effective-grouping= on a view with server =:group=, a view with =:client-group=, a cleared =:client-group=, and a non-view source.
- Persistence: =pearl--source-with-grouping= writes =:client-group= to =#+LINEAR-SOURCE=; a refresh reproduces the grouping; =none= clears it so a view falls back to its server grouping and a filter falls back to flat.
- Refresh integration: the full render and both merge-refresh paths group by the effective value, not raw =:group=.
- Normal: group a flat buffer by status — issue headings shift level 2→3, group headings at level 2, in first-appearance order; "No status" bucket for unset.
- Boundary: =none= on a grouped buffer shifts issue headings 3→2 and removes group headings; group→none round-trips to the flat shape.
- Edit preservation: a locally-edited description and comment survive the regroup (exact-string), under the right group at the right level.
- Malformed: a non-issue subtree lands last and stable in both directions.
- Validation: unknown interactive input refused before any header write; a legacy unsectioned =:group= renders flat.
- No-issues buffer: refuses with a message, no header write.

* Implementation phases

1. *Source + resolver plumbing.* =pearl--grouping-choices= + display→value coercion, =pearl--check-grouping=, =pearl--source-with-grouping=, =pearl--effective-grouping=, header write/clear. Tests: coercion, validation, persistence/clear, resolver precedence.
2. *Buffer regroup core.* Capture issue + non-issue subtrees; compute each issue's label from drawer data; shift heading levels; emit grouped/flat block; malformed-last; re-fold. Tests: level shift both directions, edit preservation, round-trip, malformed placement.
3. *Command + UX.* =pearl-set-grouping= completion, outcome messages, no-issues and missing-=#+LINEAR-SOURCE= handling (shared active-source error).
4. *Refresh / render integration.* Convert =pearl--build-org-content=, the dirty-merge branch, =pearl--merge-query-result=, and =pearl--merge-issues-into-buffer= / =pearl--merge-append-grouped= to the resolver. Tests: each path groups by effective value.
5. *Docs.* README "Grouping the current view"; source-level naming/coverage check.

** Implementation tasks (drop-in for todo.org)

#+begin_src org
** TODO [#B] Implement interactive grouping spec                   :feature:
Spec: docs/interactive-grouping-spec.org (Ready). Group the active buffer on demand, client-side, parallel to pearl-set-sort.
*** TODO [#B] Grouping source + resolver plumbing :feature:solo:
pearl--grouping-choices + coercion, pearl--check-grouping, pearl--source-with-grouping, pearl--effective-grouping, header write/clear. Phase 1.
*** TODO [#B] Buffer regroup core (capture, level-shift, malformed-last, re-fold) :feature:solo:
Phase 2. Move subtrees byte-for-byte; read group label from drawer; shift heading stars by the level delta; non-issue subtrees last+stable; re-fold after.
*** TODO [#B] pearl-set-grouping command + UX :feature:solo:
Phase 3. completing-read dimension incl. none; outcome messages; no-issues + missing-source handling.
*** TODO [#B] Route every grouping consumer through pearl--effective-grouping :feature:solo:
Phase 4. build-org-content, dirty-merge branch, merge-query-result, merge-issues-into-buffer / merge-append-grouped.
*** TODO [#C] README "Grouping the current view" + naming check :docs:solo:
Phase 5.
*** TODO [#B] Interactive grouping test surface :test:solo:
Cover resolver precedence, header persistence/clear, effective grouping in render and both merge-refresh paths, level shift both directions, edit preservation, malformed-last placement, validation, and no-issues behavior. Spec: docs/interactive-grouping-spec.org (Test plan).
#+end_src

* vNext / out of scope

- Grouping by cycle — blocked on rendering cycle into the issue drawer (filed [#C] "Render cycle into the issue drawer"); re-enable =cycle= in =pearl-set-grouping= once it lands.
- Interactive sort within a grouped view (filed [#C]) — pairs with this feature.
- Label grouping and two-level sub-grouping (an issue with several labels files under each; Linear's =issueGrouping= + sub-grouping).
- Natural per-dimension group ordering (priority Urgent→None, workflow-state position); v1 ships first-appearance.
- Transient/keymap entry for =pearl-set-grouping=.

* Review dispositions

Round 1 (Codex, 2026-06-06) — rubric Not ready, 4 HP + 3 MP. All accepted; HP3 resolved by deferral.

- *HP1 (no implementation phases / drop-in task block)* — accepted. Added Implementation phases (5) + the drop-in todo.org block.
- *HP2 (define the effective-grouping resolver and route every consumer)* — accepted. Added =pearl--effective-grouping=; enumerated the four call sites (=build-org-content=, dirty-merge branch, =merge-query-result=, =merge-issues-into-buffer= / =merge-append-grouped=) that must switch from raw =:group=.
- *HP3 (cycle isn't in the buffer)* — accepted, resolved by deferring cycle from v1. Verified against pearl.el:2882: the drawer renders state/project/assignee/priority but not cycle. v1 offers the four buffer-backed dimensions; cycle grouping is vNext, gated on a separate task that adds cycle drawer fields. Filed [#C] "Render cycle into the issue drawer". Chosen over adding cycle fields now because that grows every rendered issue and touches the save/merge hashing surface — a bigger change than this feature should carry.
- *HP4 (malformed-subtree placement undefined)* — accepted. Defined: non-issue subtrees kept last and stable, after all groups when grouping and after all issues when flat. Tests both directions.
- *MP1 (group order)* — accepted. First-appearance locked for v1; natural per-dimension order to vNext.
- *MP2 (filter source-key model)* — accepted. Uniform =:client-group= for all source types; =:group= reserved for a view's own Linear grouping.
- *MP3 (unknown-value validation)* — accepted. =pearl--check-grouping= refuses unknown interactive input before any change; a legacy unsectioned value renders flat.

Round 2 (Codex, 2026-06-06) — rubric Ready with caveats, no HP, 2 MP. Both accepted; both cleanups.

- *R2-MP1 (stale "all five dimensions" sentence)* — accepted. The Client-side-only section still claimed all five dimensions were buffer-computable, reintroducing the HP3 error. Rewritten to name the four v1 dimensions and note cycle's deferral.
- *R2-MP2 (no explicit test-surface task)* — accepted. Added =[#B] Interactive grouping test surface :test:solo:= to the drop-in todo block, per the workflow's one-test-entry requirement. Rubric raised to Ready.

* Review and iteration history

** 2026-06-06 Saturday @ 13:21:52 -0500 — Codex — reviewer

- *What changed or was recommended:* Ran the spec-review workflow. Rubric =Not ready=. Wrote a blocking review covering missing implementation phases, incomplete =:client-group= effective-grouping plumbing, the false rendered-data assumption for =cycle= grouping, and undefined malformed-subtree placement.
- *Why:* The spec has the right user-facing direction but still leaves implementation-critical state, refresh, and handoff contracts for the implementer to invent.
- *Artifacts:* [[file:interactive-grouping-spec-review.org][interactive-grouping-spec-review.org]]; deferred vNext items logged in [[file:../todo.org][todo.org]].

** 2026-06-06 Saturday @ 13:48:12 -0500 — responder

- *What changed or was recommended:* Dispositioned all 7 findings (4 HP + 3 MP), all accepted — see Review dispositions. Added Implementation phases + the drop-in todo.org block (HP1); added =pearl--effective-grouping= and named its four consumers (HP2); deferred cycle from v1 and corrected the data-contract claim in Current state (HP3); defined malformed-subtree placement (HP4); locked first-appearance order (MP1), uniform =:client-group= (MP2), and =pearl--check-grouping= validation (MP3). Status raised Draft → Ready.
- *Why:* The review was right on every point; HP3 was a genuine factual error in the draft (cycle isn't rendered into the drawer, verified at pearl.el:2882).
- *Artifacts:* this spec; filed [[file:../todo.org][todo.org]] [#C] "Render cycle into the issue drawer (unblocks grouping by cycle)". Review file deleted after fold-in.

** 2026-06-06 Saturday @ 13:56:18 -0500 — responder (round 2)

- *What changed or was recommended:* Round 2 rubric =Ready with caveats=, no HP, 2 MP — both accepted (see Review dispositions). Rewrote the stale "all five dimensions" sentence in Client-side-only to name the four v1 dimensions plus cycle's deferral (R2-MP1); added the standalone =[#B] ... test surface :test:solo:= task to the drop-in block (R2-MP2). Rubric raised to Ready.
- *Why:* Both were document-consistency cleanups, not design changes; the stale sentence re-asserted the round-1 cycle error and the workflow wants a discrete test task in the handoff block.
- *Artifacts:* this spec. Round-2 review file deleted after fold-in.

** 2026-06-06 Saturday @ 13:52:41 -0500 — Codex — reviewer

- *What changed or was recommended:* Re-ran the spec-review workflow after the response pass. Rubric =Ready with caveats=. No high-priority blockers remain; recommended fixing one stale "all five dimensions" sentence and adding the explicit test-surface task required by the workflow.
- *Why:* The round-1 design blockers are resolved, but the spec still has two small handoff/document consistency issues to clean up before marking fully =Ready=.
- *Artifacts:* [[file:interactive-grouping-spec-review.org][interactive-grouping-spec-review.org]].

** 2026-06-06 Saturday @ 14:15:36 -0500 — Codex — reviewer

- *What changed or was recommended:* Re-ran the spec-review workflow after the round-2 response. Rubric =Ready=. Confirmed the stale "all five dimensions" sentence was corrected to the four drawer-backed v1 dimensions, and the drop-in todo block now includes the required standalone test-surface task.
- *Why:* The prior caveats are resolved and no remaining implementation-blocking ambiguity was found.
- *Artifacts:* this spec; no new review file was written because the spec is implementation-ready.