feat: pearl — manage Linear issues from org-mode

Pearl fetches Linear issues into an org file and syncs edits back. It covers list / custom views / saved queries, per-issue and bulk rendering with comments inline, conflict-aware sync of descriptions, titles, and comments, field commands for priority / state / assignee / labels, and a transient dispatch menu. The render folds to a scannable outline and nests issues under a sortable parent.

Based on and inspired by Gael Blanchemain's linear-emacs.

1 files changed, 230 insertions, 0 deletions

diff --git a/docs/issue-representation-spec.org b/docs/issue-representation-spec.org
new file mode 100644
index 0000000..912cee9
--- /dev/null
+++ b/docs/issue-representation-spec.org
@@ -0,0 +1,230 @@
+#+TITLE: pearl — Issue Org Representation & Editing Spec
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-05-23
+#+STARTUP: showall
+
+* Status
+
+*DRAFT — review incorporated (2026-05-23), awaiting final go-ahead.* Design proposal; nothing in =pearl.el= has changed. v1 scope is decided (see [[*Agreed v1 decisions][Agreed v1 decisions]]); deferred items in [[*vNext][vNext]]; modifications/rejections in [[*Review dispositions][Review dispositions]].
+
+Companion to [[file:issue-query-spec.org][issue-query-spec.org]] (which covers *fetching/filtering*). This doc covers how an issue is *rendered and edited* once it's in org. They share the org→Linear write path and the single active-file output model.
+
+* Problem
+
+Open a fetched issue in org and the body below the drawer is empty — and with =org-tidy= folding the drawer, the whole entry looks blank. The one piece of free-text the issue has (its description) isn't missing; it's misfiled into a *property*. Users can't tell what they're allowed to edit.
+
+Grounded in =pearl--format-issue-as-org-entry= (=pearl.el:1114-1173=): the description is written into a =:DESCRIPTION: |= property as 2-space-indented lines (=l.1154-1158=), inside the drawer; the body after =:END:= (=l.1171=) is empty; =org-tidy= then hides the drawer and the entry reads as blank. That's the root of "I opened the task and there's nothing there / I'm not sure what I can edit."
+
+* Current rendering (what exists today)
+
+Each issue is a =***= heading =*** <TODO-state> <priority> <title>= (=l.1146=) plus a drawer carrying =:ID:= (Linear UUID), =:ID-LINEAR:= (the =ENG-123= identifier), =:TEAM:=, =:DESCRIPTION:= (the misfiled body), =:PRIORITY:=, =:LABELS:=, =:PROJECT:=, =:LINK:=, =:PROJECT-ID:= (=l.1148-1171=). Nothing below the drawer. The fetch query pulls =description= (=l.1119=) but *not* comments. State sync resolves team name → ID by network lookup each time (slow, fragile on rename/collision). The renderer strips =[ ]= from titles (=l.1145=) — existing lossy title behavior.
+
+* Agreed v1 decisions
+
+Settled in the 2026-05-23 review.
+
+1. *The org issue body is entirely Linear-owned in v1.* No local-only notes area. The active org file is a synchronized representation of Linear, not a mixed local/remote workspace.
+2. *Fetched comments are remote-owned display content.* Users can *add* comments; editing existing comments is vNext (and then only comments authored by the current Linear user, matching Linear's permissions).
+3. *New entries use only namespaced =LINEAR-*= properties.* No compatibility layer for the old =:ID:= / =:ID-LINEAR:= shape (no users yet).
+4. *Description sync starts as an explicit command only.* Automatic save-triggered description sync is vNext, after no-op detection and conflict handling are proven.
+5. *V1 conflict handling is detect / refuse / message.* Interactive diff-merge or local/remote choice is vNext.
+
+* Content ownership and refresh semantics
+
+The hard part isn't moving the description — it's distinguishing machine-owned fetched content from user edits once refresh, comments, and sync coexist. v1 makes this simple by fiat (decision 1: the whole body is Linear-owned), but the layout and refresh model still have to be explicit.
+
+** Generated entry layout
+
+#+begin_src org
+*** TODO [#B] ENG-123 Title
+:PROPERTIES:
+:LINEAR-ID:               <uuid>
+:LINEAR-IDENTIFIER:       ENG-123
+:LINEAR-URL:              https://linear.app/.../ENG-123
+:LINEAR-TEAM-ID:          <id>
+:LINEAR-TEAM-NAME:        ENG
+:LINEAR-PROJECT-ID:       <id>
+:LINEAR-PROJECT-NAME:     Foo
+:LINEAR-STATE-ID:         <id>
+:LINEAR-STATE-NAME:       In Progress
+:LINEAR-ASSIGNEE-ID:      <id>
+:LINEAR-ASSIGNEE-NAME:    Craig
+:LINEAR-LABELS:           [bug, p1]
+:LINEAR-DESC-SHA256:      <hash of last-fetched markdown>
+:LINEAR-DESC-UPDATED-AT:  <remote timestamp>
+:END:
+
+Description text managed by Linear (org-rendered).
+
+**** Comments
+***** <author> — <timestamp>
+comment body
+#+end_src
+
+*Store IDs and display names separately* for team, project, assignee, state, labels (and later cycle). Commands display names; they mutate by ID. This kills the per-render network name-lookup.
+
+*Provenance for the description* lives as a *hash + remote timestamp* in properties — not the full raw markdown. A large multiline markdown property is awkward in org and bad with folding. When the sync/no-op check needs the exact last-fetched markdown, fetch current remote markdown before deciding, or keep it in an internal cache keyed by =LINEAR-ID=. (See [[*Conflict handling][Conflict handling]].)
+
+** Refresh model — merge by ID, reconciled with the active-file output model
+
+The query spec's output model says *switching to a different view/query replaces the active file*. This spec's refresh says *don't wholesale-rewrite*. Both hold, for different actions:
+
+- *Switching source* (run a different view/query) → the issue set changes; replace the file contents after the dirty-buffer + conflict checks. One issue appears in one place.
+- *Refreshing the same source* (=refresh-current-view=, =refresh-current-issue=) → *merge by =LINEAR-ID===: update each existing issue subtree in place, add new matches, drop issues no longer in the result. Per subtree, run the conflict check before overwriting a description that was edited locally but not yet pushed.
+
+A wholesale rewrite on same-source refresh would clobber un-pushed description edits; merge-by-ID + per-subtree conflict check is what protects them.
+
+* Proposed model — body is editable content, drawer is machine-managed metadata
+
+Organizing principle: the body holds what a human reads and writes (description, comments); the drawer holds structured fields commands manage. An =org-tidy= user edits body text + runs commands and never touches the drawer.
+
+** Description → body
+
+Render the description as the heading body (org-converted — see [[*Markdown vs org — the conversion question][conversion]]). Opening a task now shows its description; the org-tidy blank-entry problem disappears. The body is the editable region; an explicit command (decision 4) pushes edits back, behind the conflict gate.
+
+** Drawer = command-managed fields
+
+State (TODO keyword), priority, labels, project, assignee live in the drawer/heading and change via dedicated commands ("Set assignee, priority, labels" task), which resolve names→IDs. =org-tidy= users never need to open the drawer.
+
+** Comments as a body subtree
+
+Fetch comments (needs a query change — not pulled today) and render *oldest-first* as =****= → =*****= sub-headings (=<author> — <timestamp>=, body beneath), so the thread reads chronologically and "add comment" appends at the end. =pearl-add-comment= creates a new comment via =commentCreate= and inserts/refreshes the returned comment. Fetched comments are remote-owned (decision 2): editing an existing comment heading does *not* sync back in v1.
+
+*Comment shape (verified against the published schema).* =Issue.comments= → =CommentConnection= (nodes/pageInfo); each =Comment= has =body= (markdown — runs through the same conversion tier as the description), =createdAt=, and =user=. *=user= is nullable* — comments from integrations or bots have no user, carrying =botActor= / =externalUser= instead. The renderer must fall back to the bot/external actor name (or a literal like "(automation)") for the author rather than assuming a =user.name=. =commentCreate(input: CommentCreateInput!)= returns =CommentPayload= (=comment=, =success=); the input takes =body= + =issueId= (and optional =parentId=), with success checked the same way as issue creation before reporting.
+
+** Affordance + discoverable commands
+
+A one-line preamble note (body = description, edit + sync via command; Comments subtree = thread, add via command; fields = drawer, change via commands). But commands matter more than a note — expose discoverable ones that work from *anywhere inside an issue subtree*:
+
+=pearl-sync-current-issue=, =pearl-open-current-issue=, =pearl-add-comment=, =pearl-set-priority=, =pearl-set-assignee=, =pearl-refresh-current-issue=.
+
+** Sub-issues (later)
+
+Optional nested headings; out of scope for v1.
+
+* Markdown vs org — the conversion question
+
+Linear stores descriptions/comments as *markdown*; we want *org* in the body. The directions differ in difficulty.
+
+- *org → markdown (push):* =ox-md= is built in, but it is *not* round-trip-faithful for the subset Linear uses (see [[*ox-md rejected for push][ox-md rejected for push]]). Push is therefore a hand-rolled inverse of the fetch converter.
+- *markdown → org (fetch):* no built-in. The only place pandoc is tempting.
+
+** ox-md rejected for push
+
+The original recommendation was =org-export-string-as ... 'md= for the push direction. Empirical testing (2026-05-23) of =org→md(md→org(x))= over the conversion matrix showed *zero of nine samples round-trip cleanly*. =ox-md= injects a =# Table of Contents= header, inverts emphasis (org =*italic*= → md =**bold**=), *drops checkbox markers* (=- [x] done= → =- done=), converts fenced code to 4-space indented blocks (losing the language), and reindents lists.
+
+This breaks the conflict gate two ways: the no-op guard compares =hash(org→md(body))= against the stored =LINEAR-DESC-SHA256= (hash of the last-fetched markdown), so a lossy push makes *every* no-op sync look like an edit; and the lossy output would *corrupt content pushed back to Linear* (dropped checkboxes, lost code-fence languages). This is the same lossy-round-trip failure mode the spec already rejected pandoc for — it applies to =ox-md= too.
+
+The push converter (=pearl--org-to-md=) is therefore hand-rolled as the symmetric inverse of the fetch converter (=pearl--md-to-org=), which makes round-trips byte-stable for the supported subset. Owning both directions also keeps the conversion tier self-consistent. *Two documented lossy edges remain* (inherent to the fetch converter, not the push side): a markdown =# heading= renders to a bold line on fetch and stays a bold line on push (restoring =#= would fork the org outline); single-asterisk markdown italics are unsupported on fetch (only =_underscore_= italics convert).
+
+** Pandoc — pros/cons
+
+- *Pros:* full-fidelity bidirectional GFM↔org; one tool; battle-tested.
+- *Cons:* hard external-binary dependency (MELPA-hostile; users without it get broken sync); subprocess per conversion; *lossy round-trip* (pandoc reflows/normalizes → spurious diffs on no-op fetch/push); cross-platform/version drift.
+
+** Recommendation — pure-elisp default, pandoc optional
+
+Hand-roll *both* directions: push via =pearl--org-to-md= (the inverse of the fetch pass — see [[*ox-md rejected for push][ox-md rejected for push]]), fetch via the lightweight md→org pass. No dependency, byte-stable round-trips. Pandoc is an *optional* enhancement: if =(executable-find "pandoc")= and a defcustom opts in, route both directions through it. Detected, never required — MELPA-safe.
+
+** Conversion matrix (the testable contract)
+
+"High-frequency constructs" needs a precise, testable subset. Unsupported constructs are *preserved as literal text*, never emitted as malformed org.
+
+| Markdown | Org | Note |
+|----------+-----+------|
+| =**bold**= | =*bold*= | |
+| =*italic*= / =_italic_= | =/italic/= | underscores in identifiers must not trigger emphasis |
+| =`code`= | =~code~= | |
+| =```lang ... ```= | =#+begin_src lang ... #+end_src= | language preserved |
+| =- item= / =* item= | =- item= | |
+| =1. item= | =1. item= | |
+| =- [ ]= / =- [x]= | =- [ ]= / =- [X]= | checkboxes |
+| =[text](url)= | =[[url][text]]= | |
+| => quote= | =#+begin_quote ... #+end_quote= | |
+| =# Heading= | *bold line*, NOT an org heading | an org heading would fork the issue subtree and corrupt structure |
+| tables / HTML / footnotes | literal pass-through | preserved, not converted |
+
+The =# Heading= → bold-line rule is load-bearing: converting a markdown heading inside a description to a real org heading would split the issue's subtree.
+
+* Conflict handling
+
+The round-trip-drift guard is necessary but not sufficient — it prevents no-op churn; it doesn't define conflicts. Promote it from a note to a *phase gate on sync-back*. The sync command compares three things:
+
+- *last-fetched* Linear markdown (hash in =:LINEAR-DESC-SHA256:=),
+- *current org-rendered* markdown (re-render the body to md, hash it),
+- *current remote* markdown / =updatedAt= (fetch before pushing).
+
+Outcomes:
+
+- org == last-fetched → no local edit → *no API call* (no-op guard).
+- org changed, remote == last-fetched → clean push.
+- org changed *and* remote changed since last fetch → *conflict*: stop, refuse to push, message the user (decision 5). Resolution workflows (diff/merge, local/remote-wins) are vNext.
+
+* Parsing — org-element, not regex
+
+Current parsing assumes a level-3 heading with a drawer immediately after and walks lines/regex. Once bodies and comment subtrees exist, that's brittle (misread drawers, nested comment headings mistaken for issues). Spec an =org-element=-based parser: locate issue headings by the durable =:LINEAR-ID:= property, read properties via org APIs, treat depth structurally — never =^\*\*\*= regexes.
+
+* Internal representation
+
+Normalize API responses into internal plists/structs *before* rendering, so the renderer never sees whether Linear returned a vector, =null=, or an omitted field. Comments, assignees, cycles, and views multiply the missing/null/vector handling otherwise. Model boundaries (filter compilation, API transport, issue/comment models, org rendering, org parsing, sync orchestration, commands) stay as *logical* sections — see [[*Review dispositions][Review dispositions]] on keeping a single file.
+
+* Actions a user wants in the body space
+
+- *Edit the description* in place → explicit sync (push, behind the conflict gate).
+- *Read the comment thread* without leaving Emacs.
+- *Add a comment* → =commentCreate= (append a sub-heading).
+- (later) navigate sub-issues.
+
+Field changes (assignee/priority/labels/state) stay command-driven, not body edits.
+
+* Impact on existing todo.org tasks
+
+Gives concrete shape to three already-open feature tasks; implement them together:
+
+- =Sync title and description back to Linear= — description-in-body + explicit push. *Phase title sync separately* from description (its own last-fetched-title hash + conflict behavior; note the existing bracket-stripping lossiness). Keep TODO-keyword state sync as the only automatic heading mutation in the first body-editing phase.
+- =Add a comment to an issue from Emacs= — the comment subtree + =commentCreate=.
+- =Set assignee, priority, and labels from Emacs= — command-driven drawer fields (mutate by ID).
+
+Cross-cuts the query spec at the shared write path and the active-file/refresh model.
+
+* Phased implementation
+
+1. *Description → body (read-only) + namespaced properties.* Move description out of =:DESCRIPTION:= into the body; switch to =LINEAR-*= properties storing IDs + display names; provenance hash + timestamp. Characterization test of the old shape first, then the new render; confirm =org-tidy= no longer shows a blank entry.
+2. *org-element parser.* Locate by =:LINEAR-ID:=, structural depth; replaces regex parsing before subtrees land.
+3. *Conversion tier.* Hand-rolled org→md push (inverse of the fetch pass; =ox-md= rejected for lossy round-trips) + lightweight md→org fetch per the matrix; unit-test the matrix (Normal/Boundary/Error) and the no-op round-trip invariant.
+4. *Refresh = merge by ID* + per-subtree conflict check; reconcile with the active-file replace-on-switch model.
+5. *Description sync-back (explicit command)* behind the conflict gate (the round-trip guard is the phase gate). Title sync as a separate step.
+6. *Comments* — add to the fetch query; render oldest-first; =add-comment= via =commentCreate=.
+7. *Pandoc optional path* + the affordance line + discoverable commands.
+8. *(later)* sub-issues, comment editing, local notes, save-hook automation, interactive conflict resolution.
+
+* Test strategy
+
+*Characterization (before changing rendering):* old shape renders description in =:DESCRIPTION:= with empty body; dirty visiting buffer not overwritten; state sync uses only matching issue headings; current parser behavior with drawer placement.
+
+*Per phase:* description after =:END:= with no =:DESCRIPTION:= property; org-element parser extracts properties even with body text + comment subtrees; comments render with IDs/timestamps oldest-first; =add-comment= makes one mutation and inserts/refreshes the returned comment; no-op description sync makes *no* API call; local-edit + remote-unchanged pushes the expected markdown; local-edit + remote-changed refuses with a conflict message; unsupported markdown stays readable and doesn't corrupt org.
+
+*Golden rendering:* small, intentional string snapshots of representative issue entries.
+
+* Open decisions
+
+None blocking v1 — the five agreed decisions resolved the ownership, conflict, sync-trigger, comment-immutability, and property-naming questions. Remaining calls are implementation-level (exact converter edge handling, command key bindings).
+
+* vNext
+
+- Local-only notes under issues, if a clean ownership representation emerges.
+- Editing existing comments — only those authored by the current Linear user.
+- Automatic description sync on save (after no-op detection + conflict handling are proven).
+- Interactive conflict handling: diff/merge, local-wins, remote-wins, manual merge.
+- Read-only text properties on remote-owned regions (after the command UX exists).
+- Sub-issue rendering.
+
+* Review dispositions
+
+All review recommendations were accepted and incorporated above except the following, modified with reasons:
+
+1. *"Split representation from network/API code" into modules → modified.* Adopted the *logical* boundaries (filter compilation / transport / models / rendering / parsing / sync / commands) and the "normalize before rendering" discipline, but *kept a single file* for v1. The package is a single-file =pearl.el= aiming at MELPA, where single-file is a virtue; splitting into multiple files is a larger restructuring with its own review. Logical sections + pure helpers get the unit-testability the review wants without the file split. Revisit multi-file only if size forces it.
+
+2. *Read-only text properties on remote-owned regions → deferred (the review's own lighter recommendation).* v1 detects edits to remote-owned generated areas and warns/refuses to push rather than making regions buffer-read-only, which would frustrate org users and complicate tests. Hard read-only is in vNext.
+
+Everything else — Linear-owned body, namespaced =LINEAR-*= properties, IDs-with-display-names, hash+timestamp provenance (not raw-markdown-in-property), merge-by-ID refresh reconciled with active-file replace, conflict detect/refuse/message as a phase gate, explicit-command sync, separate title/description sync, org-element parsing, the conversion matrix, oldest-first read/add-only comments, normalized model objects, discoverable subtree commands, and the full test strategy — was accepted as written.