#+TITLE: pearl — Issue Org Representation & Editing Spec #+AUTHOR: Craig Jennings #+DATE: 2026-05-23 #+STARTUP: showall * Status *DRAFT — review incorporated (2026-05-23), awaiting final go-ahead.* Design proposal; nothing in =pearl.el= has changed. v1 scope is decided (see [[*Agreed v1 decisions][Agreed v1 decisions]]); deferred items in [[*vNext][vNext]]; modifications/rejections in [[*Review dispositions][Review dispositions]]. Companion to [[file:issue-query-spec.org][issue-query-spec.org]] (which covers *fetching/filtering*). This doc covers how an issue is *rendered and edited* once it's in org. They share the org→Linear write path and the single active-file output model. * Problem Open a fetched issue in org and the body below the drawer is empty — and with =org-tidy= folding the drawer, the whole entry looks blank. The one piece of free-text the issue has (its description) isn't missing; it's misfiled into a *property*. Users can't tell what they're allowed to edit. Grounded in =pearl--format-issue-as-org-entry= (=pearl.el:1114-1173=): the description is written into a =:DESCRIPTION: |= property as 2-space-indented lines (=l.1154-1158=), inside the drawer; the body after =:END:= (=l.1171=) is empty; =org-tidy= then hides the drawer and the entry reads as blank. That's the root of "I opened the task and there's nothing there / I'm not sure what I can edit." * Current rendering (what exists today) Each issue is a =***= heading =*** = (=l.1146=) plus a drawer carrying =:ID:= (Linear UUID), =:ID-LINEAR:= (the =ENG-123= identifier), =:TEAM:=, =:DESCRIPTION:= (the misfiled body), =:PRIORITY:=, =:LABELS:=, =:PROJECT:=, =:LINK:=, =:PROJECT-ID:= (=l.1148-1171=). Nothing below the drawer. The fetch query pulls =description= (=l.1119=) but *not* comments. State sync resolves team name → ID by network lookup each time (slow, fragile on rename/collision). The renderer strips =[ ]= from titles (=l.1145=) — existing lossy title behavior. * Agreed v1 decisions Settled in the 2026-05-23 review. 1. *The org issue body is entirely Linear-owned in v1.* No local-only notes area. The active org file is a synchronized representation of Linear, not a mixed local/remote workspace. 2. *Fetched comments are remote-owned display content.* Users can *add* comments; editing existing comments is vNext (and then only comments authored by the current Linear user, matching Linear's permissions). 3. *New entries use only namespaced =LINEAR-*= properties.* No compatibility layer for the old =:ID:= / =:ID-LINEAR:= shape (no users yet). 4. *Description sync starts as an explicit command only.* Automatic save-triggered description sync is vNext, after no-op detection and conflict handling are proven. 5. *V1 conflict handling is detect / refuse / message.* Interactive diff-merge or local/remote choice is vNext. * Content ownership and refresh semantics The hard part isn't moving the description — it's distinguishing machine-owned fetched content from user edits once refresh, comments, and sync coexist. v1 makes this simple by fiat (decision 1: the whole body is Linear-owned), but the layout and refresh model still have to be explicit. ** Generated entry layout #+begin_src org *** TODO [#B] ENG-123 Title :PROPERTIES: :LINEAR-ID: <uuid> :LINEAR-IDENTIFIER: ENG-123 :LINEAR-URL: https://linear.app/.../ENG-123 :LINEAR-TEAM-ID: <id> :LINEAR-TEAM-NAME: ENG :LINEAR-PROJECT-ID: <id> :LINEAR-PROJECT-NAME: Foo :LINEAR-STATE-ID: <id> :LINEAR-STATE-NAME: In Progress :LINEAR-ASSIGNEE-ID: <id> :LINEAR-ASSIGNEE-NAME: Craig :LINEAR-LABELS: [bug, p1] :LINEAR-DESC-SHA256: <hash of last-fetched markdown> :LINEAR-DESC-UPDATED-AT: <remote timestamp> :END: Description text managed by Linear (org-rendered). **** Comments ***** <author> — <timestamp> comment body #+end_src *Store IDs and display names separately* for team, project, assignee, state, labels (and later cycle). Commands display names; they mutate by ID. This kills the per-render network name-lookup. *Provenance for the description* lives as a *hash + remote timestamp* in properties — not the full raw markdown. A large multiline markdown property is awkward in org and bad with folding. When the sync/no-op check needs the exact last-fetched markdown, fetch current remote markdown before deciding, or keep it in an internal cache keyed by =LINEAR-ID=. (See [[*Conflict handling][Conflict handling]].) ** Refresh model — merge by ID, reconciled with the active-file output model The query spec's output model says *switching to a different view/query replaces the active file*. This spec's refresh says *don't wholesale-rewrite*. Both hold, for different actions: - *Switching source* (run a different view/query) → the issue set changes; replace the file contents after the dirty-buffer + conflict checks. One issue appears in one place. - *Refreshing the same source* (=refresh-current-view=, =refresh-current-issue=) → *merge by =LINEAR-ID===: update each existing issue subtree in place, add new matches, drop issues no longer in the result. Per subtree, run the conflict check before overwriting a description that was edited locally but not yet pushed. A wholesale rewrite on same-source refresh would clobber un-pushed description edits; merge-by-ID + per-subtree conflict check is what protects them. * Proposed model — body is editable content, drawer is machine-managed metadata Organizing principle: the body holds what a human reads and writes (description, comments); the drawer holds structured fields commands manage. An =org-tidy= user edits body text + runs commands and never touches the drawer. ** Description → body Render the description as the heading body (org-converted — see [[*Markdown vs org — the conversion question][conversion]]). Opening a task now shows its description; the org-tidy blank-entry problem disappears. The body is the editable region; an explicit command (decision 4) pushes edits back, behind the conflict gate. ** Drawer = command-managed fields State (TODO keyword), priority, labels, project, assignee live in the drawer/heading and change via dedicated commands ("Set assignee, priority, labels" task), which resolve names→IDs. =org-tidy= users never need to open the drawer. ** Comments as a body subtree Fetch comments (needs a query change — not pulled today) and render *oldest-first* as =****= → =*****= sub-headings (=<author> — <timestamp>=, body beneath), so the thread reads chronologically and "add comment" appends at the end. =pearl-add-comment= creates a new comment via =commentCreate= and inserts/refreshes the returned comment. Fetched comments are remote-owned (decision 2): editing an existing comment heading does *not* sync back in v1. *Comment shape (verified against the published schema).* =Issue.comments= → =CommentConnection= (nodes/pageInfo); each =Comment= has =body= (markdown — runs through the same conversion tier as the description), =createdAt=, and =user=. *=user= is nullable* — comments from integrations or bots have no user, carrying =botActor= / =externalUser= instead. The renderer must fall back to the bot/external actor name (or a literal like "(automation)") for the author rather than assuming a =user.name=. =commentCreate(input: CommentCreateInput!)= returns =CommentPayload= (=comment=, =success=); the input takes =body= + =issueId= (and optional =parentId=), with success checked the same way as issue creation before reporting. ** Affordance + discoverable commands A one-line preamble note (body = description, edit + sync via command; Comments subtree = thread, add via command; fields = drawer, change via commands). But commands matter more than a note — expose discoverable ones that work from *anywhere inside an issue subtree*: =pearl-sync-current-issue=, =pearl-open-current-issue=, =pearl-add-comment=, =pearl-set-priority=, =pearl-set-assignee=, =pearl-refresh-current-issue=. ** Sub-issues (later) Optional nested headings; out of scope for v1. * Markdown vs org — the conversion question Linear stores descriptions/comments as *markdown*; we want *org* in the body. The directions differ in difficulty. - *org → markdown (push):* =ox-md= is built in, but it is *not* round-trip-faithful for the subset Linear uses (see [[*ox-md rejected for push][ox-md rejected for push]]). Push is therefore a hand-rolled inverse of the fetch converter. - *markdown → org (fetch):* no built-in. The only place pandoc is tempting. ** ox-md rejected for push The original recommendation was =org-export-string-as ... 'md= for the push direction. Empirical testing (2026-05-23) of =org→md(md→org(x))= over the conversion matrix showed *zero of nine samples round-trip cleanly*. =ox-md= injects a =# Table of Contents= header, inverts emphasis (org =*italic*= → md =**bold**=), *drops checkbox markers* (=- [x] done= → =- done=), converts fenced code to 4-space indented blocks (losing the language), and reindents lists. This breaks the conflict gate two ways: the no-op guard compares =hash(org→md(body))= against the stored =LINEAR-DESC-SHA256= (hash of the last-fetched markdown), so a lossy push makes *every* no-op sync look like an edit; and the lossy output would *corrupt content pushed back to Linear* (dropped checkboxes, lost code-fence languages). This is the same lossy-round-trip failure mode the spec already rejected pandoc for — it applies to =ox-md= too. The push converter (=pearl--org-to-md=) is therefore hand-rolled as the symmetric inverse of the fetch converter (=pearl--md-to-org=), which makes round-trips byte-stable for the supported subset. Owning both directions also keeps the conversion tier self-consistent. *Two documented lossy edges remain* (inherent to the fetch converter, not the push side): a markdown =# heading= renders to a bold line on fetch and stays a bold line on push (restoring =#= would fork the org outline); single-asterisk markdown italics are unsupported on fetch (only =_underscore_= italics convert). ** Pandoc — pros/cons - *Pros:* full-fidelity bidirectional GFM↔org; one tool; battle-tested. - *Cons:* hard external-binary dependency (MELPA-hostile; users without it get broken sync); subprocess per conversion; *lossy round-trip* (pandoc reflows/normalizes → spurious diffs on no-op fetch/push); cross-platform/version drift. ** Recommendation — pure-elisp default, pandoc optional Hand-roll *both* directions: push via =pearl--org-to-md= (the inverse of the fetch pass — see [[*ox-md rejected for push][ox-md rejected for push]]), fetch via the lightweight md→org pass. No dependency, byte-stable round-trips. Pandoc is an *optional* enhancement: if =(executable-find "pandoc")= and a defcustom opts in, route both directions through it. Detected, never required — MELPA-safe. ** Conversion matrix (the testable contract) "High-frequency constructs" needs a precise, testable subset. Unsupported constructs are *preserved as literal text*, never emitted as malformed org. | Markdown | Org | Note | |----------+-----+------| | =**bold**= | =*bold*= | | | =*italic*= / =_italic_= | =/italic/= | underscores in identifiers must not trigger emphasis | | =`code`= | =~code~= | | | =```lang ... ```= | =#+begin_src lang ... #+end_src= | language preserved | | =- item= / =* item= | =- item= | | | =1. item= | =1. item= | | | =- [ ]= / =- [x]= | =- [ ]= / =- [X]= | checkboxes | | =[text](url)= | =[[url][text]]= | | | => quote= | =#+begin_quote ... #+end_quote= | | | =# Heading= | *bold line*, NOT an org heading | an org heading would fork the issue subtree and corrupt structure | | tables / HTML / footnotes | literal pass-through | preserved, not converted | The =# Heading= → bold-line rule is load-bearing: converting a markdown heading inside a description to a real org heading would split the issue's subtree. * Conflict handling The round-trip-drift guard is necessary but not sufficient — it prevents no-op churn; it doesn't define conflicts. Promote it from a note to a *phase gate on sync-back*. The sync command compares three things: - *last-fetched* Linear markdown (hash in =:LINEAR-DESC-SHA256:=), - *current org-rendered* markdown (re-render the body to md, hash it), - *current remote* markdown / =updatedAt= (fetch before pushing). Outcomes: - org == last-fetched → no local edit → *no API call* (no-op guard). - org changed, remote == last-fetched → clean push. - org changed *and* remote changed since last fetch → *conflict*: stop, refuse to push, message the user (decision 5). Resolution workflows (diff/merge, local/remote-wins) are vNext. * Parsing — org-element, not regex Current parsing assumes a level-3 heading with a drawer immediately after and walks lines/regex. Once bodies and comment subtrees exist, that's brittle (misread drawers, nested comment headings mistaken for issues). Spec an =org-element=-based parser: locate issue headings by the durable =:LINEAR-ID:= property, read properties via org APIs, treat depth structurally — never =^\*\*\*= regexes. * Internal representation Normalize API responses into internal plists/structs *before* rendering, so the renderer never sees whether Linear returned a vector, =null=, or an omitted field. Comments, assignees, cycles, and views multiply the missing/null/vector handling otherwise. Model boundaries (filter compilation, API transport, issue/comment models, org rendering, org parsing, sync orchestration, commands) stay as *logical* sections — see [[*Review dispositions][Review dispositions]] on keeping a single file. * Actions a user wants in the body space - *Edit the description* in place → explicit sync (push, behind the conflict gate). - *Read the comment thread* without leaving Emacs. - *Add a comment* → =commentCreate= (append a sub-heading). - (later) navigate sub-issues. Field changes (assignee/priority/labels/state) stay command-driven, not body edits. * Impact on existing todo.org tasks Gives concrete shape to three already-open feature tasks; implement them together: - =Sync title and description back to Linear= — description-in-body + explicit push. *Phase title sync separately* from description (its own last-fetched-title hash + conflict behavior; note the existing bracket-stripping lossiness). Keep TODO-keyword state sync as the only automatic heading mutation in the first body-editing phase. - =Add a comment to an issue from Emacs= — the comment subtree + =commentCreate=. - =Set assignee, priority, and labels from Emacs= — command-driven drawer fields (mutate by ID). Cross-cuts the query spec at the shared write path and the active-file/refresh model. * Phased implementation 1. *Description → body (read-only) + namespaced properties.* Move description out of =:DESCRIPTION:= into the body; switch to =LINEAR-*= properties storing IDs + display names; provenance hash + timestamp. Characterization test of the old shape first, then the new render; confirm =org-tidy= no longer shows a blank entry. 2. *org-element parser.* Locate by =:LINEAR-ID:=, structural depth; replaces regex parsing before subtrees land. 3. *Conversion tier.* Hand-rolled org→md push (inverse of the fetch pass; =ox-md= rejected for lossy round-trips) + lightweight md→org fetch per the matrix; unit-test the matrix (Normal/Boundary/Error) and the no-op round-trip invariant. 4. *Refresh = merge by ID* + per-subtree conflict check; reconcile with the active-file replace-on-switch model. 5. *Description sync-back (explicit command)* behind the conflict gate (the round-trip guard is the phase gate). Title sync as a separate step. 6. *Comments* — add to the fetch query; render oldest-first; =add-comment= via =commentCreate=. 7. *Pandoc optional path* + the affordance line + discoverable commands. 8. *(later)* sub-issues, comment editing, local notes, save-hook automation, interactive conflict resolution. * Test strategy *Characterization (before changing rendering):* old shape renders description in =:DESCRIPTION:= with empty body; dirty visiting buffer not overwritten; state sync uses only matching issue headings; current parser behavior with drawer placement. *Per phase:* description after =:END:= with no =:DESCRIPTION:= property; org-element parser extracts properties even with body text + comment subtrees; comments render with IDs/timestamps oldest-first; =add-comment= makes one mutation and inserts/refreshes the returned comment; no-op description sync makes *no* API call; local-edit + remote-unchanged pushes the expected markdown; local-edit + remote-changed refuses with a conflict message; unsupported markdown stays readable and doesn't corrupt org. *Golden rendering:* small, intentional string snapshots of representative issue entries. * Open decisions None blocking v1 — the five agreed decisions resolved the ownership, conflict, sync-trigger, comment-immutability, and property-naming questions. Remaining calls are implementation-level (exact converter edge handling, command key bindings). * vNext - Local-only notes under issues, if a clean ownership representation emerges. - Editing existing comments — only those authored by the current Linear user. - Automatic description sync on save (after no-op detection + conflict handling are proven). - Interactive conflict handling: diff/merge, local-wins, remote-wins, manual merge. - Read-only text properties on remote-owned regions (after the command UX exists). - Sub-issue rendering. * Review dispositions All review recommendations were accepted and incorporated above except the following, modified with reasons: 1. *"Split representation from network/API code" into modules → modified.* Adopted the *logical* boundaries (filter compilation / transport / models / rendering / parsing / sync / commands) and the "normalize before rendering" discipline, but *kept a single file* for v1. The package is a single-file =pearl.el= aiming at MELPA, where single-file is a virtue; splitting into multiple files is a larger restructuring with its own review. Logical sections + pure helpers get the unit-testability the review wants without the file split. Revisit multi-file only if size forces it. 2. *Read-only text properties on remote-owned regions → deferred (the review's own lighter recommendation).* v1 detects edits to remote-owned generated areas and warns/refuses to push rather than making regions buffer-read-only, which would frustrate org users and complicate tests. Hard read-only is in vNext. Everything else — Linear-owned body, namespaced =LINEAR-*= properties, IDs-with-display-names, hash+timestamp provenance (not raw-markdown-in-property), merge-by-ID refresh reconciled with active-file replace, conflict detect/refuse/message as a phase gate, explicit-command sync, separate title/description sync, org-element parsing, the conversion matrix, oldest-first read/add-only comments, normalized model objects, discoverable subtree commands, and the full test strategy — was accepted as written.