path: root/docs/issue-representation-spec.org

#+TITLE: pearl — Issue Org Representation & Editing Spec
#+AUTHOR: Craig Jennings
#+DATE: 2026-05-23
#+STARTUP: showall

* Status

*DRAFT — review incorporated (2026-05-23), awaiting final go-ahead.* Design proposal; nothing in =pearl.el= has changed. v1 scope is decided (see [[*Agreed v1 decisions][Agreed v1 decisions]]); deferred items in [[*vNext][vNext]]; modifications/rejections in [[*Review dispositions][Review dispositions]].

Companion to [[file:issue-query-spec.org][issue-query-spec.org]] (which covers *fetching/filtering*). This doc covers how an issue is *rendered and edited* once it's in org. They share the org→Linear write path and the single active-file output model.

* Problem

Open a fetched issue in org and the body below the drawer is empty — and with =org-tidy= folding the drawer, the whole entry looks blank. The one piece of free-text the issue has (its description) isn't missing; it's misfiled into a *property*. Users can't tell what they're allowed to edit.

Grounded in =pearl--format-issue-as-org-entry= (=pearl.el:1114-1173=): the description is written into a =:DESCRIPTION: |= property as 2-space-indented lines (=l.1154-1158=), inside the drawer; the body after =:END:= (=l.1171=) is empty; =org-tidy= then hides the drawer and the entry reads as blank. That's the root of "I opened the task and there's nothing there / I'm not sure what I can edit."

* Current rendering (what exists today)

Each issue is a =***= heading =*** <TODO-state> <priority> <title>= (=l.1146=) plus a drawer carrying =:ID:= (Linear UUID), =:ID-LINEAR:= (the =ENG-123= identifier), =:TEAM:=, =:DESCRIPTION:= (the misfiled body), =:PRIORITY:=, =:LABELS:=, =:PROJECT:=, =:LINK:=, =:PROJECT-ID:= (=l.1148-1171=). Nothing below the drawer. The fetch query pulls =description= (=l.1119=) but *not* comments. State sync resolves team name → ID by network lookup each time (slow, fragile on rename/collision). The renderer strips =[ ]= from titles (=l.1145=) — existing lossy title behavior.

* Agreed v1 decisions

Settled in the 2026-05-23 review.

1. *The org issue body is entirely Linear-owned in v1.* No local-only notes area. The active org file is a synchronized representation of Linear, not a mixed local/remote workspace.
2. *Fetched comments are remote-owned display content.* Users can *add* comments; editing existing comments is vNext (and then only comments authored by the current Linear user, matching Linear's permissions).
3. *New entries use only namespaced =LINEAR-*= properties.* No compatibility layer for the old =:ID:= / =:ID-LINEAR:= shape (no users yet).
4. *Description sync starts as an explicit command only.* Automatic save-triggered description sync is vNext, after no-op detection and conflict handling are proven.
5. *V1 conflict handling is detect / refuse / message.* Interactive diff-merge or local/remote choice is vNext.

* Content ownership and refresh semantics

The hard part isn't moving the description — it's distinguishing machine-owned fetched content from user edits once refresh, comments, and sync coexist. v1 makes this simple by fiat (decision 1: the whole body is Linear-owned), but the layout and refresh model still have to be explicit.

** Generated entry layout

#+begin_src org
*** TODO [#B] ENG-123 Title
:PROPERTIES:
:LINEAR-ID:               <uuid>
:LINEAR-IDENTIFIER:       ENG-123
:LINEAR-URL:              https://linear.app/.../ENG-123
:LINEAR-TEAM-ID:          <id>
:LINEAR-TEAM-NAME:        ENG
:LINEAR-PROJECT-ID:       <id>
:LINEAR-PROJECT-NAME:     Foo
:LINEAR-STATE-ID:         <id>
:LINEAR-STATE-NAME:       In Progress
:LINEAR-ASSIGNEE-ID:      <id>
:LINEAR-ASSIGNEE-NAME:    Craig
:LINEAR-LABELS:           [bug, p1]
:LINEAR-DESC-SHA256:      <hash of last-fetched markdown>
:LINEAR-DESC-UPDATED-AT:  <remote timestamp>
:END:

Description text managed by Linear (org-rendered).

**** Comments
***** <author> — <timestamp>
comment body
#+end_src

*Store IDs and display names separately* for team, project, assignee, state, labels (and later cycle). Commands display names; they mutate by ID. This kills the per-render network name-lookup.

*Provenance for the description* lives as a *hash + remote timestamp* in properties — not the full raw markdown. A large multiline markdown property is awkward in org and bad with folding. When the sync/no-op check needs the exact last-fetched markdown, fetch current remote markdown before deciding, or keep it in an internal cache keyed by =LINEAR-ID=. (See [[*Conflict handling][Conflict handling]].)

** Refresh model — merge by ID, reconciled with the active-file output model

The query spec's output model says *switching to a different view/query replaces the active file*. This spec's refresh says *don't wholesale-rewrite*. Both hold, for different actions:

- *Switching source* (run a different view/query) → the issue set changes; replace the file contents after the dirty-buffer + conflict checks. One issue appears in one place.
- *Refreshing the same source* (=refresh-current-view=, =refresh-current-issue=) → *merge by =LINEAR-ID===: update each existing issue subtree in place, add new matches, drop issues no longer in the result. Per subtree, run the conflict check before overwriting a description that was edited locally but not yet pushed.

A wholesale rewrite on same-source refresh would clobber un-pushed description edits; merge-by-ID + per-subtree conflict check is what protects them.

* Proposed model — body is editable content, drawer is machine-managed metadata

Organizing principle: the body holds what a human reads and writes (description, comments); the drawer holds structured fields commands manage. An =org-tidy= user edits body text + runs commands and never touches the drawer.

** Description → body

Render the description as the heading body (org-converted — see [[*Markdown vs org — the conversion question][conversion]]). Opening a task now shows its description; the org-tidy blank-entry problem disappears. The body is the editable region; an explicit command (decision 4) pushes edits back, behind the conflict gate.

** Drawer = command-managed fields

State (TODO keyword), priority, labels, project, assignee live in the drawer/heading and change via dedicated commands ("Set assignee, priority, labels" task), which resolve names→IDs. =org-tidy= users never need to open the drawer.

** Comments as a body subtree

Fetch comments (needs a query change — not pulled today) and render *oldest-first* as =****= → =*****= sub-headings (=<author> — <timestamp>=, body beneath), so the thread reads chronologically and "add comment" appends at the end. =pearl-add-comment= creates a new comment via =commentCreate= and inserts/refreshes the returned comment. Fetched comments are remote-owned (decision 2): editing an existing comment heading does *not* sync back in v1.

*Comment shape (verified against the published schema).* =Issue.comments= → =CommentConnection= (nodes/pageInfo); each =Comment= has =body= (markdown — runs through the same conversion tier as the description), =createdAt=, and =user=. *=user= is nullable* — comments from integrations or bots have no user, carrying =botActor= / =externalUser= instead. The renderer must fall back to the bot/external actor name (or a literal like "(automation)") for the author rather than assuming a =user.name=. =commentCreate(input: CommentCreateInput!)= returns =CommentPayload= (=comment=, =success=); the input takes =body= + =issueId= (and optional =parentId=), with success checked the same way as issue creation before reporting.

** Affordance + discoverable commands

A one-line preamble note (body = description, edit + sync via command; Comments subtree = thread, add via command; fields = drawer, change via commands). But commands matter more than a note — expose discoverable ones that work from *anywhere inside an issue subtree*:

=pearl-sync-current-issue=, =pearl-open-current-issue=, =pearl-add-comment=, =pearl-set-priority=, =pearl-set-assignee=, =pearl-refresh-current-issue=.

** Sub-issues (later)

Optional nested headings; out of scope for v1.

* Markdown vs org — the conversion question

Linear stores descriptions/comments as *markdown*; we want *org* in the body. The directions differ in difficulty.

- *org → markdown (push):* =ox-md= is built in, but it is *not* round-trip-faithful for the subset Linear uses (see [[*ox-md rejected for push][ox-md rejected for push]]). Push is therefore a hand-rolled inverse of the fetch converter.
- *markdown → org (fetch):* no built-in. The only place pandoc is tempting.

** ox-md rejected for push

The original recommendation was =org-export-string-as ... 'md= for the push direction. Empirical testing (2026-05-23) of =org→md(md→org(x))= over the conversion matrix showed *zero of nine samples round-trip cleanly*. =ox-md= injects a =# Table of Contents= header, inverts emphasis (org =*italic*= → md =**bold**=), *drops checkbox markers* (=- [x] done= → =- done=), converts fenced code to 4-space indented blocks (losing the language), and reindents lists.

This breaks the conflict gate two ways: the no-op guard compares =hash(org→md(body))= against the stored =LINEAR-DESC-SHA256= (hash of the last-fetched markdown), so a lossy push makes *every* no-op sync look like an edit; and the lossy output would *corrupt content pushed back to Linear* (dropped checkboxes, lost code-fence languages). This is the same lossy-round-trip failure mode the spec already rejected pandoc for — it applies to =ox-md= too.

The push converter (=pearl--org-to-md=) is therefore hand-rolled as the symmetric inverse of the fetch converter (=pearl--md-to-org=), which makes round-trips byte-stable for the supported subset. Owning both directions also keeps the conversion tier self-consistent. *Two documented lossy edges remain* (inherent to the fetch converter, not the push side): a markdown =# heading= renders to a bold line on fetch and stays a bold line on push (restoring =#= would fork the org outline); single-asterisk markdown italics are unsupported on fetch (only =_underscore_= italics convert).

** Pandoc — pros/cons

- *Pros:* full-fidelity bidirectional GFM↔org; one tool; battle-tested.
- *Cons:* hard external-binary dependency (MELPA-hostile; users without it get broken sync); subprocess per conversion; *lossy round-trip* (pandoc reflows/normalizes → spurious diffs on no-op fetch/push); cross-platform/version drift.

** Recommendation — pure-elisp default, pandoc optional

Hand-roll *both* directions: push via =pearl--org-to-md= (the inverse of the fetch pass — see [[*ox-md rejected for push][ox-md rejected for push]]), fetch via the lightweight md→org pass. No dependency, byte-stable round-trips. Pandoc is an *optional* enhancement: if =(executable-find "pandoc")= and a defcustom opts in, route both directions through it. Detected, never required — MELPA-safe.

** Conversion matrix (the testable contract)

"High-frequency constructs" needs a precise, testable subset. Unsupported constructs are *preserved as literal text*, never emitted as malformed org.

| Markdown | Org | Note |
|----------+-----+------|
| =**bold**= | =*bold*= | |
| =*italic*= / =_italic_= | =/italic/= | underscores in identifiers must not trigger emphasis |
| =`code`= | =~code~= | |
| =```lang ... ```= | =#+begin_src lang ... #+end_src= | language preserved |
| =- item= / =* item= | =- item= | |
| =1. item= | =1. item= | |
| =- [ ]= / =- [x]= | =- [ ]= / =- [X]= | checkboxes |
| =[text](url)= | =[[url][text]]= | |
| => quote= | =#+begin_quote ... #+end_quote= | |
| =# Heading= | *bold line*, NOT an org heading | an org heading would fork the issue subtree and corrupt structure |
| tables / HTML / footnotes | literal pass-through | preserved, not converted |

The =# Heading= → bold-line rule is load-bearing: converting a markdown heading inside a description to a real org heading would split the issue's subtree.

* Conflict handling

The round-trip-drift guard is necessary but not sufficient — it prevents no-op churn; it doesn't define conflicts. Promote it from a note to a *phase gate on sync-back*. The sync command compares three things:

- *last-fetched* Linear markdown (hash in =:LINEAR-DESC-SHA256:=),
- *current org-rendered* markdown (re-render the body to md, hash it),
- *current remote* markdown / =updatedAt= (fetch before pushing).

Outcomes:

- org == last-fetched → no local edit → *no API call* (no-op guard).
- org changed, remote == last-fetched → clean push.
- org changed *and* remote changed since last fetch → *conflict*: stop, refuse to push, message the user (decision 5). Resolution workflows (diff/merge, local/remote-wins) are vNext.

* Parsing — org-element, not regex

Current parsing assumes a level-3 heading with a drawer immediately after and walks lines/regex. Once bodies and comment subtrees exist, that's brittle (misread drawers, nested comment headings mistaken for issues). Spec an =org-element=-based parser: locate issue headings by the durable =:LINEAR-ID:= property, read properties via org APIs, treat depth structurally — never =^\*\*\*= regexes.

* Internal representation

Normalize API responses into internal plists/structs *before* rendering, so the renderer never sees whether Linear returned a vector, =null=, or an omitted field. Comments, assignees, cycles, and views multiply the missing/null/vector handling otherwise. Model boundaries (filter compilation, API transport, issue/comment models, org rendering, org parsing, sync orchestration, commands) stay as *logical* sections — see [[*Review dispositions][Review dispositions]] on keeping a single file.

* Actions a user wants in the body space

- *Edit the description* in place → explicit sync (push, behind the conflict gate).
- *Read the comment thread* without leaving Emacs.
- *Add a comment* → =commentCreate= (append a sub-heading).
- (later) navigate sub-issues.

Field changes (assignee/priority/labels/state) stay command-driven, not body edits.

* Impact on existing todo.org tasks

Gives concrete shape to three already-open feature tasks; implement them together:

- =Sync title and description back to Linear= — description-in-body + explicit push. *Phase title sync separately* from description (its own last-fetched-title hash + conflict behavior; note the existing bracket-stripping lossiness). Keep TODO-keyword state sync as the only automatic heading mutation in the first body-editing phase.
- =Add a comment to an issue from Emacs= — the comment subtree + =commentCreate=.
- =Set assignee, priority, and labels from Emacs= — command-driven drawer fields (mutate by ID).

Cross-cuts the query spec at the shared write path and the active-file/refresh model.

* Phased implementation

1. *Description → body (read-only) + namespaced properties.* Move description out of =:DESCRIPTION:= into the body; switch to =LINEAR-*= properties storing IDs + display names; provenance hash + timestamp. Characterization test of the old shape first, then the new render; confirm =org-tidy= no longer shows a blank entry.
2. *org-element parser.* Locate by =:LINEAR-ID:=, structural depth; replaces regex parsing before subtrees land.
3. *Conversion tier.* Hand-rolled org→md push (inverse of the fetch pass; =ox-md= rejected for lossy round-trips) + lightweight md→org fetch per the matrix; unit-test the matrix (Normal/Boundary/Error) and the no-op round-trip invariant.
4. *Refresh = merge by ID* + per-subtree conflict check; reconcile with the active-file replace-on-switch model.
5. *Description sync-back (explicit command)* behind the conflict gate (the round-trip guard is the phase gate). Title sync as a separate step.
6. *Comments* — add to the fetch query; render oldest-first; =add-comment= via =commentCreate=.
7. *Pandoc optional path* + the affordance line + discoverable commands.
8. *(later)* sub-issues, comment editing, local notes, save-hook automation, interactive conflict resolution.

* Test strategy

*Characterization (before changing rendering):* old shape renders description in =:DESCRIPTION:= with empty body; dirty visiting buffer not overwritten; state sync uses only matching issue headings; current parser behavior with drawer placement.

*Per phase:* description after =:END:= with no =:DESCRIPTION:= property; org-element parser extracts properties even with body text + comment subtrees; comments render with IDs/timestamps oldest-first; =add-comment= makes one mutation and inserts/refreshes the returned comment; no-op description sync makes *no* API call; local-edit + remote-unchanged pushes the expected markdown; local-edit + remote-changed refuses with a conflict message; unsupported markdown stays readable and doesn't corrupt org.

*Golden rendering:* small, intentional string snapshots of representative issue entries.

* Open decisions

None blocking v1 — the five agreed decisions resolved the ownership, conflict, sync-trigger, comment-immutability, and property-naming questions. Remaining calls are implementation-level (exact converter edge handling, command key bindings).

* vNext

- Local-only notes under issues, if a clean ownership representation emerges.
- Editing existing comments — only those authored by the current Linear user.
- Automatic description sync on save (after no-op detection + conflict handling are proven).
- Interactive conflict handling: diff/merge, local-wins, remote-wins, manual merge.
- Read-only text properties on remote-owned regions (after the command UX exists).
- Sub-issue rendering.

* Review dispositions

All review recommendations were accepted and incorporated above except the following, modified with reasons:

1. *"Split representation from network/API code" into modules → modified.* Adopted the *logical* boundaries (filter compilation / transport / models / rendering / parsing / sync / commands) and the "normalize before rendering" discipline, but *kept a single file* for v1. The package is a single-file =pearl.el= aiming at MELPA, where single-file is a virtue; splitting into multiple files is a larger restructuring with its own review. Logical sections + pure helpers get the unit-testability the review wants without the file split. Revisit multi-file only if size forces it.

2. *Read-only text properties on remote-owned regions → deferred (the review's own lighter recommendation).* v1 detects edits to remote-owned generated areas and warns/refuses to push rather than making regions buffer-read-only, which would frustrate org users and complicate tests. Hard read-only is in vNext.

Everything else — Linear-owned body, namespaced =LINEAR-*= properties, IDs-with-display-names, hash+timestamp provenance (not raw-markdown-in-property), merge-by-ID refresh reconciled with active-file replace, conflict detect/refuse/message as a phase gate, explicit-command sync, separate title/description sync, org-element parsing, the conversion matrix, oldest-first read/add-only comments, normalized model objects, discoverable subtree commands, and the full test strategy — was accepted as written.