diff options
| -rw-r--r-- | todo.org | 1718 |
1 files changed, 868 insertions, 850 deletions
@@ -2315,519 +2315,6 @@ From the color-assignment guide work (2026-06-08): make the tool support the gui [[id:b70b37f2-37df-4c8e-ac2f-1f20d12e33dd][theme-studio-seeding-engine-spec-doing.org]] — role table + face→role maps for syntax/UI/org, OKLCH shade generation, reseed dupre-revised to the compact mapping. Codex-reviewed, Ready. Implementation tracked under the seeding-engine parent below. *** TODO Guide-support views and advisories spec Five optional surfaces, all dismissible and non-blocking, in one collapsible panel where they advise: (1) CVD-simulation toggle on previews (deuteranopia/protanopia/tritanopia); (2) squint/blur preview toggle; (3) lightness-ramp view + palette advisories (accent count over 6-8, roles separated only by red/green) — depends on the OKLCH/ΔE core; (4) definition-vs-call / weight advisories; (5) state-over-syntax preview (region/search/diff tint over real syntax-colored text). Sequence: rewritten guide reviewed → seeding-engine spec → this. Advisories (3, 4) layer on the perceptual-metrics feature. -** CANCELLED [#C] GPTel Feature Extension Brainstorm :feature: -CLOSED: [2026-06-23 Tue] -gptel archived 2026-06-23 to archive/gptel/ (rarely used); extension ideas are moot. Child design notes left below as reference. -:PROPERTIES: -:LAST_REVIEWED: 2026-06-01 -:END: - -Categories below thematize the agent affordances the design doc -[[file:docs/design/gptel-agentic-tool-ideas.org][gptel-agentic-tool-ideas.org]] -points at -- Git, Org, messaging, file / buffer / workspace state, -media, and the dev loop. The shortlist's first-batch ADOPT tools -(git_status / git_log / git_diff / web_fetch) already shipped; the -themes below are next-tier work where the agent treats Emacs as a -structured workspace, not a text terminal. Per-theme spec lives in -the task body once written; implementation tasks land as siblings -of the spec heading once the spec is approved. The magit-backend -reimplementation of the shipped git tools is tracked separately in -[[id:bd47c9a8-aae1-4a3d-ad5b-b8767f2fd580][gptel-git-tools-magit-backend-spec.org]]. - -*** TODO [#C] Wire Up MCP.el so That GPTel Has Access to MCP Servers via GPTel Tools - -**** 2026-05-16 Sat @ 15:44:36 -0500 Spec - -Design doc: [[id:b4c274c5-8572-4a7b-b657-d315712bd6af][docs/specs/mcp-el-gptel-integration-spec-doing.org]] - -**** 2026-05-17 Sun @ 14:14:34 -0500 Landed ai-mcp.el pure-helper foundation - -Commit =54d231be=. Sections 1 (constants + defcustoms) and 3 (pure helpers) of the seven-section outline. 41 ERT tests, all green. Refactor audit caught two duplications during Phase 4 and folded them into the same commit (=cj/mcp--get-server-entry= and =cj/mcp--name-matches-p=). Phase 1.5 (confirmation contract) is next. - -**** TODO [#C] Phase 1.5 -- GPTel confirmation contract - -*Goal:* flip =gptel-confirm-tool-calls= to ='auto= and gate the existing local tools that need it. - -*Entry:* Phase 1 module exists and helpers tested. - -*DECISION (cj):* which of the existing local tools register with =:confirm t= once ='auto= is in effect? Reads (=read_buffer=, =read_text_file=, =list_directory_files=, =git_status=, =git_log=, =git_diff=) clearly stay =:confirm nil=. Judgment calls: -- =web_fetch= -- fetches arbitrary URLs the agent supplies. Spec recommends gating. -- =write_text_file= -- writes any path under =$HOME= with agent-supplied content. -- =update_text_file= -- modifies an existing file with an agent-supplied transform. -- =move_to_trash= -- moves a path to trash (reversible but disruptive). - -*Deliverables:* -- =ai-mcp.el= setup section runs =(setq gptel-confirm-tool-calls 'auto)=. -- Remove =(setq gptel-confirm-tool-calls nil)= from =modules/ai-config.el:386= with a comment pointing at =ai-mcp.el=. -- For each tool the decision marks "gate," add =:confirm t= to its =gptel-make-tool= form. -- Tests in =tests/test-ai-mcp-confirm-contract.el= asserting: =gptel-confirm-tool-calls= is ='auto= after load; write-classified stub MCP tool with =:confirm t= triggers the confirm branch in =gptel-send='s dispatch (stub the prompt); read-classified MCP tool with =:confirm nil= does not; =git_log= (=:confirm nil=) still runs without prompting; each newly-gated local tool does prompt. - -*Exit:* tests green. Manual smoke: open GPTel, call a gated tool, confirm prompt appears. Call =git_log=, no prompt. - -**** TODO [#B] Phase 2 -- Compat layer + registration pipeline (fake inventory) - -*Goal:* implement the mcp.el compat wrappers and the tool-registration pipeline against stubbed =mcp-server-connections=. - -*Entry:* Phase 1.5 proves gptel respects per-tool =:confirm= slot. - -*Deliverables:* -- Section 4 of =ai-mcp.el= (compat layer): =cj/mcp--server-status=, =cj/mcp--server-tools=, =cj/mcp--server-name=, =cj/mcp--assert-capabilities=. Each helper documents the upstream commit / file location it targets. -- Section 5 of =ai-mcp.el= (registration pipeline): =cj/mcp--register-tool=, =cj/mcp--register-server-tools=, =cj/mcp--deregister-server-tools=, =cj/mcp--rewrite-plist=, =cj/mcp--registered-tools= hash. -- All MCP tools register with =:async t=. -- Tests in =tests/test-ai-mcp-registration.el=. - -*Exit:* with a stubbed =mcp-server-connections=, registration produces correctly prefixed =mcp__SERVER__TOOL= entries in =gptel-tools=; closures call =mcp-call-tool SERVER REMOTE-NAME= (verified by stubbing =mcp-async-call-tool=); deregistration removes only MCP-owned tools and leaves a pre-populated local =git_log= entry intact; re-registration replaces function pointer without duplicating menu entries; confirm overrides win over patterns. - -**** TODO [#B] Phase 3 -- Async state machine + timer-race timeout wrapper - -*Goal:* implement the lifecycle state machine and the per-call timer-race timeout. - -*Entry:* Phase 2 registration works against stubs. - -*Deliverables:* -- Section 6 of =ai-mcp.el= (async state machine): =cj/mcp--state=, =cj/mcp--server-status= alist, =cj/mcp--stall-timer=, =cj/mcp-ensure-started=, =cj/mcp--on-hub-callback=, =cj/mcp--poll-status=, =cj/mcp--start-stall-timer=, =cj/mcp--build-status-from-specs=. -- =cj/mcp--wrap-async-with-timeout= (timer/callback race; both branches set =done= before invoking gptel callback so late responses are ignored). -- Tests in =tests/test-ai-mcp-async.el=. - -*Exit:* =cj/mcp-ensure-started= returns in <100 ms with delayed-callback stubs; stall timer fires for stuck servers; timer-race wrapper handles all three orderings (MCP-first, timer-first, late-MCP-after-timer); async error path (=:error-callback= without inited callback) reaches =failed= state via polling. - -**** TODO [#B] Phase 4 -- First real connection (drawio or slack-deepsat) - -*Goal:* wire one real no-auth server end-to-end against actual mcp.el and prove the stubbed Phase 3 behavior matches reality. - -*Entry:* Phase 3 async works against stubs. - -*Deliverables:* -- Add =use-package mcp= to =ai-mcp.el= (MELPA active, =:load-path= for local checkout commented). -- =cj/mcp--assert-capabilities= called at load time; signals clearly if mcp.el is too old. -- Set =cj/mcp-enabled-servers= temporarily to =("drawio")= (or =("slack-deepsat")= if the local proxy is running). -- First real =cj/mcp-ensure-started= invocation from =cj/toggle-gptel=. - -*Exit:* manual smoke -- =C-; a t= opens GPTel without blocking; within 30 s, drawio (or slack-deepsat) tools appear in =gptel-menu= grouped by category; calling a tool returns expected output; killing the subprocess externally surfaces as =failed= in =cj/mcp--server-status=. - -**** TODO [#B] Phase 5 -- Status UX + commands + doctor (static) - -*Goal:* ship the full server-management UX so partial-availability and failures are visible. - -*Entry:* Phase 4 proves a real connection works. - -*Deliverables:* -- Section 7 of =ai-mcp.el= (UI). -- Commands: =cj/mcp-status= (echo-area summary keyed off =cj/mcp--state=), =cj/mcp-list-tools= (tabulated buffer with failed servers at top in red face; keys =g r c RET q=), =cj/mcp-doctor= (static mode only -- capability, =npx=/=uvx=, Claude config, per-server env, local endpoints; output buffer keys =c r q=), =cj/mcp-wait-until-ready=, =cj/mcp-hub= (thin wrapper that ensures startup first), =cj/mcp-restart-failed=, =cj/mcp-restart-server=, =cj/mcp-stop-all=. -- Keymap: =C-; a C= subprefix bound in =ai-config.el='s autoload section. Keys =h s l r R S d w=. -- which-key labels for every binding. -- =kill-emacs-hook= registration for =cj/mcp-stop-all=. -- Investigation: does =gptel-menu= refresh after mid-call tool registration? Document the answer in =ai-mcp.el= commentary; if it requires close+reopen, add to known UX caveats. - -*Exit:* all keymap bindings work; audit buffer surfaces failed servers prominently; doctor identifies each scenario in the manual test matrix; status command shows the right state for each phase transition. - -**** TODO [#B] Phase 6 -- HTTP servers (linear, notion) - -*Goal:* add the two HTTP-transport servers with in-protocol OAuth. - -*Entry:* Phase 5 UX shipped. - -*Deliverables:* -- Add =linear= and =notion= back to =cj/mcp-enabled-servers=. -- Doctor gains live-auth-check mode (=C-u C-; a C d=): invokes a single safe read per auth class to verify OAuth tokens haven't silently expired. Static checks first; live probe only fires after static passes. -- OAuth recovery pattern matcher surfaces auth URLs in =cj/mcp-status= on first connect. - -*Exit:* first connect surfaces the OAuth URL through the recovery pattern; after browser handshake completes, subsequent connects succeed without prompt; live-auth-check correctly identifies a deliberately revoked token; both servers appear ready in the audit buffer. - -**** TODO [#B] Phase 7 -- Env-dependent stdio servers (figma, google-*) - -*Goal:* add the remaining five env-dependent servers. - -*Entry:* Phase 6 HTTP servers connect cleanly. - -*Deliverables:* -- Add =figma=, =google-calendar=, =google-docs-personal=, =google-docs-work=, =google-keep= to =cj/mcp-enabled-servers=. -- Verify env-merge from =~/.claude.json= for each (the mtime-cached reader from Phase 1). -- Verify figma's =:secret-args= splicing places the API key correctly without echoing it. -- Manual smoke: simulate token expiry on one Google server; recovery message points at "re-auth via Claude Code, then C-; a C r SERVER". - -*Exit:* all 9 servers reach =ready= state on a clean machine. Sentinel-grep check across status / audit / hub / errors / audit-log shows zero secret leakage. Doctor's live-auth covers each auth class (oauth, token, args-token, in-protocol, local, none). - -**** TODO [#B] Phase 8 -- Privacy + audit polish - -*Goal:* land the final UX polish and documentation. - -*Entry:* all 9 servers working. - -*Deliverables:* -- Audit buffer privacy header: "Tool results land in =gptel-tools= responses; saved conversations persist them. Use =cj/gptel-autosave-toggle= per buffer to opt out." -- =cj/mcp-tool-audit-log-enabled= defcustom + log writer (=~/.emacs.d/data/mcp-tool-log/YYYY-MM-DD.log= -- metadata only, one line per call, daily rotation). -- =ai-mcp.el= commentary updated with the code-organization outline as a table of contents. -- Final pass on tests covering saved-conversation behavior (autosave persists MCP tool results; toggling off prevents persistence). - -*Exit:* all 10 acceptance criteria from the spec pass. Manual matrix run end-to-end on a fresh Emacs. Working tree clean. - -*** TODO [#C] Wrap the gh CLI as a GPTel tool - -**** 2026-05-16 Sat @ 16:20:00 -0500 Spec - -Design doc: [[id:a124dd0f-1f40-4533-aeb8-595d93e20865][docs/specs/gptel-gh-tool-spec.org]] - -*** TODO [#C] GPTel should autosave regularly after a conversation is saved -*** TODO [#B] Org Workflow Related Tools - -Affordances that expose the Org workspace -- agenda state, capture -targets, org-roam nodes and backlinks, dailies, drill review state -- -to the agent as structured context, not raw .org buffer text. - -**** TODO [#B] Agenda state tools :feature: - -Read scheduled / deadline / waiting tasks for a date range; query by -tag, priority, or TODO keyword; list what's blocking today. Lets the -agent answer "what's on the critical path this week" without me -pasting agenda output, and feeds the daily-prep / wrap-up workflows. - -**** TODO [#B] Org-roam node tools :feature: - -Resolve a topic to its node; return body + backlinks; list nodes by -tag; surface dailies for a date range. Lets the agent reason over -the personal knowledge graph and write back into it via the capture -tools below. - -**** TODO [#B] Capture creation tools :feature: - -Drive =org-capture= from a template key + body string. Lets the -agent file inbox items, reading notes, journal entries, or roam -nodes without me leaving the chat. Tight pairing with the -=cj/org-capture= optimization task in todo.org. - -**** TODO [#B] Org-drill review tools :feature: - -Surface next-due drill cards in =drill-dir=; let the agent quiz on a -topic and report performance. Useful for prompted recall sessions -("ask me five medical-Spanish cards") and for "did this card stick" -analysis. - -*** TODO [#B] Git Related Tools - -Affordances that expose magit's structured view of a repo -- sections, -staged-vs-unstaged, commit metadata, rebase / conflict state -- as -first-class tools rather than asking the model to reason over raw -diff text. - -**** TODO [#B] Section-aware git tools :feature: - -Expose Magit sections as first-class GPTel tools: current section type, -heading, file, hunk range, and content; sibling sections under the same -file; staged / unstaged / untracked status; commit metadata around the -selected commit or branch; the exact staged patch that would be -committed. Lets prompts say "review the file section at point" or -"explain this hunk in the context of adjacent hunks" without manual -context-copying. - -**** TODO [#B] Commit intent workbench :feature: - -Transient that builds a commit intentionally: -1. Agent reads unstaged + staged changes. -2. Agent proposes coherent commit groups. -3. User selects groups in a Magit-style buffer. -4. Agent stages those paths or hunks only after confirmation. -5. Agent generates a message reflecting the selected intent. - -Addresses the common case of two or three unrelated edits in one -working tree -- a single commit-message generator can't handle that -cleanly. - -**** TODO [#B] Patch narrative buffer :feature: - -Generate an Org buffer that explains a change set as a reviewable -narrative: -- "What changed" by subsystem. -- "Why it appears to have changed" inferred from names, tests, and docs. -- "Risk areas" with links back to Magit file sections. -- "Suggested verification" using local Makefile targets when present. - -Reusable artifact: paste into a PR description, save with an AI -session, or file into org-roam. - -**** TODO [#B] Review-thread simulator :feature: - -Before opening a PR, create a local review buffer with inline comments -attached to Magit diff positions. The agent writes comments as if -reviewing someone else's patch: -- Comments grouped by severity. -- Each comment links to file and line. -- Resolved comments check off in Org. -- Accepted suggestions apply through the existing text-update tools. - -Makes "review my diff" less ephemeral and avoids losing useful findings -inside a chat transcript. - -**** TODO [#B] Rebase and conflict coach :feature: - -When Magit enters a rebase, cherry-pick, merge, or conflict state, -expose an agent command that reads: -- Git operation state from =.git/=. -- Conflict markers in the worktree. -- Relevant commits from =git log --merge= or the rebase todo. -- The current Magit status sections. - -The agent explains the conflict in domain terms and proposes a -resolution patch; the actual edit and =git add= stay under explicit -user control. - -**** TODO [#B] Regression archaeology :feature: - -Magit transient that runs a bisect-like reasoning workflow: -- Ask for a symptom and a known-good / known-bad range. -- Summarize candidate commits in small batches. -- Use tests or user-provided repro commands when available. -- Maintain a bisect journal in an Org buffer. - -Even when the agent can't run the whole bisect, it keeps the -investigation structured and preserves why each commit was judged -good or bad. - -**** TODO [#B] Messaging Related Tools - -Affordances over mu4e, Slack, Telegram, and ERC. Same shape across -protocols: read recent threads, search by sender / topic, compose a -draft from a prompt + thread context, leave the send under explicit -user control. - -***** TODO [#B] Mu4e thread and compose tools :feature: - -Read the message at point and surrounding thread (with attachments -summarized); query the inbox by =from:= / =subject:= / date range; -compose a draft from a prompt + thread context using =org-msg=. -Pairs with the existing =mu4e-org-contacts-integration.el=. - -***** TODO [#B] Slack thread and compose tools :feature: - -Read channel / DM / thread history through =emacs-slack=; search by -user or channel; compose a draft message but leave sending to me. -Mirrors the mu4e shape so the agent's interface is uniform across -messaging protocols. - -***** TODO [#B] Telegram and IRC read tools :feature: - -Same shape as Slack for =telega= (Telegram) and =erc= (IRC): -recent-message reads, search, and draft compose. Bundled because -the API shape is identical even if the underlying clients differ. - -***** TODO [#B] Contact resolution tools :feature: - -Resolve a name to email / Slack ID / Telegram handle via -=org-contacts= and the configured address books. Removes the -"who's this person again" friction from the compose flows above. - -**** TODO [#B] File and Buffer Related Tools - -Affordances that expose the user's actual workspace -- open buffers, -narrowed regions, marked files, vterm / eshell sessions -- as -structured context. Stops the model from asking "what file are you -looking at" or "what region is selected." - -***** TODO [#B] Buffer state tools :feature: - -List visible buffers with major-mode + file (when any); read the -narrowed region instead of the whole buffer; report point + mark -positions and the active region's text. The single most-asked -question between turns becomes a tool call. - -***** TODO [#B] Dirvish / Dired tools :feature: - -Read marked files, sort state, and filter state from a Dired or -Dirvish buffer. Lets the agent operate on "the files I just marked" -rather than "files in this directory" -- a real distinction in any -review or refactor workflow. - -***** TODO [#B] Vterm session tools :feature: - -Recent command output from a named vterm session; scroll-history -search. Pairs naturally with the =ai-vterm= design: the agent -running in one project's vterm can read another project's vterm -without leaving the chat. - -***** TODO [#B] Eshell session tools :feature: - -Same shape as the vterm tools for =eshell= sessions -- last-command -output, history search, current directory. Most useful for -agent-driven inspection of long-running pipelines. - -**** TODO [#B] Filesystem Related Tools - -Affordances that let the agent operate on actual files on disk and -run common CLI utilities -- pandoc, ffmpeg, imagemagick, ripgrep, -fd, jq -- rather than relying on me to paste content or run -commands by hand. - -*Design tension to resolve before any of these ship: one tool per -utility, or one generic =run_shell_command=?* - -The shortlist's first pass DEFERRED a generic =run_shell_command=: -sandboxing to HOME + /tmp with a denylist for destructive ops is -straightforward, but the denylist can never be exhaustive, and -"confirmation for everything else" becomes click-fatigue. - -The children below take the other path -- *one gptel tool per -binary*, with a strictly-typed argv shape (e.g. -=pandoc_convert(input_path, output_format)=, not -=pandoc_convert(args_string)=). Each tool: - -- Validates its own paths (must be under HOME, outputs in a - sandboxed dir). -- Rejects dangerous flags explicitly (pandoc =--filter=, ffmpeg's - =-protocol_whitelist= chicanery, imagemagick's policy bypasses). -- Runs via =call-process= with an argv list -- no shell parsing, - no string-interpolation injection. -- Caps output and reports truncation inline. - -The trade-off is breadth: every new CLI tool means a new gptel tool -file. Acceptable because (a) the list of utilities I actually need -agent access to is small (~8 below covers most of it), and (b) each -wrapper gets type-checked argv and a focused description the model -can reason over, which is genuinely better than a free-form -=run_shell_command(string)=. - -The =eshell_submit= entry at the end is the escape hatch for one- -off needs the wrappers don't cover -- =:confirm t= always. - -Adjacent categories: the existing =gptel-tools/= file CRUD -(=read_text_file=, =write_text_file=, =update_text_file=, -=list_directory_files=, =move_to_trash=) is the foundation this -category extends. =web_fetch= is the network-fetch counterpart. - -***** TODO [#B] Document conversion (pandoc) :feature: - -Convert between markdown, org, html, pdf, docx, latex, epub, plain -text. Most common use: "extract this docx to markdown so I can -read it inline." Strict argv: input path, output format, optional -output path. Reject =--filter= and =--lua-filter= (arbitrary code -execution). Output written to a sandbox dir unless explicit -override. - -***** TODO [#B] Image manipulation (imagemagick) :feature: - -Resize, format-convert, get-metadata (=identify=), optionally crop / -rotate / annotate. Common use: "resize this PNG to a thumbnail" or -"convert these HEICs to JPEGs." Strict argv per operation. -Reject pre-validated dangerous formats (the historical EXR / SVG / -MVG CVE surface) unless explicitly enabled. ImageMagick's -=policy.xml= is the underlying defense; the wrapper enforces it at -the tool boundary too. - -***** TODO [#B] Audio / video processing (ffmpeg) :feature: - -Trim, transcode, extract audio, get-metadata (=ffprobe=). Paths -under HOME only; reject network-protocol inputs (=http:= / =rtmp:= -/ =rtsp:=) so the model can't pull from arbitrary sources. Pairs -with the existing transcription module -- the same "extract audio -from video" path =cj/transcribe-media= uses internally. - -***** TODO [#B] Content search (ripgrep) :feature: - -=rg= wrapper with path / glob filtering, result-count cap, optional -literal-vs-regex mode. Pure read. Was in the shortlist's ADOPT -bucket as =search_in_files=. Highest-leverage filesystem tool by -expected call frequency -- "where in this repo is X" is the -question I paste agent output for most often. - -***** TODO [#B] File discovery (fd) :feature: - -=fd= (or =find= fallback) wrapper, capped result count. Pure -read, lower stakes than =search_in_files= (filenames only, no -content). Common pairing: =find_file_by_name= then -=read_text_file=. - -***** TODO [#B] Metadata extraction (file / exiftool) :feature: - -=file= for MIME-type detection; =exiftool= for image / video / -audio metadata. Lets the agent answer "what is this file" or -"when was this photo taken" without me opening external tools. -Pure read. - -***** TODO [#B] Structured data processing (jq / yq) :feature: - -=jq= for JSON, =yq= for YAML / TOML. Filter / project / transform -structured data into a smaller, more focused view before reading. -Strictly read-only -- output goes to the chat, not to disk. The -agent often wants "the third element of .results" from a JSON file -and this is much cheaper than pasting the whole thing. - -***** TODO [#B] Eshell command submission :feature: - -Submit a single eshell command line, return output (capped). -=:confirm t= always -- this is the escape hatch where the -strictly-typed wrappers above don't fit, so each invocation needs -my eyeball. Eshell parses in-process (no /bin/sh fork) so the -security surface is narrower than a shell command runner, but it's -still effectively arbitrary execution -- treat it as such. - -**** TODO [#B] Media and Reading Related Tools - -Affordances over non-code content: feeds, PDFs, EPUBs, music. The -agent's job here is summarize / extract / queue, not produce. - -***** TODO [#B] Elfeed entry tools :feature: - -Read entry body; list unread by feed or tag; mark read after a -summary lands in a roam node or inbox. Enables "give me the -non-noise headlines from this week's feeds" flows. - -***** TODO [#B] PDF and EPUB text tools :feature: - -Extract plain text from a PDF page or page range (via =pdftotext=) -and from an EPUB (via the existing nov-mode pipeline). Lets the -agent summarize / quote a research paper or book chapter without -me pasting passages. - -***** TODO [#B] EMMS playback and queue tools :feature: - -Current track, queue contents, playback state; queue or play a -path; compose a playlist from a prompt ("play something focusing -that's not Nick Cave"). Light tools, but a frequent friction -point. - -**** TODO [#B] Development Workflow Related Tools - -Affordances over the dev loop: compilation output, test invocation, -coverage / profile data, flycheck / flymake diagnostics. - -***** TODO [#B] Compilation buffer tools :feature: - -Read the most recent =compile= buffer output; parse error locations -to =file:line=; summarize what broke. Pairs with the F6 test-runner -flow -- "tell me what's failing" becomes a single agent turn -instead of paste + parse. - -***** TODO [#B] Project test invocation tools :feature: - -Run =make test-file FILE=X= / =make test-name TEST=Y= / -project-equivalent and return results. Currently each agent guesses -the project convention; expose the canonical invocation explicitly -per project so the agent can run focused tests itself. - -***** TODO [#B] Coverage and profile tools :feature: - -Read the most recent SimpleCov JSON or profile dump. Lets the -agent answer "what's still uncovered after this push" or "what -function dominates startup time" against real measured data. - -***** TODO [#B] Diagnostic tools (flycheck / flymake) :feature: - -Surface current-buffer or project-wide errors and warnings. Useful -both as a "what's broken right now" check and as input to the -patch-narrative buffer / commit-intent workbench above. - -**** CANCELLED [#B] gptel-magit activation fails on velox :bug:quick: -:PROPERTIES: -:LAST_REVIEWED: 2026-06-01 -:END: -Surfaced 2026-05-25 while diagnosing an unrelated load failure over SSH. velox-specific — the workstation has a current gptel and does not show it. - -At startup (and reproducibly in batch) velox logs: "Unable to activate package `gptel-magit'. Required package `gptel-0.9.8' is unavailable." gptel-magit depends on gptel >= 0.9.8 and velox's installed gptel is older or missing, so it can't activate. A startup warning, not a blocker. - -Reproduce: -: emacs --batch --no-site-file -L . -L modules --eval "(package-initialize)" --eval "(message \"done\")" 2>&1 | grep -i gptel - -Next step: check the installed gptel version (=(assq 'gptel package-alist)= or =M-x package-list-packages=), update gptel to >= 0.9.8, then re-evaluate gptel-magit activation. If gptel was pinned/held on velox, reconcile the pin against the gptel-magit dependency. - ** PROJECT [#C] Music Open Work Parent grouping the open music / EMMS issues; close each child independently. *** VERIFY [#C] music: extract faces for music config :refactor:quick:solo:next: @@ -4078,343 +3565,6 @@ The "buffer differs from file" confirmation currently gives only yes/no. Craig w :LAST_REVIEWED: 2026-06-21 :END: Replace topic tagging with single-word module tags: :studio: for everything under scripts/theme-studio/, module-named tags elsewhere, :multi: for cross-area work. Drop bug/enhancement-style tags since work should be chosen on other bases. This changes the current six-tag convention, so update the priority-scheme section to document it, rewrite the task-audit workflow to reconcile tasks against the module scheme, then run the audit. Queue for end of session. From the roam inbox. -** TODO [#C] Build an Org-native API workspace :feature:test: -:PROPERTIES: -:LAST_REVIEWED: 2026-06-21 -:END: - -Build an Emacs-native API workspace layer that keeps =restclient.el= useful for -lightweight request execution while adding OpenAPI import, Org notebooks, -response capture, inline images, and =gptel=-assisted API documentation. - -*** Spec - -**** Summary -Build a small Emacs API-workspace layer that keeps =restclient.el= as the -plain-text request executor, while adding higher-level discovery, generation, -Org notebook, response handling, and =gptel=-assisted documentation workflows. - -The intended result is not a Postman clone. It should feel like an -Emacs-native API lab: -- generate request collections from OpenAPI/Swagger specs -- browse and execute requests from =.rest= files or Org notebooks -- save useful responses back into Org -- display image responses inline -- use =gptel= to explain APIs, endpoints, request bodies, and responses -- leave room for =verb.el= or Hurl where they are a better fit - -**** Goals -- Preserve the current lightweight =restclient.el= workflow for ad-hoc calls. -- Add an OpenAPI importer that can generate useful first-draft request files. -- Add an Org workspace format for API notes, executable request blocks, results, - response links, and generated summaries. -- Add response post-processing helpers for JSON, binary/image data, and model - summaries. -- Make common auth flows easy to scaffold without storing secrets in generated - files. -- Keep generated artifacts readable and editable by hand. -- Keep implementation modular enough that =verb.el= can be evaluated as a richer - Org-native frontend. - -**** Non-Goals -- Do not build a full graphical API client. -- Do not store secrets directly in Org or =.rest= files. -- Do not attempt complete OpenAPI 3.1 schema synthesis in the first pass. -- Do not require =gptel= for ordinary request execution. -- Do not make Hurl a runtime dependency for the restclient workflow. - -**** Primary User Flows -***** Import An API -1. User runs =cj/restclient-import-openapi=. -2. Command prompts for a URL or local OpenAPI/Swagger file. -3. Command parses the spec into normalized endpoint records. -4. Command prompts for output format: - - =restclient= request collection - - Org API workspace - - both -5. Command writes generated files under a project/API workspace directory. -6. Generated file includes host variables, auth placeholders, request examples, - and a short summary of how to start using the collection. - -***** Describe An API -1. User runs =cj/restclient-describe-api= from a generated workspace or spec file. -2. Command extracts API metadata, paths, operations, schemas, auth, and tags. -3. Command sends a bounded summary payload to =gptel=. -4. Result is inserted under an Org =Overview= subtree. - -***** Execute And Capture A Request -1. User executes a request via =restclient.el= or =ob-restclient=. -2. Command captures response status, headers, content type, body, and timestamp - when available. -3. JSON/XML responses can be inserted as source blocks or summarized. -4. Image responses are saved to an assets directory and linked inline in Org. -5. Binary responses that are not image-like are saved as file links. - -***** Iterate On A Response -1. User runs helper commands against the last response: - - pretty-print JSON/XML - - run =jq= - - summarize with =gptel= - - extract a value into a restclient variable - - save body to a file -2. The result is inserted near the endpoint subtree or shown in a temporary - buffer depending on command prefix/options. - -**** Proposed Modules -***** =modules/restclient-workspace.el= -Main configuration and user-facing commands. Owns keybindings, workspace -creation, dispatch commands, and integration with existing restclient config. - -***** =modules/restclient-openapi.el= -OpenAPI/Swagger loading and normalization: -- fetch URL or read file -- parse JSON/YAML where available -- normalize servers, paths, methods, parameters, request bodies, auth schemes, - tags, operation IDs, and examples -- expose pure helper functions suitable for unit tests - -***** =modules/restclient-generate.el= -Generation layer: -- render =.rest= / =.http= request collections -- render Org workspaces -- synthesize simple JSON request bodies from schemas/examples -- generate variables and auth placeholders - -***** =modules/restclient-response.el= -Response helpers: -- identify content type -- save binary/image bodies -- insert Org links -- display inline images -- route JSON/XML/text handling -- track last response metadata - -***** =modules/restclient-ai.el= -=gptel= integration: -- summarize API specs -- explain endpoint behavior -- generate example payload notes -- summarize responses -- keep prompts bounded and avoid sending secrets - -**** File And Workspace Layout -Default generated workspace: - -#+begin_src text -data/apis/<api-slug>/ - <api-slug>.rest - <api-slug>.org - openapi.json - assets/ - responses/ -#+end_src - -The exact root should be configurable, defaulting to the existing REST data -directory if one is already present in the config. - -**** Org Workspace Shape -#+begin_example -,* API Name -:PROPERTIES: -:OPENAPI_SOURCE: https://example.com/openapi.json -:BASE_URL: https://api.example.com -:END: - -,** Overview -Generated or user-written API notes. - -,** Auth -Token acquisition notes and placeholders. No secrets stored here. - -,** Endpoints -,*** GET /users -:PROPERTIES: -:OPERATION_ID: listUsers -:METHOD: GET -:PATH: /users -:END: - -#+begin_src restclient :results raw -GET :host/users -Authorization: Bearer :token -Accept: application/json -#+end_src - -,**** Notes -,**** Responses -#+end_example - -**** Commands -- =cj/restclient-new-scratch= :: open a scratch =restclient-mode= buffer. -- =cj/restclient-open-file= :: open a REST file from the configured API data dir. -- =cj/restclient-import-openapi= :: generate a workspace from OpenAPI/Swagger. -- =cj/restclient-describe-api= :: insert a =gptel=-generated API overview. -- =cj/restclient-describe-endpoint= :: explain the endpoint at point. -- =cj/restclient-generate-example-body= :: generate or refresh sample JSON body. -- =cj/restclient-save-last-response= :: save response body and metadata. -- =cj/restclient-insert-last-response-org= :: insert response under Org subtree. -- =cj/restclient-display-image-response= :: save and display image response. -- =cj/restclient-response-jq= :: run =jq= against the last JSON response. -- =cj/restclient-copy-curl= :: copy the request at point as curl if supported. - -**** Keybindings -Keep the existing prefix shape under =C-; R=: -- =C-; R n= :: new scratch request buffer -- =C-; R o= :: open REST file -- =C-; R i= :: import OpenAPI/Swagger -- =C-; R d= :: describe API or endpoint -- =C-; R s= :: save/capture last response -- =C-; R j= :: run =jq= on last response -- =C-; R m= :: transient menu for workspace commands - -**** OpenAPI Import Details -Minimum first-pass support: -- OpenAPI 3.x JSON -- Swagger 2.0 JSON if easy to normalize -- YAML via available Emacs YAML library or external converter if already present -- =servers= / =host= + =basePath= -- path-level and operation-level parameters -- query/path/header parameters -- JSON request bodies -- examples and default schema values -- security schemes: - - HTTP bearer - - basic - - API key header - - API key query - -First pass can skip or mark as unsupported: -- polymorphic schemas beyond simple =oneOf= / =anyOf= note generation -- multipart body generation -- full OAuth flows -- callbacks/webhooks -- complete response schema rendering - -**** Secret Handling -- Generated files must use placeholders like =:token=, =:api-key=, or - =:basic-auth=. -- Do not write tokens into generated files. -- Prefer variables loaded from auth-source, environment variables, or manually - entered restclient variables. -- =gptel= prompts must strip obvious auth headers and secret-like values from - request/response payloads before sending context. - -**** Response Handling -- Text JSON/XML/HTML/plain responses may be inserted into Org as source blocks. -- JSON responses should support optional =jq= filters. -- Image responses should be saved under =assets/responses/= and inserted as Org - file links. -- After inserting image links, call =org-display-inline-images= when in Org. -- Large responses should be saved as files and linked rather than inserted. -- Binary non-image responses should be saved and linked with content type notes. - -**** Integration Choices To Evaluate -- =restclient.el= remains the default executor for =.rest= files. -- =ob-restclient= powers executable Org source blocks. -- =verb.el= should be evaluated as an alternate frontend for Org-native API - workspaces because it already has tree inheritance, embedded Elisp, and image - response display. -- Hurl should be evaluated as an export/test target, especially for chained - request tests and CI assertions. - -**** Testing Strategy -- Unit-test OpenAPI normalization with small fixture specs. -- Unit-test request generation for GET, POST, auth, query params, path params, - and request-body examples. -- Unit-test secret redaction before =gptel= calls. -- Unit-test response classification and Org insertion helpers. -- Integration-test generated =.rest= files for representative APIs. -- Integration-test generated Org workspaces with =ob-restclient= where feasible. -- Avoid live network tests by default; use fixtures/stubs unless explicitly - tagged =:network=. - -**** First Milestone -1. Keep/verify existing restclient scratch/open commands. -2. Add a tiny OpenAPI JSON parser/normalizer for a fixture spec. -3. Generate a readable =.rest= collection with variables and examples. -4. Generate a matching Org workspace. -5. Add response save/link helpers for JSON and images. -6. Add a =gptel= API-summary command with secret redaction. -7. Add focused unit tests and one sample fixture. - -**** Open Questions -- Should the canonical workspace be =.rest= first, Org first, or generated in - both formats by default? -- Is =verb.el= a better primary target than =ob-restclient= for the Org notebook - experience? -- Which YAML parser should be used if no YAML package is already configured? -- Where should API workspaces live by default: =data/=, =docs/=, or project root? -- Should response capture hook into restclient internals, or should capture be a - separate explicit command operating on response buffers? -- How much schema synthesis is useful before it becomes noisy? - -*** Ideas -- Treat =restclient.el= as the lightweight request scratchpad, and layer richer workflow commands around it instead of trying to turn it into Postman. -- Add an OpenAPI/Swagger importer: - - prompt for a spec URL or local file - - parse paths, methods, auth schemes, request bodies, and examples - - generate a =.http= / =.rest= request collection - - optionally generate an Org notebook with one subtree per endpoint -- Add an API description command using =gptel=: - - summarize the API from an OpenAPI spec - - explain each endpoint, auth requirement, parameters, and response shape - - write the summary into the generated Org workspace -- Generate example requests automatically: - - =GET= examples with query params - - =POST= / =PUT= / =PATCH= examples with sample JSON bodies - - =DELETE= examples with safe placeholder IDs - - shared variables like =:host=, =:token=, =:api-version=, =:tenant= -- Add auth helpers: - - bearer token insertion - - basic auth - - API key header/query param - - OAuth token fetch request template -- Improve Org integration: - - use =ob-restclient= for executable API notebooks - - support =:jq= filters for clean JSON results - - save response bodies under endpoint subtrees - - capture request/response metadata as Org properties -- Handle images and binary responses: - - detect image =Content-Type= - - save response data into an assets directory - - insert =[[file:...]]= links into Org - - call =org-display-inline-images= after execution -- Add response-processing commands: - - pretty-print JSON/XML - - run =jq= against the last response - - summarize response with =gptel= - - extract fields into restclient variables for follow-up calls -- Add request collection ergonomics: - - imenu/consult navigation by endpoint - - transient menu for send/copy curl/jq/save response - - templates for common headers - - per-project API workspace discovery -- Investigate =verb.el= as the richer Org-native frontend: - - Org tree inheritance for host/header/auth defaults - - embedded Elisp in request specs - - built-in display of image/PDF/SVG responses - - likely better base for a notebook-style API client -- Investigate Hurl for repeatable API tests: - - request chaining - - captures - - assertions - - CI-friendly execution - - possible export target from generated API workspaces - -*** Original Goals -**** Keybindings to test -- C-; R n — new scratch *restclient* buffer (should open in restclient-mode) -- C-; R o — open .rest file (should default to data/ directory) - -**** Functional tests -1. Open tutorial-api.rest, run JSONPlaceholder GET (C-c C-c) — verify response inline -2. Run POST example — verify 201 response with fake ID -3. Run httpbin header echo — verify custom headers echoed back -4. Navigate between requests with C-c C-n / C-c C-p -5. Test jq filtering (requires jq installed): restclient-jq loaded? -6. Open scratch buffer (C-; R n), type a request manually, execute -7. which-key shows "REST client" menu under C-; R - ** TODO [#C] Build debug-profiling.el module :feature:solo: :PROPERTIES: :LAST_REVIEWED: 2026-06-21 @@ -4912,6 +4062,874 @@ Three small reveal.js improvements; collected into one task because each on its ** TODO [#D] Treesitter grammar offline cache :feature: Treesitter grammars are downloaded by =treesit-auto= on first use and live outside the localrepo. For true offline reproducibility, cache the grammars next to the localrepo (a =.localrepo/treesitter/= tier, or a separate mirror script). Cross-linked from =docs/design/localrepo.org=. * Emacs Someday/Maybe + +** TODO [#D] GPTel orphan tasks and useful ideas :feature: +:PROPERTIES: +:LAST_REVIEWED: 2026-06-01 +:END: + +On 2026-06-23 gptel was archived out of the live config (its modules, +tools, tests, and specs moved to archive/gptel/) because it sees little +use. This subtree was the gptel agentic-tool feature backlog. I kept it +here in Someday/Maybe instead of deleting it: most of the child ideas +below are agent-tool concepts that are not gptel-specific (org-roam node +tools, git section tools, ripgrep / pandoc / ffmpeg / jq wrappers, +messaging and buffer-state tools) and would carry over to the ai-term +Claude agents or an MCP tool layer if that path is taken. They are +reference, not active work. + +Categories below thematize the agent affordances the design doc +[[file:docs/design/gptel-agentic-tool-ideas.org][gptel-agentic-tool-ideas.org]] +points at -- Git, Org, messaging, file / buffer / workspace state, +media, and the dev loop. The shortlist's first-batch ADOPT tools +(git_status / git_log / git_diff / web_fetch) already shipped; the +themes below are next-tier work where the agent treats Emacs as a +structured workspace, not a text terminal. Per-theme spec lives in +the task body once written; implementation tasks land as siblings +of the spec heading once the spec is approved. The magit-backend +reimplementation of the shipped git tools is tracked separately in +[[id:bd47c9a8-aae1-4a3d-ad5b-b8767f2fd580][gptel-git-tools-magit-backend-spec.org]]. + +*** TODO [#C] Wire Up MCP.el so That GPTel Has Access to MCP Servers via GPTel Tools + +**** 2026-05-16 Sat @ 15:44:36 -0500 Spec + +Design doc: [[id:b4c274c5-8572-4a7b-b657-d315712bd6af][docs/specs/mcp-el-gptel-integration-spec-doing.org]] + +**** 2026-05-17 Sun @ 14:14:34 -0500 Landed ai-mcp.el pure-helper foundation + +Commit =54d231be=. Sections 1 (constants + defcustoms) and 3 (pure helpers) of the seven-section outline. 41 ERT tests, all green. Refactor audit caught two duplications during Phase 4 and folded them into the same commit (=cj/mcp--get-server-entry= and =cj/mcp--name-matches-p=). Phase 1.5 (confirmation contract) is next. + +**** TODO [#C] Phase 1.5 -- GPTel confirmation contract + +*Goal:* flip =gptel-confirm-tool-calls= to ='auto= and gate the existing local tools that need it. + +*Entry:* Phase 1 module exists and helpers tested. + +*DECISION (cj):* which of the existing local tools register with =:confirm t= once ='auto= is in effect? Reads (=read_buffer=, =read_text_file=, =list_directory_files=, =git_status=, =git_log=, =git_diff=) clearly stay =:confirm nil=. Judgment calls: +- =web_fetch= -- fetches arbitrary URLs the agent supplies. Spec recommends gating. +- =write_text_file= -- writes any path under =$HOME= with agent-supplied content. +- =update_text_file= -- modifies an existing file with an agent-supplied transform. +- =move_to_trash= -- moves a path to trash (reversible but disruptive). + +*Deliverables:* +- =ai-mcp.el= setup section runs =(setq gptel-confirm-tool-calls 'auto)=. +- Remove =(setq gptel-confirm-tool-calls nil)= from =modules/ai-config.el:386= with a comment pointing at =ai-mcp.el=. +- For each tool the decision marks "gate," add =:confirm t= to its =gptel-make-tool= form. +- Tests in =tests/test-ai-mcp-confirm-contract.el= asserting: =gptel-confirm-tool-calls= is ='auto= after load; write-classified stub MCP tool with =:confirm t= triggers the confirm branch in =gptel-send='s dispatch (stub the prompt); read-classified MCP tool with =:confirm nil= does not; =git_log= (=:confirm nil=) still runs without prompting; each newly-gated local tool does prompt. + +*Exit:* tests green. Manual smoke: open GPTel, call a gated tool, confirm prompt appears. Call =git_log=, no prompt. + +**** TODO [#B] Phase 2 -- Compat layer + registration pipeline (fake inventory) + +*Goal:* implement the mcp.el compat wrappers and the tool-registration pipeline against stubbed =mcp-server-connections=. + +*Entry:* Phase 1.5 proves gptel respects per-tool =:confirm= slot. + +*Deliverables:* +- Section 4 of =ai-mcp.el= (compat layer): =cj/mcp--server-status=, =cj/mcp--server-tools=, =cj/mcp--server-name=, =cj/mcp--assert-capabilities=. Each helper documents the upstream commit / file location it targets. +- Section 5 of =ai-mcp.el= (registration pipeline): =cj/mcp--register-tool=, =cj/mcp--register-server-tools=, =cj/mcp--deregister-server-tools=, =cj/mcp--rewrite-plist=, =cj/mcp--registered-tools= hash. +- All MCP tools register with =:async t=. +- Tests in =tests/test-ai-mcp-registration.el=. + +*Exit:* with a stubbed =mcp-server-connections=, registration produces correctly prefixed =mcp__SERVER__TOOL= entries in =gptel-tools=; closures call =mcp-call-tool SERVER REMOTE-NAME= (verified by stubbing =mcp-async-call-tool=); deregistration removes only MCP-owned tools and leaves a pre-populated local =git_log= entry intact; re-registration replaces function pointer without duplicating menu entries; confirm overrides win over patterns. + +**** TODO [#B] Phase 3 -- Async state machine + timer-race timeout wrapper + +*Goal:* implement the lifecycle state machine and the per-call timer-race timeout. + +*Entry:* Phase 2 registration works against stubs. + +*Deliverables:* +- Section 6 of =ai-mcp.el= (async state machine): =cj/mcp--state=, =cj/mcp--server-status= alist, =cj/mcp--stall-timer=, =cj/mcp-ensure-started=, =cj/mcp--on-hub-callback=, =cj/mcp--poll-status=, =cj/mcp--start-stall-timer=, =cj/mcp--build-status-from-specs=. +- =cj/mcp--wrap-async-with-timeout= (timer/callback race; both branches set =done= before invoking gptel callback so late responses are ignored). +- Tests in =tests/test-ai-mcp-async.el=. + +*Exit:* =cj/mcp-ensure-started= returns in <100 ms with delayed-callback stubs; stall timer fires for stuck servers; timer-race wrapper handles all three orderings (MCP-first, timer-first, late-MCP-after-timer); async error path (=:error-callback= without inited callback) reaches =failed= state via polling. + +**** TODO [#B] Phase 4 -- First real connection (drawio or slack-deepsat) + +*Goal:* wire one real no-auth server end-to-end against actual mcp.el and prove the stubbed Phase 3 behavior matches reality. + +*Entry:* Phase 3 async works against stubs. + +*Deliverables:* +- Add =use-package mcp= to =ai-mcp.el= (MELPA active, =:load-path= for local checkout commented). +- =cj/mcp--assert-capabilities= called at load time; signals clearly if mcp.el is too old. +- Set =cj/mcp-enabled-servers= temporarily to =("drawio")= (or =("slack-deepsat")= if the local proxy is running). +- First real =cj/mcp-ensure-started= invocation from =cj/toggle-gptel=. + +*Exit:* manual smoke -- =C-; a t= opens GPTel without blocking; within 30 s, drawio (or slack-deepsat) tools appear in =gptel-menu= grouped by category; calling a tool returns expected output; killing the subprocess externally surfaces as =failed= in =cj/mcp--server-status=. + +**** TODO [#B] Phase 5 -- Status UX + commands + doctor (static) + +*Goal:* ship the full server-management UX so partial-availability and failures are visible. + +*Entry:* Phase 4 proves a real connection works. + +*Deliverables:* +- Section 7 of =ai-mcp.el= (UI). +- Commands: =cj/mcp-status= (echo-area summary keyed off =cj/mcp--state=), =cj/mcp-list-tools= (tabulated buffer with failed servers at top in red face; keys =g r c RET q=), =cj/mcp-doctor= (static mode only -- capability, =npx=/=uvx=, Claude config, per-server env, local endpoints; output buffer keys =c r q=), =cj/mcp-wait-until-ready=, =cj/mcp-hub= (thin wrapper that ensures startup first), =cj/mcp-restart-failed=, =cj/mcp-restart-server=, =cj/mcp-stop-all=. +- Keymap: =C-; a C= subprefix bound in =ai-config.el='s autoload section. Keys =h s l r R S d w=. +- which-key labels for every binding. +- =kill-emacs-hook= registration for =cj/mcp-stop-all=. +- Investigation: does =gptel-menu= refresh after mid-call tool registration? Document the answer in =ai-mcp.el= commentary; if it requires close+reopen, add to known UX caveats. + +*Exit:* all keymap bindings work; audit buffer surfaces failed servers prominently; doctor identifies each scenario in the manual test matrix; status command shows the right state for each phase transition. + +**** TODO [#B] Phase 6 -- HTTP servers (linear, notion) + +*Goal:* add the two HTTP-transport servers with in-protocol OAuth. + +*Entry:* Phase 5 UX shipped. + +*Deliverables:* +- Add =linear= and =notion= back to =cj/mcp-enabled-servers=. +- Doctor gains live-auth-check mode (=C-u C-; a C d=): invokes a single safe read per auth class to verify OAuth tokens haven't silently expired. Static checks first; live probe only fires after static passes. +- OAuth recovery pattern matcher surfaces auth URLs in =cj/mcp-status= on first connect. + +*Exit:* first connect surfaces the OAuth URL through the recovery pattern; after browser handshake completes, subsequent connects succeed without prompt; live-auth-check correctly identifies a deliberately revoked token; both servers appear ready in the audit buffer. + +**** TODO [#B] Phase 7 -- Env-dependent stdio servers (figma, google-*) + +*Goal:* add the remaining five env-dependent servers. + +*Entry:* Phase 6 HTTP servers connect cleanly. + +*Deliverables:* +- Add =figma=, =google-calendar=, =google-docs-personal=, =google-docs-work=, =google-keep= to =cj/mcp-enabled-servers=. +- Verify env-merge from =~/.claude.json= for each (the mtime-cached reader from Phase 1). +- Verify figma's =:secret-args= splicing places the API key correctly without echoing it. +- Manual smoke: simulate token expiry on one Google server; recovery message points at "re-auth via Claude Code, then C-; a C r SERVER". + +*Exit:* all 9 servers reach =ready= state on a clean machine. Sentinel-grep check across status / audit / hub / errors / audit-log shows zero secret leakage. Doctor's live-auth covers each auth class (oauth, token, args-token, in-protocol, local, none). + +**** TODO [#B] Phase 8 -- Privacy + audit polish + +*Goal:* land the final UX polish and documentation. + +*Entry:* all 9 servers working. + +*Deliverables:* +- Audit buffer privacy header: "Tool results land in =gptel-tools= responses; saved conversations persist them. Use =cj/gptel-autosave-toggle= per buffer to opt out." +- =cj/mcp-tool-audit-log-enabled= defcustom + log writer (=~/.emacs.d/data/mcp-tool-log/YYYY-MM-DD.log= -- metadata only, one line per call, daily rotation). +- =ai-mcp.el= commentary updated with the code-organization outline as a table of contents. +- Final pass on tests covering saved-conversation behavior (autosave persists MCP tool results; toggling off prevents persistence). + +*Exit:* all 10 acceptance criteria from the spec pass. Manual matrix run end-to-end on a fresh Emacs. Working tree clean. + +*** TODO [#C] Wrap the gh CLI as a GPTel tool + +**** 2026-05-16 Sat @ 16:20:00 -0500 Spec + +Design doc: [[id:a124dd0f-1f40-4533-aeb8-595d93e20865][docs/specs/gptel-gh-tool-spec.org]] + +*** TODO [#C] GPTel should autosave regularly after a conversation is saved +*** TODO [#B] Org Workflow Related Tools + +Affordances that expose the Org workspace -- agenda state, capture +targets, org-roam nodes and backlinks, dailies, drill review state -- +to the agent as structured context, not raw .org buffer text. + +**** TODO [#B] Agenda state tools :feature: + +Read scheduled / deadline / waiting tasks for a date range; query by +tag, priority, or TODO keyword; list what's blocking today. Lets the +agent answer "what's on the critical path this week" without me +pasting agenda output, and feeds the daily-prep / wrap-up workflows. + +**** TODO [#B] Org-roam node tools :feature: + +Resolve a topic to its node; return body + backlinks; list nodes by +tag; surface dailies for a date range. Lets the agent reason over +the personal knowledge graph and write back into it via the capture +tools below. + +**** TODO [#B] Capture creation tools :feature: + +Drive =org-capture= from a template key + body string. Lets the +agent file inbox items, reading notes, journal entries, or roam +nodes without me leaving the chat. Tight pairing with the +=cj/org-capture= optimization task in todo.org. + +**** TODO [#B] Org-drill review tools :feature: + +Surface next-due drill cards in =drill-dir=; let the agent quiz on a +topic and report performance. Useful for prompted recall sessions +("ask me five medical-Spanish cards") and for "did this card stick" +analysis. + +*** TODO [#B] Git Related Tools + +Affordances that expose magit's structured view of a repo -- sections, +staged-vs-unstaged, commit metadata, rebase / conflict state -- as +first-class tools rather than asking the model to reason over raw +diff text. + +**** TODO [#B] Section-aware git tools :feature: + +Expose Magit sections as first-class GPTel tools: current section type, +heading, file, hunk range, and content; sibling sections under the same +file; staged / unstaged / untracked status; commit metadata around the +selected commit or branch; the exact staged patch that would be +committed. Lets prompts say "review the file section at point" or +"explain this hunk in the context of adjacent hunks" without manual +context-copying. + +**** TODO [#B] Commit intent workbench :feature: + +Transient that builds a commit intentionally: +1. Agent reads unstaged + staged changes. +2. Agent proposes coherent commit groups. +3. User selects groups in a Magit-style buffer. +4. Agent stages those paths or hunks only after confirmation. +5. Agent generates a message reflecting the selected intent. + +Addresses the common case of two or three unrelated edits in one +working tree -- a single commit-message generator can't handle that +cleanly. + +**** TODO [#B] Patch narrative buffer :feature: + +Generate an Org buffer that explains a change set as a reviewable +narrative: +- "What changed" by subsystem. +- "Why it appears to have changed" inferred from names, tests, and docs. +- "Risk areas" with links back to Magit file sections. +- "Suggested verification" using local Makefile targets when present. + +Reusable artifact: paste into a PR description, save with an AI +session, or file into org-roam. + +**** TODO [#B] Review-thread simulator :feature: + +Before opening a PR, create a local review buffer with inline comments +attached to Magit diff positions. The agent writes comments as if +reviewing someone else's patch: +- Comments grouped by severity. +- Each comment links to file and line. +- Resolved comments check off in Org. +- Accepted suggestions apply through the existing text-update tools. + +Makes "review my diff" less ephemeral and avoids losing useful findings +inside a chat transcript. + +**** TODO [#B] Rebase and conflict coach :feature: + +When Magit enters a rebase, cherry-pick, merge, or conflict state, +expose an agent command that reads: +- Git operation state from =.git/=. +- Conflict markers in the worktree. +- Relevant commits from =git log --merge= or the rebase todo. +- The current Magit status sections. + +The agent explains the conflict in domain terms and proposes a +resolution patch; the actual edit and =git add= stay under explicit +user control. + +**** TODO [#B] Regression archaeology :feature: + +Magit transient that runs a bisect-like reasoning workflow: +- Ask for a symptom and a known-good / known-bad range. +- Summarize candidate commits in small batches. +- Use tests or user-provided repro commands when available. +- Maintain a bisect journal in an Org buffer. + +Even when the agent can't run the whole bisect, it keeps the +investigation structured and preserves why each commit was judged +good or bad. + +**** TODO [#B] Messaging Related Tools + +Affordances over mu4e, Slack, Telegram, and ERC. Same shape across +protocols: read recent threads, search by sender / topic, compose a +draft from a prompt + thread context, leave the send under explicit +user control. + +***** TODO [#B] Mu4e thread and compose tools :feature: + +Read the message at point and surrounding thread (with attachments +summarized); query the inbox by =from:= / =subject:= / date range; +compose a draft from a prompt + thread context using =org-msg=. +Pairs with the existing =mu4e-org-contacts-integration.el=. + +***** TODO [#B] Slack thread and compose tools :feature: + +Read channel / DM / thread history through =emacs-slack=; search by +user or channel; compose a draft message but leave sending to me. +Mirrors the mu4e shape so the agent's interface is uniform across +messaging protocols. + +***** TODO [#B] Telegram and IRC read tools :feature: + +Same shape as Slack for =telega= (Telegram) and =erc= (IRC): +recent-message reads, search, and draft compose. Bundled because +the API shape is identical even if the underlying clients differ. + +***** TODO [#B] Contact resolution tools :feature: + +Resolve a name to email / Slack ID / Telegram handle via +=org-contacts= and the configured address books. Removes the +"who's this person again" friction from the compose flows above. + +**** TODO [#B] File and Buffer Related Tools + +Affordances that expose the user's actual workspace -- open buffers, +narrowed regions, marked files, vterm / eshell sessions -- as +structured context. Stops the model from asking "what file are you +looking at" or "what region is selected." + +***** TODO [#B] Buffer state tools :feature: + +List visible buffers with major-mode + file (when any); read the +narrowed region instead of the whole buffer; report point + mark +positions and the active region's text. The single most-asked +question between turns becomes a tool call. + +***** TODO [#B] Dirvish / Dired tools :feature: + +Read marked files, sort state, and filter state from a Dired or +Dirvish buffer. Lets the agent operate on "the files I just marked" +rather than "files in this directory" -- a real distinction in any +review or refactor workflow. + +***** TODO [#B] Vterm session tools :feature: + +Recent command output from a named vterm session; scroll-history +search. Pairs naturally with the =ai-vterm= design: the agent +running in one project's vterm can read another project's vterm +without leaving the chat. + +***** TODO [#B] Eshell session tools :feature: + +Same shape as the vterm tools for =eshell= sessions -- last-command +output, history search, current directory. Most useful for +agent-driven inspection of long-running pipelines. + +**** TODO [#B] Filesystem Related Tools + +Affordances that let the agent operate on actual files on disk and +run common CLI utilities -- pandoc, ffmpeg, imagemagick, ripgrep, +fd, jq -- rather than relying on me to paste content or run +commands by hand. + +*Design tension to resolve before any of these ship: one tool per +utility, or one generic =run_shell_command=?* + +The shortlist's first pass DEFERRED a generic =run_shell_command=: +sandboxing to HOME + /tmp with a denylist for destructive ops is +straightforward, but the denylist can never be exhaustive, and +"confirmation for everything else" becomes click-fatigue. + +The children below take the other path -- *one gptel tool per +binary*, with a strictly-typed argv shape (e.g. +=pandoc_convert(input_path, output_format)=, not +=pandoc_convert(args_string)=). Each tool: + +- Validates its own paths (must be under HOME, outputs in a + sandboxed dir). +- Rejects dangerous flags explicitly (pandoc =--filter=, ffmpeg's + =-protocol_whitelist= chicanery, imagemagick's policy bypasses). +- Runs via =call-process= with an argv list -- no shell parsing, + no string-interpolation injection. +- Caps output and reports truncation inline. + +The trade-off is breadth: every new CLI tool means a new gptel tool +file. Acceptable because (a) the list of utilities I actually need +agent access to is small (~8 below covers most of it), and (b) each +wrapper gets type-checked argv and a focused description the model +can reason over, which is genuinely better than a free-form +=run_shell_command(string)=. + +The =eshell_submit= entry at the end is the escape hatch for one- +off needs the wrappers don't cover -- =:confirm t= always. + +Adjacent categories: the existing =gptel-tools/= file CRUD +(=read_text_file=, =write_text_file=, =update_text_file=, +=list_directory_files=, =move_to_trash=) is the foundation this +category extends. =web_fetch= is the network-fetch counterpart. + +***** TODO [#B] Document conversion (pandoc) :feature: + +Convert between markdown, org, html, pdf, docx, latex, epub, plain +text. Most common use: "extract this docx to markdown so I can +read it inline." Strict argv: input path, output format, optional +output path. Reject =--filter= and =--lua-filter= (arbitrary code +execution). Output written to a sandbox dir unless explicit +override. + +***** TODO [#B] Image manipulation (imagemagick) :feature: + +Resize, format-convert, get-metadata (=identify=), optionally crop / +rotate / annotate. Common use: "resize this PNG to a thumbnail" or +"convert these HEICs to JPEGs." Strict argv per operation. +Reject pre-validated dangerous formats (the historical EXR / SVG / +MVG CVE surface) unless explicitly enabled. ImageMagick's +=policy.xml= is the underlying defense; the wrapper enforces it at +the tool boundary too. + +***** TODO [#B] Audio / video processing (ffmpeg) :feature: + +Trim, transcode, extract audio, get-metadata (=ffprobe=). Paths +under HOME only; reject network-protocol inputs (=http:= / =rtmp:= +/ =rtsp:=) so the model can't pull from arbitrary sources. Pairs +with the existing transcription module -- the same "extract audio +from video" path =cj/transcribe-media= uses internally. + +***** TODO [#B] Content search (ripgrep) :feature: + +=rg= wrapper with path / glob filtering, result-count cap, optional +literal-vs-regex mode. Pure read. Was in the shortlist's ADOPT +bucket as =search_in_files=. Highest-leverage filesystem tool by +expected call frequency -- "where in this repo is X" is the +question I paste agent output for most often. + +***** TODO [#B] File discovery (fd) :feature: + +=fd= (or =find= fallback) wrapper, capped result count. Pure +read, lower stakes than =search_in_files= (filenames only, no +content). Common pairing: =find_file_by_name= then +=read_text_file=. + +***** TODO [#B] Metadata extraction (file / exiftool) :feature: + +=file= for MIME-type detection; =exiftool= for image / video / +audio metadata. Lets the agent answer "what is this file" or +"when was this photo taken" without me opening external tools. +Pure read. + +***** TODO [#B] Structured data processing (jq / yq) :feature: + +=jq= for JSON, =yq= for YAML / TOML. Filter / project / transform +structured data into a smaller, more focused view before reading. +Strictly read-only -- output goes to the chat, not to disk. The +agent often wants "the third element of .results" from a JSON file +and this is much cheaper than pasting the whole thing. + +***** TODO [#B] Eshell command submission :feature: + +Submit a single eshell command line, return output (capped). +=:confirm t= always -- this is the escape hatch where the +strictly-typed wrappers above don't fit, so each invocation needs +my eyeball. Eshell parses in-process (no /bin/sh fork) so the +security surface is narrower than a shell command runner, but it's +still effectively arbitrary execution -- treat it as such. + +**** TODO [#B] Media and Reading Related Tools + +Affordances over non-code content: feeds, PDFs, EPUBs, music. The +agent's job here is summarize / extract / queue, not produce. + +***** TODO [#B] Elfeed entry tools :feature: + +Read entry body; list unread by feed or tag; mark read after a +summary lands in a roam node or inbox. Enables "give me the +non-noise headlines from this week's feeds" flows. + +***** TODO [#B] PDF and EPUB text tools :feature: + +Extract plain text from a PDF page or page range (via =pdftotext=) +and from an EPUB (via the existing nov-mode pipeline). Lets the +agent summarize / quote a research paper or book chapter without +me pasting passages. + +***** TODO [#B] EMMS playback and queue tools :feature: + +Current track, queue contents, playback state; queue or play a +path; compose a playlist from a prompt ("play something focusing +that's not Nick Cave"). Light tools, but a frequent friction +point. + +**** TODO [#B] Development Workflow Related Tools + +Affordances over the dev loop: compilation output, test invocation, +coverage / profile data, flycheck / flymake diagnostics. + +***** TODO [#B] Compilation buffer tools :feature: + +Read the most recent =compile= buffer output; parse error locations +to =file:line=; summarize what broke. Pairs with the F6 test-runner +flow -- "tell me what's failing" becomes a single agent turn +instead of paste + parse. + +***** TODO [#B] Project test invocation tools :feature: + +Run =make test-file FILE=X= / =make test-name TEST=Y= / +project-equivalent and return results. Currently each agent guesses +the project convention; expose the canonical invocation explicitly +per project so the agent can run focused tests itself. + +***** TODO [#B] Coverage and profile tools :feature: + +Read the most recent SimpleCov JSON or profile dump. Lets the +agent answer "what's still uncovered after this push" or "what +function dominates startup time" against real measured data. + +***** TODO [#B] Diagnostic tools (flycheck / flymake) :feature: + +Surface current-buffer or project-wide errors and warnings. Useful +both as a "what's broken right now" check and as input to the +patch-narrative buffer / commit-intent workbench above. + +**** CANCELLED [#B] gptel-magit activation fails on velox :bug:quick: +:PROPERTIES: +:LAST_REVIEWED: 2026-06-01 +:END: +Surfaced 2026-05-25 while diagnosing an unrelated load failure over SSH. velox-specific — the workstation has a current gptel and does not show it. + +At startup (and reproducibly in batch) velox logs: "Unable to activate package `gptel-magit'. Required package `gptel-0.9.8' is unavailable." gptel-magit depends on gptel >= 0.9.8 and velox's installed gptel is older or missing, so it can't activate. A startup warning, not a blocker. + +Reproduce: +: emacs --batch --no-site-file -L . -L modules --eval "(package-initialize)" --eval "(message \"done\")" 2>&1 | grep -i gptel + +Next step: check the installed gptel version (=(assq 'gptel package-alist)= or =M-x package-list-packages=), update gptel to >= 0.9.8, then re-evaluate gptel-magit activation. If gptel was pinned/held on velox, reconcile the pin against the gptel-magit dependency. + + +** TODO [#C] Build an Org-native API workspace :feature:test: +:PROPERTIES: +:LAST_REVIEWED: 2026-06-21 +:END: + +Moved to Someday/Maybe on 2026-06-23 when gptel was archived. This +design used gptel for its AI-assisted parts (restclient-ai.el, API +summaries, the AI-debugging ideas); with gptel archived those modules +are orphaned. The non-AI parts (OpenAPI import, request execution, +response capture) still stand on their own, so I shelved the whole design +here rather than dropping it. Pull it back to Open Work if the non-AI +core is worth building. + +Build an Emacs-native API workspace layer that keeps =restclient.el= useful for +lightweight request execution while adding OpenAPI import, Org notebooks, +response capture, inline images, and =gptel=-assisted API documentation. + +*** Spec + +**** Summary +Build a small Emacs API-workspace layer that keeps =restclient.el= as the +plain-text request executor, while adding higher-level discovery, generation, +Org notebook, response handling, and =gptel=-assisted documentation workflows. + +The intended result is not a Postman clone. It should feel like an +Emacs-native API lab: +- generate request collections from OpenAPI/Swagger specs +- browse and execute requests from =.rest= files or Org notebooks +- save useful responses back into Org +- display image responses inline +- use =gptel= to explain APIs, endpoints, request bodies, and responses +- leave room for =verb.el= or Hurl where they are a better fit + +**** Goals +- Preserve the current lightweight =restclient.el= workflow for ad-hoc calls. +- Add an OpenAPI importer that can generate useful first-draft request files. +- Add an Org workspace format for API notes, executable request blocks, results, + response links, and generated summaries. +- Add response post-processing helpers for JSON, binary/image data, and model + summaries. +- Make common auth flows easy to scaffold without storing secrets in generated + files. +- Keep generated artifacts readable and editable by hand. +- Keep implementation modular enough that =verb.el= can be evaluated as a richer + Org-native frontend. + +**** Non-Goals +- Do not build a full graphical API client. +- Do not store secrets directly in Org or =.rest= files. +- Do not attempt complete OpenAPI 3.1 schema synthesis in the first pass. +- Do not require =gptel= for ordinary request execution. +- Do not make Hurl a runtime dependency for the restclient workflow. + +**** Primary User Flows +***** Import An API +1. User runs =cj/restclient-import-openapi=. +2. Command prompts for a URL or local OpenAPI/Swagger file. +3. Command parses the spec into normalized endpoint records. +4. Command prompts for output format: + - =restclient= request collection + - Org API workspace + - both +5. Command writes generated files under a project/API workspace directory. +6. Generated file includes host variables, auth placeholders, request examples, + and a short summary of how to start using the collection. + +***** Describe An API +1. User runs =cj/restclient-describe-api= from a generated workspace or spec file. +2. Command extracts API metadata, paths, operations, schemas, auth, and tags. +3. Command sends a bounded summary payload to =gptel=. +4. Result is inserted under an Org =Overview= subtree. + +***** Execute And Capture A Request +1. User executes a request via =restclient.el= or =ob-restclient=. +2. Command captures response status, headers, content type, body, and timestamp + when available. +3. JSON/XML responses can be inserted as source blocks or summarized. +4. Image responses are saved to an assets directory and linked inline in Org. +5. Binary responses that are not image-like are saved as file links. + +***** Iterate On A Response +1. User runs helper commands against the last response: + - pretty-print JSON/XML + - run =jq= + - summarize with =gptel= + - extract a value into a restclient variable + - save body to a file +2. The result is inserted near the endpoint subtree or shown in a temporary + buffer depending on command prefix/options. + +**** Proposed Modules +***** =modules/restclient-workspace.el= +Main configuration and user-facing commands. Owns keybindings, workspace +creation, dispatch commands, and integration with existing restclient config. + +***** =modules/restclient-openapi.el= +OpenAPI/Swagger loading and normalization: +- fetch URL or read file +- parse JSON/YAML where available +- normalize servers, paths, methods, parameters, request bodies, auth schemes, + tags, operation IDs, and examples +- expose pure helper functions suitable for unit tests + +***** =modules/restclient-generate.el= +Generation layer: +- render =.rest= / =.http= request collections +- render Org workspaces +- synthesize simple JSON request bodies from schemas/examples +- generate variables and auth placeholders + +***** =modules/restclient-response.el= +Response helpers: +- identify content type +- save binary/image bodies +- insert Org links +- display inline images +- route JSON/XML/text handling +- track last response metadata + +***** =modules/restclient-ai.el= +=gptel= integration: +- summarize API specs +- explain endpoint behavior +- generate example payload notes +- summarize responses +- keep prompts bounded and avoid sending secrets + +**** File And Workspace Layout +Default generated workspace: + +#+begin_src text +data/apis/<api-slug>/ + <api-slug>.rest + <api-slug>.org + openapi.json + assets/ + responses/ +#+end_src + +The exact root should be configurable, defaulting to the existing REST data +directory if one is already present in the config. + +**** Org Workspace Shape +#+begin_example +,* API Name +:PROPERTIES: +:OPENAPI_SOURCE: https://example.com/openapi.json +:BASE_URL: https://api.example.com +:END: + +,** Overview +Generated or user-written API notes. + +,** Auth +Token acquisition notes and placeholders. No secrets stored here. + +,** Endpoints +,*** GET /users +:PROPERTIES: +:OPERATION_ID: listUsers +:METHOD: GET +:PATH: /users +:END: + +#+begin_src restclient :results raw +GET :host/users +Authorization: Bearer :token +Accept: application/json +#+end_src + +,**** Notes +,**** Responses +#+end_example + +**** Commands +- =cj/restclient-new-scratch= :: open a scratch =restclient-mode= buffer. +- =cj/restclient-open-file= :: open a REST file from the configured API data dir. +- =cj/restclient-import-openapi= :: generate a workspace from OpenAPI/Swagger. +- =cj/restclient-describe-api= :: insert a =gptel=-generated API overview. +- =cj/restclient-describe-endpoint= :: explain the endpoint at point. +- =cj/restclient-generate-example-body= :: generate or refresh sample JSON body. +- =cj/restclient-save-last-response= :: save response body and metadata. +- =cj/restclient-insert-last-response-org= :: insert response under Org subtree. +- =cj/restclient-display-image-response= :: save and display image response. +- =cj/restclient-response-jq= :: run =jq= against the last JSON response. +- =cj/restclient-copy-curl= :: copy the request at point as curl if supported. + +**** Keybindings +Keep the existing prefix shape under =C-; R=: +- =C-; R n= :: new scratch request buffer +- =C-; R o= :: open REST file +- =C-; R i= :: import OpenAPI/Swagger +- =C-; R d= :: describe API or endpoint +- =C-; R s= :: save/capture last response +- =C-; R j= :: run =jq= on last response +- =C-; R m= :: transient menu for workspace commands + +**** OpenAPI Import Details +Minimum first-pass support: +- OpenAPI 3.x JSON +- Swagger 2.0 JSON if easy to normalize +- YAML via available Emacs YAML library or external converter if already present +- =servers= / =host= + =basePath= +- path-level and operation-level parameters +- query/path/header parameters +- JSON request bodies +- examples and default schema values +- security schemes: + - HTTP bearer + - basic + - API key header + - API key query + +First pass can skip or mark as unsupported: +- polymorphic schemas beyond simple =oneOf= / =anyOf= note generation +- multipart body generation +- full OAuth flows +- callbacks/webhooks +- complete response schema rendering + +**** Secret Handling +- Generated files must use placeholders like =:token=, =:api-key=, or + =:basic-auth=. +- Do not write tokens into generated files. +- Prefer variables loaded from auth-source, environment variables, or manually + entered restclient variables. +- =gptel= prompts must strip obvious auth headers and secret-like values from + request/response payloads before sending context. + +**** Response Handling +- Text JSON/XML/HTML/plain responses may be inserted into Org as source blocks. +- JSON responses should support optional =jq= filters. +- Image responses should be saved under =assets/responses/= and inserted as Org + file links. +- After inserting image links, call =org-display-inline-images= when in Org. +- Large responses should be saved as files and linked rather than inserted. +- Binary non-image responses should be saved and linked with content type notes. + +**** Integration Choices To Evaluate +- =restclient.el= remains the default executor for =.rest= files. +- =ob-restclient= powers executable Org source blocks. +- =verb.el= should be evaluated as an alternate frontend for Org-native API + workspaces because it already has tree inheritance, embedded Elisp, and image + response display. +- Hurl should be evaluated as an export/test target, especially for chained + request tests and CI assertions. + +**** Testing Strategy +- Unit-test OpenAPI normalization with small fixture specs. +- Unit-test request generation for GET, POST, auth, query params, path params, + and request-body examples. +- Unit-test secret redaction before =gptel= calls. +- Unit-test response classification and Org insertion helpers. +- Integration-test generated =.rest= files for representative APIs. +- Integration-test generated Org workspaces with =ob-restclient= where feasible. +- Avoid live network tests by default; use fixtures/stubs unless explicitly + tagged =:network=. + +**** First Milestone +1. Keep/verify existing restclient scratch/open commands. +2. Add a tiny OpenAPI JSON parser/normalizer for a fixture spec. +3. Generate a readable =.rest= collection with variables and examples. +4. Generate a matching Org workspace. +5. Add response save/link helpers for JSON and images. +6. Add a =gptel= API-summary command with secret redaction. +7. Add focused unit tests and one sample fixture. + +**** Open Questions +- Should the canonical workspace be =.rest= first, Org first, or generated in + both formats by default? +- Is =verb.el= a better primary target than =ob-restclient= for the Org notebook + experience? +- Which YAML parser should be used if no YAML package is already configured? +- Where should API workspaces live by default: =data/=, =docs/=, or project root? +- Should response capture hook into restclient internals, or should capture be a + separate explicit command operating on response buffers? +- How much schema synthesis is useful before it becomes noisy? + +*** Ideas +- Treat =restclient.el= as the lightweight request scratchpad, and layer richer workflow commands around it instead of trying to turn it into Postman. +- Add an OpenAPI/Swagger importer: + - prompt for a spec URL or local file + - parse paths, methods, auth schemes, request bodies, and examples + - generate a =.http= / =.rest= request collection + - optionally generate an Org notebook with one subtree per endpoint +- Add an API description command using =gptel=: + - summarize the API from an OpenAPI spec + - explain each endpoint, auth requirement, parameters, and response shape + - write the summary into the generated Org workspace +- Generate example requests automatically: + - =GET= examples with query params + - =POST= / =PUT= / =PATCH= examples with sample JSON bodies + - =DELETE= examples with safe placeholder IDs + - shared variables like =:host=, =:token=, =:api-version=, =:tenant= +- Add auth helpers: + - bearer token insertion + - basic auth + - API key header/query param + - OAuth token fetch request template +- Improve Org integration: + - use =ob-restclient= for executable API notebooks + - support =:jq= filters for clean JSON results + - save response bodies under endpoint subtrees + - capture request/response metadata as Org properties +- Handle images and binary responses: + - detect image =Content-Type= + - save response data into an assets directory + - insert =[[file:...]]= links into Org + - call =org-display-inline-images= after execution +- Add response-processing commands: + - pretty-print JSON/XML + - run =jq= against the last response + - summarize response with =gptel= + - extract fields into restclient variables for follow-up calls +- Add request collection ergonomics: + - imenu/consult navigation by endpoint + - transient menu for send/copy curl/jq/save response + - templates for common headers + - per-project API workspace discovery +- Investigate =verb.el= as the richer Org-native frontend: + - Org tree inheritance for host/header/auth defaults + - embedded Elisp in request specs + - built-in display of image/PDF/SVG responses + - likely better base for a notebook-style API client +- Investigate Hurl for repeatable API tests: + - request chaining + - captures + - assertions + - CI-friendly execution + - possible export target from generated API workspaces + +*** Original Goals +**** Keybindings to test +- C-; R n — new scratch *restclient* buffer (should open in restclient-mode) +- C-; R o — open .rest file (should default to data/ directory) + +**** Functional tests +1. Open tutorial-api.rest, run JSONPlaceholder GET (C-c C-c) — verify response inline +2. Run POST example — verify 201 response with fake ID +3. Run httpbin header echo — verify custom headers echoed back +4. Navigate between requests with C-c C-n / C-c C-p +5. Test jq filtering (requires jq installed): restclient-jq loaded? +6. Open scratch buffer (C-; R n), type a request manually, execute +7. which-key shows "REST client" menu under C-; R + * Emacs Resolved ** DONE [#B] Fix likely =elpa-mirror-location= path bug :bug:quick: CLOSED: [2026-05-03 Sun] |