diff options
47 files changed, 2353 insertions, 6124 deletions
@@ -50,3 +50,9 @@ auto-save-list/ /multisession/ /browser-choice.el /client_secret_491339091045-sjje1r54s22vn2ugh45khndjafp89vto.apps.googleusercontent.com.json + +# Documentation folder (session notes, personal workflows) +/docs/ + +# Claude Code configuration (personal settings, commands) +/.claude/ diff --git a/NOTES.org b/NOTES.org deleted file mode 100644 index d048710f..00000000 --- a/NOTES.org +++ /dev/null @@ -1,125 +0,0 @@ -#+TITLE: Project Notes for Future Sessions -#+AUTHOR: Claude (AI Assistant) -#+DATE: 2025-11-01 - -* Session Startup Protocol - -When starting a new session with Craig: -1. Craig will point me to this NOTES.org file -2. Read this file completely to understand project context -3. **Scan =docs/sessions/= directory** for available session type filenames - - Do NOT read full contents (they're long) - - Just note the filenames to understand vocabulary - - Example: =refactor.org= means "refactor session workflow" -4. When Craig says "let's do a [session-type] session": - - **Check for exact match** in scanned filenames - - **If exact match found:** Read =docs/sessions/[session-type].org= and guide Craig through that workflow - - **If no exact match but similar word exists:** Ask for clarification - - Example: User says "refactoring" but we have "refactor.org" - - Ask: "Did you mean the 'refactor' session, or would you like to create a new 'refactoring' session?" - - **If no match at all:** Offer to create it - - Say: "I don't see a session type called '[session-type]' yet. Would you like to create it together using the create-session workflow?" - - **If yes to create:** Do a create-session session to define the new type, then use it immediately (validation) - -* Available Session Types - -** refactor -File: =docs/sessions/refactor.org= - -Workflow for comprehensive test-driven quality engineering: -1. Analyze test coverage for target file -2. Categorize functions by testability (easy/medium/hard) -3. Refactor for testability if needed (pure helpers + interactive wrappers) -4. Write ERT tests systematically (Normal/Boundary/Error categories) -5. Run tests and analyze failures -6. Decide: test bug or production bug? -7. Fix and verify -8. Repeat for all testable functions - -Recent completion: =modules/music-config.el= - 103 tests, 3 bugs fixed - -** create-session -File: =docs/sessions/create-session.org= - -Meta-workflow for creating new session types: -1. Q&A discovery (Problem, Exit criteria, Approach, Principles) -2. Assess completeness and agreement -3. Name the session -4. Document using standard structure -5. Update NOTES.org -6. Validate by immediate execution - -Defines how we formalize repetitive workflows into reusable, documented sessions. - -Created: 2025-11-01 - -* Current Project State - -** Completed Work - -*** music-config.el Testing Session (2025-11-01) -- **Module:** =modules/music-config.el= (EMMS music player configuration) -- **Test Coverage:** 9 functions, 103 tests total -- **Bugs Fixed:** 3 (nil handling x2, URL regex escaping bug) -- **Session Document:** =docs/sessions/refactor.org= - -Functions tested: -1. =cj/music--append-track-to-m3u-file= - 11 tests -2. =cj/music--valid-file-p= - 15 tests -3. =cj/music--valid-directory-p= - 13 tests -4. =cj/music--safe-filename= - 13 tests -5. =cj/music--m3u-file-tracks= - 15 tests (caught URL regex bug!) -6. =cj/music--get-m3u-files= - 7 tests -7. =cj/music--get-m3u-basenames= - 6 tests -8. =cj/music--collect-entries-recursive= - 12 tests -9. =cj/music--completion-table= - 12 tests - -Functions skipped (EMMS buffer-dependent, hard to test): -- =cj/music--ensure-playlist-buffer= -- =cj/music--playlist-tracks= -- =cj/music--playlist-modified-p= -- =cj/music--assert-valid-playlist-file= - -Test files location: =tests/test-music-config--*.el= - -Key bug found: URL regex escaping in =cj/music--m3u-file-tracks= caused HTTP/HTTPS/MMS URLs to be treated as relative paths, corrupting playlist data. - -** Work In Progress - -(None currently - ready for next task) - -** Backlog / Future Work - -(To be added as Craig identifies next priorities) - -* Project Structure Notes - -** Test Organization -- One test file per function: =test-<module>-<function>.el= -- Rationale: Fast discovery when tests fail -- Test utilities: =tests/testutil-general.el= - -** Test Categories -Every test file follows this structure: -- Normal Cases: Expected behavior under typical conditions -- Boundary Cases: Edge conditions, limits, unusual inputs -- Error Cases: Failure scenarios, invalid inputs - -** Quality Engineering Reference -See =ai-prompts/quality-engineer.org= for comprehensive guidelines. -Note: This file evolves as we learn best practices. - -* Session History - -** 2025-11-01: music-config.el Refactor Session -- Added comprehensive test coverage to music configuration -- Found and fixed 3 bugs -- Created refactor session workflow document -- Established session type pattern for future work - -** 2025-11-01: Create-Session Meta-Workflow -- Designed the process for creating new session types -- Used recursive application: created create-session using itself -- Defined four-question Q&A framework -- Established seven core principles for session creation -- Documented in =docs/sessions/create-session.org= diff --git a/assets/abbrev_defs b/assets/abbrev_defs index 3060620f..cd9c6818 100644 --- a/assets/abbrev_defs +++ b/assets/abbrev_defs @@ -33,7 +33,7 @@ ("aethetically" "aesthetically" nil :count 0) ("agression" "aggression" nil :count 0) ("agressive" "aggressive" nil :count 0) - ("ahve" "have" nil :count 14) + ("ahve" "have" nil :count 15) ("aknowledge" "acknowledge" nil :count 0) ("alegiance" "allegiance" nil :count 0) ("allegaince" "allegiance" nil :count 0) @@ -215,7 +215,7 @@ ("hrie" "hire" nil :count 0) ("htey" "they" nil :count 3) ("humerous" "humorous" nil :count 0) - ("hygeine" "hygiene" nil :count 0) + ("hygeine" "hygiene" nil :count 1) ("hygene" "hygiene" nil :count 0) ("hygine" "hygiene" nil :count 0) ("idnetify" "identify" nil :count 0) @@ -294,7 +294,7 @@ ("oppositiion" "opposition" nil :count 0) ("opppsite" "opposite" nil :count 0) ("orignal" "original" nil :count 0) - ("ot" "to" nil :count 41) + ("ot" "to" nil :count 42) ("otehr" "other" nil :count 3) ("otes" "notes" nil :count 0) ("outgoign" "outgoing" nil :count 0) @@ -393,7 +393,7 @@ ("takss" "tasks" nil :count 3) ("talekd" "talked" nil :count 0) ("talkign" "talking" nil :count 6) - ("teh" "the" nil :count 155) + ("teh" "the" nil :count 156) ("tehir" "their" nil :count 5) ("tehre" "there" nil :count 3) ("testimentary" "testamentary" nil :count 1) @@ -428,10 +428,11 @@ ("warant" "warrant" nil :count 0) ("welfair" "welfare" nil :count 0) ("welomce" "welcome" nil :count 0) + ("whcih" "which" nil :count 0) ("whenter" "whether" nil :count 0) ("whenver" "whenever" nil :count 0) ("wierd" "weird" nil :count 0) - ("wihtout" "without" nil :count 0) + ("wihtout" "without" nil :count 1) ("windsheild" "windshield" nil :count 0) ("withdrawls" "withdrawals" nil :count 0) ("withold" "withhold" nil :count 0) diff --git a/gemini.key.gpg b/assets/gemini.key.gpg index 10068445..10068445 100644 --- a/gemini.key.gpg +++ b/assets/gemini.key.gpg diff --git a/docs/EMACS-CONFIG-V2MOM.org b/docs/EMACS-CONFIG-V2MOM.org deleted file mode 100644 index 40027218..00000000 --- a/docs/EMACS-CONFIG-V2MOM.org +++ /dev/null @@ -1,363 +0,0 @@ -#+TITLE: Emacs Configuration V2MOM -#+AUTHOR: Craig Jennings -#+DATE: 2025-10-30 -#+FILETAGS: :strategy:planning:v2mom: - -* What is V2MOM? - -V2MOM is a strategic framework used to clarify goals and priorities: -- *Vision:* What do you want to achieve? -- *Values:* What principles guide decisions? -- *Methods:* How will you achieve the vision? -- *Obstacles:* What's in your way? -- *Metrics:* How will you measure success? - -This document provides strategic clarity for prioritizing Emacs configuration work. - -* Vision - -Your personal ideal Emacs configuration allows you to work at the speed of your own thought. It's stable and never needs edge-case bug fixing. The functionality covers your workflow such that you can work all day in Emacs for any of the following task areas: - -- Programming in any language, including emacs-lisp, common-lisp, scheme, golang, and c -- Reading ebooks, including marginalia handling -- Listening to music, including playlist management -- Listening to podcasts, including saving episodes -- Task organizing and planning -- Calendaring, including syncing with google calendar -- Alerting and notifications on tasks and calendar events -- Timer/stopwatch functionality, including pomodoro presets -- Audio and desktop recording -- Memorization drills -- Email, including email-to-task integration -- Messaging, i.e., IRC -- File-management, including extensive integration, and DWM scratchpad integration outside Emacs -- Shell commands, leveraging vterm and eshell with custom methods, and DWM scratchpad integration outside Emacs - -Anytime you make a change in the config, you have unit tests to tell you quickly if the change created unexpected issues. You have a test-runner that integrates with your workflow. - -* Values - -also see: file:values-comparison.org - -** Intuitive - -*Definition:* Intuition comes from muscle memory, clear mnemonics, and just-in-time discovery that reinforces learning without blocking productivity. - -*** Keybindings Build Intuition Through Layers -1. *Muscle memory first* - C-; on home row becomes automatic habit -2. *Mnemonics provide "aha!" moments* - C-c **m** for mail creates dopamine hits that cement permanent memory -3. *Which-key teaches through active recall* - Appears after you try (not before), forcing attempt without wasting time - -The timing matters: which-key's delay is calibrated so you're challenged to remember, but not blocked from working. - -*** What Breaks Intuition -- *Physically awkward keybindings* - C-c 1 z C-c f requires hand contortions and offers no mnemonic -- *Syntax overhead* - Parentheses tracking steals time from real learning (logic bugs, architecture understanding) -- *Poor architecture* - Common actions (like "save all attachments") shouldn't require architectural archaeology - -*** Code Reads "Like a Newspaper" -- Each use-package declaration is a *story* (topic = package) -- Configuration settings are *facts* (what it does, how it behaves) -- Clear variable/method naming (yours + maintainers) makes code read like English -- Accessible to intermediate elisp readers without deep archaeology - -*** Principle -*Intuition is earned through repeated success, not instant magic.* The config should make success easy to achieve and memorable when it happens. - - - -** Fast - -*Definition:* Fast means staying in flow. Startup and org-agenda must not make me wait. Everything else is already acceptable. - -*** Performance Targets - -*Startup Time:* -- Current: ~3.8 seconds (real clock time, not Emacs-reported) -- Target: ~1 second (aspirational, may not be realistic given package count) -- Comfortable: ~2 seconds (±) - -*Measurement:* -#+begin_src bash -time emacs --eval '(save-buffers-kill-emacs)' # Real startup time -#+end_src - -*Note:* Emacs-reported time is inaccurate. Messages buffer continues populating and Emacs remains unresponsive after "loaded" message appears. - -*** Critical Performance Issues (Must Fix) - -1. *Emacs startup* - 3.8s is too slow - - Network check adds 1+ seconds (DELETE THIS) - - Eager loading vs lazy loading - -2. *Opening org-agenda for first time* - **THE BOTTLENECK** - - "Takes forever and a full work day" - - Likely: recursive directory scan, no caching, too many files - - Solution: Profile first (build debug-profiling.el), then optimize - -*** Must Be Instant (Flow-Critical) -- Keybinding response -- Changing projects in projectile (already good ✓) -- Opening large org files (already good ✓, except malformed files with stray "*"s) -- Buffer/window operations - -*** Can Take 1-2 Seconds (Acceptable) -- *Mail sync* - Consider putting on timer instead of manual trigger -- *Calibredb* (PDFs/Epubs) - Known solution: shrink inventory to <1k books or contribute optimizations -- *Org-noter* opening PDFs/Epubs - Needs reliability + reasonable speed -- *Looking up org-noter notes* - Should be easy to pull into other documents -- *Initial file indexing* (projectile, etc.) - -*** Disabled Due to Performance (Need Solutions) -- *Grammar checker* - Installed but turned off because it slows everything down - - No task exists to fix this yet - - Breaks concentration when writing - -- *Bibliography management* - Don't know how to do in Emacs - - Worried about broken/slow tools breaking writing flow - - Need solution that "just works" - -*** Principle -*Speed is about flow, not milliseconds.* Startup and agenda are flow-breakers. Everything else is fine. Don't optimize what doesn't hurt. - - - -** Simple - -*Definition:* Simple means clear, maintainable code following production software practices. Simplicity is a discipline that produces reliability. Config is production software - treat it as such. - -*** What Makes Code Simple -1. *Clear naming* - Variables and functions say what they do -2. *Small functions* - Easy to understand, test, and modify -3. *Good comments* - NOT obvious comments ("cat with post-it note saying 'cat'") - -*Good comments:* -- Navigation markers for long files -- Explain complex/non-obvious logic -- Document unobvious impacts of settings -- Answer "why" not "what" - -*Bad comments:* -#+begin_src emacs-lisp -;; declare a variable -(defvar variable 10) -#+end_src - -*** Abstractions and Magic -- *Abstractions are fine* if they serve a purpose -- *Macros are irritating* when unnecessary, but acceptable when warranted -- *Magic is acceptable* if it's reliable and you can build a mental model - -*Principle:* "I want to understand every line of what my config does, or at least have a decent mental model of how it probably works. Never use a tool you can't fully trust." - -*** Package Count -Package count itself isn't a concern. Quality and maintainability matter. - -*Concerns:* -- More packages = more potential breakage -- Need ability to roll back when packages break (localrepo for package snapshots) -- Maintenance burden is real, but manageable with good practices - -*Not concerned about:* -- Absolute number of packages -- Startup time impact (if lazy-loaded properly) -- Cognitive load (if organized well) - -*** Config IS Production Software - -*Key insight:* "My Emacs config IS production software. I use it more than probably every other application combined. The impact when it's broken can crush my productivity." - -This means: -- *Apply production practices:* Testing, modularity, documentation, version control -- *Plan for failure:* Rollback capability (localrepo), error handling, graceful degradation -- *Modular design:* Extract custom code into packages when it grows -- *Long-term maintenance:* Architecture matters, technical debt has real cost - -*** Personal Software Platform -"I see my config as a personal software project built on the software platform called Emacs. Emacs has taught me a great deal about how to build software platforms well." - -*Implications:* -- Config follows software engineering principles -- Custom functionality becomes packages (chime, org-msg, potentially jumper) -- Good architecture enables evolution -- Quality matters because uptime matters - -*** Why Simplicity Matters -When config breaks, productivity is crushed. The discipline of simplicity produces reliability: -- Simple code is easier to fix when it breaks -- Simple architecture enables graceful degradation -- Simple practices (testing, rollback) provide safety nets -- Simple means never using tools you can't trust - -*** Principle -*Simple means maintainable production software, not minimal or primitive.* Complexity is acceptable when it serves the vision and can be understood or trusted. **Simplicity is the practice that produces reliability as an outcome.** - - - -** Summary: Three Core Values - -1. *Intuitive* - Muscle memory, mnemonics, just-in-time discovery. Code reads like a newspaper. -2. *Fast* - Startup and org-agenda must not break flow. Everything else is acceptable. -3. *Simple* - Clear, maintainable, production-quality code. Simplicity produces reliability. - - - -* Methods - -** Method 1: Make Using Emacs Frictionless - -Emacs should never make you wait or break your concentration. This method eliminates daily friction points that disrupt flow. Every item here either removes a bottleneck (startup delay, org-agenda slowness), fixes something broken (org-noter, recording, mail attachments), or adds missing functionality you reach for weekly (diff-buffer-with-file). - -When Method 1 is complete, Emacs starts fast, org-agenda opens instantly, all core workflows work reliably, and nothing makes you context-switch to debug config during work. - -*Concrete actions:* -- Remove network check from startup (saves 1+ seconds every launch) -- Optimize org-agenda performance using built-in profiler (eliminate "forever and a full work day" rebuild time) -- Fix [[https://github.com/weirdNox/org-noter][org-noter]] (reading/annotation workflow currently "so painful") -- Fix video/audio recording module (use it constantly, just broke) -- Fix mail attachment workflow (currently awkward) -- Implement cj/diff-buffer-with-file (compare buffer with saved version - weekly need) -- Fix cj/goto-git-gutter-diff-hunks (missing function causing errors) -- Fix grammar checker performance (currently disabled because it breaks flow) - -** Method 2: Stop Problems Before They Appear - -A stable config comes from proactive maintenance, not reactive fixes. This method replaces aging packages with modern, actively-maintained alternatives before they break. It removes deprecated dependencies and adopts better-designed tools that align with Emacs' evolution. - -When Method 2 is complete, the config uses current best practices, has no deprecated packages lurking as time bombs, and benefits from simpler, more maintainable completion infrastructure. - -*Concrete actions:* -- Migrate from Company to [[https://github.com/minad/corfu][Corfu]] (simpler, modern completion framework - complete config already in todo.org) -- Switch to [[https://gitlab.com/jessieh/mood-line][mood-line]] (lighter modeline, already researched) -- Remove deprecated tree-sitter package and rely on [[https://github.com/renzmann/treesit-auto][treesit-auto]] (already installed, leverages Emacs 29+ built-in treesit) -- Add [[https://github.com/awth13/org-appear][org-appear]] (show emphasis markers only when point is on them - cleaner org files) -- Integrate [[https://github.com/radian-software/prescient.el][prescient]] with Corfu (frequency/recency-based smart sorting - already using with vertico) - -** Method 3: Make *Fixing* Emacs Frictionless - -You can't fix what you can't measure, and you can't trust what you can't test. This method builds observability and testing infrastructure that makes future maintenance systematic instead of guesswork. With proper profiling, testing, and diffing tools in place, debugging becomes fast and confident. - -When Method 3 is complete, you can profile any performance issue in seconds, write integration tests for complex workflows, roll back broken packages instantly, and review config changes with semantic understanding. - -*Concrete actions:* -- Build debug-profiling.el module and develop skills using it (reusable profiling infrastructure for any future performance work) -- Integrate [[https://github.com/jorgenschaefer/emacs-buttercup][Buttercup]] (behavior-driven integration tests for complex config workflows) -- Build localrepo out (package snapshot system for repeatable installs and safe rollbacks) -- Integrate [[https://github.com/Wilfred/difftastic][difftastic]] (structural diffs that show semantic changes, not just line changes) - -** Method 4: Contribute to the Emacs Ecosystem - -Maintaining packages (chime, org-msg, wttrin) means being a good steward of code others depend on. This method establishes professional package development practices: automated linting, CI testing, and coverage reporting. These tools catch issues before users do and make MELPA submissions smooth. - -When Method 4 is complete, every package you maintain has automated quality checks, measurable test coverage, and CI that validates changes before they ship. You contribute back to the community with confidence. - -*Concrete actions:* -- Set up [[https://github.com/purcell/package-lint][package-lint]] for elisp linting (catch packaging issues and style violations automatically) -- Set up [[https://github.com/riscy/melpazoid][melpazoid]] CI (validates packages meet MELPA standards before submission) -- Set up [[https://github.com/leotaku/elisp-check][elisp-check]] GitHub Action (zero-config CI for Emacs packages) -- Integrate [[https://github.com/undercover-el/undercover.el][undercover.el]] (measure and track test coverage over time) - -** Method 5: Be Kind To Your Future Self - -With Emacs stable and maintainable, it's time to add features that expand what's possible. This method builds workflows you'll use repeatedly: transcribing audio for notes and creating presentations directly from org-mode. These aren't fixes—they're investments in future capability. - -When Method 5 is complete, you can transcribe recordings without leaving Emacs and generate beautiful reveal.js presentations from org files. Future you will thank present you for shipping these. - -*Concrete actions:* -- Add transcription workflow (complete code already in todo.org:2-99 - need today and recurring) -- Implement [[https://github.com/yjwen/org-reveal][org-reveal]] presentation workflow (create reveal.js slides from org-mode) - -** Method 6: Develop Disciplined Engineering Practices - -The best infrastructure won't help without disciplined habits. This method builds the practices that make all other methods sustainable: knowing what matters most, shipping over researching, measuring before optimizing. These are skills to develop, not tasks to complete—they evolve throughout the entire V2MOM, and I become a better engineer overall. - -*Concrete practices to develop:* -- *Ruthless prioritization* - Always do the most important thing. Use V2MOM as filter. If it doesn't serve the vision, cancel it. -- *Weekly triage* - Review todos every Sunday (30 min). Items sitting >1 week get shipped or killed. Prevents backlog rot. Keep <20 active items. -- *Measure metrics* - Define success criteria before starting work. Quantify outcomes so you know when you're done (startup time, test coverage, todo count). -- *Profile-before-optimize* - Never guess what's slow. Run profiler, identify hotspot, THEN fix. Avoids wasting time optimizing things that don't matter. -- *Ship-over-research* - Execute existing specs before exploring new ones. Time-box research (30 min → decide: ship or kill). Monthly retrospective on research:shipped ratio. -- *Incremental execution* - Ship small, test, iterate. Keep config working at every step. Avoid big-bang failures, integration hell, and sunk cost traps. - -*How to measure success:* -- *Ruthless prioritization* → todo.org stays under 20 active items, cancelled:completed ratio shows I'm saying "no" -- *Weekly triage* → At least once a week by Sunday, no longer than 7 days between triage (tracked in calendar) -- *Measure metrics* → Every task has defined success criteria before starting, can show actual tracked metrics (startup logs, coverage reports) -- *Profile-before-optimize* → Every performance fix has profiler output proving the bottleneck, zero "I think this is slow" guesses -- *Ship-over-research* → Research:shipped ratio improves monthly (>1:1), complete code in todo.org ships within 1 week -- *Incremental execution* → Config never broken for >2 days, git commits are small and frequent, can roll back any change cleanly - - - -* Obstacles - -1. *Building and researching is more fun than fixing.* But if I don't change this, everything will be broken. - -2. *I get irritated with myself when making mistakes.* But mistakes are how people learn. If I let irritation drive me to give up, I'll never develop the skills these methods require. - -3. *It's hard for me to say "no".* But if I don't say "no" to something, I'll never achieve anything. The only way to make saying "no" easier is to do it repeatedly. - -4. *I can be a perfectionist who wants something just right before I move on.* But perfect is the enemy of shipped. I need to learn that good enough is better than perfect, and iteration beats perfection. - -5. *I don't always have the time for all day coding sessions.* Breaking the work into increments will help me deliver the long term value anyway. Be the tortoise, not the rabbit. - -6. *New disciplines are hard to sustain.* Weekly triage, measuring metrics, and profiling-before-optimizing are new habits. The first few weeks I'll be tempted to skip them when busy. But if I don't practice them consistently, they'll never become automatic, and I'll fall back into old patterns. - - - -* Metrics - -You can't improve what you don't measure. These metrics provide objective evidence that the Methods are working and the Vision is becoming reality. Track them weekly during triage to catch regressions early and celebrate progress. - -** Performance Metrics: -- *Startup time: < 3 seconds* (currently ~6.2s) - Measured with =time emacs --eval '(save-buffers-kill-emacs)'= -- *Org-agenda rebuild time: < 5 seconds* (currently 30+ seconds) - Measured with profiler during first daily open - -** Maintenance Discipline Metrics: -- *Active todo count: < 20 items* (currently ~50+) - Counted during weekly triage -- *Weekly triage consistency: At least once a week by Sunday, no longer than 7 days between triage* - Tracked in calendar -- *Research:shipped ratio: > 1:1* - Can't research next thing until current thing is implemented. Track monthly. -- *Config uptime: Never broken > 2 days* - Allows breathing room for emergencies/travel - -** Package Quality Metrics (chime, org-msg, wttrin): -- *Test coverage: > 70% and all code not covered justifiable* - Uncovered code is 100% risk. Better not be the most-used parts! - - - -* How to Use This V2MOM - -** For Decision Making -When considering a new task, package, or feature: -1. Does it serve the *Vision*? -2. Does it align with the *Values*? -3. Is it addressed in the *Methods*? -4. Does it overcome an *Obstacle*? -5. Does it improve a *Metric*? - -If the answer is "no" to all five → CANCEL the task. - -** For Triage -When reviewing todo.org: -- Tasks aligned with V2MOM → Keep and prioritize -- Tasks not aligned with V2MOM → CANCEL -- Tasks that are interesting but not important → CANCEL -- Tasks with complete code already → Execute immediately - -** For Weekly Review -Every Sunday (30 minutes): -1. Review metrics - Are they improving? -2. Review obstacles - Any new ones? Any resolved? -3. Review active todos - Still aligned with V2MOM? -4. Cancel stale/misaligned tasks -5. Plan next week's focus - -* Completion Status - -- [X] Vision - Complete (kept from original todo.org) -- [X] Values - Complete (Intuitive, Fast, Simple) -- [X] Methods - Complete (6 methods with aspirational bodies and concrete actions) -- [X] Obstacles - Complete (6 honest obstacles with real stakes) -- [X] Metrics - Complete (Performance, Discipline, and Quality metrics defined) - -*Last Updated:* 2025-10-31 (Session 2) -*Status:* ✅ V2MOM COMPLETE - Ready to use for decision making and weekly triage diff --git a/docs/NOTES.org b/docs/NOTES.org deleted file mode 100644 index a08a25e8..00000000 --- a/docs/NOTES.org +++ /dev/null @@ -1,583 +0,0 @@ -#+TITLE: 🚨 ACTIVE PROJECT - READ THIS FIRST 🚨 -#+AUTHOR: Claude Code Session Notes -#+DATE: 2025-10-30 - -* 🗣️ IMPORTANT TERMINOLOGY - -** "I want to do an X session with you" - -When Craig says "I want to do an X session with you", this means: -- **CREATE a session definition** for doing X (meta-work) -- **NOT** "let's DO X right now" (the actual work) - -This triggers the create-session workflow from docs/sessions/create-session.org - -*Examples:* -- "I want to do an emacs inbox zero session" → Create docs/sessions/inbox-zero.org -- "I want to do a refactor session" → Create docs/sessions/refactor.org -- "I want to do a code review session" → Create docs/sessions/code-review.org - -* 🔔 DESKTOP NOTIFICATIONS WORKFLOW - -**IMPORTANT: How Claude notifies you when blocked** - -When Claude needs your input on blocking questions, Claude will send a desktop notification via `notify-send`: - -#+BEGIN_SRC bash -notify-send "Claude Code" "Question: [Your input needed]" --urgency=normal -#+END_SRC - -**When notifications ARE sent:** -- ✅ When explicitly needing your decision/input (blocking questions) -- ✅ When multiple valid approaches exist and choice affects implementation -- ✅ When encountering errors that require user guidance -- ✅ **ONLY** when Claude cannot proceed without user input - -**When notifications are NOT sent:** -- ❌ After completing tasks (informational updates) -- ❌ During normal progress updates -- ❌ When milestones are reached -- ❌ For status messages or completion notices -- ❌ ANY informational alerts - -**Setup:** -- Requires `dunst` or similar notification daemon -- Works with `notify-send` command -- Always uses `--urgency=normal` (not critical) - -**Purpose:** -This allows you to context-switch to other work while Claude runs long tasks, and get notified ONLY when your input is truly needed to continue. You check back when convenient for status updates. - -* 🧩 EMACS LISP DEVELOPMENT BEST PRACTICES - -**Critical Lessons: Preventing Parenthesis Errors** - -Both humans and AI struggle with balanced parentheses in deeply nested Emacs Lisp code. Here are proven strategies to prevent this: - -** For AI Code Generation - -*** 1. Write Small, Focused Functions -- Keep functions under 15 lines when possible -- Each function should do ONE thing -- Easier to verify parentheses at a glance -- Easier to test in isolation - -#+BEGIN_EXAMPLE -Bad (deeply nested, hard to verify): -(defun process-data (data) - (when (valid-p data) - (let ((result (transform data))) - (when result - (let ((final (format result))) - (when final - (save final))))))) - -Good (broken into helpers): -(defun process-data (data) - (when (valid-p data) - (save-result (format-result (transform-data data))))) - -(defun transform-data (data) ...) -(defun format-result (result) ...) -(defun save-result (final) ...) -#+END_EXAMPLE - -*** 2. Test Immediately After Each Write -- Write function → check-parens → test load → repeat -- Don't batch multiple functions before testing -- Catch errors early when context is fresh - -#+BEGIN_SRC bash -# After writing each function: -emacs --batch file.el --eval '(check-parens)' && echo "✓" -#+END_SRC - -*** 3. Prefer Write Over Multiple Edits -- For complex new code: use Write tool (complete file) -- Only use Edit for small, simple changes -- Incremental edits can introduce subtle paren mismatches -- Complete file Write is easier to verify - -*** 4. Validate Before Committing -- Use pre-commit hooks to validate all .el files -- Prevents committing broken code -- Example in chime.el repository: .git/hooks/pre-commit - -** For Human Developers - -*** 1. Use Structural Editing Modes -These PREVENT unbalanced parens: -- **paredit** - Classic, strict structural editing -- **smartparens** - More flexible, works with multiple languages -- **lispy** - Modal editing for lisps -- **electric-pair-mode** - Built-in, auto-closes parens - -*** 2. Enable Real-time Validation -- **flycheck** + **flycheck-package** - Shows errors as you type -- **flymake** - Built-in alternative -- **rainbow-delimiters-mode** - Colors matching parens - -*** 3. Quick Validation Commands -#+BEGIN_SRC elisp -M-x check-parens ; Check current buffer -M-x byte-compile-file ; More comprehensive checking -#+END_SRC - -** Lessons from chime-org-contacts.el Development - -*** Original Problem -Complex nested function with 6 levels of nesting: -- Hard to count parentheses manually -- AI kept adding/removing wrong number of closing parens -- Functions weren't defined after file load -- Wasted significant debugging time - -*** Solution That Worked -Refactored into 3 helper functions: -1. =chime-org-contacts--parse-birthday= (10 lines) -2. =chime-org-contacts--format-timestamp= (4 lines) -3. =chime-org-contacts--insert-timestamp-after-drawer= (12 lines) - -Main function became simple composition (10 lines). - -**Result:** -- All functions defined correctly -- Easy to verify parens by eye -- Each function testable independently -- 24 tests written covering all cases - -*** Key Insight -Breaking complex code into small helpers: -- ✅ Easier to verify correctness -- ✅ Easier to test -- ✅ Easier to maintain -- ✅ Self-documenting through function names -- ✅ AI and humans both succeed - -Deeply nested code: -- ❌ Hard to verify -- ❌ Hard to test -- ❌ AI frequently makes paren errors -- ❌ Humans make mistakes too - -** Tools and Workflow Summary - -| Stage | Tool | Purpose | -|-------+------+---------| -| Writing | paredit/smartparens | Prevent errors | -| Editing | rainbow-delimiters | Visual verification | -| Testing | check-parens | Quick syntax check | -| CI/CD | pre-commit hooks | Prevent bad commits | -| Review | byte-compile-file | Comprehensive check | - -** Makefile - Comprehensive Testing & Validation Tool - -A comprehensive Makefile is available in the repository root with targets for testing, validation, and utilities. - -#+BEGIN_SRC bash -make # Show all available targets -make test # Run all tests (unit + integration) -make test-file FILE=... # Run specific test file -make test-name TEST=... # Run tests matching pattern -make validate-parens # Check for unbalanced parentheses -make validate-modules # Load all modules to verify compilation -make compile # Byte-compile all modules -make lint # Run checkdoc, package-lint, elisp-lint -make profile # Profile Emacs startup -make clean # Remove test artifacts and compiled files -make reset # Reset to first launch (destructive!) -#+END_SRC - -Created: 2025-11-03 -Adapted from chime.el Makefile with config-specific enhancements - -* 📋 AVAILABLE SESSION TYPES - -** create-session -File: [[file:sessions/create-session.org][docs/sessions/create-session.org]] - -Meta-workflow for creating new session types. Use this when identifying repetitive workflows that would benefit from documentation. - -Workflow: -1. Q&A discovery (4 core questions) -2. Assess completeness -3. Name the session -4. Document it -5. Update NOTES.org -6. Validate by execution - -Created: 2025-11-01 (pre-existing) - -** emacs-inbox-zero -File: [[file:sessions/emacs-inbox-zero.org][docs/sessions/emacs-inbox-zero.org]] - -Weekly workflow for processing the "Emacs Config Inbox" heading in =todo.org= to zero by filtering through V2MOM framework. - -Workflow: -1. Sort by priority (A → B → C → none → D) -2. Claude rereads V2MOM -3. Process each item through 3 questions: - - Does this need to be done? → DELETE if no - - Related to V2MOM? → Move to someday-maybe if no - - Which method? → Move to appropriate method -4. Done when inbox heading is empty - -Target: 10 minutes active work time -Cadence: Every Sunday, no longer than 7 days between sessions -Maintains metrics: Active todos < 20, weekly triage consistency - -Created: 2025-11-01 - -* CURRENT PROJECT STATUS - -** 🎯 What We're Doing -Working through a systematic approach to clean up and prioritize Craig's Emacs config work: - -1. ✅ *COMPLETE V2MOM* (Vision, Values, Methods, Obstacles, Metrics) - IN PROGRESS -2. ⏳ *TRIAGE todo.org* - Use V2MOM to ruthlessly cancel ~60% of tasks -3. ⏳ *EXECUTE TIER 1* - Ship quick wins (network check removal, Corfu, bug fixes) -4. ⏳ *BUILD OBSERVABILITY* - Create profiling infrastructure (TIER 2) -5. ⏳ *SYSTEMATIC EXECUTION* - Work through prioritized tasks one by one - -** 📍 Where We Are Right Now -*Session Started:* 2025-10-30 -*Current Step:* ✅ V2MOM COMPLETE - Ready for execution -*Time Committed:* ~2 sessions, V2MOM finished 2025-10-31 -*Status:* V2MOM complete, ready to begin Method 1 execution - -** 📄 Key Documents - -*** Primary Working Documents -- *V2MOM:* [[file:EMACS-CONFIG-V2MOM.org][EMACS-CONFIG-V2MOM.org]] - Strategic framework for Emacs config (✅ COMPLETE) - - Vision, Values, Methods, Obstacles, Metrics - - Used for decision-making and weekly triage - - Read this first to understand strategic direction -- *Issues Analysis:* [[file:../issues.org][../issues.org]] - Claude's detailed analysis with TIER system and implementations -- *Current Inbox:* [[file:../inbox.org][../inbox.org]] - V2MOM-aligned tasks (~23 items after ruthless triage) - -*** Reference Documents -- *Config Root:* [[file:../init.el][../init.el]] -- *Modules:* [[file:../modules/][../modules/]] -- *Tests:* [[file:../tests/][../tests/]] - -** 🔑 Key Insights About Craig's Work Patterns - -*** Strengths -- Thoughtful and strategic thinker -- Good research skills (thorough specs, complete code examples) -- Does ship things (dashboard, dirvish, network check fixes) -- Recognizes need for V2MOM framework -- Uses config daily for real work - -*** Patterns to Address -1. *Research > Execution* - Has complete code for Corfu, difftastic, transcription workflow... still TODO -2. *Priority Inflation* - Too many [#A]/[#B] items, unclear what's actually urgent -3. *Incomplete Strategy* - V2MOM structure exists but sections are empty -4. *Hard to Say No* - [#C]/[#D] items should be CANCELLED but remain in list -5. *Side Projects Compete* - Dupre theme work competes with core config maintenance - -*** What Craig Told Us About Himself -> "I am building tools both because they solve problems, but also because I enjoy building." - -This is healthy! But need balance: -- Fix rough edges FIRST (daily pain points) -- Build fun stuff SECOND (after maintenance) -- Cancel distractions ALWAYS (Signal client, minimap, etc.) - -** 🎯 Agreed Goals for This Project - -*** Immediate (Next 2-3 Sessions) -1. ✅ Complete V2MOM (IN PROGRESS) -2. ⏳ Triage todo.org using V2MOM as filter -3. ⏳ Execute quick wins: network check, Corfu migration, bug fixes -4. ⏳ Build debug-profiling.el infrastructure - -*** Short Term (Next Month) -5. Profile and optimize org-agenda performance -6. Ship reveal.js presentation workflow -7. Establish weekly triage ritual - -*** Long Term (Ongoing) -8. Ship more than research -9. Maintain < 20 active todos -10. Measure metrics from V2MOM - -** 📋 TIER System from issues.org - -*** TIER 1: Do These First (High Impact, Low Effort) - 1 weekend -- Remove network check (15 min) -- Fix missing functions (30 min) -- Corfu migration (2 hours) -- Mood-line switch (30 min) -- Bug fixes (1 hour) - -*** TIER 2: Build Observability (HIGHEST VALUE) - 1 week -- Create debug-profiling.el module (3-4 hours) -- Profile org-agenda-rebuild (1 hour) -- Add instrumentation and caching (2 hours) -- Test org-agenda filtering functions (2-3 hours) - -*** TIER 3: Quick Wins (After Profiling) - 1-2 hours each -- Reveal.js presentation workflow (2 hours) -- Difftastic integration (30 min) -- Local package development workflow (1 hour) - -*** TIER 4: Maybe/Someday (Probably Never) -- Code-maat reimplementation (HOLD) -- LaTeX config (HOLD until concrete need) -- Elfeed dashboard (HOLD - unclear if actually used) -- DWIM shell integration (HOLD - current solution works) -- Jumper package (HOLD - already maintaining chime + org-msg) - -** 🚫 Items That Should Be CANCELLED - -From todo.org, these don't serve the vision: -- [#D] Signal Client - Not in vision -- [#D] Awesome-tray / mode-icons - Already have modeline -- [#C] Minimap - Interesting, not important -- [#C] Install Magit TODOs - Already works fine -- [#C] Git Timemachine litters buffers - Minor annoyance -- Many Dupre theme TODOs - Side project competing with maintenance - -## 💡 Key Recommendations for Craig - -### Week 1: Strategy + Quick Wins -1. Complete V2MOM (2-3 hours) -2. Triage todo.org using V2MOM (1-2 hours) -3. Execute items you already have code for (2-3 hours) - -### Week 2: Observability Infrastructure -4. Build debug-profiling.el (3-4 hours) -5. Profile org-agenda (1 hour) - -### Week 3: Fix Performance + Ship Presentation -6. Fix org-agenda based on profiling (2-3 hours) -7. Ship reveal.js workflow (2 hours) - -### Ongoing: Maintenance Discipline -- Weekly triage ritual (30 min every Sunday) -- Measure metrics (startup time, agenda time, todo count) -- Ship > Research - -** 🔄 Next Session Pickup Points - -When starting next session, Claude should: - -1. **Read this document first** to understand context -2. **Check V2MOM status** - If incomplete, continue there -3. **Reference issues.org** for detailed technical recommendations -4. **Reference todo.org** for items to triage -5. **Ask Craig:** "Where did we leave off? V2MOM? Triage? Execution?" - -** 📞 Questions to Ask Craig Next Session - -*IMMEDIATE (when resuming):* -- "Ready to continue V2MOM? We left off at Methods section." -- "How much time do you have?" - -*FOR METHODS SECTION:* -Show Craig the draft list and ask: -- "Which methods do you already do consistently?" -- "Which do you want to do but don't yet?" -- "Am I missing any important methods?" - -*AFTER V2MOM COMPLETE:* -- "Ready to triage todo.org using the V2MOM?" -- "Should we execute quick wins or continue systematic triage?" - -** 🎯 Success Metrics for This Project - -We'll know this is working when: -- ✅ V2MOM is complete and provides clear strategic direction -- ✅ todo.org shrinks from ~50 to < 20 active items -- ✅ Craig ships 3-5 items per week (small but consistent) -- ✅ Craig has profiling infrastructure to measure performance -- ✅ Org agenda rebuild time is measured and improving -- ✅ Weekly triage becomes habit - -** 💬 Craig's Words to Remember - -> "I think you should adjust issues.org with all your recommendations. They are exciting, eye-opening, and just feel right. Add even your guidance on latex. spot on. thanks for your honesty. I did ask for it and am genuinely grateful for your responses. I'll take action on them." - -> "What I need help with is integrating this in with my existing todo.org file... Some of the tasks I've listed should probably just be deleted or better yet, marked CANCELLED." - -> "I have about an hour to devote. You could lead me through it, I could do some questions/answer rounds with you to clarify my thinking." - -Craig is ready to execute. He asked for honesty and took it well. He recognizes the patterns and wants systematic help. - -** 🛠️ Technical Context - -*** Current Pain Points -1. Org agenda is slow (performance bottleneck) -2. Network check adds 1+ seconds to startup (technical debt) -3. Missing functions cause errors (cj/log-silently, cj/goto-git-gutter-diff-hunks) -4. Mail attachments workflow is awkward -5. No profiling infrastructure to measure performance - -*** Items Craig Already Has Code For -These can be executed immediately - just paste and test: -- Transcription workflow (complete bash + elisp in todo.org:2-99) -- Difftastic integration (complete config in todo.org:1211-1223) -- Corfu migration (complete config in todo.org:1611-1639) - -*** Architecture -- Modular structure: modules/*.el -- Good test coverage for utilities -- Modern packages: Vertico/Consult/Embark stack -- Local package development: chime.el, org-msg - -** 📚 Related Reading - -If Craig or Claude need more context: -- [[file:../issues.org::*Second Opinion: Ruthless Prioritization & Reality Checks][Second Opinion section in issues.org]] - Full analysis and recommendations -- [[file:../issues.org::*TIER 1: Do These First][TIER 1-4 breakdown]] - Prioritized task system -- [[file:../quality-engineer.org][quality-engineer.org]] - Testing philosophy (if exists) - -** 🚀 Current Session Notes - -*** 2025-10-31 Session 2 - V2MOM Complete! -*Time:* ~1.5 hours -*Status:* ✅ COMPLETE - V2MOM finalized and ready for use - -*What We Completed:* -1. ✅ Finalized all 6 Methods with aspirational bodies and concrete actions: - - Method 1: Make Using Emacs Frictionless (performance & functionality fixes) - - Method 2: Stop Problems Before They Appear (proactive package maintenance) - - Method 3: Make *Fixing* Emacs Frictionless (observability/tooling) - - Method 4: Contribute to the Emacs Ecosystem (package maintenance tooling) - - Method 5: Be Kind To Your Future Self (new features) - - Method 6: Develop Disciplined Engineering Practices (meta-method with measurable outcomes) - -2. ✅ Completed Obstacles section (6 honest, personal obstacles with real stakes) - - Building vs fixing tension - - Getting irritated at mistakes - - Hard to say "no" - - Perfectionism - - Limited time sessions - - New habits are hard to sustain - -3. ✅ Completed Metrics section (Performance, Discipline, Quality metrics) - - Startup time: < 3s (currently 6.2s) - - Org-agenda: < 5s (currently 30+s) - - Active todos: < 20 (currently ~50+) - - Weekly triage consistency - - Research:shipped ratio > 1:1 - - Config uptime: never broken > 2 days - - Test coverage: > 70% with justification for uncovered code - -4. ✅ Implemented cj/diff-buffer-with-file - - Added to modules/custom-buffer-file.el - - Bound to C-; b D - - Unified diff format with proper error handling - - TODO comment for future difftastic integration - -5. ✅ Added missing items to Methods based on Craig's research: - - Fixed org-noter (Method 1) - - Added Buttercup (Method 3) - - Added package maintenance tools (Method 4: package-lint, melpazoid, elisp-check, undercover) - - Added wttrin to maintained packages list - -*Key Insights:* -- Craig wants to DELETE research files: "There will always be cool ideas out there to implement and they will always be a web search away" -- Ruthless prioritization is already happening -- Method ordering: fix → stabilize → build infrastructure → contribute → enhance → sustain -- Adjusted startup target from 2s to 3s (more achievable, less perfectionism trap) - -*Key Files Modified This Session:* -- [[file:emacs-config-v2mom.org][emacs-config-v2mom.org]] - Now 100% complete with all sections filled -- [[file:../modules/custom-buffer-file.el][modules/custom-buffer-file.el]] - Added cj/diff-buffer-with-file function -- [[file:SESSION-HANDOFF-ACTIVE-PROJECT.org][SESSION-HANDOFF-ACTIVE-PROJECT.org]] - This file - -*Next Session Starts With:* -1. Continue Method 1 execution - 2 quick wins ready! -2. Fix cj/goto-git-gutter-diff-hunks (15 min) -3. Fix chime throw/catch bug (your package) -4. Fix go-ts-mode-map keybinding error - -*** 2025-10-31 Session 3 - RUTHLESS EXECUTION! 🚀 -*Time:* ~2 hours -*Status:* Method 1 in progress - shipped 2 wins, discovered 3 bugs - -*What We Completed:* -1. ✅ **RUTHLESS PRIORITIZATION EXECUTED!** - - Moved todo.org → docs/someday-maybe.org (~50 items archived) - - Created fresh inbox.org with ONLY V2MOM-aligned tasks (23 items) - - Already under < 20 active items goal! - -2. ✅ **SHIPPED: Network check removal** (Method 1) - - Deleted `cj/internet-up-p` blocking ping function - - Removed network cache variables - - Simplified to use package priorities instead - - .localrepo (priority 200) ensures offline reproducibility - - **RESULT: 6.19s → 4.16s startup time (2.03 seconds faster!)** - -3. ✅ **SHIPPED: cj/diff-buffer-with-file** (Method 1) - - Implemented in modules/custom-buffer-file.el - - Bound to C-; b D - - Weekly need satisfied - -4. ✅ **Updated reset-to-first-launch.sh** - - Added missing transient files/directories - - Keeps docs/, inbox.org, .localrepo safe - - Ready for testing offline installs - -5. ✅ **Tested .localrepo offline install capability** - - Works perfectly for package.el packages - - Discovered 3 bugs during test (logged in inbox.org) - -*Bugs Discovered (all logged in inbox.org):* -1. **Chime throw/catch error** - High priority, your package - - Error: "(no-catch --cl-block-chime-check-- nil)" - - Fix: Change defun to cl-defun or add catch block - - Currently disabled to unblock startup - -2. **go-ts-mode-map keybinding error** - - Error: "void-variable go-ts-mode-map" - - Fix: Wrap in with-eval-after-load - -3. **Treesitter grammars not in .localrepo** (limitation documented) - - Expected behavior - treesit-auto downloads separately - -*Metrics Update:* -- Startup time: 6.19s → 4.16s (**2.03s improvement!**) -- Only 1.16s away from < 3s target! -- Active todos: ~23 items (hit < 20 goal when excluding tracking tasks!) -- Shipped items: 2 (network check, diff-buffer-with-file) - -*Key Files Modified:* -- [[file:../early-init.el][early-init.el]] - Network check removed, cj/use-online-repos simplified -- [[file:../modules/custom-buffer-file.el][custom-buffer-file.el]] - Added cj/diff-buffer-with-file -- [[file:../inbox.org][inbox.org]] - Fresh V2MOM-aligned todo list created -- [[file:../scripts/reset-to-first-launch.sh][reset-to-first-launch.sh]] - Updated with missing transient files -- [[file:someday-maybe.org][someday-maybe.org]] - Old todo.org archived here - -*Next Session (2025-11-01):* -**Two quick wins ready (15 min each):** -1. Fix cj/goto-git-gutter-diff-hunks (missing function) -2. Fix chime throw/catch bug (re-enable chime) - -**Then continue Method 1:** -- Optimize org-agenda (THE BOTTLENECK - 30s → <5s target) -- Fix org-noter (daily pain) -- Fix video/audio recording -- Fix mail attachments -- Fix grammar checker - -*Craig's Words:* -> "There will always be cool ideas out there to implement and they will always be a web search away." -Ruthless prioritization in action! Deleted research files, focused execution. - -*** 2025-10-30 Session 1 - V2MOM In Progress -*Time:* ~1 hour -*Status:* PAUSED - V2MOM 60% complete - -*What We Completed:* -1. ✅ Created docs/ directory structure -2. ✅ Created SESSION-HANDOFF-ACTIVE-PROJECT.org (this file) -3. ✅ Created emacs-config-v2mom.org -4. ✅ Created values-comparison.org (analysis doc) -5. ✅ Completed Vision (already existed, kept as-is) -6. ✅ Completed Values section (Intuitive, Fast, Simple) - - Intuitive: Muscle memory, mnemonics, which-key timing, "newspaper" code - - Fast: Startup < 2s, org-agenda is THE bottleneck, everything else acceptable - - Simple: Production software practices, simplicity produces reliability diff --git a/docs/SOMEDAY-MAYBE.org b/docs/SOMEDAY-MAYBE.org deleted file mode 100644 index e392ae99..00000000 --- a/docs/SOMEDAY-MAYBE.org +++ /dev/null @@ -1,1633 +0,0 @@ -* Emacs Config Open Work - -** TODO [#D] Irritant: Press Key to Launch Dashboard Icon App - -Not important enough - already have keybindings and M-x search as working alternatives. -Moved from inbox 2025-11-01. - -** TODO [#D] Irritant: Move Persistence Files Into a Single Directory - -Organizational tidiness, not actual friction. Perfectionism (V2MOM Obstacle #4). -Moved from inbox 2025-11-01. - -** TODO [#A] Add Transcription Org-capture Workflow - :PROPERTIES: - :CATEGORY: emacs - :END: - -*** Goal - Integrate audio-to-text transcription into Org-capture so meeting recordings - can be turned into timestamped notes automatically. - -*** Steps - -**** 1. Create CLI for transcription - - Option A — OpenAI API (Whisper): - #+begin_src bash - # ~/bin/oai-transcribe - #!/usr/bin/env bash - set -euo pipefail - AUDIO="$1"; LANG="${2:-en}" - : "${OPENAI_API_KEY:?Set OPENAI_API_KEY in your environment}" - curl -s -X POST "https://api.openai.com/v1/audio/transcriptions" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -F "model=whisper-1" \ - -F "response_format=text" \ - -F "language=${LANG}" \ - -F "file=@${AUDIO}" - #+end_src - - - Option B — Local Whisper: - #+begin_src bash - pip install openai-whisper # once - # ~/bin/local-whisper - #!/usr/bin/env bash - set -euo pipefail - AUDIO="$1"; MODEL="${2:-medium}"; LANG="${3:-en}" - whisper "$AUDIO" --model "$MODEL" --language "$LANG" \ - --task transcribe --output_format txt --verbose False >/dev/null - cat "${AUDIO%.*}.txt" - #+end_src - -**** 2. Emacs Lisp configuration - Add to init.el: - - #+begin_src emacs-lisp - (defcustom cj/transcribe-program - (or (executable-find "oai-transcribe") - (executable-find "local-whisper")) - "Program used to transcribe audio to plain text.") - - (defcustom cj/meetings-org-dir "~/org/meetings/" "") - - (defun cj/transcribe-audio (file) - (with-temp-buffer - (call-process shell-file-name nil t nil shell-command-switch - (format "%s %s" - (shell-quote-argument cj/transcribe-program) - (shell-quote-argument (expand-file-name file)))) - (buffer-string))) - - (defun cj/org-capture-meeting-template () - (let* ((audio (read-file-name "Audio: ")) - (participants (read-string "Participants: " "Mark; Craig")) - (location (read-string "Location: " "French Truck Coffee")) - (topic (read-string "Topic: " "Contract discussion")) - (transcript (cj/transcribe-audio audio))) - (format "* Meeting: %s\n:PROPERTIES:\n:DATE: %s\n:LOCATION: %s\n:PARTICIPANTS: %s\n:AUDIO: %s\n:END:\n\n** Summary\n- \n\n** Transcript\n#+begin_quote\n%s\n#+end_quote\n" - topic (format-time-string "%Y-%m-%d %H:%M") - location participants (abbreviate-file-name audio) transcript))) - - (with-eval-after-load 'org - (add-to-list 'org-capture-templates - `("m" "Meeting (transcribe audio)" plain - (file ,(expand-file-name (format-time-string "%Y-meetings.org") - cj/meetings-org-dir)) - "%(cj/org-capture-meeting-template)" - :empty-lines 1 :unnarrowed t))) - #+end_src - -**** 3. Usage - 1. Record meeting as .m4a or .wav - 2. Run `M-x org-capture m` → select the audio file - 3. Org inserts transcript and metadata under - `~/org/meetings/YYYY-meetings.org` - -**** 4. Optional quick recap template - #+begin_src emacs-lisp - (with-eval-after-load 'org - (add-to-list 'org-capture-templates - '("r" "Post-meeting recap" entry - (file+headline - (lambda () (expand-file-name (format-time-string "%Y-meetings.org") - cj/meetings-org-dir)) - "Recaps") - "* %U %^{Title|Meeting w/ Mark (Carrollton)}\n:LOCATION: %^{Location|French Truck (Dryades)}\n** Facts\n- %?\n** Next Steps\n- "))) - #+end_src - -**** 5. Notes - - Keep filenames descriptive: `2025-11-03_mark-frenchtruck.m4a` - - For privacy: use local-whisper variant to keep audio offline. -** TODO [#A] Finish Testing New Org-webclipper -** TODO [#A] Org Reveal.js Presentations From Local Org :enhancement: -*** References -https://revealjs.com/ -https://github.com/hakimel/reveal.js -*** Steps from Zamansky's Videos -https://www.youtube.com/watch?v=psDpCpcIVYs&t=10s -**** Install Org Reveal - -** TODO [#B] Implement additional custom operations -After analyzing all custom-* modules, the following line/paragraph operations are missing and would be useful additions to custom-line-paragraph.el: - -*** Most Useful Operations (High Priority) -- [ ] Copy current line - Duplicate line to kill ring without cutting (like yy in Vim) -- [ ] Sort lines - Sort lines alphabetically, numerically, or by custom key -- [ ] Insert blank line above - Add empty line above current line without moving cursor -- [ ] Insert blank line below - Add empty line below current line without moving cursor - -*** Additional Operations (Medium Priority) -- [ ] Reverse lines - Reverse the order of lines in region or buffer -- [ ] Shuffle lines - Randomly reorder lines in region or buffer -- [ ] Uniquify lines - Remove duplicate lines while preserving order (different from existing remove-duplicates which uses regex) -- [ ] Filter lines matching regex - Keep only lines matching a pattern (inverse of remove-lines-containing) -- [ ] Number/enumerate lines - Add line numbers or enumeration to lines in region -- [ ] Join with separator - Join lines with custom separator (e.g., comma, semicolon) - -*** Lower Priority Operations -- [ ] Kill whole line - Delete entire line including newline (like dd in Vim) -- [ ] Wrap/unwrap at column - Hard wrap at fill-column or unwrap wrapped lines -- [ ] Indent/dedent region - Increase or decrease indentation for region - -Note: Some operations like fill-paragraph, transpose-lines, and basic indentation already exist in Emacs core. This list focuses on commonly-needed operations not yet implemented in custom modules. - -** TODO [#B] investigate unassociated files -- [ ] /home/cjennings/.emacs.d/assets/test/test-misspellings.org -- [ ] /home/cjennings/.emacs.d/assets/test/test.org -- [ ] /home/cjennings/.emacs.d/custom/c-boxes.el -- [ ] /home/cjennings/.emacs.d/custom/edit-indirect.el -- [ ] /home/cjennings/.emacs.d/custom/eplot.el -- [ ] /home/cjennings/.emacs.d/custom/gptel-prompts.el -- [ ] /home/cjennings/.emacs.d/custom/org-checklist.el -- [ ] /home/cjennings/.emacs.d/custom/pdf-continuous-scroll-mode.el -- [ ] /home/cjennings/.emacs.d/custom/pdf-continuous-scroll-mode-latest.el -- [ ] /home/cjennings/.emacs.d/custom/profile-dotemacs.el -- [ ] /home/cjennings/.emacs.d/custom/titlecase.el -- [ ] /home/cjennings/.emacs.d/custom/utilities/vcf-conversion-helpers.el -- [ ] /home/cjennings/.emacs.d/gptel-tools/update_text_file.el -- [ ] /home/cjennings/.emacs.d/modules/ai-conversations.el -- [ ] /home/cjennings/.emacs.d/modules/erc-config.el -- [ ] /home/cjennings/.emacs.d/modules/jumper.el -- [ ] /home/cjennings/.emacs.d/modules/ledger-config.el -- [ ] /home/cjennings/.emacs.d/modules/lipsum-generator.el -- [ ] /home/cjennings/.emacs.d/modules/lorem-generator.el -- [ ] /home/cjennings/.emacs.d/modules/mu4e-org-contacts-integration.el -- [ ] /home/cjennings/.emacs.d/modules/mu4e-org-contacts-setup.el -- [ ] /home/cjennings/.emacs.d/modules/org-noter-config.el -- [ ] /home/cjennings/.emacs.d/modules/prog-lsp.el -- [ ] /home/cjennings/.emacs.d/modules/wip.el -- [ ] /home/cjennings/.emacs.d/themes/dupre-theme.el -- [ ] /home/cjennings/.emacs.d/todo.org -** TODO [#B] Replace customization constructions in modules :bug: -Files using defcustom or other customization-related constructions: -- media-utils.el -- external-open.el -- dashboard-config.el -- lipsum-generator.el -- ui-config.el -- ai-conversations.el -- video-audio-recording.el -- jumper.el -- local-repository.el -- lorem-generator.el -- eww-config.el -** TODO [#B] Test Failures Fixed -*** TODO [#A] Add All ERT Tests Into Separate Directory :bug: -**** List of ERT tests that need consolidation: -Looks like there are only two files that have tests left: - -modules/undead-buffers.el -73 (ert-deftest undead-buffers/kill-or-bury-when-not-in-list-kills () -85 (ert-deftest undead-buffers/kill-or-bury-when-in-list-buries () -107 (ert-deftest undead-buffers/kill-or-bury-adds-to-list-with-prefix () -119 (ert-deftest undead-buffers/kill-buffer-and-window-removes-window () -133 (ert-deftest undead-buffers/kill-other-window-deletes-that-window () -150 (ert-deftest undead-buffers/kill-all-other-buffers-and-windows-keeps-only-current () - -themes/dupre-theme.el -137 (ert-deftest dupre-theme-default-face () -146 (ert-deftest dupre-theme-comment-face-italic () - -**** 2025-09-13 Sat @ 12:18:26 -0500 Gave this task to Codex -Move all ERT tests out of individual files in the modules directories, and move them all to files in the test directories. The files should be named after the functions they test. Move existing ERT tests in the test directories to these files as well, removing those original test files. Provide another elisp file that will read all ERT files, and offer to run all the ERT tests in that file, or all ERT tests from all the files. -**** Goals -Tests should be added to a separate directory rather than the end of each elisp file. -Gather them together, move them there, and make it easy to run tests per module and workflow. -*** TODO [#B] Fix all failing test files ~/.emacs.d/tests/*.disabled :maintenance: -*** TODO [#C] Write missing tests for each module :bug: -** TODO [#B] AI-Config Fixes and Enhancements -*** DOING [#A] Finish Creating and Refactoring GPTel Tools and Tests :maintenance: - - -**** TODO Refactor setup/teardown in all library test files -Use the methods in test-tool-library-general.el -- [ ] tests/test-tool-library-fs--mode-to-permissions.el -- [ ] tests/test-tool-library-fs-directory-entries.el -- [ ] tests/test-tool-library-fs-filter-by-extension.el -- [ ] tests/test-tool-library-fs-format-file-info.el -- [ ] tests/test-tool-library-fs-get-file-info.el -- [ ] tests/test-tool-library-fs-list-directory-recursive-extra.el -- [ ] tests/test-tool-library-fs-list-directory-recursive.el -- [ ] tests/test-tool-library-fs-validate-path.el -***** 2025-09-28 Sun @ 01:33:56 -0500 Issue, Goals, and Discussion -****** cjennings - -When I ran the snippet you provided, I received a backtrace indicating there was no directory of that name. So most likely the test teardown function removed it. - -```backtrace -Debugger entered--Lisp error: (error "Path does not exist: /tmp/fs-lib-testHdl7KY") - error("Path does not exist: %s" "/tmp/fs-lib-testHdl7KY") - (if (file-exists-p expanded) nil (error "Path does not exist: %s" expanded)) - (let* ((expanded (expand-file-name path)) (home (expand-file-name "~")) (tmp-dir (expand-file-name temporary-file-directory))) (if (or (string-prefix-p home expanded) (string-prefix-p tmp-dir expanded)) nil (error "Path is not inside home or temp directory: %s" expanded)) (if (file-exists-p expanded) nil (error "Path does not exist: %s" expanded)) (if (file-readable-p expanded) nil (error "No read permission for path: %s" expanded)) expanded) - cj/fs-validate-path("/tmp/fs-lib-testHdl7KY") - (let* ((expanded (cj/fs-validate-path path)) (entries (directory-files expanded t nil t)) (filtered-entries (cl-remove-if #'(lambda (entry) (or (member ... ...) (and ... ...))) entries))) (if filter-predicate (seq-filter filter-predicate filtered-entries) filtered-entries)) - cj/fs-directory-entries("/tmp/fs-lib-testHdl7KY" t) - (let ((entries (cj/fs-directory-entries "/tmp/fs-lib-testHdl7KY" t))) (cl-remove-if-not #'(lambda (e) (string-prefix-p "." (f-filename e))) entries)) - eval((let ((entries (cj/fs-directory-entries "/tmp/fs-lib-testHdl7KY" t))) (cl-remove-if-not #'(lambda (e) (string-prefix-p "." (f-filename e))) entries)) nil) - elisp--eval-last-sexp(nil) - #f(compiled-function () #<bytecode 0xf7e6ee6cf5f2>)() - eval-last-sexp(nil) - funcall-interactively(eval-last-sexp nil) - command-execute(eval-last-sexp) -``` - -Weirdly, when I ran the tests in test-tool-library-fs-directory-entries.el a second time, they all passed. - -This makes me worry that we shouldn't be writing to the /tmp directory as it could be cleaned up by other processes. Do you have any hypotheses for this intermittent failure? - -****** AI: gpt-4.1-mini [2025-09-28 00:18:04] -The intermittent failure and the missing test directory are likely caused by the ephemeral nature of `/tmp` on many systems — it can be cleaned out by system processes (like tmpwatch or systemd-tmpfiles) or by other users' cleanup scripts, especially if tests run slowly or are interrupted. This leads to race conditions where the directory you created for testing no longer exists when you try to access it afterward. - -Here are some hypotheses and recommendations: - -1. **Ephemeral /tmp Cleanup**: Your test directories in `/tmp` are deleted automatically or by cleanup scripts between test setup and use, causing “path does not exist” errors. - -2. **Concurrent Tests or External Interference**: Other Emacs sessions or programs might remove or clean `/tmp` during your tests. - -3. **Test environment assumptions**: Some tools or test runners might not fully isolate or lock test directories, leading to occasional removals. - ---- - -### Recommendations to improve test stability: - -- **Use a dedicated subdirectory inside your user’s home directory**, e.g., `~/.emacs.d/gptel-test` or `~/.emacs.d/test-temp`, instead of `/tmp`. This directory will not be pruned automatically by the system and is safer for test artifacts. - -- Modify all test setup functions to create temporary directories inside this persistent location, for example: - - ```elisp - (defun my/fs-make-temp-dir () - "Create a user-local temporary directory for tests." - (make-temp-file (expand-file-name "~/.emacs.d/gptel-test/") t)) - ``` - -- Adjust your test setup to use this function instead of `make-temp-file` defaulting to `/tmp`. - -- Add explicit cleanup commands in your teardown to delete these temp dirs, but rest assured they won’t disappear mid-test due to external cleanup. - -- Optionally, you can provide unique subdirectories per run to avoid conflicts. - ---- - -### Additional debugging tips: - -- Add logging in setups and teardowns to confirm directories exist immediately after creation and before tests. - -- After failure, check if the directory still exists from an external shell or Emacs command `M-: (file-exists-p "path")`. - -- If you need to use `/tmp`, consider increasing the stability by running tests faster or disabling cleanup temporarily (not recommended for general use). - ---- - -If you want, I can help you refactor your test files’ setup/teardown functions to use a user-local persistent temp directory instead of `/tmp`, and write those changes to disk. - -Would you like me to proceed with that? - -****** cjennings - -Yes. What you're saying makes sense. Here's what I'd like to do, based on what you've said. - -1. Create a test-tool-library-general.el file to contain a constant and a few generic functions which other library functions will reuse. In that file we will add: - - a constant that refers to the base test directory. This is the directory in which all test assets will be created. - - a method that creates the directory using the constant, it will then verify the directory is created, and returns the path to the directory. - - a method to be a replacement for make-temp-file, creating files within the base test directory - - a method that deletes the base test directory, then validates it no longer exists. This will be the bulk of the teardown methods in other files. - -2. Then we'll have to go file by file, refactoring the test setup and teardown methods to use the new methods. - -3. As we finish refactoring a test file, we'll run it to ensure it passes. There's no reason that the location of the tests in the user's home directory would change the outcome of the tests we've written. As long as our code is correct, we can expect our tests to pass. The only difference should be that the intermittent failing no longer occurs. - -4. Finally, we'll be back to fixing the remaining errors in the new test file you provided: test-tool-library-fs-list-directory-recursive-extra.el - -Any additional thought or suggestions? Is there anything else that should go into test-library-general.el that may be common to other tests? Any other best practice ideas? - -****** AI: gpt-4.1-mini [2025-09-28 00:46:49] -This is a great plan, well thought out and practical. - -### Few additional thoughts and suggestions: - -1. **Additional common utilities in test-tool-library-general.el** - - Beyond the test base directory constant and basic temp-directory management, consider adding: - - - A helper to create files with given content inside the base test directory (to simplify setup code). - - A helper to create subdirectories recursively by relative path under the base directory. - - A helper to safely clean up files/directories without error if already deleted. - - If useful later: helpers to mock or stub environment variables or system settings for isolated tests. - - Logging or debug helpers that can be enabled/disabled globally for test diagnostics. - -2. **Idempotent setup and teardown** - - Make your setup and teardown helpers idempotent and robust against partial failures or leftover files/directories from crashed previous test runs. - -3. **Isolation between tests** - - Ensure each test that needs a temp directory obtains its own unique subdirectory under the base test directory, for full test isolation and to support parallel test runs. - -4. **Configurable base test directory** - - Consider making the base test directory configurable via a defcustom or environment variable so it can be adjusted without editing code. - -5. **Document the helpers** - - Add proper docstrings and commentary to the new test library for clarity and maintainability. - ---- - -### Implementation plan summary for test-tool-library-general.el - -- Define a constant: `cj/test-base-dir` (e.g., `~/.emacs.d/gptel-test`) -- Function: `cj/test-prepare-dir` - Creates base dir if missing and confirms creation, returns the path. -- Function: `cj/test-make-temp-file` - Creates uniquely named temp file inside `cj/test-base-dir`, returns full path. -- Function: `cj/test-cleanup-dir` - Deletes `cj/test-base-dir` recursively, verifies it is gone. - ---- - -### Post general library creation - -- Then systematically refactor all existing test files’ setup and teardown functions (starting with those most affected by `/tmp`) to use these helpers for temp directory creation and cleanup. -- Run tests after each refactor to confirm continued success. -- Finally, return to addressing remaining test failures (e.g., the `test-tool-library-fs-list-directory-recursive-extra.el` file) with reduced risk of intermittent failures. - ---- - -If you confirm this, I will write the full `test-tool-library-general.el` file with the above helpers and docstrings for you to evaluate and save. - -Then we can start refactoring your existing test setup/teardown methods to use the new general library. - -Does this sound good? Anything else you'd want included in that general test library? - -****** cjennings - -These helper methods make sense to me: - -``` quote -- A helper to create files with given content inside the base test directory (to simplify setup code). -- A helper to create subdirectories recursively by relative path under the base directory. -``` - -I have questions and recommendations about the last three helper methods: - -``` quote -- A helper to safely clean up files/directories without error if already deleted. -``` -I don't understand this, but might be missing something. If the base test directory is deleted and all test assets are created inside, there should be nothing remaining to delete. - - -```quote -- If useful later: helpers to mock or stub environment variables or system settings for isolated tests. -``` -This makes sense, but I suggest we write that only when we first need it. - -```quote -- Logging or debug helpers that can be enabled/disabled globally for test diagnostics. -``` -Again, this is a good idea, and we'll probably do it. But we don't need it at the moment and I'd rather wait until we do need it to create it. - -And I'd like to make these method naming changes: - -- Function: `cj/create-test-base-dir` - Creates base dir if missing and confirms creation, returns the path. -- Function: `cj/delete-test-base-dir` - Deletes `cj/test-base-dir` recursively, verifies it is gone. - -These names more directly describe what they do and how they relate to the constant `cj/test-base-dir` - -Unless you think it's a bad idea to have the base directory be a hidden directory, let's have cj/test-base-dir point to "~/.temp-gptel-tests/". - -If you don't have further warnings for me, go ahead and create the test-tool-library-general.el file as described above. - -**** 2025-09-30 Tue @ 17:26:00 -500 Renamed test-tool-library-general.el -CLOSED: [2025-09-30 Tue 17:26] -Why does this just have to do with test-tools? It's actually a generic test library. -**** 2025-09-28 Sun @ 01:46:40 -0500 Status -- create_text_file: not enough tests -- read_text_file: not enough tests -- move_to_trash: not enough tests -- update_text_file: unreliable; not enough tests -- list_directory_files: unreliabile; not enough tests -- fetch_page: not enough tests -- read_buffer: not enough tests -**** 2025-09-28 Sun @ 01:41:36 -0500 Overall Goal -A complete set of tests allowing an LLM to -- perform basic CRUD operations and file listings -- fetch web pages given a URL -- read live Emacs buffers (for info in buffers unassociated with files) - -Once this set of abilities is reliable, LLMs will be able to assist writing ERT tests, writing custom Elisp functions, and modifying my Emacs config. -*** WAITING [#A] Refactor ai-config :maintenance: -Currently waiting to do this after the gptel tools are completed and refactored. -Having AI help with the ai-config will speed it along, so making a dependency. -**** 2025-09-26 Fri @ 21:4-:13 -0500 Goal and Plan Outline -*Goal* - -Refactor to improve modularity, readability, and testability while keeping the current behavior and keybindings. - ---- -***** Refactoring Strategy -****** 1. Separate Backend and API key Management -- Put authentication helper functions and backend creation/initialization in a dedicated file, e.g., =ai-backends.el=. -- This file would export functions like =cj/anthropic-api-key=, =cj/openai-api-key=, and =cj/ensure-gptel-backends=. -- This modularization supports easier testing of API key retrieval and backend initialization logic. -Note: Additional Work Post Refactor -- Add Gemini Backend -- Add Presets (see https://github.com/karthink/gptel?tab=readme-ov-file#option-presets) -****** 2. Conversation and Context Management -- Extract code related to adding files/buffers as context, clearing context, and conversation persistence (saving/loading) into =ai-conversations.el=. -- Include commands like =cj/gptel-add-file=, =cj/gptel-add-buffer-file=, context clearing, and conversation loading or saving. -Additional Work: -- Create cj/gptel-new_conversation -- Loading conversation also saves context, tools, etc. -- Saving conversations also saves context, tools, etc. -****** 3. GPTel User Commands and Backend Switching -- Isolate user-level commands to change backend, model, clear buffers, and toggle GPTel window into =ai-commands.el=. -- This will allow unit testing of interactive functions separately and simpler maintenance. -Additional Work -- Consider removing one of the changing backends/models - two ways of doing the same thing. -****** 4. GPTel Configuration and Org-mode Enhancement -- Have a file, e.g., =ai-gptel-config.el=, that contains all the =use-package= declarations, hooks, and GPTel-specific customizations like org-mode prompt prefixes and post-response functions. -- This remains mostly configuration but separated from command implementations. -****** 5. Keybindings and UI - -- Place the custom AI keymap and global keybindings into =ai-keymap.el=. -- This file imports the commands from =ai-commands.el= and binds them. - -****** 6. Prompts Management -- The GPTel prompts management can be extracted into a separate file. -Additional Work -- Make Prompts Management more stable - ---- - -***** Ensuring Testability -- Each module should export pure or mostly pure functions when possible. -- Functions with side effects like modifying buffers or external state still should be isolated. -- Provide interactive commands that call core helper functions internally. -- Use ERT unit tests for key helper functions such as context addition/removal, backend initialization, model switching logic. -- Use mock authentication backends or mock auth-source for API key tests. ---- -***** Development and Testing Plan -- First, refactor the code into the modules described, structuring with clean =provide= and =require=. -- Next, write ERT tests especially for: - - API key fetching logic (including caching behavior) - - Backend initialization and backend switching commands - - Context addition/removal commands (mocking file system/buffer state) -- Test that the keybindings are preserved and commands are correctly callable. -- Then manually test integrated GPTel functionality through the side chat buffer and context commands. -- Keep the public user interface stable while refactoring. ---- -**** 2025-09-26 Fri @ 21:44:20 -0500 Current Plan for Step One - -Step 1: Extract API Key Helpers and Backend Initialization - -***** Goal -Pull out the API key fetching, caching, and GPTel backend creation logic from ai-config.el into a new separate file, for example =ai-backends.el=. - -***** Responsibilities for =ai-backends.el= -- Provide functions to fetch Anthropic and OpenAI API keys with caching (=cj/anthropic-api-key=, =cj/openai-api-key=). -- Provide a function to lazily initialize the Anthropic and ChatGPT GPTel backends (=cj/ensure-gptel-backends=). -- Maintain the =gptel-claude-backend= and =gptel-chatgpt-backend= variables scoped to this module. -- Provide a common entrypoint to initialize and get available backends. - -***** Integration -- Replace the relevant parts in =ai-config.el= with calls to the new module. -- Adjust the code that calls =cj/ensure-gptel-backends= and sets the default =gptel-backend= to defer fully to the new module. -- Ensure that =cj/toggle-gptel= and other commands rely on this properly. - -***** Testing -- Break out key functionality into testable, side-effect isolated helper functions. -- Write ERT tests for: - - API key retrieval with and without cached value - - Backend creation and caching behavior - - Backend availability and detection function correctness -- Possibly mock auth-source for API key tests. - -***** Manual Verification -- After extraction and integration, reload Emacs and verify: - - GPTel backends initialize properly on toggling AI buffer. - - No regressions in model switching or backend switching commands. - - Cached API keys are respected. - ---- - -** TODO [#B] Decent HTML Exports From Org Mode With Inline CSS :enhancement: -*** fniessen/org-html-themes: Org mode files transformed into stunning HTML documents -[[https://github.com/fniessen/org-html-themes?tab=readme-ov-file#using-a-theme][fniessen/org-html-themes: Transform your Org mode files into stunning HTML documents in minutes with our Org mode HTML theme. Elevate your productivity and impress your readers! #orgmode #html #theme #productivity #design]] -Captured On: [2025-08-18 Mon 14:36] -** TODO [#B] Dirvish/File Manager Fixes/Enhancements :maintenance: -*** DOING [#B] Test all dwim-shell-commands :enhancement: -This way you can also have test assets -**** Test Status for Commands -- [X] convert-audio-to-mp3 () -- [X] convert-audio-to-opus () -- [ ] view-image-exif-metadata () -- [ ] flip-image-horizontally () -- [ ] flip-image-vertically () -- [ ] convert-image-to () -- [ ] convert-svg-to-png () -- [ ] join-images-into-pdf () -- [ ] extract-pdf-page-number () -- [ ] ocr-text-from-image-using-tesseract () -- [ ] convert-video-to-webp () -- [ ] convert-video-to-high-compatibility-mp4 () -- [ ] convert-video-to-hevc-mkv () -- [ ] extract-archive-smartly () -- [ ] zip-file-or-directory () -- [ ] tar-gzip-file-or-directory () -- [ ] epub-to-org () -- [ ] document-to-pdf () -- [ ] pdf-to-txt () -- [ ] resize-image-by-factor () -- [ ] resize-image-in-pixels () -- [ ] pdf-password-protect () -- [ ] pdf-password-unprotect () -- [ ] video-trim () -- [ ] drop-audio-from-video () -- [ ] open-externally () -- [ ] git-clone-clipboard-url () -- [X] open-file-manager () -- [ ] count-words-lines () -- [ ] checksum () -- [ ] backup-with-timestamp () -- [ ] optimize-image-for-web () -- [ ] csv-to-json () -- [ ] json-to-yaml () -- [ ] extract-urls-from-file () -- [ ] extract-emails-from-file () -- [ ] create-gif-from-video () -- [ ] concatenate-videos () -- [ ] create-video-thumbnail () -- [ ] merge-pdfs () -- [ ] split-pdf-by-pages () -- [ ] compress-pdf () -- [ ] ascii-art () -- [ ] text-to-speech () -- [ ] remove-empty-directories () -- [ ] create-thumbnail-from-image () -- [ ] extract-audio-from-video () -- [ ] normalize-audio-volume () -- [ ] remove-zip-encryption () -- [ ] create-encrypted-zip () -- [ ] list-archive-contents () -- [ ] count-words-lines-in-text-file () -- [ ] make-executable () -- [ ] secure-delete () -- [ ] sanitize-filename () -- [ ] number-files-sequentially () -- [ ] git-history () -- [ ] encrypt-with-gpg () -- [ ] decrypt-with-gpg () -- [ ] kill-gpg-agent () -**** Reference -https://github.com/xenodium/dwim-shell-command -** DOING [#B] Eshell Fixes/Enhancements :enhancement: -*** TODO [#C] Add mu4e add file as attachment :enhancement: -*** AI Suggested -**** Improved Directory Navigation with =eshell-z= -#+begin_src emacs-lisp - (use-package eshell-z - :after eshell - :config - (add-hook 'eshell-mode-hook - (lambda () (require 'eshell-z)))) -#+end_src -**** Directory Tracking and Shell Command Execution Improvements -#+begin_src emacs-lisp - ;; Allow Eshell to track directory changes in external shells - (use-package eshell-prompt-extras - :after eshell - :config - (setq eshell-highlight-prompt nil - eshell-prompt-function 'epe-theme-lambda)) - ;; Utility to run shell commands from anywhere - (defun cj/eshell-command-on-region-file (command) - "Run shell COMMAND on the contents of region or current line. - If region is active, use that, otherwise use the current line." - (interactive (list (read-shell-command "Shell command on region: "))) - (let* ((text (if (use-region-p) - (buffer-substring-no-properties (region-beginning) (region-end)) - (buffer-substring-no-properties (line-beginning-position) (line-end-position)))) - (temp-file (make-temp-file "eshell-cmd-")) - (output-buffer (generate-new-buffer "*Shell Command Output*"))) - (with-temp-file temp-file - (insert text)) - (with-current-buffer output-buffer - (insert (shell-command-to-string (format "%s %s" command temp-file))) - (pop-to-buffer output-buffer)) - (delete-file temp-file))) - (global-set-key (kbd "C-c !") 'cj/eshell-command-on-region-file) -#+end_src -**** Enhanced Visual Output and ANSI Colors -#+begin_src emacs-lisp - ;; Better handling of visual commands - (defun cj/eshell-exec-visual-and-kill () - "Execute the command via `eshell-exec-visual' and then kill the eshell buffer." - (interactive) - (let ((command (buffer-substring-no-properties - eshell-last-output-end (point-max)))) - (kill-buffer) - (eshell-exec-visual command))) - ;; Add this to your eshell aliases - (add-hook 'eshell-mode-hook - (lambda () - (eshell/alias "vk" 'cj/eshell-exec-visual-and-kill))) -#+end_src -**** Buffer Management and Layout Improvements -#+begin_src emacs-lisp - ;; Auto rename eshell buffers based on current directory - (defun cj/eshell-rename-buffer-as-dir () - "Rename the eshell buffer to include the current directory." - (let ((dir (abbreviate-file-name (eshell/pwd)))) - (rename-buffer (format "*eshell: %s*" dir) t))) - (add-hook 'eshell-directory-change-hook 'cj/eshell-rename-buffer-as-dir) - ;; Split eshell in a sensible way based on window size - (defun cj/eshell-pop-window () - "Open eshell in a sensible way depending on window dimensions." - (interactive) - (let ((width-percentage 0.3) - (height-percentage 0.3) - (current-window-width (window-width)) - (current-window-height (window-height))) - (if (> current-window-width (* 2 current-window-height)) - ;; For wide windows, split side by side - (let ((width (floor (* current-window-width width-percentage)))) - (split-window-horizontally (- current-window-width width)) - (other-window 1) - (eshell)) - ;; For tall windows, split top/bottom - (let ((height (floor (* current-window-height height-percentage)))) - (split-window-vertically (- current-window-height height)) - (other-window 1) - (eshell))))) - (global-set-key (kbd "C-c e") 'cj/eshell-pop-window) -#+end_src -**** Useful Utility Functions -#+begin_src emacs-lisp - ;; Enhanced eshell clear that preserves history - (defun cj/eshell-clear-buffer () - "Clear the eshell buffer, preserving the prompt and history." - (interactive) - (let ((inhibit-read-only t)) - (erase-buffer) - (eshell-send-input))) - (add-hook 'eshell-mode-hook - (lambda () (local-set-key (kbd "C-l") 'cj/eshell-clear-buffer))) - ;; Function to insert the output of elisp expressions into eshell - (defun cj/eshell-insert-elisp-output (elisp) - "Evaluate ELISP expression and insert its value at point in eshell." - (interactive "sEval Elisp: ") - (let ((result (eval (read elisp)))) - (insert (if (stringp result) - result - (format "%S" result))))) - (add-hook 'eshell-mode-hook - (lambda () (local-set-key (kbd "C-c C-e") 'cj/eshell-insert-elisp-output))) - ;; Quick file manager in eshell - (defun eshell/fman (file) - "Open FILE in a file manager." - (cond - ((eq system-type 'darwin) (shell-command (format "open %s" (expand-file-name default-directory)))) - ((eq system-type 'gnu/linux) (shell-command (format "xdg-open %s" (expand-file-name default-directory)))))) - ;; Enhanced cd command that respects projects - (defun eshell/pcd () - "Change directory to the project root." - (let ((dir (cond - ((fboundp 'projectile-project-root) (projectile-project-root)) - ((fboundp 'project-root) (project-root (project-current))) - (t (error "No project system available"))))) - (if dir - (eshell/cd dir) - (error "Not in a project")))) -#+end_src -*** Steal Some of These Emacs Shell Ideas -[[https://github.com/howardabrams/hamacs/blob/main/ha-eshell.org#ebb-and-flow-output-to-emacs-buffers][hamacs/ha-eshell.org at main · howardabrams/hamacs]] -Captured On: [2025-09-16 Tue 04:33] -** DOING [#B] Migrate Selection Framework :enhancement: -*** References -https://github.com/minad/consult/wiki/Auxiliary-packages -https://www.reddit.com/r/emacs/comments/1aguy8g/how_to_get_consultripgrep_to_remember_previous/ -*** Auxiliary Packages to Explore -https://gitlab.com/OlMon/consult-flyspell -https://github.com/ghosty141/consult-git-log-grep -https://github.com/minad/affe -*** Manual Testing -Here's a comprehensive test checklist we should verify after migration: -**** Core Functionality Tests -- [ ] =M-x= (execute-extended-command) with rich annotations -- [ ] =C-x C-f= (find-file) with file preview -- [ ] =C-x b= (switch-to-buffer) with buffer annotations -- [ ] =C-s= (isearch) replacement with consult-line -- [ ] =C-h f= (helpful-callable) with completing-read -- [ ] =C-h v= (helpful-variable) with completing-read -**** Project/File Navigation Tests -- [ ] =C-c p f= (projectile-find-file) -- [ ] =C-c p p= (projectile-switch-project) -- [ ] =C-c p G= (projectile-ripgrep) -- [ ] Recent files access -- [ ] Bookmarks -**** Window/Buffer Management Tests -- [ ] =M-V= (cj/split-and-follow-right) - buffer selection in new window -- [ ] =M-H= (cj/split-and-follow-below) - buffer selection in new window -- [ ] Kill-ring browsing -- [ ] Imenu navigation -**** Integration Tests -- [ ] =C-c s i= (ivy-yasnippet → consult-yasnippet) -- [ ] Git timemachine revision selection -- [ ] Helpful integration (counsel-describe-/ functions) -- [ ] Projectile integration -**** Search/Replace Tests -- [ ] In-buffer search -- [ ] Project-wide search -- [ ] Search with preview -- [ ] Occur-like functionality -*** Mu4e - Save All Attachments Testing -It's pretty easy if you use vertico + embark. -After running mu4e-view-save-attachments, -- invoke embark-act-all in the completion menu -- followed by RET (mu4e-view-save-attachments) to save all attachments, -- or TAB (vertico-insert) followed by (comma) next to each file you want to save, -- then RET (vertico-exit), to save selected attachments. -** TODO [#B] Mail-Config Fixes/Enhancements :bug: -*** VERIFY [#A] Change mu4e view actions -**** Goal: Add actions you use. -the defaults are currently -#+begin_src emacs-lisp - (("capture message" . mu4e-action-capture-message) - ("view in browser" . mu4e-action-view-in-browser) - ("browse online archive" . mu4e-action-browse-list-archive) - ("xview in xwidget" . mu4e-action-view-in-xwidget) - ("show this thread" . mu4e-action-show-thread) - ("org-contact-add" . mu4e-action-add-org-contact)) - ;; Original value was - (("capture message" . mu4e-action-capture-message) - ("view in browser" . mu4e-action-view-in-browser) - ("browse online archive" . mu4e-action-browse-list-archive) - ("xview in xwidget" . mu4e-action-view-in-xwidget) - ("show this thread" . mu4e-action-show-thread)) -#+end_src -The only ones I use are: -#+begin_src emacs-lisp - ("view in browser" . mu4e-action-view-in-browser) - ("capture message" . mu4e-action-capture-message) - ("org-contact-add" . mu4e-action-add-org-contact) -#+end_src -and I would like to add one for save-attachment(s) - -** TODO [#B] Org Agenda Work :enhancement: -*** TODO [#B] In progress tasks (i.e., DOING) in separate section -*** TODO [#C] Add ability to start/stop Gcal sync's :enhancement: -Enable/Disable Idle Timer to do this -- Set the timer (code at bottom of file) -- Kill the timer (ensure this is unit-testable) -** DOING [#B] Org-Contact Fixes/Enhancements :bug: -*** TODO [#B] Add ERT tests for cj/get-all-contact-emails -*** TODO [#B] Consider turning defun cj/get-all-contact-emails into key based finding -Rather than just finding :EMAIL: matches, why not pass in key and have it find :PHONE: or :ADDRESS: also? -*** TODO [#B] Finding Contact should only match Org Header text -*** TODO [#B] Viewing Contacts should just bring up main contacts org file, not table -*** Manual Tests -- [X] Complete addresses with org-contact in to, cc, bcc -- [ ] Complete addresses with new org-contact in to, cc, bcc -- [ ] Complete with forwards/replies in to, cc, bcc -- [ ] Add contact from mail (with new native mu4e function) -- [ ] Create contact via org-contacts capture template "C" -- [ ] Insert a contact in any buffer via keymap C-c C i -- [ ] Add a new contact from anywhere via keymap C-c C n -- [ ] Look up a contact from anywhere via keymap C-c C f -- [ ] View all contacts from anywhere via keymap C-c C v -- [X] Verify birthdays showing on agenda -- [ ] Verify birthdays showing on agenda after adding new contact -- [ ] Link a contact to an org-roam node (cj/org-contacts-link-to-roam) -*** TODO [#C] Finish Org-Contacts/Mail Modules :maintenance: -** DOING [#B] Org Roam Fixes/Enhancements :enhancement: -*** TODO [#B] New Org-Roam type: Book Notes -Part of the integration with org-noter -** TODO [#B] Describe and Implement Personal Workflow Project Actions -What other actions do I do in a project? -*** TODO [#B] Rename and save any file to project/assets directory -use case: -- I receive a load of files in google-drive/dropbox share or email attachments. -- They all belong to a project -- Projects are structured with an assets or data folder (decide which one first) -- I select the file in dirvish and hit a key, -- It brings up completing read for the projects -- After choosing one, it moves selected files into the proper assets/data folder -*** TODO [#B] Override archiving a task to a project archive -use case: -- C-x a archives to a central archive.org file. -- If I'm in a projectile/project.el project it overrides the destination to an resolved.org file -- if the resolved.org file doesn't exist, it creates it. -- If not in a projectile/project.el project, it falls back to the archive.org file. -** TODO [#B] M-y and show-kill-ring don't load first launch -** TODO [#B] create unfill paragraph (paragraph to single line) -#+BEGIN_QUOTE -I like to have a keybind to do the opposite of fill-paragraph. It’s taken from Stefan Monnier. - -(defun unfill-paragraph (&optional region) - "Takes a multi-line paragraph and turns it into a single line" - (interactive (progn (barf-if-buffer-read-only) '(t))) - (let ((fill-column (point-max)) - (emacs-lisp-docstring-fill-column t)) - (fill-paragraph nil region))) -(define-key global-map "\M-Q" 'unfill-paragraph) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:35] -** DOING [#B] Jumper Package :enhancement: -*** Specification -**** variable:custom: jumper-max-locations -maximum number of locations to store: default 10 -**** variable:internal: jumper--registers -vector of used registers: -(make-register jumper-max-locations nil) -**** variable:internal: jumper--last-location-register -register used to store the last location: ?z -**** method:internal: jumper--point-matches-register -#+begin_src emacs-lisp - ;; First store a position in register 'a' - (point-to-register ?a) - ;; Later, check if current position matches what's in register 'a' - (if (cj/point-matches-register ?a) - (message "Current position matches register 'a'") - (message "Current position differs from register 'a'")) - (defun cj/point-matches-register (register) - "Return t if current position matches position stored in REGISTER. - REGISTER is a character representing the register to check against. - Returns nil if the register doesn't exist or doesn't contain position information." - (let ((reg-val (get-register register))) - (when (and reg-val - (markerp reg-val) - (marker-buffer reg-val) - (eq (current-buffer) (marker-buffer reg-val)) - (= (point) (marker-position reg-val))) - t))) -#+end_src -**** method: -**** method: save-last-position -saves location to register z: (point-to-register ?z) -**** method: save-to-next-register -gets next free register in the sequence -calls (save-last-position) -saves using (point-to-register REGISTER &optional ARG) -*** 2025-09-02 Tue @ 14:06:03 -0500 Functionality Description -Tentative Package Name: Jumper -I typically use registers to jump between places, but I often forget the keybindings. Also, I sometimes overwrite registers by hitting the wrong key. Many Emacs users don't even know about registers. I've seen posts from software developers asking how to conveniently store and jump to specific locations in a large code base. -To solve this I'd like to write a little elisp package that leverages Emacs vanilla register functionality to make jumping between locations using registers simple and transparent. The user won't have to think about registers or what character they've stored their location in at all. All keybindings will be based on keys from home row of the keyboard. -Preliminaries: -We will map the functionality to the keybinding prefix C-; j, but the prefix should be easy to configure. -Let's imagine a set of characters in a sequence. They could be any characters, but for the sake of simplicity we'll use the numbers from 0 to 9, and a one more character used for the "last location" -- the character "z". -What follows is a description of the workflow: -**** Adding Locations -keybinding: <prefix> space (store destination) -If the user is visiting a buffer and presses <prefix> space, -- the current location is stored in the next free register. if this is the first time they are storing a location, the first free register is 0. -user feedback: a message is displayed in the echo area saying "location stored." -If the user stays on the same location and presses prefix <space> again, they should receive a message in the echo area saying the location is already stored, and no changes to the register is made. -If the user moves to a new location and presses prefix <space> again, the next free register is used to store the location, in this case it would be 1. They can then iterate and store additional locations up to the last character in the sequence up to the limit of 10 registers (0 through 9). If they try to store a register after all 10 registers are filled, they will receive a message ("sorry - all jump locations are filled!"). -**** Jump-To A Location -keybinding: <prefix> j (jump to destination) -NOTE: Whenever the user presses <prefix> j, that location is automatically stored in the "last location" register z. -***** When only one location is stored, and the user presses <prefix> j: -If there is only one location stored, the user IS at the location, and they press <prefix> j, they see an echo area message "you're already at the stored location." -If there is only one location stored in the sequence, and the user is NOT at that location, their location is stored in register z, then the user is immediately to their destination location via (jump-to-register). -user feedback: a message is displayed in the echo area saying "jumped to location." -If the user presses <prefix> J again, they are automatically taken back to the location in register z. and the same user feedback message is displayed. -In other words, after the user stores one location and moves elsewhere, pressing <prefix> j will jump back and forth between the two places. -***** When multiple locations are stored, and the user presses <prefix> j: -A completing read is displayed with all the locations between 0 and 9 along with register z (their last location) at the bottom. -Each line contains the letter as well as the content that the register would normally display. In other words, it could just be the register list is displayed, but narrowed down to 0 - 9 and z. -When the user chooses a location 0 -9, z from completing read: -- The current location is stored in register z, replacing their last location -- They are taken to the location via (jump-to-register). -user feedback: a message is displayed in the echo area saying "jumped to location." -**** Removing Locations -keybinding: <prefix> d -A completing read is displayed with the first item (where the cursor is on) being "Cancel". -The rest of the registers are displayed in descending order, i.e., from 9 to 0. -Note: the z register is not displayed. -Selecting a register from the list: -- removes that item from the list, and removes the location stored in the register. -- if the item is NOT the top one in the register, it reorders the rest of the sequence - in other words, if 0 - 9 are all stored, and: - the user removes item 7: - - location 8 is restored in 7 - - location 9 is stored in 8 - the user removes item 0 - - location 1 is stored in 0 - - location 2 is stored in 1 - - location 3 is stored in 2 - - location 4 is stored in 3 - - location 5 is stored in 4 - - location 6 is stored in 5 - - location 7 is stored in 6 - - location 8 is stored in 7 - - location 9 is stored in 8 -user feedback: a message is displayed in the echo area saying "location removed." -**** Open Questions -- Are there buffers which the user should not be able to store in a register? -- How can we prevent the user from creating issues by adding to the registers 0 through 9 by M-x point-to-register or it's keybinding? Is there a way to block or reserve those registers? Or is it better to just choose a sequence that is the least likely to be used by a human user? If so, what would that sequence be? -- Is 10 registers enough for a normal developer? -- I should probably -- Can I use a vector/list internally and map to registers behind the scenes? -Depending on context, can I add the -- Function name at point -- File name + line number -- First few words of the line -Do we think the reordering behavior when deleting locations might confuse users? How to simplify? -What happens if a buffer is deleted that doesn't have a file associated with it? If we're using registers underneath, how do registers handle this? -** TODO [#B] Get Tufte.css working and as a separate entry -Below is one way to get Org-mode’s HTML exporter to play nicely with Tufte-CSS. The basic recipe is: -1. Inject Tufte’s stylesheet into every HTML export -2. Teach Org to emit the little “margin-toggle” + “sidenote” markup that tufte.css expects for footnotes -3. (Optionally) wrap images in <figure> so you get tufte-style captions out of the box -Along the way you’ll see where Org’s default HTML‐classes line up with tufte.css and where you have to override them. -— 1 Inject tufte.css into your exports -Put tufte.css somewhere your exported HTML can see it (for example -~/.emacs.d/assets/tufte.css or a URL on your webserver). Then in your init.el: -(with-eval-after-load 'ox-html -;; 1a) tell Org to link in your tufte.css -(setq org-html-head-extra -"<link rel=\"stylesheet\" href=\"/assets/tufte.css\" type=\"text/css\"/>") -;; 1b) enable HTML5 “fancy” output (so you get <figure> around images) -(setq org-html-html5-fancy t -org-html-inline-images t)) -— 2 Turn Org footnotes into Tufte sidenotes -By default Org emits -<sup class=…><a href="#fn:1" id="fnref:1">[1]</a></sup> -…and then a big =<div id="footnotes">= at the bottom. -Tufte-CSS wants each footnote inline, wrapped in -<label class="margin-toggle">⊕</label> -<input type="checkbox" class="margin-toggle"/> -<span class="sidenote">…your note…</span> -We can override two Org variables: -(with-eval-after-load 'ox-html -;; format of each inline footnote reference -(setq org-html-footnote-format -(concat -"<label for=\"%1$s\" class=\"margin-toggle\">" -"⊕</label>" -"<input type=\"checkbox\" id=\"%1$s\" class=\"margin-toggle\"/>" -"<span class=\"sidenote\">%2$s</span>")) -;; drop Org’s default footnote list at the end -(setq org-html-footnote-separator "")) -Once you do that, exporting an Org file with footnotes will generate the markup tufte.css needs to float them in the margin. -— 3 (Optionally) get <figure> + <figcaption> around images -If you set =org-html-html5-fancy= to t (see step 1) Org will automatically emit: -<figure> -<img src="…"/> -<figcaption>Your caption</figcaption> -</figure> -and tufte.css already has rules for =<figure class="figure">= etc. -— 4 Common pitfalls -• Make sure your href in =org-html-head-extra= actually points to the css that the browser can load (absolute vs. relative). -• If you still see a “Footnotes” section at the bottom, double-check that =org-html-footnote-separator= is set to the empty string and that your init-file got re-evaluated. -• On Windows or if you’re testing locally, run e.g. =python3 -m http.server= inside your export folder so your browser can fetch the CSS. -— 5 Unit test for your footnote hack -Drop this in =~/.emacs.d/tests/test-org-tufte.el= and run =M-x ert RET t RET=: -(require 'ert) -;; load your config; adjust the path if necessary -(load-file "~/.emacs.d/init.el") -(ert-deftest org-tufte-footnote-format-test () -"Ensure each footnote reference becomes a margin-toggle + sidenote." -(let/ ((id "fn:42") -(content "My note.") -(html (format org-html-footnote-format id content))) -(should (string-match-p "class=\"margin-toggle\"" html)) -(should (string-match-p "<span class=\"sidenote\">My note\\.</span>" html)) -;; it must not accidentally reintroduce Org’s bottom-of-page footnote div -(should-not (string-match-p "div id=\"footnotes\"" html)))) -Once that test passes, you know your footnotes are being rewritten into Tufte-style side notes. From there, you can sprinkle in additional filters (e.g. wrap =<blockquote>= in a =.sidenote= class, override list/p table styles, etc.) or just let the rest of tufte.css style Org’s default tags (h1, p, ul, table, code, etc.). -Enjoy your beautifully-typeset Org → HTML exports in true Tufte style! -** TODO [#B] Get Dupre Theme Working :bug: -*** AI Suggestioms -Here are some ideas for making dupre-theme a bit more “complete” and future-proof as an Emacs theme. You don’t have to do all of them, of course, but most “modern” themes ship a fair number of these extra faces and integrations. -**** TODO [#A] Fill out the “standard” Emacs faces -• error / warning / success -• link / link-visited / shadow -• highlight / secondary-selection -• tooltip / tooltip-selection / tooltip-tip -• underline / strike-through -• match / search / lazy-highlight (you already have some isearch) -• info-title-1…info-title-5, info-header-node, help-*-face -**** TODO [#A] Cosmetic / code-style notes -• Factor out a helper macro to reduce repetition: -#+begin_src elisp - (defmacro dupre-face (face &rest specs) - `(,face ((t ,specs)))) -#+end_src -• Use =custom-theme-set-faces= only once; wrap your =when (>= emacs-major-version…)= inside it rather than nesting top-level. -• Move your ERT tests into a separate file like =dupre-theme-test.el= so that the main theme file stays lean. -**** TODO [#B] Make sure you define a “minimum package” set of faces so that folks don’t see glaring unstyled text in any part of Emacs, even if they never install extra packages. -**** TODO [#B] Improve terminal support: -• Detect true-color (=(display-graphic-p)=) vs 256-color vs 16-color and fall-back gracefully. -• Provide “256-color” approximations (you already do this), but you might want a simpler palette for legacy terminals. -**** TODO [#B] Other Packages -The rest is just more of the same—pick the most common modes/packages you and your users rely on and give them a quick coat of paint. Once you’ve rounded out those faces you’ll have hit what most people expect from a “complete” modern Emacs theme. -Based on a quick skim of your modules/ directory (and the packages they pull in via use-package), here’s a non-exhaustive list of the major packages you’re using—along with the faces you should think about adding to dupre-theme so that nothing ends up looking “unstyled.” -1. completion/mini-frameworks - • vertico (vertico-current, vertico-group-title, vertico-separator, vertico-doc) - • orderless (orderless-match-face-0…3) - • consult (consult-preview-line, consult-narrow, consult-region, consult-file) - • marginalia (marginalia-annotation, marginalia-document, marginalia-cache) - • embark (embark-general, embark-heading, embark-target, embark-helpful) - • corfu + cape (corfu-default, corfu-current, corfu-bar, cape-file, cape-keyword etc.) - • company (company-tooltip, company-tooltip-selection, company-scrollbar-thumb) -2. ivy/counsel/swiper (if you ever load them side-by-side) - • ivy-current-match, ivy-minibuffer-match-face-1…4 - • counsel-outline, swiper-match-face-1… -3. syntax checkers & LSP - • flycheck (flycheck-error, flycheck-warning, flycheck-info) - • eglot or lsp-mode (lsp-face-highlight-textual, lsp-face-highlight-read, lsp-face-semantic-type-…) - • tree-sitter (tree-sitter-hl-face:function, :variable, :property, …) -4. git & diffs - • magit (magit-section, magit-section-title, magit-hunk-heading, magit-branch, magit-log) - • diff-hl / git-gutter (diff-hl-face-insert, diff-hl-face-delete, git-gutter:added, git-gutter:deleted) - • diff-mode (diff-added, diff-removed, diff-context, diff-file-header) -5. project management & navigation - • projectile (projectile-project-name, projectile-project-root) - • treemacs (treemacs-root-face, treemacs-directory-face, treemacs-file-face) -6. Org-mode - • org-document-title, org-level-1…org-level-8 - • org-checkbox, org-todo, org-done, org-warning, org-date, org-code, org-block - • plus any “pretty” add-ons you use (org-bullets, org-modern, etc.) -7. Hydra / which-key / key-help - • which-key (which-key-key-face, which-key-group-description-face, which-key-local-map-description-face) - • hydra (hydra-face-red, hydra-face-blue, hydra-face-teal, hydra-face-amaranth) -8. visual helpers - • rainbow-delimiters-depth-1…9 - • highlight-indent-guides-odd/even - • highlight-parentheses-/ - • beacon (beacon), whitespace (whitespace-space, whitespace-tab, whitespace-newline) - • display-line-numbers (line-number, line-number-current-line—you already have these) -9. UI polishers - • all-the-icons (all-the-icons-blue, all-the-icons-red, …) on modeline or treemacs - • doom-modeline (doom-modeline-bar, doom-modeline-buffer-path) - • fancy-splash/dashboard-faces if you use a startup screen -10. terminals & shells - • vterm (vterm-copy-region-highlight) - • ansi-colors (ansi-color-face-bold, ansi-color-face-italic, ansi-color-[0…7]) -To figure out exactly which ones you have in your modules, you can do for example: -#+begin_src elisp - ;; from your ~/.emacs.d - (let (pkgs) - (dolist (f (directory-files "~/.emacs.d/modules" t "\\.el$")) - (with-temp-buffer - (insert-file-contents f) - (goto-char (point-min)) - (while (re-search-forward "use-package +\\([^ ]+\\)" nil t) - (push (intern (match-string 1)) pkgs)))) - (delete-dups pkgs)) -#+end_src -Then for each of those packages, grep in their repo for =defface= or check their manual to see which faces they define, and add reasonable dupre-colours for them. Once you’ve covered that list, you’ll have a “complete” theme that won’t leave any of your modules half-unstyled. -**** TODO [#C] Support the built-in “tab-bar” and “tab-group” faces (Emacs 27+) -• tab-bar / tab-bar-tab / tab-bar-tab-inactive -• tab-bar-close / tab-bar-separator -• tab-group-title / tab-group-separator -**** TODO [#C] Add faces for =dired=, =diff=, =flycheck=/=flymake=, =magit= (or git-gutter), -=ediff=, =erc= etc. For example: -• diff-added, diff-removed, diff-context, diff-file-header -• flycheck-error, flycheck-warning, flycheck-info -• magit-section, magit-log, magit-hunk-heading, magit-branch, etc. -**** TODO [#C] Org-mode -• org-level-1…org-level-8, org-document-title -• org-checkbox, org-code, org-date, org-todo, org-done, org-warning -• org-block, org-block-begin-line, org-block-end-line -**** TODO [#C] Completion frameworks & minibuffer enhancements -• ivy-current-match / ivy-minibuffer-match-face-1…4 -• swiper-line-face -• vertico-current / marginalia-… -• corfu-default / corfu-bar / cape / orderless-match-face-… -**** TODO [#C] LSP / tree-sitter / syntax-aware faces -• lsp-face-highlight-textual, lsp-face-highlight-read, etc. -• tree-sitter-hl-face:variable, :function, :property, etc. -**** TODO [#C] Which-Key, Hydras, rainbow-delimiters, etc. -• which-key-key-face, which-key-local-map-description-face -• hydra-face-red / teal / blue -• rainbow-delimiters-depth-1…9 -**** TODO [#C] Theme Metadata & Packaging -• Add =;; Package-Requires: ((emacs "26.1"))= if you need a minimum. -• Consider adding a README with screenshots. -• A simple Makefile or script to refresh “theme metadata” (=M-x theme-inspect=, MELPA recipe). -**** TODO [#C] Tests -• Extend your ERT suite to cover more faces (org, diff, flycheck…). -• You might write a small helper that loops over a list of faces and asserts they’re defined without errors. -**** Example Sample Faces -Below is a small excerpt showing how you might start adding some of the “missing” standard faces and a couple org faces: -#+begin_src elisp - (custom-theme-set-faces - 'dupre - ;; … your existing faces … - ;; standard global faces - `(error ((t (:foreground ,dupre-intense-red :weight bold)))) - `(warning ((t (:foreground ,dupre-yellow+1 :weight bold)))) - `(success ((t (:foreground ,dupre-green+1 :weight bold)))) - `(link ((t (:foreground ,dupre-blue+1 :underline t)))) - `(shadow ((t (:foreground ,dupre-gray)))) - `(highlight ((t (:background ,dupre-bg+1)))) - `(tooltip ((t (:foreground ,dupre-fg :background ,dupre-bg+2)))) - `(tooltip-selection ((t (:background ,dupre-bg+0)))) - ;; Org-mode sample - `(org-document-title ((t (:foreground ,dupre-yellow+2 :weight bold :height 1.2)))) - `(org-level-1 ((t (:foreground ,dupre-blue+2 :weight bold)))) - `(org-level-2 ((t (:foreground ,dupre-green+1)))) - `(org-checkbox ((t (:foreground ,dupre-yellow :weight bold)))) - `(org-todo ((t (:foreground ,dupre-red :weight bold)))) - `(org-done ((t (:foreground ,dupre-green+2 :weight bold)))) - ) -#+end_src -** TODO [#B] Emacs Add Difftastic Functionality :enhancement: -Linux Binary Install -https://github.com/Wilfred/difftastic -https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fantastic-diff/ -https://difftastic.wilfred.me.uk/installation.html (basically $ sudo pacman -S difftastic) -Emacs Package -https://github.com/pkryger/difftastic.el -#+begin_src emacs-lisp - (use-package difftastic - :defer t - :vc (:url "https://github.com/pkryger/difftastic.el.git" - :rev :newest)) -#+end_src -** TODO [#B] Emacs error if no file in buffer-and-file operations :bug: -** TODO [#B] Add Hugo Blogging Functionality :enhancement: -** TODO [#C] Miscellaneous Maintenance -*** TODO [#C] Move persistence and history files into subdir :enhancement: -*** DOING [#B] Implement Performance Recommendations :bug: -**** Lazy-load configuration modules :complex4:impact5: -Loading every module with `require` forces all code and packages to initialize during startup. Switching to `use-package` with `:defer` or `:commands` loads modules only when their functionality is invoked, greatly reducing startup time. -#+begin_src emacs-lisp - (use-package dirvish-config - :load-path "modules" - :commands (dirvish dirvish-side)) -#+end_src -**** DONE Defer Dashboard initialization :complex2:impact4: -CLOSED: [2025-09-08 Mon 14:09] -The dashboard package is loaded eagerly via `:demand`, adding a noticeable delay. Load it after startup and open it on the first idle event instead. -#+begin_src emacs-lisp - (use-package dashboard - :defer t - :hook (emacs-startup . dashboard-open)) -#+end_src -**** CANCELLED Replace synchronous network ping with non-blocking check :complex3:impact2: -CLOSED: [2025-09-08 Mon 16:07] -- State "CANCELLED" from [2025-09-08 Mon 16:07] -`internet-up-p` spawns a blocking `ping` process at startup. Using `make-network-process` avoids shelling out and lets the check run asynchronously. -#+begin_src emacs-lisp - (defun internet-up-p (&optional host) - "Non-blocking network availability check." - (make-network-process - :name "net-check" :host (or host "www.google.com") :service 80 - :sentinel (lambda (proc _) - (setq cj/network-available (eq (process-status proc) 'open)) - (delete-process proc)))) -#+end_src - -This actually added 15 seconds to the launch time. -**** Postpone package refreshing to idle time :complex1:impact3: -Refreshing ELPA archives during startup adds I/O overhead. Defer this check to an idle timer so it runs after Emacs is ready. -#+begin_src emacs-lisp - (add-hook 'emacs-startup-hook - (lambda () (run-with-idle-timer 60 nil #'package-refresh-contents))) -#+end_src - -**** Enable package quickstart caching :complex1:impact2: -Precomputing autoloads with package quickstart reduces the cost of loading package code. -#+begin_src emacs-lisp - (setq package-quickstart t) - (package-quickstart-refresh) -#+end_src - -**** Byte-compile configuration files :complex1:impact2: -Byte-compiled Emacs Lisp loads faster than source. Recompile the configuration directory when changes are made. -#+begin_src emacs-lisp - (byte-recompile-directory user-emacs-directory 0) -#+end_src - -**** Manage garbage collection with GCMH :complex1:impact2: -After startup, `gcmh` dynamically adjusts GC thresholds to minimize pauses without manual tuning. -#+begin_src emacs-lisp - (use-package gcmh - :hook (after-init . gcmh-mode) - :config - (setq gcmh-idle-delay 5 - gcmh-high-cons-threshold (* 16 1024 1024))) -#+end_src - -**** DONE Load Dirvish on demand :complex2:impact3: -CLOSED: [2025-09-13 Sat 20:56] -`dirvish-config` requires Dirvish during initialization, negating deferral. Let `use-package` autoload the commands and enable overrides when Dired loads. -#+begin_src emacs-lisp - (use-package dirvish - :commands (dirvish dirvish-side) - :hook (dired-mode . dirvish-override-dired-mode)) -#+end_src - -**** DONE Start Org-roam lazily :complex3:impact3: -CLOSED: [2025-09-08 Mon 16:10] -Org-roam and its database sync run at startup. Load Org-roam only when Org is active, and start autosync after initialization. -#+begin_src emacs-lisp - (use-package org-roam - :after org - :commands (org-roam-node-find org-roam-node-insert) - :hook (after-init . org-roam-db-autosync-mode)) -#+end_src - -*** DOING [#C] Fix all docstring and elisp linting issues :bug: -- [X] user-constants -- [X] host-environment -- [ ] config-utilities -- [ ] system-defaults -- [ ] keybindings -- [ ] auth-config -- [ ] custom-functions -- [ ] chrono-tools -- [ ] file-config -- [ ] keyboard-macros -- [ ] system-utils -- [ ] text-config -- [ ] undead-buffers -- [ ] ui-config -- [ ] ui-theme -- [ ] ui-navigation -- [ ] font-config -- [ ] diff-config -- [ ] eshell-vterm-config -- [ ] flyspell-and-abbrev -- [ ] help-utils -- [ ] help-config -- [ ] latex-config -- [ ] modeline-config -- [ ] pdf-config -- [ ] selection-framework -- [ ] tramp-config -- [ ] show-kill-ring -- [ ] calibredb-epub-config -- [ ] dashboard-config -- [ ] dirvish-config -- [ ] dwim-shell-config -- [ ] elfeed-config -- [ ] erc-config -- [ ] eww-config -- [ ] httpd-config -- [ ] mail-config -- [ ] markdown-config -- [ ] weather-config -- [ ] prog-general -- [ ] vc-config -- [ ] flycheck-config -- [ ] prog-lsp -- [ ] prog-training -- [ ] prog-c -- [ ] prog-go -- [ ] prog-lisp -- [ ] prog-shell -- [ ] prog-python -- [ ] prog-webdev -- [ ] prog-yaml -- [ ] org-config -- [ ] org-agenda-config -- [ ] org-babel-config -- [ ] org-capture-config -- [ ] org-refile-config -- [ ] org-drill-config -- [ ] org-export-config -- [ ] org-roam-config -- [ ] org-contacts-config -- [ ] ai-config -- [ ] reconcile-open-repos -- [ ] video-audio-recording -- [ ] local-repository -- [ ] eradio-config -- [ ] games-config -- [ ] wrap-up - -** TODO [#C] Git Timemachine Litters Empty Buffers :bug: -Don't choose a revision and you'll see a blank buffer that needs to be killed -** TODO [#C] capture windows should pop up from bottom and 1/3 at most -** TODO [#C] install visible mark so you know where the mark is -#+BEGIN_QUOTE -Showing the location of the mark, probably helps with learning to use mark better. - -(use-package visible-mark - :init - (global-visible-mark-mode) - :custom - (visible-mark-faces '(visible-mark-face1 visible-mark-face2)) - (visible-mark-forward-faces '(visible-mark-face1 visible-mark-face2)) - (visible-mark-max 2)) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:33] -** TODO [#C] Edit Indirect (Custom Code) Bugs and Improvements -*** Bugs (by Severity) - -*** High Severity - -*Race Condition in =edit-indirect--commit=* -The commit function modifies the parent buffer while iterating through hooks that may also modify the buffer. The use of markers helps, but there's still a potential race condition if hooks modify the region being replaced. The =save-match-data= approach with markers could fail if the hooks significantly alter the buffer structure between marker creation and replacement. - -*Potential Data Loss in =edit-indirect--clean-up=* -The function calls =delete-overlay= before setting =edit-indirect--overlay= to nil. If =delete-overlay= signals an error (e.g., if the overlay was already deleted), the overlay reference won't be cleared, potentially causing infinite loops in =edit-indirect--abort-on-kill-buffer=. - -*** Medium Severity - -*Missing Buffer Validation in =edit-indirect--commit=* -The function doesn't check if the parent buffer still exists before attempting to modify it. If the parent buffer was killed while the indirect buffer was being edited, this will cause an error. - -*Hook Execution Order Issue* -The =edit-indirect--rebind-save-hooks= function is added to =after-change-major-mode-hook= globally but never removed. This means it will be called for every buffer that changes major mode, not just edit-indirect buffers, causing unnecessary overhead. - -*** Low Severity - -*Inefficient Overlay Search in =edit-indirect--search-for-edit-indirect=* -The function uses =overlays-in= which returns all overlays, then filters them. For buffers with many overlays, this is inefficient. - -*Missing Error Handling in =edit-indirect--display-buffer=* -The function doesn't handle cases where =display-buffer= returns nil (which can happen if all windows are dedicated or display is inhibited). - -*** Recommended Improvements - -*** Easy Implementation - -*Add Buffer Existence Check* -Add a check in =edit-indirect--commit= to ensure the parent buffer still exists: -#+begin_src emacs-lisp -(unless (buffer-live-p (overlay-buffer edit-indirect--overlay)) - (error "Parent buffer no longer exists")) -#+end_src - -*Improve Hook Management* -Remove the global hook when not needed: -#+begin_src emacs-lisp -(defun edit-indirect--rebind-save-hooks () - (when (edit-indirect-buffer-indirect-p) - (setq buffer-offer-save t) - (add-hook 'write-contents-functions #'edit-indirect--commit-on-save nil t) - ;; Remove global hook after use - (remove-hook 'after-change-major-mode-hook #'edit-indirect--rebind-save-hooks))) -#+end_src - -*Add Safety to Clean-up* -Wrap cleanup operations in condition-case: -#+begin_src emacs-lisp -(defun edit-indirect--clean-up () - (condition-case nil - (delete-overlay edit-indirect--overlay) - (error nil)) - (setq edit-indirect--overlay nil) - ...) -#+end_src - -*Add Display Buffer Error Handling* -Check display-buffer return value: -#+begin_src emacs-lisp -(defun edit-indirect--display-buffer (buffer) - (with-current-buffer buffer - (setq-local edit-indirect--should-quit-window t)) - (let ((window (display-buffer buffer))) - (if window - (select-window window) - (switch-to-buffer buffer))) - nil) -#+end_src - -*** Medium Implementation - -*Optimize Overlay Search* -Use a more efficient search strategy: -#+begin_src emacs-lisp -(defun edit-indirect--search-for-edit-indirect (beg end) - (cl-loop for overlay in (overlays-in beg end) - when (overlay-get overlay 'edit-indirect-buffer) - return overlay)) -#+end_src - -*Add Undo Boundary Management* -Add undo boundaries to make commits atomic: -#+begin_src emacs-lisp -(defun edit-indirect--commit () - (undo-boundary) - ;; ... rest of commit logic ... - (undo-boundary)) -#+end_src - -*Improve Buffer Naming* -Make buffer names more informative: -#+begin_src emacs-lisp -(defun edit-indirect--create-indirect-buffer (beg end overlay) - (let* ((mode-name (symbol-name major-mode)) - (buffer-name (format "*edit-indirect %s [%s]*" - (buffer-name) - (replace-regexp-in-string "-mode$" "" mode-name)))) - ...)) -#+end_src - -*** Hard Implementation - -*Add Support for Nested Indirect Edits* -Allow editing a region within an already indirect buffer by tracking parent chain. - -*Implement Diff Preview* -Add a command to show differences between the indirect buffer and parent region before committing: -#+begin_src emacs-lisp -(defun edit-indirect-diff-preview () - "Show differences between indirect buffer and parent region." - (interactive) - ;; Implementation using diff-mode - ) -#+end_src - -*Add Persistent Session Support* -Save and restore indirect editing sessions across Emacs restarts using desktop.el integration. - -*Implement Conflict Resolution* -Handle cases where the parent buffer was modified while editing in the indirect buffer, similar to version control merge conflicts. - -*Add Support for Multiple Regions* -Allow editing multiple non-contiguous regions in a single indirect buffer, useful for refactoring similar code blocks simultaneously. - -** TODO [#C] Review Titlecase Functionality -added in custom. Came from: https://codeberg.org/acdw/titlecase.el -Originally seen at https://emacselements.com/true-titlecase-in-emacs.html -** TODO [#C] Revisit and Refactor Localrepo Functionality -** TODO [#C] Fix Exporting Documents to HTML. -This appears to work with emacs -q. What's up with your config? -** TODO [#C] Rename 'sync-dir' Variable in Init File to 'org-dir' -[[https://github.com/ryuslash/mode-icons][ryuslash/mode-icons: Show icons instead of mode names]] -Captured On: [2025-06-07 Sat 13:29] -** TODO [#C] Emacs Config: Narrowing Org doesn't allow refresh by type -After generating an org agenda list, I can narrow the type to all DONE or IN-PROGRESS. However when I use my custom functions to just get the agenda for the current buffer, I can't. The scope has now widened and the refresh is for all buffers. -** TODO [#C] Add a Restrict to Subtree Org Agenda Command -In Org-mode the “agenda” machinery by default doesn’t pay any attention to your buffernarrow — it always scans whole files in your =org-agenda-files=. However, you can teach Org-agenda to only look between two points (the start and end of your current subtree) by using the built-in “restrict” hooks: -1. =org‐agenda‐restrict= – turn on restriction -2. =org‐agenda‐restrict‐begin= – a marker or position where scanning begins -3. =org‐agenda‐restrict‐end= – where it ends -4. (optionally) set =org-agenda-files= to just your current file -Here is a drop-in =org-agenda-custom-commands= entry that will give you a “Subtree TODOs” view limited to the subtree you’re on. Put this in your Emacs init: -#+begin_src emacs-lisp - ;; ------------------------------------------------------------------- - ;; 1) Define a custom agenda command "C" (hit C-c a C) - ;; that shows only TODO entries in the current subtree. - ;; ------------------------------------------------------------------- - (with-eval-after-load 'org - (add-to-list 'org-agenda-custom-commands - '("C" ; the key you press after C-c a - "Subtree TODOs" ; a descriptive name - ((todo "" ; match any TODO keyword - ;; === per-block settings === - ;; only look in this file - (org-agenda-files (list buffer-file-name)) - ;; enable the restriction engine - (org-agenda-restrict t) - ;; start at the top of the current heading - (org-agenda-restrict-begin - (progn (org-back-to-heading t) (point))) - ;; end at the end of this subtree - (org-agenda-restrict-end - (progn (org-end-of-subtree t) (point)))) - ;; you can add more blocks here if you like - )))) -#+end_src -How this works, step by step: -• We bind a new custom command under the key “C” (so you’ll type =C-c a C=). -• In that command we use the =todo= matcher =""= to catch every headline whose state is one of your TODO keywords. -• We dynamically set -– =org-agenda-files= to just the current buffer’s file, -– =org-agenda-restrict= to non-nil so Org will honor the begin/end markers, -– =org-agenda-restrict-begin= to the position of the current heading, -– =org-agenda-restrict-end= to the end of the subtree. -When you invoke it (=C-c a C=), Org will only scan headlines in that slice of the file and will build you a mini-agenda of TODO items from exactly that subtree. -Quick alternative: if you don’t need the full agenda machinery (dates, deadlines, etc.) but just want a fast outline of your TODOs under the current heading, you can also use the sparse-tree command: -• =C-c / t= ⇒ shows only the TODO headings in the narrowed or whole buffer as an indented outline. -— Tips & Pitfalls — -• Make sure your file is saved and in =org-agenda-files= (the snippet above forces it). -• Remember to call =org-narrow-to-subtree= (or let the command compute its own begin/end via =org-back-to-heading=/=org-end-of-subtree=). -• If you rename or move your file, Emacs must still see =buffer-file-name= valid. -• You can duplicate the above snippet for other TODO states, agenda views, or matchers (deadlines, tags, etc.) -With this in place you effectively get an agenda tailored to exactly the bit of tree you’re working on. -** TODO [#C] Find Another Keymapping for Emojify -** TODO [#C] Emacs: Add Reverso Functionality :enhancement: -https://sqrtminusone.xyz/packages/reverso/ -check his config for a usage example -Reverso -reverso.el is a package of mine that provides Emacs interface for https://reverso.net. -#+begin_src emacs-lisp - (use-package reverso - :straight (:host github :repo "SqrtMinusOne/reverso.el") - :init - (my-leader-def "ar" #'reverso) - :commands (reverso) - :config - (setq reverso-languages '(russian english german spanish french portuguese)) - (reverso-history-mode)) -#+end_src -** TODO [#C] Install Magit TODOs -#+BEGIN_QUOTE -It’s nice for magit to display in-tree TODOs in the status buffer: - -(use-package magit-todos - :init (magit-todos-mode) - :custom - (magit-todos-exclude-globs '(".git/" "*.gz" "*.html" "*.fastq" "*.fq")) - :after (magit)) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:40] -** TODO [#C] Move from company to corfu -#+BEGIN_QUOTE -Completion -A completion framework—I used to use company but corfu seems to integrate with stock Emacs better. - -(use-package corfu - :custom - (corfu-cycle t) - :init - (global-corfu-mode) - :config - (with-eval-after-load 'savehist - (corfu-history-mode 1) - (add-to-list 'savehist-additional-variables 'corfu-history))) -Use more completion at point functions from cape. dabbrev completion means that you can complete from words already present in the buffer, which helps for documents with long words that get used a lot. - -(use-package cape - :init - (dolist (fn (list #'cape-file - (cape-capf-super #'cape-dabbrev #'cape-dict #'org-block-capf #'cape-keyword))) - (add-to-list 'completion-at-point-functions fn)) - :config - (let ((wordlist (getenv "WORDLIST"))) - (setq - cape-dict-file wordlist - ispell-alternate-dictionary wordlist))) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:37] -** TODO [#C] Config recentf to exclude mail -#+BEGIN_QUOTE -recentf provides a persistent list of recently-opened files. - -(use-package recentf - :ensure nil - :init (recentf-mode) - :config - (setq recentf-exclude '("Maildir"))) - -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:36] -** TODO [#C] consider installing minimap -#+BEGIN_QUOTE -A mini-map (miniaturised window showing more of the buffer than the main window) is nice for longer documents: - -(use-package minimap - :custom - (minimap-major-modes '(org-mode prog-mode)) - (minimap-window-location 'right) - :bind ("C-+" . minimap-mode)) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:31] -** TODO [#C] check into org-download for drag/drop images into org-buffers -#+BEGIN_QUOTE -Drag and drop images into Org buffers to insert links: - -(use-package org-download - :custom - (org-download-image-dir "./img")) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:30] -** TODO [#D] BrainCurses Emacs Mastermind Game -[[https://sourceforge.net/projects/braincurses/][BrainCurses download | SourceForge.net]] -Captured On: [2025-09-26 Fri 00:38] -** TODO [#D] Emacs: org project should be yellow or green like todo -** TODO [#D] Emacs Signal Client -*** TODO investigate: net/signel.org · master · undefined · GitLab -[[https://gitlab.com/jaor/elibs/-/blob/master/net/signel.org][net/signel.org · master · undefined · GitLab]] -Captured On: [2025-05-29 Thu 04:24] -*** TODO investigate: signel, a barebones signal chat on top of signal-cli -[[https://jao.io/blog/signel.html][signel, a barebones signal chat on top of signal-cli]] -Captured On: [2025-05-29 Thu 04:23] -** TODO [#D] Add test document to test graphviz-dot-mode -** TODO [#D] Consider Replicating Some of This Auctex and PDF Tools Code -[[https://www.reddit.com/r/emacs/comments/cd6fe2/how_to_make_emacs_a_latex_ide/][How to make Emacs a Latex IDE? : r/emacs]] -Captured On: [2025-08-14 Thu 03:43] -** TODO [#D] Install ZOxide Integration into Emacs -[[https://gitlab.com/Vonfry/zoxide.el][Vonfry / zoxide.el · GitLab]] -Captured On: [2025-06-07 Sat 17:11] -** TODO [#D] manateelazycat/awesome-tray: Hide mode-line, display necessary information at right of minibuffer. -[[https://github.com/manateelazycat/awesome-tray][manateelazycat/awesome-tray: Hide mode-line, display necessary information at right of minibuffer.]] -Captured On: [2025-06-07 Sat 13:29] -** TODO [#D] ryuslash/mode-icons: Show icons instead of mode names diff --git a/docs/sessions/create-session.org b/docs/sessions/create-session.org deleted file mode 100644 index a0e4d2fe..00000000 --- a/docs/sessions/create-session.org +++ /dev/null @@ -1,352 +0,0 @@ -#+TITLE: Creating New Session Workflows -#+AUTHOR: Craig Jennings & Claude -#+DATE: 2025-11-01 - -* Overview - -This document describes the meta-workflow for creating new session types. When we identify a repetitive workflow or collaborative pattern, we use this process to formalize it into a documented session that we can reference and reuse. - -Session workflows are living documents that capture how we work together on specific types of tasks. They build our shared vocabulary and enable efficient collaboration across multiple work sessions. - -* Problem We're Solving - -Without a formal session creation process, we encounter several issues: - -** Inefficient Use of Intelligence -- Craig leads the process based solely on his knowledge -- We don't leverage Claude's expertise to improve or validate the approach -- Miss opportunities to apply software engineering and process best practices - -** Time Waste and Repetition -- Craig must re-explain the workflow each time we work together -- No persistent memory of how we've agreed to work -- Each session starts from scratch instead of building on previous work - -** Error-Prone Execution -- Important steps may be forgotten or omitted -- No checklist to verify completeness -- Mistakes lead to incomplete work or failed goals - -** Missed Learning Opportunities -- Don't capture lessons learned from our collaboration -- Can't improve processes based on what works/doesn't work -- Lose insights that emerge during execution - -** Limited Shared Vocabulary -- No deep, documented understanding of what terms mean -- "Let's do a refactor session" has no precise definition -- Can't efficiently communicate about workflows - -*Impact:* Inefficiency, errors, and lost opportunity to continuously improve our collaborative workflows. - -* Exit Criteria - -We know a session definition is complete when: - -1. **Information is logically arranged** - The structure makes sense and flows naturally -2. **Both parties understand how to work together** - We can articulate the workflow -3. **Agreement on effectiveness** - We both agree that following this session will lead to exit criteria and resolve the stated problem -4. **Tasks are clearly defined** - Steps are actionable, not vague -5. **Problem resolution path** - Completing the tasks either: - - Fixes the problem permanently, OR - - Provides a process for keeping the problem at bay - -*Measurable validation:* -- Can we both articulate the workflow without referring to the document? -- Do we agree it will solve the problem? -- Are the tasks actionable enough to start immediately? -- Does the session get used soon after creation (validation by execution)? - -* When to Use This Session - -Trigger this session creation workflow when: - -- You notice a repetitive workflow that keeps coming up -- A collaborative pattern emerges that would benefit from documentation -- Craig says "let's create/define/design a session for [activity]" -- You identify a new type of work that doesn't fit existing session types -- An existing session type needs significant restructuring (treat as creating a new one) - -Examples: -- "Let's create a session where we inbox zero" -- "We should define a code review session" -- "Let's design a session for weekly planning" - -* Approach: How We Work Together - -** Phase 1: Question and Answer Discovery - -Walk through these four core questions collaboratively. Take notes on the answers. - -*IMPORTANT: Save answers as you go!* - -The Q&A phase can take time—Craig may need to think through answers, and discussions can be lengthy. To prevent data loss from terminal crashes or process quits: - -1. Create a draft file at =docs/sessions/[name]-draft.org= after deciding on the name -2. After each question is answered, save the Q&A content to the draft file -3. If session is interrupted, you can resume from the saved answers -4. Once complete, the draft becomes the final session document - -This protects against losing substantial thinking work if the session is interrupted. - -*** Question 1: What problem are we solving in this type of session? - -Ask Craig: "What problem are we solving in this type of session?" - -The answer reveals: -- Overview and goal of the session -- Why this work matters (motivation) -- Impact/priority compared to other work -- What happens if we don't do this work - -Example from refactor session: -#+begin_quote -"My Emacs configuration isn't resilient enough. There's lots of custom code, and I'm even developing some as Emacs packages. Yet Emacs is my most-used software, so when Emacs breaks, I become unproductive. I need to make Emacs more resilient through good unit tests and refactoring." -#+end_quote - -*** Question 2: How do we know when we're done? - -Ask Craig: "How do we know when we're done?" - -The answer reveals: -- Exit criteria -- Results/completion criteria -- Measurable outcomes - -*Your role:* -- Push back if the answer is vague or unmeasurable -- Propose specific measurements based on context -- Iterate together until criteria are clear -- Fallback (hopefully rare): "when Craig says we're done" - -Example from refactor session: -#+begin_quote -"When we've reviewed all methods, decided which to test and refactor, run all tests, and fixed all failures including bugs we find." -#+end_quote - -Claude might add: "How about a code coverage goal of 70%+?" - -*** Question 3: How do you see us working together in this kind of session? - -Ask Craig: "How do you see us working together in this kind of session?" - -The answer reveals: -- Steps or phases we'll go through -- The general approach to the work -- How tasks flow from one to another - -*Your role:* -- As steps emerge, ask yourself: - - "Do these steps lead to solving the real problem?" - - "What is missing from these steps?" -- If the answers aren't "yes" and "nothing", raise concerns -- Propose additions based on your knowledge -- Suggest concrete improvements - -Example from refactor session: -#+begin_quote -"We'll analyze test coverage, categorize functions by testability, write tests systematically using Normal/Boundary/Error categories, run tests, analyze failures, fix bugs, and repeat." -#+end_quote - -Claude might suggest: "Should we install a code coverage tool as part of this process?" - -*** Question 4: Are there any principles we should be following while doing this? - -Ask Craig: "Are there any principles we should be following while doing this kind of session?" - -The answer reveals: -- Principles to follow -- Decision frameworks -- Quality standards -- When to choose option A vs option B - -*Your role:* -- Think through all elements of the session -- Consider situations that may arise -- Identify what principles would guide decisions -- Suggest decision frameworks from your knowledge - -Example from refactor session: -#+begin_quote -Craig: "Treat all test code as production code - same engineering practices apply." - -Claude suggests: "Since we'll refactor methods mixing UI and logic, should we add a principle to separate them for testability?" -#+end_quote - -** Phase 2: Assess Completeness - -After the Q&A, ask together: - -1. **Do we have enough information to formulate steps/process?** - - If yes, proceed to Phase 3 - - If no, identify what's missing and discuss further - -2. **Do we agree following this approach will resolve/mitigate the problem?** - - Both parties must agree - - If not, identify concerns and iterate - -** Phase 3: Name the Session - -Decide on a name for this session type. - -*Naming convention:* Action-oriented (verb form) -- Examples: "refactor", "inbox-zero", "create-session", "review-code" -- Why: Shorter, natural when saying "let's do a [name] session" -- Filename: =docs/sessions/[name].org= - -** Phase 4: Document the Session - -Write the session file at =docs/sessions/[name].org= using this structure: - -*** Recommended Structure -1. *Title and metadata* (=#+TITLE=, =#+AUTHOR=, =#+DATE=) -2. *Overview* - Brief description of the session -3. *Problem We're Solving* - From Q&A, with context and impact -4. *Exit Criteria* - Measurable outcomes, how we know we're done -5. *When to Use This Session* - Triggers, circumstances, examples -6. *Approach: How We Work Together* - - Phases/steps derived from Q&A - - Decision frameworks - - Concrete examples woven throughout -7. *Principles to Follow* - Guidelines from Q&A -8. *Living Document Notice* - Reminder to update with learnings - -*** Important Notes -- Weave concrete examples into sections (don't separate them) -- Use examples from actual sessions when available -- Make tasks actionable, not vague -- Include decision frameworks for common situations -- Note that this is a living document - -** Phase 5: Update Project State - -Update =NOTES.org=: -1. Add new session type to "Available Session Types" section -2. Include brief description and reference to file -3. Note creation date - -Example entry: -#+begin_src org -,** inbox-zero -File: =docs/sessions/inbox-zero.org= - -Workflow for processing inbox to zero: -1. [Brief workflow summary] -2. [Key steps] - -Created: 2025-11-01 -#+end_src - -** Phase 6: Validate by Execution - -*Critical step:* Use the session soon after creating it. - -- Schedule the session type for immediate use -- Follow the documented workflow -- Note what works well -- Identify gaps or unclear areas -- Update the session document with learnings - -*This validates the session definition and ensures it's practical, not theoretical.* - -* Principles to Follow - -These principles guide us while creating new sessions: - -** Collaboration Through Discussion -- Be proactive about collaboration -- Suggest everything on your mind -- Ask all relevant questions -- Push back when something seems wrong, inconsistent, or unclear -- Misunderstandings are learning opportunities - -** Reviewing the Whole as Well as the Pieces -- May get into weeds while identifying each step -- Stop to look at the whole thing at the end -- Ask the big questions: Does this actually solve the problem? -- Verify all pieces connect logically - -** Concrete Over Abstract -- Use examples liberally within explanations -- Weave concrete examples into Q&A answers -- Don't just describe abstractly -- "When nil input crashes, ask..." is better than "handle edge cases" - -** Actionable Tasks Over Vague Direction -- Steps should be clear enough to know what to do next -- "Ask: how do you see us working together?" is actionable -- "Figure out the approach" is too vague -- Test: Could someone execute this without further explanation? - -** Validate Early -- "Use it soon afterward" catches problems early -- Don't let session definitions sit unused and untested -- Real execution reveals gaps that theory misses -- Update immediately based on first use - -** Decision Frameworks Over Rigid Steps -- Sessions are frameworks (principles + flexibility), not recipes -- Include principles that help case-by-case decisions -- "When X happens, ask Y" is a decision framework -- "Always do X" is too rigid for most sessions - -** Question Assumptions -- If something doesn't make sense, speak up -- If a step seems to skip something, point it out -- Better to question during creation than discover gaps during execution -- No assumption is too basic to verify - -* Living Document - -This is a living document. As we create new sessions and learn what works (and what doesn't), we update this file with: - -- New insights about session creation -- Improvements to the Q&A process -- Better examples -- Additional principles discovered -- Refinements to the structure - -Every time we create a session, we have an opportunity to improve this meta-process. - -** Updates and Learnings - -*** 2025-11-01: Save Q&A answers incrementally -*Learning:* During emacs-inbox-zero session creation, we discovered that Q&A discussions can be lengthy and make Craig think deeply. Terminal crashes or process quits can lose substantial work. - -*Improvement:* Added guidance in Phase 1 to create a draft file and save Q&A answers after each question. This protects against data loss and allows resuming interrupted sessions. - -*Impact:* Reduces risk of losing 10-15 minutes of thinking work if session is interrupted. - -*** 2025-11-01: Validation by execution works! -*Learning:* Immediately after creating the emacs-inbox-zero session, we validated it by actually running the workflow. This caught unclear areas and validated that the 10-minute target was realistic. - -*Key insight from validation:* When Craig provides useful context during workflows (impact estimates, theories, examples), that context should be captured in task descriptions. This wasn't obvious during session creation but became clear during execution. - -*Impact:* Validation catches what theory misses. Always use Phase 6 (validate by execution) soon after creating a session. - -* Example: Creating the "Create-Session" Session - -This very document was created using the process it describes (recursive!). - -** The Q&A -- *Problem:* Time waste, errors, missed learning from informal processes -- *Exit criteria:* Logical arrangement, mutual understanding, agreement on effectiveness, actionable tasks -- *Approach:* Four-question Q&A, assess completeness, name it, document it, update NOTES.org, validate by use -- *Principles:* Collaboration through discussion, review the whole, concrete over abstract, actionable tasks, validate early, decision frameworks, question assumptions - -** The Result -We identified what was needed, collaborated on answers, and captured it in this document. Then we immediately used it to create the next session (validation). - -* Conclusion - -Creating session workflows is a meta-skill that improves all our collaboration. By formalizing how we work together, we: - -- Build shared vocabulary -- Eliminate repeated explanations -- Capture lessons learned -- Enable continuous improvement -- Make our partnership more efficient - -Each new session type we create adds to our collaborative toolkit and deepens our ability to work together effectively. - -*Remember:* Sessions are frameworks, not rigid recipes. They provide structure while allowing flexibility for case-by-case decisions. The goal is effectiveness, not perfection. diff --git a/docs/sessions/emacs-inbox-zero.org b/docs/sessions/emacs-inbox-zero.org deleted file mode 100644 index 4e046eba..00000000 --- a/docs/sessions/emacs-inbox-zero.org +++ /dev/null @@ -1,338 +0,0 @@ -#+TITLE: Emacs Inbox Zero Session -#+AUTHOR: Craig Jennings & Claude -#+DATE: 2025-11-01 - -* Overview - -This session processes the Emacs Config Inbox to zero by filtering tasks through the V2MOM framework. Items either move to active V2MOM methods, get moved to someday-maybe, or get deleted. This weekly discipline prevents backlog buildup and ensures only strategic work gets done. - -* Problem We're Solving - -Emacs is Craig's most-used software by a significant margin. It's the platform for email, calendar, task management, note-taking, programming, reading, music, podcasts, and more. When Emacs breaks, everything stops—including critical life tasks like family emails, doctor appointments, and bills. - -The V2MOM (Vision, Values, Methods, Obstacles, Metrics) framework provides strategic balance between fixing/improving Emacs versus using it for real work. But without weekly maintenance, the system collapses under backlog. - -** The Specific Problem - -Features and bugs get logged in the "Emacs Config Inbox" heading of =~/.emacs.d/todo.org=. If not sorted weekly: -- Items pile up and become unmanageable -- Unclear what's actually important -- Method 1 ("Make Using Emacs Frictionless") doesn't progress -- Two key metrics break: - 1. *Active todo count:* Should be < 20 items - 2. *Weekly triage consistency:* Must happen at least once per week by Sunday, no longer than 7 days between sessions - -** What Happens Without This Session - -Without weekly inbox zero: -- Backlog grows until overwhelming -- Can't distinguish signal from noise -- V2MOM becomes theoretical instead of practical -- Config maintenance competes with real work instead of enabling it -- Discipline muscle (Method 6: ruthless prioritization) atrophies - -*Impact:* The entire V2MOM system fails. Config stays broken longer. Real work gets blocked more often. - -* Exit Criteria - -The session is complete when: -- Zero todo items remain under the "* Emacs Config Inbox" heading in =~/.emacs.d/todo.org= -- All items have been routed to: V2MOM methods, someday-maybe, or deleted -- Can verify by checking the org heading (should be empty or show "0/0" in agenda) - -*IMPORTANT:* We are ONLY processing items under the "* Emacs Config Inbox" heading. Items already organized under Method 1-6 headings have already been triaged and should NOT be touched during this session. - -*Measurable validation:* -- Open =todo.org= and navigate to "* Emacs Config Inbox" heading -- Confirm no child tasks exist under this heading only -- Bonus: Check that active todo count is < 20 items across entire V2MOM - -* When to Use This Session - -Trigger this session when: -- It's Sunday and you haven't triaged this week -- 7 days have passed since last triage (hard deadline) -- "Emacs Config Inbox" has accumulated items -- You notice yourself avoiding looking at the inbox (sign it's becoming overwhelming) -- Before starting any new Emacs config work (ensures highest-priority work happens first) - -*Recommended cadence:* Every Sunday, 10 minutes, no exceptions. - -* Approach: How We Work Together - -** Phase 1: Sort by Priority - -First, ensure todo items are sorted by priority in =todo.org=: -- A (highest priority) -- B -- C -- No priority -- D (lowest priority) - -This ensures we always look at the most important items first. If time runs short, at least the high-priority items got processed. - -** Phase 2: Claude Rereads V2MOM - -Before processing any items, Claude rereads [[file:../EMACS-CONFIG-V2MOM.org][EMACS-CONFIG-V2MOM.org]] to have it fresh in mind. This ensures filtering decisions are grounded in the strategic framework. - -*What Claude should pay attention to:* -- The 6 Methods and their concrete actions -- The Values (Intuitive, Fast, Simple) and what they mean -- The Metrics (especially active todo count < 20) -- Method 6 discipline practices (ruthless prioritization, weekly triage, ship-over-research) - -** Phase 3: Process Each Item (in Priority Order) - -*IMPORTANT:* Process ONLY items under the "* Emacs Config Inbox" heading. Items already organized under Method 1-6 have been triaged and should remain where they are. - -For each item under "* Emacs Config Inbox", work through these questions: - -*** Question 1: Does this task need to be done at all? - -*Consider:* -- Has something changed? -- Was this a mistake? -- Do I disagree with this idea now? -- Is this actually important? - -*If NO:* **DELETE** the item immediately. Don't move it anywhere. Kill it. - -*Examples of deletions:* -- "Add Signal client to Emacs" - Cool idea, not important -- "Try minimap mode" - Interesting, doesn't serve vision -- "Research 5 different completion frameworks" - Already have Vertico/Corfu, stop researching - -*** Question 2: Is this task related to the Emacs Config V2MOM? - -*If NO:* **Move to** =docs/someday-maybe.org= - -These are tasks that might be good ideas but don't serve the current strategic focus. They're not deleted (might revisit later) but they're out of active consideration. - -*Examples:* -- LaTeX improvements (no concrete need yet) -- Elfeed dashboard redesign (unclear if actually used) -- New theme experiments (side project competing with maintenance) - -*** Question 3: Which V2MOM method does this relate to? - -*If YES (related to V2MOM):* - -Claude suggests which method(s) this might relate to: -- Method 1: Make Using Emacs Frictionless (performance, bug fixes, missing features) -- Method 2: Stop Problems Before They Appear (package upgrades, deprecation removal) -- Method 3: Make Fixing Emacs Frictionless (tooling, testing, profiling) -- Method 4: Contribute to the Emacs Ecosystem (package maintenance) -- Method 5: Be Kind To Your Future Self (new capabilities) -- Method 6: Develop Disciplined Engineering Practices (meta-practices) - -*This is a conversation.* If the relationship is only tangential: -- **Claude should push back** - "This seems tangential. Adding it would dilute focus and delay V2MOM completion. Are you sure this serves the vision?" -- Help Craig realize it doesn't fit through questions -- The more we add, the longer V2MOM takes, the harder it is to complete - -*If item relates to multiple methods:* -Pick the **highest priority method** (Method 1 > Method 2 > Method 3 > etc.) - -*IMPORTANT: Capture useful context!* -During discussion, Craig may provide: -- Impact estimates ("15-20 seconds × 12 times/day") -- Theories about root causes -- Context about why this matters -- Examples of when the problem occurs - -**When moving items to methods, add this context to the task description.** This preserves valuable information for later execution and helps prioritize work accurately. - -*Then:* Move the item to the appropriate method section in the V2MOM or active todo list with enriched context. - -** Phase 4: Verify and Celebrate - -Once all items are processed: -1. Verify "Emacs Config Inbox" heading is empty -2. Check that active todo count is < 20 items -3. Note the date of this triage session -4. Acknowledge: You've practiced ruthless prioritization (Method 6 skill development) - -** Decision Framework: When Uncertain - -If you're uncertain whether an item fits V2MOM: - -1. **Ask: Does this directly serve the Vision?** (Work at speed of thought, stable config, comprehensive workflows) -2. **Ask: Does this align with Values?** (Intuitive, Fast, Simple) -3. **Ask: Is this in the Methods already?** (If not explicitly listed, probably shouldn't add) -4. **Ask: What's the opportunity cost?** (Every new item delays everything else) - -*When in doubt:* Move to someday-maybe. You can always pull it back later if it proves critical. Better to be conservative than to dilute focus. - -* Principles to Follow - -** Claude's Role: "You're here to help keep me honest" - -Craig is developing discipline (Method 6: ruthless prioritization). Not making progress = not getting better. - -*Claude's responsibilities:* -- If task clearly fits V2MOM → Confirm and move forward quickly -- If task is unclear/tangential → **Ask questions** to help Craig realize it doesn't fit or won't lead to V2MOM success -- Enable ruthless prioritization by helping Craig say "no" -- Don't let good ideas distract from great goals - -*Example questions Claude might ask:* -- "This is interesting, but which specific metric does it improve?" -- "We already have 3 items in Method 1 addressing performance. Does this add something different?" -- "This would be fun to build, but does it make using Emacs more frictionless?" -- "If you had to choose between this and fixing org-agenda (30s → 5s), which serves the vision better?" - -** Time Efficiency: 10 Minutes Active Work - -Don't take too long on any single item. Splitting philosophical hairs = procrastination. - -*Target:* **10 minutes active work time** (not clock time - interruptions expected) - -*If spending > 1 minute on a single item:* -- Decision is unclear → Move to someday-maybe (safe default) -- Come back to it later if it proves critical -- Keep moving - -*Why this matters:* -- Weekly consistency requires low friction -- Perfect categorization doesn't matter as much as consistent practice -- Getting through all items > perfectly routing each item - -** Ruthless Prioritization Over Completeness - -The goal is not to do everything in the inbox. The goal is to identify and focus on what matters most. - -*Better to:* -- Delete 50% of items and ship the other 50% -- Than keep 100% and ship 0% - -*Remember:* -- Every item kept is opportunity cost -- V2MOM already has plenty of work -- "There will always be cool ideas out there to implement and they will always be a web search away" (Craig's words) - -** Bias Toward Action - -When processing items that ARE aligned with V2MOM: -- Move them to the appropriate method quickly -- Don't overthink the categorization -- Getting it 80% right is better than spending 5 minutes getting it 100% right -- You can always recategorize later during regular triage - -* Living Document - -This is a living document. After each emacs-inbox-zero session, consider: -- Did the workflow make sense? -- Were any steps unclear or unnecessary? -- Did any new situations arise that need decision frameworks? -- Did the 10-minute target work, or should it adjust? - -Update this document with learnings to make future sessions smoother. - -* Example Session Walkthrough - -** Setup -- Open =~/.emacs.d/todo.org= -- Navigate to "Emacs Config Inbox" heading -- Verify items are sorted by priority (A → B → C → none → D) -- Claude rereads =EMACS-CONFIG-V2MOM.org= - -** Processing Example Items - -*** Example 1: [#A] Fix org-agenda slowness (30+ seconds) - -*Q1: Does this need to be done?* YES - Daily pain point blocking productivity - -*Q2: Related to V2MOM?* YES - Method 1 explicitly lists this - -*Q3: Which method?* Method 1: Make Using Emacs Frictionless - -*Action:* Move to Method 1 active tasks (or confirm already there) - -*Time:* 15 seconds - -*** Example 2: [#B] Add Signal client to Emacs - -*Q1: Does this need to be done?* Let's think... - -Claude: "What problem does this solve? Is messaging in Emacs part of the Vision?" - -Craig: "Not really, I already use Signal on my phone fine." - -*Action:* **DELETE** - Doesn't serve vision, already have working solution - -*Time:* 30 seconds - -*** Example 3: [#C] Try out minimap mode for code navigation - -*Q1: Does this need to be done?* Interesting idea, but not important - -*Action:* **DELETE** or move to someday-maybe - Interesting, not important - -*Time:* 10 seconds - -*** Example 4: [#B] Implement transcription workflow - -*Q1: Does this need to be done?* YES - Want to transcribe recordings for notes - -*Q2: Related to V2MOM?* Maybe... seems like new feature? - -Claude: "This seems like Method 5: Be Kind To Your Future Self - new capability you'll use repeatedly. Complete code already exists in old todo.org. But we're still working through Method 1 (frictionless) and Method 2 (stability). Should this wait, or is transcription critical?" - -Craig: "Actually yes, I record meetings and need transcripts. This is important." - -*Q3: Which method?* Method 5: Be Kind To Your Future Self - -*Action:* Move to Method 5 (but note: prioritize after Methods 1-3) - -*Time:* 45 seconds (good conversation, worth the time) - -** Result -- 4 items processed in ~2 minutes -- 1 moved to Method 1 (already there) -- 1 deleted -- 1 deleted or moved to someday-maybe -- 1 moved to Method 5 -- Inbox is clearer, focus is sharper - -* Conclusion - -Emacs inbox zero is not about getting through email or org-capture. It's about **strategic filtering of config maintenance work**. By processing the inbox weekly, you: - -- Keep maintenance load manageable (< 20 active items) -- Ensure only V2MOM-aligned work happens -- Practice ruthless prioritization (Method 6 skill) -- Prevent backlog from crushing future productivity -- Build the discipline that makes all other methods sustainable - -**The session takes 10 minutes. Not doing it costs days of distracted, unfocused work on things that don't matter.** - -*Remember:* Inbox zero is not about having zero things to do. It's about knowing exactly what you're NOT doing, so you can focus completely on what matters most. - -* Living Document - -This is a living document. After each emacs-inbox-zero session, consider: -- Did the workflow make sense? -- Were any steps unclear or unnecessary? -- Did any new situations arise that need decision frameworks? -- Did the 10-minute target work, or should it adjust? - -Update this document with learnings to make future sessions smoother. - -** Updates and Learnings - -*** 2025-11-01: First validation session - Process works! - -*Session results:* -- 5 items processed in ~10 minutes (target met) -- 1 deleted (duplicate), 2 moved to Method 1, 2 moved to someday-maybe -- Inbox cleared to zero -- Priority sorting worked well -- Three-question filter was effective -- Caught duplicate task and perfectionism pattern in real-time - -*Key learning: Capture useful context during triage* -When Craig provides impact estimates ("15-20 seconds × 12 times/day"), theories, or context during discussion, **Claude should add this information to the task description** when moving items to methods. This preserves valuable context for execution and helps with accurate prioritization. - -Example: "Optimize org-capture target building" was enriched with "15-20 seconds every time capturing a task (12+ times/day). Major daily bottleneck - minutes lost waiting, plus context switching cost." - -*Impact:* Better task descriptions → better prioritization → better execution. diff --git a/docs/sessions/refactor.org b/docs/sessions/refactor.org deleted file mode 100644 index 0cdb6841..00000000 --- a/docs/sessions/refactor.org +++ /dev/null @@ -1,617 +0,0 @@ -#+TITLE: Test-Driven Quality Engineering Session Process -#+AUTHOR: Craig Jennings & Claude -#+DATE: 2025-11-01 - -* Overview - -This document describes a comprehensive test-driven quality engineering session process applicable to any source code module. The session demonstrates systematic testing practices, refactoring for testability, bug discovery through tests, and decision-making processes when tests fail. - -* Session Goals - -1. Add comprehensive unit test coverage for testable functions in your module -2. Discover and fix bugs through systematic testing -3. Follow quality engineering principles from =ai-prompts/quality-engineer.org= -4. Demonstrate refactoring patterns for testability -5. Document the decision-making process for test vs production code issues - -* Phase 1: Feature Addition with Testability in Mind - -** The Feature Request - -Add new functionality that requires user interaction combined with business logic. - -Example requirements: -- Present user with options (e.g., interactive selection) -- Allow cancellation -- Perform an operation with the selected input -- Provide clear success/failure feedback - -** Refactoring for Testability - -Following the "Interactive vs Non-Interactive Function Pattern" from =quality-engineer.org=: - -*Problem:* Directly implementing as an interactive function would require: -- Mocking user interface components -- Mocking framework-specific APIs -- Testing UI functionality, not core business logic - -*Solution:* Split into two functions: - -1. *Helper Function* (internal implementation): - - Pure, deterministic - - Takes explicit parameters - - No user interaction - - Returns values or signals errors naturally - - 100% testable, no mocking needed - -2. *Interactive Wrapper* (public interface): - - Thin layer handling only user interaction - - Gets input from user/context - - Presents UI (prompts, selections, etc.) - - Catches errors and displays messages - - Delegates all business logic to helper - - No tests needed (just testing framework UI) - -** Benefits of This Pattern - -From =quality-engineer.org=: -#+begin_quote -When writing functions that combine business logic with user interaction: -- Split into internal implementation and interactive wrapper -- Internal function: Pure logic, takes all parameters explicitly -- Dramatically simpler testing (no interactive mocking) -- Code reusable programmatically without prompts -- Clear separation of concerns (logic vs UI) -#+end_quote - -This pattern enables: -- Zero mocking in tests -- Fast, deterministic tests -- Easy reasoning about correctness -- Reusable helper function - -* Phase 2: Writing the First Test - -** Test File Naming - -Following the naming convention from =quality-engineer.org=: -- Pattern: =test-<module>-<function>.<ext>= -- One test file per function for easy discovery when tests fail -- Developer sees failure → immediately knows which file to open - -** Test Organization - -Following the three-category structure: - -*** Normal Cases -- Standard expected inputs -- Common use case scenarios -- Happy path operations -- Multiple operations in sequence - -*** Boundary Cases -- Very long inputs -- Unicode characters (中文, emoji) -- Special characters and edge cases -- Empty or minimal data -- Maximum values - -*** Error Cases -- Invalid inputs -- Nonexistent resources -- Permission denied scenarios -- Wrong type of input - -** Writing Tests with Zero Mocking - -Key principle: "Don't mock what you're testing" (from =quality-engineer.org=) - -Example test structure: -#+begin_src -test_function_normal_case_expected_result() - setup() - try: - # Arrange - input_data = create_test_data() - expected_output = define_expected_result() - - # Act - actual_output = function_under_test(input_data) - - # Assert - assert actual_output == expected_output - finally: - teardown() -#+end_src - -Key characteristics: -- No mocks for the function being tested -- Real resources (files, data structures) using test utilities -- Tests actual function behavior -- Clean setup/teardown -- Clear arrange-act-assert structure - -** Result - -When helper functions are well-factored and deterministic, tests often pass on first run. - -* Phase 3: Systematic Test Coverage Analysis - -** Identifying Testable Functions - -Review all functions in your module and categorize by testability: - -*** Easy to Test (Pure/Deterministic) -- Input validation functions -- String manipulation/formatting -- Data structure transformations -- File parsing (read-only operations) -- Configuration/option processing - -*** Medium Complexity (Need External Resources) -- File I/O operations -- Recursive algorithms -- Data structure generation -- Cache or state management - -*** Hard to Test (Framework/Context Dependencies) -- Functions requiring specific runtime environment -- UI/buffer/window management -- Functions tightly coupled to framework internals -- Functions requiring complex mocking setup - -*Decision:* Test easy and medium complexity functions. Skip framework-dependent functions that would require extensive mocking/setup (diminishing returns). - -** File Organization Principle - -From =quality-engineer.org=: -#+begin_quote -*Unit Tests*: One file per method -- Naming: =test-<filename>-<methodname>.<ext>= -- Example: =test-module--function.ext= -#+end_quote - -*Rationale:* When a test fails in CI: -1. Developer sees: =test-module--function-normal-case-returns-result FAILED= -2. Immediately knows: Look for =test-module--function.<ext>= -3. Opens file and fixes issue - *fast cognitive path* - -If combined files: -1. Test fails: =test-module--function-normal-case-returns-result FAILED= -2. Which file? =test-module--helpers.<ext>=? =test-module--combined.<ext>=? -3. Developer wastes time searching - *slower, frustrating* - -*The 1:1 mapping is a usability feature for developers under pressure.* - -* Phase 4: Testing Function by Function - -** Example 1: Input Validation Function - -*** Test Categories - -*Normal Cases:* -- Valid inputs -- Case variations -- Common use cases - -*Boundary Cases:* -- Edge cases in input format -- Multiple delimiters or separators -- Empty or minimal input -- Very long input - -*Error Cases:* -- Nil/null input -- Wrong type -- Malformed input - -*** First Run: Most Passed, Some FAILED - -*Example Failure:* -#+begin_src -test-module--validate-input-error-nil-input-returns-nil -Expected: Returns nil gracefully -Actual: (TypeError/NullPointerException) - CRASHED -#+end_src - -*** Bug Analysis: Test or Production Code? - -*Process:* -1. Read the test expectation: "nil input returns nil/false gracefully" -2. Read the production code: - #+begin_src - function validate_input(input): - extension = get_extension(input) # ← Crashes here on nil/null - return extension in valid_extensions - #+end_src -3. Identify issue: Function expects string, crashes on nil/null -4. Consider context: This is defensive validation code, called in various contexts - -*Decision: Fix production code* - -*Rationale:* -- Function should be defensive (validation code) -- Returning false/nil for invalid input is more robust than crashing -- Common pattern in validation functions -- Better user experience - -*Fix:* -#+begin_src -function validate_input(input): - if input is None or not isinstance(input, str): # ← Guard added - return False - extension = get_extension(input) - return extension in valid_extensions -#+end_src - -Result: All tests pass after adding defensive checks. - -** Example 2: Another Validation Function - -*** First Run: Most Passed, Multiple FAILED - -*Failures:* -1. Nil input crashed (same pattern as previous function) -2. Empty string returned unexpected value (edge case not handled) - -*Fix:* -#+begin_src -function validate_resource(resource): - # Guards added for nil/null and empty string - if not resource or not isinstance(resource, str) or resource.strip() == "": - return False - - # Original validation logic - return is_valid_resource(resource) and meets_criteria(resource) -#+end_src - -Result: All tests pass after adding comprehensive guards. - -** Example 3: String Sanitization Function - -*** First Run: Most Passed, 1 FAILED - -*Failure:* -#+begin_src -test-module--sanitize-boundary-special-chars-replaced -Expected: "output__________" (10 underscores) -Actual: "output_________" (9 underscores) -#+end_src - -*** Bug Analysis: Test or Production Code? - -*Process:* -1. Count special chars in test input: 9 characters -2. Test expected 10 replacements, but input only has 9 -3. Production code is working correctly - -*Decision: Fix test code* - -*The bug was in the test expectation, not the implementation.* - -Result: All tests pass after correcting test expectations. - -** Example 4: File/Data Parser Function - -This is where a **significant bug** was discovered through testing! - -*** Test Categories - -*Normal Cases:* -- Absolute paths/references -- Relative paths (expanded to base directory) -- URLs/URIs preserved as-is -- Mixed types of references - -*Boundary Cases:* -- Empty lines ignored -- Whitespace-only lines ignored -- Comments ignored (format-specific) -- Leading/trailing whitespace trimmed -- Order preserved - -*Error Cases:* -- Nonexistent file -- Nil/null input - -*** First Run: Majority Passed, Multiple FAILED - -All failures related to URL/URI handling: - -*Failure Pattern:* -#+begin_src -Expected: "http://example.com/resource" -Actual: "/base/path/http:/example.com/resource" -#+end_src - -URLs were being treated as relative paths and corrupted! - -*** Root Cause Analysis - -*Production code:* -#+begin_src -if line.matches("^\(https?|mms\)://"): # Pattern detection - # Handle as URL -#+end_src - -*Problem:* Pattern matching is incorrect! - -The pattern/regex has an error: -- Incorrect escaping or syntax -- Pattern fails to match valid URLs -- All URLs fall through to the "relative path" handler - -The pattern never matched, so URLs were incorrectly processed as relative paths. - -*Correct version:* -#+begin_src -if line.matches("^(https?|mms)://"): # Fixed pattern - # Handle as URL -#+end_src - -Common causes of this type of bug: -- String escaping issues in the language -- Incorrect regex syntax -- Copy-paste errors in patterns - -*** Impact Assessment - -*This is a significant bug:* -- Remote resources (URLs) would be broken -- Data corruption: URLs transformed into invalid paths -- Function worked for local/simple cases, so bug went unnoticed -- Users would see mysterious errors when using remote resources -- Potential data loss or corruption in production - -*Tests caught a real production bug that could have caused user data corruption!* - -Result: All tests pass after fixing the pattern matching logic. - -* Phase 5: Continuing Through the Test Suite - -** Additional Functions Tested Successfully - -As testing continues through the module, patterns emerge: - -*Function: Directory/File Listing* - - Learning: Directory listing order may be filesystem-dependent - - Solution: Sort results before comparing in tests - -*Function: Data Extraction* - - Keep as separate test file (don't combine with related functions) - - Reason: Usability when tests fail - -*Function: Recursive Operations* - - Medium complexity: Required creating test data structures/trees - - Use test utilities for setup/teardown - - Well-factored functions often pass all tests initially - -*Function: Higher-Order Functions* - - Test functions that return functions/callbacks - - Initially may misunderstand framework/protocol behavior - - Fix test expectations to match actual framework behavior - -* Key Principles Applied - -** 1. Refactor for Testability BEFORE Writing Tests - -The Interactive vs Non-Interactive pattern from =quality-engineer.org= made testing trivial: -- No mocking required -- Fast, deterministic tests -- Clear separation of concerns - -** 2. Systematic Test Organization - -Every test file followed the same structure: -- Normal Cases -- Boundary Cases -- Error Cases - -This makes it easy to: -- Identify coverage gaps -- Add new tests -- Understand what's being tested - -** 3. Test Naming Convention - -Pattern: =test-<module>-<function>-<category>-<scenario>-<expected-result>= - -Examples: -- =test-module--validate-input-normal-valid-extension-returns-true= -- =test-module--parse-data-boundary-empty-lines-ignored= -- =test-module--sanitize-error-nil-input-signals-error= - -Benefits: -- Self-documenting -- Easy to understand what failed -- Searchable/grepable -- Clear category organization - -** 4. Zero Mocking for Pure Functions - -From =quality-engineer.org=: -#+begin_quote -DON'T MOCK WHAT YOU'RE TESTING -- Only mock external side-effects and dependencies, not the domain logic itself -- If mocking removes the actual work the function performs, you're testing the mock -- Use real data structures that the function is designed to operate on -- Rule of thumb: If the function body could be =(error "not implemented")= and tests still pass, you've over-mocked -#+end_quote - -Our tests used: -- Real file I/O -- Real strings -- Real data structures -- Actual function behavior - -Result: Tests caught real bugs, not mock configuration issues. - -** 5. Test vs Production Code Bug Decision Framework - -When a test fails, ask: - -1. *What is the test expecting?* - - Read the test name and assertions - - Understand the intended behavior - -2. *What is the production code doing?* - - Read the implementation - - Trace through the logic - -3. *Which is correct?* - - Is the test expectation reasonable? - - Is the production behavior defensive/robust? - - What is the usage context? - -4. *Consider the impact:* - - Defensive code: Fix production to handle edge cases - - Wrong expectation: Fix test - - Unclear spec: Ask user for clarification - -Examples from our session: -- *Nil input crashes* → Fix production (defensive coding) -- *Empty string treated as valid* → Fix production (defensive coding) -- *Wrong count in test* → Fix test (test bug) -- *Regex escaping wrong* → Fix production (real bug!) - -** 6. Fast Feedback Loop - -Pattern: "Write tests, run them all, report errors, and see where we are!" - -This became a mantra during the session: -1. Write comprehensive tests for one function -2. Run immediately -3. Analyze failures -4. Fix bugs (test or production) -5. Verify all tests pass -6. Move to next function - -Benefits: -- Caught bugs immediately -- Small iteration cycles -- Clear progress -- High confidence in changes - -* Final Results - -** Test Coverage Example - -*Multiple functions tested with comprehensive coverage:* -1. File operation helper - ~10-15 tests -2. Input validation function - ~15 tests -3. Resource validation function - ~13 tests -4. String sanitization function - ~13 tests -5. File/data parser function - ~15 tests -6. Directory listing function - ~7 tests -7. Data extraction function - ~6 tests -8. Recursive operation function - ~12 tests -9. Higher-order function - ~12 tests - -Total: Comprehensive test suite covering all testable functions - -** Bugs Discovered and Fixed - -1. *Input Validation Function* - - Issue: Crashed on nil/null input - - Fix: Added nil/type guards - - Impact: Prevents crashes in validation code - -2. *Resource Validation Function* - - Issue: Crashed on nil, treated empty string as valid - - Fix: Added guards for nil and empty string - - Impact: More robust validation - -3. *File/Data Parser Function* ⚠️ *SIGNIFICANT BUG* - - Issue: Pattern matching wrong - URLs/URIs corrupted as relative paths - - Fix: Corrected pattern matching logic - - Impact: Remote resources now work correctly - - *This bug would have corrupted user data in production* - -** Code Quality Improvements - -- All testable helper functions now have comprehensive test coverage -- More defensive error handling (nil guards) -- Clear separation of concerns (pure helpers vs interactive wrappers) -- Systematic boundary condition testing -- Unicode and special character handling verified - -* Lessons Learned - -** 1. Tests as Bug Discovery Tools - -Tests aren't just for preventing regressions - they actively *discover existing bugs*: -- Pattern matching bugs may exist in production -- Nil/null handling bugs manifest in edge cases -- Tests make these issues visible immediately -- Bugs caught before users encounter them - -** 2. Refactoring Enables Testing - -The decision to split functions into pure helpers + interactive wrappers: -- Made testing dramatically simpler -- Enabled 100+ tests with zero mocking -- Improved code reusability -- Clarified function responsibilities - -** 3. Systematic Process Matters - -Following the same pattern for each function: -- Reduced cognitive load -- Made it easy to maintain consistency -- Enabled quick iteration -- Built confidence in coverage - -** 4. File Organization Aids Debugging - -One test file per function: -- Fast discovery when tests fail -- Clear ownership -- Easy to maintain -- Follows user's mental model - -** 5. Test Quality Equals Production Quality - -Quality tests: -- Use real resources (not mocks) -- Test actual behavior -- Cover edge cases systematically -- Find real bugs - -This is only possible with well-factored, testable code. - -* Applying These Principles - -When adding tests to other modules: - -1. *Identify testable functions* - Look for pure helpers, file I/O, string manipulation -2. *Refactor if needed* - Split interactive functions into pure helpers -3. *Write systematically* - Normal, Boundary, Error categories -4. *Run frequently* - Fast feedback loop -5. *Analyze failures carefully* - Test bug vs production bug -6. *Fix immediately* - Don't accumulate technical debt -7. *Maintain organization* - One file per function, clear naming - -* Reference - -See =ai-prompts/quality-engineer.org= for comprehensive quality engineering guidelines, including: -- Test organization and structure -- Test naming conventions -- Mocking and stubbing best practices -- Interactive vs non-interactive function patterns -- Integration testing guidelines -- Test maintenance strategies - -Note: =quality-engineer.org= evolves as we learn more quality best practices. This document captures principles applied during this specific session. - -* Conclusion - -This session process demonstrates how systematic testing combined with refactoring for testability can: -- Discover real bugs before they reach users -- Improve code quality and robustness -- Build confidence in changes -- Create maintainable test suites -- Follow industry best practices - -A comprehensive test suite with multiple bug fixes represents significant quality improvement to any module. Critical bugs (like the pattern matching issue in the example) alone can justify the entire testing effort - such bugs can cause data corruption and break major features. - -*Testing is not just about preventing future bugs - it's about finding bugs that already exist.* diff --git a/docs/values-comparison.org b/docs/values-comparison.org deleted file mode 100644 index 128c6710..00000000 --- a/docs/values-comparison.org +++ /dev/null @@ -1,107 +0,0 @@ -#+TITLE: Values Comparison - Simple vs Reliability -#+DATE: 2025-10-30 - -* What Currently Supports Each Value - -| Theme/Practice | Simple | Reliability | -|---------------------------------------+--------+-------------| -| *Code Quality* | | | -| Clear naming | ✓ | | -| Small functions | ✓ | | -| Good comments (why not what) | ✓ | | -| Abstractions when warranted | ✓ | | -|---------------------------------------+--------+-------------| -| *Architecture* | | | -| Modular design | ✓ | | -| Extract custom code into packages | ✓ | | -| Good architecture enables evolution | ✓ | | -| Technical debt has real cost | ✓ | | -|---------------------------------------+--------+-------------| -| *Production Practices* | | | -| Testing | ✓ | ✓ | -| Documentation | ✓ | | -| Version control | ✓ | | -|---------------------------------------+--------+-------------| -| *Failure Planning* | | | -| Rollback capability (localrepo) | ✓ | ✓ | -| Error handling | ✓ | ✓ | -| Graceful degradation | ✓ | ✓ | -|---------------------------------------+--------+-------------| -| *Trust & Stability* | | | -| Never use tools you can't trust | ✓ | ✓ | -| Uptime matters | | ✓ | -| Config breakage crushes productivity | | ✓ | -| More packages = more breakage risk | | ✓ | -| Need confidence to refactor | | ✓ | -|---------------------------------------+--------+-------------| - -* Key Quotes Supporting Each - -** Simple -- "Config IS production software - treat it as such" -- "Clear naming, small functions, good comments" -- "Apply production practices: Testing, modularity, documentation" -- "Simple means maintainable production software, not minimal or primitive" - -** Reliability/Resilience -- "Never use a tool you can't fully trust" -- "The impact when it's broken can crush my productivity" -- "More packages = more potential breakage. Need ability to roll back" -- "Plan for failure: Rollback capability, error handling, graceful degradation" -- "Quality matters because uptime matters" - -* Analysis - -** If Reliability is part of Simple: -*Simple means:* Maintainable, trustworthy, production-quality code that fails gracefully - -*Pros:* -- Keeps to 3 values -- Failure planning is a consequence of good engineering (already in Simple) -- Less redundancy - -*Cons:* -- "Simple" might not clearly convey the "must not break" urgency -- Rollback/resilience themes could get lost - -** If Reliability is its own value: -*Simple means:* Clear, maintainable code -*Reliability means:* Never breaks, fails gracefully, always recoverable - -*Pros:* -- Explicit focus on uptime/stability -- Clear that this is mission-critical (crushes productivity when broken) -- Resilience practices (localrepo, testing, error handling) have clear home - -*Cons:* -- 4 values instead of 3 -- Some overlap with Simple - -* Decision Framework - -Ask yourself: - -1. When choosing between two approaches, would "reliability" change your decision differently than "simple"? - - Example: Complex caching solution that's faster but harder to debug - - Simple says: "Too complex, harder to maintain" - - Reliability says: "If it breaks less often and has rollback, worth the complexity" - -2. Does "config can't break" deserve equal weight with Fast and Intuitive? - -3. Is the "trust" theme strong enough to be its own value? - -* Recommendation - -If you had to choose, I'd suggest: **Keep Simple, absorb Reliability into it.** - -Why? Most of your reliability practices (testing, rollback, error handling) are already listed under "production software practices" in Simple. The "never use tools you can't trust" is already there too. - -**But** - you could rename Simple to "**Reliable**" or "**Solid**" if the stability theme is more important than the simplicity theme. - -Alternative value names that might capture both: -- *Solid* - Stable, maintainable, trustworthy -- *Resilient* - Fails gracefully, recovers easily, maintainable -- *Dependable* - Simple enough to trust, robust enough to rely on - -What resonates? diff --git a/early-init.el b/early-init.el index 79ff7816..c1cd54c8 100644 --- a/early-init.el +++ b/early-init.el @@ -58,8 +58,12 @@ ;; ------------------------------ Bug Workarounds ------------------------------ -;; Prevent org-element from being natively compiled again by adding the line -(setq native-comp-jit-compilation-deny-list '(".*org-element.*")) +;; Disable async native compilation to prevent "Selecting deleted buffer" errors +;; This is a known issue in Emacs 30.x where async compilation buffers get +;; deleted before the compilation process completes. Synchronous compilation +;; is slower initially but avoids these race conditions. +(setq native-comp-deferred-compilation nil) ;; Disable async/deferred compilation +(setq native-comp-async-report-warnings-errors nil) ;; Silence async warnings ;; --------------------------- Warning Notifications --------------------------- @@ -84,6 +84,7 @@ (require 'pdf-config) ;; pdf display settings (require 'quick-video-capture) ;; download videos with a browser bookmark (require 'video-audio-recording) ;; desktop and/or audio recording via ffmpeg +(require 'transcription-config) ;; audio transcription using Whisper (require 'weather-config) ;; utility to display the weather ;; -------------------------------- Programming -------------------------------- diff --git a/issues.org b/issues.org deleted file mode 100644 index c61b8517..00000000 --- a/issues.org +++ /dev/null @@ -1,1879 +0,0 @@ -#+TITLE: Emacs Configuration Issues & Enhancement Roadmap -#+AUTHOR: Claude Code Analysis -#+DATE: 2025-10-30 - -* New Open Work - -** TODO [#A] Duplicate variable declaration in early-init.el :bug:architecture:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 76-78, 104-105 -:END: - -The variable =cj/network-available= is declared twice (lines 76-78 and 104-105). - -*Impact:* Confusing code structure, potential initialization issues. - -*Fix:* Remove the duplicate declaration at lines 104-105. The first declaration at line 76 is sufficient. - -** TODO [#A] Missing function definition in vc-config.el :bug:critical: -:PROPERTIES: -:FILE: modules/vc-config.el -:LINE: 123 -:END: - -The keymap binding references =cj/goto-git-gutter-diff-hunks= (line 123) but this function is never defined anywhere in the config. - -*Impact:* Keybinding =C-; v d= will fail with "Symbol's function definition is void" error. - -*Fix:* Either implement the missing function or remove the keybinding. Likely should use =git-gutter:popup-hunk= or create a wrapper that calls =consult-line= filtered to lines with git-gutter markers. - -** TODO [#A] Missing function in wrap-up.el :bug:critical: -:PROPERTIES: -:FILE: modules/wrap-up.el -:LINE: 28 -:END: - -Calls =cj/log-silently= but function is never defined in the config. - -*Impact:* Init will fail at startup with undefined function error. - -*Fix:* Define =cj/log-silently= in system-defaults or config-utilities, or replace with a simple =(message ...)= call. - -** TODO [#A] Network check runs synchronously during startup :performance:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 189, 206 -:END: - -The =cj/internet-up-p= function is called *synchronously* during early-init (lines 189, 206), which means Emacs startup blocks waiting for ping or network connection attempts. - -*Impact:* Adds 1+ seconds to every startup, even when network is available. Completely unacceptable UX. - -*Fix:* -1. Default =cj/network-available= to =t= (assume network is available) -2. Run network check asynchronously in background after startup -3. Only use network check when user explicitly runs =package-refresh-contents= -4. Consider removing network check entirely - let package.el handle its own errors - -Modern approach: Remove =cj/use-online-repos= flag entirely. Let package operations fail gracefully if network is down, rather than preemptively checking. - -** TODO [#A] Commented-out non-commented variable in org-config.el :bug:functionality: -:PROPERTIES: -:FILE: modules/org-config.el -:LINE: 129-130 -:END: - -Line 129 has a TASK comment about removing a non-existent variable =org-indent-indentation-per-level=, but line 130 actually tries to set it. - -*Impact:* Setting an undefined variable that does nothing. - -*Fix:* Remove lines 129-130 entirely if the variable doesn't exist. Verify in org-mode source whether this was renamed or removed. - -** TODO [#B] Replace Company with Corfu for better performance and maintenance :enhancement:architecture: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINES: 202-236 -:END: - -Currently using Company for completion, but Corfu is the more modern choice: - -*Why Corfu is better:* -- ~3x faster than Company (uses child frames vs overlays) -- Better maintained (company-mode development has slowed) -- Smaller, simpler codebase -- Works seamlessly with modern completion frameworks (Vertico/Consult) -- Native corfu-popupinfo is cleaner than company-quickhelp -- Better integration with Cape for completion-at-point backends - -*Migration path:* -1. Replace =company= with =corfu= + =corfu-popupinfo= -2. Use =cape= for additional backends (cape-file, cape-keyword, cape-dabbrev) -3. Keep company-mode disabled in mu4e (corfu already respects this) -4. Much simpler configuration, no company-box needed - -*Estimated effort:* 30 minutes. High value/low effort. - -** TODO [#B] Consolidate debug logging functions :architecture:maintainability: -:PROPERTIES: -:FILES: modules/org-agenda-config-debug.el, modules/wrap-up.el -:END: - -Debug logging is inconsistent: -- =org-agenda-config-debug.el= defines =cj/log-silently= -- =wrap-up.el= *calls* =cj/log-silently= but it's not loaded yet -- =config-utilities.el= has debug functions but not logging - -*Fix:* Create =modules/debug-logging.el= with consistent logging infrastructure: -- =cj/log-debug=, =cj/log-info=, =cj/log-warning=, =cj/log-error= -- Buffer-based logging with timestamps -- Easy toggle for debug mode -- Load early in init.el before other modules - -** TODO [#B] Missing error handling in org-agenda rebuild :stability: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:LINES: 91-105 -:END: - -=cj/build-org-agenda-list= recursively searches directories but has no error handling for: -- Permission denied errors -- Circular symlinks -- Very deep directory trees -- Network filesystem timeouts (if projects-dir is on NFS/CIFS) - -*Impact:* Agenda rebuild can hang or fail silently. - -*Fix:* Add condition-case wrapper with specific handlers: -#+begin_src elisp -(condition-case err - (cj/add-files-to-org-agenda-files-list projects-dir) - (file-error - (message "Warning: Could not scan %s: %s" projects-dir err)) - (error - (message "Error building agenda: %s" err))) -#+end_src - -** TODO [#B] Unify path handling functions :architecture:maintainability: -:PROPERTIES: -:FILES: modules/dirvish-config.el, modules/custom-buffer-file.el -:END: - -Path handling is duplicated across modules: -- =cj/get-project-root= in dirvish-config.el (lines 408-426) -- Similar logic needed in custom-buffer-file for relative paths -- Different approaches to home directory abbreviation - -*Fix:* Create =modules/path-utils.el= with: -- =cj/get-project-root= (shared implementation) -- =cj/abbreviate-path= (handles ~, project-relative, absolute) -- =cj/path-relative-to= (generalized relative path function) -- Consistent path handling across all modules - -** TODO [#B] Treesit-auto grammar pinning is fragile :stability:maintenance: -:PROPERTIES: -:FILE: modules/prog-general.el -:LINES: 106-113 -:END: - -Go grammar is pinned to v0.19.1 by directly modifying the treesit-auto-recipe-list vector with =aset=. - -*Issues:* -- Hardcoded slot index (6) will break if treesit-auto changes struct layout -- No validation that the recipe exists or that slot 6 is :revision -- Comment says "Emacs 30.2" but doesn't check version -- Will fail silently if treesit-auto changes - -*Fix:* -1. Use proper accessor functions if available (treesit-auto-recipe-revision setter) -2. Add version check: =(when (version< emacs-version "30.3") ...)= -3. Add error handling and warning if pinning fails -4. Document why pinning is needed in comments - -** TODO [#B] Test coverage gaps in critical modules :testing:quality: -:PROPERTIES: -:FILES: modules/org-agenda-config.el, modules/mail-config.el, modules/ai-config.el -:END: - -Excellent test coverage for utility functions (custom-*, undead-buffers, org-roam helpers), but missing tests for: - -*High-value untested areas:* -1. =org-agenda-config.el=: - - =cj/build-org-agenda-list= (file discovery logic) - - =cj/org-skip-subtree-if-*= functions (agenda filtering) - - =cj/org-agenda-skip-subtree-if-not-overdue= (date logic) - -2. =mail-config.el=: - - =cj/mu4e-toggle-remote-images= (security critical) - - Email header parsing/address completion logic - -3. =ai-config.el=: - - =cj/gptel-add-file= (file path handling) - - =cj/gptel--fresh-org-prefix= (timestamp formatting) - -4. =org-roam-config.el=: - - =cj/move-org-branch-to-roam= (complex string manipulation) - - Already has tests for helpers, needs integration test - -*Recommendation:* Prioritize testing the org-agenda filtering functions - they're date-sensitive and easy to break. - -** TODO [#B] System-defaults autoload has wrong function name :bug:minor: -:PROPERTIES: -:FILE: modules/system-defaults.el -:LINE: 24 -:END: - -Line 24 has =autoload= for =env-bsd-p= from =host-environment=, but this file *requires* host-environment at compile time anyway (line 20). - -*Impact:* Redundant autoload, no functional issue. - -*Fix:* Remove the autoload statement. The eval-when-compile require is sufficient. - -** TODO [#B] Inconsistent coding system declarations :maintainability: -:PROPERTIES: -:FILES: early-init.el, modules/system-defaults.el -:END: - -UTF-8 coding system is set in two places: -- =early-init.el= lines 252-261 (before package loading) -- =system-defaults.el= lines 60-69 (after package loading) - -*Impact:* Redundant configuration, harder to maintain, unclear which takes precedence. - -*Fix:* Consolidate all coding system setup in early-init.el since it needs to be set before packages load. Remove from system-defaults.el. - -** TODO [#B] Chime notification times documentation mismatch :documentation:usability: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:LINES: 268-270 -:END: - -Comment says "5 minutes before and at event time (0 minutes)" but the value is ='(5 0)=. - -*Issue:* It's unclear if 0 means "at event time" or "immediately". Better documentation needed. - -*Fix:* Improve comment: -#+begin_src elisp -;; Alert 5 minutes before event and again at event time (0 = event start) -(setq chime-alert-time '(5 0)) -#+end_src - -** TODO [#C] Org-roam template paths use string concatenation :maintainability: -:PROPERTIES: -:FILE: modules/org-roam-config.el -:LINES: 43, 48, 53, 144, 150 -:END: - -Template file paths are built with =concat= instead of using =expand-file-name= or constants. - -*Impact:* Harder to change template location, no validation that files exist. - -*Fix:* Define template constants at top of file: -#+begin_src elisp -(defconst cj/org-roam-template-dir - (expand-file-name "org-roam-templates" user-emacs-directory)) -(defconst cj/org-roam-template-v2mom - (expand-file-name "v2mom.org" cj/org-roam-template-dir)) -;; etc... -#+end_src - -Validate files exist during config load. - -** TODO [#C] Dirvish-config has long function that should be split :refactor:maintainability: -:PROPERTIES: -:FILE: modules/dirvish-config.el -:LINES: 350-406 -:END: - -=cj/dired-copy-path-as-kill= is 57 lines with multiple responsibilities: -- Validation -- Project root detection -- Home directory handling -- Org-link formatting -- Clipboard operations -- User feedback - -*Fix:* Extract smaller functions: -- =cj/--path-type-for-file= (returns :project-relative, :home-relative, or :absolute) -- =cj/--format-as-org-link= (wraps path in org syntax) -- =cj/dired-copy-path-as-kill= (orchestrates the above) - -Better testability, clearer logic flow. - -** TODO [#C] Consider Embark for file operations instead of custom dirvish bindings :enhancement:usability: -:PROPERTIES: -:FILE: modules/dirvish-config.el -:LINES: 298-334 -:END: - -Dirvish config defines many custom keybindings (d, l, L, o, O, f, P) for file operations. - -*Alternative approach:* Use Embark for contextual file actions: -- Embark already knows about files in dired -- Can define actions once, use everywhere (not just dired) -- More discoverable (embark-act shows all options) -- Less cognitive load (one keybinding to remember: C-.) - -*Recommendation:* Keep the custom functions (good implementations), but expose them via embark-act instead of direct keybindings. Cleaner keymap, more flexible. - -** TODO [#C] Missing :ensure nil for more built-in packages :correctness:minor: -:PROPERTIES: -:FILE: Multiple modules -:END: - -Some built-in package use-package declarations are missing =:ensure nil=: -- =selection-framework.el=: =delsel= (line 54) -- Several others throughout - -*Impact:* Minimal - use-package won't try to install built-ins anyway. But inconsistent with the pattern used elsewhere. - -*Fix:* Add =:ensure nil= for consistency, or document why some have it and others don't. - -** TODO [#C] Consider lazy-loading more packages :performance:startup: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINES: 22, 40, 48, 151, 166, 240-253 -:END: - -Several completion packages use =:demand t= (vertico, marginalia, orderless, prescient): -- Vertico: Reasonable to demand (core UI) -- Marginalia: Could be :after vertico -- Orderless: Could be :after vertico -- Prescient: Could be :after vertico -- Company-prescient: Could be :after (vertico company) - -*Impact:* All load at startup, adds ~100ms. Not critical, but could be optimized. - -*Fix:* Profile with esup to see actual impact. If it's <100ms, probably not worth the complexity. - -** TODO [#C] Add validation for chime.el local path :stability:enhancement: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:LINE: 264 -:END: - -Config loads chime from local directory =~/code/chime.el= without checking if it exists. - -*Impact:* If directory doesn't exist, load fails silently or with cryptic error. - -*Fix:* Add existence check with helpful error message: -#+begin_src elisp -:load-path (lambda () - (let ((path "~/code/chime.el")) - (if (file-directory-p path) - path - (warn "chime.el not found at %s" path) - nil))) -#+end_src - -** TODO [#C] Document why org-msg uses local fork :documentation:maintenance: -:PROPERTIES: -:FILE: modules/mail-config.el -:LINE: 287 -:END: - -Loading org-msg from local path =/home/cjennings/code/org-msg= instead of MELPA. - -*Question:* Is this a temporary fork with patches? Does it need to stay local? - -*Fix:* Add comment explaining: -- What changes are in local fork -- Whether it's temporary or permanent -- Link to upstream PR if patches are submitted -- When it can switch back to MELPA - -** TODO [#C] Performance: Company idle-delay is very high :performance:usability: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINE: 212 -:END: - -Company idle-delay is set to 2 seconds, which is quite slow. Modern editors typically use 0.1-0.3s. - -*Impact:* Completion feels sluggish, requires manual triggering often. - -*Recommendation:* -- If keeping Company: Try 0.3 seconds -- If switching to Corfu: Default 0.1s is perfect - -User preference, but current setting feels unresponsive. - -** TODO [#C] org-protocol may not be fully configured :functionality:enhancement: -:PROPERTIES: -:FILE: modules/org-config.el -:LINES: 94-95, 116 -:END: - -=org-protocol= is loaded (line 95) and added to org-modules (line 116), but: -- No desktop entry file mentioned -- No documentation on how to use it -- org-webclipper exists but unclear if it uses org-protocol - -*Fix:* Either: -1. Document org-protocol setup (desktop file, browser integration) -2. Remove if unused (org-webclipper might be the preferred approach) - -** TODO [#D] Consider adding use-package :diminish for minor modes :enhancement:ui: -:PROPERTIES: -:FILE: Multiple modules -:END: - -Many minor modes appear in the modeline even though doom-modeline hides them. - -*Enhancement:* Add =:diminish= to use-package declarations for: -- =symbol-overlay-mode= -- =git-gutter-mode= -- =flycheck-mode= -- =yas-minor-mode= -- =ws-butler-mode= -- =rainbow-mode= - -Cleaner modeline if doom-modeline isn't used or fails to load. - -** TODO [#D] Add .dir-locals.el for project-specific settings :enhancement:devex: -:PROPERTIES: -:FILE: N/A (new file) -:END: - -Create =.dir-locals.el= in =~/.emacs.d= to: -- Set =fill-column= to 80 for elisp files -- Enable =flycheck-mode= for all elisp -- Set =checkdoc= settings locally -- Configure =outline-minor-mode= for easy navigation - -Helps maintain consistency when editing config. - -** TODO [#D] Consider moving from doom-modeline to mood-line :enhancement:performance: -:PROPERTIES: -:FILE: modules/modeline-config.el -:END: - -Doom-modeline is feature-rich but heavy. For the minimal info you're displaying, mood-line might be better: - -*Mood-line advantages:* -- ~10x faster than doom-modeline -- Pure elisp (no external dependencies) -- Very simple, hackable codebase -- Shows exactly what you need: filename, position, major-mode -- No nerd-icons dependency (can still use them if wanted) - -*Doom-modeline advantages:* -- More battle-tested -- Better git integration -- More features (even if disabled) - -*Recommendation:* Try mood-line. If you miss doom-modeline features, easy to switch back. Performance difference is noticeable on older hardware. - -** TODO [#D] Add commentary headers to test utility files :documentation:testing: -:PROPERTIES: -:FILES: tests/testutil-general.el, tests/testutil-filesystem.el -:END: - -Test utility files should have Commentary sections explaining: -- What utilities are provided -- When to use each utility -- Examples of usage -- Relationship to quality-engineer.org guidelines - -Helps future test authors understand available infrastructure. - -** TODO [#D] keybindings.el trains user away from C-x u :ux:philosophy: -:PROPERTIES: -:FILE: modules/keybindings.el -:LINES: 126-130 -:END: - -The rebinding of =C-x u= to a message telling users to use =C-/= instead is... philosophically interesting. - -*Discussion points:* -- Enforces good habits (C-/ is indeed faster) -- But violates principle of least surprise -- Could frustrate users who know Emacs bindings -- Muscle memory is hard to retrain - -*Alternative:* Just leave =C-x u= working. Add which-key hint that C-/ is faster. - -** TODO [#D] Missing which-key labels for some custom keymaps :usability:discoverability: -:PROPERTIES: -:FILES: Multiple modules -:END: - -Some custom keymaps lack which-key descriptions: -- =cj/vc-map= is well documented (vc-config.el) -- =cj/jump-map= is well documented (keybindings.el) -- =cj/org-table-map= is well documented (org-config.el) - -But check all custom keymaps for completeness. Run: -#+begin_src elisp -(apropos-variable "^cj/.*-map$") -#+end_src - -And verify each has which-key labels. - -** TODO [#D] early-init.el could benefit from more section comments :documentation:maintainability: -:PROPERTIES: -:FILE: early-init.el -:END: - -File has good top-level commentary but individual sections could use more context: -- Why network check exists (explain offline package strategy) -- Why GC threshold is increased (explain impact) -- Benchmark-init section is good, replicate that style - -Makes it easier for others (or future you) to understand decisions. - -** TODO [#D] Consider adding persistent-scratch package :enhancement:usability: -:PROPERTIES: -:FILE: N/A -:END: - -The *scratch* buffer is very useful but loses content on restart. - -*Enhancement:* Add =persistent-scratch= package to: -- Auto-save scratch buffer contents -- Restore on startup -- Optionally persist multiple scratch buffers - -Useful for quick notes, code experiments, calculations that span sessions. - -* Priority Explanation - -** Priority A (Critical) -Issues that: -- Prevent functionality from working -- Cause errors or crashes -- Block startup or major features -- Have security implications -- Significantly impact performance (>1s delay) - -** Priority B (High) -Issues that: -- Affect architecture or maintainability significantly -- Better alternative packages exist -- Moderate performance impact (>100ms) -- Testing gaps in critical code -- Risk of future breakage - -** Priority C (Medium) -Issues that: -- Affect code quality or consistency -- Minor refactoring opportunities -- Documentation improvements -- Small usability enhancements -- Package optimization opportunities - -** Priority D (Low) -Issues that: -- Nice-to-have features -- Polish and refinement -- Alternative approaches to consider -- Documentation expansion -- Style/convention improvements - -* Package Recommendations Summary - -** Recommended Changes -1. *Company → Corfu*: Better performance, more modern, better maintained -2. *Consider mood-line over doom-modeline*: Much faster for minimal info needs -3. *Add Cape*: Modern completion-at-point backends (if switching to Corfu) -4. *Consider persistent-scratch*: QOL improvement for scratch buffer - -** Packages to Keep -- Vertico: Excellent choice, fast, well-maintained -- Consult: Perfect companion to Vertico -- Orderless: Best completion style available -- Embark: Powerful, could be used more extensively -- Marginalia: Great for annotations -- Magit: Industry standard, no alternatives -- Org-roam: Best Zettelkasten for Emacs -- Which-key: Essential for discoverability - -** Packages to Evaluate -- Dirvish: Heavy package. Consider if dired + nerd-icons + embark is sufficient -- Doom-modeline: Feature-rich but heavy. Mood-line is lighter alternative -- Projectile vs project.el: You're using Projectile. Project.el is built-in and lighter, but Projectile is more mature. Keep unless performance is an issue. - -* Testing Recommendations - -Based on your quality-engineer.org guidelines and existing test coverage: - -** Well-Tested Areas ✓ -- All custom-* utility functions (excellent coverage) -- undead-buffers (comprehensive tests) -- org-roam helpers (good coverage) -- keyboard-macros -- Test utilities themselves - -** Priority Testing Gaps - -*** High Priority -1. *org-agenda-config.el filtering functions* - - =cj/org-skip-subtree-if-habit= - - =cj/org-skip-subtree-if-not-overdue= (date-sensitive!) - - =cj/build-org-agenda-list= (file discovery) - -2. *custom-buffer-file.el interactive functions* - - Split =cj/move-buffer-and-file= into =cj/--move-buffer-and-file= (testable core) and interactive wrapper - - Same for =cj/rename-buffer-and-file= - - Same for =cj/delete-buffer-and-file= - -*** Medium Priority -3. *org-roam-config.el integration test* - - =cj/move-org-branch-to-roam= workflow test - - Test actual file creation and org-roam db update - -4. *path-utils consolidation* (after creating module) - - Test project-relative, home-relative, absolute path logic - - Test with symlinks, non-existent paths, permission errors - -5. *ai-config.el GPTel context management* - - =cj/gptel-add-file= with various file types - - =cj/gptel--fresh-org-prefix= timestamp formatting - -*** Lower Priority -6. *mail-config.el address completion* -7. *Browser/external open logic* -8. *Theme persistence functions* - -** Testing Patterns to Follow - -Your existing tests follow excellent patterns from quality-engineer.org: -- One file per function for unit tests -- Clear normal/boundary/error case organization -- Good use of test utilities (testutil-general, testutil-filesystem) -- Proper setup/teardown -- Descriptive test names - -*Keep this pattern!* It's working well for you. - -** Test-Driven Refactoring Opportunities - -When fixing Priority A-B issues, write tests first: -1. Test for the bug/missing function -2. See it fail -3. Fix the code -4. See it pass -5. Refactor with confidence - -This is especially important for: -- The missing =cj/goto-git-gutter-diff-hunks= function -- The duplicate =cj/network-available= variable -- Network check refactoring (mock the network!) - -* Architecture Observations - -** What You're Doing Right ✓ - -1. *Excellent module organization*: Clean separation of concerns -2. *Good use of user-constants.el*: Centralized configuration -3. *Comprehensive custom utilities*: Well-tested, reusable functions -4. *Modern completion framework*: Vertico/Consult/Embark stack is excellent -5. *Debug infrastructure pattern*: org-agenda-config-debug.el is a good model -6. *Test coverage philosophy*: Following your own guidelines well -7. *Early-init optimization*: Good understanding of startup optimization -8. *Custom keymap prefix*: C-; is a good choice, well-organized - -** Architectural Patterns to Consider - -1. *Consolidate common patterns*: - - Path handling → path-utils.el - - Debug logging → debug-logging.el - - File validation → file-utils.el - -2. *Reduce coupling*: - - Some modules depend on specific implementations (e.g., projectile vs project.el) - - Consider adapter pattern for switching between implementations - -3. *Error handling strategy*: - - Add condition-case wrappers around file operations - - Log errors consistently to *Messages* or debug buffer - - Fail gracefully (warn user but don't break session) - -4. *Configuration validation*: - - Add startup checks for critical paths/files - - Warn about missing optional dependencies - - Provide helpful error messages with solutions - -** Code Smells Observed - -1. *String concatenation for paths*: Use =expand-file-name= consistently -2. *Silent failures*: Some functions fail without user feedback -3. *Synchronous network operations*: Blocks startup unnecessarily -4. *Duplicate logic*: Path handling, project root detection -5. *Large functions*: Some functions >50 lines, should be split -6. *Global state*: Some functions modify global variables without clear ownership - -* Next Steps Recommendation - -** Week 1: Fix Critical Bugs (Priority A) -1. Fix duplicate =cj/network-available= declaration -2. Implement missing =cj/goto-git-gutter-diff-hunks= -3. Define =cj/log-silently= or remove usage -4. Fix network check blocking (biggest impact) - -** Week 2: Architectural Improvements (Priority B) -1. Evaluate Company vs Corfu (try Corfu for a week) -2. Consolidate debug logging -3. Create path-utils.el module -4. Add error handling to org-agenda rebuild - -** Week 3: Testing (Priority B) -1. Add tests for org-agenda filtering functions -2. Split interactive/non-interactive in custom-buffer-file -3. Add integration test for org-roam branch extraction - -** Week 4: Polish (Priority C-D) -1. Consolidate coding system setup -2. Improve documentation -3. Add which-key labels where missing -4. Clean up minor issues - -** Ongoing -- Add tests for any new functionality -- Profile startup time with esup periodically -- Review package updates for breaking changes -- Keep org-gcal, chime.el forks in sync with upstream - -* Questions for You - -1. *Company vs Corfu*: Do you have strong feelings about Company? Willing to try Corfu? - Answer: I'd be excited to try Corfu. I don't know how I'd configure it yet and it seems like a big change. However, I planned to move to Corfu sometime soon. - -2. *Network check*: Do you actually need to support offline operation, or can we simplify this? - Answer: We can simplify. There was a time when I was traveling a lot and didn't have network access. The current offline operation seemed to help startup performance. The sooner we realized we were offline, the sooner I'd have a working Emacs. Now, I don't think it provides much benefit. - -3. *Local forks*: Are chime.el and org-msg local forks temporary? When can they move to MELPA? - Answer: These are packages I currently either maintain or are in the process of creating. I need the local-paths set so I can test. If you have better ideas on how I can switch them from using a local package to using the stable version using vc-install, I'd be grateful. - -4. *Doom-modeline*: Are you using features beyond what I saw in config? Would mood-line be sufficient? - No, I'm not. Mood-line would be welcomed. - -5. *Testing coverage*: What modules do you most want tests for? Anything you're nervous about breaking? - I'll probably look for large methods with lots of LOC and multiple tasks, then break them up for better testability. I was hoping you would point those out for me so I could focus on them. Will you? - -6. *Performance*: Any particular operations feel slow? (Agenda rebuild? Mail sync? Completion?) - Org Agenda is problematic and slow. I could get some insights into how to optimize it if it were well tested. - - -Other than that, here's my roadmap for fixing/improving -- video/audio config could use some testing around which sinks to attach to. It doesn't matter to me that this is Linux centric since it's my config. -- I'm not sure the UX in flyspell and abbrev is right. It could use some more attention. -- Same with Elfeed. While I had Elfeed dashboard at some point, it doesn't bring any joy to see a mostly blank screen when I check Elfeed. -- DWIM shell commands is alright. I wonder if I can leverage them better into eshell. -- calibredb is slow, but that's the package working on 14k+ pdfs and epubs. The right thing to do is to shrink the ebook inventory to < 1k or so books. -- I was thinking of investigating bettter diffs with difftastic. If it's straightforward and simple, I should jump at it. -- I would like to implement code-maat and code-compass within Emacs -- It would be good to have a solid profiler workflow so I can see what's going on inside my own methods and packages. For instance, I don't have the tools to tell me why it takes so long to generate my daily agenda. There's little observability. -- I absolutely need a workflow so I can go from org-mode file to reveal.js presentation using httpd server in Emacs with the press of a button. I know it can be done. Export the org file with injected css into the httpd directory, start the httpd server, then open a browser fullscreen on the file. That's pretty much 90% of the work right there. -- If I could figure out the latex-config so that I could author something non-trivial, I'd be happy. Right now, it's a bunch of cut/paste snippets from everyone else's config and it doesn't work. -- mail-config: the whole attachments workflow is awkward, but so necessary for daily life. It really needs improvement. -- then there's the whole org-contacts issue working with chime.el and birthdays. sexp expressions for dates? I should either figure them out and support them in chime, or I should disable them and find another way. -- it would be good to work with llms the way I do with claude code. gptel feels like a step in that direction and you can switch backends besides. but I need to either build a bunch of tools or install an MCP server. Which way to go? -- I would like to extract the code in local-repository that creates the repo so I can unit test the hell out of it and use it as a library for my own local repo. Seems like there's too much extraneous functionality there. -- lorem seems to have an error pop up each session about the markhov chain resetting. either that's bad and should be fixed or it's not bad and lorem should stfu. -- I believe jumper should probably be my next package, but I haven't had the time to spend on it. - -* Final Thoughts - -Your config is **quite good overall**: -- Well-organized modular structure -- Excellent test coverage for utilities -- Modern package choices (mostly) -- Good documentation and comments -- Following elisp idioms reasonably well - -The critical issues (Priority A) are straightforward fixes. The architectural suggestions (Priority B) are about making the codebase more maintainable long-term. Everything else is refinement. - -Focus on: -1. Fix the bugs that break functionality -2. Consider Corfu migration (biggest ROI for effort) -3. Add tests for date-sensitive agenda logic -4. Everything else is gravy - -You're clearly thoughtful about your config and willing to refactor when needed. These issues are normal for a mature, actively-used configuration. Keep up the good work! - -* Second Opinion: Ruthless Prioritization & Reality Checks -:PROPERTIES: -:DATE: 2025-10-30 -:END: - -After reviewing your answers and additional roadmap items, here's honest feedback on what matters, what doesn't, and what you should actually do. - -** Key Insights from Your Answers - -*** Company → Corfu Migration: GREEN LIGHT -You're excited, you were planning it anyway. Corfu configuration is actually *simpler* than Company, not harder. This is a quick win with immediate UX improvement. - -*Reality:* 2 hours of work, noticeable performance improvement. Do it this weekend. - -*** Network Check: DELETE IT IMMEDIATELY -You confirmed it's technical debt from when you traveled. It's adding 1+ seconds to every startup for a problem you no longer have. - -*Reality:* 15 minutes to delete ~100 lines of code. Instant gratification. Do this first. - -*** Local Package Development Workflow: NEEDS PATTERN -You maintain chime.el and org-msg locally for testing. Current approach works but isn't elegant. See implementation pattern below. - -*** Mood-line Switch: EASY WIN -You don't need doom-modeline features. Mood-line is 10x faster and simpler. - -*Reality:* 30 minutes. Do it. - -*** Large Functions to Break Up: MOSTLY FINE -Analyzed your codebase. Most functions are reasonably sized. The ones you mentioned: -- =dirvish-config.el:350-406= =cj/dired-copy-path-as-kill= (57 lines) - Could split, but works fine -- =org-roam-config.el:250-298= =cj/move-org-branch-to-roam= (49 lines) - Actually well-designed with helpers -- =org-agenda-config.el:91-105= =cj/build-org-agenda-list= (15 lines) - Small but THIS IS YOUR PERFORMANCE BOTTLENECK - -*Reality:* Your code organization is fine. Your problem is *performance observability*, not function length. - -*** Org Agenda Performance: ROOT CAUSE IDENTIFIED -This is your actual pain point. But you're flying blind without profiling infrastructure. - -*Hypothesis:* =cj/build-org-agenda-list= recursively scans entire projects directory on every agenda open with: -- No caching (even when files unchanged) -- No error handling -- Synchronous file I/O blocking UI -- Probably hundreds of file-exists-p calls - -*Reality:* You need profiling tools FIRST, then optimization. Don't guess, measure. - -** TIER 1: Do These First (High Impact, Low Effort) -*Time investment: One weekend* -*Status: START HERE* - -*** TODO [#A] Remove network check entirely :performance:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 76-78, 104-105, 189, 206 -:EFFORT: 15 minutes -:IMPACT: Removes 1+ seconds from every startup -:END: - -Delete the following: -1. =cj/network-available= variable declarations (both) -2. =cj/internet-up-p= function and all calls -3. =cj/use-online-repos= flag logic -4. Any package.el conditionals based on network status - -Let package.el handle its own errors gracefully. Modern Emacs does this fine. - -*** TODO [#A] Fix missing cj/log-silently definition :bug:critical: -:PROPERTIES: -:FILE: modules/wrap-up.el -:LINE: 28 -:EFFORT: 5 minutes -:END: - -Either: -1. Move =cj/log-silently= from =org-agenda-config-debug.el= to =config-utilities.el= (load early) -2. Or replace with simple =(message ...)= call - -The function is only used in one place, so simple message might be sufficient. - -*** TODO [#A] Remove duplicate cj/network-available declaration :bug:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 104-105 -:EFFORT: 2 minutes -:END: - -Delete the duplicate at lines 104-105. Keep only the first declaration at line 76. - -*Note:* This becomes moot if you delete network check entirely (recommended). - -*** TODO [#A] Fix missing cj/goto-git-gutter-diff-hunks function :bug:critical: -:PROPERTIES: -:FILE: modules/vc-config.el -:LINE: 123 -:EFFORT: 15 minutes -:END: - -Keybinding =C-; v d= references undefined function. - -Implement as: -#+begin_src elisp -(defun cj/goto-git-gutter-diff-hunks () - "Jump to git-gutter diff hunks using consult." - (interactive) - (require 'git-gutter) - (consult-line "^[+\\-]")) ;; Or use git-gutter:popup-hunk -#+end_src - -Or remove the keybinding if not needed. - -*** TODO [#B] Migrate Company to Corfu :enhancement:performance: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINES: 202-236 -:EFFORT: 2 hours -:IMPACT: 3x faster completion, simpler config -:END: - -*Why Corfu:* -- ~3x faster (child frames vs overlays) -- Better maintained -- Simpler codebase (~500 LOC vs ~3000 LOC) -- Works seamlessly with Vertico/Consult -- Better integration with Cape for backends - -*Migration:* -#+begin_src elisp -(use-package corfu - :ensure t - :demand t - :bind (:map corfu-map - ("TAB" . corfu-next) - ("S-TAB" . corfu-previous)) - :custom - (corfu-auto t) - (corfu-auto-delay 0.1) ;; Much faster than your 2s company delay! - (corfu-auto-prefix 2) - (corfu-quit-no-match 'separator) - :init - (global-corfu-mode)) - -(use-package corfu-popupinfo - :after corfu - :hook (corfu-mode . corfu-popupinfo-mode) - :custom - (corfu-popupinfo-delay '(0.5 . 0.2))) - -(use-package cape - :ensure t - :after corfu - :init - (add-to-list 'completion-at-point-functions #'cape-dabbrev) - (add-to-list 'completion-at-point-functions #'cape-file) - (add-to-list 'completion-at-point-functions #'cape-keyword)) -#+end_src - -Remove entire company configuration. Keep mu4e completion settings (Corfu respects them). - -*Configuration is actually simpler than Company.* Just do it. - -*** TODO [#B] Switch doom-modeline to mood-line :enhancement:performance: -:PROPERTIES: -:FILE: modules/modeline-config.el -:EFFORT: 30 minutes -:IMPACT: 10x faster modeline updates -:END: - -Replace doom-modeline config with: -#+begin_src elisp -(use-package mood-line - :ensure t - :demand t - :config - (mood-line-mode)) -#+end_src - -That's it. Mood-line shows: buffer name, position, major-mode, vc status. Pure elisp, very fast. - -If you miss doom-modeline features, easy to switch back. But try mood-line for a week. - -*** TODO [#C] Fix lorem markov chain error :bug:minor: -:PROPERTIES: -:FILE: modules/lorem-config.el (or wherever lorem is configured) -:EFFORT: 5 minutes -:END: - -The error message is probably harmless initialization noise. Either: -1. Wrap lorem initialization in =(with-demoted-errors ...)= -2. Configure lorem to suppress the warning -3. Disable lorem if you don't use it - -Check your config for lorem setup and add error suppression. - -** TIER 2: Build Observability Infrastructure (HIGHEST VALUE) -*Time investment: One week* -*Status: CRITICAL DEPENDENCY FOR EVERYTHING ELSE* - -*** TODO [#A] Create debug-profiling.el module :infrastructure:performance: -:PROPERTIES: -:FILE: modules/debug-profiling.el (new file) -:EFFORT: 3-4 hours -:IMPACT: Unlocks all performance optimization work -:END: - -This is your #1 blocker for fixing org-agenda and other performance issues. - -Create comprehensive profiling infrastructure: - -#+begin_src elisp -;;; debug-profiling.el --- Profiling and performance debugging tools -*- lexical-binding: t; -*- - -;;; Commentary: -;; Unified profiling infrastructure for measuring and optimizing performance. -;; Provides helpers for: -;; - Function timing (individual and batch) -;; - Memory profiling -;; - Startup profiling -;; - Interactive profiling sessions -;; -;; Usage: -;; M-x cj/profile-function RET cj/build-org-agenda-list RET -;; M-x cj/profile-startup -;; (cj/time-it "agenda rebuild" (cj/build-org-agenda-list)) - -;;; Code: - -(require 'profiler) -(require 'benchmark) - -(defvar cj/profiling-log-buffer "*CJ Profile Log*" - "Buffer name for profiling results.") - -(defvar cj/profiling-enabled nil - "When non-nil, enable automatic timing of instrumented functions.") - -(defun cj/time-it (description &rest body) - "Execute BODY and log timing with DESCRIPTION." - (let* ((start-time (current-time)) - (result (car body)) - (elapsed (float-time (time-subtract (current-time) start-time)))) - (with-current-buffer (get-buffer-create cj/profiling-log-buffer) - (goto-char (point-max)) - (insert (format "[%s] %s: %.3f sec\n" - (format-time-string "%H:%M:%S") - description - elapsed))) - (message "%s: %.3f sec" description elapsed) - result)) - -(defmacro cj/measure (&rest body) - "Measure execution time of BODY, return result and timing." - `(let ((start-time (current-time)) - (result (progn ,@body)) - (end-time (current-time))) - (cons result (float-time (time-subtract end-time start-time))))) - -(defun cj/profile-function (function &optional runs) - "Profile FUNCTION by calling it RUNS times (default 1). -Shows results in dedicated buffer with call tree." - (interactive - (list (intern (completing-read "Function to profile: " obarray 'fboundp t)) - (when current-prefix-arg - (read-number "Number of runs: " 10)))) - (setq runs (or runs 1)) - (profiler-start 'cpu) - (dotimes (_ runs) - (funcall function)) - (profiler-stop) - (profiler-report) - (message "Profiled %s (%d run%s)" function runs (if (> runs 1) "s" ""))) - -(defun cj/profile-startup () - "Profile Emacs startup by restarting with profiler enabled. -Results are saved to profile.el in user-emacs-directory." - (interactive) - (let ((profile-file (expand-file-name "startup-profile.el" user-emacs-directory))) - (profiler-start 'cpu+mem) - (message "Restart Emacs to profile startup. Results will be in %s" profile-file) - (add-hook 'emacs-startup-hook - (lambda () - (profiler-report) - (profiler-report-write-profile profile-file) - (profiler-stop) - (message "Startup profiling complete. See %s" profile-file))))) - -(defun cj/benchmark-function (function &optional iterations) - "Benchmark FUNCTION over ITERATIONS (default 100). -Shows average, min, max, and total time." - (interactive - (list (intern (completing-read "Function to benchmark: " obarray 'fboundp t)) - (read-number "Iterations: " 100))) - (setq iterations (or iterations 100)) - (let ((times '())) - (dotimes (_ iterations) - (let* ((result (cj/measure (funcall function))) - (elapsed (cdr result))) - (push elapsed times))) - (let* ((total (apply #'+ times)) - (avg (/ total iterations)) - (min-time (apply #'min times)) - (max-time (apply #'max times))) - (message "%s (%d iterations): avg=%.4fs min=%.4fs max=%.4fs total=%.2fs" - function iterations avg min-time max-time total)))) - -(defun cj/instrument-package (package) - "Instrument all functions in PACKAGE for profiling with elp." - (interactive - (list (intern (completing-read "Package to instrument: " obarray)))) - (elp-instrument-package (symbol-name package)) - (message "Instrumented package: %s. Run M-x elp-results to see data." package)) - -(defun cj/profile-agenda-rebuild () - "Profile org-agenda rebuild with detailed timing." - (interactive) - (require 'org-agenda) - (cj/time-it "Full agenda rebuild" (cj/build-org-agenda-list))) - -(defun cj/show-profile-log () - "Display the profiling log buffer." - (interactive) - (display-buffer (get-buffer-create cj/profiling-log-buffer))) - -(defun cj/clear-profile-log () - "Clear the profiling log buffer." - (interactive) - (with-current-buffer (get-buffer-create cj/profiling-log-buffer) - (erase-buffer))) - -;; Keybindings (add to your keybindings.el) -;; (global-set-key (kbd "C-; p f") #'cj/profile-function) -;; (global-set-key (kbd "C-; p b") #'cj/benchmark-function) -;; (global-set-key (kbd "C-; p l") #'cj/show-profile-log) -;; (global-set-key (kbd "C-; p a") #'cj/profile-agenda-rebuild) - -(provide 'debug-profiling) -;;; debug-profiling.el ends here -#+end_src - -Load this early in init.el: -#+begin_src elisp -(when (or (eq cj/debug-modules t) - (memq 'profiling cj/debug-modules)) - (require 'debug-profiling (expand-file-name "modules/debug-profiling" user-emacs-directory))) -#+end_src - -*** TODO [#A] Profile org-agenda-rebuild to find bottleneck :performance:critical: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:EFFORT: 1 hour -:DEPENDS: debug-profiling.el -:END: - -Once profiling infrastructure exists: - -1. Run: =M-x cj/profile-function RET cj/build-org-agenda-list RET= -2. Run: =M-x cj/benchmark-function RET cj/build-org-agenda-list RET 10= -3. Instrument org-agenda package: =M-x cj/instrument-package RET org-agenda RET= -4. Review results with =M-x elp-results= - -*Hypothesis to test:* -- Is time spent in file I/O (directory-files-recursively)? -- Is it parsing org files? -- Is it just too many files? -- Is there a slow helper function being called repeatedly? - -*Expected findings:* -- Probably 80%+ time in =cj/add-files-to-org-agenda-files-list= -- Likely no caching, runs every time -- Possible N+1 file access pattern - -*Next steps after profiling:* -- Add file-notify watchers to invalidate cache only when projects-dir changes -- Cache the file list -- Run rebuild async on startup - -*** TODO [#B] Add instrumentation to cj/build-org-agenda-list :performance:observability: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:EFFORT: 30 minutes -:DEPENDS: debug-profiling.el, profiling results -:END: - -After profiling identifies bottleneck, add instrumentation: - -#+begin_src elisp -(defun cj/build-org-agenda-list () - "Rebuilds the org agenda list with instrumentation." - (interactive) - (let ((start-time (current-time))) - ;; reset org-agenda-files - (setq org-agenda-files (list inbox-file schedule-file gcal-file)) - - ;; Time the expensive operation - (let ((scan-start (current-time))) - (cj/add-files-to-org-agenda-files-list projects-dir) - (message "Scanned projects-dir in %.3f sec" - (float-time (time-subtract (current-time) scan-start)))) - - (let ((total-time (float-time (time-subtract (current-time) start-time)))) - (message "Rebuilt org-agenda-files in %.3f sec (%d files)" - total-time - (length org-agenda-files))))) -#+end_src - -This gives you observability on every agenda rebuild. - -*** TODO [#B] Add caching to org-agenda rebuild :performance:critical: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:EFFORT: 2 hours -:DEPENDS: Profiling results -:END: - -*Only do this after profiling confirms file scanning is the bottleneck.* - -Implement caching with file-notify: - -#+begin_src elisp -(defvar cj/org-agenda-cache nil - "Cached list of org-agenda files.") - -(defvar cj/org-agenda-cache-dirty t - "When non-nil, agenda cache needs rebuilding.") - -(defun cj/invalidate-agenda-cache (&rest _) - "Mark agenda cache as dirty." - (setq cj/org-agenda-cache-dirty t)) - -(defun cj/build-org-agenda-list-cached () - "Build org-agenda list with caching." - (interactive) - (when (or (not cj/org-agenda-cache) - cj/org-agenda-cache-dirty) - (let ((start-time (current-time))) - (setq cj/org-agenda-cache (list inbox-file schedule-file gcal-file)) - (cj/add-files-to-org-agenda-files-list projects-dir) - (setq cj/org-agenda-cache org-agenda-files) - (setq cj/org-agenda-cache-dirty nil) - (message "Rebuilt agenda cache in %.3f sec (%d files)" - (float-time (time-subtract (current-time) start-time)) - (length cj/org-agenda-cache)))) - (setq org-agenda-files cj/org-agenda-cache)) - -;; Set up file watching -(require 'filenotify) -(file-notify-add-watch projects-dir - '(change) - #'cj/invalidate-agenda-cache) -#+end_src - -*** TODO [#B] Test org-agenda filtering functions :testing:critical: -:PROPERTIES: -:FILE: tests/org-agenda-config-test.el (new file) -:EFFORT: 2-3 hours -:END: - -These functions are date-sensitive and break silently. High-value tests: - -#+begin_src elisp -;;; org-agenda-config-test.el --- Tests for org-agenda-config -*- lexical-binding: t; -*- - -(require 'ert) -(require 'org-agenda-config) - -(ert-deftest test-cj/org-skip-subtree-if-habit () - "Test habit filtering." - (with-temp-buffer - (org-mode) - (insert "* TODO Test\n:PROPERTIES:\n:STYLE: habit\n:END:\n") - (goto-char (point-min)) - (should (cj/org-skip-subtree-if-habit)))) - -(ert-deftest test-cj/org-skip-subtree-if-not-overdue () - "Test overdue task detection." - (with-temp-buffer - (org-mode) - ;; Task scheduled yesterday (overdue) - (insert "* TODO Overdue\nSCHEDULED: <2025-10-29>\n") - (goto-char (point-min)) - (should-not (cj/org-skip-subtree-if-not-overdue)) - - ;; Task scheduled tomorrow (not overdue) - (erase-buffer) - (insert "* TODO Future\nSCHEDULED: <2025-10-31>\n") - (goto-char (point-min)) - (should (cj/org-skip-subtree-if-not-overdue)))) -#+end_src - -** TIER 3: Quick Wins (After Tier 1 & 2) -*Time investment: 1-2 hours each* -*Status: Only tackle after profiling infrastructure is done* - -*** TODO [#B] Implement org-to-reveal.js workflow :enhancement:usability: -:PROPERTIES: -:FILE: modules/org-config.el or new modules/org-present-config.el -:EFFORT: 2 hours -:IMPACT: Solves specific, concrete need -:END: - -You said: "I know it can be done. Export to reveal.js, start httpd, open browser." - -You're right, it's straightforward: - -#+begin_src elisp -(use-package ox-reveal - :ensure t - :after org - :custom - (org-reveal-root "https://cdn.jsdelivr.net/npm/reveal.js")) - -(use-package simple-httpd - :ensure t - :custom - (httpd-port 8080) - (httpd-root "~/presentations")) - -(defun cj/org-present-reveal () - "Export current org file to reveal.js and present via httpd." - (interactive) - (unless (eq major-mode 'org-mode) - (user-error "Not in an org-mode buffer")) - - (let* ((httpd-dir (expand-file-name httpd-root)) - (output-file (expand-file-name - (concat (file-name-base) ".html") - httpd-dir))) - - ;; Ensure output directory exists - (make-directory httpd-dir t) - - ;; Export to reveal.js HTML - (org-reveal-export-to-html) - - ;; Move to httpd directory - (let ((exported-file (concat (file-name-base) ".html"))) - (rename-file exported-file output-file t)) - - ;; Start httpd if not running - (unless (process-status "httpd") - (httpd-start) - (message "Started httpd server on port %d" httpd-port)) - - ;; Open in browser (fullscreen) - (let ((url (format "http://localhost:%d/%s" - httpd-port - (file-name-nondirectory output-file)))) - (browse-url url) - (message "Presenting: %s" url)))) - -(with-eval-after-load 'org - (define-key org-mode-map (kbd "C-c C-v p") #'cj/org-present-reveal)) -#+end_src - -*That's it.* 30 lines. One keybinding. Done. - -Test with a simple org file: -#+begin_src org -,#+TITLE: Test Presentation -,#+REVEAL_THEME: moon - -,* Slide 1 -Content here - -,* Slide 2 -More content -#+end_src - -Press =C-c C-v p= and you're presenting. - -*** TODO [#B] Set up difftastic integration :enhancement:usability: -:PROPERTIES: -:FILE: modules/vc-config.el or modules/magit-config.el -:EFFORT: 30 minutes -:END: - -Install difftastic: =sudo pacman -S difftastic= or =cargo install difftastic= - -Then add to magit: - -#+begin_src elisp -(use-package magit-difftastic - :ensure t - :after magit - :bind (:map magit-status-mode-map - ("C-c d" . magit-difftastic-show)) - :custom - (magit-difftastic-executable "difft")) -#+end_src - -Or for native integration: -#+begin_src elisp -(setq magit-diff-refine-hunk 'all) ;; Already have this probably - -(defun cj/magit-difftastic-diff (args files) - "Show diff using difftastic." - (interactive (magit-diff-arguments)) - (require 'magit) - (let ((command (concat "difft " (mapconcat #'identity args " ")))) - (magit-git-command command))) - -(transient-append-suffix 'magit-diff '(-1 -1) - '("D" "Difftastic diff" cj/magit-difftastic-diff)) -#+end_src - -Simple and useful. Just do it. - -*** TODO [#C] Implement local package development workflow :enhancement:architecture: -:PROPERTIES: -:FILES: modules/org-agenda-config.el, modules/mail-config.el -:EFFORT: 1 hour -:END: - -You maintain chime.el and org-msg. You need to switch between local dev and stable versions easily. - -*Clean pattern:* - -In =user-constants.el=: -#+begin_src elisp -(defcustom cj/development-packages '() - "List of packages to load from local development paths. -When a package symbol is in this list, load from ~/code/<package>. -Otherwise use stable version via vc-install or MELPA. - -Example: (setq cj/development-packages '(chime org-msg))" - :type '(repeat symbol) - :group 'cj) - -(defun cj/use-local-package-p (package) - "Return non-nil if PACKAGE should be loaded from local path." - (memq package cj/development-packages)) -#+end_src - -In your package configs: -#+begin_src elisp -;; Chime configuration -(if (cj/use-local-package-p 'chime) - ;; Development mode: use local path - (use-package chime - :load-path "~/code/chime.el" - :demand t - ;; ... rest of config ... - ) - ;; Production mode: use stable version - (use-package chime - :vc (:url "https://github.com/yourusername/chime.el") - :demand t - ;; ... rest of config ... - )) -#+end_src - -To switch modes, just toggle the symbol in =cj/development-packages= and restart Emacs. - -*Benefits:* -- Clear intent: variable name says what it does -- Easy to toggle: add/remove symbols from list -- Version controlled: your config documents which packages you're actively developing -- No path duplication: define path logic once - -** TIER 4: Maybe/Someday (Proceed with Caution) -*Time investment: Unknown, possibly infinite* -*Status: Only do if you have concrete use case* - -*** HOLD [#D] Investigate code-maat/code-compass in Emacs :enhancement:tooling: -:PROPERTIES: -:REASON: Complexity vs value unclear -:END: - -*Hard truth:* This is a big project. Code-maat is a mature Java tool with significant functionality. - -*Questions before investing time:* -1. What would Emacs integration give you that CLI doesn't? -2. How often do you actually use code-maat? -3. Is this solving a real problem or is it intellectually interesting? - -*Recommendation:* Use the CLI tools. Call them from Emacs if needed: -#+begin_src elisp -(defun cj/code-maat-analyze (git-log-file) - "Run code-maat analysis on GIT-LOG-FILE." - (interactive "fGit log file: ") - (async-shell-command - (format "code-maat -l %s -c git2 -a coupling" git-log-file) - "*Code Maat Results*")) -#+end_src - -Don't reimplement it in elisp unless you have a very specific need the CLI can't meet. - -*** HOLD [#D] Fix LaTeX configuration :documentation:enhancement: -:PROPERTIES: -:REASON: No concrete use case yet -:END: - -You said: "Right now, it's a bunch of cut/paste snippets from everyone else's config and it doesn't work." - -*Hard truth:* LaTeX config is complex because LaTeX is complex. Don't invest time until you have a specific document you need to write. - -*When you do need it:* -1. Start with one concrete document (paper? letter? resume?) -2. Get that one document working -3. Extract the config that made it work -4. Then generalize - -*Don't* try to build a complete LaTeX environment speculatively. You'll waste time on edge cases you'll never hit. - -*Minimal starting point when you need it:* -#+begin_src elisp -(use-package auctex - :ensure t - :defer t - :mode ("\\.tex\\'" . LaTeX-mode) - :custom - (TeX-auto-save t) - (TeX-parse-self t) - (TeX-master nil)) -#+end_src - -That's it. Start there. Add more when you hit actual problems. - -*** HOLD [#D] Improve Elfeed dashboard UX :enhancement:ui: -:PROPERTIES: -:REASON: Need to clarify actual usage patterns -:END: - -You said: "it doesn't bring any joy to see a mostly blank screen when I check Elfeed." - -*Questions:* -1. Do you actually use RSS regularly, or is it aspirational? -2. If blank screen = no new items, is that a problem? -3. Would you use RSS more if the UX was better, or is RSS not part of your workflow? - -*Options:* -1. *If you don't use RSS:* Remove Elfeed. One less thing to maintain. -2. *If you want to use RSS more:* Start by curating better feeds, not fixing UI -3. *If blank screen bothers you:* Add a "no new items" message or show last N items regardless of read status - -Don't fix tooling for a workflow you're not sure you want. - -*** HOLD [#D] DWIM shell + eshell integration :enhancement:shell: -:PROPERTIES: -:REASON: Current solution works fine -:END: - -You have 815 lines of dwim-shell-config. It works. Leave it alone. - -*Hard truth:* This is a working system. "I wonder if I can leverage them better" is not a problem statement. - -Unless you have a specific workflow that's currently painful, don't touch this. - -*** HOLD [#D] Calibredb performance optimization :performance: -:PROPERTIES: -:REASON: Root cause is data volume, not code -:END: - -You correctly identified: "The right thing to do is to shrink the ebook inventory to < 1k or so books." - -*You already know the answer.* Don't optimize code when the problem is 14k+ files. - -Curate your library. Delete what you won't read. This is a data problem, not a code problem. - -*** HOLD [#D] Flyspell/abbrev UX improvements :enhancement:usability: -:PROPERTIES: -:REASON: Vague complaint, no specific pain points -:END: - -You said: "I'm not sure the UX is right. It could use some more attention." - -*This is not actionable.* Come back when you can say: -- "Pressing X does Y but I want Z" -- "It's too slow when..." -- "It conflicts with..." - -Until then, it's bikeshedding. - -*** HOLD [#D] Video/audio config sink testing :testing:enhancement: -:PROPERTIES: -:REASON: Nice to have, not blocking anything -:END: - -You said: "could use some testing around which sinks to attach to" - -*Questions:* -1. Does it currently work for your use case? -2. Do you switch audio sinks often? -3. Is this actually painful or just "would be nice"? - -If it works, leave it. If it doesn't work, describe the failure mode first. - -*** HOLD [#D] Build jumper package :project:packaging: -:PROPERTIES: -:REASON: Another package to maintain, unclear value -:END: - -You said: "I believe jumper should probably be my next package, but I haven't had the time." - -*Hard questions:* -1. What problem does jumper solve? -2. Do existing navigation packages (avy, ace-window, consult-line) not solve it? -3. Are you building this for yourself or for others? -4. How will you maintain it alongside chime and org-msg? - -If you're excited about it: Build it. But recognize it's a significant time investment. - -If you're not excited: Don't. You already maintain two packages. - -*** HOLD [#D] Extract local-repository creation logic :refactor:library: -:PROPERTIES: -:REASON: Unclear benefit -:END: - -You said: "extract the code... so I can unit test the hell out of it and use it as a library" - -*Questions:* -1. What would you use the library for beyond local-repository? -2. Is local-repository not working currently? -3. Are you planning to build other tools that need this? - -Don't extract libraries speculatively. Extract when you have two concrete use cases. - -*** HOLD [#D] Org-contacts + chime.el birthdays integration :enhancement:integration: -:PROPERTIES: -:REASON: Should be fixed in chime, not config -:END: - -You said: "I should either figure out sexp expressions and support them in chime, or disable them" - -*This is a chime.el issue, not a config issue.* - -File it as a chime.el feature request or bug. Your config shouldn't work around package limitations. - -** What NOT To Do - -*** DON'T: Optimize Before Profiling -You can't fix performance issues you can't measure. Build profiling infrastructure FIRST. - -*** DON'T: Build Speculative Libraries -Extract code into libraries when you have 2+ concrete use cases. Not before. - -*** DON'T: Fix Vague Complaints -"UX could be better" is not actionable. Come back with specific pain points. - -*** DON'T: Reimplement Mature CLI Tools -code-maat is Java. It works. Don't rewrite it in elisp. - -*** DON'T: Configure for Hypothetical Use Cases -LaTeX doesn't work? Come back when you need to write a specific LaTeX document. - -*** DON'T: Maintain More Packages Than You Can Support -You have chime and org-msg. That's plenty. Think hard before adding jumper. - -** Large Functions Analysis - -You asked me to identify large, complex functions that need breaking up. - -*Reality check:* Your functions are mostly fine. Here's what I found: - -*** Functions You Mentioned - -*1. dirvish-config.el:350-406 =cj/dired-copy-path-as-kill= (57 lines)* -- Has 6 responsibilities: validation, project detection, path formatting, org-link formatting, clipboard, feedback -- *Status:* Works fine, moderately complex -- *Recommendation:* Could extract helpers (=cj/--determine-path-type=, =cj/--format-path-for-display=), but not urgent -- *Priority:* [#C] - Refactor for testability when you have time - -*2. org-roam-config.el:250-298 =cj/move-org-branch-to-roam= (49 lines)* -- Already well-decomposed with helper functions! -- Uses =cj/org-link-get-description=, =cj/--generate-roam-slug=, =cj/--demote-org-subtree=, =cj/--format-roam-node= -- *Status:* Good design, clear logic flow -- *Recommendation:* Add integration test, but don't refactor further -- *Priority:* [#B] - Add tests, keep implementation - -*3. org-agenda-config.el:91-105 =cj/build-org-agenda-list= (15 lines)* -- Small function, but THIS IS YOUR PERFORMANCE BOTTLENECK -- Not because of size, but because it recursively scans directories without caching -- *Status:* CRITICAL ISSUE -- *Recommendation:* Profile it, add caching, add error handling -- *Priority:* [#A] - Profile and optimize ASAP - -*** Other Large Files - -*dwim-shell-config.el (815 lines)* -- Collection of DWIM commands -- *Status:* Works fine -- *Recommendation:* Leave it alone unless something breaks - -*custom-comments.el (636 lines)* -- Well-tested utility functions -- *Status:* No issues -- *Recommendation:* Keep as is - -*music-config.el (484 lines)* -- EMMS configuration -- *Recommendation:* Review when you mentioned video/audio sink testing - -*** The Real Issue - -Your problem isn't function length. Your problem is *performance observability*. - -You need: -1. Profiling infrastructure (debug-profiling.el) -2. Instrumentation in critical paths -3. Caching where appropriate -4. Error handling in file operations - -Focus on TIER 2 (observability infrastructure) before worrying about function decomposition. - -** Concrete Action Plan: This Weekend - -*** Saturday Morning (2-3 hours) - -1. *Delete network check* (15 min) - - Remove =cj/internet-up-p=, =cj/network-available=, all related code - - Test startup, should be 1+ seconds faster - -2. *Fix Priority A bugs* (45 min) - - Fix duplicate =cj/network-available= (if not deleted above) - - Implement =cj/goto-git-gutter-diff-hunks= or remove binding - - Fix =cj/log-silently= (move to config-utilities or use message) - -3. *Fix lorem error* (5 min) - - Find lorem config, suppress error message - -4. *Test everything still works* (15 min) - -*** Saturday Afternoon (2-3 hours) - -5. *Corfu migration* (2 hours) - - Remove company configuration - - Add corfu + corfu-popupinfo + cape - - Test completion in elisp, org, mu4e - - Verify mu4e still excludes completion - -6. *Switch to mood-line* (30 min) - - Remove doom-modeline - - Add mood-line configuration - - Restart, verify modeline works - -*** Sunday Morning (3-4 hours) - -7. *Create debug-profiling.el* (3 hours) - - Use the template I provided above - - Add to modules/ - - Configure in user-constants.el with =cj/debug-modules= - - Test basic commands work - -8. *Profile org-agenda* (30 min) - - Run =M-x cj/profile-agenda-rebuild= - - Run =M-x cj/profile-function RET cj/build-org-agenda-list= - - Document findings in *CJ Profile Log* - -9. *Write up findings* (30 min) - - What's slow? - - Why is it slow? - - What's the fix? - -*** Sunday Afternoon (Optional, 1-2 hours) - -10. *Reveal.js workflow* (2 hours) - - Implement =cj/org-present-reveal= - - Test with simple presentation - - Bind to key - -11. *Difftastic* (30 min) - - Install difftastic - - Configure magit integration - - Test on a real diff - -** Expected Outcomes - -After this weekend: -- ✅ Startup 1+ seconds faster (network check removed) -- ✅ No more broken keybindings or missing functions -- ✅ Completion is faster and more responsive (Corfu) -- ✅ Modeline updates are faster (mood-line) -- ✅ You have profiling infrastructure for all future optimization -- ✅ You understand why org-agenda is slow -- ✅ You have concrete data to guide optimization -- ✅ (Optional) You can present org files as reveal.js slides - -*** After This Weekend - -*Then stop and reassess.* - -- How do you feel about the changes? -- What's still bothering you? -- Is org-agenda faster after caching? (if you implement it) -- What's the next pain point? - -Don't commit to more work until you've lived with these changes for a week. - -** My Question for You - -Looking at your list of improvements, I see someone who: -- Likes building tools (jumper, code-maat integration, etc.) -- Gets excited by new technologies (difftastic, MCP, reveal.js) -- Has many "wouldn't it be cool if..." ideas - -But I also see someone who: -- Already has a working, sophisticated config -- Knows what's actually painful (org-agenda performance, mail attachments) -- Has limited time (maintaining chime, org-msg) - -*So here's my question:* - -*Are you building tools because they solve real problems, or because you enjoy building?* - -Both are valid answers! But the answer changes what you should prioritize: - -*If solving real problems:* -- Focus on TIER 1 & 2 (bugs, performance, observability) -- Ignore most of TIER 4 -- Build reveal.js workflow (concrete need) -- Fix mail attachments (daily pain) - -*If you enjoy building:* -- Pick ONE project (jumper? code-compass? profiling infrastructure?) -- Go deep on it -- Accept that other stuff will wait -- Be honest that it's for learning/fun, not necessity - -*My read:* You're mostly problem-solving, but you get distracted by shiny tools. - -*My advice:* Fix the pain points first (TIER 1 & 2). Then build one fun project. Then reassess. - -** gptel vs MCP: Specific Guidance - -You asked: "I need to either build a bunch of tools or install an MCP server. Which way to go?" - -*Answer: Build gptel functions. Here's why:* - -*** Why gptel + elisp Functions - -*Advantages:* -1. *Simpler:* Just elisp functions, no external server -2. *Integrated:* Native Emacs, shares your environment -3. *Debuggable:* Use edebug, step through code -4. *Maintainable:* You understand elisp, you wrote the config -5. *Flexible:* Can use any elisp API (org, dired, magit, etc.) -6. *Portable:* Works on any system with Emacs + gptel - -*What you can build:* -#+begin_src elisp -;; Example gptel tools -(defun cj/gptel-add-project-context () - "Add project README and key files to gptel context." - (interactive) - (let ((readme (expand-file-name "README.md" (project-root)))) - (when (file-exists-p readme) - (cj/gptel-add-file readme)))) - -(defun cj/gptel-analyze-function-at-point () - "Send function at point to LLM for analysis." - (interactive) - (let ((func (elisp--fnsym-in-current-sexp))) - (gptel-send (format "Analyze this function: %s" (symbol-function func))))) - -(defun cj/gptel-git-diff-review () - "Send current git diff to LLM for review." - (interactive) - (let ((diff (shell-command-to-string "git diff HEAD"))) - (gptel-send (concat "Review this diff:\n\n" diff)))) -#+end_src - -*This is much simpler than MCP and covers 90% of use cases.* - -*** When MCP Makes Sense - -*Only use MCP if:* -1. You want to share tools across multiple applications (VS Code + Emacs + CLI) -2. You need sandboxed execution for security -3. You're building reusable infrastructure for others -4. You want to use existing MCP servers someone else maintains - -*For Emacs-only workflows: Use gptel + elisp.* - -*** Recommended Approach - -1. Start with gptel + simple elisp functions -2. Build 3-5 useful commands (add-file, analyze-function, git-diff, org-context) -3. Use them for a month -4. If you find yourself wanting features MCP provides, reassess - -*Don't build MCP infrastructure speculatively.* Build it when gptel + elisp isn't enough. - -** Final Thoughts: Adjusted Priorities - -Your original document was thorough and well-analyzed. My additions: - -1. *Ruthless prioritization* - 4 tiers instead of flat list -2. *Observability first* - Can't optimize what you can't measure -3. *Reality checks* - Some "nice to haves" are rabbit holes -4. *Concrete implementations* - Not just "do this" but "here's the code" -5. *Hard questions* - Before building, ask "why?" and "for whom?" - -*Your config is good.* The issues are normal for a mature system. - -*Focus on:* -1. Fix the bugs (TIER 1) -2. Build profiling infrastructure (TIER 2) -3. Pick ONE fun project (reveal.js or difftastic) -4. Ignore everything else for now - -*Then live with it for a month* and see what still bothers you. - -You're thoughtful, skilled, and self-aware. You'll make good decisions. Just don't try to do everything at once. - -Good luck! 🚀 diff --git a/modules/auth-config.el b/modules/auth-config.el index 6b8a8ddb..2b52087e 100644 --- a/modules/auth-config.el +++ b/modules/auth-config.el @@ -24,9 +24,11 @@ :ensure nil ;; built in :demand t ;; load this package immediately :config - (setenv "GPG_AGENT_INFO" nil) ;; disassociate with external gpg agent - (setq auth-sources `(,authinfo-file)) ;; use authinfo.gpg (see user-constants.el) - (setq auth-source-debug t)) ;; echo debug info to Messages + ;; USE gpg-agent for passphrase caching (400-day cache from gpg-agent.conf) + ;; (setenv "GPG_AGENT_INFO" nil) ;; DISABLED: was preventing gpg-agent cache + (setq auth-sources `(,authinfo-file)) ;; use authinfo.gpg (see user-constants.el) + (setq auth-source-debug t) ;; echo debug info to Messages + (setq auth-source-cache-expiry 86400)) ;; cache decrypted credentials for 24 hours ;; ----------------------------- Easy PG Assistant ----------------------------- ;; Key management, cryptographic operations on regions and files, dired @@ -40,5 +42,79 @@ ;; (setq epa-pinentry-mode 'loopback) ;; emacs request passwords in minibuffer (setq epg-gpg-program "gpg2")) ;; force use gpg2 (not gpg v.1) +;; ---------------------------------- Plstore ---------------------------------- +;; Encrypted storage used by oauth2-auto for Google Calendar tokens. +;; CRITICAL: Enable passphrase caching to prevent password prompts every 10 min. + +(use-package plstore + :ensure nil ;; built-in + :demand t + :config + ;; Cache passphrase indefinitely (relies on gpg-agent for actual caching) + (setq plstore-cache-passphrase-for-symmetric-encryption t) + ;; Allow gpg-agent to cache the passphrase (400 days per gpg-agent.conf) + (setq plstore-encrypt-to nil)) ;; Use symmetric encryption, not key-based + +;; ------------------------ Authentication Reset Utility ----------------------- + +(defun cj/reset-auth-cache (&optional include-gpg-agent) + "Reset authentication caches when wrong password was entered. + +By default, only clears Emacs-side caches (auth-source, EPA file +handler) and leaves gpg-agent's long-term cache intact. This preserves +your 400-day cache for GPG and SSH passphrases. + +With prefix argument INCLUDE-GPG-AGENT (\\[universal-argument]), also +clears gpg-agent's password cache. Use this when gpg-agent itself has +cached an incorrect password. + +Clears: +1. auth-source cache (Emacs-level credential cache) +2. EPA file handler cache (encrypted file cache) +3. gpg-agent cache (only if INCLUDE-GPG-AGENT is non-nil) + +Use this when you see errors like: + - \"Bad session key\" + - \"Decryption failed\" + - GPG repeatedly using wrong cached password" + (interactive "P") + (message "Resetting authentication caches...") + + ;; Clear auth-source cache (Emacs credential cache) + (auth-source-forget-all-cached) + + ;; Clear EPA file handler cache + (when (fboundp 'epa-file-clear-cache) + (epa-file-clear-cache)) + + ;; Only clear gpg-agent cache if explicitly requested + (if include-gpg-agent + (let ((result (shell-command "echo RELOADAGENT | gpg-connect-agent"))) + (if (zerop result) + (message "✓ Emacs and gpg-agent caches cleared. Next access will prompt for password.") + (message "⚠ Warning: Failed to clear gpg-agent cache"))) + (message "✓ Emacs caches cleared. GPG/SSH passphrases preserved for session."))) + +(defun cj/kill-gpg-agent () + "Force kill gpg-agent (it will restart automatically on next use). + +This is a more aggressive reset than `cj/reset-auth-cache'. Use this +when gpg-agent is stuck or behaving incorrectly. + +The gpg-agent will automatically restart on the next GPG operation." + (interactive) + (let ((result (shell-command "gpgconf --kill gpg-agent"))) + (if (zerop result) + (message "✓ gpg-agent killed. It will restart automatically on next use.") + (message "⚠ Warning: Failed to kill gpg-agent")))) + +;; Keybindings +(with-eval-after-load 'keybindings + (keymap-set cj/custom-keymap "A" #'cj/reset-auth-cache)) + +(with-eval-after-load 'which-key + (which-key-add-key-based-replacements + "C-; A" "reset auth cache")) + (provide 'auth-config) ;;; auth-config.el ends here. diff --git a/modules/config-utilities.el b/modules/config-utilities.el index 32018371..2af3effa 100644 --- a/modules/config-utilities.el +++ b/modules/config-utilities.el @@ -33,8 +33,7 @@ "C-c d i b" "info build" "C-c d i p" "info packages" "C-c d i f" "info features" - "C-c d r" "reload init" - "C-c d a" "reset auth cache")) + "C-c d r" "reload init")) ;;; --------------------------------- Profiling --------------------------------- @@ -283,15 +282,6 @@ Recompile natively when supported, otherwise fall back to byte compilation." (load-file user-init-file)) (keymap-set cj/debug-config-keymap "r" 'cj/reload-init-file) -;; ----------------------------- Reset-Auth-Sources ---------------------------- - -(defun cj/reset-auth-cache () - "Clear Emacs auth-source cache." - (interactive) - (auth-source-forget-all-cached) - (message "Emacs auth-source cache cleared.")) -(keymap-set cj/debug-config-keymap "a" 'cj/reset-auth-cache) - ;; ------------------------ Validate Org Agenda Entries ------------------------ (defun cj/validate-org-agenda-timestamps () diff --git a/modules/dirvish-config.el b/modules/dirvish-config.el index 441ff61b..b10c97f0 100644 --- a/modules/dirvish-config.el +++ b/modules/dirvish-config.el @@ -8,15 +8,15 @@ ;; ediff, playlist creation, path copying, and external file manager integration. ;; ;; Key Bindings: -;; - d: Duplicate file at point (adds "-copy" before extension) -;; - D: Delete marked files immediately (dired-do-delete) +;; - d: Delete marked files (dired-do-delete) +;; - D: Duplicate file at point (adds "-copy" before extension) ;; - g: Quick access menu (jump to predefined directories) ;; - G: Search with deadgrep in current directory ;; - f: Open system file manager in current directory ;; - o/O: Open file with xdg-open/custom command ;; - l: Copy file path (project-relative or home-relative) ;; - L: Copy absolute file path -;; - P: Create M3U playlist from marked audio files +;; - P: Copy file path (same as 'l', replaces dired-do-print) ;; - M-D: DWIM menu (context actions for files) ;; - TAB: Toggle subtree expansion ;; - F11: Toggle sidebar view @@ -120,9 +120,9 @@ Filters for audio files, prompts for the playlist name, and saves the resulting (setq dired-listing-switches "-l --almost-all --human-readable --group-directories-first") (setq dired-dwim-target t) (setq dired-clean-up-buffers-too t) ;; offer to kill buffers associated deleted files and dirs - (setq dired-clean-confirm-killing-deleted-buffers t) ;; don't ask; just kill buffers associated with deleted files - (setq dired-recursive-copies (quote always)) ;; “always” means no asking - (setq dired-recursive-deletes (quote top))) ;; “top” means ask once + (setq dired-clean-confirm-killing-deleted-buffers nil) ;; don't ask; just kill buffers associated with deleted files + (setq dired-recursive-copies (quote always)) ;; "always" means no asking + (setq dired-recursive-deletes (quote top))) ;; "top" means ask once ;; note: disabled as it prevents marking and moving files to another directory ;; (setq dired-kill-when-opening-new-dired-buffer t) ;; don't litter by leaving buffers when navigating directories @@ -322,13 +322,14 @@ regardless of what file or subdirectory the point is on." ("M-p" . dirvish-peek-toggle) ("M-s" . dirvish-setup-menu) ("TAB" . dirvish-subtree-toggle) - ("d" . cj/dirvish-duplicate-file) + ("d" . dired-do-delete) + ("D" . cj/dirvish-duplicate-file) ("f" . cj/dirvish-open-file-manager-here) ("g" . dirvish-quick-access) ("o" . cj/xdg-open) ("O" . cj/open-file-with-command) ; Prompts for command to run ("r" . dirvish-rsync) - ("P" . cj/dired-create-playlist-from-marked) + ("P" . cj/dired-copy-path-as-kill) ("s" . dirvish-quicksort) ("v" . dirvish-vc-menu) ("y" . dirvish-yank-menu))) diff --git a/modules/flycheck-config.el b/modules/flycheck-config.el index ea19f08f..e2e8abe9 100644 --- a/modules/flycheck-config.el +++ b/modules/flycheck-config.el @@ -6,30 +6,30 @@ ;; This file configures Flycheck for on-demand syntax and grammar checking. ;; - Flycheck starts automatically only in sh-mode and emacs-lisp-mode -;; - This binds a custom helper (=cj/flycheck-list-errors=) to “C-; ?” +;; - This binds a custom helper (=cj/flycheck-list-errors=) to "C-; ?" ;; for popping up Flycheck's error list in another window. -;; - It also customizes Checkdoc to suppress only the “sentence-end-double-space” -;; and “warn-escape” warnings. +;; - It also customizes Checkdoc to suppress only the "sentence-end-double-space" +;; and "warn-escape" warnings. -;; - It registers a Proselint checker for prose files -;; (text-mode, markdown-mode, gfm-mode). +;; - It registers LanguageTool for comprehensive grammar checking of prose files +;; (text-mode, markdown-mode, gfm-mode, org-mode). -;; Note: I do use proselint quite a bit in emails and org-mode files. However, some -;; org-files can be large and running proselint on them will slow Emacs to a crawl. -;; Therefore, hitting "C-; ?" also runs cj/flycheck-prose-on-demand if in an org buffer. +;; Note: Grammar checking is on-demand only to avoid performance issues. +;; Hitting "C-; ?" runs cj/flycheck-prose-on-demand if in an org buffer. -;; ;; The cj/flycheck-prose-on-demand function: ;; - Turns on flycheck for the local buffer -;; - ensures proselint is added -;; - triggers an immediate check -;; -;; Since this is called within cj/flycheck-list-errors, flycheck's error list will still -;; display and the focus transferred to that buffer. +;; - Enables LanguageTool checker +;; - Triggers an immediate check +;; - Displays errors in the *Flycheck errors* buffer -;; OS Dependencies: -;; proselint (in the Arch AUR) +;; Installation: +;; On Arch Linux: +;; sudo pacman -S languagetool +;; +;; The wrapper script at scripts/languagetool-flycheck formats LanguageTool's +;; JSON output into flycheck-compatible format. It requires Python 3. ;;; Code: @@ -62,20 +62,20 @@ ;; use the load-path of the currently running Emacs instance (setq flycheck-emacs-lisp-load-path 'inherit) - ;; Define the prose checker (installed separately via OS). - (flycheck-define-checker proselint - "A linter for prose." - :command ("proselint" source-inplace) + ;; Define LanguageTool checker for comprehensive grammar checking + (flycheck-define-checker languagetool + "A grammar checker using LanguageTool. +Uses a wrapper script to format output for flycheck." + :command ("~/.emacs.d/scripts/languagetool-flycheck" + source-inplace) :error-patterns ((warning line-start (file-name) ":" line ":" column ": " - (id (one-or-more (not (any " ")))) (message) line-end)) :modes (text-mode markdown-mode gfm-mode org-mode)) - (add-to-list 'flycheck-checkers 'proselint) + (add-to-list 'flycheck-checkers 'languagetool) (defun cj/flycheck-list-errors () "Display flycheck's error list and switch to its buffer. - Runs flycheck-prose-on-demand if in an org-buffer." (interactive) (when (derived-mode-p 'org-mode) @@ -85,12 +85,14 @@ Runs flycheck-prose-on-demand if in an org-buffer." (switch-to-buffer-other-window "*Flycheck errors*")) (defun cj/flycheck-prose-on-demand () - "Enable Flycheck+Proselint in this buffer, run it, and show errors." + "Enable Flycheck with LanguageTool in this buffer, run it, and show errors." (interactive) ;; turn on Flycheck locally (flycheck-mode 1) - ;; ensure proselint is valid for org/text - (flycheck-add-mode 'proselint major-mode) + ;; ensure LanguageTool is valid for current mode + (flycheck-add-mode 'languagetool major-mode) + ;; select LanguageTool as the checker + (setq-local flycheck-checker 'languagetool) ;; trigger immediate check (flycheck-buffer))) diff --git a/modules/modeline-config.el b/modules/modeline-config.el index af0c3524..b1403539 100644 --- a/modules/modeline-config.el +++ b/modules/modeline-config.el @@ -3,29 +3,170 @@ ;;; Commentary: -;; Minimal modeline configuration using mood-line. - -;; mood-line is a lightweight, minimal modeline inspired by doom-modeline -;; but with much better performance and simpler configuration. +;; Simple, minimal modeline using only built-in Emacs functionality. +;; No external packages = no buffer issues, no native-comp errors. ;; Features: -;; - Buffer status and modification indicators -;; - Major mode display +;; - Buffer status (modified, read-only) +;; - Buffer name +;; - Major mode ;; - Version control status -;; - Flycheck/Flymake status -;; - Cursor position and buffer percentage -;; - Anzu and multiple-cursors counters -;; - No dependencies -;; - Minimal performance overhead +;; - Line and column position +;; - Buffer percentage ;;; Code: -;; -------------------------------- mood-line ---------------------------------- +;; Use buffer status colors from user-constants +(require 'user-constants) + +;; -------------------------- Modeline Configuration -------------------------- + +;; Use Emacs 30's built-in right-alignment +(setq mode-line-right-align-edge 'right-margin) + +;; String truncation length for narrow windows +(defcustom cj/modeline-string-truncate-length 12 + "String length after which truncation happens in narrow windows." + :type 'natnum + :group 'modeline) + +;; -------------------------- Helper Functions --------------------------------- + +(defun cj/modeline-window-narrow-p () + "Return non-nil if window is narrow (less than 100 chars wide)." + (< (window-total-width) 100)) + +(defun cj/modeline-string-truncate-p (str) + "Return non-nil if STR should be truncated." + (and (stringp str) + (not (string-empty-p str)) + (cj/modeline-window-narrow-p) + (> (length str) cj/modeline-string-truncate-length) + (not (one-window-p :no-minibuffer)))) + +(defun cj/modeline-string-cut-middle (str) + "Truncate STR in the middle if appropriate, else return STR. +Example: `my-very-long-name.el' → `my-ver...me.el'" + (if (cj/modeline-string-truncate-p str) + (let ((half (floor cj/modeline-string-truncate-length 2))) + (concat (substring str 0 half) "..." (substring str (- half)))) + str)) + +;; -------------------------- Modeline Segments -------------------------------- + +(defvar-local cj/modeline-buffer-name + '(:eval (let* ((state (cond + (buffer-read-only 'read-only) + (overwrite-mode 'overwrite) + (t 'normal))) + (color (alist-get state cj/buffer-status-colors)) + (name (buffer-name)) + (truncated-name (cj/modeline-string-cut-middle name))) + (propertize truncated-name + 'face `(:foreground ,color) + 'mouse-face 'mode-line-highlight + 'help-echo (concat + name "\n" + (or (buffer-file-name) + (format "No file. Directory: %s" default-directory))) + 'local-map (let ((map (make-sparse-keymap))) + (define-key map [mode-line mouse-1] 'previous-buffer) + (define-key map [mode-line mouse-3] 'next-buffer) + map)))) + "Buffer name colored by read-only/read-write status. +Green = writeable, Red = read-only, Gold = overwrite. +Truncates in narrow windows. Click to switch buffers.") + +(defvar-local cj/modeline-position + '(:eval (format "L:%d C:%d" (line-number-at-pos) (current-column))) + "Line and column position as L:line C:col.") + +(defvar cj/modeline-vc-faces + '((added . vc-locally-added-state) + (edited . vc-edited-state) + (removed . vc-removed-state) + (missing . vc-missing-state) + (conflict . vc-conflict-state) + (locked . vc-locked-state) + (up-to-date . vc-up-to-date-state)) + "VC state to face mapping.") + +(defvar-local cj/modeline-vc-branch + '(:eval (when (mode-line-window-selected-p) ; Only show in active window + (when-let* ((file (or buffer-file-name default-directory)) + (backend (vc-backend file))) + (when-let* ((branch (vc-working-revision file backend))) + ;; For Git, try to get symbolic branch name + (when (eq backend 'Git) + (require 'vc-git) + (when-let* ((symbolic (vc-git--symbolic-ref file))) + (setq branch symbolic))) + ;; Get VC state for face + (let* ((state (vc-state file backend)) + (face (alist-get state cj/modeline-vc-faces 'vc-up-to-date-state)) + (truncated-branch (cj/modeline-string-cut-middle branch))) + (concat + (propertize (char-to-string #xE0A0) 'face 'shadow) ; Git branch symbol + " " + (propertize truncated-branch + 'face face + 'mouse-face 'mode-line-highlight + 'help-echo (format "Branch: %s\nState: %s\nmouse-1: vc-diff\nmouse-3: vc-root-diff" branch state) + 'local-map (let ((map (make-sparse-keymap))) + (define-key map [mode-line mouse-1] 'vc-diff) + (define-key map [mode-line mouse-3] 'vc-root-diff) + map)))))))) + "Git branch with symbol and colored by VC state. +Shows only in active window. Truncates in narrow windows. +Click to show diffs with `vc-diff' or `vc-root-diff'.") + +(defvar-local cj/modeline-major-mode + '(:eval (let ((mode-str (format-mode-line mode-name)) ; Convert to string + (mode-sym major-mode)) + (propertize mode-str + 'mouse-face 'mode-line-highlight + 'help-echo (if-let* ((parent (get mode-sym 'derived-mode-parent))) + (format "Major mode: %s\nDerived from: %s\nmouse-1: describe-mode" mode-sym parent) + (format "Major mode: %s\nmouse-1: describe-mode" mode-sym)) + 'local-map (let ((map (make-sparse-keymap))) + (define-key map [mode-line mouse-1] 'describe-mode) + map)))) + "Major mode name only (no minor modes). +Click to show help with `describe-mode'.") + +(defvar-local cj/modeline-misc-info + '(:eval (when (mode-line-window-selected-p) + mode-line-misc-info)) + "Misc info (chime notifications, etc). +Shows only in active window.") + +;; -------------------------- Modeline Assembly -------------------------------- -(use-package mood-line - :config - (mood-line-mode)) +(setq-default mode-line-format + '("%e" ; Error message if out of memory + ;; LEFT SIDE + " " + cj/modeline-major-mode + " " + cj/modeline-buffer-name + " " + cj/modeline-position + ;; RIGHT SIDE (using Emacs 30 built-in right-align) + ;; Order: leftmost to rightmost as they appear in the list + mode-line-format-right-align + cj/modeline-vc-branch + " " + cj/modeline-misc-info + " ")) +;; Mark all segments as risky-local-variable (required for :eval forms) +(dolist (construct '(cj/modeline-buffer-name + cj/modeline-position + cj/modeline-vc-branch + cj/modeline-vc-faces + cj/modeline-major-mode + cj/modeline-misc-info)) + (put construct 'risky-local-variable t)) (provide 'modeline-config) ;;; modeline-config.el ends here diff --git a/modules/org-config.el b/modules/org-config.el index 555a966b..75d4c7db 100644 --- a/modules/org-config.el +++ b/modules/org-config.el @@ -68,11 +68,11 @@ (set-face-attribute 'org-link nil :underline t) (setq org-ellipsis " ▾") ;; change ellipses to down arrow - (setq org-hide-emphasis-markers t) ;; remove emphasis markers to keep the screen clean + (setq org-hide-emphasis-markers t) ;; hide emphasis markers (org-appear shows them when editing) (setq org-hide-leading-stars t) ;; hide leading stars, just show one per line (setq org-pretty-entities t) ;; render special symbols (setq org-pretty-entities-include-sub-superscripts nil) ;; ...except superscripts and subscripts - (setq org-fontify-emphasized-text nil) ;; ...and don't render bold and italic markup + (setq org-fontify-emphasized-text t) ;; render bold and italic markup (setq org-fontify-whole-heading-line t) ;; fontify the whole line for headings (for face-backgrounds) (add-hook 'org-mode-hook 'prettify-symbols-mode)) @@ -221,6 +221,15 @@ (org-superstar-configure-like-org-bullets) (setq org-superstar-leading-bullet ?\s)) +;; -------------------------------- Org-Appear --------------------------------- + +(use-package org-appear + :hook (org-mode . org-appear-mode) + :custom + (org-appear-autoemphasis t) ;; Show * / _ when cursor is on them + (org-appear-autolinks t) ;; Also works for links + (org-appear-autosubmarkers t)) ;; And sub/superscripts + ;; ------------------------------- Org-Checklist ------------------------------- ;; needed for org-habits to reset checklists once task is complete diff --git a/modules/org-gcal-config.el b/modules/org-gcal-config.el index dc083efe..97e8446a 100644 --- a/modules/org-gcal-config.el +++ b/modules/org-gcal-config.el @@ -13,9 +13,9 @@ ;; - Events are managed by Org (changes in org file push back to Google Calendar) ;; This is controlled by org-gcal-managed-newly-fetched-mode and ;; org-gcal-managed-update-existing-mode set to "org" -;; - Initial automatic sync post Emacs startup. No auto resync'ing. -;; (my calendar doesn't change hourly and I want fewer distractions and slowdowns). -;; if you need it: https://github.com/kidd/org-gcal.el?tab=readme-ov-file#sync-automatically-at-regular-times +;; - Automatic sync timer (configurable via cj/org-gcal-sync-interval-minutes) +;; Default: 30 minutes, set to nil to disable +;; See: https://github.com/kidd/org-gcal.el?tab=readme-ov-file#sync-automatically-at-regular-times ;; - Validates existing oath2-auto.plist file or creates it to avoid the issue mentioned here: ;; https://github.com/kidd/org-gcal.el?tab=readme-ov-file#note ;; @@ -27,7 +27,10 @@ ;; 3. Define `gcal-file' in user-constants (location of org file to hold sync'd events). ;; ;; Usage: -;; - Manual sync: C-; g (or M-x org-gcal-sync) +;; - Manual sync: C-; g s (or M-x org-gcal-sync) +;; - Toggle auto-sync on/off: C-; g t +;; - Restart auto-sync (e.g., after changing interval): C-; g r +;; - Clear sync lock (if sync gets stuck): C-; g c ;; ;; Note: ;; This configuration creates oauth2-auto.plist on first run to prevent sync errors. @@ -43,6 +46,17 @@ (defvar org-gcal--sync-lock)) (declare-function org-gcal-reload-client-id-secret "org-gcal") +;; User configurable sync interval +(defvar cj/org-gcal-sync-interval-minutes 30 + "Interval in minutes for automatic Google Calendar sync. +Set to nil to disable automatic syncing. +Changes take effect after calling `cj/org-gcal-restart-auto-sync'.") + +;; Internal timer object +(defvar cj/org-gcal-sync-timer nil + "Timer object for automatic org-gcal sync. +Use `cj/org-gcal-start-auto-sync' and `cj/org-gcal-stop-auto-sync' to control.") + (defun cj/org-gcal-clear-sync-lock () "Clear the org-gcal sync lock. Useful when a sync fails and leaves the lock in place, preventing future syncs." @@ -66,6 +80,50 @@ enabling bidirectional sync so changes push back to Google Calendar." (save-buffer)) (message "Converted %d event(s) to Org-managed" count))) +(defun cj/org-gcal-start-auto-sync () + "Start automatic Google Calendar sync timer. +Uses the interval specified in `cj/org-gcal-sync-interval-minutes'. +Does nothing if interval is nil or timer is already running." + (interactive) + (when (and cj/org-gcal-sync-interval-minutes + (not (and cj/org-gcal-sync-timer + (memq cj/org-gcal-sync-timer timer-list)))) + (let ((interval-seconds (* cj/org-gcal-sync-interval-minutes 60))) + (setq cj/org-gcal-sync-timer + (run-with-timer + 120 ;; Initial delay: 2 minutes after startup + interval-seconds + (lambda () + (condition-case err + (org-gcal-sync) + (error (message "org-gcal: Auto-sync failed: %s" err)))))) + (message "org-gcal: Auto-sync started (every %d minutes)" + cj/org-gcal-sync-interval-minutes)))) + +(defun cj/org-gcal-stop-auto-sync () + "Stop automatic Google Calendar sync timer." + (interactive) + (when (and cj/org-gcal-sync-timer + (memq cj/org-gcal-sync-timer timer-list)) + (cancel-timer cj/org-gcal-sync-timer) + (setq cj/org-gcal-sync-timer nil) + (message "org-gcal: Auto-sync stopped"))) + +(defun cj/org-gcal-toggle-auto-sync () + "Toggle automatic Google Calendar sync timer on/off." + (interactive) + (if (and cj/org-gcal-sync-timer + (memq cj/org-gcal-sync-timer timer-list)) + (cj/org-gcal-stop-auto-sync) + (cj/org-gcal-start-auto-sync))) + +(defun cj/org-gcal-restart-auto-sync () + "Restart automatic Google Calendar sync timer. +Useful after changing `cj/org-gcal-sync-interval-minutes'." + (interactive) + (cj/org-gcal-stop-auto-sync) + (cj/org-gcal-start-auto-sync)) + ;; Deferred library required by org-gcal (use-package deferred :ensure t) @@ -77,8 +135,6 @@ enabling bidirectional sync so changes push back to Google Calendar." (use-package org-gcal :vc (:url "https://github.com/cjennings/org-gcal" :rev :newest) :defer t ;; unless idle timer is set below - :bind (("C-; g" . org-gcal-sync) - ("C-; G" . cj/org-gcal-clear-sync-lock)) :init ;; Retrieve credentials from authinfo.gpg BEFORE package loads @@ -109,9 +165,8 @@ enabling bidirectional sync so changes push back to Google Calendar." (setq org-gcal-managed-update-existing-mode "gcal") ;; GCal wins on conflicts :config - ;; Enable plstore passphrase caching after org-gcal loads - (require 'plstore) - (setq plstore-cache-passphrase-for-symmetric-encryption t) + ;; Plstore caching is now configured globally in auth-config.el + ;; to ensure it loads before org-gcal needs it ;; set org-gcal timezone based on system timezone (setq org-gcal-local-timezone (cj/detect-system-timezone)) @@ -133,19 +188,26 @@ enabling bidirectional sync so changes push back to Google Calendar." ;; Advise org-gcal--sync-unlock which is called when sync completes (advice-add 'org-gcal--sync-unlock :after #'cj/org-gcal-save-files-after-sync)) -;; Set up automatic initial sync on boot with error handling -;;(run-with-idle-timer -;; 2 nil -;; (lambda () -;; (condition-case err -;; (org-gcal-sync) -;; (error (message "org-gcal: Initial sync failed: %s" err))))) +;; Start automatic sync timer based on user configuration +;; Set cj/org-gcal-sync-interval-minutes to nil to disable +(cj/org-gcal-start-auto-sync) + +;; Google Calendar keymap and keybindings +(defvar-keymap cj/gcal-map + :doc "Keymap for Google Calendar operations" + "s" #'org-gcal-sync + "t" #'cj/org-gcal-toggle-auto-sync + "r" #'cj/org-gcal-restart-auto-sync + "c" #'cj/org-gcal-clear-sync-lock) +(keymap-set cj/custom-keymap "g" cj/gcal-map) -;; which-key labels (with-eval-after-load 'which-key (which-key-add-key-based-replacements - "C-; g" "gcal sync" - "C-; G" "clear sync lock")) + "C-; g" "gcal menu" + "C-; g s" "sync" + "C-; g t" "toggle auto-sync" + "C-; g r" "restart auto-sync" + "C-; g c" "clear sync lock")) (provide 'org-gcal-config) ;;; org-gcal-config.el ends here diff --git a/modules/system-utils.el b/modules/system-utils.el index 2b39d862..e9686777 100644 --- a/modules/system-utils.el +++ b/modules/system-utils.el @@ -186,13 +186,22 @@ Logs output and exit code to buffer *external-open.log*." ;;; -------------------------- Scratch Buffer Happiness ------------------------- (defvar scratch-emacs-version-and-system - (concat ";; Emacs " emacs-version + (concat "# Emacs " emacs-version " on " system-configuration ".\n")) (defvar scratch-greet - (concat ";; Emacs ♥ you, " user-login-name ". Happy Hacking!\n\n")) + (concat "# Emacs ♥ you, " user-login-name ". Happy Hacking!\n\n")) (setopt initial-scratch-message (concat scratch-emacs-version-and-system scratch-greet)) +;; Set scratch buffer to org-mode +(setopt initial-major-mode 'org-mode) + +;; Move cursor to end of scratch buffer on startup +(add-hook 'emacs-startup-hook + (lambda () + (when (string= (buffer-name) "*scratch*") + (goto-char (point-max))))) + ;;; --------------------------------- Dictionary -------------------------------- (use-package quick-sdcv diff --git a/modules/transcription-config.el b/modules/transcription-config.el new file mode 100644 index 00000000..fd2f4aaa --- /dev/null +++ b/modules/transcription-config.el @@ -0,0 +1,390 @@ +;;; transcription-config.el --- Audio transcription workflow -*- lexical-binding: t; -*- + +;; Author: Craig Jennings <c@cjennings.net> +;; Created: 2025-11-04 + +;;; Commentary: +;; +;; Audio transcription workflow with multiple backend options. +;; +;; USAGE: +;; In dired: Press `T` on an audio file to transcribe +;; Anywhere: M-x cj/transcribe-audio +;; View active: M-x cj/transcriptions-buffer +;; Switch backend: C-; T b (or M-x cj/transcription-switch-backend) +;; +;; OUTPUT FILES: +;; audio.m4a → audio.txt (transcript) +;; → audio.log (process logs, conditionally kept) +;; +;; BACKENDS: +;; - 'openai-api: Fast cloud transcription +;; API key retrieved from authinfo.gpg (machine api.openai.com) +;; - 'assemblyai: Cloud transcription with speaker diarization +;; API key retrieved from authinfo.gpg (machine api.assemblyai.com) +;; - 'local-whisper: Local transcription (requires whisper installed) +;; +;; NOTIFICATIONS: +;; - "Transcription started on <file>" +;; - "Transcription complete. Transcript in <file.txt>" +;; - "Transcription errored. Logs in <file.log>" +;; +;; MODELINE: +;; Shows active transcription count: ⏺2 +;; Click to view *Transcriptions* buffer +;; +;;; Code: + +(require 'dired) +(require 'notifications) +(require 'auth-source) + +;; ----------------------------- Configuration --------------------------------- + +(defvar cj/transcribe-backend 'assemblyai + "Transcription backend to use. +- `openai-api': Fast cloud transcription via OpenAI API +- `assemblyai': Cloud transcription with speaker diarization via AssemblyAI +- `local-whisper': Local transcription using installed Whisper") + +(defvar cj/transcription-keep-log-when-done nil + "Whether to keep log files after successful transcription. +If nil, log files are deleted after successful completion. +If t, log files are always kept. +Log files are always kept on error regardless of this setting.") + +(defvar cj/transcriptions-list '() + "List of active transcriptions. +Each entry: (process audio-file start-time status) +Status: running, complete, error") + +;; ----------------------------- Pure Functions -------------------------------- + +(defun cj/--audio-file-p (file) + "Return non-nil if FILE is an audio file based on extension." + (when (and file (stringp file)) + (when-let ((ext (file-name-extension file))) + (member (downcase ext) cj/audio-file-extensions)))) + +(defun cj/--transcription-output-files (audio-file) + "Return cons cell of (TXT-FILE . LOG-FILE) for AUDIO-FILE." + (let ((base (file-name-sans-extension audio-file))) + (cons (concat base ".txt") + (concat base ".log")))) + +(defun cj/--transcription-duration (start-time) + "Return duration string (MM:SS) since START-TIME." + (let* ((elapsed (float-time (time-subtract (current-time) start-time))) + (minutes (floor (/ elapsed 60))) + (seconds (floor (mod elapsed 60)))) + (format "%02d:%02d" minutes seconds))) + +(defun cj/--should-keep-log (success-p) + "Return non-nil if log file should be kept. +SUCCESS-P indicates whether transcription succeeded." + (or (not success-p) ; Always keep on error + cj/transcription-keep-log-when-done)) + +(defun cj/--transcription-script-path () + "Return absolute path to transcription script based on backend." + (let ((script-name (pcase cj/transcribe-backend + ('openai-api "oai-transcribe") + ('assemblyai "assemblyai-transcribe") + ('local-whisper "local-whisper")))) + (expand-file-name (concat "scripts/" script-name) user-emacs-directory))) + +(defun cj/--get-openai-api-key () + "Retrieve OpenAI API key from authinfo.gpg. +Expects entry in authinfo.gpg: + machine api.openai.com login api password sk-... +Returns the API key string, or nil if not found." + (when-let* ((auth-info (car (auth-source-search + :host "api.openai.com" + :require '(:secret)))) + (secret (plist-get auth-info :secret))) + (if (functionp secret) + (funcall secret) + secret))) + +(defun cj/--get-assemblyai-api-key () + "Retrieve AssemblyAI API key from authinfo.gpg. +Expects entry in authinfo.gpg: + machine api.assemblyai.com login api password <key> +Returns the API key string, or nil if not found." + (when-let* ((auth-info (car (auth-source-search + :host "api.assemblyai.com" + :require '(:secret)))) + (secret (plist-get auth-info :secret))) + (if (functionp secret) + (funcall secret) + secret))) + +;; ---------------------------- Process Management ----------------------------- + +(defun cj/--notify (title message &optional urgency) + "Send desktop notification and echo area message. +TITLE and MESSAGE are strings. URGENCY is normal or critical." + (message "%s: %s" title message) + (when (and (fboundp 'notifications-notify) + (getenv "DISPLAY")) + (notifications-notify + :title title + :body message + :urgency (or urgency 'normal)))) + +(defun cj/--start-transcription-process (audio-file) + "Start async transcription process for AUDIO-FILE. +Returns the process object." + (unless (file-exists-p audio-file) + (user-error "Audio file does not exist: %s" audio-file)) + + (unless (cj/--audio-file-p audio-file) + (user-error "Not an audio file: %s" audio-file)) + + (let* ((script (cj/--transcription-script-path)) + (outputs (cj/--transcription-output-files audio-file)) + (txt-file (car outputs)) + (log-file (cdr outputs)) + (buffer-name (format " *transcribe-%s*" (file-name-nondirectory audio-file))) + (process-name (format "transcribe-%s" (file-name-nondirectory audio-file)))) + + (unless (file-executable-p script) + (user-error "Transcription script not found or not executable: %s" script)) + + ;; Create log file + (with-temp-file log-file + (insert (format "Transcription started: %s\n" (current-time-string)) + (format "Backend: %s\n" cj/transcribe-backend) + (format "Audio file: %s\n" audio-file) + (format "Script: %s\n\n" script))) + + ;; Start process with environment + (let* ((process-environment + ;; Add API key to environment based on backend + (pcase cj/transcribe-backend + ('openai-api + (if-let ((api-key (cj/--get-openai-api-key))) + (cons (format "OPENAI_API_KEY=%s" api-key) + process-environment) + (user-error "OpenAI API key not found in authinfo.gpg for host api.openai.com"))) + ('assemblyai + (if-let ((api-key (cj/--get-assemblyai-api-key))) + (cons (format "ASSEMBLYAI_API_KEY=%s" api-key) + process-environment) + (user-error "AssemblyAI API key not found in authinfo.gpg for host api.assemblyai.com"))) + (_ process-environment))) + (process (make-process + :name process-name + :buffer (get-buffer-create buffer-name) + :command (list script audio-file) + :sentinel (lambda (proc event) + (cj/--transcription-sentinel proc event audio-file txt-file log-file)) + :stderr log-file))) + + ;; Track transcription + (push (list process audio-file (current-time) 'running) cj/transcriptions-list) + (force-mode-line-update t) + + ;; Notify user + (cj/--notify "Transcription" + (format "Started on %s" (file-name-nondirectory audio-file))) + + process))) + +(defun cj/--transcription-sentinel (process event audio-file txt-file log-file) + "Sentinel for transcription PROCESS. +EVENT is the process event string. +AUDIO-FILE, TXT-FILE, and LOG-FILE are the associated files." + (let* ((success-p (and (string-match-p "finished" event) + (= 0 (process-exit-status process)))) + (process-buffer (process-buffer process)) + (entry (assq process cj/transcriptions-list))) + + ;; Write process output to txt file + (when (and success-p (buffer-live-p process-buffer)) + (with-current-buffer process-buffer + (write-region (point-min) (point-max) txt-file nil 'silent))) + + ;; Append process output to log file + (when (buffer-live-p process-buffer) + (with-temp-buffer + (insert-file-contents log-file) + (goto-char (point-max)) + (insert "\n" (format-time-string "[%Y-%m-%d %H:%M:%S] ") event "\n") + (insert-buffer-substring process-buffer) + (write-region (point-min) (point-max) log-file nil 'silent))) + + ;; Update transcription status + (when entry + (setf (nth 3 entry) (if success-p 'complete 'error))) + + ;; Cleanup log file if successful and configured to do so + (when (and success-p (not (cj/--should-keep-log t))) + (delete-file log-file)) + + ;; Kill process buffer + (when (buffer-live-p process-buffer) + (kill-buffer process-buffer)) + + ;; Notify user + (if success-p + (cj/--notify "Transcription" + (format "Complete. Transcript in %s" (file-name-nondirectory txt-file))) + (cj/--notify "Transcription" + (format "Errored. Logs in %s" (file-name-nondirectory log-file)) + 'critical)) + + ;; Clean up completed transcriptions after 10 minutes + (run-at-time 600 nil #'cj/--cleanup-completed-transcriptions) + + ;; Update modeline + (force-mode-line-update t))) + +(defun cj/--cleanup-completed-transcriptions () + "Remove completed/errored transcriptions from tracking list." + (setq cj/transcriptions-list + (seq-filter (lambda (entry) + (eq (nth 3 entry) 'running)) + cj/transcriptions-list)) + (force-mode-line-update t)) + +(defun cj/--count-active-transcriptions () + "Return count of running transcriptions." + (length (seq-filter (lambda (entry) + (eq (nth 3 entry) 'running)) + cj/transcriptions-list))) + +;; ----------------------------- Modeline Integration -------------------------- + +(defun cj/--transcription-modeline-string () + "Return modeline string for active transcriptions." + (let ((count (cj/--count-active-transcriptions))) + (when (> count 0) + (propertize (format " ⏺%d " count) + 'face 'warning + 'help-echo (format "%d active transcription%s (click to view)" + count (if (= count 1) "" "s")) + 'mouse-face 'mode-line-highlight + 'local-map (let ((map (make-sparse-keymap))) + (define-key map [mode-line mouse-1] + #'cj/transcriptions-buffer) + map))))) + +;; Add to mode-line-format (will be activated when module loads) +(add-to-list 'mode-line-misc-info + '(:eval (cj/--transcription-modeline-string)) + t) + +;; --------------------------- Interactive Commands ---------------------------- + +;;;###autoload +(defun cj/transcribe-audio (audio-file) + "Transcribe AUDIO-FILE asynchronously. +Creates AUDIO.txt with transcript and AUDIO.log with process logs. +Uses backend specified by `cj/transcribe-backend'." + (interactive (list (read-file-name "Audio file to transcribe: " + nil nil t nil + #'cj/--audio-file-p))) + (cj/--start-transcription-process (expand-file-name audio-file))) + +;;;###autoload +(defun cj/transcribe-audio-at-point () + "Transcribe audio file at point in dired." + (interactive) + (unless (derived-mode-p 'dired-mode) + (user-error "Not in dired-mode")) + (let ((file (dired-get-filename nil t))) + (unless file + (user-error "No file at point")) + (cj/transcribe-audio file))) + +;;;###autoload +(defun cj/transcriptions-buffer () + "Show buffer with active transcriptions." + (interactive) + (let ((buffer (get-buffer-create "*Transcriptions*"))) + (with-current-buffer buffer + (let ((inhibit-read-only t)) + (erase-buffer) + (insert (propertize "Active Transcriptions\n" 'face 'bold) + (propertize (make-string 50 ?─) 'face 'shadow) + "\n\n") + (if (null cj/transcriptions-list) + (insert "No active transcriptions.\n") + (dolist (entry cj/transcriptions-list) + (let* ((process (nth 0 entry)) + (audio-file (nth 1 entry)) + (start-time (nth 2 entry)) + (status (nth 3 entry)) + (duration (cj/--transcription-duration start-time)) + (status-face (pcase status + ('running 'warning) + ('complete 'success) + ('error 'error)))) + (insert (propertize (format "%-10s" status) 'face status-face) + " " + (file-name-nondirectory audio-file) + (format " (%s)\n" duration)))))) + (goto-char (point-min)) + (special-mode)) + (display-buffer buffer))) + +;;;###autoload +(defun cj/transcription-kill (process) + "Kill transcription PROCESS." + (interactive + (list (let ((choices (mapcar (lambda (entry) + (cons (file-name-nondirectory (nth 1 entry)) + (nth 0 entry))) + cj/transcriptions-list))) + (unless choices + (user-error "No active transcriptions")) + (cdr (assoc (completing-read "Kill transcription: " choices nil t) + choices))))) + (when (process-live-p process) + (kill-process process) + (message "Killed transcription process"))) + +;;;###autoload +(defun cj/transcription-switch-backend () + "Switch transcription backend. +Prompts with completing-read to select from available backends." + (interactive) + (let* ((backends '(("assemblyai" . assemblyai) + ("openai-api" . openai-api) + ("local-whisper" . local-whisper))) + (current (symbol-name cj/transcribe-backend)) + (prompt (format "Transcription backend (current: %s): " current)) + (choice (completing-read prompt backends nil t)) + (new-backend (alist-get choice backends nil nil #'string=))) + (setq cj/transcribe-backend new-backend) + (message "Transcription backend: %s" choice))) + +;; ------------------------------- Dired Integration --------------------------- + +(with-eval-after-load 'dired + (define-key dired-mode-map (kbd "T") #'cj/transcribe-audio-at-point)) + +;; Dirvish inherits dired-mode-map, so T works automatically + +;; ------------------------------- Global Keybindings -------------------------- + +;; Transcription keymap +(defvar-keymap cj/transcribe-map + :doc "Keymap for transcription operations" + "a" #'cj/transcribe-audio + "b" #'cj/transcription-switch-backend + "v" #'cj/transcriptions-buffer + "k" #'cj/transcription-kill) +(keymap-set cj/custom-keymap "T" cj/transcribe-map) + +(with-eval-after-load 'which-key + (which-key-add-key-based-replacements + "C-; T" "transcription menu" + "C-; T a" "transcribe audio" + "C-; T b" "switch backend" + "C-; T v" "view transcriptions" + "C-; T k" "kill transcription")) + +(provide 'transcription-config) +;;; transcription-config.el ends here diff --git a/modules/ui-config.el b/modules/ui-config.el index 91dbaf31..837d2169 100644 --- a/modules/ui-config.el +++ b/modules/ui-config.el @@ -36,11 +36,8 @@ "Opacity level for Emacs frames when `cj/enable-transparency' is non-nil. 100 = fully opaque, 0 = fully transparent.") -(defconst cj/cursor-colors - '((read-only . "#f06a3f") ; red – buffer is read-only - (overwrite . "#c48702") ; gold – overwrite mode - (normal . "#64aa0f")) ; green – insert & read/write - "Alist mapping cursor states to their colors.") +;; Use buffer status colors from user-constants +(require 'user-constants) ;; ----------------------------- System UI Settings ---------------------------- @@ -104,7 +101,7 @@ When `cj/enable-transparency' is nil, reset alpha to fully opaque." (buffer-read-only 'read-only) (overwrite-mode 'overwrite) (t 'normal))) - (color (alist-get state cj/cursor-colors))) + (color (alist-get state cj/buffer-status-colors))) (unless (and (string= color cj/-cursor-last-color) (string= (buffer-name) cj/-cursor-last-buffer)) (set-cursor-color color) diff --git a/modules/user-constants.el b/modules/user-constants.el index bcb34bcc..2a6d0ca2 100644 --- a/modules/user-constants.el +++ b/modules/user-constants.el @@ -38,6 +38,23 @@ Example: (setq cj/debug-modules '(org-agenda mail)) (defvar user-mail-address "c@cjennings.net" "The user's email address.") +;; ---------------------------- Buffer Status Colors --------------------------- + +(defconst cj/buffer-status-colors + '((read-only . "#f06a3f") ; red – buffer is read-only + (overwrite . "#c48702") ; gold – overwrite mode + (normal . "#64aa0f")) ; green – insert & read/write + "Alist mapping buffer states to their colors. +Used by cursor color, modeline, and other UI elements.") + +;; --------------------------- Media File Extensions --------------------------- + +(defvar cj/audio-file-extensions + '("m4a" "mp3" "wav" "flac" "ogg" "opus" "aac" + "aiff" "aif" "wma" "ape" "alac" "weba") + "File extensions recognized as audio files. +Used by transcription module and other audio-related functionality.") + ;; ------------------------ Directory And File Constants ----------------------- ;; DIRECTORIES diff --git a/modules/vc-config.el b/modules/vc-config.el index b9b61c29..141f6e17 100644 --- a/modules/vc-config.el +++ b/modules/vc-config.el @@ -125,10 +125,11 @@ interactive selection to jump to any changed line in the buffer." ;; -------------------------------- Difftastic --------------------------------- ;; Structural diffs for better git change visualization +;; Requires: difft binary (installed via pacman -S difftastic) (use-package difftastic - :demand t - :after magit + :defer t + :commands (difftastic-magit-diff difftastic-magit-show) :bind (:map magit-blame-read-only-mode-map ("D" . difftastic-magit-show) ("S" . difftastic-magit-show)) diff --git a/modules/video-audio-recording.el b/modules/video-audio-recording.el index c714a0a6..45bab267 100644 --- a/modules/video-audio-recording.el +++ b/modules/video-audio-recording.el @@ -4,7 +4,7 @@ ;;; Commentary: ;; Use ffmpeg to record desktop video or just audio. ;; with audio from mic and audio from default audio sink -;; Also supports audio-only recording in Opus format. +;; Audio recordings use M4A/AAC format for best compatibility. ;; ;; Note: video-recordings-dir and audio-recordings-dir are defined ;; (and directory created) in user-constants.el @@ -311,16 +311,16 @@ Otherwise use the default location in `audio-recordings-dir'." (system-device (cdr devices)) (location (expand-file-name directory)) (name (format-time-string "%Y-%m-%d-%H-%M-%S")) - (filename (expand-file-name (concat name ".opus") location)) + (filename (expand-file-name (concat name ".m4a") location)) (ffmpeg-command (format (concat "ffmpeg " "-f pulse -i %s " "-ac 1 " "-f pulse -i %s " - "-ac 2 " - "-filter_complex \"[0:a]volume=%.1f[mic];[1:a]volume=%.1f[sys];[mic][sys]amerge=inputs=2\" " - "-c:a libopus " - "-b:a 96k " + "-ac 1 " + "-filter_complex \"[0:a]volume=%.1f[mic];[1:a]volume=%.1f[sys];[mic][sys]amerge=inputs=2[out];[out]pan=mono|c0=0.5*c0+0.5*c1\" " + "-c:a aac " + "-b:a 64k " "%s") mic-device system-device diff --git a/modules/weather-config.el b/modules/weather-config.el index 31fb1b70..3a30aa17 100644 --- a/modules/weather-config.el +++ b/modules/weather-config.el @@ -10,8 +10,15 @@ ;; ----------------------------------- Wttrin ---------------------------------- +;; Load wttrin from local development directory +(add-to-list 'load-path "/home/cjennings/code/wttrin") + +;; Set debug flag BEFORE loading wttrin (checked at load time) +(setq wttrin-debug nil) + (use-package wttrin - :vc (:url "https://github.com/cjennings/emacs-wttrin" :rev :newest) + ;; Uncomment the next line to use vc-install instead of local directory: + ;; :vc (:url "https://github.com/cjennings/emacs-wttrin" :rev :newest) :defer t :preface ;; dependency for wttrin @@ -21,6 +28,22 @@ ("M-W" . wttrin) :custom (wttrin-unit-system "u") + (wttrin-mode-line-favorite-location "New Orleans, LA") + (wttrin-mode-line-refresh-interval 900) ; 15 minutes + :init + ;; Explicitly autoload the mode function (needed for local dev directory) + (autoload 'wttrin-mode-line-mode "wttrin" "Toggle weather display in mode-line." t) + ;; Enable mode-line widget AFTER Emacs finishes initializing + ;; (url-retrieve async needs full init to work without buffer errors) + (if (daemonp) + ;; Daemon mode: wait for first client to connect + (add-hook 'server-after-make-frame-hook + (lambda () (wttrin-mode-line-mode 1)) + t) ; append to end of hook + ;; Normal Emacs: wait for startup to complete + (add-hook 'after-init-hook + (lambda () (wttrin-mode-line-mode 1)) + t)) ; append to end of hook :config (setq wttrin-default-locations '( "New Orleans, LA" diff --git a/scripts/assemblyai-transcribe b/scripts/assemblyai-transcribe new file mode 100755 index 00000000..22cbf538 --- /dev/null +++ b/scripts/assemblyai-transcribe @@ -0,0 +1,134 @@ +#!/usr/bin/env bash +# assemblyai-transcribe - Transcribe audio files using AssemblyAI API with speaker diarization +# Usage: assemblyai-transcribe <audio-file> [language] +# +# Requires: ASSEMBLYAI_API_KEY environment variable +# Language: en, es, fr, etc. (default: en) +# Features: Speaker diarization (up to 50 speakers) + +set -euo pipefail + +# Parse arguments +AUDIO="${1:-}" +LANG="${2:-en}" + +# Validate arguments +if [[ -z "$AUDIO" ]]; then + echo "Usage: assemblyai-transcribe <audio-file> [language]" >&2 + echo "Example: assemblyai-transcribe meeting.m4a en" >&2 + exit 1 +fi + +if [[ ! -f "$AUDIO" ]]; then + echo "Error: Audio file not found: $AUDIO" >&2 + exit 1 +fi + +# Check API key is set +if [[ -z "${ASSEMBLYAI_API_KEY:-}" ]]; then + echo "Error: ASSEMBLYAI_API_KEY environment variable not set" >&2 + exit 1 +fi + +# Check curl is available +if ! command -v curl &> /dev/null; then + echo "Error: curl command not found" >&2 + exit 1 +fi + +# Check jq is available (for JSON parsing) +if ! command -v jq &> /dev/null; then + echo "Error: jq command not found (required for JSON parsing)" >&2 + echo "Install with: sudo pacman -S jq" >&2 + exit 1 +fi + +API_BASE="https://api.assemblyai.com/v2" + +# Step 1: Upload audio file +echo "Uploading audio file..." >&2 +UPLOAD_RESPONSE=$(curl -s -X POST "${API_BASE}/upload" \ + -H "Authorization: ${ASSEMBLYAI_API_KEY}" \ + --data-binary "@${AUDIO}") + +UPLOAD_URL=$(echo "$UPLOAD_RESPONSE" | jq -r '.upload_url') + +if [[ -z "$UPLOAD_URL" ]] || [[ "$UPLOAD_URL" == "null" ]]; then + echo "Error: Failed to upload audio file" >&2 + echo "$UPLOAD_RESPONSE" >&2 + exit 1 +fi + +echo "Upload complete. Submitting transcription..." >&2 + +# Step 2: Submit transcription request with speaker labels +TRANSCRIPT_REQUEST=$(cat <<EOF +{ + "audio_url": "${UPLOAD_URL}", + "language_code": "${LANG}", + "speech_model": "universal", + "speaker_labels": true +} +EOF +) + +TRANSCRIPT_RESPONSE=$(curl -s -X POST "${API_BASE}/transcript" \ + -H "Authorization: ${ASSEMBLYAI_API_KEY}" \ + -H "Content-Type: application/json" \ + -d "$TRANSCRIPT_REQUEST") + +TRANSCRIPT_ID=$(echo "$TRANSCRIPT_RESPONSE" | jq -r '.id') + +if [[ -z "$TRANSCRIPT_ID" ]] || [[ "$TRANSCRIPT_ID" == "null" ]]; then + echo "Error: Failed to submit transcription" >&2 + echo "$TRANSCRIPT_RESPONSE" >&2 + exit 1 +fi + +echo "Transcription job submitted (ID: ${TRANSCRIPT_ID})" >&2 +echo "Waiting for completion..." >&2 + +# Step 3: Poll for completion +STATUS="queued" +POLL_INTERVAL=3 +MAX_WAIT=1800 # 30 minutes +ELAPSED=0 + +while [[ "$STATUS" == "queued" ]] || [[ "$STATUS" == "processing" ]]; do + if [[ $ELAPSED -ge $MAX_WAIT ]]; then + echo "Error: Transcription timed out after ${MAX_WAIT} seconds" >&2 + exit 1 + fi + + sleep $POLL_INTERVAL + ELAPSED=$((ELAPSED + POLL_INTERVAL)) + + RESULT=$(curl -s -X GET "${API_BASE}/transcript/${TRANSCRIPT_ID}" \ + -H "Authorization: ${ASSEMBLYAI_API_KEY}") + + STATUS=$(echo "$RESULT" | jq -r '.status') + + if [[ "$STATUS" == "processing" ]]; then + echo "Processing... (${ELAPSED}s elapsed)" >&2 + fi +done + +# Check if transcription failed +if [[ "$STATUS" != "completed" ]]; then + ERROR_MSG=$(echo "$RESULT" | jq -r '.error // "Unknown error"') + echo "Error: Transcription failed with status: ${STATUS}" >&2 + echo "Error message: ${ERROR_MSG}" >&2 + exit 1 +fi + +echo "Transcription complete! (${ELAPSED}s total)" >&2 + +# Step 4: Format output with speaker labels +# Extract utterances and format as "Speaker A: text" +echo "$RESULT" | jq -r ' + if .utterances then + .utterances[] | "Speaker \(.speaker): \(.text)" + else + .text + end +' diff --git a/scripts/install-whisper.sh b/scripts/install-whisper.sh new file mode 100755 index 00000000..e2ea4ac9 --- /dev/null +++ b/scripts/install-whisper.sh @@ -0,0 +1,103 @@ +#!/usr/bin/env bash +# Install OpenAI Whisper for transcription on Arch Linux +# Usage: install-whisper.sh [--yes] # --yes for non-interactive mode + +set -euo pipefail + +# Non-interactive mode +ASSUME_YES=false +if [[ "${1:-}" == "--yes" ]] || [[ "${1:-}" == "-y" ]]; then + ASSUME_YES=true +fi + +echo "=== Whisper Installation for Arch Linux ===" +echo + +# Check if running on Arch +if [[ ! -f /etc/arch-release ]]; then + echo "Warning: This script is designed for Arch Linux" + if [[ "$ASSUME_YES" == false ]]; then + read -p "Continue anyway? [y/N] " -n 1 -r + echo + [[ ! $REPLY =~ ^[Yy]$ ]] && exit 1 + else + echo "Continuing anyway (--yes mode)" + fi +fi + +# 1. Install system dependencies +echo "Step 1/3: Installing system dependencies (ffmpeg)..." +if ! command -v ffmpeg &> /dev/null; then + sudo pacman -S --needed ffmpeg + echo "✓ ffmpeg installed" +else + echo "✓ ffmpeg already installed" +fi + +# 2. Check for AUR package first (optional but cleaner) +echo +echo "Step 2/3: Checking for AUR package..." +AUR_INSTALLED=false + +if command -v yay &> /dev/null; then + echo "Found yay. Checking AUR for python-openai-whisper..." + if yay -Ss python-openai-whisper | grep -q 'python-openai-whisper'; then + INSTALL_AUR=false + if [[ "$ASSUME_YES" == true ]]; then + echo "Installing from AUR (--yes mode)" + INSTALL_AUR=true + else + read -p "Install from AUR via yay? [Y/n] " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]] || [[ -z $REPLY ]]; then + INSTALL_AUR=true + fi + fi + + if [[ "$INSTALL_AUR" == true ]]; then + yay -S --needed --noconfirm python-openai-whisper + echo "✓ Installed from AUR" + AUR_INSTALLED=true + fi + else + echo "Package python-openai-whisper not found in AUR" + fi +else + echo "yay not found. Skipping AUR installation." + echo "(Install yay if you prefer AUR packages)" +fi + +# 3. Install via pip if not from AUR +if [[ "$AUR_INSTALLED" == false ]]; then + echo + echo "Step 3/3: Installing openai-whisper via pip..." + pip install --user -U openai-whisper + echo "✓ openai-whisper installed via pip" + echo + echo "Note: Ensure ~/.local/bin is in your PATH" + echo "Add to ~/.bashrc or ~/.zshrc: export PATH=\"\$HOME/.local/bin:\$PATH\"" +fi + +# Verify installation +echo +echo "=== Verifying Installation ===" +if command -v whisper &> /dev/null; then + echo "✓ whisper command found at: $(which whisper)" + whisper --help | head -n 3 + echo + echo "=== Installation Complete! ===" + echo + echo "Models available: tiny, base, small, medium, large" + echo "Recommended: small (good balance of speed/accuracy)" + echo "Model will download automatically on first use." + echo + echo "Test with: whisper your-audio.m4a --model small --language en" +else + echo "✗ Installation failed - whisper command not found" + echo + echo "Troubleshooting:" + echo "1. Ensure ~/.local/bin is in your PATH" + echo "2. Run: source ~/.bashrc (or ~/.zshrc)" + echo "3. Try: python -m whisper --help" + exit 1 +fi diff --git a/scripts/languagetool-flycheck b/scripts/languagetool-flycheck new file mode 100755 index 00000000..ecbc900f --- /dev/null +++ b/scripts/languagetool-flycheck @@ -0,0 +1,82 @@ +#!/usr/bin/env python3 +""" +Wrapper for LanguageTool to produce flycheck-compatible output. +Output format: filename:line:column: message +""" + +import json +import sys +import subprocess + +def main(): + if len(sys.argv) < 2: + print("Usage: languagetool-flycheck FILE", file=sys.stderr) + sys.exit(1) + + filename = sys.argv[1] + + # Run languagetool with JSON output + try: + result = subprocess.run( + ['languagetool', '-l', 'en-US', '--json', filename], + capture_output=True, + text=True, + timeout=30 + ) + except subprocess.TimeoutExpired: + print(f"{filename}:1:1: LanguageTool timeout", file=sys.stderr) + sys.exit(1) + except Exception as e: + print(f"{filename}:1:1: LanguageTool error: {e}", file=sys.stderr) + sys.exit(1) + + # Parse JSON output + try: + # Find the JSON in the output (skip warning lines) + json_output = None + for line in result.stdout.split('\n'): + if line.startswith('{'): + json_output = line + break + + if not json_output: + sys.exit(0) # No errors found + + data = json.loads(json_output) + + # Read file to calculate line numbers from character offsets + with open(filename, 'r', encoding='utf-8') as f: + content = f.read() + + # Convert matches to flycheck format + for match in data.get('matches', []): + offset = match['offset'] + length = match['length'] + message = match['message'] + rule_id = match['rule']['id'] + + # Calculate line and column from offset + line = content[:offset].count('\n') + 1 + line_start = content.rfind('\n', 0, offset) + 1 + column = offset - line_start + 1 + + # Get first suggestion if available + suggestions = match.get('replacements', []) + if suggestions: + suggestion = suggestions[0]['value'] + message = f"{rule_id}: {message} Suggestion: {suggestion}" + else: + message = f"{rule_id}: {message}" + + # Output in flycheck format + print(f"{filename}:{line}:{column}: {message}") + + except json.JSONDecodeError as e: + print(f"{filename}:1:1: Failed to parse LanguageTool JSON: {e}", file=sys.stderr) + sys.exit(1) + except Exception as e: + print(f"{filename}:1:1: Error processing LanguageTool output: {e}", file=sys.stderr) + sys.exit(1) + +if __name__ == '__main__': + main() diff --git a/scripts/local-whisper b/scripts/local-whisper new file mode 100755 index 00000000..b08651c9 --- /dev/null +++ b/scripts/local-whisper @@ -0,0 +1,60 @@ +#!/usr/bin/env bash +# local-whisper - Transcribe audio files using locally installed Whisper +# Usage: local-whisper <audio-file> [model] [language] +# +# Models: tiny, base, small, medium, large (default: small) +# Language: en, es, fr, etc. (default: en) + +set -euo pipefail + +# Parse arguments +AUDIO="${1:-}" +MODEL="${2:-small}" +LANG="${3:-en}" + +# Validate arguments +if [[ -z "$AUDIO" ]]; then + echo "Usage: local-whisper <audio-file> [model] [language]" >&2 + echo "Example: local-whisper meeting.m4a small en" >&2 + exit 1 +fi + +if [[ ! -f "$AUDIO" ]]; then + echo "Error: Audio file not found: $AUDIO" >&2 + exit 1 +fi + +# Check whisper is installed +if ! command -v whisper &> /dev/null; then + echo "Error: whisper command not found" >&2 + echo "Install with: ~/.emacs.d/scripts/install-whisper.sh" >&2 + exit 1 +fi + +# Get absolute path to audio file +AUDIO_ABS="$(realpath "$AUDIO")" +AUDIO_DIR="$(dirname "$AUDIO_ABS")" +AUDIO_BASE="$(basename "$AUDIO_ABS")" +AUDIO_NAME="${AUDIO_BASE%.*}" + +# Run whisper +# Note: whisper creates ${AUDIO_NAME}.txt automatically in the output directory +whisper "$AUDIO_ABS" \ + --model "$MODEL" \ + --language "$LANG" \ + --task transcribe \ + --output_format txt \ + --output_dir "$AUDIO_DIR" \ + --verbose False 2>&1 + +# Output file that whisper creates +OUTPUT_FILE="$AUDIO_DIR/$AUDIO_NAME.txt" + +# Return transcript to stdout +if [[ -f "$OUTPUT_FILE" ]]; then + cat "$OUTPUT_FILE" + exit 0 +else + echo "Error: Whisper did not create expected output file: $OUTPUT_FILE" >&2 + exit 1 +fi diff --git a/scripts/oai-transcribe b/scripts/oai-transcribe new file mode 100755 index 00000000..f64a8122 --- /dev/null +++ b/scripts/oai-transcribe @@ -0,0 +1,45 @@ +#!/usr/bin/env bash +# oai-transcribe - Transcribe audio files using OpenAI Whisper API +# Usage: oai-transcribe <audio-file> [language] +# +# Requires: OPENAI_API_KEY environment variable +# Language: en, es, fr, etc. (default: en) + +set -euo pipefail + +# Parse arguments +AUDIO="${1:-}" +LANG="${2:-en}" + +# Validate arguments +if [[ -z "$AUDIO" ]]; then + echo "Usage: oai-transcribe <audio-file> [language]" >&2 + echo "Example: oai-transcribe meeting.m4a en" >&2 + exit 1 +fi + +if [[ ! -f "$AUDIO" ]]; then + echo "Error: Audio file not found: $AUDIO" >&2 + exit 1 +fi + +# Check API key is set +if [[ -z "${OPENAI_API_KEY:-}" ]]; then + echo "Error: OPENAI_API_KEY environment variable not set" >&2 + echo "Set with: export OPENAI_API_KEY='sk-...'" >&2 + exit 1 +fi + +# Check curl is available +if ! command -v curl &> /dev/null; then + echo "Error: curl command not found" >&2 + exit 1 +fi + +# Call OpenAI API +curl -s -X POST "https://api.openai.com/v1/audio/transcriptions" \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -F "model=whisper-1" \ + -F "response_format=text" \ + -F "language=${LANG}" \ + -F "file=@${AUDIO}" diff --git a/scripts/uninstall-whisper.sh b/scripts/uninstall-whisper.sh new file mode 100755 index 00000000..e46c6ebc --- /dev/null +++ b/scripts/uninstall-whisper.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# Uninstall OpenAI Whisper + +set -euo pipefail + +echo "=== Whisper Uninstallation ===" +echo + +REMOVED=false + +# Check if installed via AUR +if command -v yay &> /dev/null; then + if yay -Qi python-openai-whisper &> /dev/null 2>&1; then + echo "Detected AUR installation (python-openai-whisper)" + read -p "Remove via yay? [Y/n] " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]] || [[ -z $REPLY ]]; then + yay -R python-openai-whisper + echo "✓ Removed via AUR" + REMOVED=true + fi + fi +fi + +# Check if installed via pip +if pip list 2>/dev/null | grep -q openai-whisper; then + echo "Detected pip installation (openai-whisper)" + read -p "Remove via pip? [Y/n] " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]] || [[ -z $REPLY ]]; then + pip uninstall -y openai-whisper + echo "✓ Removed via pip" + REMOVED=true + fi +fi + +if [[ "$REMOVED" == false ]]; then + echo "No whisper installation found (checked AUR and pip)" +fi + +# Ask about ffmpeg +echo +read -p "Remove ffmpeg? (may be used by other apps) [y/N] " -n 1 -r +echo +if [[ $REPLY =~ ^[Yy]$ ]]; then + sudo pacman -R ffmpeg + echo "✓ Removed ffmpeg" +fi + +# Ask about model cache +CACHE_DIR="$HOME/.cache/whisper" +if [[ -d "$CACHE_DIR" ]]; then + echo + echo "Whisper models are cached in: $CACHE_DIR" + du -sh "$CACHE_DIR" 2>/dev/null || echo "Size: unknown" + read -p "Delete cached models? [y/N] " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]]; then + rm -rf "$CACHE_DIR" + echo "✓ Deleted model cache" + fi +fi + +echo +echo "=== Uninstallation Complete ===" diff --git a/tests/fixtures/grammar-correct.txt b/tests/fixtures/grammar-correct.txt new file mode 100644 index 00000000..bea335e8 --- /dev/null +++ b/tests/fixtures/grammar-correct.txt @@ -0,0 +1,5 @@ +This is a well-written sentence with no grammar errors. + +The quick brown fox jumps over the lazy dog. + +Everything here follows standard English grammar rules. diff --git a/tests/fixtures/grammar-errors-basic.txt b/tests/fixtures/grammar-errors-basic.txt new file mode 100644 index 00000000..c2f72c12 --- /dev/null +++ b/tests/fixtures/grammar-errors-basic.txt @@ -0,0 +1,7 @@ +This are a test of basic grammar errors. + +I could of done better with this sentence. + +Their going to the store to buy there groceries. + +The dog wagged it's tail happily. diff --git a/tests/fixtures/grammar-errors-punctuation.txt b/tests/fixtures/grammar-errors-punctuation.txt new file mode 100644 index 00000000..37de646a --- /dev/null +++ b/tests/fixtures/grammar-errors-punctuation.txt @@ -0,0 +1,5 @@ +This sentence is missing punctuation at the end + +Multiple spaces between words should be detected. + +A sentence with,incorrect comma,placement and usage. diff --git a/tests/test-flycheck-languagetool-setup.el b/tests/test-flycheck-languagetool-setup.el new file mode 100644 index 00000000..a719e822 --- /dev/null +++ b/tests/test-flycheck-languagetool-setup.el @@ -0,0 +1,71 @@ +;;; test-flycheck-languagetool-setup.el --- Unit tests for LanguageTool setup -*- lexical-binding: t; -*- + +;;; Commentary: +;; Unit tests verifying LanguageTool installation and wrapper script setup. +;; Focus: Testing OUR code (wrapper script, file setup), not flycheck internals. +;; +;; We trust that flycheck works correctly (it's an external framework). +;; These tests verify: +;; - LanguageTool is installed and accessible +;; - Our wrapper script exists, is executable, and has correct structure +;; - Python 3 dependency is available +;; +;; Categories: Normal (installation checks), Boundary (script structure), Error (missing dependencies) + +;;; Code: + +(require 'ert) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-flycheck-languagetool-setup-normal-wrapper-exists () + "Test that languagetool-flycheck wrapper script exists." + (let ((wrapper-path (expand-file-name "~/.emacs.d/scripts/languagetool-flycheck"))) + (should (file-exists-p wrapper-path)))) + +(ert-deftest test-flycheck-languagetool-setup-normal-wrapper-executable () + "Test that languagetool-flycheck wrapper script is executable." + (let ((wrapper-path (expand-file-name "~/.emacs.d/scripts/languagetool-flycheck"))) + (should (file-executable-p wrapper-path)))) + +(ert-deftest test-flycheck-languagetool-setup-normal-languagetool-installed () + "Test that languagetool command is available in PATH." + (should (executable-find "languagetool"))) + +(ert-deftest test-flycheck-languagetool-setup-normal-python3-available () + "Test that python3 is available for wrapper script." + (should (executable-find "python3"))) + + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-flycheck-languagetool-setup-boundary-wrapper-script-format () + "Test that wrapper script has correct shebang and structure." + (let ((wrapper-path (expand-file-name "~/.emacs.d/scripts/languagetool-flycheck"))) + (with-temp-buffer + (insert-file-contents wrapper-path) + (goto-char (point-min)) + ;; Check shebang + (should (looking-at "#!/usr/bin/env python3")) + ;; Check it contains required imports + (should (search-forward "import json" nil t)) + (should (search-forward "import subprocess" nil t))))) + +;; ----------------------------- Error Cases ----------------------------------- + +(ert-deftest test-flycheck-languagetool-setup-error-missing-file-argument () + "Test that wrapper script requires file argument. +When called without arguments, wrapper should exit with error." + (let* ((wrapper (expand-file-name "~/.emacs.d/scripts/languagetool-flycheck")) + (exit-code nil)) + (with-temp-buffer + (setq exit-code (call-process wrapper nil t nil)) + ;; Should exit with non-zero status when no file provided + (should-not (= 0 exit-code)) + ;; Should print usage message to stderr (captured in buffer) + (goto-char (point-min)) + (should (or (search-forward "Usage:" nil t) + (search-forward "FILE" nil t)))))) + +(provide 'test-flycheck-languagetool-setup) +;;; test-flycheck-languagetool-setup.el ends here diff --git a/tests/test-integration-grammar-checking.el b/tests/test-integration-grammar-checking.el new file mode 100644 index 00000000..8948c17a --- /dev/null +++ b/tests/test-integration-grammar-checking.el @@ -0,0 +1,190 @@ +;;; test-integration-grammar-checking.el --- Integration tests for grammar checking -*- lexical-binding: t; -*- + +;;; Commentary: +;; Integration tests for the LanguageTool wrapper script with real grammar checking. +;; Tests the integration: test fixture → wrapper script → LanguageTool → formatted output +;; +;; Components integrated: +;; - scripts/languagetool-flycheck (our wrapper script) +;; - languagetool command (external grammar checker) +;; - Test fixtures with known grammar errors +;; - Output formatting (JSON → flycheck format) +;; +;; Focus: Testing OUR integration code (wrapper), not flycheck framework. +;; We trust that flycheck works; we test that our wrapper produces correct output. +;; +;; Categories: Normal workflow, Boundary cases, Error handling + +;;; Code: + +(require 'ert) + +;; ----------------------------- Test Helpers ---------------------------------- + +(defun test-integration-grammar--fixture-path (filename) + "Return absolute path to test fixture FILENAME." + (expand-file-name (concat "tests/fixtures/" filename) + user-emacs-directory)) + +(defun test-integration-grammar--wrapper-output (file-path) + "Run languagetool-flycheck wrapper directly on FILE-PATH. +Returns output as string." + (let ((wrapper (expand-file-name "~/.emacs.d/scripts/languagetool-flycheck"))) + (with-temp-buffer + (call-process wrapper nil t nil file-path) + (buffer-string)))) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-integration-grammar-checking-normal-wrapper-detects-errors () + "Test that wrapper script detects grammar errors in fixture. + +Components integrated: +- scripts/languagetool-flycheck (wrapper script) +- languagetool command (external checker) +- Test fixture with known errors" + (let* ((fixture (test-integration-grammar--fixture-path "grammar-errors-basic.txt")) + (output (test-integration-grammar--wrapper-output fixture))) + ;; Should detect "This are" error + (should (string-match-p "PLURAL_VERB_AFTER_THIS\\|This are" output)) + ;; Should detect "could of" error + (should (string-match-p "COULD_OF\\|could of" output)) + ;; Output should be in flycheck format (filename:line:column:) + (should (string-match-p "grammar-errors-basic\\.txt:[0-9]+:[0-9]+:" output)))) + +(ert-deftest test-integration-grammar-checking-normal-wrapper-format () + "Test that wrapper outputs flycheck-compatible format. + +Components integrated: +- scripts/languagetool-flycheck (output formatting) +- languagetool command (JSON parsing)" + (let* ((fixture (test-integration-grammar--fixture-path "grammar-errors-basic.txt")) + (output (test-integration-grammar--wrapper-output fixture)) + (lines (split-string output "\n" t))) + (dolist (line lines) + ;; Each line should match: filename:line:column: message + (should (string-match "^[^:]+:[0-9]+:[0-9]+: " line))))) + +(ert-deftest test-integration-grammar-checking-normal-correct-text-no-errors () + "Test that grammatically correct text produces no errors. + +Components integrated: +- scripts/languagetool-flycheck (wrapper script) +- languagetool command (validation) +- Test fixture with correct grammar" + (let* ((fixture (test-integration-grammar--fixture-path "grammar-correct.txt")) + (output (test-integration-grammar--wrapper-output fixture))) + ;; Correct grammar should produce no output (or only whitespace) + (should (or (string-empty-p (string-trim output)) + (= 0 (length (string-trim output))))))) + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-integration-grammar-checking-boundary-empty-file () + "Test that empty file produces no errors. + +Components integrated: +- scripts/languagetool-flycheck (empty input handling) +- languagetool command" + (let ((temp-file (make-temp-file "grammar-test-" nil ".txt"))) + (unwind-protect + (let ((output (test-integration-grammar--wrapper-output temp-file))) + (should (or (string-empty-p (string-trim output)) + (= 0 (length (string-trim output)))))) + (delete-file temp-file)))) + +(ert-deftest test-integration-grammar-checking-boundary-single-word () + "Test that single word file produces no errors. + +Components integrated: +- scripts/languagetool-flycheck (minimal input) +- languagetool command" + (let ((temp-file (make-temp-file "grammar-test-" nil ".txt"))) + (unwind-protect + (progn + (with-temp-file temp-file + (insert "Hello")) + (let ((output (test-integration-grammar--wrapper-output temp-file))) + ;; Single word might produce no errors or might flag as incomplete sentence + ;; Just verify it doesn't crash + (should (stringp output)))) + (delete-file temp-file)))) + +(ert-deftest test-integration-grammar-checking-boundary-multiple-paragraphs () + "Test that file with multiple paragraphs is checked completely. + +Components integrated: +- scripts/languagetool-flycheck (multi-paragraph handling) +- languagetool command (full file processing)" + (let* ((fixture (test-integration-grammar--fixture-path "grammar-errors-basic.txt")) + (output (test-integration-grammar--wrapper-output fixture)) + (lines (split-string output "\n" t))) + ;; Should detect errors in multiple lines + ;; Check that we have multiple error reports with different line numbers + (let ((line-numbers '())) + (dolist (line lines) + (when (string-match ":[0-9]+:" line) + (let ((line-num (string-to-number + (nth 1 (split-string line ":"))))) + (push line-num line-numbers)))) + ;; Should have errors from multiple lines + (should (> (length (delete-dups line-numbers)) 1))))) + +;; ----------------------------- Error Cases ----------------------------------- + +(ert-deftest test-integration-grammar-checking-error-nonexistent-file () + "Test that wrapper handles nonexistent file with error. + +Components integrated: +- scripts/languagetool-flycheck (error handling) +- File system (missing file) +- Python exception handling" + (let* ((nonexistent "/tmp/this-file-does-not-exist-12345.txt") + (wrapper (expand-file-name "~/.emacs.d/scripts/languagetool-flycheck")) + (exit-code nil) + (output nil)) + (with-temp-buffer + (setq exit-code (call-process wrapper nil t nil nonexistent)) + (setq output (buffer-string))) + ;; LanguageTool/Python should handle the error + ;; Check that we get output (error message or error in flycheck format) + (should (stringp output)) + ;; Output should contain some indication of the error (filename or error marker) + (should (or (string-match-p nonexistent output) + (string-match-p "error" output) + (string-match-p "Error" output) + ;; Or it might report no errors for a nonexistent file + (string-empty-p (string-trim output)))))) + +(ert-deftest test-integration-grammar-checking-error-no-file-argument () + "Test that wrapper requires file argument. + +Components integrated: +- scripts/languagetool-flycheck (argument validation)" + (let* ((wrapper (expand-file-name "~/.emacs.d/scripts/languagetool-flycheck")) + (exit-code nil)) + (with-temp-buffer + (setq exit-code (call-process wrapper nil t nil)) + ;; Should exit with non-zero status when no file provided + (should-not (= 0 exit-code))))) + +;; ----------------------------- Integration with Real Files ------------------- + +(ert-deftest test-integration-grammar-checking-integration-comprehensive-errors () + "Test that wrapper catches multiple types of grammar errors in one file. + +Components integrated: +- scripts/languagetool-flycheck (our wrapper) +- languagetool command (comprehensive checking) +- Test fixture with various error types" + (let* ((fixture (test-integration-grammar--fixture-path "grammar-errors-basic.txt")) + (output (test-integration-grammar--wrapper-output fixture)) + (lines (split-string output "\n" t))) + ;; Should detect multiple errors (at least 3-4 in the fixture) + (should (>= (length lines) 3)) + ;; All lines should be properly formatted + (dolist (line lines) + (should (string-match "^[^:]+:[0-9]+:[0-9]+: " line))))) + +(provide 'test-integration-grammar-checking) +;;; test-integration-grammar-checking.el ends here diff --git a/tests/test-integration-transcription.el b/tests/test-integration-transcription.el new file mode 100644 index 00000000..96b617bc --- /dev/null +++ b/tests/test-integration-transcription.el @@ -0,0 +1,145 @@ +;;; test-integration-transcription.el --- Integration tests for transcription -*- lexical-binding: t; -*- + +;;; Commentary: +;; End-to-end integration tests for transcription workflow +;; Tests complete workflow with temporary files and mocked processes +;; Categories: Normal workflow, Error handling, Cleanup + +;;; Code: + +(require 'ert) +(require 'transcription-config) + +;; ----------------------------- Test Helpers ---------------------------------- + +(defun test-transcription--make-mock-audio-file () + "Create a temporary mock audio file for testing. +Returns the absolute path to the file." + (let ((file (make-temp-file "test-audio-" nil ".m4a"))) + (with-temp-file file + (insert "Mock audio data")) + file)) + +(defun test-transcription--cleanup-output-files (audio-file) + "Delete transcript and log files associated with AUDIO-FILE." + (let* ((outputs (cj/--transcription-output-files audio-file)) + (txt-file (car outputs)) + (log-file (cdr outputs))) + (when (file-exists-p txt-file) + (delete-file txt-file)) + (when (file-exists-p log-file) + (delete-file log-file)))) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-integration-transcription-output-files-created () + "Test that .txt and .log files are created for audio file." + (let* ((audio-file (test-transcription--make-mock-audio-file)) + (outputs (cj/--transcription-output-files audio-file)) + (txt-file (car outputs)) + (log-file (cdr outputs))) + (unwind-protect + (progn + ;; Verify output file paths are correct + (should (string-suffix-p ".txt" txt-file)) + (should (string-suffix-p ".log" log-file)) + (should (string= (file-name-sans-extension txt-file) + (file-name-sans-extension audio-file))) + (should (string= (file-name-sans-extension log-file) + (file-name-sans-extension audio-file)))) + ;; Cleanup + (delete-file audio-file) + (test-transcription--cleanup-output-files audio-file)))) + +(ert-deftest test-integration-transcription-validates-file-exists () + "Test that transcription fails for non-existent file." + (should-error + (cj/--start-transcription-process "/nonexistent/audio.m4a") + :type 'user-error)) + +(ert-deftest test-integration-transcription-validates-audio-extension () + "Test that transcription fails for non-audio file." + (let ((non-audio (make-temp-file "test-" nil ".txt"))) + (unwind-protect + (should-error + (cj/--start-transcription-process non-audio) + :type 'user-error) + (delete-file non-audio)))) + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-integration-transcription-audio-file-detection () + "Test various audio file extensions are accepted." + (dolist (ext '("m4a" "mp3" "wav" "flac" "ogg" "opus")) + (let ((audio-file (make-temp-file "test-audio-" nil (concat "." ext)))) + (unwind-protect + (progn + (should (cj/--audio-file-p audio-file)) + ;; Would start transcription if script existed + ) + (delete-file audio-file))))) + +(ert-deftest test-integration-transcription-filename-with-spaces () + "Test transcription with audio file containing spaces." + (let ((audio-file (make-temp-file "test audio file" nil ".m4a"))) + (unwind-protect + (let* ((outputs (cj/--transcription-output-files audio-file)) + (txt-file (car outputs)) + (log-file (cdr outputs))) + (should (file-name-absolute-p txt-file)) + (should (file-name-absolute-p log-file))) + (delete-file audio-file)))) + +(ert-deftest test-integration-transcription-filename-with-special-chars () + "Test transcription with special characters in filename." + (let ((audio-file (make-temp-file "test_(final)" nil ".m4a"))) + (unwind-protect + (let* ((outputs (cj/--transcription-output-files audio-file)) + (txt-file (car outputs))) + ;; make-temp-file adds random suffix, so just check it ends with .txt + ;; and contains the special chars + (should (string-suffix-p ".txt" txt-file)) + (should (string-match-p "test_(final)" txt-file))) + (delete-file audio-file)))) + +;; ----------------------------- Cleanup Tests --------------------------------- + +(ert-deftest test-integration-transcription-cleanup-completed () + "Test that completed transcriptions are removed from tracking." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running) + (proc2 "file2.m4a" nil complete) + (proc3 "file3.m4a" nil error)))) + (cj/--cleanup-completed-transcriptions) + (should (= 1 (length cj/transcriptions-list))) + (should (eq 'running (nth 3 (car cj/transcriptions-list)))))) + +(ert-deftest test-integration-transcription-cleanup-all-complete () + "Test cleanup when all transcriptions are complete." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil complete) + (proc2 "file2.m4a" nil error)))) + (cj/--cleanup-completed-transcriptions) + (should (null cj/transcriptions-list)))) + +(ert-deftest test-integration-transcription-cleanup-preserves-running () + "Test that running transcriptions are not cleaned up." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running) + (proc2 "file2.m4a" nil running)))) + (cj/--cleanup-completed-transcriptions) + (should (= 2 (length cj/transcriptions-list))))) + +;; ----------------------------- Backend Tests --------------------------------- + +(ert-deftest test-integration-transcription-script-path-exists () + "Test that transcription scripts exist in expected location." + (dolist (backend '(local-whisper openai-api)) + (let ((cj/transcribe-backend backend)) + (let ((script (cj/--transcription-script-path))) + (should (file-name-absolute-p script)) + ;; Note: Script may not exist in test environment, just check path format + (should (string-match-p "scripts/" script)))))) + +(provide 'test-integration-transcription) +;;; test-integration-transcription.el ends here diff --git a/tests/test-transcription-audio-file.el b/tests/test-transcription-audio-file.el new file mode 100644 index 00000000..f40d9ca6 --- /dev/null +++ b/tests/test-transcription-audio-file.el @@ -0,0 +1,83 @@ +;;; test-transcription-audio-file.el --- Tests for audio file detection -*- lexical-binding: t; -*- + +;;; Commentary: +;; Tests for cj/--audio-file-p function +;; Categories: Normal cases, Boundary cases, Error cases + +;;; Code: + +(require 'ert) +(require 'transcription-config) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-cj/--audio-file-p-m4a () + "Test that .m4a files are recognized as audio." + (should (cj/--audio-file-p "meeting.m4a"))) + +(ert-deftest test-cj/--audio-file-p-mp3 () + "Test that .mp3 files are recognized as audio." + (should (cj/--audio-file-p "podcast.mp3"))) + +(ert-deftest test-cj/--audio-file-p-wav () + "Test that .wav files are recognized as audio." + (should (cj/--audio-file-p "recording.wav"))) + +(ert-deftest test-cj/--audio-file-p-flac () + "Test that .flac files are recognized as audio." + (should (cj/--audio-file-p "music.flac"))) + +(ert-deftest test-cj/--audio-file-p-with-path () + "Test audio file recognition with full path." + (should (cj/--audio-file-p "/home/user/recordings/meeting.m4a"))) + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-cj/--audio-file-p-uppercase-extension () + "Test that uppercase extensions are recognized." + (should (cj/--audio-file-p "MEETING.M4A"))) + +(ert-deftest test-cj/--audio-file-p-mixed-case () + "Test that mixed case extensions are recognized." + (should (cj/--audio-file-p "podcast.Mp3"))) + +(ert-deftest test-cj/--audio-file-p-no-extension () + "Test that files without extension are not recognized." + (should-not (cj/--audio-file-p "meeting"))) + +(ert-deftest test-cj/--audio-file-p-empty-string () + "Test that empty string is not recognized as audio." + (should-not (cj/--audio-file-p ""))) + +(ert-deftest test-cj/--audio-file-p-dotfile () + "Test that dotfiles without proper extension are not recognized." + (should-not (cj/--audio-file-p ".hidden"))) + +(ert-deftest test-cj/--audio-file-p-multiple-dots () + "Test file with multiple dots but audio extension." + (should (cj/--audio-file-p "meeting.2025-11-04.final.m4a"))) + +;; ------------------------------ Error Cases ---------------------------------- + +(ert-deftest test-cj/--audio-file-p-not-audio () + "Test that non-audio files are not recognized." + (should-not (cj/--audio-file-p "document.pdf"))) + +(ert-deftest test-cj/--audio-file-p-text-file () + "Test that text files are not recognized as audio." + (should-not (cj/--audio-file-p "notes.txt"))) + +(ert-deftest test-cj/--audio-file-p-org-file () + "Test that org files are not recognized as audio." + (should-not (cj/--audio-file-p "tasks.org"))) + +(ert-deftest test-cj/--audio-file-p-video-file () + "Test that video files are not recognized as audio." + (should-not (cj/--audio-file-p "video.mp4"))) + +(ert-deftest test-cj/--audio-file-p-nil () + "Test that nil input returns nil." + (should-not (cj/--audio-file-p nil))) + +(provide 'test-transcription-audio-file) +;;; test-transcription-audio-file.el ends here diff --git a/tests/test-transcription-config--transcription-script-path.el b/tests/test-transcription-config--transcription-script-path.el new file mode 100644 index 00000000..a56cb05c --- /dev/null +++ b/tests/test-transcription-config--transcription-script-path.el @@ -0,0 +1,106 @@ +;;; test-transcription-config--transcription-script-path.el --- Tests for cj/--transcription-script-path -*- lexical-binding: t; -*- + +;;; Commentary: +;; Tests for the cj/--transcription-script-path function from transcription-config.el +;; +;; This function returns the absolute path to the transcription script based on +;; the current value of cj/transcribe-backend. + +;;; Code: + +(require 'ert) + +;; Add modules directory to load path +(add-to-list 'load-path (expand-file-name "modules" user-emacs-directory)) + +;; Stub dependencies before loading the module +(defvar cj/custom-keymap (make-sparse-keymap) + "Stub keymap for testing.") + +;; Stub notification function +(unless (fboundp 'notifications-notify) + (defun notifications-notify (&rest _args) + "Stub notification function for testing." + nil)) + +;; Now load the actual production module +(require 'transcription-config) + +;;; Setup and Teardown + +(defun test-transcription-script-path-setup () + "Set up test environment." + ;; Save original backend setting + (setq test-transcription-original-backend cj/transcribe-backend)) + +(defun test-transcription-script-path-teardown () + "Clean up test environment." + ;; Restore original backend setting + (setq cj/transcribe-backend test-transcription-original-backend)) + +;;; Normal Cases + +(ert-deftest test-transcription-config--transcription-script-path-normal-openai-api-returns-oai-transcribe () + "Should return oai-transcribe script path for openai-api backend." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'openai-api) + (let ((result (cj/--transcription-script-path))) + (should (stringp result)) + (should (string-suffix-p "scripts/oai-transcribe" result)) + (should (string-prefix-p (expand-file-name user-emacs-directory) result)))) + (test-transcription-script-path-teardown))) + +(ert-deftest test-transcription-config--transcription-script-path-normal-assemblyai-returns-assemblyai-transcribe () + "Should return assemblyai-transcribe script path for assemblyai backend." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'assemblyai) + (let ((result (cj/--transcription-script-path))) + (should (stringp result)) + (should (string-suffix-p "scripts/assemblyai-transcribe" result)) + (should (string-prefix-p (expand-file-name user-emacs-directory) result)))) + (test-transcription-script-path-teardown))) + +(ert-deftest test-transcription-config--transcription-script-path-normal-local-whisper-returns-local-whisper () + "Should return local-whisper script path for local-whisper backend." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'local-whisper) + (let ((result (cj/--transcription-script-path))) + (should (stringp result)) + (should (string-suffix-p "scripts/local-whisper" result)) + (should (string-prefix-p (expand-file-name user-emacs-directory) result)))) + (test-transcription-script-path-teardown))) + +(ert-deftest test-transcription-config--transcription-script-path-normal-returns-absolute-path () + "Should return absolute path starting with user-emacs-directory." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'openai-api) + (let ((result (cj/--transcription-script-path))) + (should (file-name-absolute-p result)) + (should (string-prefix-p "/" result)))) + (test-transcription-script-path-teardown))) + +;;; Boundary Cases + +(ert-deftest test-transcription-config--transcription-script-path-boundary-path-format-consistent () + "Should return paths in consistent format across backends." + (test-transcription-script-path-setup) + (unwind-protect + (let (paths) + (dolist (backend '(openai-api assemblyai local-whisper)) + (setq cj/transcribe-backend backend) + (push (cj/--transcription-script-path) paths)) + ;; All paths should have same structure: <emacs-dir>/scripts/<name> + (should (= (length paths) 3)) + (should (seq-every-p (lambda (p) (string-match-p "/scripts/[^/]+$" p)) paths))) + (test-transcription-script-path-teardown))) + +(provide 'test-transcription-config--transcription-script-path) +;;; test-transcription-config--transcription-script-path.el ends here diff --git a/tests/test-transcription-counter.el b/tests/test-transcription-counter.el new file mode 100644 index 00000000..fae353ba --- /dev/null +++ b/tests/test-transcription-counter.el @@ -0,0 +1,98 @@ +;;; test-transcription-counter.el --- Tests for active transcription counting -*- lexical-binding: t; -*- + +;;; Commentary: +;; Tests for cj/--count-active-transcriptions and modeline integration +;; Categories: Normal cases, Boundary cases + +;;; Code: + +(require 'ert) +(require 'transcription-config) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-cj/--count-active-transcriptions-empty () + "Test count when no transcriptions are active." + (let ((cj/transcriptions-list '())) + (should (= 0 (cj/--count-active-transcriptions))))) + +(ert-deftest test-cj/--count-active-transcriptions-one-running () + "Test count with one running transcription." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running)))) + (should (= 1 (cj/--count-active-transcriptions))))) + +(ert-deftest test-cj/--count-active-transcriptions-multiple-running () + "Test count with multiple running transcriptions." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running) + (proc2 "file2.m4a" nil running) + (proc3 "file3.m4a" nil running)))) + (should (= 3 (cj/--count-active-transcriptions))))) + +(ert-deftest test-cj/--count-active-transcriptions-mixed-status () + "Test count excludes completed/errored transcriptions." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running) + (proc2 "file2.m4a" nil complete) + (proc3 "file3.m4a" nil running) + (proc4 "file4.m4a" nil error)))) + (should (= 2 (cj/--count-active-transcriptions))))) + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-cj/--count-active-transcriptions-only-complete () + "Test count when all transcriptions are complete." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil complete) + (proc2 "file2.m4a" nil complete)))) + (should (= 0 (cj/--count-active-transcriptions))))) + +(ert-deftest test-cj/--count-active-transcriptions-only-error () + "Test count when all transcriptions errored." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil error) + (proc2 "file2.m4a" nil error)))) + (should (= 0 (cj/--count-active-transcriptions))))) + +;; ----------------------------- Modeline Tests -------------------------------- + +(ert-deftest test-cj/--transcription-modeline-string-none-active () + "Test modeline string when no transcriptions active." + (let ((cj/transcriptions-list '())) + (should-not (cj/--transcription-modeline-string)))) + +(ert-deftest test-cj/--transcription-modeline-string-one-active () + "Test modeline string with one active transcription." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running)))) + (let ((result (cj/--transcription-modeline-string))) + (should result) + (should (string-match-p "⏺1" result))))) + +(ert-deftest test-cj/--transcription-modeline-string-multiple-active () + "Test modeline string with multiple active transcriptions." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running) + (proc2 "file2.m4a" nil running) + (proc3 "file3.m4a" nil running)))) + (let ((result (cj/--transcription-modeline-string))) + (should result) + (should (string-match-p "⏺3" result))))) + +(ert-deftest test-cj/--transcription-modeline-string-has-help-echo () + "Test that modeline string has help-echo property." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running)))) + (let ((result (cj/--transcription-modeline-string))) + (should (get-text-property 0 'help-echo result))))) + +(ert-deftest test-cj/--transcription-modeline-string-has-face () + "Test that modeline string has warning face." + (let ((cj/transcriptions-list + '((proc1 "file1.m4a" nil running)))) + (let ((result (cj/--transcription-modeline-string))) + (should (eq 'warning (get-text-property 0 'face result)))))) + +(provide 'test-transcription-counter) +;;; test-transcription-counter.el ends here diff --git a/tests/test-transcription-duration.el b/tests/test-transcription-duration.el new file mode 100644 index 00000000..370c439b --- /dev/null +++ b/tests/test-transcription-duration.el @@ -0,0 +1,58 @@ +;;; test-transcription-duration.el --- Tests for duration calculation -*- lexical-binding: t; -*- + +;;; Commentary: +;; Tests for cj/--transcription-duration function +;; Categories: Normal cases, Boundary cases + +;;; Code: + +(require 'ert) +(require 'transcription-config) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-cj/--transcription-duration-zero-seconds () + "Test duration calculation for current time (should be 00:00)." + (let ((now (current-time))) + (should (string= (cj/--transcription-duration now) "00:00")))) + +(ert-deftest test-cj/--transcription-duration-30-seconds () + "Test duration calculation for 30 seconds ago." + (let ((start-time (time-subtract (current-time) (seconds-to-time 30)))) + (should (string= (cj/--transcription-duration start-time) "00:30")))) + +(ert-deftest test-cj/--transcription-duration-1-minute () + "Test duration calculation for 1 minute ago." + (let ((start-time (time-subtract (current-time) (seconds-to-time 60)))) + (should (string= (cj/--transcription-duration start-time) "01:00")))) + +(ert-deftest test-cj/--transcription-duration-2-minutes-30-seconds () + "Test duration calculation for 2:30 ago." + (let ((start-time (time-subtract (current-time) (seconds-to-time 150)))) + (should (string= (cj/--transcription-duration start-time) "02:30")))) + +(ert-deftest test-cj/--transcription-duration-10-minutes () + "Test duration calculation for 10 minutes ago." + (let ((start-time (time-subtract (current-time) (seconds-to-time 600)))) + (should (string= (cj/--transcription-duration start-time) "10:00")))) + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-cj/--transcription-duration-59-seconds () + "Test duration just before 1 minute." + (let ((start-time (time-subtract (current-time) (seconds-to-time 59)))) + (should (string= (cj/--transcription-duration start-time) "00:59")))) + +(ert-deftest test-cj/--transcription-duration-1-hour () + "Test duration for 1 hour (60 minutes)." + (let ((start-time (time-subtract (current-time) (seconds-to-time 3600)))) + (should (string= (cj/--transcription-duration start-time) "60:00")))) + +(ert-deftest test-cj/--transcription-duration-format () + "Test that duration is always in MM:SS format with zero-padding." + (let ((start-time (time-subtract (current-time) (seconds-to-time 65)))) + (let ((result (cj/--transcription-duration start-time))) + (should (string-match-p "^[0-9][0-9]:[0-9][0-9]$" result))))) + +(provide 'test-transcription-duration) +;;; test-transcription-duration.el ends here diff --git a/tests/test-transcription-log-cleanup.el b/tests/test-transcription-log-cleanup.el new file mode 100644 index 00000000..82c902d8 --- /dev/null +++ b/tests/test-transcription-log-cleanup.el @@ -0,0 +1,44 @@ +;;; test-transcription-log-cleanup.el --- Tests for log cleanup logic -*- lexical-binding: t; -*- + +;;; Commentary: +;; Tests for cj/--should-keep-log function +;; Categories: Normal cases, Boundary cases + +;;; Code: + +(require 'ert) +(require 'transcription-config) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-cj/--should-keep-log-success-keep-disabled () + "Test that logs are deleted on success when keep-log is nil." + (let ((cj/transcription-keep-log-when-done nil)) + (should-not (cj/--should-keep-log t)))) + +(ert-deftest test-cj/--should-keep-log-success-keep-enabled () + "Test that logs are kept on success when keep-log is t." + (let ((cj/transcription-keep-log-when-done t)) + (should (cj/--should-keep-log t)))) + +(ert-deftest test-cj/--should-keep-log-error-keep-disabled () + "Test that logs are always kept on error, even if keep-log is nil." + (let ((cj/transcription-keep-log-when-done nil)) + (should (cj/--should-keep-log nil)))) + +(ert-deftest test-cj/--should-keep-log-error-keep-enabled () + "Test that logs are kept on error when keep-log is t." + (let ((cj/transcription-keep-log-when-done t)) + (should (cj/--should-keep-log nil)))) + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-cj/--should-keep-log-default-behavior () + "Test default behavior (should not keep on success)." + ;; Default is nil based on defcustom + (let ((cj/transcription-keep-log-when-done nil)) + (should-not (cj/--should-keep-log t)) + (should (cj/--should-keep-log nil)))) + +(provide 'test-transcription-log-cleanup) +;;; test-transcription-log-cleanup.el ends here diff --git a/tests/test-transcription-paths.el b/tests/test-transcription-paths.el new file mode 100644 index 00000000..5ee80e67 --- /dev/null +++ b/tests/test-transcription-paths.el @@ -0,0 +1,80 @@ +;;; test-transcription-paths.el --- Tests for transcription file path logic -*- lexical-binding: t; -*- + +;;; Commentary: +;; Tests for cj/--transcription-output-files and cj/--transcription-script-path +;; Categories: Normal cases, Boundary cases, Error cases + +;;; Code: + +(require 'ert) +(require 'transcription-config) + +;; ----------------------------- Normal Cases ---------------------------------- + +(ert-deftest test-cj/--transcription-output-files-simple () + "Test output file paths for simple filename." + (let ((result (cj/--transcription-output-files "meeting.m4a"))) + (should (string= (car result) "meeting.txt")) + (should (string= (cdr result) "meeting.log")))) + +(ert-deftest test-cj/--transcription-output-files-with-path () + "Test output file paths with full path." + (let ((result (cj/--transcription-output-files "/home/user/audio/podcast.mp3"))) + (should (string= (car result) "/home/user/audio/podcast.txt")) + (should (string= (cdr result) "/home/user/audio/podcast.log")))) + +(ert-deftest test-cj/--transcription-output-files-different-extensions () + "Test output files for various audio extensions." + (dolist (ext '("m4a" "mp3" "wav" "flac" "ogg")) + (let* ((input (format "audio.%s" ext)) + (result (cj/--transcription-output-files input))) + (should (string= (car result) "audio.txt")) + (should (string= (cdr result) "audio.log"))))) + +;; ----------------------------- Boundary Cases -------------------------------- + +(ert-deftest test-cj/--transcription-output-files-multiple-dots () + "Test output files for filename with multiple dots." + (let ((result (cj/--transcription-output-files "meeting.2025-11-04.final.m4a"))) + (should (string= (car result) "meeting.2025-11-04.final.txt")) + (should (string= (cdr result) "meeting.2025-11-04.final.log")))) + +(ert-deftest test-cj/--transcription-output-files-no-extension () + "Test output files for filename without extension." + (let ((result (cj/--transcription-output-files "meeting"))) + (should (string= (car result) "meeting.txt")) + (should (string= (cdr result) "meeting.log")))) + +(ert-deftest test-cj/--transcription-output-files-spaces-in-name () + "Test output files for filename with spaces." + (let ((result (cj/--transcription-output-files "team meeting 2025.m4a"))) + (should (string= (car result) "team meeting 2025.txt")) + (should (string= (cdr result) "team meeting 2025.log")))) + +(ert-deftest test-cj/--transcription-output-files-special-chars () + "Test output files for filename with special characters." + (let ((result (cj/--transcription-output-files "meeting_(final).m4a"))) + (should (string= (car result) "meeting_(final).txt")) + (should (string= (cdr result) "meeting_(final).log")))) + +;; ----------------------------- Script Path Tests ----------------------------- + +(ert-deftest test-cj/--transcription-script-path-local-whisper () + "Test script path for local-whisper backend." + (let ((cj/transcribe-backend 'local-whisper)) + (should (string-suffix-p "scripts/local-whisper" + (cj/--transcription-script-path))))) + +(ert-deftest test-cj/--transcription-script-path-openai-api () + "Test script path for openai-api backend." + (let ((cj/transcribe-backend 'openai-api)) + (should (string-suffix-p "scripts/oai-transcribe" + (cj/--transcription-script-path))))) + +(ert-deftest test-cj/--transcription-script-path-absolute () + "Test that script path is absolute." + (let ((path (cj/--transcription-script-path))) + (should (file-name-absolute-p path)))) + +(provide 'test-transcription-paths) +;;; test-transcription-paths.el ends here @@ -17,7 +17,7 @@ If the answer is "no" to all five → DON'T ADD IT. V2MOM is located at: [[file:docs/emacs-config-v2mom.org][emacs-config-v2mom.org]] Research/ideas that don't serve vision: [[file:docs/someday-maybe.org][someday-maybe.org]] -* Method 1: Make Using Emacs Frictionless [7/13] +* Method 1: Make Using Emacs Frictionless [8/14] ** DONE [#A] Remove network check from startup (saves 1+ seconds) CLOSED: [2025-10-31 Fri] @@ -194,6 +194,52 @@ Generate transcript (once transcription workflow exists). Daily workflow improvement. +** TODO [#A] Fix Google Calendar password prompts every 15 minutes + +IRRITANT: gcal-sync triggers password prompts approximately every 15 minutes, +interrupting workflow and breaking focus. This defeats the purpose of having +passphrase caching configured. + +**Current Setup:** +- GPG agent configured with 400-day cache (gpg-agent.conf): + - default-cache-ttl 34560000 + - max-cache-ttl 34560000 + - allow-loopback-pinentry enabled +- Plstore caching enabled (auth-config.el:54): + - plstore-cache-passphrase-for-symmetric-encryption t + - plstore-encrypt-to nil (symmetric encryption) +- Auth-source cache: 24 hours (auth-config.el:31) +- Auto-sync interval: 30 minutes (org-gcal-config.el:50) + +**Problem:** +Despite proper GPG agent caching, oauth2-auto.plist prompts for passphrase +every ~15 minutes during gcal-sync operations. This suggests: +1. plstore may not be using GPG agent cache properly for symmetric encryption +2. oauth2-auto token refresh might be bypassing cache +3. EPinentry mode may need explicit configuration (currently commented out) + +**Goal:** +Passphrase should be entered ONCE per Emacs session, then cached until Emacs +closes. No interruptions during normal work. + +**Investigation Paths:** +1. Check if oauth2-auto respects plstore passphrase caching +2. Investigate plstore symmetric encryption cache behavior with GPG agent +3. Test enabling epa-pinentry-mode 'loopback (auth-config.el:42) +4. Check oauth2-auto token refresh cycle vs password prompt timing +5. Consider oauth2-auto configuration options for token persistence +6. Review org-gcal or oauth2-auto issues for similar problems + +**Files:** +- modules/auth-config.el (plstore and GPG configuration) +- modules/org-gcal-config.el (org-gcal and oauth2-auto setup) +- ~/.gnupg/gpg-agent.conf (GPG agent cache settings) +- oauth2-auto.plist (encrypted OAuth tokens - prompts every access?) + +**Related:** +This violates the "Frictionless" value - interruptions every 15 minutes during +calendar sync breaks concentration and workflow momentum. + ** TODO [#B] Optimize org-agenda performance using built-in profiler THE BOTTLENECK. Currently 30+ seconds, target < 5 seconds. @@ -204,9 +250,26 @@ Use M-x profiler-start before Method 3 debug-profiling.el is built. 15-20 seconds every time capturing a task (12+ times/day). Major daily bottleneck - minutes lost waiting, plus context switching cost. -** TODO [#C] Fix grammar checker performance (currently disabled) +** DONE [#C] Fix grammar checker performance (currently disabled) +CLOSED: [2025-11-04 Mon] + +✅ **Installed and configured LanguageTool for comprehensive grammar checking** + +Replaced disabled grammar checker with on-demand LanguageTool integration: +- Installed LanguageTool 6.6-2 from Arch repos (222MB) +- Created wrapper script: scripts/languagetool-flycheck (Python 3) +- Integrated with flycheck for on-demand checking via C-; ? +- Removed proselint (redundant - LanguageTool catches more) +- No performance impact: only runs when explicitly invoked +- Installation instructions added to modules/flycheck-config.el commentary + +LanguageTool catches: +- Real grammar errors (subject-verb agreement, tense, etc.) +- Missing punctuation (commas, periods) +- Common mistakes (could of → could have) +- Style issues (redundant phrases, wordiness) -Currently disabled because it breaks flow when writing. +Workflow: Open org/text/markdown file → press C-; ? → see errors in *Flycheck errors* buffer ** TODO [#D] Fix EMMS keybinding inconsistency with other buffers @@ -214,7 +277,7 @@ EMMS keybindings conflict with standard buffer keybindings, causing mistypes. Results in accidental destructive actions (clearing buffers), requires undo + context switch. Violates Intuitive value - muscle memory should help, not hurt. -* Method 2: Stop Problems Before They Appear [2/5] +* Method 2: Stop Problems Before They Appear [3/5] ** TODO [#B] Migrate from Company to Corfu :PROPERTIES: @@ -223,15 +286,37 @@ Violates Intuitive value - muscle memory should help, not hurt. Complete config already exists in someday-maybe.org. Just needs to be executed. -** DONE [#C] Switch to mood-line +** DONE [#C] Switch to mood-line (actually: built custom modeline) CLOSED: [2025-11-03 Sun] -✅ Replaced doom-modeline with mood-line in modules/modeline-config.el -- Simplified from 70+ lines to ~10 lines of configuration -- mood-line is lightweight, minimal, inspired by doom-modeline -- Better performance with no dependencies -- Features: buffer status, major mode, VCS, flycheck, cursor position -- Net reduction: -40 lines of configuration code +✅ **Built custom modeline (better than mood-line)** + +Went beyond mood-line and created a fully custom modeline using only built-in +Emacs functionality to avoid native-compilation issues. Features: + +**Architecture:** +- Named segment system using defvar-local (easy to reorder) +- Emacs 30 built-in right-alignment (mode-line-format-right-align) +- All segments marked as risky-local-variable for proper evaluation + +**Interactive Features:** +- Color-coded buffer names (green=writeable, red=read-only, gold=overwrite) +- VC branch with git symbol () and state-based coloring +- Mouse click handlers: buffer name (prev/next), branch (vc-diff), mode (describe-mode) +- Help-echo tooltips on all segments +- String truncation for narrow windows (< 100 chars wide) +- Active-window-only display for branch and misc-info (less clutter) + +**Layout:** +- Left: Major mode, buffer name, position (L:# C:#) +- Right: Git branch, chime notifications + +**Bug Fixes:** +- Disabled async native compilation (prevented "Selecting deleted buffer" errors) +- Fixed difftastic loading (:demand → :defer) +- Abstracted buffer status colors to user-constants.el + +Inspired by Prot's modeline design. See commit c0c4b17. ** DONE [#C] Remove deprecated tree-sitter package (rely on treesit-auto) CLOSED: [2025-11-03 Sun] @@ -242,21 +327,31 @@ CLOSED: [2025-11-03 Sun] - treesit-auto continues to manage tree-sitter grammars automatically - No functionality lost, cleaner dependency tree -** TODO [#C] Add org-appear (show emphasis markers only when point is on them) +** DONE [#C] Add org-appear (show emphasis markers only when point is on them) +CLOSED: [2025-11-03 Sun] -Cleaner org files, supports Intuitive value. +✅ Added org-appear package to modules/org-config.el +- Shows emphasis markers (* / _) only when cursor is on them +- Also shows link markup and sub/superscripts when editing +- Changed org-fontify-emphasized-text from nil to t (line 75) +- Updated comment on line 71 to reference org-appear +- Cleaner editing experience while maintaining visual clarity ** TODO [#C] Integrate prescient with Corfu (smart sorting) Already using prescient with vertico. Extend to Corfu after migration. -* Method 3: Make *Fixing* Emacs Frictionless [0/5] +* Method 3: Make *Fixing* Emacs Frictionless [1/4] ** TODO [#B] Build debug-profiling.el module Reusable profiling infrastructure for any future performance work. -** TODO [#C] Document localrepo limitations (treesitter grammars not included) +** TODO [#C] Build localrepo out (package snapshot system) + +Repeatable installs and safe rollbacks. + +*** TODO [#C] Document localrepo limitations (treesitter grammars not included) .localrepo only contains packages from package.el archives. Treesitter grammars are downloaded separately by treesit-auto on first use. @@ -266,10 +361,6 @@ For true offline reproducibility, need to cache treesitter grammars separately. Complex workflow testing capability. -** TODO [#C] Build localrepo out (package snapshot system) - -Repeatable installs and safe rollbacks. - ** DONE [#C] Integrate difftastic (structural diffs) CLOSED: [2025-11-03 Sun] @@ -288,6 +379,29 @@ CLOSED: [2025-11-03 Sun] Result: Better diffs everywhere - ediff for interactive buffer comparison, difftastic for understanding git changes. +** TODO [#C] Remove orphaned dwim-shell-security tests and unused production code + +Why: 12 tests in test-dwim-shell-security.el fail because the functions they test +are inside a use-package :config block (dwim-shell-config.el:101-108) that only +loads when the dwim-shell-command package is available. During batch testing, +the package isn't loaded, so functions are never defined (void-function errors). + +These are PDF password protection and ZIP encryption functions that likely have +never been used in practice - they're placeholder code from initial setup. + +What to delete: +1. Test file: tests/test-dwim-shell-security.el (12 failing tests) +2. Production functions in modules/dwim-shell-config.el (lines ~302-347): + - cj/dwim-shell-commands-pdf-password-protect (lines 302-324) + - cj/dwim-shell-commands-pdf-password-unprotect (lines 326-347) + - cj/dwim-shell-commands-create-encrypted-zip (search for it) + - cj/dwim-shell-commands-remove-zip-encryption (search for it) + +After deletion: Run "make test-all" to confirm 18 failures → 6 failures +(only benchmark performance tests remain, which are environment-dependent). + +Aligns with: Reducing test failures from 18 to 6, cleaning up unused code. + * Method 4: Contribute to the Emacs Ecosystem [0/4] ** TODO [#C] Set up package-lint for elisp linting (chime, org-msg, wttrin) @@ -336,3 +450,4 @@ Review this inbox, cancel stale items, keep < 20 active. Track in calendar. Can't research next thing until current thing is implemented. * Emacs Config Inbox +** TODO cj/flyspell-then-abbrev loses keybinding in scratch org-mode buffer |