diff options
| -rw-r--r-- | .claude/commands/wrap-it-up.md | 25 | ||||
| -rw-r--r-- | .gitignore | 6 | ||||
| -rw-r--r-- | NOTES.org | 184 | ||||
| -rw-r--r-- | assets/abbrev_defs | 11 | ||||
| -rw-r--r-- | assets/gemini.key.gpg (renamed from gemini.key.gpg) | 0 | ||||
| -rw-r--r-- | docs/EMACS-CONFIG-V2MOM.org | 363 | ||||
| -rw-r--r-- | docs/NOTES.org | 887 | ||||
| -rw-r--r-- | docs/SOMEDAY-MAYBE.org | 1633 | ||||
| -rw-r--r-- | docs/sessions/create-session.org | 352 | ||||
| -rw-r--r-- | docs/sessions/emacs-inbox-zero.org | 338 | ||||
| -rw-r--r-- | docs/sessions/refactor.org | 617 | ||||
| -rw-r--r-- | docs/values-comparison.org | 107 | ||||
| -rw-r--r-- | early-init.el | 8 | ||||
| -rw-r--r-- | init.el | 1 | ||||
| -rw-r--r-- | issues.org | 1881 | ||||
| -rw-r--r-- | modules/auth-config.el | 82 | ||||
| -rw-r--r-- | modules/config-utilities.el | 12 | ||||
| -rw-r--r-- | modules/dirvish-config.el | 17 | ||||
| -rw-r--r-- | modules/org-gcal-config.el | 5 | ||||
| -rw-r--r-- | modules/system-utils.el | 13 | ||||
| -rw-r--r-- | modules/transcription-config.el | 96 | ||||
| -rw-r--r-- | modules/video-audio-recording.el | 12 | ||||
| -rwxr-xr-x | scripts/assemblyai-transcribe | 134 | ||||
| -rw-r--r-- | tests/test-transcription-config--transcription-script-path.el | 106 | ||||
| -rw-r--r-- | todo.org | 84 |
25 files changed, 522 insertions, 6452 deletions
diff --git a/.claude/commands/wrap-it-up.md b/.claude/commands/wrap-it-up.md deleted file mode 100644 index cdf743d5..00000000 --- a/.claude/commands/wrap-it-up.md +++ /dev/null @@ -1,25 +0,0 @@ -# Wrap Up Session - -When the user says "let's wrap it up", "that's a wrap", or "let's call it a wrap", execute this complete workflow: - -## 1. Write Session Notes -Add anything important to `docs/NOTES.org` for next session: -- Key decisions made -- Work completed -- Context needed for continuation -- Any pending issues or blockers - -## 2. Commit and Push Changes -- Run `git status` to see what changed -- Run `git diff` to review changes -- Create a descriptive commit message following repository conventions -- Commit with co-author attribution (Claude) -- Push to ALL remotes (origin + github) - -## 3. Valediction -Give a friendly goodbye with: -- Brief summary of what was accomplished -- What's ready for next session -- Any important reminders - -Execute this workflow now. @@ -50,3 +50,9 @@ auto-save-list/ /multisession/ /browser-choice.el /client_secret_491339091045-sjje1r54s22vn2ugh45khndjafp89vto.apps.googleusercontent.com.json + +# Documentation folder (session notes, personal workflows) +/docs/ + +# Claude Code configuration (personal settings, commands) +/.claude/ diff --git a/NOTES.org b/NOTES.org deleted file mode 100644 index 0e7b92a2..00000000 --- a/NOTES.org +++ /dev/null @@ -1,184 +0,0 @@ -#+TITLE: Project Notes for Future Sessions -#+AUTHOR: Claude (AI Assistant) -#+DATE: 2025-11-01 - -* Session Startup Protocol - -When starting a new session with Craig: -1. Craig will point me to this NOTES.org file -2. Read this file completely to understand project context -3. **Scan =docs/sessions/= directory** for available session type filenames - - Do NOT read full contents (they're long) - - Just note the filenames to understand vocabulary - - Example: =refactor.org= means "refactor session workflow" -4. When Craig says "let's do a [session-type] session": - - **Check for exact match** in scanned filenames - - **If exact match found:** Read =docs/sessions/[session-type].org= and guide Craig through that workflow - - **If no exact match but similar word exists:** Ask for clarification - - Example: User says "refactoring" but we have "refactor.org" - - Ask: "Did you mean the 'refactor' session, or would you like to create a new 'refactoring' session?" - - **If no match at all:** Offer to create it - - Say: "I don't see a session type called '[session-type]' yet. Would you like to create it together using the create-session workflow?" - - **If yes to create:** Do a create-session session to define the new type, then use it immediately (validation) - -* Available Session Types - -** refactor -File: =docs/sessions/refactor.org= - -Workflow for comprehensive test-driven quality engineering: -1. Analyze test coverage for target file -2. Categorize functions by testability (easy/medium/hard) -3. Refactor for testability if needed (pure helpers + interactive wrappers) -4. Write ERT tests systematically (Normal/Boundary/Error categories) -5. Run tests and analyze failures -6. Decide: test bug or production bug? -7. Fix and verify -8. Repeat for all testable functions - -Recent completion: =modules/music-config.el= - 103 tests, 3 bugs fixed - -** create-session -File: =docs/sessions/create-session.org= - -Meta-workflow for creating new session types: -1. Q&A discovery (Problem, Exit criteria, Approach, Principles) -2. Assess completeness and agreement -3. Name the session -4. Document using standard structure -5. Update NOTES.org -6. Validate by immediate execution - -Defines how we formalize repetitive workflows into reusable, documented sessions. - -Created: 2025-11-01 - -* Current Project State - -** Completed Work - -*** music-config.el Testing Session (2025-11-01) -- **Module:** =modules/music-config.el= (EMMS music player configuration) -- **Test Coverage:** 9 functions, 103 tests total -- **Bugs Fixed:** 3 (nil handling x2, URL regex escaping bug) -- **Session Document:** =docs/sessions/refactor.org= - -Functions tested: -1. =cj/music--append-track-to-m3u-file= - 11 tests -2. =cj/music--valid-file-p= - 15 tests -3. =cj/music--valid-directory-p= - 13 tests -4. =cj/music--safe-filename= - 13 tests -5. =cj/music--m3u-file-tracks= - 15 tests (caught URL regex bug!) -6. =cj/music--get-m3u-files= - 7 tests -7. =cj/music--get-m3u-basenames= - 6 tests -8. =cj/music--collect-entries-recursive= - 12 tests -9. =cj/music--completion-table= - 12 tests - -Functions skipped (EMMS buffer-dependent, hard to test): -- =cj/music--ensure-playlist-buffer= -- =cj/music--playlist-tracks= -- =cj/music--playlist-modified-p= -- =cj/music--assert-valid-playlist-file= - -Test files location: =tests/test-music-config--*.el= - -Key bug found: URL regex escaping in =cj/music--m3u-file-tracks= caused HTTP/HTTPS/MMS URLs to be treated as relative paths, corrupting playlist data. - -** Work In Progress - -(None currently - ready for next task) - -** Backlog / Future Work - -(To be added as Craig identifies next priorities) - -* Project Structure Notes - -** Test Organization -- One test file per function: =test-<module>-<function>.el= -- Rationale: Fast discovery when tests fail -- Test utilities: =tests/testutil-general.el= - -** Test Categories -Every test file follows this structure: -- Normal Cases: Expected behavior under typical conditions -- Boundary Cases: Edge conditions, limits, unusual inputs -- Error Cases: Failure scenarios, invalid inputs - -** Quality Engineering Reference -See =ai-prompts/quality-engineer.org= for comprehensive guidelines. -Note: This file evolves as we learn best practices. - -* Session History - -** 2025-11-01: music-config.el Refactor Session -- Added comprehensive test coverage to music configuration -- Found and fixed 3 bugs -- Created refactor session workflow document -- Established session type pattern for future work - -** 2025-11-01: Create-Session Meta-Workflow -- Designed the process for creating new session types -- Used recursive application: created create-session using itself -- Defined four-question Q&A framework -- Established seven core principles for session creation -- Documented in =docs/sessions/create-session.org= - -** 2025-11-03: Method 2 Quick Wins Session -- Completed 6 tasks across Methods 1, 2, and 3 -- Focus: Low-risk improvements aligned with V2MOM Methods - -*** Key Technical Learnings - -**** Difftastic vs Ediff - Complementary Tools -- **Difftastic**: Read-only structural diffs for git (language-aware) -- **Ediff**: Interactive side-by-side editing for buffer comparison -- **Discovery**: User had ediff at C-c D but C-; b D used basic diff -- **Solution**: Hybrid approach - ediff for buffers, difftastic for git -- **Lesson**: Don't assume one replaces the other; they serve different purposes - -**** Org-mode Visual Emphasis -- =org-fontify-emphasized-text= controls bold/italic rendering -- =org-hide-emphasis-markers= hides markup characters (* / _) -- =org-appear= shows markers only when cursor is on emphasized text -- **Pattern**: Hide markers globally, reveal contextually with org-appear - -*** User Workflow Preferences - -**** Task Selection Strategy -- User prefers quick wins when energy is lower -- Appreciates effort estimates BEFORE committing to work -- Will defer mission-critical changes (e.g., Company → Corfu) -- Quote: "now I'm hungry for more" after successful quick wins - -**** Communication Patterns -- User sometimes needs clarification iterations ("whoops. I wasn't clear again") -- Always confirm understanding before major changes -- User values knowing keybindings for new features -- Appreciates when tasks are already complete (just need verification) - -**** V2MOM Task Management -- Tasks organized by 6 Methods with progress tracking (e.g., 7/13, 3/5) -- User maintains detailed completion notes in todo.org -- Each task includes context, rationale, and completion details -- Weekly triage scheduled to keep active todos < 20 - -*** Development Patterns - -**** Git Workflow -- Always push to BOTH remotes: origin (cjennings.net) AND github -- Commit messages include Claude Code attribution -- User reviews git status and diffs before committing - -**** Configuration Management -- User runs Emacs 30.2 with use-package -- Prefers editing existing files over creating new ones -- Values code reduction (mood-line: -40 lines, ediff upgrade: -12 lines) - -*** Tasks Completed -1. Switched doom-modeline → mood-line (Method 2) -2. Removed deprecated tree-sitter package (Method 2) -3. Fixed cj/goto-git-gutter-diff-hunks void-function (Method 1) -4. Cleaned up go-ts-mode-map declarations (Method 1) -5. Upgraded buffer diff to ediff + added difftastic for git (Method 3) -6. Added org-appear and enabled emphasis rendering (Method 2) diff --git a/assets/abbrev_defs b/assets/abbrev_defs index 3060620f..cd9c6818 100644 --- a/assets/abbrev_defs +++ b/assets/abbrev_defs @@ -33,7 +33,7 @@ ("aethetically" "aesthetically" nil :count 0) ("agression" "aggression" nil :count 0) ("agressive" "aggressive" nil :count 0) - ("ahve" "have" nil :count 14) + ("ahve" "have" nil :count 15) ("aknowledge" "acknowledge" nil :count 0) ("alegiance" "allegiance" nil :count 0) ("allegaince" "allegiance" nil :count 0) @@ -215,7 +215,7 @@ ("hrie" "hire" nil :count 0) ("htey" "they" nil :count 3) ("humerous" "humorous" nil :count 0) - ("hygeine" "hygiene" nil :count 0) + ("hygeine" "hygiene" nil :count 1) ("hygene" "hygiene" nil :count 0) ("hygine" "hygiene" nil :count 0) ("idnetify" "identify" nil :count 0) @@ -294,7 +294,7 @@ ("oppositiion" "opposition" nil :count 0) ("opppsite" "opposite" nil :count 0) ("orignal" "original" nil :count 0) - ("ot" "to" nil :count 41) + ("ot" "to" nil :count 42) ("otehr" "other" nil :count 3) ("otes" "notes" nil :count 0) ("outgoign" "outgoing" nil :count 0) @@ -393,7 +393,7 @@ ("takss" "tasks" nil :count 3) ("talekd" "talked" nil :count 0) ("talkign" "talking" nil :count 6) - ("teh" "the" nil :count 155) + ("teh" "the" nil :count 156) ("tehir" "their" nil :count 5) ("tehre" "there" nil :count 3) ("testimentary" "testamentary" nil :count 1) @@ -428,10 +428,11 @@ ("warant" "warrant" nil :count 0) ("welfair" "welfare" nil :count 0) ("welomce" "welcome" nil :count 0) + ("whcih" "which" nil :count 0) ("whenter" "whether" nil :count 0) ("whenver" "whenever" nil :count 0) ("wierd" "weird" nil :count 0) - ("wihtout" "without" nil :count 0) + ("wihtout" "without" nil :count 1) ("windsheild" "windshield" nil :count 0) ("withdrawls" "withdrawals" nil :count 0) ("withold" "withhold" nil :count 0) diff --git a/gemini.key.gpg b/assets/gemini.key.gpg index 10068445..10068445 100644 --- a/gemini.key.gpg +++ b/assets/gemini.key.gpg diff --git a/docs/EMACS-CONFIG-V2MOM.org b/docs/EMACS-CONFIG-V2MOM.org deleted file mode 100644 index 40027218..00000000 --- a/docs/EMACS-CONFIG-V2MOM.org +++ /dev/null @@ -1,363 +0,0 @@ -#+TITLE: Emacs Configuration V2MOM -#+AUTHOR: Craig Jennings -#+DATE: 2025-10-30 -#+FILETAGS: :strategy:planning:v2mom: - -* What is V2MOM? - -V2MOM is a strategic framework used to clarify goals and priorities: -- *Vision:* What do you want to achieve? -- *Values:* What principles guide decisions? -- *Methods:* How will you achieve the vision? -- *Obstacles:* What's in your way? -- *Metrics:* How will you measure success? - -This document provides strategic clarity for prioritizing Emacs configuration work. - -* Vision - -Your personal ideal Emacs configuration allows you to work at the speed of your own thought. It's stable and never needs edge-case bug fixing. The functionality covers your workflow such that you can work all day in Emacs for any of the following task areas: - -- Programming in any language, including emacs-lisp, common-lisp, scheme, golang, and c -- Reading ebooks, including marginalia handling -- Listening to music, including playlist management -- Listening to podcasts, including saving episodes -- Task organizing and planning -- Calendaring, including syncing with google calendar -- Alerting and notifications on tasks and calendar events -- Timer/stopwatch functionality, including pomodoro presets -- Audio and desktop recording -- Memorization drills -- Email, including email-to-task integration -- Messaging, i.e., IRC -- File-management, including extensive integration, and DWM scratchpad integration outside Emacs -- Shell commands, leveraging vterm and eshell with custom methods, and DWM scratchpad integration outside Emacs - -Anytime you make a change in the config, you have unit tests to tell you quickly if the change created unexpected issues. You have a test-runner that integrates with your workflow. - -* Values - -also see: file:values-comparison.org - -** Intuitive - -*Definition:* Intuition comes from muscle memory, clear mnemonics, and just-in-time discovery that reinforces learning without blocking productivity. - -*** Keybindings Build Intuition Through Layers -1. *Muscle memory first* - C-; on home row becomes automatic habit -2. *Mnemonics provide "aha!" moments* - C-c **m** for mail creates dopamine hits that cement permanent memory -3. *Which-key teaches through active recall* - Appears after you try (not before), forcing attempt without wasting time - -The timing matters: which-key's delay is calibrated so you're challenged to remember, but not blocked from working. - -*** What Breaks Intuition -- *Physically awkward keybindings* - C-c 1 z C-c f requires hand contortions and offers no mnemonic -- *Syntax overhead* - Parentheses tracking steals time from real learning (logic bugs, architecture understanding) -- *Poor architecture* - Common actions (like "save all attachments") shouldn't require architectural archaeology - -*** Code Reads "Like a Newspaper" -- Each use-package declaration is a *story* (topic = package) -- Configuration settings are *facts* (what it does, how it behaves) -- Clear variable/method naming (yours + maintainers) makes code read like English -- Accessible to intermediate elisp readers without deep archaeology - -*** Principle -*Intuition is earned through repeated success, not instant magic.* The config should make success easy to achieve and memorable when it happens. - - - -** Fast - -*Definition:* Fast means staying in flow. Startup and org-agenda must not make me wait. Everything else is already acceptable. - -*** Performance Targets - -*Startup Time:* -- Current: ~3.8 seconds (real clock time, not Emacs-reported) -- Target: ~1 second (aspirational, may not be realistic given package count) -- Comfortable: ~2 seconds (±) - -*Measurement:* -#+begin_src bash -time emacs --eval '(save-buffers-kill-emacs)' # Real startup time -#+end_src - -*Note:* Emacs-reported time is inaccurate. Messages buffer continues populating and Emacs remains unresponsive after "loaded" message appears. - -*** Critical Performance Issues (Must Fix) - -1. *Emacs startup* - 3.8s is too slow - - Network check adds 1+ seconds (DELETE THIS) - - Eager loading vs lazy loading - -2. *Opening org-agenda for first time* - **THE BOTTLENECK** - - "Takes forever and a full work day" - - Likely: recursive directory scan, no caching, too many files - - Solution: Profile first (build debug-profiling.el), then optimize - -*** Must Be Instant (Flow-Critical) -- Keybinding response -- Changing projects in projectile (already good ✓) -- Opening large org files (already good ✓, except malformed files with stray "*"s) -- Buffer/window operations - -*** Can Take 1-2 Seconds (Acceptable) -- *Mail sync* - Consider putting on timer instead of manual trigger -- *Calibredb* (PDFs/Epubs) - Known solution: shrink inventory to <1k books or contribute optimizations -- *Org-noter* opening PDFs/Epubs - Needs reliability + reasonable speed -- *Looking up org-noter notes* - Should be easy to pull into other documents -- *Initial file indexing* (projectile, etc.) - -*** Disabled Due to Performance (Need Solutions) -- *Grammar checker* - Installed but turned off because it slows everything down - - No task exists to fix this yet - - Breaks concentration when writing - -- *Bibliography management* - Don't know how to do in Emacs - - Worried about broken/slow tools breaking writing flow - - Need solution that "just works" - -*** Principle -*Speed is about flow, not milliseconds.* Startup and agenda are flow-breakers. Everything else is fine. Don't optimize what doesn't hurt. - - - -** Simple - -*Definition:* Simple means clear, maintainable code following production software practices. Simplicity is a discipline that produces reliability. Config is production software - treat it as such. - -*** What Makes Code Simple -1. *Clear naming* - Variables and functions say what they do -2. *Small functions* - Easy to understand, test, and modify -3. *Good comments* - NOT obvious comments ("cat with post-it note saying 'cat'") - -*Good comments:* -- Navigation markers for long files -- Explain complex/non-obvious logic -- Document unobvious impacts of settings -- Answer "why" not "what" - -*Bad comments:* -#+begin_src emacs-lisp -;; declare a variable -(defvar variable 10) -#+end_src - -*** Abstractions and Magic -- *Abstractions are fine* if they serve a purpose -- *Macros are irritating* when unnecessary, but acceptable when warranted -- *Magic is acceptable* if it's reliable and you can build a mental model - -*Principle:* "I want to understand every line of what my config does, or at least have a decent mental model of how it probably works. Never use a tool you can't fully trust." - -*** Package Count -Package count itself isn't a concern. Quality and maintainability matter. - -*Concerns:* -- More packages = more potential breakage -- Need ability to roll back when packages break (localrepo for package snapshots) -- Maintenance burden is real, but manageable with good practices - -*Not concerned about:* -- Absolute number of packages -- Startup time impact (if lazy-loaded properly) -- Cognitive load (if organized well) - -*** Config IS Production Software - -*Key insight:* "My Emacs config IS production software. I use it more than probably every other application combined. The impact when it's broken can crush my productivity." - -This means: -- *Apply production practices:* Testing, modularity, documentation, version control -- *Plan for failure:* Rollback capability (localrepo), error handling, graceful degradation -- *Modular design:* Extract custom code into packages when it grows -- *Long-term maintenance:* Architecture matters, technical debt has real cost - -*** Personal Software Platform -"I see my config as a personal software project built on the software platform called Emacs. Emacs has taught me a great deal about how to build software platforms well." - -*Implications:* -- Config follows software engineering principles -- Custom functionality becomes packages (chime, org-msg, potentially jumper) -- Good architecture enables evolution -- Quality matters because uptime matters - -*** Why Simplicity Matters -When config breaks, productivity is crushed. The discipline of simplicity produces reliability: -- Simple code is easier to fix when it breaks -- Simple architecture enables graceful degradation -- Simple practices (testing, rollback) provide safety nets -- Simple means never using tools you can't trust - -*** Principle -*Simple means maintainable production software, not minimal or primitive.* Complexity is acceptable when it serves the vision and can be understood or trusted. **Simplicity is the practice that produces reliability as an outcome.** - - - -** Summary: Three Core Values - -1. *Intuitive* - Muscle memory, mnemonics, just-in-time discovery. Code reads like a newspaper. -2. *Fast* - Startup and org-agenda must not break flow. Everything else is acceptable. -3. *Simple* - Clear, maintainable, production-quality code. Simplicity produces reliability. - - - -* Methods - -** Method 1: Make Using Emacs Frictionless - -Emacs should never make you wait or break your concentration. This method eliminates daily friction points that disrupt flow. Every item here either removes a bottleneck (startup delay, org-agenda slowness), fixes something broken (org-noter, recording, mail attachments), or adds missing functionality you reach for weekly (diff-buffer-with-file). - -When Method 1 is complete, Emacs starts fast, org-agenda opens instantly, all core workflows work reliably, and nothing makes you context-switch to debug config during work. - -*Concrete actions:* -- Remove network check from startup (saves 1+ seconds every launch) -- Optimize org-agenda performance using built-in profiler (eliminate "forever and a full work day" rebuild time) -- Fix [[https://github.com/weirdNox/org-noter][org-noter]] (reading/annotation workflow currently "so painful") -- Fix video/audio recording module (use it constantly, just broke) -- Fix mail attachment workflow (currently awkward) -- Implement cj/diff-buffer-with-file (compare buffer with saved version - weekly need) -- Fix cj/goto-git-gutter-diff-hunks (missing function causing errors) -- Fix grammar checker performance (currently disabled because it breaks flow) - -** Method 2: Stop Problems Before They Appear - -A stable config comes from proactive maintenance, not reactive fixes. This method replaces aging packages with modern, actively-maintained alternatives before they break. It removes deprecated dependencies and adopts better-designed tools that align with Emacs' evolution. - -When Method 2 is complete, the config uses current best practices, has no deprecated packages lurking as time bombs, and benefits from simpler, more maintainable completion infrastructure. - -*Concrete actions:* -- Migrate from Company to [[https://github.com/minad/corfu][Corfu]] (simpler, modern completion framework - complete config already in todo.org) -- Switch to [[https://gitlab.com/jessieh/mood-line][mood-line]] (lighter modeline, already researched) -- Remove deprecated tree-sitter package and rely on [[https://github.com/renzmann/treesit-auto][treesit-auto]] (already installed, leverages Emacs 29+ built-in treesit) -- Add [[https://github.com/awth13/org-appear][org-appear]] (show emphasis markers only when point is on them - cleaner org files) -- Integrate [[https://github.com/radian-software/prescient.el][prescient]] with Corfu (frequency/recency-based smart sorting - already using with vertico) - -** Method 3: Make *Fixing* Emacs Frictionless - -You can't fix what you can't measure, and you can't trust what you can't test. This method builds observability and testing infrastructure that makes future maintenance systematic instead of guesswork. With proper profiling, testing, and diffing tools in place, debugging becomes fast and confident. - -When Method 3 is complete, you can profile any performance issue in seconds, write integration tests for complex workflows, roll back broken packages instantly, and review config changes with semantic understanding. - -*Concrete actions:* -- Build debug-profiling.el module and develop skills using it (reusable profiling infrastructure for any future performance work) -- Integrate [[https://github.com/jorgenschaefer/emacs-buttercup][Buttercup]] (behavior-driven integration tests for complex config workflows) -- Build localrepo out (package snapshot system for repeatable installs and safe rollbacks) -- Integrate [[https://github.com/Wilfred/difftastic][difftastic]] (structural diffs that show semantic changes, not just line changes) - -** Method 4: Contribute to the Emacs Ecosystem - -Maintaining packages (chime, org-msg, wttrin) means being a good steward of code others depend on. This method establishes professional package development practices: automated linting, CI testing, and coverage reporting. These tools catch issues before users do and make MELPA submissions smooth. - -When Method 4 is complete, every package you maintain has automated quality checks, measurable test coverage, and CI that validates changes before they ship. You contribute back to the community with confidence. - -*Concrete actions:* -- Set up [[https://github.com/purcell/package-lint][package-lint]] for elisp linting (catch packaging issues and style violations automatically) -- Set up [[https://github.com/riscy/melpazoid][melpazoid]] CI (validates packages meet MELPA standards before submission) -- Set up [[https://github.com/leotaku/elisp-check][elisp-check]] GitHub Action (zero-config CI for Emacs packages) -- Integrate [[https://github.com/undercover-el/undercover.el][undercover.el]] (measure and track test coverage over time) - -** Method 5: Be Kind To Your Future Self - -With Emacs stable and maintainable, it's time to add features that expand what's possible. This method builds workflows you'll use repeatedly: transcribing audio for notes and creating presentations directly from org-mode. These aren't fixes—they're investments in future capability. - -When Method 5 is complete, you can transcribe recordings without leaving Emacs and generate beautiful reveal.js presentations from org files. Future you will thank present you for shipping these. - -*Concrete actions:* -- Add transcription workflow (complete code already in todo.org:2-99 - need today and recurring) -- Implement [[https://github.com/yjwen/org-reveal][org-reveal]] presentation workflow (create reveal.js slides from org-mode) - -** Method 6: Develop Disciplined Engineering Practices - -The best infrastructure won't help without disciplined habits. This method builds the practices that make all other methods sustainable: knowing what matters most, shipping over researching, measuring before optimizing. These are skills to develop, not tasks to complete—they evolve throughout the entire V2MOM, and I become a better engineer overall. - -*Concrete practices to develop:* -- *Ruthless prioritization* - Always do the most important thing. Use V2MOM as filter. If it doesn't serve the vision, cancel it. -- *Weekly triage* - Review todos every Sunday (30 min). Items sitting >1 week get shipped or killed. Prevents backlog rot. Keep <20 active items. -- *Measure metrics* - Define success criteria before starting work. Quantify outcomes so you know when you're done (startup time, test coverage, todo count). -- *Profile-before-optimize* - Never guess what's slow. Run profiler, identify hotspot, THEN fix. Avoids wasting time optimizing things that don't matter. -- *Ship-over-research* - Execute existing specs before exploring new ones. Time-box research (30 min → decide: ship or kill). Monthly retrospective on research:shipped ratio. -- *Incremental execution* - Ship small, test, iterate. Keep config working at every step. Avoid big-bang failures, integration hell, and sunk cost traps. - -*How to measure success:* -- *Ruthless prioritization* → todo.org stays under 20 active items, cancelled:completed ratio shows I'm saying "no" -- *Weekly triage* → At least once a week by Sunday, no longer than 7 days between triage (tracked in calendar) -- *Measure metrics* → Every task has defined success criteria before starting, can show actual tracked metrics (startup logs, coverage reports) -- *Profile-before-optimize* → Every performance fix has profiler output proving the bottleneck, zero "I think this is slow" guesses -- *Ship-over-research* → Research:shipped ratio improves monthly (>1:1), complete code in todo.org ships within 1 week -- *Incremental execution* → Config never broken for >2 days, git commits are small and frequent, can roll back any change cleanly - - - -* Obstacles - -1. *Building and researching is more fun than fixing.* But if I don't change this, everything will be broken. - -2. *I get irritated with myself when making mistakes.* But mistakes are how people learn. If I let irritation drive me to give up, I'll never develop the skills these methods require. - -3. *It's hard for me to say "no".* But if I don't say "no" to something, I'll never achieve anything. The only way to make saying "no" easier is to do it repeatedly. - -4. *I can be a perfectionist who wants something just right before I move on.* But perfect is the enemy of shipped. I need to learn that good enough is better than perfect, and iteration beats perfection. - -5. *I don't always have the time for all day coding sessions.* Breaking the work into increments will help me deliver the long term value anyway. Be the tortoise, not the rabbit. - -6. *New disciplines are hard to sustain.* Weekly triage, measuring metrics, and profiling-before-optimizing are new habits. The first few weeks I'll be tempted to skip them when busy. But if I don't practice them consistently, they'll never become automatic, and I'll fall back into old patterns. - - - -* Metrics - -You can't improve what you don't measure. These metrics provide objective evidence that the Methods are working and the Vision is becoming reality. Track them weekly during triage to catch regressions early and celebrate progress. - -** Performance Metrics: -- *Startup time: < 3 seconds* (currently ~6.2s) - Measured with =time emacs --eval '(save-buffers-kill-emacs)'= -- *Org-agenda rebuild time: < 5 seconds* (currently 30+ seconds) - Measured with profiler during first daily open - -** Maintenance Discipline Metrics: -- *Active todo count: < 20 items* (currently ~50+) - Counted during weekly triage -- *Weekly triage consistency: At least once a week by Sunday, no longer than 7 days between triage* - Tracked in calendar -- *Research:shipped ratio: > 1:1* - Can't research next thing until current thing is implemented. Track monthly. -- *Config uptime: Never broken > 2 days* - Allows breathing room for emergencies/travel - -** Package Quality Metrics (chime, org-msg, wttrin): -- *Test coverage: > 70% and all code not covered justifiable* - Uncovered code is 100% risk. Better not be the most-used parts! - - - -* How to Use This V2MOM - -** For Decision Making -When considering a new task, package, or feature: -1. Does it serve the *Vision*? -2. Does it align with the *Values*? -3. Is it addressed in the *Methods*? -4. Does it overcome an *Obstacle*? -5. Does it improve a *Metric*? - -If the answer is "no" to all five → CANCEL the task. - -** For Triage -When reviewing todo.org: -- Tasks aligned with V2MOM → Keep and prioritize -- Tasks not aligned with V2MOM → CANCEL -- Tasks that are interesting but not important → CANCEL -- Tasks with complete code already → Execute immediately - -** For Weekly Review -Every Sunday (30 minutes): -1. Review metrics - Are they improving? -2. Review obstacles - Any new ones? Any resolved? -3. Review active todos - Still aligned with V2MOM? -4. Cancel stale/misaligned tasks -5. Plan next week's focus - -* Completion Status - -- [X] Vision - Complete (kept from original todo.org) -- [X] Values - Complete (Intuitive, Fast, Simple) -- [X] Methods - Complete (6 methods with aspirational bodies and concrete actions) -- [X] Obstacles - Complete (6 honest obstacles with real stakes) -- [X] Metrics - Complete (Performance, Discipline, and Quality metrics defined) - -*Last Updated:* 2025-10-31 (Session 2) -*Status:* ✅ V2MOM COMPLETE - Ready to use for decision making and weekly triage diff --git a/docs/NOTES.org b/docs/NOTES.org deleted file mode 100644 index 5d63d1f0..00000000 --- a/docs/NOTES.org +++ /dev/null @@ -1,887 +0,0 @@ -#+TITLE: 🚨 ACTIVE PROJECT - READ THIS FIRST 🚨 -#+AUTHOR: Claude Code Session Notes -#+DATE: 2025-10-30 - -* 🗣️ IMPORTANT TERMINOLOGY - -** "I want to do an X session with you" - -When Craig says "I want to do an X session with you", this means: -- **CREATE a session definition** for doing X (meta-work) -- **NOT** "let's DO X right now" (the actual work) - -This triggers the create-session workflow from docs/sessions/create-session.org - -*Examples:* -- "I want to do an emacs inbox zero session" → Create docs/sessions/inbox-zero.org -- "I want to do a refactor session" → Create docs/sessions/refactor.org -- "I want to do a code review session" → Create docs/sessions/code-review.org - -** "Let's wrap it up" / "That's a wrap" / "Let's call it a wrap" - -When Craig says any of these phrases, execute the wrap-up workflow: - -1. **Write session notes** to docs/NOTES.org - - Key decisions made - - Work completed - - Context needed for next session - - Any pending issues or blockers - -2. **Git commit and push all changes** - - Check git status and diff - - Create descriptive commit message - - Include co-author attribution (Claude) - - Push to ALL remotes (origin + github) - -3. **Valediction** - - Friendly goodbye summary - - What was accomplished - - What's ready for next session - - Any important reminders - -*Slash command:* `/wrap-it-up` triggers this same workflow - -* 🤔 PENDING DECISIONS - -** Flymake/Flycheck Modeline Integration - -Craig was interested in adding error/warning counts to the modeline but hasn't decided on implementation details yet. - -**Questions to decide:** - -1. **Flymake vs Flycheck?** - - Flymake is built-in and works with LSP (eglot) - - Flycheck requires installation but has more checker backends - -2. **Format preference?** - - Just counts: `E:3 W:5` - - With symbols: `⚠ 3 ✖ 5` - - Clickable to show diagnostics buffer? - -3. **Placement?** - - Left side near major-mode, or right side with VC branch? - -4. **Active window only?** - - Like your VC branch - only show in selected window - -**Implementation is ready** - just need your preferences and I can add it exactly how you want it. - -* 🔔 DESKTOP NOTIFICATIONS WORKFLOW - -**IMPORTANT: How Claude notifies you when blocked** - -When Claude needs your input on blocking questions, Claude will send a desktop notification via `notify-send`: - -#+BEGIN_SRC bash -notify-send "Claude Code" "Question: [Your input needed]" --urgency=normal -#+END_SRC - -**When notifications ARE sent:** -- ✅ When explicitly needing your decision/input (blocking questions) -- ✅ When multiple valid approaches exist and choice affects implementation -- ✅ When encountering errors that require user guidance -- ✅ **ONLY** when Claude cannot proceed without user input - -**When notifications are NOT sent:** -- ❌ After completing tasks (informational updates) -- ❌ During normal progress updates -- ❌ When milestones are reached -- ❌ For status messages or completion notices -- ❌ ANY informational alerts - -**Setup:** -- Requires `dunst` or similar notification daemon -- Works with `notify-send` command -- Always uses `--urgency=normal` (not critical) - -**Purpose:** -This allows you to context-switch to other work while Claude runs long tasks, and get notified ONLY when your input is truly needed to continue. You check back when convenient for status updates. - -* 🧩 EMACS LISP DEVELOPMENT BEST PRACTICES - -**Critical Lessons: Preventing Parenthesis Errors** - -Both humans and AI struggle with balanced parentheses in deeply nested Emacs Lisp code. Here are proven strategies to prevent this: - -** For AI Code Generation - -*** 1. Write Small, Focused Functions -- Keep functions under 15 lines when possible -- Each function should do ONE thing -- Easier to verify parentheses at a glance -- Easier to test in isolation - -#+BEGIN_EXAMPLE -Bad (deeply nested, hard to verify): -(defun process-data (data) - (when (valid-p data) - (let ((result (transform data))) - (when result - (let ((final (format result))) - (when final - (save final))))))) - -Good (broken into helpers): -(defun process-data (data) - (when (valid-p data) - (save-result (format-result (transform-data data))))) - -(defun transform-data (data) ...) -(defun format-result (result) ...) -(defun save-result (final) ...) -#+END_EXAMPLE - -*** 2. Test Immediately After Each Write -- Write function → check-parens → test load → repeat -- Don't batch multiple functions before testing -- Catch errors early when context is fresh - -#+BEGIN_SRC bash -# After writing each function: -emacs --batch file.el --eval '(check-parens)' && echo "✓" -#+END_SRC - -*** 3. Prefer Write Over Multiple Edits -- For complex new code: use Write tool (complete file) -- Only use Edit for small, simple changes -- Incremental edits can introduce subtle paren mismatches -- Complete file Write is easier to verify - -*** 4. Validate Before Committing -- Use pre-commit hooks to validate all .el files -- Prevents committing broken code -- Example in chime.el repository: .git/hooks/pre-commit - -*** 5. Test Emacs Launch After Non-Trivial Changes -**CRITICAL: Always test that Emacs launches without errors after making changes** - -After completing any non-trivial code changes (new modules, integrations, etc.): - -#+BEGIN_SRC bash -# 1. Run unit tests (if applicable) -make test - -# 2. Test that Emacs launches without backtrace -emacs --eval "(kill-emacs)" -#+END_SRC - -This catches: -- Syntax errors in :command specifications -- Missing dependencies at load time -- Invalid use-package configurations -- Broken requires/provides - -**When to do this:** -- ✅ After adding new checker definitions (flycheck, flymake) -- ✅ After creating new modules -- ✅ After modifying init.el or early-init.el -- ✅ Before committing changes -- ✅ After running all unit tests - -**Example lesson:** The LanguageTool integration used =(eval (expand-file-name ...))= in -a =:command= specification, which caused a backtrace on startup. Testing Emacs launch -would have caught this immediately instead of discovering it on next restart. - -** For Human Developers - -*** 1. Use Structural Editing Modes -These PREVENT unbalanced parens: -- **paredit** - Classic, strict structural editing -- **smartparens** - More flexible, works with multiple languages -- **lispy** - Modal editing for lisps -- **electric-pair-mode** - Built-in, auto-closes parens - -*** 2. Enable Real-time Validation -- **flycheck** + **flycheck-package** - Shows errors as you type -- **flymake** - Built-in alternative -- **rainbow-delimiters-mode** - Colors matching parens - -*** 3. Quick Validation Commands -#+BEGIN_SRC elisp -M-x check-parens ; Check current buffer -M-x byte-compile-file ; More comprehensive checking -#+END_SRC - -** Lessons from chime-org-contacts.el Development - -*** Original Problem -Complex nested function with 6 levels of nesting: -- Hard to count parentheses manually -- AI kept adding/removing wrong number of closing parens -- Functions weren't defined after file load -- Wasted significant debugging time - -*** Solution That Worked -Refactored into 3 helper functions: -1. =chime-org-contacts--parse-birthday= (10 lines) -2. =chime-org-contacts--format-timestamp= (4 lines) -3. =chime-org-contacts--insert-timestamp-after-drawer= (12 lines) - -Main function became simple composition (10 lines). - -**Result:** -- All functions defined correctly -- Easy to verify parens by eye -- Each function testable independently -- 24 tests written covering all cases - -*** Key Insight -Breaking complex code into small helpers: -- ✅ Easier to verify correctness -- ✅ Easier to test -- ✅ Easier to maintain -- ✅ Self-documenting through function names -- ✅ AI and humans both succeed - -Deeply nested code: -- ❌ Hard to verify -- ❌ Hard to test -- ❌ AI frequently makes paren errors -- ❌ Humans make mistakes too - -** Tools and Workflow Summary - -| Stage | Tool | Purpose | -|-------+------+---------| -| Writing | paredit/smartparens | Prevent errors | -| Editing | rainbow-delimiters | Visual verification | -| Testing | check-parens | Quick syntax check | -| Testing | emacs --eval "(kill-emacs)" | Verify Emacs launches | -| CI/CD | pre-commit hooks | Prevent bad commits | -| Review | byte-compile-file | Comprehensive check | - -** Makefile - Comprehensive Testing & Validation Tool - -A comprehensive Makefile is available in the repository root with targets for testing, validation, and utilities. - -#+BEGIN_SRC bash -make # Show all available targets -make test # Run all tests (unit + integration) -make test-file FILE=... # Run specific test file -make test-name TEST=... # Run tests matching pattern -make validate-parens # Check for unbalanced parentheses -make validate-modules # Load all modules to verify compilation -make compile # Byte-compile all modules -make lint # Run checkdoc, package-lint, elisp-lint -make profile # Profile Emacs startup -make clean # Remove test artifacts and compiled files -make reset # Reset to first launch (destructive!) -#+END_SRC - -Created: 2025-11-03 -Adapted from chime.el Makefile with config-specific enhancements - -* 📋 AVAILABLE SESSION TYPES - -** create-session -File: [[file:sessions/create-session.org][docs/sessions/create-session.org]] - -Meta-workflow for creating new session types. Use this when identifying repetitive workflows that would benefit from documentation. - -Workflow: -1. Q&A discovery (4 core questions) -2. Assess completeness -3. Name the session -4. Document it -5. Update NOTES.org -6. Validate by execution - -Created: 2025-11-01 (pre-existing) - -** emacs-inbox-zero -File: [[file:sessions/emacs-inbox-zero.org][docs/sessions/emacs-inbox-zero.org]] - -Weekly workflow for processing the "Emacs Config Inbox" heading in =todo.org= to zero by filtering through V2MOM framework. - -Workflow: -1. Sort by priority (A → B → C → none → D) -2. Claude rereads V2MOM -3. Process each item through 3 questions: - - Does this need to be done? → DELETE if no - - Related to V2MOM? → Move to someday-maybe if no - - Which method? → Move to appropriate method -4. Done when inbox heading is empty - -Target: 10 minutes active work time -Cadence: Every Sunday, no longer than 7 days between sessions -Maintains metrics: Active todos < 20, weekly triage consistency - -Created: 2025-11-01 - -* CURRENT PROJECT STATUS - -** 🎯 What We're Doing -Working through a systematic approach to clean up and prioritize Craig's Emacs config work: - -1. ✅ *COMPLETE V2MOM* (Vision, Values, Methods, Obstacles, Metrics) - IN PROGRESS -2. ⏳ *TRIAGE todo.org* - Use V2MOM to ruthlessly cancel ~60% of tasks -3. ⏳ *EXECUTE TIER 1* - Ship quick wins (network check removal, Corfu, bug fixes) -4. ⏳ *BUILD OBSERVABILITY* - Create profiling infrastructure (TIER 2) -5. ⏳ *SYSTEMATIC EXECUTION* - Work through prioritized tasks one by one - -** 📍 Where We Are Right Now -*Session Started:* 2025-10-30 -*Current Step:* ✅ V2MOM COMPLETE - Ready for execution -*Time Committed:* ~2 sessions, V2MOM finished 2025-10-31 -*Status:* V2MOM complete, ready to begin Method 1 execution - -** 📄 Key Documents - -*** Primary Working Documents -- *V2MOM:* [[file:EMACS-CONFIG-V2MOM.org][EMACS-CONFIG-V2MOM.org]] - Strategic framework for Emacs config (✅ COMPLETE) - - Vision, Values, Methods, Obstacles, Metrics - - Used for decision-making and weekly triage - - Read this first to understand strategic direction -- *Issues Analysis:* [[file:../issues.org][../issues.org]] - Claude's detailed analysis with TIER system and implementations -- *Current Inbox:* [[file:../inbox.org][../inbox.org]] - V2MOM-aligned tasks (~23 items after ruthless triage) - -*** Reference Documents -- *Config Root:* [[file:../init.el][../init.el]] -- *Modules:* [[file:../modules/][../modules/]] -- *Tests:* [[file:../tests/][../tests/]] - -** 🔑 Key Insights About Craig's Work Patterns - -*** Strengths -- Thoughtful and strategic thinker -- Good research skills (thorough specs, complete code examples) -- Does ship things (dashboard, dirvish, network check fixes) -- Recognizes need for V2MOM framework -- Uses config daily for real work - -*** Patterns to Address -1. *Research > Execution* - Has complete code for Corfu, difftastic, transcription workflow... still TODO -2. *Priority Inflation* - Too many [#A]/[#B] items, unclear what's actually urgent -3. *Incomplete Strategy* - V2MOM structure exists but sections are empty -4. *Hard to Say No* - [#C]/[#D] items should be CANCELLED but remain in list -5. *Side Projects Compete* - Dupre theme work competes with core config maintenance - -*** What Craig Told Us About Himself -> "I am building tools both because they solve problems, but also because I enjoy building." - -This is healthy! But need balance: -- Fix rough edges FIRST (daily pain points) -- Build fun stuff SECOND (after maintenance) -- Cancel distractions ALWAYS (Signal client, minimap, etc.) - -** 🎯 Agreed Goals for This Project - -*** Immediate (Next 2-3 Sessions) -1. ✅ Complete V2MOM (IN PROGRESS) -2. ⏳ Triage todo.org using V2MOM as filter -3. ⏳ Execute quick wins: network check, Corfu migration, bug fixes -4. ⏳ Build debug-profiling.el infrastructure - -*** Short Term (Next Month) -5. Profile and optimize org-agenda performance -6. Ship reveal.js presentation workflow -7. Establish weekly triage ritual - -*** Long Term (Ongoing) -8. Ship more than research -9. Maintain < 20 active todos -10. Measure metrics from V2MOM - -** 📋 TIER System from issues.org - -*** TIER 1: Do These First (High Impact, Low Effort) - 1 weekend -- Remove network check (15 min) -- Fix missing functions (30 min) -- Corfu migration (2 hours) -- Mood-line switch (30 min) -- Bug fixes (1 hour) - -*** TIER 2: Build Observability (HIGHEST VALUE) - 1 week -- Create debug-profiling.el module (3-4 hours) -- Profile org-agenda-rebuild (1 hour) -- Add instrumentation and caching (2 hours) -- Test org-agenda filtering functions (2-3 hours) - -*** TIER 3: Quick Wins (After Profiling) - 1-2 hours each -- Reveal.js presentation workflow (2 hours) -- Difftastic integration (30 min) -- Local package development workflow (1 hour) - -*** TIER 4: Maybe/Someday (Probably Never) -- Code-maat reimplementation (HOLD) -- LaTeX config (HOLD until concrete need) -- Elfeed dashboard (HOLD - unclear if actually used) -- DWIM shell integration (HOLD - current solution works) -- Jumper package (HOLD - already maintaining chime + org-msg) - -** 🚫 Items That Should Be CANCELLED - -From todo.org, these don't serve the vision: -- [#D] Signal Client - Not in vision -- [#D] Awesome-tray / mode-icons - Already have modeline -- [#C] Minimap - Interesting, not important -- [#C] Install Magit TODOs - Already works fine -- [#C] Git Timemachine litters buffers - Minor annoyance -- Many Dupre theme TODOs - Side project competing with maintenance - -## 💡 Key Recommendations for Craig - -### Week 1: Strategy + Quick Wins -1. Complete V2MOM (2-3 hours) -2. Triage todo.org using V2MOM (1-2 hours) -3. Execute items you already have code for (2-3 hours) - -### Week 2: Observability Infrastructure -4. Build debug-profiling.el (3-4 hours) -5. Profile org-agenda (1 hour) - -### Week 3: Fix Performance + Ship Presentation -6. Fix org-agenda based on profiling (2-3 hours) -7. Ship reveal.js workflow (2 hours) - -### Ongoing: Maintenance Discipline -- Weekly triage ritual (30 min every Sunday) -- Measure metrics (startup time, agenda time, todo count) -- Ship > Research - -** 🔄 Next Session Pickup Points - -When starting next session, Claude should: - -1. **Read this document first** to understand context -2. **Check V2MOM status** - If incomplete, continue there -3. **Reference issues.org** for detailed technical recommendations -4. **Reference todo.org** for items to triage -5. **Ask Craig:** "Where did we leave off? V2MOM? Triage? Execution?" - -** 📞 Questions to Ask Craig Next Session - -*IMMEDIATE (when resuming):* -- "Ready to continue V2MOM? We left off at Methods section." -- "How much time do you have?" - -*FOR METHODS SECTION:* -Show Craig the draft list and ask: -- "Which methods do you already do consistently?" -- "Which do you want to do but don't yet?" -- "Am I missing any important methods?" - -*AFTER V2MOM COMPLETE:* -- "Ready to triage todo.org using the V2MOM?" -- "Should we execute quick wins or continue systematic triage?" - -** 🎯 Success Metrics for This Project - -We'll know this is working when: -- ✅ V2MOM is complete and provides clear strategic direction -- ✅ todo.org shrinks from ~50 to < 20 active items -- ✅ Craig ships 3-5 items per week (small but consistent) -- ✅ Craig has profiling infrastructure to measure performance -- ✅ Org agenda rebuild time is measured and improving -- ✅ Weekly triage becomes habit - -** 💬 Craig's Words to Remember - -> "I think you should adjust issues.org with all your recommendations. They are exciting, eye-opening, and just feel right. Add even your guidance on latex. spot on. thanks for your honesty. I did ask for it and am genuinely grateful for your responses. I'll take action on them." - -> "What I need help with is integrating this in with my existing todo.org file... Some of the tasks I've listed should probably just be deleted or better yet, marked CANCELLED." - -> "I have about an hour to devote. You could lead me through it, I could do some questions/answer rounds with you to clarify my thinking." - -Craig is ready to execute. He asked for honesty and took it well. He recognizes the patterns and wants systematic help. - -** 🛠️ Technical Context - -*** Current Pain Points -1. Org agenda is slow (performance bottleneck) -2. Network check adds 1+ seconds to startup (technical debt) -3. Missing functions cause errors (cj/log-silently, cj/goto-git-gutter-diff-hunks) -4. Mail attachments workflow is awkward -5. No profiling infrastructure to measure performance - -*** Items Craig Already Has Code For -These can be executed immediately - just paste and test: -- Transcription workflow (complete bash + elisp in todo.org:2-99) -- Difftastic integration (complete config in todo.org:1211-1223) -- Corfu migration (complete config in todo.org:1611-1639) - -*** Architecture -- Modular structure: modules/*.el -- Good test coverage for utilities -- Modern packages: Vertico/Consult/Embark stack -- Local package development: chime.el, org-msg - -** 📚 Related Reading - -If Craig or Claude need more context: -- [[file:../issues.org::*Second Opinion: Ruthless Prioritization & Reality Checks][Second Opinion section in issues.org]] - Full analysis and recommendations -- [[file:../issues.org::*TIER 1: Do These First][TIER 1-4 breakdown]] - Prioritized task system -- [[file:../quality-engineer.org][quality-engineer.org]] - Testing philosophy (if exists) - -** 🚀 Current Session Notes - -*** 2025-11-04 Session 3 - LanguageTool Grammar Checker Test Suite -*Time:* ~1 hour -*Status:* ✅ COMPLETE - Comprehensive test suite with 15 passing tests - -*What We Completed:* - -1. ✅ **Created Test Fixtures** (tests/fixtures/) - - grammar-errors-basic.txt - Common grammar errors (subject-verb, could of, etc.) - - grammar-errors-punctuation.txt - Punctuation and spacing issues - - grammar-correct.txt - Clean baseline for comparison - - Deterministic fixtures for reliable testing - -2. ✅ **Unit Tests** (test-flycheck-languagetool-setup.el) - - 6 tests verifying LanguageTool installation and setup - - Checks binary availability, wrapper script existence - - Validates wrapper script structure (shebang, imports) - - Tests error handling for missing arguments - - All tests pass ✓ (< 1 second) - -3. ✅ **Integration Tests** (test-integration-grammar-checking.el) - - 9 tests covering end-to-end grammar checking workflow - - Tests wrapper script with real LanguageTool execution - - Validates flycheck-compatible output format - - Normal cases: error detection, output formatting - - Boundary cases: empty files, single word, multiple paragraphs - - Error cases: nonexistent files, missing arguments - - All tests pass ✓ (~35 seconds with real LanguageTool execution) - -4. ✅ **Applied Testing Philosophy** from quality-engineer.org - - Focus on OUR code (wrapper script), not flycheck internals - - Trust external frameworks work correctly - - Test real integration with actual LanguageTool execution - - Comprehensive docstrings listing integrated components - - Clear test categories: Normal, Boundary, Error cases - - No over-mocking of domain logic - -*Key Decisions:* -- Don't test flycheck framework internals (trust it works) -- Test our integration code with real external tools -- Use real fixtures instead of generated test data -- Integration tests run slower but provide high confidence - -*Files Created:* -- tests/test-flycheck-languagetool-setup.el -- tests/test-integration-grammar-checking.el -- tests/fixtures/grammar-errors-basic.txt -- tests/fixtures/grammar-errors-punctuation.txt -- tests/fixtures/grammar-correct.txt - -*Makefile Integration:* -Tests automatically discovered via wildcards: -- make test-unit (includes our unit test) -- make test-integration (includes our integration test) -- make test-file FILE=test-flycheck-languagetool-setup.el -- make test-file FILE=test-integration-grammar-checking.el - -*Test Results:* -- Unit tests: 6/6 passing -- Integration tests: 9/9 passing -- Total: 15/15 tests ✓ - -*Status:* -- All code committed and pushed to origin + github ✅ -- Tests run regularly via Makefile ✅ -- Grammar checker fully tested and production-ready ✅ - -*** 2025-11-04 Session 2 - Emergency Bug Fixes & Modeline Polish -*Time:* ~30 minutes -*Status:* ✅ COMPLETE - Fixed async buffer error and improved modeline spacing - -*What We Completed:* - -1. ✅ **Fixed Critical Async Buffer Error on Startup** - - Problem: "Selecting deleted buffer" error in async.el during init - - Root cause: wttrin-mode-line-mode starting async HTTP request during init - - Solution: Delayed activation using after-init-hook - - Weather widget now loads AFTER Emacs finishes initializing - - Prevents async buffer from being killed before request completes - -2. ✅ **Fixed Parenthesis Error in weather-config.el** - - Removed extra closing parenthesis that prevented file from loading - - Verified with check-parens - -3. ✅ **Improved Modeline Right-Side Spacing** - - Problem: Right-most icon (weather) appeared flush with window edge - - Discovered: Regular spaces were being trimmed by right-align mechanism - - Solution: Added 2 non-breaking spaces (U+00A0) after misc-info segment - - Provides visual breathing room without being excessive - -*Key Technical Insights:* -- Async operations during init need careful timing (use after-init-hook) -- mode-line-right-align-edge 'right-margin trims trailing regular spaces -- Non-breaking spaces (U+00A0) survive trimming in modeline format - -*Files Modified:* -- modules/weather-config.el - Added after-init-hook delay for mode-line widget -- modules/modeline-config.el - Added 2 non-breaking spaces at end - -*Status:* -- Emacs launches without errors ✅ -- Weather widget appears in modeline after startup ✅ -- Modeline spacing looks clean and professional ✅ - -*** 2025-11-04 Session 1 - Complete Transcription Workflow Implementation -*Time:* ~3 hours -*Status:* ✅ COMPLETE - Full async transcription system with 60 passing tests - -*What We Completed:* - -1. ✅ **Installed Whisper Locally** - - Created install-whisper.sh with AUR support (python-openai-whisper) - - Created uninstall-whisper.sh for clean removal - - Installed via AUR successfully - - Added --yes flag for non-interactive automation - -2. ✅ **Created CLI Transcription Scripts** - - scripts/local-whisper - Uses installed Whisper (works offline) - - scripts/oai-transcribe - Uses OpenAI API (faster, requires API key) - - Both scripts output to stdout, log to stderr - - Proper error handling and validation - -3. ✅ **Implemented Full Transcription Module** (modules/transcription-config.el) - - Async transcription workflow (non-blocking) - - Desktop notifications (started, complete, error) - - Output: audio.txt (transcript) + audio.log (process logs) - - Log cleanup: auto-delete on success (configurable) - - Modeline integration: Shows ⏺count of active transcriptions - - Clickable modeline to view *Transcriptions* buffer - - Process tracking and management - -4. ✅ **Comprehensive Test Suite - 60 Tests, All Passing** - - test-transcription-audio-file.el (16 tests) - Extension detection - - test-transcription-paths.el (11 tests) - File path logic - - test-transcription-log-cleanup.el (5 tests) - Log retention - - test-transcription-duration.el (9 tests) - Time formatting - - test-transcription-counter.el (11 tests) - Active count & modeline - - test-integration-transcription.el (8 tests) - End-to-end workflows - - Tests found and fixed 1 bug (nil handling in audio detection) - - Normal, boundary, and error cases covered - -5. ✅ **Reorganized Keybindings** - - Moved gcal from C-; g/t/r/G to C-; g s/t/r/c submenu - - Created C-; t transcription submenu: - - C-; t a → transcribe audio - - C-; t v → view transcriptions - - C-; t k → kill transcription - - Dired/Dirvish: T → transcribe file at point - - which-key integration for discoverability - -6. ✅ **Added Audio Extensions to user-constants.el** - - Centralized cj/audio-file-extensions list - - Shared across transcription and future audio features - - Used defvar (not defcustom) per Craig's preference - -*Key Decisions:* -- **Simplified UX:** No org-capture integration (initially planned), just file in/out -- **Minimalist approach:** Audio files → .txt transcripts (no complex templates) -- **Testable architecture:** Pure functions separated from I/O -- **defvar over defcustom:** All configuration variables use defvar - -*Files Created:* -- scripts/install-whisper.sh -- scripts/uninstall-whisper.sh -- scripts/local-whisper -- scripts/oai-transcribe -- modules/transcription-config.el -- tests/test-transcription-*.el (5 test files) - -*Files Modified:* -- modules/user-constants.el (added cj/audio-file-extensions) -- modules/org-gcal-config.el (reorganized keybindings to C-; g submenu) - -*Pending for Next Session:* -- Manual test with real audio file -- True integration tests (run actual transcription process) -- Create test fixtures (small audio samples) -- Consolidate issues.org with inbox.org (deferred) - -*Next Steps:* -1. Add `(require 'transcription-config)` to init.el -2. Test with: M-x cj/transcribe-audio or T in dired on audio file -3. Verify .txt and .log files created -4. Check modeline shows active count -5. Review output quality - -*** 2025-11-03 Session - Modeline Polish & Wrap-Up Workflow -*Time:* ~30 minutes -*Status:* ✅ COMPLETE - Code quality improvements and workflow automation - -*What We Completed:* -1. ✅ Fixed all checkdoc linting errors in modeline-config.el (8 warnings → 0) - - Removed embedded keycodes from docstrings (mouse-1, mouse-3) - - Added double spaces after periods per Emacs conventions - - Quoted Lisp symbols with backticks (`vc-diff`, `describe-mode`) - - Fixed single quotes in examples to use proper backtick quoting - - File now passes checkdoc and byte-compilation cleanly - -2. ✅ Created `/wrap-it-up` workflow automation - - Slash command: `.claude/commands/wrap-it-up.md` - - Phrase recognition: "let's wrap it up", "that's a wrap", "let's call it a wrap" - - Workflow: Write notes → Git commit & push → Valediction summary - - Documented in both slash command and NOTES.org terminology section - - Ensures clean session endings with no lost work - -3. ✅ Added reminder for Flymake/Flycheck modeline integration - - Created "PENDING DECISIONS" section in NOTES.org - - Questions documented: Flymake vs Flycheck, format, placement, active window - - Implementation ready - just needs user preferences - - Craig will be reminded proactively at next session start - -*Key Decisions:* -- Dual documentation approach (slash command + phrase recognition) for reliability -- Craig expressed gratitude: "my life is genuinely improving while working with you" -- Modeline work quality bar: must pass all linters before shipping - -*Commits:* -- aacc1e7: style: Fix checkdoc linting errors in modeline-config.el -- bb40315: feat: Add /wrap-it-up slash command and phrase recognition -- 1a9b8db: docs: Add reminder for Flymake/Flycheck modeline integration decision - -*Next Session:* -- Remind Craig about Flymake/Flycheck decision (proactively!) -- Ready to implement error/warning counts in modeline once preferences decided - -*** 2025-10-31 Session 2 - V2MOM Complete! -*Time:* ~1.5 hours -*Status:* ✅ COMPLETE - V2MOM finalized and ready for use - -*What We Completed:* -1. ✅ Finalized all 6 Methods with aspirational bodies and concrete actions: - - Method 1: Make Using Emacs Frictionless (performance & functionality fixes) - - Method 2: Stop Problems Before They Appear (proactive package maintenance) - - Method 3: Make *Fixing* Emacs Frictionless (observability/tooling) - - Method 4: Contribute to the Emacs Ecosystem (package maintenance tooling) - - Method 5: Be Kind To Your Future Self (new features) - - Method 6: Develop Disciplined Engineering Practices (meta-method with measurable outcomes) - -2. ✅ Completed Obstacles section (6 honest, personal obstacles with real stakes) - - Building vs fixing tension - - Getting irritated at mistakes - - Hard to say "no" - - Perfectionism - - Limited time sessions - - New habits are hard to sustain - -3. ✅ Completed Metrics section (Performance, Discipline, Quality metrics) - - Startup time: < 3s (currently 6.2s) - - Org-agenda: < 5s (currently 30+s) - - Active todos: < 20 (currently ~50+) - - Weekly triage consistency - - Research:shipped ratio > 1:1 - - Config uptime: never broken > 2 days - - Test coverage: > 70% with justification for uncovered code - -4. ✅ Implemented cj/diff-buffer-with-file - - Added to modules/custom-buffer-file.el - - Bound to C-; b D - - Unified diff format with proper error handling - - TODO comment for future difftastic integration - -5. ✅ Added missing items to Methods based on Craig's research: - - Fixed org-noter (Method 1) - - Added Buttercup (Method 3) - - Added package maintenance tools (Method 4: package-lint, melpazoid, elisp-check, undercover) - - Added wttrin to maintained packages list - -*Key Insights:* -- Craig wants to DELETE research files: "There will always be cool ideas out there to implement and they will always be a web search away" -- Ruthless prioritization is already happening -- Method ordering: fix → stabilize → build infrastructure → contribute → enhance → sustain -- Adjusted startup target from 2s to 3s (more achievable, less perfectionism trap) - -*Key Files Modified This Session:* -- [[file:emacs-config-v2mom.org][emacs-config-v2mom.org]] - Now 100% complete with all sections filled -- [[file:../modules/custom-buffer-file.el][modules/custom-buffer-file.el]] - Added cj/diff-buffer-with-file function -- [[file:SESSION-HANDOFF-ACTIVE-PROJECT.org][SESSION-HANDOFF-ACTIVE-PROJECT.org]] - This file - -*Next Session Starts With:* -1. Continue Method 1 execution - 2 quick wins ready! -2. Fix cj/goto-git-gutter-diff-hunks (15 min) -3. Fix chime throw/catch bug (your package) -4. Fix go-ts-mode-map keybinding error - -*** 2025-10-31 Session 3 - RUTHLESS EXECUTION! 🚀 -*Time:* ~2 hours -*Status:* Method 1 in progress - shipped 2 wins, discovered 3 bugs - -*What We Completed:* -1. ✅ **RUTHLESS PRIORITIZATION EXECUTED!** - - Moved todo.org → docs/someday-maybe.org (~50 items archived) - - Created fresh inbox.org with ONLY V2MOM-aligned tasks (23 items) - - Already under < 20 active items goal! - -2. ✅ **SHIPPED: Network check removal** (Method 1) - - Deleted `cj/internet-up-p` blocking ping function - - Removed network cache variables - - Simplified to use package priorities instead - - .localrepo (priority 200) ensures offline reproducibility - - **RESULT: 6.19s → 4.16s startup time (2.03 seconds faster!)** - -3. ✅ **SHIPPED: cj/diff-buffer-with-file** (Method 1) - - Implemented in modules/custom-buffer-file.el - - Bound to C-; b D - - Weekly need satisfied - -4. ✅ **Updated reset-to-first-launch.sh** - - Added missing transient files/directories - - Keeps docs/, inbox.org, .localrepo safe - - Ready for testing offline installs - -5. ✅ **Tested .localrepo offline install capability** - - Works perfectly for package.el packages - - Discovered 3 bugs during test (logged in inbox.org) - -*Bugs Discovered (all logged in inbox.org):* -1. **Chime throw/catch error** - High priority, your package - - Error: "(no-catch --cl-block-chime-check-- nil)" - - Fix: Change defun to cl-defun or add catch block - - Currently disabled to unblock startup - -2. **go-ts-mode-map keybinding error** - - Error: "void-variable go-ts-mode-map" - - Fix: Wrap in with-eval-after-load - -3. **Treesitter grammars not in .localrepo** (limitation documented) - - Expected behavior - treesit-auto downloads separately - -*Metrics Update:* -- Startup time: 6.19s → 4.16s (**2.03s improvement!**) -- Only 1.16s away from < 3s target! -- Active todos: ~23 items (hit < 20 goal when excluding tracking tasks!) -- Shipped items: 2 (network check, diff-buffer-with-file) - -*Key Files Modified:* -- [[file:../early-init.el][early-init.el]] - Network check removed, cj/use-online-repos simplified -- [[file:../modules/custom-buffer-file.el][custom-buffer-file.el]] - Added cj/diff-buffer-with-file -- [[file:../inbox.org][inbox.org]] - Fresh V2MOM-aligned todo list created -- [[file:../scripts/reset-to-first-launch.sh][reset-to-first-launch.sh]] - Updated with missing transient files -- [[file:someday-maybe.org][someday-maybe.org]] - Old todo.org archived here - -*Next Session (2025-11-01):* -**Two quick wins ready (15 min each):** -1. Fix cj/goto-git-gutter-diff-hunks (missing function) -2. Fix chime throw/catch bug (re-enable chime) - -**Then continue Method 1:** -- Optimize org-agenda (THE BOTTLENECK - 30s → <5s target) -- Fix org-noter (daily pain) -- Fix video/audio recording -- Fix mail attachments -- Fix grammar checker - -*Craig's Words:* -> "There will always be cool ideas out there to implement and they will always be a web search away." -Ruthless prioritization in action! Deleted research files, focused execution. - -*** 2025-10-30 Session 1 - V2MOM In Progress -*Time:* ~1 hour -*Status:* PAUSED - V2MOM 60% complete - -*What We Completed:* -1. ✅ Created docs/ directory structure -2. ✅ Created SESSION-HANDOFF-ACTIVE-PROJECT.org (this file) -3. ✅ Created emacs-config-v2mom.org -4. ✅ Created values-comparison.org (analysis doc) -5. ✅ Completed Vision (already existed, kept as-is) -6. ✅ Completed Values section (Intuitive, Fast, Simple) - - Intuitive: Muscle memory, mnemonics, which-key timing, "newspaper" code - - Fast: Startup < 2s, org-agenda is THE bottleneck, everything else acceptable - - Simple: Production software practices, simplicity produces reliability diff --git a/docs/SOMEDAY-MAYBE.org b/docs/SOMEDAY-MAYBE.org deleted file mode 100644 index e392ae99..00000000 --- a/docs/SOMEDAY-MAYBE.org +++ /dev/null @@ -1,1633 +0,0 @@ -* Emacs Config Open Work - -** TODO [#D] Irritant: Press Key to Launch Dashboard Icon App - -Not important enough - already have keybindings and M-x search as working alternatives. -Moved from inbox 2025-11-01. - -** TODO [#D] Irritant: Move Persistence Files Into a Single Directory - -Organizational tidiness, not actual friction. Perfectionism (V2MOM Obstacle #4). -Moved from inbox 2025-11-01. - -** TODO [#A] Add Transcription Org-capture Workflow - :PROPERTIES: - :CATEGORY: emacs - :END: - -*** Goal - Integrate audio-to-text transcription into Org-capture so meeting recordings - can be turned into timestamped notes automatically. - -*** Steps - -**** 1. Create CLI for transcription - - Option A — OpenAI API (Whisper): - #+begin_src bash - # ~/bin/oai-transcribe - #!/usr/bin/env bash - set -euo pipefail - AUDIO="$1"; LANG="${2:-en}" - : "${OPENAI_API_KEY:?Set OPENAI_API_KEY in your environment}" - curl -s -X POST "https://api.openai.com/v1/audio/transcriptions" \ - -H "Authorization: Bearer $OPENAI_API_KEY" \ - -F "model=whisper-1" \ - -F "response_format=text" \ - -F "language=${LANG}" \ - -F "file=@${AUDIO}" - #+end_src - - - Option B — Local Whisper: - #+begin_src bash - pip install openai-whisper # once - # ~/bin/local-whisper - #!/usr/bin/env bash - set -euo pipefail - AUDIO="$1"; MODEL="${2:-medium}"; LANG="${3:-en}" - whisper "$AUDIO" --model "$MODEL" --language "$LANG" \ - --task transcribe --output_format txt --verbose False >/dev/null - cat "${AUDIO%.*}.txt" - #+end_src - -**** 2. Emacs Lisp configuration - Add to init.el: - - #+begin_src emacs-lisp - (defcustom cj/transcribe-program - (or (executable-find "oai-transcribe") - (executable-find "local-whisper")) - "Program used to transcribe audio to plain text.") - - (defcustom cj/meetings-org-dir "~/org/meetings/" "") - - (defun cj/transcribe-audio (file) - (with-temp-buffer - (call-process shell-file-name nil t nil shell-command-switch - (format "%s %s" - (shell-quote-argument cj/transcribe-program) - (shell-quote-argument (expand-file-name file)))) - (buffer-string))) - - (defun cj/org-capture-meeting-template () - (let* ((audio (read-file-name "Audio: ")) - (participants (read-string "Participants: " "Mark; Craig")) - (location (read-string "Location: " "French Truck Coffee")) - (topic (read-string "Topic: " "Contract discussion")) - (transcript (cj/transcribe-audio audio))) - (format "* Meeting: %s\n:PROPERTIES:\n:DATE: %s\n:LOCATION: %s\n:PARTICIPANTS: %s\n:AUDIO: %s\n:END:\n\n** Summary\n- \n\n** Transcript\n#+begin_quote\n%s\n#+end_quote\n" - topic (format-time-string "%Y-%m-%d %H:%M") - location participants (abbreviate-file-name audio) transcript))) - - (with-eval-after-load 'org - (add-to-list 'org-capture-templates - `("m" "Meeting (transcribe audio)" plain - (file ,(expand-file-name (format-time-string "%Y-meetings.org") - cj/meetings-org-dir)) - "%(cj/org-capture-meeting-template)" - :empty-lines 1 :unnarrowed t))) - #+end_src - -**** 3. Usage - 1. Record meeting as .m4a or .wav - 2. Run `M-x org-capture m` → select the audio file - 3. Org inserts transcript and metadata under - `~/org/meetings/YYYY-meetings.org` - -**** 4. Optional quick recap template - #+begin_src emacs-lisp - (with-eval-after-load 'org - (add-to-list 'org-capture-templates - '("r" "Post-meeting recap" entry - (file+headline - (lambda () (expand-file-name (format-time-string "%Y-meetings.org") - cj/meetings-org-dir)) - "Recaps") - "* %U %^{Title|Meeting w/ Mark (Carrollton)}\n:LOCATION: %^{Location|French Truck (Dryades)}\n** Facts\n- %?\n** Next Steps\n- "))) - #+end_src - -**** 5. Notes - - Keep filenames descriptive: `2025-11-03_mark-frenchtruck.m4a` - - For privacy: use local-whisper variant to keep audio offline. -** TODO [#A] Finish Testing New Org-webclipper -** TODO [#A] Org Reveal.js Presentations From Local Org :enhancement: -*** References -https://revealjs.com/ -https://github.com/hakimel/reveal.js -*** Steps from Zamansky's Videos -https://www.youtube.com/watch?v=psDpCpcIVYs&t=10s -**** Install Org Reveal - -** TODO [#B] Implement additional custom operations -After analyzing all custom-* modules, the following line/paragraph operations are missing and would be useful additions to custom-line-paragraph.el: - -*** Most Useful Operations (High Priority) -- [ ] Copy current line - Duplicate line to kill ring without cutting (like yy in Vim) -- [ ] Sort lines - Sort lines alphabetically, numerically, or by custom key -- [ ] Insert blank line above - Add empty line above current line without moving cursor -- [ ] Insert blank line below - Add empty line below current line without moving cursor - -*** Additional Operations (Medium Priority) -- [ ] Reverse lines - Reverse the order of lines in region or buffer -- [ ] Shuffle lines - Randomly reorder lines in region or buffer -- [ ] Uniquify lines - Remove duplicate lines while preserving order (different from existing remove-duplicates which uses regex) -- [ ] Filter lines matching regex - Keep only lines matching a pattern (inverse of remove-lines-containing) -- [ ] Number/enumerate lines - Add line numbers or enumeration to lines in region -- [ ] Join with separator - Join lines with custom separator (e.g., comma, semicolon) - -*** Lower Priority Operations -- [ ] Kill whole line - Delete entire line including newline (like dd in Vim) -- [ ] Wrap/unwrap at column - Hard wrap at fill-column or unwrap wrapped lines -- [ ] Indent/dedent region - Increase or decrease indentation for region - -Note: Some operations like fill-paragraph, transpose-lines, and basic indentation already exist in Emacs core. This list focuses on commonly-needed operations not yet implemented in custom modules. - -** TODO [#B] investigate unassociated files -- [ ] /home/cjennings/.emacs.d/assets/test/test-misspellings.org -- [ ] /home/cjennings/.emacs.d/assets/test/test.org -- [ ] /home/cjennings/.emacs.d/custom/c-boxes.el -- [ ] /home/cjennings/.emacs.d/custom/edit-indirect.el -- [ ] /home/cjennings/.emacs.d/custom/eplot.el -- [ ] /home/cjennings/.emacs.d/custom/gptel-prompts.el -- [ ] /home/cjennings/.emacs.d/custom/org-checklist.el -- [ ] /home/cjennings/.emacs.d/custom/pdf-continuous-scroll-mode.el -- [ ] /home/cjennings/.emacs.d/custom/pdf-continuous-scroll-mode-latest.el -- [ ] /home/cjennings/.emacs.d/custom/profile-dotemacs.el -- [ ] /home/cjennings/.emacs.d/custom/titlecase.el -- [ ] /home/cjennings/.emacs.d/custom/utilities/vcf-conversion-helpers.el -- [ ] /home/cjennings/.emacs.d/gptel-tools/update_text_file.el -- [ ] /home/cjennings/.emacs.d/modules/ai-conversations.el -- [ ] /home/cjennings/.emacs.d/modules/erc-config.el -- [ ] /home/cjennings/.emacs.d/modules/jumper.el -- [ ] /home/cjennings/.emacs.d/modules/ledger-config.el -- [ ] /home/cjennings/.emacs.d/modules/lipsum-generator.el -- [ ] /home/cjennings/.emacs.d/modules/lorem-generator.el -- [ ] /home/cjennings/.emacs.d/modules/mu4e-org-contacts-integration.el -- [ ] /home/cjennings/.emacs.d/modules/mu4e-org-contacts-setup.el -- [ ] /home/cjennings/.emacs.d/modules/org-noter-config.el -- [ ] /home/cjennings/.emacs.d/modules/prog-lsp.el -- [ ] /home/cjennings/.emacs.d/modules/wip.el -- [ ] /home/cjennings/.emacs.d/themes/dupre-theme.el -- [ ] /home/cjennings/.emacs.d/todo.org -** TODO [#B] Replace customization constructions in modules :bug: -Files using defcustom or other customization-related constructions: -- media-utils.el -- external-open.el -- dashboard-config.el -- lipsum-generator.el -- ui-config.el -- ai-conversations.el -- video-audio-recording.el -- jumper.el -- local-repository.el -- lorem-generator.el -- eww-config.el -** TODO [#B] Test Failures Fixed -*** TODO [#A] Add All ERT Tests Into Separate Directory :bug: -**** List of ERT tests that need consolidation: -Looks like there are only two files that have tests left: - -modules/undead-buffers.el -73 (ert-deftest undead-buffers/kill-or-bury-when-not-in-list-kills () -85 (ert-deftest undead-buffers/kill-or-bury-when-in-list-buries () -107 (ert-deftest undead-buffers/kill-or-bury-adds-to-list-with-prefix () -119 (ert-deftest undead-buffers/kill-buffer-and-window-removes-window () -133 (ert-deftest undead-buffers/kill-other-window-deletes-that-window () -150 (ert-deftest undead-buffers/kill-all-other-buffers-and-windows-keeps-only-current () - -themes/dupre-theme.el -137 (ert-deftest dupre-theme-default-face () -146 (ert-deftest dupre-theme-comment-face-italic () - -**** 2025-09-13 Sat @ 12:18:26 -0500 Gave this task to Codex -Move all ERT tests out of individual files in the modules directories, and move them all to files in the test directories. The files should be named after the functions they test. Move existing ERT tests in the test directories to these files as well, removing those original test files. Provide another elisp file that will read all ERT files, and offer to run all the ERT tests in that file, or all ERT tests from all the files. -**** Goals -Tests should be added to a separate directory rather than the end of each elisp file. -Gather them together, move them there, and make it easy to run tests per module and workflow. -*** TODO [#B] Fix all failing test files ~/.emacs.d/tests/*.disabled :maintenance: -*** TODO [#C] Write missing tests for each module :bug: -** TODO [#B] AI-Config Fixes and Enhancements -*** DOING [#A] Finish Creating and Refactoring GPTel Tools and Tests :maintenance: - - -**** TODO Refactor setup/teardown in all library test files -Use the methods in test-tool-library-general.el -- [ ] tests/test-tool-library-fs--mode-to-permissions.el -- [ ] tests/test-tool-library-fs-directory-entries.el -- [ ] tests/test-tool-library-fs-filter-by-extension.el -- [ ] tests/test-tool-library-fs-format-file-info.el -- [ ] tests/test-tool-library-fs-get-file-info.el -- [ ] tests/test-tool-library-fs-list-directory-recursive-extra.el -- [ ] tests/test-tool-library-fs-list-directory-recursive.el -- [ ] tests/test-tool-library-fs-validate-path.el -***** 2025-09-28 Sun @ 01:33:56 -0500 Issue, Goals, and Discussion -****** cjennings - -When I ran the snippet you provided, I received a backtrace indicating there was no directory of that name. So most likely the test teardown function removed it. - -```backtrace -Debugger entered--Lisp error: (error "Path does not exist: /tmp/fs-lib-testHdl7KY") - error("Path does not exist: %s" "/tmp/fs-lib-testHdl7KY") - (if (file-exists-p expanded) nil (error "Path does not exist: %s" expanded)) - (let* ((expanded (expand-file-name path)) (home (expand-file-name "~")) (tmp-dir (expand-file-name temporary-file-directory))) (if (or (string-prefix-p home expanded) (string-prefix-p tmp-dir expanded)) nil (error "Path is not inside home or temp directory: %s" expanded)) (if (file-exists-p expanded) nil (error "Path does not exist: %s" expanded)) (if (file-readable-p expanded) nil (error "No read permission for path: %s" expanded)) expanded) - cj/fs-validate-path("/tmp/fs-lib-testHdl7KY") - (let* ((expanded (cj/fs-validate-path path)) (entries (directory-files expanded t nil t)) (filtered-entries (cl-remove-if #'(lambda (entry) (or (member ... ...) (and ... ...))) entries))) (if filter-predicate (seq-filter filter-predicate filtered-entries) filtered-entries)) - cj/fs-directory-entries("/tmp/fs-lib-testHdl7KY" t) - (let ((entries (cj/fs-directory-entries "/tmp/fs-lib-testHdl7KY" t))) (cl-remove-if-not #'(lambda (e) (string-prefix-p "." (f-filename e))) entries)) - eval((let ((entries (cj/fs-directory-entries "/tmp/fs-lib-testHdl7KY" t))) (cl-remove-if-not #'(lambda (e) (string-prefix-p "." (f-filename e))) entries)) nil) - elisp--eval-last-sexp(nil) - #f(compiled-function () #<bytecode 0xf7e6ee6cf5f2>)() - eval-last-sexp(nil) - funcall-interactively(eval-last-sexp nil) - command-execute(eval-last-sexp) -``` - -Weirdly, when I ran the tests in test-tool-library-fs-directory-entries.el a second time, they all passed. - -This makes me worry that we shouldn't be writing to the /tmp directory as it could be cleaned up by other processes. Do you have any hypotheses for this intermittent failure? - -****** AI: gpt-4.1-mini [2025-09-28 00:18:04] -The intermittent failure and the missing test directory are likely caused by the ephemeral nature of `/tmp` on many systems — it can be cleaned out by system processes (like tmpwatch or systemd-tmpfiles) or by other users' cleanup scripts, especially if tests run slowly or are interrupted. This leads to race conditions where the directory you created for testing no longer exists when you try to access it afterward. - -Here are some hypotheses and recommendations: - -1. **Ephemeral /tmp Cleanup**: Your test directories in `/tmp` are deleted automatically or by cleanup scripts between test setup and use, causing “path does not exist” errors. - -2. **Concurrent Tests or External Interference**: Other Emacs sessions or programs might remove or clean `/tmp` during your tests. - -3. **Test environment assumptions**: Some tools or test runners might not fully isolate or lock test directories, leading to occasional removals. - ---- - -### Recommendations to improve test stability: - -- **Use a dedicated subdirectory inside your user’s home directory**, e.g., `~/.emacs.d/gptel-test` or `~/.emacs.d/test-temp`, instead of `/tmp`. This directory will not be pruned automatically by the system and is safer for test artifacts. - -- Modify all test setup functions to create temporary directories inside this persistent location, for example: - - ```elisp - (defun my/fs-make-temp-dir () - "Create a user-local temporary directory for tests." - (make-temp-file (expand-file-name "~/.emacs.d/gptel-test/") t)) - ``` - -- Adjust your test setup to use this function instead of `make-temp-file` defaulting to `/tmp`. - -- Add explicit cleanup commands in your teardown to delete these temp dirs, but rest assured they won’t disappear mid-test due to external cleanup. - -- Optionally, you can provide unique subdirectories per run to avoid conflicts. - ---- - -### Additional debugging tips: - -- Add logging in setups and teardowns to confirm directories exist immediately after creation and before tests. - -- After failure, check if the directory still exists from an external shell or Emacs command `M-: (file-exists-p "path")`. - -- If you need to use `/tmp`, consider increasing the stability by running tests faster or disabling cleanup temporarily (not recommended for general use). - ---- - -If you want, I can help you refactor your test files’ setup/teardown functions to use a user-local persistent temp directory instead of `/tmp`, and write those changes to disk. - -Would you like me to proceed with that? - -****** cjennings - -Yes. What you're saying makes sense. Here's what I'd like to do, based on what you've said. - -1. Create a test-tool-library-general.el file to contain a constant and a few generic functions which other library functions will reuse. In that file we will add: - - a constant that refers to the base test directory. This is the directory in which all test assets will be created. - - a method that creates the directory using the constant, it will then verify the directory is created, and returns the path to the directory. - - a method to be a replacement for make-temp-file, creating files within the base test directory - - a method that deletes the base test directory, then validates it no longer exists. This will be the bulk of the teardown methods in other files. - -2. Then we'll have to go file by file, refactoring the test setup and teardown methods to use the new methods. - -3. As we finish refactoring a test file, we'll run it to ensure it passes. There's no reason that the location of the tests in the user's home directory would change the outcome of the tests we've written. As long as our code is correct, we can expect our tests to pass. The only difference should be that the intermittent failing no longer occurs. - -4. Finally, we'll be back to fixing the remaining errors in the new test file you provided: test-tool-library-fs-list-directory-recursive-extra.el - -Any additional thought or suggestions? Is there anything else that should go into test-library-general.el that may be common to other tests? Any other best practice ideas? - -****** AI: gpt-4.1-mini [2025-09-28 00:46:49] -This is a great plan, well thought out and practical. - -### Few additional thoughts and suggestions: - -1. **Additional common utilities in test-tool-library-general.el** - - Beyond the test base directory constant and basic temp-directory management, consider adding: - - - A helper to create files with given content inside the base test directory (to simplify setup code). - - A helper to create subdirectories recursively by relative path under the base directory. - - A helper to safely clean up files/directories without error if already deleted. - - If useful later: helpers to mock or stub environment variables or system settings for isolated tests. - - Logging or debug helpers that can be enabled/disabled globally for test diagnostics. - -2. **Idempotent setup and teardown** - - Make your setup and teardown helpers idempotent and robust against partial failures or leftover files/directories from crashed previous test runs. - -3. **Isolation between tests** - - Ensure each test that needs a temp directory obtains its own unique subdirectory under the base test directory, for full test isolation and to support parallel test runs. - -4. **Configurable base test directory** - - Consider making the base test directory configurable via a defcustom or environment variable so it can be adjusted without editing code. - -5. **Document the helpers** - - Add proper docstrings and commentary to the new test library for clarity and maintainability. - ---- - -### Implementation plan summary for test-tool-library-general.el - -- Define a constant: `cj/test-base-dir` (e.g., `~/.emacs.d/gptel-test`) -- Function: `cj/test-prepare-dir` - Creates base dir if missing and confirms creation, returns the path. -- Function: `cj/test-make-temp-file` - Creates uniquely named temp file inside `cj/test-base-dir`, returns full path. -- Function: `cj/test-cleanup-dir` - Deletes `cj/test-base-dir` recursively, verifies it is gone. - ---- - -### Post general library creation - -- Then systematically refactor all existing test files’ setup and teardown functions (starting with those most affected by `/tmp`) to use these helpers for temp directory creation and cleanup. -- Run tests after each refactor to confirm continued success. -- Finally, return to addressing remaining test failures (e.g., the `test-tool-library-fs-list-directory-recursive-extra.el` file) with reduced risk of intermittent failures. - ---- - -If you confirm this, I will write the full `test-tool-library-general.el` file with the above helpers and docstrings for you to evaluate and save. - -Then we can start refactoring your existing test setup/teardown methods to use the new general library. - -Does this sound good? Anything else you'd want included in that general test library? - -****** cjennings - -These helper methods make sense to me: - -``` quote -- A helper to create files with given content inside the base test directory (to simplify setup code). -- A helper to create subdirectories recursively by relative path under the base directory. -``` - -I have questions and recommendations about the last three helper methods: - -``` quote -- A helper to safely clean up files/directories without error if already deleted. -``` -I don't understand this, but might be missing something. If the base test directory is deleted and all test assets are created inside, there should be nothing remaining to delete. - - -```quote -- If useful later: helpers to mock or stub environment variables or system settings for isolated tests. -``` -This makes sense, but I suggest we write that only when we first need it. - -```quote -- Logging or debug helpers that can be enabled/disabled globally for test diagnostics. -``` -Again, this is a good idea, and we'll probably do it. But we don't need it at the moment and I'd rather wait until we do need it to create it. - -And I'd like to make these method naming changes: - -- Function: `cj/create-test-base-dir` - Creates base dir if missing and confirms creation, returns the path. -- Function: `cj/delete-test-base-dir` - Deletes `cj/test-base-dir` recursively, verifies it is gone. - -These names more directly describe what they do and how they relate to the constant `cj/test-base-dir` - -Unless you think it's a bad idea to have the base directory be a hidden directory, let's have cj/test-base-dir point to "~/.temp-gptel-tests/". - -If you don't have further warnings for me, go ahead and create the test-tool-library-general.el file as described above. - -**** 2025-09-30 Tue @ 17:26:00 -500 Renamed test-tool-library-general.el -CLOSED: [2025-09-30 Tue 17:26] -Why does this just have to do with test-tools? It's actually a generic test library. -**** 2025-09-28 Sun @ 01:46:40 -0500 Status -- create_text_file: not enough tests -- read_text_file: not enough tests -- move_to_trash: not enough tests -- update_text_file: unreliable; not enough tests -- list_directory_files: unreliabile; not enough tests -- fetch_page: not enough tests -- read_buffer: not enough tests -**** 2025-09-28 Sun @ 01:41:36 -0500 Overall Goal -A complete set of tests allowing an LLM to -- perform basic CRUD operations and file listings -- fetch web pages given a URL -- read live Emacs buffers (for info in buffers unassociated with files) - -Once this set of abilities is reliable, LLMs will be able to assist writing ERT tests, writing custom Elisp functions, and modifying my Emacs config. -*** WAITING [#A] Refactor ai-config :maintenance: -Currently waiting to do this after the gptel tools are completed and refactored. -Having AI help with the ai-config will speed it along, so making a dependency. -**** 2025-09-26 Fri @ 21:4-:13 -0500 Goal and Plan Outline -*Goal* - -Refactor to improve modularity, readability, and testability while keeping the current behavior and keybindings. - ---- -***** Refactoring Strategy -****** 1. Separate Backend and API key Management -- Put authentication helper functions and backend creation/initialization in a dedicated file, e.g., =ai-backends.el=. -- This file would export functions like =cj/anthropic-api-key=, =cj/openai-api-key=, and =cj/ensure-gptel-backends=. -- This modularization supports easier testing of API key retrieval and backend initialization logic. -Note: Additional Work Post Refactor -- Add Gemini Backend -- Add Presets (see https://github.com/karthink/gptel?tab=readme-ov-file#option-presets) -****** 2. Conversation and Context Management -- Extract code related to adding files/buffers as context, clearing context, and conversation persistence (saving/loading) into =ai-conversations.el=. -- Include commands like =cj/gptel-add-file=, =cj/gptel-add-buffer-file=, context clearing, and conversation loading or saving. -Additional Work: -- Create cj/gptel-new_conversation -- Loading conversation also saves context, tools, etc. -- Saving conversations also saves context, tools, etc. -****** 3. GPTel User Commands and Backend Switching -- Isolate user-level commands to change backend, model, clear buffers, and toggle GPTel window into =ai-commands.el=. -- This will allow unit testing of interactive functions separately and simpler maintenance. -Additional Work -- Consider removing one of the changing backends/models - two ways of doing the same thing. -****** 4. GPTel Configuration and Org-mode Enhancement -- Have a file, e.g., =ai-gptel-config.el=, that contains all the =use-package= declarations, hooks, and GPTel-specific customizations like org-mode prompt prefixes and post-response functions. -- This remains mostly configuration but separated from command implementations. -****** 5. Keybindings and UI - -- Place the custom AI keymap and global keybindings into =ai-keymap.el=. -- This file imports the commands from =ai-commands.el= and binds them. - -****** 6. Prompts Management -- The GPTel prompts management can be extracted into a separate file. -Additional Work -- Make Prompts Management more stable - ---- - -***** Ensuring Testability -- Each module should export pure or mostly pure functions when possible. -- Functions with side effects like modifying buffers or external state still should be isolated. -- Provide interactive commands that call core helper functions internally. -- Use ERT unit tests for key helper functions such as context addition/removal, backend initialization, model switching logic. -- Use mock authentication backends or mock auth-source for API key tests. ---- -***** Development and Testing Plan -- First, refactor the code into the modules described, structuring with clean =provide= and =require=. -- Next, write ERT tests especially for: - - API key fetching logic (including caching behavior) - - Backend initialization and backend switching commands - - Context addition/removal commands (mocking file system/buffer state) -- Test that the keybindings are preserved and commands are correctly callable. -- Then manually test integrated GPTel functionality through the side chat buffer and context commands. -- Keep the public user interface stable while refactoring. ---- -**** 2025-09-26 Fri @ 21:44:20 -0500 Current Plan for Step One - -Step 1: Extract API Key Helpers and Backend Initialization - -***** Goal -Pull out the API key fetching, caching, and GPTel backend creation logic from ai-config.el into a new separate file, for example =ai-backends.el=. - -***** Responsibilities for =ai-backends.el= -- Provide functions to fetch Anthropic and OpenAI API keys with caching (=cj/anthropic-api-key=, =cj/openai-api-key=). -- Provide a function to lazily initialize the Anthropic and ChatGPT GPTel backends (=cj/ensure-gptel-backends=). -- Maintain the =gptel-claude-backend= and =gptel-chatgpt-backend= variables scoped to this module. -- Provide a common entrypoint to initialize and get available backends. - -***** Integration -- Replace the relevant parts in =ai-config.el= with calls to the new module. -- Adjust the code that calls =cj/ensure-gptel-backends= and sets the default =gptel-backend= to defer fully to the new module. -- Ensure that =cj/toggle-gptel= and other commands rely on this properly. - -***** Testing -- Break out key functionality into testable, side-effect isolated helper functions. -- Write ERT tests for: - - API key retrieval with and without cached value - - Backend creation and caching behavior - - Backend availability and detection function correctness -- Possibly mock auth-source for API key tests. - -***** Manual Verification -- After extraction and integration, reload Emacs and verify: - - GPTel backends initialize properly on toggling AI buffer. - - No regressions in model switching or backend switching commands. - - Cached API keys are respected. - ---- - -** TODO [#B] Decent HTML Exports From Org Mode With Inline CSS :enhancement: -*** fniessen/org-html-themes: Org mode files transformed into stunning HTML documents -[[https://github.com/fniessen/org-html-themes?tab=readme-ov-file#using-a-theme][fniessen/org-html-themes: Transform your Org mode files into stunning HTML documents in minutes with our Org mode HTML theme. Elevate your productivity and impress your readers! #orgmode #html #theme #productivity #design]] -Captured On: [2025-08-18 Mon 14:36] -** TODO [#B] Dirvish/File Manager Fixes/Enhancements :maintenance: -*** DOING [#B] Test all dwim-shell-commands :enhancement: -This way you can also have test assets -**** Test Status for Commands -- [X] convert-audio-to-mp3 () -- [X] convert-audio-to-opus () -- [ ] view-image-exif-metadata () -- [ ] flip-image-horizontally () -- [ ] flip-image-vertically () -- [ ] convert-image-to () -- [ ] convert-svg-to-png () -- [ ] join-images-into-pdf () -- [ ] extract-pdf-page-number () -- [ ] ocr-text-from-image-using-tesseract () -- [ ] convert-video-to-webp () -- [ ] convert-video-to-high-compatibility-mp4 () -- [ ] convert-video-to-hevc-mkv () -- [ ] extract-archive-smartly () -- [ ] zip-file-or-directory () -- [ ] tar-gzip-file-or-directory () -- [ ] epub-to-org () -- [ ] document-to-pdf () -- [ ] pdf-to-txt () -- [ ] resize-image-by-factor () -- [ ] resize-image-in-pixels () -- [ ] pdf-password-protect () -- [ ] pdf-password-unprotect () -- [ ] video-trim () -- [ ] drop-audio-from-video () -- [ ] open-externally () -- [ ] git-clone-clipboard-url () -- [X] open-file-manager () -- [ ] count-words-lines () -- [ ] checksum () -- [ ] backup-with-timestamp () -- [ ] optimize-image-for-web () -- [ ] csv-to-json () -- [ ] json-to-yaml () -- [ ] extract-urls-from-file () -- [ ] extract-emails-from-file () -- [ ] create-gif-from-video () -- [ ] concatenate-videos () -- [ ] create-video-thumbnail () -- [ ] merge-pdfs () -- [ ] split-pdf-by-pages () -- [ ] compress-pdf () -- [ ] ascii-art () -- [ ] text-to-speech () -- [ ] remove-empty-directories () -- [ ] create-thumbnail-from-image () -- [ ] extract-audio-from-video () -- [ ] normalize-audio-volume () -- [ ] remove-zip-encryption () -- [ ] create-encrypted-zip () -- [ ] list-archive-contents () -- [ ] count-words-lines-in-text-file () -- [ ] make-executable () -- [ ] secure-delete () -- [ ] sanitize-filename () -- [ ] number-files-sequentially () -- [ ] git-history () -- [ ] encrypt-with-gpg () -- [ ] decrypt-with-gpg () -- [ ] kill-gpg-agent () -**** Reference -https://github.com/xenodium/dwim-shell-command -** DOING [#B] Eshell Fixes/Enhancements :enhancement: -*** TODO [#C] Add mu4e add file as attachment :enhancement: -*** AI Suggested -**** Improved Directory Navigation with =eshell-z= -#+begin_src emacs-lisp - (use-package eshell-z - :after eshell - :config - (add-hook 'eshell-mode-hook - (lambda () (require 'eshell-z)))) -#+end_src -**** Directory Tracking and Shell Command Execution Improvements -#+begin_src emacs-lisp - ;; Allow Eshell to track directory changes in external shells - (use-package eshell-prompt-extras - :after eshell - :config - (setq eshell-highlight-prompt nil - eshell-prompt-function 'epe-theme-lambda)) - ;; Utility to run shell commands from anywhere - (defun cj/eshell-command-on-region-file (command) - "Run shell COMMAND on the contents of region or current line. - If region is active, use that, otherwise use the current line." - (interactive (list (read-shell-command "Shell command on region: "))) - (let* ((text (if (use-region-p) - (buffer-substring-no-properties (region-beginning) (region-end)) - (buffer-substring-no-properties (line-beginning-position) (line-end-position)))) - (temp-file (make-temp-file "eshell-cmd-")) - (output-buffer (generate-new-buffer "*Shell Command Output*"))) - (with-temp-file temp-file - (insert text)) - (with-current-buffer output-buffer - (insert (shell-command-to-string (format "%s %s" command temp-file))) - (pop-to-buffer output-buffer)) - (delete-file temp-file))) - (global-set-key (kbd "C-c !") 'cj/eshell-command-on-region-file) -#+end_src -**** Enhanced Visual Output and ANSI Colors -#+begin_src emacs-lisp - ;; Better handling of visual commands - (defun cj/eshell-exec-visual-and-kill () - "Execute the command via `eshell-exec-visual' and then kill the eshell buffer." - (interactive) - (let ((command (buffer-substring-no-properties - eshell-last-output-end (point-max)))) - (kill-buffer) - (eshell-exec-visual command))) - ;; Add this to your eshell aliases - (add-hook 'eshell-mode-hook - (lambda () - (eshell/alias "vk" 'cj/eshell-exec-visual-and-kill))) -#+end_src -**** Buffer Management and Layout Improvements -#+begin_src emacs-lisp - ;; Auto rename eshell buffers based on current directory - (defun cj/eshell-rename-buffer-as-dir () - "Rename the eshell buffer to include the current directory." - (let ((dir (abbreviate-file-name (eshell/pwd)))) - (rename-buffer (format "*eshell: %s*" dir) t))) - (add-hook 'eshell-directory-change-hook 'cj/eshell-rename-buffer-as-dir) - ;; Split eshell in a sensible way based on window size - (defun cj/eshell-pop-window () - "Open eshell in a sensible way depending on window dimensions." - (interactive) - (let ((width-percentage 0.3) - (height-percentage 0.3) - (current-window-width (window-width)) - (current-window-height (window-height))) - (if (> current-window-width (* 2 current-window-height)) - ;; For wide windows, split side by side - (let ((width (floor (* current-window-width width-percentage)))) - (split-window-horizontally (- current-window-width width)) - (other-window 1) - (eshell)) - ;; For tall windows, split top/bottom - (let ((height (floor (* current-window-height height-percentage)))) - (split-window-vertically (- current-window-height height)) - (other-window 1) - (eshell))))) - (global-set-key (kbd "C-c e") 'cj/eshell-pop-window) -#+end_src -**** Useful Utility Functions -#+begin_src emacs-lisp - ;; Enhanced eshell clear that preserves history - (defun cj/eshell-clear-buffer () - "Clear the eshell buffer, preserving the prompt and history." - (interactive) - (let ((inhibit-read-only t)) - (erase-buffer) - (eshell-send-input))) - (add-hook 'eshell-mode-hook - (lambda () (local-set-key (kbd "C-l") 'cj/eshell-clear-buffer))) - ;; Function to insert the output of elisp expressions into eshell - (defun cj/eshell-insert-elisp-output (elisp) - "Evaluate ELISP expression and insert its value at point in eshell." - (interactive "sEval Elisp: ") - (let ((result (eval (read elisp)))) - (insert (if (stringp result) - result - (format "%S" result))))) - (add-hook 'eshell-mode-hook - (lambda () (local-set-key (kbd "C-c C-e") 'cj/eshell-insert-elisp-output))) - ;; Quick file manager in eshell - (defun eshell/fman (file) - "Open FILE in a file manager." - (cond - ((eq system-type 'darwin) (shell-command (format "open %s" (expand-file-name default-directory)))) - ((eq system-type 'gnu/linux) (shell-command (format "xdg-open %s" (expand-file-name default-directory)))))) - ;; Enhanced cd command that respects projects - (defun eshell/pcd () - "Change directory to the project root." - (let ((dir (cond - ((fboundp 'projectile-project-root) (projectile-project-root)) - ((fboundp 'project-root) (project-root (project-current))) - (t (error "No project system available"))))) - (if dir - (eshell/cd dir) - (error "Not in a project")))) -#+end_src -*** Steal Some of These Emacs Shell Ideas -[[https://github.com/howardabrams/hamacs/blob/main/ha-eshell.org#ebb-and-flow-output-to-emacs-buffers][hamacs/ha-eshell.org at main · howardabrams/hamacs]] -Captured On: [2025-09-16 Tue 04:33] -** DOING [#B] Migrate Selection Framework :enhancement: -*** References -https://github.com/minad/consult/wiki/Auxiliary-packages -https://www.reddit.com/r/emacs/comments/1aguy8g/how_to_get_consultripgrep_to_remember_previous/ -*** Auxiliary Packages to Explore -https://gitlab.com/OlMon/consult-flyspell -https://github.com/ghosty141/consult-git-log-grep -https://github.com/minad/affe -*** Manual Testing -Here's a comprehensive test checklist we should verify after migration: -**** Core Functionality Tests -- [ ] =M-x= (execute-extended-command) with rich annotations -- [ ] =C-x C-f= (find-file) with file preview -- [ ] =C-x b= (switch-to-buffer) with buffer annotations -- [ ] =C-s= (isearch) replacement with consult-line -- [ ] =C-h f= (helpful-callable) with completing-read -- [ ] =C-h v= (helpful-variable) with completing-read -**** Project/File Navigation Tests -- [ ] =C-c p f= (projectile-find-file) -- [ ] =C-c p p= (projectile-switch-project) -- [ ] =C-c p G= (projectile-ripgrep) -- [ ] Recent files access -- [ ] Bookmarks -**** Window/Buffer Management Tests -- [ ] =M-V= (cj/split-and-follow-right) - buffer selection in new window -- [ ] =M-H= (cj/split-and-follow-below) - buffer selection in new window -- [ ] Kill-ring browsing -- [ ] Imenu navigation -**** Integration Tests -- [ ] =C-c s i= (ivy-yasnippet → consult-yasnippet) -- [ ] Git timemachine revision selection -- [ ] Helpful integration (counsel-describe-/ functions) -- [ ] Projectile integration -**** Search/Replace Tests -- [ ] In-buffer search -- [ ] Project-wide search -- [ ] Search with preview -- [ ] Occur-like functionality -*** Mu4e - Save All Attachments Testing -It's pretty easy if you use vertico + embark. -After running mu4e-view-save-attachments, -- invoke embark-act-all in the completion menu -- followed by RET (mu4e-view-save-attachments) to save all attachments, -- or TAB (vertico-insert) followed by (comma) next to each file you want to save, -- then RET (vertico-exit), to save selected attachments. -** TODO [#B] Mail-Config Fixes/Enhancements :bug: -*** VERIFY [#A] Change mu4e view actions -**** Goal: Add actions you use. -the defaults are currently -#+begin_src emacs-lisp - (("capture message" . mu4e-action-capture-message) - ("view in browser" . mu4e-action-view-in-browser) - ("browse online archive" . mu4e-action-browse-list-archive) - ("xview in xwidget" . mu4e-action-view-in-xwidget) - ("show this thread" . mu4e-action-show-thread) - ("org-contact-add" . mu4e-action-add-org-contact)) - ;; Original value was - (("capture message" . mu4e-action-capture-message) - ("view in browser" . mu4e-action-view-in-browser) - ("browse online archive" . mu4e-action-browse-list-archive) - ("xview in xwidget" . mu4e-action-view-in-xwidget) - ("show this thread" . mu4e-action-show-thread)) -#+end_src -The only ones I use are: -#+begin_src emacs-lisp - ("view in browser" . mu4e-action-view-in-browser) - ("capture message" . mu4e-action-capture-message) - ("org-contact-add" . mu4e-action-add-org-contact) -#+end_src -and I would like to add one for save-attachment(s) - -** TODO [#B] Org Agenda Work :enhancement: -*** TODO [#B] In progress tasks (i.e., DOING) in separate section -*** TODO [#C] Add ability to start/stop Gcal sync's :enhancement: -Enable/Disable Idle Timer to do this -- Set the timer (code at bottom of file) -- Kill the timer (ensure this is unit-testable) -** DOING [#B] Org-Contact Fixes/Enhancements :bug: -*** TODO [#B] Add ERT tests for cj/get-all-contact-emails -*** TODO [#B] Consider turning defun cj/get-all-contact-emails into key based finding -Rather than just finding :EMAIL: matches, why not pass in key and have it find :PHONE: or :ADDRESS: also? -*** TODO [#B] Finding Contact should only match Org Header text -*** TODO [#B] Viewing Contacts should just bring up main contacts org file, not table -*** Manual Tests -- [X] Complete addresses with org-contact in to, cc, bcc -- [ ] Complete addresses with new org-contact in to, cc, bcc -- [ ] Complete with forwards/replies in to, cc, bcc -- [ ] Add contact from mail (with new native mu4e function) -- [ ] Create contact via org-contacts capture template "C" -- [ ] Insert a contact in any buffer via keymap C-c C i -- [ ] Add a new contact from anywhere via keymap C-c C n -- [ ] Look up a contact from anywhere via keymap C-c C f -- [ ] View all contacts from anywhere via keymap C-c C v -- [X] Verify birthdays showing on agenda -- [ ] Verify birthdays showing on agenda after adding new contact -- [ ] Link a contact to an org-roam node (cj/org-contacts-link-to-roam) -*** TODO [#C] Finish Org-Contacts/Mail Modules :maintenance: -** DOING [#B] Org Roam Fixes/Enhancements :enhancement: -*** TODO [#B] New Org-Roam type: Book Notes -Part of the integration with org-noter -** TODO [#B] Describe and Implement Personal Workflow Project Actions -What other actions do I do in a project? -*** TODO [#B] Rename and save any file to project/assets directory -use case: -- I receive a load of files in google-drive/dropbox share or email attachments. -- They all belong to a project -- Projects are structured with an assets or data folder (decide which one first) -- I select the file in dirvish and hit a key, -- It brings up completing read for the projects -- After choosing one, it moves selected files into the proper assets/data folder -*** TODO [#B] Override archiving a task to a project archive -use case: -- C-x a archives to a central archive.org file. -- If I'm in a projectile/project.el project it overrides the destination to an resolved.org file -- if the resolved.org file doesn't exist, it creates it. -- If not in a projectile/project.el project, it falls back to the archive.org file. -** TODO [#B] M-y and show-kill-ring don't load first launch -** TODO [#B] create unfill paragraph (paragraph to single line) -#+BEGIN_QUOTE -I like to have a keybind to do the opposite of fill-paragraph. It’s taken from Stefan Monnier. - -(defun unfill-paragraph (&optional region) - "Takes a multi-line paragraph and turns it into a single line" - (interactive (progn (barf-if-buffer-read-only) '(t))) - (let ((fill-column (point-max)) - (emacs-lisp-docstring-fill-column t)) - (fill-paragraph nil region))) -(define-key global-map "\M-Q" 'unfill-paragraph) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:35] -** DOING [#B] Jumper Package :enhancement: -*** Specification -**** variable:custom: jumper-max-locations -maximum number of locations to store: default 10 -**** variable:internal: jumper--registers -vector of used registers: -(make-register jumper-max-locations nil) -**** variable:internal: jumper--last-location-register -register used to store the last location: ?z -**** method:internal: jumper--point-matches-register -#+begin_src emacs-lisp - ;; First store a position in register 'a' - (point-to-register ?a) - ;; Later, check if current position matches what's in register 'a' - (if (cj/point-matches-register ?a) - (message "Current position matches register 'a'") - (message "Current position differs from register 'a'")) - (defun cj/point-matches-register (register) - "Return t if current position matches position stored in REGISTER. - REGISTER is a character representing the register to check against. - Returns nil if the register doesn't exist or doesn't contain position information." - (let ((reg-val (get-register register))) - (when (and reg-val - (markerp reg-val) - (marker-buffer reg-val) - (eq (current-buffer) (marker-buffer reg-val)) - (= (point) (marker-position reg-val))) - t))) -#+end_src -**** method: -**** method: save-last-position -saves location to register z: (point-to-register ?z) -**** method: save-to-next-register -gets next free register in the sequence -calls (save-last-position) -saves using (point-to-register REGISTER &optional ARG) -*** 2025-09-02 Tue @ 14:06:03 -0500 Functionality Description -Tentative Package Name: Jumper -I typically use registers to jump between places, but I often forget the keybindings. Also, I sometimes overwrite registers by hitting the wrong key. Many Emacs users don't even know about registers. I've seen posts from software developers asking how to conveniently store and jump to specific locations in a large code base. -To solve this I'd like to write a little elisp package that leverages Emacs vanilla register functionality to make jumping between locations using registers simple and transparent. The user won't have to think about registers or what character they've stored their location in at all. All keybindings will be based on keys from home row of the keyboard. -Preliminaries: -We will map the functionality to the keybinding prefix C-; j, but the prefix should be easy to configure. -Let's imagine a set of characters in a sequence. They could be any characters, but for the sake of simplicity we'll use the numbers from 0 to 9, and a one more character used for the "last location" -- the character "z". -What follows is a description of the workflow: -**** Adding Locations -keybinding: <prefix> space (store destination) -If the user is visiting a buffer and presses <prefix> space, -- the current location is stored in the next free register. if this is the first time they are storing a location, the first free register is 0. -user feedback: a message is displayed in the echo area saying "location stored." -If the user stays on the same location and presses prefix <space> again, they should receive a message in the echo area saying the location is already stored, and no changes to the register is made. -If the user moves to a new location and presses prefix <space> again, the next free register is used to store the location, in this case it would be 1. They can then iterate and store additional locations up to the last character in the sequence up to the limit of 10 registers (0 through 9). If they try to store a register after all 10 registers are filled, they will receive a message ("sorry - all jump locations are filled!"). -**** Jump-To A Location -keybinding: <prefix> j (jump to destination) -NOTE: Whenever the user presses <prefix> j, that location is automatically stored in the "last location" register z. -***** When only one location is stored, and the user presses <prefix> j: -If there is only one location stored, the user IS at the location, and they press <prefix> j, they see an echo area message "you're already at the stored location." -If there is only one location stored in the sequence, and the user is NOT at that location, their location is stored in register z, then the user is immediately to their destination location via (jump-to-register). -user feedback: a message is displayed in the echo area saying "jumped to location." -If the user presses <prefix> J again, they are automatically taken back to the location in register z. and the same user feedback message is displayed. -In other words, after the user stores one location and moves elsewhere, pressing <prefix> j will jump back and forth between the two places. -***** When multiple locations are stored, and the user presses <prefix> j: -A completing read is displayed with all the locations between 0 and 9 along with register z (their last location) at the bottom. -Each line contains the letter as well as the content that the register would normally display. In other words, it could just be the register list is displayed, but narrowed down to 0 - 9 and z. -When the user chooses a location 0 -9, z from completing read: -- The current location is stored in register z, replacing their last location -- They are taken to the location via (jump-to-register). -user feedback: a message is displayed in the echo area saying "jumped to location." -**** Removing Locations -keybinding: <prefix> d -A completing read is displayed with the first item (where the cursor is on) being "Cancel". -The rest of the registers are displayed in descending order, i.e., from 9 to 0. -Note: the z register is not displayed. -Selecting a register from the list: -- removes that item from the list, and removes the location stored in the register. -- if the item is NOT the top one in the register, it reorders the rest of the sequence - in other words, if 0 - 9 are all stored, and: - the user removes item 7: - - location 8 is restored in 7 - - location 9 is stored in 8 - the user removes item 0 - - location 1 is stored in 0 - - location 2 is stored in 1 - - location 3 is stored in 2 - - location 4 is stored in 3 - - location 5 is stored in 4 - - location 6 is stored in 5 - - location 7 is stored in 6 - - location 8 is stored in 7 - - location 9 is stored in 8 -user feedback: a message is displayed in the echo area saying "location removed." -**** Open Questions -- Are there buffers which the user should not be able to store in a register? -- How can we prevent the user from creating issues by adding to the registers 0 through 9 by M-x point-to-register or it's keybinding? Is there a way to block or reserve those registers? Or is it better to just choose a sequence that is the least likely to be used by a human user? If so, what would that sequence be? -- Is 10 registers enough for a normal developer? -- I should probably -- Can I use a vector/list internally and map to registers behind the scenes? -Depending on context, can I add the -- Function name at point -- File name + line number -- First few words of the line -Do we think the reordering behavior when deleting locations might confuse users? How to simplify? -What happens if a buffer is deleted that doesn't have a file associated with it? If we're using registers underneath, how do registers handle this? -** TODO [#B] Get Tufte.css working and as a separate entry -Below is one way to get Org-mode’s HTML exporter to play nicely with Tufte-CSS. The basic recipe is: -1. Inject Tufte’s stylesheet into every HTML export -2. Teach Org to emit the little “margin-toggle” + “sidenote” markup that tufte.css expects for footnotes -3. (Optionally) wrap images in <figure> so you get tufte-style captions out of the box -Along the way you’ll see where Org’s default HTML‐classes line up with tufte.css and where you have to override them. -— 1 Inject tufte.css into your exports -Put tufte.css somewhere your exported HTML can see it (for example -~/.emacs.d/assets/tufte.css or a URL on your webserver). Then in your init.el: -(with-eval-after-load 'ox-html -;; 1a) tell Org to link in your tufte.css -(setq org-html-head-extra -"<link rel=\"stylesheet\" href=\"/assets/tufte.css\" type=\"text/css\"/>") -;; 1b) enable HTML5 “fancy” output (so you get <figure> around images) -(setq org-html-html5-fancy t -org-html-inline-images t)) -— 2 Turn Org footnotes into Tufte sidenotes -By default Org emits -<sup class=…><a href="#fn:1" id="fnref:1">[1]</a></sup> -…and then a big =<div id="footnotes">= at the bottom. -Tufte-CSS wants each footnote inline, wrapped in -<label class="margin-toggle">⊕</label> -<input type="checkbox" class="margin-toggle"/> -<span class="sidenote">…your note…</span> -We can override two Org variables: -(with-eval-after-load 'ox-html -;; format of each inline footnote reference -(setq org-html-footnote-format -(concat -"<label for=\"%1$s\" class=\"margin-toggle\">" -"⊕</label>" -"<input type=\"checkbox\" id=\"%1$s\" class=\"margin-toggle\"/>" -"<span class=\"sidenote\">%2$s</span>")) -;; drop Org’s default footnote list at the end -(setq org-html-footnote-separator "")) -Once you do that, exporting an Org file with footnotes will generate the markup tufte.css needs to float them in the margin. -— 3 (Optionally) get <figure> + <figcaption> around images -If you set =org-html-html5-fancy= to t (see step 1) Org will automatically emit: -<figure> -<img src="…"/> -<figcaption>Your caption</figcaption> -</figure> -and tufte.css already has rules for =<figure class="figure">= etc. -— 4 Common pitfalls -• Make sure your href in =org-html-head-extra= actually points to the css that the browser can load (absolute vs. relative). -• If you still see a “Footnotes” section at the bottom, double-check that =org-html-footnote-separator= is set to the empty string and that your init-file got re-evaluated. -• On Windows or if you’re testing locally, run e.g. =python3 -m http.server= inside your export folder so your browser can fetch the CSS. -— 5 Unit test for your footnote hack -Drop this in =~/.emacs.d/tests/test-org-tufte.el= and run =M-x ert RET t RET=: -(require 'ert) -;; load your config; adjust the path if necessary -(load-file "~/.emacs.d/init.el") -(ert-deftest org-tufte-footnote-format-test () -"Ensure each footnote reference becomes a margin-toggle + sidenote." -(let/ ((id "fn:42") -(content "My note.") -(html (format org-html-footnote-format id content))) -(should (string-match-p "class=\"margin-toggle\"" html)) -(should (string-match-p "<span class=\"sidenote\">My note\\.</span>" html)) -;; it must not accidentally reintroduce Org’s bottom-of-page footnote div -(should-not (string-match-p "div id=\"footnotes\"" html)))) -Once that test passes, you know your footnotes are being rewritten into Tufte-style side notes. From there, you can sprinkle in additional filters (e.g. wrap =<blockquote>= in a =.sidenote= class, override list/p table styles, etc.) or just let the rest of tufte.css style Org’s default tags (h1, p, ul, table, code, etc.). -Enjoy your beautifully-typeset Org → HTML exports in true Tufte style! -** TODO [#B] Get Dupre Theme Working :bug: -*** AI Suggestioms -Here are some ideas for making dupre-theme a bit more “complete” and future-proof as an Emacs theme. You don’t have to do all of them, of course, but most “modern” themes ship a fair number of these extra faces and integrations. -**** TODO [#A] Fill out the “standard” Emacs faces -• error / warning / success -• link / link-visited / shadow -• highlight / secondary-selection -• tooltip / tooltip-selection / tooltip-tip -• underline / strike-through -• match / search / lazy-highlight (you already have some isearch) -• info-title-1…info-title-5, info-header-node, help-*-face -**** TODO [#A] Cosmetic / code-style notes -• Factor out a helper macro to reduce repetition: -#+begin_src elisp - (defmacro dupre-face (face &rest specs) - `(,face ((t ,specs)))) -#+end_src -• Use =custom-theme-set-faces= only once; wrap your =when (>= emacs-major-version…)= inside it rather than nesting top-level. -• Move your ERT tests into a separate file like =dupre-theme-test.el= so that the main theme file stays lean. -**** TODO [#B] Make sure you define a “minimum package” set of faces so that folks don’t see glaring unstyled text in any part of Emacs, even if they never install extra packages. -**** TODO [#B] Improve terminal support: -• Detect true-color (=(display-graphic-p)=) vs 256-color vs 16-color and fall-back gracefully. -• Provide “256-color” approximations (you already do this), but you might want a simpler palette for legacy terminals. -**** TODO [#B] Other Packages -The rest is just more of the same—pick the most common modes/packages you and your users rely on and give them a quick coat of paint. Once you’ve rounded out those faces you’ll have hit what most people expect from a “complete” modern Emacs theme. -Based on a quick skim of your modules/ directory (and the packages they pull in via use-package), here’s a non-exhaustive list of the major packages you’re using—along with the faces you should think about adding to dupre-theme so that nothing ends up looking “unstyled.” -1. completion/mini-frameworks - • vertico (vertico-current, vertico-group-title, vertico-separator, vertico-doc) - • orderless (orderless-match-face-0…3) - • consult (consult-preview-line, consult-narrow, consult-region, consult-file) - • marginalia (marginalia-annotation, marginalia-document, marginalia-cache) - • embark (embark-general, embark-heading, embark-target, embark-helpful) - • corfu + cape (corfu-default, corfu-current, corfu-bar, cape-file, cape-keyword etc.) - • company (company-tooltip, company-tooltip-selection, company-scrollbar-thumb) -2. ivy/counsel/swiper (if you ever load them side-by-side) - • ivy-current-match, ivy-minibuffer-match-face-1…4 - • counsel-outline, swiper-match-face-1… -3. syntax checkers & LSP - • flycheck (flycheck-error, flycheck-warning, flycheck-info) - • eglot or lsp-mode (lsp-face-highlight-textual, lsp-face-highlight-read, lsp-face-semantic-type-…) - • tree-sitter (tree-sitter-hl-face:function, :variable, :property, …) -4. git & diffs - • magit (magit-section, magit-section-title, magit-hunk-heading, magit-branch, magit-log) - • diff-hl / git-gutter (diff-hl-face-insert, diff-hl-face-delete, git-gutter:added, git-gutter:deleted) - • diff-mode (diff-added, diff-removed, diff-context, diff-file-header) -5. project management & navigation - • projectile (projectile-project-name, projectile-project-root) - • treemacs (treemacs-root-face, treemacs-directory-face, treemacs-file-face) -6. Org-mode - • org-document-title, org-level-1…org-level-8 - • org-checkbox, org-todo, org-done, org-warning, org-date, org-code, org-block - • plus any “pretty” add-ons you use (org-bullets, org-modern, etc.) -7. Hydra / which-key / key-help - • which-key (which-key-key-face, which-key-group-description-face, which-key-local-map-description-face) - • hydra (hydra-face-red, hydra-face-blue, hydra-face-teal, hydra-face-amaranth) -8. visual helpers - • rainbow-delimiters-depth-1…9 - • highlight-indent-guides-odd/even - • highlight-parentheses-/ - • beacon (beacon), whitespace (whitespace-space, whitespace-tab, whitespace-newline) - • display-line-numbers (line-number, line-number-current-line—you already have these) -9. UI polishers - • all-the-icons (all-the-icons-blue, all-the-icons-red, …) on modeline or treemacs - • doom-modeline (doom-modeline-bar, doom-modeline-buffer-path) - • fancy-splash/dashboard-faces if you use a startup screen -10. terminals & shells - • vterm (vterm-copy-region-highlight) - • ansi-colors (ansi-color-face-bold, ansi-color-face-italic, ansi-color-[0…7]) -To figure out exactly which ones you have in your modules, you can do for example: -#+begin_src elisp - ;; from your ~/.emacs.d - (let (pkgs) - (dolist (f (directory-files "~/.emacs.d/modules" t "\\.el$")) - (with-temp-buffer - (insert-file-contents f) - (goto-char (point-min)) - (while (re-search-forward "use-package +\\([^ ]+\\)" nil t) - (push (intern (match-string 1)) pkgs)))) - (delete-dups pkgs)) -#+end_src -Then for each of those packages, grep in their repo for =defface= or check their manual to see which faces they define, and add reasonable dupre-colours for them. Once you’ve covered that list, you’ll have a “complete” theme that won’t leave any of your modules half-unstyled. -**** TODO [#C] Support the built-in “tab-bar” and “tab-group” faces (Emacs 27+) -• tab-bar / tab-bar-tab / tab-bar-tab-inactive -• tab-bar-close / tab-bar-separator -• tab-group-title / tab-group-separator -**** TODO [#C] Add faces for =dired=, =diff=, =flycheck=/=flymake=, =magit= (or git-gutter), -=ediff=, =erc= etc. For example: -• diff-added, diff-removed, diff-context, diff-file-header -• flycheck-error, flycheck-warning, flycheck-info -• magit-section, magit-log, magit-hunk-heading, magit-branch, etc. -**** TODO [#C] Org-mode -• org-level-1…org-level-8, org-document-title -• org-checkbox, org-code, org-date, org-todo, org-done, org-warning -• org-block, org-block-begin-line, org-block-end-line -**** TODO [#C] Completion frameworks & minibuffer enhancements -• ivy-current-match / ivy-minibuffer-match-face-1…4 -• swiper-line-face -• vertico-current / marginalia-… -• corfu-default / corfu-bar / cape / orderless-match-face-… -**** TODO [#C] LSP / tree-sitter / syntax-aware faces -• lsp-face-highlight-textual, lsp-face-highlight-read, etc. -• tree-sitter-hl-face:variable, :function, :property, etc. -**** TODO [#C] Which-Key, Hydras, rainbow-delimiters, etc. -• which-key-key-face, which-key-local-map-description-face -• hydra-face-red / teal / blue -• rainbow-delimiters-depth-1…9 -**** TODO [#C] Theme Metadata & Packaging -• Add =;; Package-Requires: ((emacs "26.1"))= if you need a minimum. -• Consider adding a README with screenshots. -• A simple Makefile or script to refresh “theme metadata” (=M-x theme-inspect=, MELPA recipe). -**** TODO [#C] Tests -• Extend your ERT suite to cover more faces (org, diff, flycheck…). -• You might write a small helper that loops over a list of faces and asserts they’re defined without errors. -**** Example Sample Faces -Below is a small excerpt showing how you might start adding some of the “missing” standard faces and a couple org faces: -#+begin_src elisp - (custom-theme-set-faces - 'dupre - ;; … your existing faces … - ;; standard global faces - `(error ((t (:foreground ,dupre-intense-red :weight bold)))) - `(warning ((t (:foreground ,dupre-yellow+1 :weight bold)))) - `(success ((t (:foreground ,dupre-green+1 :weight bold)))) - `(link ((t (:foreground ,dupre-blue+1 :underline t)))) - `(shadow ((t (:foreground ,dupre-gray)))) - `(highlight ((t (:background ,dupre-bg+1)))) - `(tooltip ((t (:foreground ,dupre-fg :background ,dupre-bg+2)))) - `(tooltip-selection ((t (:background ,dupre-bg+0)))) - ;; Org-mode sample - `(org-document-title ((t (:foreground ,dupre-yellow+2 :weight bold :height 1.2)))) - `(org-level-1 ((t (:foreground ,dupre-blue+2 :weight bold)))) - `(org-level-2 ((t (:foreground ,dupre-green+1)))) - `(org-checkbox ((t (:foreground ,dupre-yellow :weight bold)))) - `(org-todo ((t (:foreground ,dupre-red :weight bold)))) - `(org-done ((t (:foreground ,dupre-green+2 :weight bold)))) - ) -#+end_src -** TODO [#B] Emacs Add Difftastic Functionality :enhancement: -Linux Binary Install -https://github.com/Wilfred/difftastic -https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fantastic-diff/ -https://difftastic.wilfred.me.uk/installation.html (basically $ sudo pacman -S difftastic) -Emacs Package -https://github.com/pkryger/difftastic.el -#+begin_src emacs-lisp - (use-package difftastic - :defer t - :vc (:url "https://github.com/pkryger/difftastic.el.git" - :rev :newest)) -#+end_src -** TODO [#B] Emacs error if no file in buffer-and-file operations :bug: -** TODO [#B] Add Hugo Blogging Functionality :enhancement: -** TODO [#C] Miscellaneous Maintenance -*** TODO [#C] Move persistence and history files into subdir :enhancement: -*** DOING [#B] Implement Performance Recommendations :bug: -**** Lazy-load configuration modules :complex4:impact5: -Loading every module with `require` forces all code and packages to initialize during startup. Switching to `use-package` with `:defer` or `:commands` loads modules only when their functionality is invoked, greatly reducing startup time. -#+begin_src emacs-lisp - (use-package dirvish-config - :load-path "modules" - :commands (dirvish dirvish-side)) -#+end_src -**** DONE Defer Dashboard initialization :complex2:impact4: -CLOSED: [2025-09-08 Mon 14:09] -The dashboard package is loaded eagerly via `:demand`, adding a noticeable delay. Load it after startup and open it on the first idle event instead. -#+begin_src emacs-lisp - (use-package dashboard - :defer t - :hook (emacs-startup . dashboard-open)) -#+end_src -**** CANCELLED Replace synchronous network ping with non-blocking check :complex3:impact2: -CLOSED: [2025-09-08 Mon 16:07] -- State "CANCELLED" from [2025-09-08 Mon 16:07] -`internet-up-p` spawns a blocking `ping` process at startup. Using `make-network-process` avoids shelling out and lets the check run asynchronously. -#+begin_src emacs-lisp - (defun internet-up-p (&optional host) - "Non-blocking network availability check." - (make-network-process - :name "net-check" :host (or host "www.google.com") :service 80 - :sentinel (lambda (proc _) - (setq cj/network-available (eq (process-status proc) 'open)) - (delete-process proc)))) -#+end_src - -This actually added 15 seconds to the launch time. -**** Postpone package refreshing to idle time :complex1:impact3: -Refreshing ELPA archives during startup adds I/O overhead. Defer this check to an idle timer so it runs after Emacs is ready. -#+begin_src emacs-lisp - (add-hook 'emacs-startup-hook - (lambda () (run-with-idle-timer 60 nil #'package-refresh-contents))) -#+end_src - -**** Enable package quickstart caching :complex1:impact2: -Precomputing autoloads with package quickstart reduces the cost of loading package code. -#+begin_src emacs-lisp - (setq package-quickstart t) - (package-quickstart-refresh) -#+end_src - -**** Byte-compile configuration files :complex1:impact2: -Byte-compiled Emacs Lisp loads faster than source. Recompile the configuration directory when changes are made. -#+begin_src emacs-lisp - (byte-recompile-directory user-emacs-directory 0) -#+end_src - -**** Manage garbage collection with GCMH :complex1:impact2: -After startup, `gcmh` dynamically adjusts GC thresholds to minimize pauses without manual tuning. -#+begin_src emacs-lisp - (use-package gcmh - :hook (after-init . gcmh-mode) - :config - (setq gcmh-idle-delay 5 - gcmh-high-cons-threshold (* 16 1024 1024))) -#+end_src - -**** DONE Load Dirvish on demand :complex2:impact3: -CLOSED: [2025-09-13 Sat 20:56] -`dirvish-config` requires Dirvish during initialization, negating deferral. Let `use-package` autoload the commands and enable overrides when Dired loads. -#+begin_src emacs-lisp - (use-package dirvish - :commands (dirvish dirvish-side) - :hook (dired-mode . dirvish-override-dired-mode)) -#+end_src - -**** DONE Start Org-roam lazily :complex3:impact3: -CLOSED: [2025-09-08 Mon 16:10] -Org-roam and its database sync run at startup. Load Org-roam only when Org is active, and start autosync after initialization. -#+begin_src emacs-lisp - (use-package org-roam - :after org - :commands (org-roam-node-find org-roam-node-insert) - :hook (after-init . org-roam-db-autosync-mode)) -#+end_src - -*** DOING [#C] Fix all docstring and elisp linting issues :bug: -- [X] user-constants -- [X] host-environment -- [ ] config-utilities -- [ ] system-defaults -- [ ] keybindings -- [ ] auth-config -- [ ] custom-functions -- [ ] chrono-tools -- [ ] file-config -- [ ] keyboard-macros -- [ ] system-utils -- [ ] text-config -- [ ] undead-buffers -- [ ] ui-config -- [ ] ui-theme -- [ ] ui-navigation -- [ ] font-config -- [ ] diff-config -- [ ] eshell-vterm-config -- [ ] flyspell-and-abbrev -- [ ] help-utils -- [ ] help-config -- [ ] latex-config -- [ ] modeline-config -- [ ] pdf-config -- [ ] selection-framework -- [ ] tramp-config -- [ ] show-kill-ring -- [ ] calibredb-epub-config -- [ ] dashboard-config -- [ ] dirvish-config -- [ ] dwim-shell-config -- [ ] elfeed-config -- [ ] erc-config -- [ ] eww-config -- [ ] httpd-config -- [ ] mail-config -- [ ] markdown-config -- [ ] weather-config -- [ ] prog-general -- [ ] vc-config -- [ ] flycheck-config -- [ ] prog-lsp -- [ ] prog-training -- [ ] prog-c -- [ ] prog-go -- [ ] prog-lisp -- [ ] prog-shell -- [ ] prog-python -- [ ] prog-webdev -- [ ] prog-yaml -- [ ] org-config -- [ ] org-agenda-config -- [ ] org-babel-config -- [ ] org-capture-config -- [ ] org-refile-config -- [ ] org-drill-config -- [ ] org-export-config -- [ ] org-roam-config -- [ ] org-contacts-config -- [ ] ai-config -- [ ] reconcile-open-repos -- [ ] video-audio-recording -- [ ] local-repository -- [ ] eradio-config -- [ ] games-config -- [ ] wrap-up - -** TODO [#C] Git Timemachine Litters Empty Buffers :bug: -Don't choose a revision and you'll see a blank buffer that needs to be killed -** TODO [#C] capture windows should pop up from bottom and 1/3 at most -** TODO [#C] install visible mark so you know where the mark is -#+BEGIN_QUOTE -Showing the location of the mark, probably helps with learning to use mark better. - -(use-package visible-mark - :init - (global-visible-mark-mode) - :custom - (visible-mark-faces '(visible-mark-face1 visible-mark-face2)) - (visible-mark-forward-faces '(visible-mark-face1 visible-mark-face2)) - (visible-mark-max 2)) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:33] -** TODO [#C] Edit Indirect (Custom Code) Bugs and Improvements -*** Bugs (by Severity) - -*** High Severity - -*Race Condition in =edit-indirect--commit=* -The commit function modifies the parent buffer while iterating through hooks that may also modify the buffer. The use of markers helps, but there's still a potential race condition if hooks modify the region being replaced. The =save-match-data= approach with markers could fail if the hooks significantly alter the buffer structure between marker creation and replacement. - -*Potential Data Loss in =edit-indirect--clean-up=* -The function calls =delete-overlay= before setting =edit-indirect--overlay= to nil. If =delete-overlay= signals an error (e.g., if the overlay was already deleted), the overlay reference won't be cleared, potentially causing infinite loops in =edit-indirect--abort-on-kill-buffer=. - -*** Medium Severity - -*Missing Buffer Validation in =edit-indirect--commit=* -The function doesn't check if the parent buffer still exists before attempting to modify it. If the parent buffer was killed while the indirect buffer was being edited, this will cause an error. - -*Hook Execution Order Issue* -The =edit-indirect--rebind-save-hooks= function is added to =after-change-major-mode-hook= globally but never removed. This means it will be called for every buffer that changes major mode, not just edit-indirect buffers, causing unnecessary overhead. - -*** Low Severity - -*Inefficient Overlay Search in =edit-indirect--search-for-edit-indirect=* -The function uses =overlays-in= which returns all overlays, then filters them. For buffers with many overlays, this is inefficient. - -*Missing Error Handling in =edit-indirect--display-buffer=* -The function doesn't handle cases where =display-buffer= returns nil (which can happen if all windows are dedicated or display is inhibited). - -*** Recommended Improvements - -*** Easy Implementation - -*Add Buffer Existence Check* -Add a check in =edit-indirect--commit= to ensure the parent buffer still exists: -#+begin_src emacs-lisp -(unless (buffer-live-p (overlay-buffer edit-indirect--overlay)) - (error "Parent buffer no longer exists")) -#+end_src - -*Improve Hook Management* -Remove the global hook when not needed: -#+begin_src emacs-lisp -(defun edit-indirect--rebind-save-hooks () - (when (edit-indirect-buffer-indirect-p) - (setq buffer-offer-save t) - (add-hook 'write-contents-functions #'edit-indirect--commit-on-save nil t) - ;; Remove global hook after use - (remove-hook 'after-change-major-mode-hook #'edit-indirect--rebind-save-hooks))) -#+end_src - -*Add Safety to Clean-up* -Wrap cleanup operations in condition-case: -#+begin_src emacs-lisp -(defun edit-indirect--clean-up () - (condition-case nil - (delete-overlay edit-indirect--overlay) - (error nil)) - (setq edit-indirect--overlay nil) - ...) -#+end_src - -*Add Display Buffer Error Handling* -Check display-buffer return value: -#+begin_src emacs-lisp -(defun edit-indirect--display-buffer (buffer) - (with-current-buffer buffer - (setq-local edit-indirect--should-quit-window t)) - (let ((window (display-buffer buffer))) - (if window - (select-window window) - (switch-to-buffer buffer))) - nil) -#+end_src - -*** Medium Implementation - -*Optimize Overlay Search* -Use a more efficient search strategy: -#+begin_src emacs-lisp -(defun edit-indirect--search-for-edit-indirect (beg end) - (cl-loop for overlay in (overlays-in beg end) - when (overlay-get overlay 'edit-indirect-buffer) - return overlay)) -#+end_src - -*Add Undo Boundary Management* -Add undo boundaries to make commits atomic: -#+begin_src emacs-lisp -(defun edit-indirect--commit () - (undo-boundary) - ;; ... rest of commit logic ... - (undo-boundary)) -#+end_src - -*Improve Buffer Naming* -Make buffer names more informative: -#+begin_src emacs-lisp -(defun edit-indirect--create-indirect-buffer (beg end overlay) - (let* ((mode-name (symbol-name major-mode)) - (buffer-name (format "*edit-indirect %s [%s]*" - (buffer-name) - (replace-regexp-in-string "-mode$" "" mode-name)))) - ...)) -#+end_src - -*** Hard Implementation - -*Add Support for Nested Indirect Edits* -Allow editing a region within an already indirect buffer by tracking parent chain. - -*Implement Diff Preview* -Add a command to show differences between the indirect buffer and parent region before committing: -#+begin_src emacs-lisp -(defun edit-indirect-diff-preview () - "Show differences between indirect buffer and parent region." - (interactive) - ;; Implementation using diff-mode - ) -#+end_src - -*Add Persistent Session Support* -Save and restore indirect editing sessions across Emacs restarts using desktop.el integration. - -*Implement Conflict Resolution* -Handle cases where the parent buffer was modified while editing in the indirect buffer, similar to version control merge conflicts. - -*Add Support for Multiple Regions* -Allow editing multiple non-contiguous regions in a single indirect buffer, useful for refactoring similar code blocks simultaneously. - -** TODO [#C] Review Titlecase Functionality -added in custom. Came from: https://codeberg.org/acdw/titlecase.el -Originally seen at https://emacselements.com/true-titlecase-in-emacs.html -** TODO [#C] Revisit and Refactor Localrepo Functionality -** TODO [#C] Fix Exporting Documents to HTML. -This appears to work with emacs -q. What's up with your config? -** TODO [#C] Rename 'sync-dir' Variable in Init File to 'org-dir' -[[https://github.com/ryuslash/mode-icons][ryuslash/mode-icons: Show icons instead of mode names]] -Captured On: [2025-06-07 Sat 13:29] -** TODO [#C] Emacs Config: Narrowing Org doesn't allow refresh by type -After generating an org agenda list, I can narrow the type to all DONE or IN-PROGRESS. However when I use my custom functions to just get the agenda for the current buffer, I can't. The scope has now widened and the refresh is for all buffers. -** TODO [#C] Add a Restrict to Subtree Org Agenda Command -In Org-mode the “agenda” machinery by default doesn’t pay any attention to your buffernarrow — it always scans whole files in your =org-agenda-files=. However, you can teach Org-agenda to only look between two points (the start and end of your current subtree) by using the built-in “restrict” hooks: -1. =org‐agenda‐restrict= – turn on restriction -2. =org‐agenda‐restrict‐begin= – a marker or position where scanning begins -3. =org‐agenda‐restrict‐end= – where it ends -4. (optionally) set =org-agenda-files= to just your current file -Here is a drop-in =org-agenda-custom-commands= entry that will give you a “Subtree TODOs” view limited to the subtree you’re on. Put this in your Emacs init: -#+begin_src emacs-lisp - ;; ------------------------------------------------------------------- - ;; 1) Define a custom agenda command "C" (hit C-c a C) - ;; that shows only TODO entries in the current subtree. - ;; ------------------------------------------------------------------- - (with-eval-after-load 'org - (add-to-list 'org-agenda-custom-commands - '("C" ; the key you press after C-c a - "Subtree TODOs" ; a descriptive name - ((todo "" ; match any TODO keyword - ;; === per-block settings === - ;; only look in this file - (org-agenda-files (list buffer-file-name)) - ;; enable the restriction engine - (org-agenda-restrict t) - ;; start at the top of the current heading - (org-agenda-restrict-begin - (progn (org-back-to-heading t) (point))) - ;; end at the end of this subtree - (org-agenda-restrict-end - (progn (org-end-of-subtree t) (point)))) - ;; you can add more blocks here if you like - )))) -#+end_src -How this works, step by step: -• We bind a new custom command under the key “C” (so you’ll type =C-c a C=). -• In that command we use the =todo= matcher =""= to catch every headline whose state is one of your TODO keywords. -• We dynamically set -– =org-agenda-files= to just the current buffer’s file, -– =org-agenda-restrict= to non-nil so Org will honor the begin/end markers, -– =org-agenda-restrict-begin= to the position of the current heading, -– =org-agenda-restrict-end= to the end of the subtree. -When you invoke it (=C-c a C=), Org will only scan headlines in that slice of the file and will build you a mini-agenda of TODO items from exactly that subtree. -Quick alternative: if you don’t need the full agenda machinery (dates, deadlines, etc.) but just want a fast outline of your TODOs under the current heading, you can also use the sparse-tree command: -• =C-c / t= ⇒ shows only the TODO headings in the narrowed or whole buffer as an indented outline. -— Tips & Pitfalls — -• Make sure your file is saved and in =org-agenda-files= (the snippet above forces it). -• Remember to call =org-narrow-to-subtree= (or let the command compute its own begin/end via =org-back-to-heading=/=org-end-of-subtree=). -• If you rename or move your file, Emacs must still see =buffer-file-name= valid. -• You can duplicate the above snippet for other TODO states, agenda views, or matchers (deadlines, tags, etc.) -With this in place you effectively get an agenda tailored to exactly the bit of tree you’re working on. -** TODO [#C] Find Another Keymapping for Emojify -** TODO [#C] Emacs: Add Reverso Functionality :enhancement: -https://sqrtminusone.xyz/packages/reverso/ -check his config for a usage example -Reverso -reverso.el is a package of mine that provides Emacs interface for https://reverso.net. -#+begin_src emacs-lisp - (use-package reverso - :straight (:host github :repo "SqrtMinusOne/reverso.el") - :init - (my-leader-def "ar" #'reverso) - :commands (reverso) - :config - (setq reverso-languages '(russian english german spanish french portuguese)) - (reverso-history-mode)) -#+end_src -** TODO [#C] Install Magit TODOs -#+BEGIN_QUOTE -It’s nice for magit to display in-tree TODOs in the status buffer: - -(use-package magit-todos - :init (magit-todos-mode) - :custom - (magit-todos-exclude-globs '(".git/" "*.gz" "*.html" "*.fastq" "*.fq")) - :after (magit)) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:40] -** TODO [#C] Move from company to corfu -#+BEGIN_QUOTE -Completion -A completion framework—I used to use company but corfu seems to integrate with stock Emacs better. - -(use-package corfu - :custom - (corfu-cycle t) - :init - (global-corfu-mode) - :config - (with-eval-after-load 'savehist - (corfu-history-mode 1) - (add-to-list 'savehist-additional-variables 'corfu-history))) -Use more completion at point functions from cape. dabbrev completion means that you can complete from words already present in the buffer, which helps for documents with long words that get used a lot. - -(use-package cape - :init - (dolist (fn (list #'cape-file - (cape-capf-super #'cape-dabbrev #'cape-dict #'org-block-capf #'cape-keyword))) - (add-to-list 'completion-at-point-functions fn)) - :config - (let ((wordlist (getenv "WORDLIST"))) - (setq - cape-dict-file wordlist - ispell-alternate-dictionary wordlist))) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:37] -** TODO [#C] Config recentf to exclude mail -#+BEGIN_QUOTE -recentf provides a persistent list of recently-opened files. - -(use-package recentf - :ensure nil - :init (recentf-mode) - :config - (setq recentf-exclude '("Maildir"))) - -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:36] -** TODO [#C] consider installing minimap -#+BEGIN_QUOTE -A mini-map (miniaturised window showing more of the buffer than the main window) is nice for longer documents: - -(use-package minimap - :custom - (minimap-major-modes '(org-mode prog-mode)) - (minimap-window-location 'right) - :bind ("C-+" . minimap-mode)) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:31] -** TODO [#C] check into org-download for drag/drop images into org-buffers -#+BEGIN_QUOTE -Drag and drop images into Org buffers to insert links: - -(use-package org-download - :custom - (org-download-image-dir "./img")) -#+END_QUOTE -[[https://gitlab.com/jdm204/dotfiles/-/blob/master/config.org][config.org · master · jdm204 / dotfiles · GitLab]] -Captured On: [2025-10-15 Wed 18:30] -** TODO [#D] BrainCurses Emacs Mastermind Game -[[https://sourceforge.net/projects/braincurses/][BrainCurses download | SourceForge.net]] -Captured On: [2025-09-26 Fri 00:38] -** TODO [#D] Emacs: org project should be yellow or green like todo -** TODO [#D] Emacs Signal Client -*** TODO investigate: net/signel.org · master · undefined · GitLab -[[https://gitlab.com/jaor/elibs/-/blob/master/net/signel.org][net/signel.org · master · undefined · GitLab]] -Captured On: [2025-05-29 Thu 04:24] -*** TODO investigate: signel, a barebones signal chat on top of signal-cli -[[https://jao.io/blog/signel.html][signel, a barebones signal chat on top of signal-cli]] -Captured On: [2025-05-29 Thu 04:23] -** TODO [#D] Add test document to test graphviz-dot-mode -** TODO [#D] Consider Replicating Some of This Auctex and PDF Tools Code -[[https://www.reddit.com/r/emacs/comments/cd6fe2/how_to_make_emacs_a_latex_ide/][How to make Emacs a Latex IDE? : r/emacs]] -Captured On: [2025-08-14 Thu 03:43] -** TODO [#D] Install ZOxide Integration into Emacs -[[https://gitlab.com/Vonfry/zoxide.el][Vonfry / zoxide.el · GitLab]] -Captured On: [2025-06-07 Sat 17:11] -** TODO [#D] manateelazycat/awesome-tray: Hide mode-line, display necessary information at right of minibuffer. -[[https://github.com/manateelazycat/awesome-tray][manateelazycat/awesome-tray: Hide mode-line, display necessary information at right of minibuffer.]] -Captured On: [2025-06-07 Sat 13:29] -** TODO [#D] ryuslash/mode-icons: Show icons instead of mode names diff --git a/docs/sessions/create-session.org b/docs/sessions/create-session.org deleted file mode 100644 index a0e4d2fe..00000000 --- a/docs/sessions/create-session.org +++ /dev/null @@ -1,352 +0,0 @@ -#+TITLE: Creating New Session Workflows -#+AUTHOR: Craig Jennings & Claude -#+DATE: 2025-11-01 - -* Overview - -This document describes the meta-workflow for creating new session types. When we identify a repetitive workflow or collaborative pattern, we use this process to formalize it into a documented session that we can reference and reuse. - -Session workflows are living documents that capture how we work together on specific types of tasks. They build our shared vocabulary and enable efficient collaboration across multiple work sessions. - -* Problem We're Solving - -Without a formal session creation process, we encounter several issues: - -** Inefficient Use of Intelligence -- Craig leads the process based solely on his knowledge -- We don't leverage Claude's expertise to improve or validate the approach -- Miss opportunities to apply software engineering and process best practices - -** Time Waste and Repetition -- Craig must re-explain the workflow each time we work together -- No persistent memory of how we've agreed to work -- Each session starts from scratch instead of building on previous work - -** Error-Prone Execution -- Important steps may be forgotten or omitted -- No checklist to verify completeness -- Mistakes lead to incomplete work or failed goals - -** Missed Learning Opportunities -- Don't capture lessons learned from our collaboration -- Can't improve processes based on what works/doesn't work -- Lose insights that emerge during execution - -** Limited Shared Vocabulary -- No deep, documented understanding of what terms mean -- "Let's do a refactor session" has no precise definition -- Can't efficiently communicate about workflows - -*Impact:* Inefficiency, errors, and lost opportunity to continuously improve our collaborative workflows. - -* Exit Criteria - -We know a session definition is complete when: - -1. **Information is logically arranged** - The structure makes sense and flows naturally -2. **Both parties understand how to work together** - We can articulate the workflow -3. **Agreement on effectiveness** - We both agree that following this session will lead to exit criteria and resolve the stated problem -4. **Tasks are clearly defined** - Steps are actionable, not vague -5. **Problem resolution path** - Completing the tasks either: - - Fixes the problem permanently, OR - - Provides a process for keeping the problem at bay - -*Measurable validation:* -- Can we both articulate the workflow without referring to the document? -- Do we agree it will solve the problem? -- Are the tasks actionable enough to start immediately? -- Does the session get used soon after creation (validation by execution)? - -* When to Use This Session - -Trigger this session creation workflow when: - -- You notice a repetitive workflow that keeps coming up -- A collaborative pattern emerges that would benefit from documentation -- Craig says "let's create/define/design a session for [activity]" -- You identify a new type of work that doesn't fit existing session types -- An existing session type needs significant restructuring (treat as creating a new one) - -Examples: -- "Let's create a session where we inbox zero" -- "We should define a code review session" -- "Let's design a session for weekly planning" - -* Approach: How We Work Together - -** Phase 1: Question and Answer Discovery - -Walk through these four core questions collaboratively. Take notes on the answers. - -*IMPORTANT: Save answers as you go!* - -The Q&A phase can take time—Craig may need to think through answers, and discussions can be lengthy. To prevent data loss from terminal crashes or process quits: - -1. Create a draft file at =docs/sessions/[name]-draft.org= after deciding on the name -2. After each question is answered, save the Q&A content to the draft file -3. If session is interrupted, you can resume from the saved answers -4. Once complete, the draft becomes the final session document - -This protects against losing substantial thinking work if the session is interrupted. - -*** Question 1: What problem are we solving in this type of session? - -Ask Craig: "What problem are we solving in this type of session?" - -The answer reveals: -- Overview and goal of the session -- Why this work matters (motivation) -- Impact/priority compared to other work -- What happens if we don't do this work - -Example from refactor session: -#+begin_quote -"My Emacs configuration isn't resilient enough. There's lots of custom code, and I'm even developing some as Emacs packages. Yet Emacs is my most-used software, so when Emacs breaks, I become unproductive. I need to make Emacs more resilient through good unit tests and refactoring." -#+end_quote - -*** Question 2: How do we know when we're done? - -Ask Craig: "How do we know when we're done?" - -The answer reveals: -- Exit criteria -- Results/completion criteria -- Measurable outcomes - -*Your role:* -- Push back if the answer is vague or unmeasurable -- Propose specific measurements based on context -- Iterate together until criteria are clear -- Fallback (hopefully rare): "when Craig says we're done" - -Example from refactor session: -#+begin_quote -"When we've reviewed all methods, decided which to test and refactor, run all tests, and fixed all failures including bugs we find." -#+end_quote - -Claude might add: "How about a code coverage goal of 70%+?" - -*** Question 3: How do you see us working together in this kind of session? - -Ask Craig: "How do you see us working together in this kind of session?" - -The answer reveals: -- Steps or phases we'll go through -- The general approach to the work -- How tasks flow from one to another - -*Your role:* -- As steps emerge, ask yourself: - - "Do these steps lead to solving the real problem?" - - "What is missing from these steps?" -- If the answers aren't "yes" and "nothing", raise concerns -- Propose additions based on your knowledge -- Suggest concrete improvements - -Example from refactor session: -#+begin_quote -"We'll analyze test coverage, categorize functions by testability, write tests systematically using Normal/Boundary/Error categories, run tests, analyze failures, fix bugs, and repeat." -#+end_quote - -Claude might suggest: "Should we install a code coverage tool as part of this process?" - -*** Question 4: Are there any principles we should be following while doing this? - -Ask Craig: "Are there any principles we should be following while doing this kind of session?" - -The answer reveals: -- Principles to follow -- Decision frameworks -- Quality standards -- When to choose option A vs option B - -*Your role:* -- Think through all elements of the session -- Consider situations that may arise -- Identify what principles would guide decisions -- Suggest decision frameworks from your knowledge - -Example from refactor session: -#+begin_quote -Craig: "Treat all test code as production code - same engineering practices apply." - -Claude suggests: "Since we'll refactor methods mixing UI and logic, should we add a principle to separate them for testability?" -#+end_quote - -** Phase 2: Assess Completeness - -After the Q&A, ask together: - -1. **Do we have enough information to formulate steps/process?** - - If yes, proceed to Phase 3 - - If no, identify what's missing and discuss further - -2. **Do we agree following this approach will resolve/mitigate the problem?** - - Both parties must agree - - If not, identify concerns and iterate - -** Phase 3: Name the Session - -Decide on a name for this session type. - -*Naming convention:* Action-oriented (verb form) -- Examples: "refactor", "inbox-zero", "create-session", "review-code" -- Why: Shorter, natural when saying "let's do a [name] session" -- Filename: =docs/sessions/[name].org= - -** Phase 4: Document the Session - -Write the session file at =docs/sessions/[name].org= using this structure: - -*** Recommended Structure -1. *Title and metadata* (=#+TITLE=, =#+AUTHOR=, =#+DATE=) -2. *Overview* - Brief description of the session -3. *Problem We're Solving* - From Q&A, with context and impact -4. *Exit Criteria* - Measurable outcomes, how we know we're done -5. *When to Use This Session* - Triggers, circumstances, examples -6. *Approach: How We Work Together* - - Phases/steps derived from Q&A - - Decision frameworks - - Concrete examples woven throughout -7. *Principles to Follow* - Guidelines from Q&A -8. *Living Document Notice* - Reminder to update with learnings - -*** Important Notes -- Weave concrete examples into sections (don't separate them) -- Use examples from actual sessions when available -- Make tasks actionable, not vague -- Include decision frameworks for common situations -- Note that this is a living document - -** Phase 5: Update Project State - -Update =NOTES.org=: -1. Add new session type to "Available Session Types" section -2. Include brief description and reference to file -3. Note creation date - -Example entry: -#+begin_src org -,** inbox-zero -File: =docs/sessions/inbox-zero.org= - -Workflow for processing inbox to zero: -1. [Brief workflow summary] -2. [Key steps] - -Created: 2025-11-01 -#+end_src - -** Phase 6: Validate by Execution - -*Critical step:* Use the session soon after creating it. - -- Schedule the session type for immediate use -- Follow the documented workflow -- Note what works well -- Identify gaps or unclear areas -- Update the session document with learnings - -*This validates the session definition and ensures it's practical, not theoretical.* - -* Principles to Follow - -These principles guide us while creating new sessions: - -** Collaboration Through Discussion -- Be proactive about collaboration -- Suggest everything on your mind -- Ask all relevant questions -- Push back when something seems wrong, inconsistent, or unclear -- Misunderstandings are learning opportunities - -** Reviewing the Whole as Well as the Pieces -- May get into weeds while identifying each step -- Stop to look at the whole thing at the end -- Ask the big questions: Does this actually solve the problem? -- Verify all pieces connect logically - -** Concrete Over Abstract -- Use examples liberally within explanations -- Weave concrete examples into Q&A answers -- Don't just describe abstractly -- "When nil input crashes, ask..." is better than "handle edge cases" - -** Actionable Tasks Over Vague Direction -- Steps should be clear enough to know what to do next -- "Ask: how do you see us working together?" is actionable -- "Figure out the approach" is too vague -- Test: Could someone execute this without further explanation? - -** Validate Early -- "Use it soon afterward" catches problems early -- Don't let session definitions sit unused and untested -- Real execution reveals gaps that theory misses -- Update immediately based on first use - -** Decision Frameworks Over Rigid Steps -- Sessions are frameworks (principles + flexibility), not recipes -- Include principles that help case-by-case decisions -- "When X happens, ask Y" is a decision framework -- "Always do X" is too rigid for most sessions - -** Question Assumptions -- If something doesn't make sense, speak up -- If a step seems to skip something, point it out -- Better to question during creation than discover gaps during execution -- No assumption is too basic to verify - -* Living Document - -This is a living document. As we create new sessions and learn what works (and what doesn't), we update this file with: - -- New insights about session creation -- Improvements to the Q&A process -- Better examples -- Additional principles discovered -- Refinements to the structure - -Every time we create a session, we have an opportunity to improve this meta-process. - -** Updates and Learnings - -*** 2025-11-01: Save Q&A answers incrementally -*Learning:* During emacs-inbox-zero session creation, we discovered that Q&A discussions can be lengthy and make Craig think deeply. Terminal crashes or process quits can lose substantial work. - -*Improvement:* Added guidance in Phase 1 to create a draft file and save Q&A answers after each question. This protects against data loss and allows resuming interrupted sessions. - -*Impact:* Reduces risk of losing 10-15 minutes of thinking work if session is interrupted. - -*** 2025-11-01: Validation by execution works! -*Learning:* Immediately after creating the emacs-inbox-zero session, we validated it by actually running the workflow. This caught unclear areas and validated that the 10-minute target was realistic. - -*Key insight from validation:* When Craig provides useful context during workflows (impact estimates, theories, examples), that context should be captured in task descriptions. This wasn't obvious during session creation but became clear during execution. - -*Impact:* Validation catches what theory misses. Always use Phase 6 (validate by execution) soon after creating a session. - -* Example: Creating the "Create-Session" Session - -This very document was created using the process it describes (recursive!). - -** The Q&A -- *Problem:* Time waste, errors, missed learning from informal processes -- *Exit criteria:* Logical arrangement, mutual understanding, agreement on effectiveness, actionable tasks -- *Approach:* Four-question Q&A, assess completeness, name it, document it, update NOTES.org, validate by use -- *Principles:* Collaboration through discussion, review the whole, concrete over abstract, actionable tasks, validate early, decision frameworks, question assumptions - -** The Result -We identified what was needed, collaborated on answers, and captured it in this document. Then we immediately used it to create the next session (validation). - -* Conclusion - -Creating session workflows is a meta-skill that improves all our collaboration. By formalizing how we work together, we: - -- Build shared vocabulary -- Eliminate repeated explanations -- Capture lessons learned -- Enable continuous improvement -- Make our partnership more efficient - -Each new session type we create adds to our collaborative toolkit and deepens our ability to work together effectively. - -*Remember:* Sessions are frameworks, not rigid recipes. They provide structure while allowing flexibility for case-by-case decisions. The goal is effectiveness, not perfection. diff --git a/docs/sessions/emacs-inbox-zero.org b/docs/sessions/emacs-inbox-zero.org deleted file mode 100644 index 4e046eba..00000000 --- a/docs/sessions/emacs-inbox-zero.org +++ /dev/null @@ -1,338 +0,0 @@ -#+TITLE: Emacs Inbox Zero Session -#+AUTHOR: Craig Jennings & Claude -#+DATE: 2025-11-01 - -* Overview - -This session processes the Emacs Config Inbox to zero by filtering tasks through the V2MOM framework. Items either move to active V2MOM methods, get moved to someday-maybe, or get deleted. This weekly discipline prevents backlog buildup and ensures only strategic work gets done. - -* Problem We're Solving - -Emacs is Craig's most-used software by a significant margin. It's the platform for email, calendar, task management, note-taking, programming, reading, music, podcasts, and more. When Emacs breaks, everything stops—including critical life tasks like family emails, doctor appointments, and bills. - -The V2MOM (Vision, Values, Methods, Obstacles, Metrics) framework provides strategic balance between fixing/improving Emacs versus using it for real work. But without weekly maintenance, the system collapses under backlog. - -** The Specific Problem - -Features and bugs get logged in the "Emacs Config Inbox" heading of =~/.emacs.d/todo.org=. If not sorted weekly: -- Items pile up and become unmanageable -- Unclear what's actually important -- Method 1 ("Make Using Emacs Frictionless") doesn't progress -- Two key metrics break: - 1. *Active todo count:* Should be < 20 items - 2. *Weekly triage consistency:* Must happen at least once per week by Sunday, no longer than 7 days between sessions - -** What Happens Without This Session - -Without weekly inbox zero: -- Backlog grows until overwhelming -- Can't distinguish signal from noise -- V2MOM becomes theoretical instead of practical -- Config maintenance competes with real work instead of enabling it -- Discipline muscle (Method 6: ruthless prioritization) atrophies - -*Impact:* The entire V2MOM system fails. Config stays broken longer. Real work gets blocked more often. - -* Exit Criteria - -The session is complete when: -- Zero todo items remain under the "* Emacs Config Inbox" heading in =~/.emacs.d/todo.org= -- All items have been routed to: V2MOM methods, someday-maybe, or deleted -- Can verify by checking the org heading (should be empty or show "0/0" in agenda) - -*IMPORTANT:* We are ONLY processing items under the "* Emacs Config Inbox" heading. Items already organized under Method 1-6 headings have already been triaged and should NOT be touched during this session. - -*Measurable validation:* -- Open =todo.org= and navigate to "* Emacs Config Inbox" heading -- Confirm no child tasks exist under this heading only -- Bonus: Check that active todo count is < 20 items across entire V2MOM - -* When to Use This Session - -Trigger this session when: -- It's Sunday and you haven't triaged this week -- 7 days have passed since last triage (hard deadline) -- "Emacs Config Inbox" has accumulated items -- You notice yourself avoiding looking at the inbox (sign it's becoming overwhelming) -- Before starting any new Emacs config work (ensures highest-priority work happens first) - -*Recommended cadence:* Every Sunday, 10 minutes, no exceptions. - -* Approach: How We Work Together - -** Phase 1: Sort by Priority - -First, ensure todo items are sorted by priority in =todo.org=: -- A (highest priority) -- B -- C -- No priority -- D (lowest priority) - -This ensures we always look at the most important items first. If time runs short, at least the high-priority items got processed. - -** Phase 2: Claude Rereads V2MOM - -Before processing any items, Claude rereads [[file:../EMACS-CONFIG-V2MOM.org][EMACS-CONFIG-V2MOM.org]] to have it fresh in mind. This ensures filtering decisions are grounded in the strategic framework. - -*What Claude should pay attention to:* -- The 6 Methods and their concrete actions -- The Values (Intuitive, Fast, Simple) and what they mean -- The Metrics (especially active todo count < 20) -- Method 6 discipline practices (ruthless prioritization, weekly triage, ship-over-research) - -** Phase 3: Process Each Item (in Priority Order) - -*IMPORTANT:* Process ONLY items under the "* Emacs Config Inbox" heading. Items already organized under Method 1-6 have been triaged and should remain where they are. - -For each item under "* Emacs Config Inbox", work through these questions: - -*** Question 1: Does this task need to be done at all? - -*Consider:* -- Has something changed? -- Was this a mistake? -- Do I disagree with this idea now? -- Is this actually important? - -*If NO:* **DELETE** the item immediately. Don't move it anywhere. Kill it. - -*Examples of deletions:* -- "Add Signal client to Emacs" - Cool idea, not important -- "Try minimap mode" - Interesting, doesn't serve vision -- "Research 5 different completion frameworks" - Already have Vertico/Corfu, stop researching - -*** Question 2: Is this task related to the Emacs Config V2MOM? - -*If NO:* **Move to** =docs/someday-maybe.org= - -These are tasks that might be good ideas but don't serve the current strategic focus. They're not deleted (might revisit later) but they're out of active consideration. - -*Examples:* -- LaTeX improvements (no concrete need yet) -- Elfeed dashboard redesign (unclear if actually used) -- New theme experiments (side project competing with maintenance) - -*** Question 3: Which V2MOM method does this relate to? - -*If YES (related to V2MOM):* - -Claude suggests which method(s) this might relate to: -- Method 1: Make Using Emacs Frictionless (performance, bug fixes, missing features) -- Method 2: Stop Problems Before They Appear (package upgrades, deprecation removal) -- Method 3: Make Fixing Emacs Frictionless (tooling, testing, profiling) -- Method 4: Contribute to the Emacs Ecosystem (package maintenance) -- Method 5: Be Kind To Your Future Self (new capabilities) -- Method 6: Develop Disciplined Engineering Practices (meta-practices) - -*This is a conversation.* If the relationship is only tangential: -- **Claude should push back** - "This seems tangential. Adding it would dilute focus and delay V2MOM completion. Are you sure this serves the vision?" -- Help Craig realize it doesn't fit through questions -- The more we add, the longer V2MOM takes, the harder it is to complete - -*If item relates to multiple methods:* -Pick the **highest priority method** (Method 1 > Method 2 > Method 3 > etc.) - -*IMPORTANT: Capture useful context!* -During discussion, Craig may provide: -- Impact estimates ("15-20 seconds × 12 times/day") -- Theories about root causes -- Context about why this matters -- Examples of when the problem occurs - -**When moving items to methods, add this context to the task description.** This preserves valuable information for later execution and helps prioritize work accurately. - -*Then:* Move the item to the appropriate method section in the V2MOM or active todo list with enriched context. - -** Phase 4: Verify and Celebrate - -Once all items are processed: -1. Verify "Emacs Config Inbox" heading is empty -2. Check that active todo count is < 20 items -3. Note the date of this triage session -4. Acknowledge: You've practiced ruthless prioritization (Method 6 skill development) - -** Decision Framework: When Uncertain - -If you're uncertain whether an item fits V2MOM: - -1. **Ask: Does this directly serve the Vision?** (Work at speed of thought, stable config, comprehensive workflows) -2. **Ask: Does this align with Values?** (Intuitive, Fast, Simple) -3. **Ask: Is this in the Methods already?** (If not explicitly listed, probably shouldn't add) -4. **Ask: What's the opportunity cost?** (Every new item delays everything else) - -*When in doubt:* Move to someday-maybe. You can always pull it back later if it proves critical. Better to be conservative than to dilute focus. - -* Principles to Follow - -** Claude's Role: "You're here to help keep me honest" - -Craig is developing discipline (Method 6: ruthless prioritization). Not making progress = not getting better. - -*Claude's responsibilities:* -- If task clearly fits V2MOM → Confirm and move forward quickly -- If task is unclear/tangential → **Ask questions** to help Craig realize it doesn't fit or won't lead to V2MOM success -- Enable ruthless prioritization by helping Craig say "no" -- Don't let good ideas distract from great goals - -*Example questions Claude might ask:* -- "This is interesting, but which specific metric does it improve?" -- "We already have 3 items in Method 1 addressing performance. Does this add something different?" -- "This would be fun to build, but does it make using Emacs more frictionless?" -- "If you had to choose between this and fixing org-agenda (30s → 5s), which serves the vision better?" - -** Time Efficiency: 10 Minutes Active Work - -Don't take too long on any single item. Splitting philosophical hairs = procrastination. - -*Target:* **10 minutes active work time** (not clock time - interruptions expected) - -*If spending > 1 minute on a single item:* -- Decision is unclear → Move to someday-maybe (safe default) -- Come back to it later if it proves critical -- Keep moving - -*Why this matters:* -- Weekly consistency requires low friction -- Perfect categorization doesn't matter as much as consistent practice -- Getting through all items > perfectly routing each item - -** Ruthless Prioritization Over Completeness - -The goal is not to do everything in the inbox. The goal is to identify and focus on what matters most. - -*Better to:* -- Delete 50% of items and ship the other 50% -- Than keep 100% and ship 0% - -*Remember:* -- Every item kept is opportunity cost -- V2MOM already has plenty of work -- "There will always be cool ideas out there to implement and they will always be a web search away" (Craig's words) - -** Bias Toward Action - -When processing items that ARE aligned with V2MOM: -- Move them to the appropriate method quickly -- Don't overthink the categorization -- Getting it 80% right is better than spending 5 minutes getting it 100% right -- You can always recategorize later during regular triage - -* Living Document - -This is a living document. After each emacs-inbox-zero session, consider: -- Did the workflow make sense? -- Were any steps unclear or unnecessary? -- Did any new situations arise that need decision frameworks? -- Did the 10-minute target work, or should it adjust? - -Update this document with learnings to make future sessions smoother. - -* Example Session Walkthrough - -** Setup -- Open =~/.emacs.d/todo.org= -- Navigate to "Emacs Config Inbox" heading -- Verify items are sorted by priority (A → B → C → none → D) -- Claude rereads =EMACS-CONFIG-V2MOM.org= - -** Processing Example Items - -*** Example 1: [#A] Fix org-agenda slowness (30+ seconds) - -*Q1: Does this need to be done?* YES - Daily pain point blocking productivity - -*Q2: Related to V2MOM?* YES - Method 1 explicitly lists this - -*Q3: Which method?* Method 1: Make Using Emacs Frictionless - -*Action:* Move to Method 1 active tasks (or confirm already there) - -*Time:* 15 seconds - -*** Example 2: [#B] Add Signal client to Emacs - -*Q1: Does this need to be done?* Let's think... - -Claude: "What problem does this solve? Is messaging in Emacs part of the Vision?" - -Craig: "Not really, I already use Signal on my phone fine." - -*Action:* **DELETE** - Doesn't serve vision, already have working solution - -*Time:* 30 seconds - -*** Example 3: [#C] Try out minimap mode for code navigation - -*Q1: Does this need to be done?* Interesting idea, but not important - -*Action:* **DELETE** or move to someday-maybe - Interesting, not important - -*Time:* 10 seconds - -*** Example 4: [#B] Implement transcription workflow - -*Q1: Does this need to be done?* YES - Want to transcribe recordings for notes - -*Q2: Related to V2MOM?* Maybe... seems like new feature? - -Claude: "This seems like Method 5: Be Kind To Your Future Self - new capability you'll use repeatedly. Complete code already exists in old todo.org. But we're still working through Method 1 (frictionless) and Method 2 (stability). Should this wait, or is transcription critical?" - -Craig: "Actually yes, I record meetings and need transcripts. This is important." - -*Q3: Which method?* Method 5: Be Kind To Your Future Self - -*Action:* Move to Method 5 (but note: prioritize after Methods 1-3) - -*Time:* 45 seconds (good conversation, worth the time) - -** Result -- 4 items processed in ~2 minutes -- 1 moved to Method 1 (already there) -- 1 deleted -- 1 deleted or moved to someday-maybe -- 1 moved to Method 5 -- Inbox is clearer, focus is sharper - -* Conclusion - -Emacs inbox zero is not about getting through email or org-capture. It's about **strategic filtering of config maintenance work**. By processing the inbox weekly, you: - -- Keep maintenance load manageable (< 20 active items) -- Ensure only V2MOM-aligned work happens -- Practice ruthless prioritization (Method 6 skill) -- Prevent backlog from crushing future productivity -- Build the discipline that makes all other methods sustainable - -**The session takes 10 minutes. Not doing it costs days of distracted, unfocused work on things that don't matter.** - -*Remember:* Inbox zero is not about having zero things to do. It's about knowing exactly what you're NOT doing, so you can focus completely on what matters most. - -* Living Document - -This is a living document. After each emacs-inbox-zero session, consider: -- Did the workflow make sense? -- Were any steps unclear or unnecessary? -- Did any new situations arise that need decision frameworks? -- Did the 10-minute target work, or should it adjust? - -Update this document with learnings to make future sessions smoother. - -** Updates and Learnings - -*** 2025-11-01: First validation session - Process works! - -*Session results:* -- 5 items processed in ~10 minutes (target met) -- 1 deleted (duplicate), 2 moved to Method 1, 2 moved to someday-maybe -- Inbox cleared to zero -- Priority sorting worked well -- Three-question filter was effective -- Caught duplicate task and perfectionism pattern in real-time - -*Key learning: Capture useful context during triage* -When Craig provides impact estimates ("15-20 seconds × 12 times/day"), theories, or context during discussion, **Claude should add this information to the task description** when moving items to methods. This preserves valuable context for execution and helps with accurate prioritization. - -Example: "Optimize org-capture target building" was enriched with "15-20 seconds every time capturing a task (12+ times/day). Major daily bottleneck - minutes lost waiting, plus context switching cost." - -*Impact:* Better task descriptions → better prioritization → better execution. diff --git a/docs/sessions/refactor.org b/docs/sessions/refactor.org deleted file mode 100644 index 0cdb6841..00000000 --- a/docs/sessions/refactor.org +++ /dev/null @@ -1,617 +0,0 @@ -#+TITLE: Test-Driven Quality Engineering Session Process -#+AUTHOR: Craig Jennings & Claude -#+DATE: 2025-11-01 - -* Overview - -This document describes a comprehensive test-driven quality engineering session process applicable to any source code module. The session demonstrates systematic testing practices, refactoring for testability, bug discovery through tests, and decision-making processes when tests fail. - -* Session Goals - -1. Add comprehensive unit test coverage for testable functions in your module -2. Discover and fix bugs through systematic testing -3. Follow quality engineering principles from =ai-prompts/quality-engineer.org= -4. Demonstrate refactoring patterns for testability -5. Document the decision-making process for test vs production code issues - -* Phase 1: Feature Addition with Testability in Mind - -** The Feature Request - -Add new functionality that requires user interaction combined with business logic. - -Example requirements: -- Present user with options (e.g., interactive selection) -- Allow cancellation -- Perform an operation with the selected input -- Provide clear success/failure feedback - -** Refactoring for Testability - -Following the "Interactive vs Non-Interactive Function Pattern" from =quality-engineer.org=: - -*Problem:* Directly implementing as an interactive function would require: -- Mocking user interface components -- Mocking framework-specific APIs -- Testing UI functionality, not core business logic - -*Solution:* Split into two functions: - -1. *Helper Function* (internal implementation): - - Pure, deterministic - - Takes explicit parameters - - No user interaction - - Returns values or signals errors naturally - - 100% testable, no mocking needed - -2. *Interactive Wrapper* (public interface): - - Thin layer handling only user interaction - - Gets input from user/context - - Presents UI (prompts, selections, etc.) - - Catches errors and displays messages - - Delegates all business logic to helper - - No tests needed (just testing framework UI) - -** Benefits of This Pattern - -From =quality-engineer.org=: -#+begin_quote -When writing functions that combine business logic with user interaction: -- Split into internal implementation and interactive wrapper -- Internal function: Pure logic, takes all parameters explicitly -- Dramatically simpler testing (no interactive mocking) -- Code reusable programmatically without prompts -- Clear separation of concerns (logic vs UI) -#+end_quote - -This pattern enables: -- Zero mocking in tests -- Fast, deterministic tests -- Easy reasoning about correctness -- Reusable helper function - -* Phase 2: Writing the First Test - -** Test File Naming - -Following the naming convention from =quality-engineer.org=: -- Pattern: =test-<module>-<function>.<ext>= -- One test file per function for easy discovery when tests fail -- Developer sees failure → immediately knows which file to open - -** Test Organization - -Following the three-category structure: - -*** Normal Cases -- Standard expected inputs -- Common use case scenarios -- Happy path operations -- Multiple operations in sequence - -*** Boundary Cases -- Very long inputs -- Unicode characters (中文, emoji) -- Special characters and edge cases -- Empty or minimal data -- Maximum values - -*** Error Cases -- Invalid inputs -- Nonexistent resources -- Permission denied scenarios -- Wrong type of input - -** Writing Tests with Zero Mocking - -Key principle: "Don't mock what you're testing" (from =quality-engineer.org=) - -Example test structure: -#+begin_src -test_function_normal_case_expected_result() - setup() - try: - # Arrange - input_data = create_test_data() - expected_output = define_expected_result() - - # Act - actual_output = function_under_test(input_data) - - # Assert - assert actual_output == expected_output - finally: - teardown() -#+end_src - -Key characteristics: -- No mocks for the function being tested -- Real resources (files, data structures) using test utilities -- Tests actual function behavior -- Clean setup/teardown -- Clear arrange-act-assert structure - -** Result - -When helper functions are well-factored and deterministic, tests often pass on first run. - -* Phase 3: Systematic Test Coverage Analysis - -** Identifying Testable Functions - -Review all functions in your module and categorize by testability: - -*** Easy to Test (Pure/Deterministic) -- Input validation functions -- String manipulation/formatting -- Data structure transformations -- File parsing (read-only operations) -- Configuration/option processing - -*** Medium Complexity (Need External Resources) -- File I/O operations -- Recursive algorithms -- Data structure generation -- Cache or state management - -*** Hard to Test (Framework/Context Dependencies) -- Functions requiring specific runtime environment -- UI/buffer/window management -- Functions tightly coupled to framework internals -- Functions requiring complex mocking setup - -*Decision:* Test easy and medium complexity functions. Skip framework-dependent functions that would require extensive mocking/setup (diminishing returns). - -** File Organization Principle - -From =quality-engineer.org=: -#+begin_quote -*Unit Tests*: One file per method -- Naming: =test-<filename>-<methodname>.<ext>= -- Example: =test-module--function.ext= -#+end_quote - -*Rationale:* When a test fails in CI: -1. Developer sees: =test-module--function-normal-case-returns-result FAILED= -2. Immediately knows: Look for =test-module--function.<ext>= -3. Opens file and fixes issue - *fast cognitive path* - -If combined files: -1. Test fails: =test-module--function-normal-case-returns-result FAILED= -2. Which file? =test-module--helpers.<ext>=? =test-module--combined.<ext>=? -3. Developer wastes time searching - *slower, frustrating* - -*The 1:1 mapping is a usability feature for developers under pressure.* - -* Phase 4: Testing Function by Function - -** Example 1: Input Validation Function - -*** Test Categories - -*Normal Cases:* -- Valid inputs -- Case variations -- Common use cases - -*Boundary Cases:* -- Edge cases in input format -- Multiple delimiters or separators -- Empty or minimal input -- Very long input - -*Error Cases:* -- Nil/null input -- Wrong type -- Malformed input - -*** First Run: Most Passed, Some FAILED - -*Example Failure:* -#+begin_src -test-module--validate-input-error-nil-input-returns-nil -Expected: Returns nil gracefully -Actual: (TypeError/NullPointerException) - CRASHED -#+end_src - -*** Bug Analysis: Test or Production Code? - -*Process:* -1. Read the test expectation: "nil input returns nil/false gracefully" -2. Read the production code: - #+begin_src - function validate_input(input): - extension = get_extension(input) # ← Crashes here on nil/null - return extension in valid_extensions - #+end_src -3. Identify issue: Function expects string, crashes on nil/null -4. Consider context: This is defensive validation code, called in various contexts - -*Decision: Fix production code* - -*Rationale:* -- Function should be defensive (validation code) -- Returning false/nil for invalid input is more robust than crashing -- Common pattern in validation functions -- Better user experience - -*Fix:* -#+begin_src -function validate_input(input): - if input is None or not isinstance(input, str): # ← Guard added - return False - extension = get_extension(input) - return extension in valid_extensions -#+end_src - -Result: All tests pass after adding defensive checks. - -** Example 2: Another Validation Function - -*** First Run: Most Passed, Multiple FAILED - -*Failures:* -1. Nil input crashed (same pattern as previous function) -2. Empty string returned unexpected value (edge case not handled) - -*Fix:* -#+begin_src -function validate_resource(resource): - # Guards added for nil/null and empty string - if not resource or not isinstance(resource, str) or resource.strip() == "": - return False - - # Original validation logic - return is_valid_resource(resource) and meets_criteria(resource) -#+end_src - -Result: All tests pass after adding comprehensive guards. - -** Example 3: String Sanitization Function - -*** First Run: Most Passed, 1 FAILED - -*Failure:* -#+begin_src -test-module--sanitize-boundary-special-chars-replaced -Expected: "output__________" (10 underscores) -Actual: "output_________" (9 underscores) -#+end_src - -*** Bug Analysis: Test or Production Code? - -*Process:* -1. Count special chars in test input: 9 characters -2. Test expected 10 replacements, but input only has 9 -3. Production code is working correctly - -*Decision: Fix test code* - -*The bug was in the test expectation, not the implementation.* - -Result: All tests pass after correcting test expectations. - -** Example 4: File/Data Parser Function - -This is where a **significant bug** was discovered through testing! - -*** Test Categories - -*Normal Cases:* -- Absolute paths/references -- Relative paths (expanded to base directory) -- URLs/URIs preserved as-is -- Mixed types of references - -*Boundary Cases:* -- Empty lines ignored -- Whitespace-only lines ignored -- Comments ignored (format-specific) -- Leading/trailing whitespace trimmed -- Order preserved - -*Error Cases:* -- Nonexistent file -- Nil/null input - -*** First Run: Majority Passed, Multiple FAILED - -All failures related to URL/URI handling: - -*Failure Pattern:* -#+begin_src -Expected: "http://example.com/resource" -Actual: "/base/path/http:/example.com/resource" -#+end_src - -URLs were being treated as relative paths and corrupted! - -*** Root Cause Analysis - -*Production code:* -#+begin_src -if line.matches("^\(https?|mms\)://"): # Pattern detection - # Handle as URL -#+end_src - -*Problem:* Pattern matching is incorrect! - -The pattern/regex has an error: -- Incorrect escaping or syntax -- Pattern fails to match valid URLs -- All URLs fall through to the "relative path" handler - -The pattern never matched, so URLs were incorrectly processed as relative paths. - -*Correct version:* -#+begin_src -if line.matches("^(https?|mms)://"): # Fixed pattern - # Handle as URL -#+end_src - -Common causes of this type of bug: -- String escaping issues in the language -- Incorrect regex syntax -- Copy-paste errors in patterns - -*** Impact Assessment - -*This is a significant bug:* -- Remote resources (URLs) would be broken -- Data corruption: URLs transformed into invalid paths -- Function worked for local/simple cases, so bug went unnoticed -- Users would see mysterious errors when using remote resources -- Potential data loss or corruption in production - -*Tests caught a real production bug that could have caused user data corruption!* - -Result: All tests pass after fixing the pattern matching logic. - -* Phase 5: Continuing Through the Test Suite - -** Additional Functions Tested Successfully - -As testing continues through the module, patterns emerge: - -*Function: Directory/File Listing* - - Learning: Directory listing order may be filesystem-dependent - - Solution: Sort results before comparing in tests - -*Function: Data Extraction* - - Keep as separate test file (don't combine with related functions) - - Reason: Usability when tests fail - -*Function: Recursive Operations* - - Medium complexity: Required creating test data structures/trees - - Use test utilities for setup/teardown - - Well-factored functions often pass all tests initially - -*Function: Higher-Order Functions* - - Test functions that return functions/callbacks - - Initially may misunderstand framework/protocol behavior - - Fix test expectations to match actual framework behavior - -* Key Principles Applied - -** 1. Refactor for Testability BEFORE Writing Tests - -The Interactive vs Non-Interactive pattern from =quality-engineer.org= made testing trivial: -- No mocking required -- Fast, deterministic tests -- Clear separation of concerns - -** 2. Systematic Test Organization - -Every test file followed the same structure: -- Normal Cases -- Boundary Cases -- Error Cases - -This makes it easy to: -- Identify coverage gaps -- Add new tests -- Understand what's being tested - -** 3. Test Naming Convention - -Pattern: =test-<module>-<function>-<category>-<scenario>-<expected-result>= - -Examples: -- =test-module--validate-input-normal-valid-extension-returns-true= -- =test-module--parse-data-boundary-empty-lines-ignored= -- =test-module--sanitize-error-nil-input-signals-error= - -Benefits: -- Self-documenting -- Easy to understand what failed -- Searchable/grepable -- Clear category organization - -** 4. Zero Mocking for Pure Functions - -From =quality-engineer.org=: -#+begin_quote -DON'T MOCK WHAT YOU'RE TESTING -- Only mock external side-effects and dependencies, not the domain logic itself -- If mocking removes the actual work the function performs, you're testing the mock -- Use real data structures that the function is designed to operate on -- Rule of thumb: If the function body could be =(error "not implemented")= and tests still pass, you've over-mocked -#+end_quote - -Our tests used: -- Real file I/O -- Real strings -- Real data structures -- Actual function behavior - -Result: Tests caught real bugs, not mock configuration issues. - -** 5. Test vs Production Code Bug Decision Framework - -When a test fails, ask: - -1. *What is the test expecting?* - - Read the test name and assertions - - Understand the intended behavior - -2. *What is the production code doing?* - - Read the implementation - - Trace through the logic - -3. *Which is correct?* - - Is the test expectation reasonable? - - Is the production behavior defensive/robust? - - What is the usage context? - -4. *Consider the impact:* - - Defensive code: Fix production to handle edge cases - - Wrong expectation: Fix test - - Unclear spec: Ask user for clarification - -Examples from our session: -- *Nil input crashes* → Fix production (defensive coding) -- *Empty string treated as valid* → Fix production (defensive coding) -- *Wrong count in test* → Fix test (test bug) -- *Regex escaping wrong* → Fix production (real bug!) - -** 6. Fast Feedback Loop - -Pattern: "Write tests, run them all, report errors, and see where we are!" - -This became a mantra during the session: -1. Write comprehensive tests for one function -2. Run immediately -3. Analyze failures -4. Fix bugs (test or production) -5. Verify all tests pass -6. Move to next function - -Benefits: -- Caught bugs immediately -- Small iteration cycles -- Clear progress -- High confidence in changes - -* Final Results - -** Test Coverage Example - -*Multiple functions tested with comprehensive coverage:* -1. File operation helper - ~10-15 tests -2. Input validation function - ~15 tests -3. Resource validation function - ~13 tests -4. String sanitization function - ~13 tests -5. File/data parser function - ~15 tests -6. Directory listing function - ~7 tests -7. Data extraction function - ~6 tests -8. Recursive operation function - ~12 tests -9. Higher-order function - ~12 tests - -Total: Comprehensive test suite covering all testable functions - -** Bugs Discovered and Fixed - -1. *Input Validation Function* - - Issue: Crashed on nil/null input - - Fix: Added nil/type guards - - Impact: Prevents crashes in validation code - -2. *Resource Validation Function* - - Issue: Crashed on nil, treated empty string as valid - - Fix: Added guards for nil and empty string - - Impact: More robust validation - -3. *File/Data Parser Function* ⚠️ *SIGNIFICANT BUG* - - Issue: Pattern matching wrong - URLs/URIs corrupted as relative paths - - Fix: Corrected pattern matching logic - - Impact: Remote resources now work correctly - - *This bug would have corrupted user data in production* - -** Code Quality Improvements - -- All testable helper functions now have comprehensive test coverage -- More defensive error handling (nil guards) -- Clear separation of concerns (pure helpers vs interactive wrappers) -- Systematic boundary condition testing -- Unicode and special character handling verified - -* Lessons Learned - -** 1. Tests as Bug Discovery Tools - -Tests aren't just for preventing regressions - they actively *discover existing bugs*: -- Pattern matching bugs may exist in production -- Nil/null handling bugs manifest in edge cases -- Tests make these issues visible immediately -- Bugs caught before users encounter them - -** 2. Refactoring Enables Testing - -The decision to split functions into pure helpers + interactive wrappers: -- Made testing dramatically simpler -- Enabled 100+ tests with zero mocking -- Improved code reusability -- Clarified function responsibilities - -** 3. Systematic Process Matters - -Following the same pattern for each function: -- Reduced cognitive load -- Made it easy to maintain consistency -- Enabled quick iteration -- Built confidence in coverage - -** 4. File Organization Aids Debugging - -One test file per function: -- Fast discovery when tests fail -- Clear ownership -- Easy to maintain -- Follows user's mental model - -** 5. Test Quality Equals Production Quality - -Quality tests: -- Use real resources (not mocks) -- Test actual behavior -- Cover edge cases systematically -- Find real bugs - -This is only possible with well-factored, testable code. - -* Applying These Principles - -When adding tests to other modules: - -1. *Identify testable functions* - Look for pure helpers, file I/O, string manipulation -2. *Refactor if needed* - Split interactive functions into pure helpers -3. *Write systematically* - Normal, Boundary, Error categories -4. *Run frequently* - Fast feedback loop -5. *Analyze failures carefully* - Test bug vs production bug -6. *Fix immediately* - Don't accumulate technical debt -7. *Maintain organization* - One file per function, clear naming - -* Reference - -See =ai-prompts/quality-engineer.org= for comprehensive quality engineering guidelines, including: -- Test organization and structure -- Test naming conventions -- Mocking and stubbing best practices -- Interactive vs non-interactive function patterns -- Integration testing guidelines -- Test maintenance strategies - -Note: =quality-engineer.org= evolves as we learn more quality best practices. This document captures principles applied during this specific session. - -* Conclusion - -This session process demonstrates how systematic testing combined with refactoring for testability can: -- Discover real bugs before they reach users -- Improve code quality and robustness -- Build confidence in changes -- Create maintainable test suites -- Follow industry best practices - -A comprehensive test suite with multiple bug fixes represents significant quality improvement to any module. Critical bugs (like the pattern matching issue in the example) alone can justify the entire testing effort - such bugs can cause data corruption and break major features. - -*Testing is not just about preventing future bugs - it's about finding bugs that already exist.* diff --git a/docs/values-comparison.org b/docs/values-comparison.org deleted file mode 100644 index 128c6710..00000000 --- a/docs/values-comparison.org +++ /dev/null @@ -1,107 +0,0 @@ -#+TITLE: Values Comparison - Simple vs Reliability -#+DATE: 2025-10-30 - -* What Currently Supports Each Value - -| Theme/Practice | Simple | Reliability | -|---------------------------------------+--------+-------------| -| *Code Quality* | | | -| Clear naming | ✓ | | -| Small functions | ✓ | | -| Good comments (why not what) | ✓ | | -| Abstractions when warranted | ✓ | | -|---------------------------------------+--------+-------------| -| *Architecture* | | | -| Modular design | ✓ | | -| Extract custom code into packages | ✓ | | -| Good architecture enables evolution | ✓ | | -| Technical debt has real cost | ✓ | | -|---------------------------------------+--------+-------------| -| *Production Practices* | | | -| Testing | ✓ | ✓ | -| Documentation | ✓ | | -| Version control | ✓ | | -|---------------------------------------+--------+-------------| -| *Failure Planning* | | | -| Rollback capability (localrepo) | ✓ | ✓ | -| Error handling | ✓ | ✓ | -| Graceful degradation | ✓ | ✓ | -|---------------------------------------+--------+-------------| -| *Trust & Stability* | | | -| Never use tools you can't trust | ✓ | ✓ | -| Uptime matters | | ✓ | -| Config breakage crushes productivity | | ✓ | -| More packages = more breakage risk | | ✓ | -| Need confidence to refactor | | ✓ | -|---------------------------------------+--------+-------------| - -* Key Quotes Supporting Each - -** Simple -- "Config IS production software - treat it as such" -- "Clear naming, small functions, good comments" -- "Apply production practices: Testing, modularity, documentation" -- "Simple means maintainable production software, not minimal or primitive" - -** Reliability/Resilience -- "Never use a tool you can't fully trust" -- "The impact when it's broken can crush my productivity" -- "More packages = more potential breakage. Need ability to roll back" -- "Plan for failure: Rollback capability, error handling, graceful degradation" -- "Quality matters because uptime matters" - -* Analysis - -** If Reliability is part of Simple: -*Simple means:* Maintainable, trustworthy, production-quality code that fails gracefully - -*Pros:* -- Keeps to 3 values -- Failure planning is a consequence of good engineering (already in Simple) -- Less redundancy - -*Cons:* -- "Simple" might not clearly convey the "must not break" urgency -- Rollback/resilience themes could get lost - -** If Reliability is its own value: -*Simple means:* Clear, maintainable code -*Reliability means:* Never breaks, fails gracefully, always recoverable - -*Pros:* -- Explicit focus on uptime/stability -- Clear that this is mission-critical (crushes productivity when broken) -- Resilience practices (localrepo, testing, error handling) have clear home - -*Cons:* -- 4 values instead of 3 -- Some overlap with Simple - -* Decision Framework - -Ask yourself: - -1. When choosing between two approaches, would "reliability" change your decision differently than "simple"? - - Example: Complex caching solution that's faster but harder to debug - - Simple says: "Too complex, harder to maintain" - - Reliability says: "If it breaks less often and has rollback, worth the complexity" - -2. Does "config can't break" deserve equal weight with Fast and Intuitive? - -3. Is the "trust" theme strong enough to be its own value? - -* Recommendation - -If you had to choose, I'd suggest: **Keep Simple, absorb Reliability into it.** - -Why? Most of your reliability practices (testing, rollback, error handling) are already listed under "production software practices" in Simple. The "never use tools you can't trust" is already there too. - -**But** - you could rename Simple to "**Reliable**" or "**Solid**" if the stability theme is more important than the simplicity theme. - -Alternative value names that might capture both: -- *Solid* - Stable, maintainable, trustworthy -- *Resilient* - Fails gracefully, recovers easily, maintainable -- *Dependable* - Simple enough to trust, robust enough to rely on - -What resonates? diff --git a/early-init.el b/early-init.el index 6fa3e0b2..c1cd54c8 100644 --- a/early-init.el +++ b/early-init.el @@ -51,10 +51,10 @@ (setq debug-on-error t) ;; default nil. turn on to debug issues only. (setq debug-on-quit t) ;; debug on C-g (breaking out of hangs/freezes) -;; (add-hook 'emacs-startup-hook -;; (lambda () -;; (setq debug-on-error nil) -;; (setq debug-on-quit nil))) +(add-hook 'emacs-startup-hook + (lambda () + (setq debug-on-error nil) + (setq debug-on-quit nil))) ;; ------------------------------ Bug Workarounds ------------------------------ @@ -84,6 +84,7 @@ (require 'pdf-config) ;; pdf display settings (require 'quick-video-capture) ;; download videos with a browser bookmark (require 'video-audio-recording) ;; desktop and/or audio recording via ffmpeg +(require 'transcription-config) ;; audio transcription using Whisper (require 'weather-config) ;; utility to display the weather ;; -------------------------------- Programming -------------------------------- diff --git a/issues.org b/issues.org deleted file mode 100644 index fa9bc74a..00000000 --- a/issues.org +++ /dev/null @@ -1,1881 +0,0 @@ -#+TITLE: Emacs Configuration Issues & Enhancement Roadmap -#+AUTHOR: Claude Code Analysis -#+DATE: 2025-10-30 - -* New Open Work - -** TODO [#A] Duplicate variable declaration in early-init.el :bug:architecture:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 76-78, 104-105 -:END: - -The variable =cj/network-available= is declared twice (lines 76-78 and 104-105). - -*Impact:* Confusing code structure, potential initialization issues. - -*Fix:* Remove the duplicate declaration at lines 104-105. The first declaration at line 76 is sufficient. - -** TODO [#A] Missing function definition in vc-config.el :bug:critical: -:PROPERTIES: -:FILE: modules/vc-config.el -:LINE: 123 -:END: - -The keymap binding references =cj/goto-git-gutter-diff-hunks= (line 123) but this function is never defined anywhere in the config. - -*Impact:* Keybinding =C-; v d= will fail with "Symbol's function definition is void" error. - -*Fix:* Either implement the missing function or remove the keybinding. Likely should use =git-gutter:popup-hunk= or create a wrapper that calls =consult-line= filtered to lines with git-gutter markers. - -** TODO [#A] Missing function in wrap-up.el :bug:critical: -:PROPERTIES: -:FILE: modules/wrap-up.el -:LINE: 28 -:END: - -Calls =cj/log-silently= but function is never defined in the config. - -*Impact:* Init will fail at startup with undefined function error. - -*Fix:* Define =cj/log-silently= in system-defaults or config-utilities, or replace with a simple =(message ...)= call. - -** TODO [#A] Network check runs synchronously during startup :performance:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 189, 206 -:END: - -The =cj/internet-up-p= function is called *synchronously* during early-init (lines 189, 206), which means Emacs startup blocks waiting for ping or network connection attempts. - -*Impact:* Adds 1+ seconds to every startup, even when network is available. Completely unacceptable UX. - -*Fix:* -1. Default =cj/network-available= to =t= (assume network is available) -2. Run network check asynchronously in background after startup -3. Only use network check when user explicitly runs =package-refresh-contents= -4. Consider removing network check entirely - let package.el handle its own errors - -Modern approach: Remove =cj/use-online-repos= flag entirely. Let package operations fail gracefully if network is down, rather than preemptively checking. - -** TODO [#A] Commented-out non-commented variable in org-config.el :bug:functionality: -:PROPERTIES: -:FILE: modules/org-config.el -:LINE: 129-130 -:END: - -Line 129 has a TASK comment about removing a non-existent variable =org-indent-indentation-per-level=, but line 130 actually tries to set it. - -*Impact:* Setting an undefined variable that does nothing. - -*Fix:* Remove lines 129-130 entirely if the variable doesn't exist. Verify in org-mode source whether this was renamed or removed. - -** TODO [#B] Replace Company with Corfu for better performance and maintenance :enhancement:architecture: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINES: 202-236 -:END: - -Currently using Company for completion, but Corfu is the more modern choice: - -*Why Corfu is better:* -- ~3x faster than Company (uses child frames vs overlays) -- Better maintained (company-mode development has slowed) -- Smaller, simpler codebase -- Works seamlessly with modern completion frameworks (Vertico/Consult) -- Native corfu-popupinfo is cleaner than company-quickhelp -- Better integration with Cape for completion-at-point backends - -*Migration path:* -1. Replace =company= with =corfu= + =corfu-popupinfo= -2. Use =cape= for additional backends (cape-file, cape-keyword, cape-dabbrev) -3. Keep company-mode disabled in mu4e (corfu already respects this) -4. Much simpler configuration, no company-box needed - -*Estimated effort:* 30 minutes. High value/low effort. - -** TODO [#B] Consolidate debug logging functions :architecture:maintainability: -:PROPERTIES: -:FILES: modules/org-agenda-config-debug.el, modules/wrap-up.el -:END: - -Debug logging is inconsistent: -- =org-agenda-config-debug.el= defines =cj/log-silently= -- =wrap-up.el= *calls* =cj/log-silently= but it's not loaded yet -- =config-utilities.el= has debug functions but not logging - -*Fix:* Create =modules/debug-logging.el= with consistent logging infrastructure: -- =cj/log-debug=, =cj/log-info=, =cj/log-warning=, =cj/log-error= -- Buffer-based logging with timestamps -- Easy toggle for debug mode -- Load early in init.el before other modules - -** TODO [#B] Missing error handling in org-agenda rebuild :stability: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:LINES: 91-105 -:END: - -=cj/build-org-agenda-list= recursively searches directories but has no error handling for: -- Permission denied errors -- Circular symlinks -- Very deep directory trees -- Network filesystem timeouts (if projects-dir is on NFS/CIFS) - -*Impact:* Agenda rebuild can hang or fail silently. - -*Fix:* Add condition-case wrapper with specific handlers: -#+begin_src elisp -(condition-case err - (cj/add-files-to-org-agenda-files-list projects-dir) - (file-error - (message "Warning: Could not scan %s: %s" projects-dir err)) - (error - (message "Error building agenda: %s" err))) -#+end_src - -** TODO [#B] Unify path handling functions :architecture:maintainability: -:PROPERTIES: -:FILES: modules/dirvish-config.el, modules/custom-buffer-file.el -:END: - -Path handling is duplicated across modules: -- =cj/get-project-root= in dirvish-config.el (lines 408-426) -- Similar logic needed in custom-buffer-file for relative paths -- Different approaches to home directory abbreviation - -*Fix:* Create =modules/path-utils.el= with: -- =cj/get-project-root= (shared implementation) -- =cj/abbreviate-path= (handles ~, project-relative, absolute) -- =cj/path-relative-to= (generalized relative path function) -- Consistent path handling across all modules - -** TODO [#B] Treesit-auto grammar pinning is fragile :stability:maintenance: -:PROPERTIES: -:FILE: modules/prog-general.el -:LINES: 106-113 -:END: - -Go grammar is pinned to v0.19.1 by directly modifying the treesit-auto-recipe-list vector with =aset=. - -*Issues:* -- Hardcoded slot index (6) will break if treesit-auto changes struct layout -- No validation that the recipe exists or that slot 6 is :revision -- Comment says "Emacs 30.2" but doesn't check version -- Will fail silently if treesit-auto changes - -*Fix:* -1. Use proper accessor functions if available (treesit-auto-recipe-revision setter) -2. Add version check: =(when (version< emacs-version "30.3") ...)= -3. Add error handling and warning if pinning fails -4. Document why pinning is needed in comments - -** TODO [#B] Test coverage gaps in critical modules :testing:quality: -:PROPERTIES: -:FILES: modules/org-agenda-config.el, modules/mail-config.el, modules/ai-config.el -:END: - -Excellent test coverage for utility functions (custom-*, undead-buffers, org-roam helpers), but missing tests for: - -*High-value untested areas:* -1. =org-agenda-config.el=: - - =cj/build-org-agenda-list= (file discovery logic) - - =cj/org-skip-subtree-if-*= functions (agenda filtering) - - =cj/org-agenda-skip-subtree-if-not-overdue= (date logic) - -2. =mail-config.el=: - - =cj/mu4e-toggle-remote-images= (security critical) - - Email header parsing/address completion logic - -3. =ai-config.el=: - - =cj/gptel-add-file= (file path handling) - - =cj/gptel--fresh-org-prefix= (timestamp formatting) - -4. =org-roam-config.el=: - - =cj/move-org-branch-to-roam= (complex string manipulation) - - Already has tests for helpers, needs integration test - -*Recommendation:* Prioritize testing the org-agenda filtering functions - they're date-sensitive and easy to break. - -** TODO [#B] System-defaults autoload has wrong function name :bug:minor: -:PROPERTIES: -:FILE: modules/system-defaults.el -:LINE: 24 -:END: - -Line 24 has =autoload= for =env-bsd-p= from =host-environment=, but this file *requires* host-environment at compile time anyway (line 20). - -*Impact:* Redundant autoload, no functional issue. - -*Fix:* Remove the autoload statement. The eval-when-compile require is sufficient. - -** TODO [#B] Inconsistent coding system declarations :maintainability: -:PROPERTIES: -:FILES: early-init.el, modules/system-defaults.el -:END: - -UTF-8 coding system is set in two places: -- =early-init.el= lines 252-261 (before package loading) -- =system-defaults.el= lines 60-69 (after package loading) - -*Impact:* Redundant configuration, harder to maintain, unclear which takes precedence. - -*Fix:* Consolidate all coding system setup in early-init.el since it needs to be set before packages load. Remove from system-defaults.el. - -** TODO [#B] Chime notification times documentation mismatch :documentation:usability: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:LINES: 268-270 -:END: - -Comment says "5 minutes before and at event time (0 minutes)" but the value is ='(5 0)=. - -*Issue:* It's unclear if 0 means "at event time" or "immediately". Better documentation needed. - -*Fix:* Improve comment: -#+begin_src elisp -;; Alert 5 minutes before event and again at event time (0 = event start) -(setq chime-alert-time '(5 0)) -#+end_src - -** TODO [#C] Org-roam template paths use string concatenation :maintainability: -:PROPERTIES: -:FILE: modules/org-roam-config.el -:LINES: 43, 48, 53, 144, 150 -:END: - -Template file paths are built with =concat= instead of using =expand-file-name= or constants. - -*Impact:* Harder to change template location, no validation that files exist. - -*Fix:* Define template constants at top of file: -#+begin_src elisp -(defconst cj/org-roam-template-dir - (expand-file-name "org-roam-templates" user-emacs-directory)) -(defconst cj/org-roam-template-v2mom - (expand-file-name "v2mom.org" cj/org-roam-template-dir)) -;; etc... -#+end_src - -Validate files exist during config load. - -** TODO [#C] Dirvish-config has long function that should be split :refactor:maintainability: -:PROPERTIES: -:FILE: modules/dirvish-config.el -:LINES: 350-406 -:END: - -=cj/dired-copy-path-as-kill= is 57 lines with multiple responsibilities: -- Validation -- Project root detection -- Home directory handling -- Org-link formatting -- Clipboard operations -- User feedback - -*Fix:* Extract smaller functions: -- =cj/--path-type-for-file= (returns :project-relative, :home-relative, or :absolute) -- =cj/--format-as-org-link= (wraps path in org syntax) -- =cj/dired-copy-path-as-kill= (orchestrates the above) - -Better testability, clearer logic flow. - -** TODO [#C] Consider Embark for file operations instead of custom dirvish bindings :enhancement:usability: -:PROPERTIES: -:FILE: modules/dirvish-config.el -:LINES: 298-334 -:END: - -Dirvish config defines many custom keybindings (d, l, L, o, O, f, P) for file operations. - -*Alternative approach:* Use Embark for contextual file actions: -- Embark already knows about files in dired -- Can define actions once, use everywhere (not just dired) -- More discoverable (embark-act shows all options) -- Less cognitive load (one keybinding to remember: C-.) - -*Recommendation:* Keep the custom functions (good implementations), but expose them via embark-act instead of direct keybindings. Cleaner keymap, more flexible. - -** TODO [#C] Missing :ensure nil for more built-in packages :correctness:minor: -:PROPERTIES: -:FILE: Multiple modules -:END: - -Some built-in package use-package declarations are missing =:ensure nil=: -- =selection-framework.el=: =delsel= (line 54) -- Several others throughout - -*Impact:* Minimal - use-package won't try to install built-ins anyway. But inconsistent with the pattern used elsewhere. - -*Fix:* Add =:ensure nil= for consistency, or document why some have it and others don't. - -** TODO [#C] Consider lazy-loading more packages :performance:startup: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINES: 22, 40, 48, 151, 166, 240-253 -:END: - -Several completion packages use =:demand t= (vertico, marginalia, orderless, prescient): -- Vertico: Reasonable to demand (core UI) -- Marginalia: Could be :after vertico -- Orderless: Could be :after vertico -- Prescient: Could be :after vertico -- Company-prescient: Could be :after (vertico company) - -*Impact:* All load at startup, adds ~100ms. Not critical, but could be optimized. - -*Fix:* Profile with esup to see actual impact. If it's <100ms, probably not worth the complexity. - -** TODO [#C] Add validation for chime.el local path :stability:enhancement: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:LINE: 264 -:END: - -Config loads chime from local directory =~/code/chime.el= without checking if it exists. - -*Impact:* If directory doesn't exist, load fails silently or with cryptic error. - -*Fix:* Add existence check with helpful error message: -#+begin_src elisp -:load-path (lambda () - (let ((path "~/code/chime.el")) - (if (file-directory-p path) - path - (warn "chime.el not found at %s" path) - nil))) -#+end_src - -** TODO [#C] Document why org-msg uses local fork :documentation:maintenance: -:PROPERTIES: -:FILE: modules/mail-config.el -:LINE: 287 -:END: - -Loading org-msg from local path =/home/cjennings/code/org-msg= instead of MELPA. - -*Question:* Is this a temporary fork with patches? Does it need to stay local? - -*Fix:* Add comment explaining: -- What changes are in local fork -- Whether it's temporary or permanent -- Link to upstream PR if patches are submitted -- When it can switch back to MELPA - -** TODO [#C] Performance: Company idle-delay is very high :performance:usability: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINE: 212 -:END: - -Company idle-delay is set to 2 seconds, which is quite slow. Modern editors typically use 0.1-0.3s. - -*Impact:* Completion feels sluggish, requires manual triggering often. - -*Recommendation:* -- If keeping Company: Try 0.3 seconds -- If switching to Corfu: Default 0.1s is perfect - -User preference, but current setting feels unresponsive. - -** TODO [#C] org-protocol may not be fully configured :functionality:enhancement: -:PROPERTIES: -:FILE: modules/org-config.el -:LINES: 94-95, 116 -:END: - -=org-protocol= is loaded (line 95) and added to org-modules (line 116), but: -- No desktop entry file mentioned -- No documentation on how to use it -- org-webclipper exists but unclear if it uses org-protocol - -*Fix:* Either: -1. Document org-protocol setup (desktop file, browser integration) -2. Remove if unused (org-webclipper might be the preferred approach) - -** TODO [#D] Consider adding use-package :diminish for minor modes :enhancement:ui: -:PROPERTIES: -:FILE: Multiple modules -:END: - -Many minor modes appear in the modeline even though doom-modeline hides them. - -*Enhancement:* Add =:diminish= to use-package declarations for: -- =symbol-overlay-mode= -- =git-gutter-mode= -- =flycheck-mode= -- =yas-minor-mode= -- =ws-butler-mode= -- =rainbow-mode= - -Cleaner modeline if doom-modeline isn't used or fails to load. - -** TODO [#D] Add .dir-locals.el for project-specific settings :enhancement:devex: -:PROPERTIES: -:FILE: N/A (new file) -:END: - -Create =.dir-locals.el= in =~/.emacs.d= to: -- Set =fill-column= to 80 for elisp files -- Enable =flycheck-mode= for all elisp -- Set =checkdoc= settings locally -- Configure =outline-minor-mode= for easy navigation - -Helps maintain consistency when editing config. - -** DONE [#D] Consider moving from doom-modeline to mood-line :enhancement:performance: -CLOSED: [2025-11-03 Sun] -:PROPERTIES: -:FILE: modules/modeline-config.el -:END: - -✅ **Completed: Built custom modeline (better than mood-line)** - -Went beyond mood-line and created a fully custom modeline using only built-in -Emacs functionality. Features include: -- Color-coded buffer names (green/red/gold by status) -- VC branch with git symbol and state-based coloring -- String truncation for narrow windows -- Mouse click handlers (vc-diff, describe-mode, etc.) -- Help-echo tooltips -- Active-window-only segments -- Emacs 30 built-in right-alignment - -Fast, reliable, no external packages, fully customizable. - -** TODO [#D] Add commentary headers to test utility files :documentation:testing: -:PROPERTIES: -:FILES: tests/testutil-general.el, tests/testutil-filesystem.el -:END: - -Test utility files should have Commentary sections explaining: -- What utilities are provided -- When to use each utility -- Examples of usage -- Relationship to quality-engineer.org guidelines - -Helps future test authors understand available infrastructure. - -** TODO [#D] keybindings.el trains user away from C-x u :ux:philosophy: -:PROPERTIES: -:FILE: modules/keybindings.el -:LINES: 126-130 -:END: - -The rebinding of =C-x u= to a message telling users to use =C-/= instead is... philosophically interesting. - -*Discussion points:* -- Enforces good habits (C-/ is indeed faster) -- But violates principle of least surprise -- Could frustrate users who know Emacs bindings -- Muscle memory is hard to retrain - -*Alternative:* Just leave =C-x u= working. Add which-key hint that C-/ is faster. - -** TODO [#D] Missing which-key labels for some custom keymaps :usability:discoverability: -:PROPERTIES: -:FILES: Multiple modules -:END: - -Some custom keymaps lack which-key descriptions: -- =cj/vc-map= is well documented (vc-config.el) -- =cj/jump-map= is well documented (keybindings.el) -- =cj/org-table-map= is well documented (org-config.el) - -But check all custom keymaps for completeness. Run: -#+begin_src elisp -(apropos-variable "^cj/.*-map$") -#+end_src - -And verify each has which-key labels. - -** TODO [#D] early-init.el could benefit from more section comments :documentation:maintainability: -:PROPERTIES: -:FILE: early-init.el -:END: - -File has good top-level commentary but individual sections could use more context: -- Why network check exists (explain offline package strategy) -- Why GC threshold is increased (explain impact) -- Benchmark-init section is good, replicate that style - -Makes it easier for others (or future you) to understand decisions. - -** TODO [#D] Consider adding persistent-scratch package :enhancement:usability: -:PROPERTIES: -:FILE: N/A -:END: - -The *scratch* buffer is very useful but loses content on restart. - -*Enhancement:* Add =persistent-scratch= package to: -- Auto-save scratch buffer contents -- Restore on startup -- Optionally persist multiple scratch buffers - -Useful for quick notes, code experiments, calculations that span sessions. - -* Priority Explanation - -** Priority A (Critical) -Issues that: -- Prevent functionality from working -- Cause errors or crashes -- Block startup or major features -- Have security implications -- Significantly impact performance (>1s delay) - -** Priority B (High) -Issues that: -- Affect architecture or maintainability significantly -- Better alternative packages exist -- Moderate performance impact (>100ms) -- Testing gaps in critical code -- Risk of future breakage - -** Priority C (Medium) -Issues that: -- Affect code quality or consistency -- Minor refactoring opportunities -- Documentation improvements -- Small usability enhancements -- Package optimization opportunities - -** Priority D (Low) -Issues that: -- Nice-to-have features -- Polish and refinement -- Alternative approaches to consider -- Documentation expansion -- Style/convention improvements - -* Package Recommendations Summary - -** Recommended Changes -1. *Company → Corfu*: Better performance, more modern, better maintained -2. *Consider mood-line over doom-modeline*: Much faster for minimal info needs -3. *Add Cape*: Modern completion-at-point backends (if switching to Corfu) -4. *Consider persistent-scratch*: QOL improvement for scratch buffer - -** Packages to Keep -- Vertico: Excellent choice, fast, well-maintained -- Consult: Perfect companion to Vertico -- Orderless: Best completion style available -- Embark: Powerful, could be used more extensively -- Marginalia: Great for annotations -- Magit: Industry standard, no alternatives -- Org-roam: Best Zettelkasten for Emacs -- Which-key: Essential for discoverability - -** Packages to Evaluate -- Dirvish: Heavy package. Consider if dired + nerd-icons + embark is sufficient -- Doom-modeline: Feature-rich but heavy. Mood-line is lighter alternative -- Projectile vs project.el: You're using Projectile. Project.el is built-in and lighter, but Projectile is more mature. Keep unless performance is an issue. - -* Testing Recommendations - -Based on your quality-engineer.org guidelines and existing test coverage: - -** Well-Tested Areas ✓ -- All custom-* utility functions (excellent coverage) -- undead-buffers (comprehensive tests) -- org-roam helpers (good coverage) -- keyboard-macros -- Test utilities themselves - -** Priority Testing Gaps - -*** High Priority -1. *org-agenda-config.el filtering functions* - - =cj/org-skip-subtree-if-habit= - - =cj/org-skip-subtree-if-not-overdue= (date-sensitive!) - - =cj/build-org-agenda-list= (file discovery) - -2. *custom-buffer-file.el interactive functions* - - Split =cj/move-buffer-and-file= into =cj/--move-buffer-and-file= (testable core) and interactive wrapper - - Same for =cj/rename-buffer-and-file= - - Same for =cj/delete-buffer-and-file= - -*** Medium Priority -3. *org-roam-config.el integration test* - - =cj/move-org-branch-to-roam= workflow test - - Test actual file creation and org-roam db update - -4. *path-utils consolidation* (after creating module) - - Test project-relative, home-relative, absolute path logic - - Test with symlinks, non-existent paths, permission errors - -5. *ai-config.el GPTel context management* - - =cj/gptel-add-file= with various file types - - =cj/gptel--fresh-org-prefix= timestamp formatting - -*** Lower Priority -6. *mail-config.el address completion* -7. *Browser/external open logic* -8. *Theme persistence functions* - -** Testing Patterns to Follow - -Your existing tests follow excellent patterns from quality-engineer.org: -- One file per function for unit tests -- Clear normal/boundary/error case organization -- Good use of test utilities (testutil-general, testutil-filesystem) -- Proper setup/teardown -- Descriptive test names - -*Keep this pattern!* It's working well for you. - -** Test-Driven Refactoring Opportunities - -When fixing Priority A-B issues, write tests first: -1. Test for the bug/missing function -2. See it fail -3. Fix the code -4. See it pass -5. Refactor with confidence - -This is especially important for: -- The missing =cj/goto-git-gutter-diff-hunks= function -- The duplicate =cj/network-available= variable -- Network check refactoring (mock the network!) - -* Architecture Observations - -** What You're Doing Right ✓ - -1. *Excellent module organization*: Clean separation of concerns -2. *Good use of user-constants.el*: Centralized configuration -3. *Comprehensive custom utilities*: Well-tested, reusable functions -4. *Modern completion framework*: Vertico/Consult/Embark stack is excellent -5. *Debug infrastructure pattern*: org-agenda-config-debug.el is a good model -6. *Test coverage philosophy*: Following your own guidelines well -7. *Early-init optimization*: Good understanding of startup optimization -8. *Custom keymap prefix*: C-; is a good choice, well-organized - -** Architectural Patterns to Consider - -1. *Consolidate common patterns*: - - Path handling → path-utils.el - - Debug logging → debug-logging.el - - File validation → file-utils.el - -2. *Reduce coupling*: - - Some modules depend on specific implementations (e.g., projectile vs project.el) - - Consider adapter pattern for switching between implementations - -3. *Error handling strategy*: - - Add condition-case wrappers around file operations - - Log errors consistently to *Messages* or debug buffer - - Fail gracefully (warn user but don't break session) - -4. *Configuration validation*: - - Add startup checks for critical paths/files - - Warn about missing optional dependencies - - Provide helpful error messages with solutions - -** Code Smells Observed - -1. *String concatenation for paths*: Use =expand-file-name= consistently -2. *Silent failures*: Some functions fail without user feedback -3. *Synchronous network operations*: Blocks startup unnecessarily -4. *Duplicate logic*: Path handling, project root detection -5. *Large functions*: Some functions >50 lines, should be split -6. *Global state*: Some functions modify global variables without clear ownership - -* Next Steps Recommendation - -** Week 1: Fix Critical Bugs (Priority A) -1. Fix duplicate =cj/network-available= declaration -2. Implement missing =cj/goto-git-gutter-diff-hunks= -3. Define =cj/log-silently= or remove usage -4. Fix network check blocking (biggest impact) - -** Week 2: Architectural Improvements (Priority B) -1. Evaluate Company vs Corfu (try Corfu for a week) -2. Consolidate debug logging -3. Create path-utils.el module -4. Add error handling to org-agenda rebuild - -** Week 3: Testing (Priority B) -1. Add tests for org-agenda filtering functions -2. Split interactive/non-interactive in custom-buffer-file -3. Add integration test for org-roam branch extraction - -** Week 4: Polish (Priority C-D) -1. Consolidate coding system setup -2. Improve documentation -3. Add which-key labels where missing -4. Clean up minor issues - -** Ongoing -- Add tests for any new functionality -- Profile startup time with esup periodically -- Review package updates for breaking changes -- Keep org-gcal, chime.el forks in sync with upstream - -* Questions for You - -1. *Company vs Corfu*: Do you have strong feelings about Company? Willing to try Corfu? - Answer: I'd be excited to try Corfu. I don't know how I'd configure it yet and it seems like a big change. However, I planned to move to Corfu sometime soon. - -2. *Network check*: Do you actually need to support offline operation, or can we simplify this? - Answer: We can simplify. There was a time when I was traveling a lot and didn't have network access. The current offline operation seemed to help startup performance. The sooner we realized we were offline, the sooner I'd have a working Emacs. Now, I don't think it provides much benefit. - -3. *Local forks*: Are chime.el and org-msg local forks temporary? When can they move to MELPA? - Answer: These are packages I currently either maintain or are in the process of creating. I need the local-paths set so I can test. If you have better ideas on how I can switch them from using a local package to using the stable version using vc-install, I'd be grateful. - -4. *Doom-modeline*: Are you using features beyond what I saw in config? Would mood-line be sufficient? - No, I'm not. Mood-line would be welcomed. - -5. *Testing coverage*: What modules do you most want tests for? Anything you're nervous about breaking? - I'll probably look for large methods with lots of LOC and multiple tasks, then break them up for better testability. I was hoping you would point those out for me so I could focus on them. Will you? - -6. *Performance*: Any particular operations feel slow? (Agenda rebuild? Mail sync? Completion?) - Org Agenda is problematic and slow. I could get some insights into how to optimize it if it were well tested. - - -Other than that, here's my roadmap for fixing/improving -- video/audio config could use some testing around which sinks to attach to. It doesn't matter to me that this is Linux centric since it's my config. -- I'm not sure the UX in flyspell and abbrev is right. It could use some more attention. -- Same with Elfeed. While I had Elfeed dashboard at some point, it doesn't bring any joy to see a mostly blank screen when I check Elfeed. -- DWIM shell commands is alright. I wonder if I can leverage them better into eshell. -- calibredb is slow, but that's the package working on 14k+ pdfs and epubs. The right thing to do is to shrink the ebook inventory to < 1k or so books. -- I was thinking of investigating bettter diffs with difftastic. If it's straightforward and simple, I should jump at it. -- I would like to implement code-maat and code-compass within Emacs -- It would be good to have a solid profiler workflow so I can see what's going on inside my own methods and packages. For instance, I don't have the tools to tell me why it takes so long to generate my daily agenda. There's little observability. -- I absolutely need a workflow so I can go from org-mode file to reveal.js presentation using httpd server in Emacs with the press of a button. I know it can be done. Export the org file with injected css into the httpd directory, start the httpd server, then open a browser fullscreen on the file. That's pretty much 90% of the work right there. -- If I could figure out the latex-config so that I could author something non-trivial, I'd be happy. Right now, it's a bunch of cut/paste snippets from everyone else's config and it doesn't work. -- mail-config: the whole attachments workflow is awkward, but so necessary for daily life. It really needs improvement. -- then there's the whole org-contacts issue working with chime.el and birthdays. sexp expressions for dates? I should either figure them out and support them in chime, or I should disable them and find another way. -- it would be good to work with llms the way I do with claude code. gptel feels like a step in that direction and you can switch backends besides. but I need to either build a bunch of tools or install an MCP server. Which way to go? -- I would like to extract the code in local-repository that creates the repo so I can unit test the hell out of it and use it as a library for my own local repo. Seems like there's too much extraneous functionality there. -- lorem seems to have an error pop up each session about the markhov chain resetting. either that's bad and should be fixed or it's not bad and lorem should stfu. -- I believe jumper should probably be my next package, but I haven't had the time to spend on it. - -* Final Thoughts - -Your config is **quite good overall**: -- Well-organized modular structure -- Excellent test coverage for utilities -- Modern package choices (mostly) -- Good documentation and comments -- Following elisp idioms reasonably well - -The critical issues (Priority A) are straightforward fixes. The architectural suggestions (Priority B) are about making the codebase more maintainable long-term. Everything else is refinement. - -Focus on: -1. Fix the bugs that break functionality -2. Consider Corfu migration (biggest ROI for effort) -3. Add tests for date-sensitive agenda logic -4. Everything else is gravy - -You're clearly thoughtful about your config and willing to refactor when needed. These issues are normal for a mature, actively-used configuration. Keep up the good work! - -* Second Opinion: Ruthless Prioritization & Reality Checks -:PROPERTIES: -:DATE: 2025-10-30 -:END: - -After reviewing your answers and additional roadmap items, here's honest feedback on what matters, what doesn't, and what you should actually do. - -** Key Insights from Your Answers - -*** Company → Corfu Migration: GREEN LIGHT -You're excited, you were planning it anyway. Corfu configuration is actually *simpler* than Company, not harder. This is a quick win with immediate UX improvement. - -*Reality:* 2 hours of work, noticeable performance improvement. Do it this weekend. - -*** Network Check: DELETE IT IMMEDIATELY -You confirmed it's technical debt from when you traveled. It's adding 1+ seconds to every startup for a problem you no longer have. - -*Reality:* 15 minutes to delete ~100 lines of code. Instant gratification. Do this first. - -*** Local Package Development Workflow: NEEDS PATTERN -You maintain chime.el and org-msg locally for testing. Current approach works but isn't elegant. See implementation pattern below. - -*** Mood-line Switch: EASY WIN -You don't need doom-modeline features. Mood-line is 10x faster and simpler. - -*Reality:* 30 minutes. Do it. - -*** Large Functions to Break Up: MOSTLY FINE -Analyzed your codebase. Most functions are reasonably sized. The ones you mentioned: -- =dirvish-config.el:350-406= =cj/dired-copy-path-as-kill= (57 lines) - Could split, but works fine -- =org-roam-config.el:250-298= =cj/move-org-branch-to-roam= (49 lines) - Actually well-designed with helpers -- =org-agenda-config.el:91-105= =cj/build-org-agenda-list= (15 lines) - Small but THIS IS YOUR PERFORMANCE BOTTLENECK - -*Reality:* Your code organization is fine. Your problem is *performance observability*, not function length. - -*** Org Agenda Performance: ROOT CAUSE IDENTIFIED -This is your actual pain point. But you're flying blind without profiling infrastructure. - -*Hypothesis:* =cj/build-org-agenda-list= recursively scans entire projects directory on every agenda open with: -- No caching (even when files unchanged) -- No error handling -- Synchronous file I/O blocking UI -- Probably hundreds of file-exists-p calls - -*Reality:* You need profiling tools FIRST, then optimization. Don't guess, measure. - -** TIER 1: Do These First (High Impact, Low Effort) -*Time investment: One weekend* -*Status: START HERE* - -*** TODO [#A] Remove network check entirely :performance:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 76-78, 104-105, 189, 206 -:EFFORT: 15 minutes -:IMPACT: Removes 1+ seconds from every startup -:END: - -Delete the following: -1. =cj/network-available= variable declarations (both) -2. =cj/internet-up-p= function and all calls -3. =cj/use-online-repos= flag logic -4. Any package.el conditionals based on network status - -Let package.el handle its own errors gracefully. Modern Emacs does this fine. - -*** TODO [#A] Fix missing cj/log-silently definition :bug:critical: -:PROPERTIES: -:FILE: modules/wrap-up.el -:LINE: 28 -:EFFORT: 5 minutes -:END: - -Either: -1. Move =cj/log-silently= from =org-agenda-config-debug.el= to =config-utilities.el= (load early) -2. Or replace with simple =(message ...)= call - -The function is only used in one place, so simple message might be sufficient. - -*** TODO [#A] Remove duplicate cj/network-available declaration :bug:critical: -:PROPERTIES: -:FILE: early-init.el -:LINES: 104-105 -:EFFORT: 2 minutes -:END: - -Delete the duplicate at lines 104-105. Keep only the first declaration at line 76. - -*Note:* This becomes moot if you delete network check entirely (recommended). - -*** TODO [#A] Fix missing cj/goto-git-gutter-diff-hunks function :bug:critical: -:PROPERTIES: -:FILE: modules/vc-config.el -:LINE: 123 -:EFFORT: 15 minutes -:END: - -Keybinding =C-; v d= references undefined function. - -Implement as: -#+begin_src elisp -(defun cj/goto-git-gutter-diff-hunks () - "Jump to git-gutter diff hunks using consult." - (interactive) - (require 'git-gutter) - (consult-line "^[+\\-]")) ;; Or use git-gutter:popup-hunk -#+end_src - -Or remove the keybinding if not needed. - -*** TODO [#B] Migrate Company to Corfu :enhancement:performance: -:PROPERTIES: -:FILE: modules/selection-framework.el -:LINES: 202-236 -:EFFORT: 2 hours -:IMPACT: 3x faster completion, simpler config -:END: - -*Why Corfu:* -- ~3x faster (child frames vs overlays) -- Better maintained -- Simpler codebase (~500 LOC vs ~3000 LOC) -- Works seamlessly with Vertico/Consult -- Better integration with Cape for backends - -*Migration:* -#+begin_src elisp -(use-package corfu - :ensure t - :demand t - :bind (:map corfu-map - ("TAB" . corfu-next) - ("S-TAB" . corfu-previous)) - :custom - (corfu-auto t) - (corfu-auto-delay 0.1) ;; Much faster than your 2s company delay! - (corfu-auto-prefix 2) - (corfu-quit-no-match 'separator) - :init - (global-corfu-mode)) - -(use-package corfu-popupinfo - :after corfu - :hook (corfu-mode . corfu-popupinfo-mode) - :custom - (corfu-popupinfo-delay '(0.5 . 0.2))) - -(use-package cape - :ensure t - :after corfu - :init - (add-to-list 'completion-at-point-functions #'cape-dabbrev) - (add-to-list 'completion-at-point-functions #'cape-file) - (add-to-list 'completion-at-point-functions #'cape-keyword)) -#+end_src - -Remove entire company configuration. Keep mu4e completion settings (Corfu respects them). - -*Configuration is actually simpler than Company.* Just do it. - -*** DONE [#B] Switch doom-modeline to mood-line :enhancement:performance: -CLOSED: [2025-11-03 Sun] -:PROPERTIES: -:FILE: modules/modeline-config.el -:EFFORT: 30 minutes -:IMPACT: 10x faster modeline updates -:END: - -✅ **Completed: Custom modeline implementation** - -Built a custom modeline with advanced features inspired by Prot's design: -- Named segment architecture (easy reordering) -- Color-coded buffer status -- VC state faces (green=clean, yellow=edited, red=conflict) -- Git branch symbol (U+E0A0) -- Interactive mouse handlers -- String truncation for narrow windows -- Active-window-only segments -- Help-echo tooltips -- Position format: L:# C:# - -See commit c0c4b17 for full implementation. - -*** TODO [#C] Fix lorem markov chain error :bug:minor: -:PROPERTIES: -:FILE: modules/lorem-config.el (or wherever lorem is configured) -:EFFORT: 5 minutes -:END: - -The error message is probably harmless initialization noise. Either: -1. Wrap lorem initialization in =(with-demoted-errors ...)= -2. Configure lorem to suppress the warning -3. Disable lorem if you don't use it - -Check your config for lorem setup and add error suppression. - -** TIER 2: Build Observability Infrastructure (HIGHEST VALUE) -*Time investment: One week* -*Status: CRITICAL DEPENDENCY FOR EVERYTHING ELSE* - -*** TODO [#A] Create debug-profiling.el module :infrastructure:performance: -:PROPERTIES: -:FILE: modules/debug-profiling.el (new file) -:EFFORT: 3-4 hours -:IMPACT: Unlocks all performance optimization work -:END: - -This is your #1 blocker for fixing org-agenda and other performance issues. - -Create comprehensive profiling infrastructure: - -#+begin_src elisp -;;; debug-profiling.el --- Profiling and performance debugging tools -*- lexical-binding: t; -*- - -;;; Commentary: -;; Unified profiling infrastructure for measuring and optimizing performance. -;; Provides helpers for: -;; - Function timing (individual and batch) -;; - Memory profiling -;; - Startup profiling -;; - Interactive profiling sessions -;; -;; Usage: -;; M-x cj/profile-function RET cj/build-org-agenda-list RET -;; M-x cj/profile-startup -;; (cj/time-it "agenda rebuild" (cj/build-org-agenda-list)) - -;;; Code: - -(require 'profiler) -(require 'benchmark) - -(defvar cj/profiling-log-buffer "*CJ Profile Log*" - "Buffer name for profiling results.") - -(defvar cj/profiling-enabled nil - "When non-nil, enable automatic timing of instrumented functions.") - -(defun cj/time-it (description &rest body) - "Execute BODY and log timing with DESCRIPTION." - (let* ((start-time (current-time)) - (result (car body)) - (elapsed (float-time (time-subtract (current-time) start-time)))) - (with-current-buffer (get-buffer-create cj/profiling-log-buffer) - (goto-char (point-max)) - (insert (format "[%s] %s: %.3f sec\n" - (format-time-string "%H:%M:%S") - description - elapsed))) - (message "%s: %.3f sec" description elapsed) - result)) - -(defmacro cj/measure (&rest body) - "Measure execution time of BODY, return result and timing." - `(let ((start-time (current-time)) - (result (progn ,@body)) - (end-time (current-time))) - (cons result (float-time (time-subtract end-time start-time))))) - -(defun cj/profile-function (function &optional runs) - "Profile FUNCTION by calling it RUNS times (default 1). -Shows results in dedicated buffer with call tree." - (interactive - (list (intern (completing-read "Function to profile: " obarray 'fboundp t)) - (when current-prefix-arg - (read-number "Number of runs: " 10)))) - (setq runs (or runs 1)) - (profiler-start 'cpu) - (dotimes (_ runs) - (funcall function)) - (profiler-stop) - (profiler-report) - (message "Profiled %s (%d run%s)" function runs (if (> runs 1) "s" ""))) - -(defun cj/profile-startup () - "Profile Emacs startup by restarting with profiler enabled. -Results are saved to profile.el in user-emacs-directory." - (interactive) - (let ((profile-file (expand-file-name "startup-profile.el" user-emacs-directory))) - (profiler-start 'cpu+mem) - (message "Restart Emacs to profile startup. Results will be in %s" profile-file) - (add-hook 'emacs-startup-hook - (lambda () - (profiler-report) - (profiler-report-write-profile profile-file) - (profiler-stop) - (message "Startup profiling complete. See %s" profile-file))))) - -(defun cj/benchmark-function (function &optional iterations) - "Benchmark FUNCTION over ITERATIONS (default 100). -Shows average, min, max, and total time." - (interactive - (list (intern (completing-read "Function to benchmark: " obarray 'fboundp t)) - (read-number "Iterations: " 100))) - (setq iterations (or iterations 100)) - (let ((times '())) - (dotimes (_ iterations) - (let* ((result (cj/measure (funcall function))) - (elapsed (cdr result))) - (push elapsed times))) - (let* ((total (apply #'+ times)) - (avg (/ total iterations)) - (min-time (apply #'min times)) - (max-time (apply #'max times))) - (message "%s (%d iterations): avg=%.4fs min=%.4fs max=%.4fs total=%.2fs" - function iterations avg min-time max-time total)))) - -(defun cj/instrument-package (package) - "Instrument all functions in PACKAGE for profiling with elp." - (interactive - (list (intern (completing-read "Package to instrument: " obarray)))) - (elp-instrument-package (symbol-name package)) - (message "Instrumented package: %s. Run M-x elp-results to see data." package)) - -(defun cj/profile-agenda-rebuild () - "Profile org-agenda rebuild with detailed timing." - (interactive) - (require 'org-agenda) - (cj/time-it "Full agenda rebuild" (cj/build-org-agenda-list))) - -(defun cj/show-profile-log () - "Display the profiling log buffer." - (interactive) - (display-buffer (get-buffer-create cj/profiling-log-buffer))) - -(defun cj/clear-profile-log () - "Clear the profiling log buffer." - (interactive) - (with-current-buffer (get-buffer-create cj/profiling-log-buffer) - (erase-buffer))) - -;; Keybindings (add to your keybindings.el) -;; (global-set-key (kbd "C-; p f") #'cj/profile-function) -;; (global-set-key (kbd "C-; p b") #'cj/benchmark-function) -;; (global-set-key (kbd "C-; p l") #'cj/show-profile-log) -;; (global-set-key (kbd "C-; p a") #'cj/profile-agenda-rebuild) - -(provide 'debug-profiling) -;;; debug-profiling.el ends here -#+end_src - -Load this early in init.el: -#+begin_src elisp -(when (or (eq cj/debug-modules t) - (memq 'profiling cj/debug-modules)) - (require 'debug-profiling (expand-file-name "modules/debug-profiling" user-emacs-directory))) -#+end_src - -*** TODO [#A] Profile org-agenda-rebuild to find bottleneck :performance:critical: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:EFFORT: 1 hour -:DEPENDS: debug-profiling.el -:END: - -Once profiling infrastructure exists: - -1. Run: =M-x cj/profile-function RET cj/build-org-agenda-list RET= -2. Run: =M-x cj/benchmark-function RET cj/build-org-agenda-list RET 10= -3. Instrument org-agenda package: =M-x cj/instrument-package RET org-agenda RET= -4. Review results with =M-x elp-results= - -*Hypothesis to test:* -- Is time spent in file I/O (directory-files-recursively)? -- Is it parsing org files? -- Is it just too many files? -- Is there a slow helper function being called repeatedly? - -*Expected findings:* -- Probably 80%+ time in =cj/add-files-to-org-agenda-files-list= -- Likely no caching, runs every time -- Possible N+1 file access pattern - -*Next steps after profiling:* -- Add file-notify watchers to invalidate cache only when projects-dir changes -- Cache the file list -- Run rebuild async on startup - -*** TODO [#B] Add instrumentation to cj/build-org-agenda-list :performance:observability: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:EFFORT: 30 minutes -:DEPENDS: debug-profiling.el, profiling results -:END: - -After profiling identifies bottleneck, add instrumentation: - -#+begin_src elisp -(defun cj/build-org-agenda-list () - "Rebuilds the org agenda list with instrumentation." - (interactive) - (let ((start-time (current-time))) - ;; reset org-agenda-files - (setq org-agenda-files (list inbox-file schedule-file gcal-file)) - - ;; Time the expensive operation - (let ((scan-start (current-time))) - (cj/add-files-to-org-agenda-files-list projects-dir) - (message "Scanned projects-dir in %.3f sec" - (float-time (time-subtract (current-time) scan-start)))) - - (let ((total-time (float-time (time-subtract (current-time) start-time)))) - (message "Rebuilt org-agenda-files in %.3f sec (%d files)" - total-time - (length org-agenda-files))))) -#+end_src - -This gives you observability on every agenda rebuild. - -*** TODO [#B] Add caching to org-agenda rebuild :performance:critical: -:PROPERTIES: -:FILE: modules/org-agenda-config.el -:EFFORT: 2 hours -:DEPENDS: Profiling results -:END: - -*Only do this after profiling confirms file scanning is the bottleneck.* - -Implement caching with file-notify: - -#+begin_src elisp -(defvar cj/org-agenda-cache nil - "Cached list of org-agenda files.") - -(defvar cj/org-agenda-cache-dirty t - "When non-nil, agenda cache needs rebuilding.") - -(defun cj/invalidate-agenda-cache (&rest _) - "Mark agenda cache as dirty." - (setq cj/org-agenda-cache-dirty t)) - -(defun cj/build-org-agenda-list-cached () - "Build org-agenda list with caching." - (interactive) - (when (or (not cj/org-agenda-cache) - cj/org-agenda-cache-dirty) - (let ((start-time (current-time))) - (setq cj/org-agenda-cache (list inbox-file schedule-file gcal-file)) - (cj/add-files-to-org-agenda-files-list projects-dir) - (setq cj/org-agenda-cache org-agenda-files) - (setq cj/org-agenda-cache-dirty nil) - (message "Rebuilt agenda cache in %.3f sec (%d files)" - (float-time (time-subtract (current-time) start-time)) - (length cj/org-agenda-cache)))) - (setq org-agenda-files cj/org-agenda-cache)) - -;; Set up file watching -(require 'filenotify) -(file-notify-add-watch projects-dir - '(change) - #'cj/invalidate-agenda-cache) -#+end_src - -*** TODO [#B] Test org-agenda filtering functions :testing:critical: -:PROPERTIES: -:FILE: tests/org-agenda-config-test.el (new file) -:EFFORT: 2-3 hours -:END: - -These functions are date-sensitive and break silently. High-value tests: - -#+begin_src elisp -;;; org-agenda-config-test.el --- Tests for org-agenda-config -*- lexical-binding: t; -*- - -(require 'ert) -(require 'org-agenda-config) - -(ert-deftest test-cj/org-skip-subtree-if-habit () - "Test habit filtering." - (with-temp-buffer - (org-mode) - (insert "* TODO Test\n:PROPERTIES:\n:STYLE: habit\n:END:\n") - (goto-char (point-min)) - (should (cj/org-skip-subtree-if-habit)))) - -(ert-deftest test-cj/org-skip-subtree-if-not-overdue () - "Test overdue task detection." - (with-temp-buffer - (org-mode) - ;; Task scheduled yesterday (overdue) - (insert "* TODO Overdue\nSCHEDULED: <2025-10-29>\n") - (goto-char (point-min)) - (should-not (cj/org-skip-subtree-if-not-overdue)) - - ;; Task scheduled tomorrow (not overdue) - (erase-buffer) - (insert "* TODO Future\nSCHEDULED: <2025-10-31>\n") - (goto-char (point-min)) - (should (cj/org-skip-subtree-if-not-overdue)))) -#+end_src - -** TIER 3: Quick Wins (After Tier 1 & 2) -*Time investment: 1-2 hours each* -*Status: Only tackle after profiling infrastructure is done* - -*** TODO [#B] Implement org-to-reveal.js workflow :enhancement:usability: -:PROPERTIES: -:FILE: modules/org-config.el or new modules/org-present-config.el -:EFFORT: 2 hours -:IMPACT: Solves specific, concrete need -:END: - -You said: "I know it can be done. Export to reveal.js, start httpd, open browser." - -You're right, it's straightforward: - -#+begin_src elisp -(use-package ox-reveal - :ensure t - :after org - :custom - (org-reveal-root "https://cdn.jsdelivr.net/npm/reveal.js")) - -(use-package simple-httpd - :ensure t - :custom - (httpd-port 8080) - (httpd-root "~/presentations")) - -(defun cj/org-present-reveal () - "Export current org file to reveal.js and present via httpd." - (interactive) - (unless (eq major-mode 'org-mode) - (user-error "Not in an org-mode buffer")) - - (let* ((httpd-dir (expand-file-name httpd-root)) - (output-file (expand-file-name - (concat (file-name-base) ".html") - httpd-dir))) - - ;; Ensure output directory exists - (make-directory httpd-dir t) - - ;; Export to reveal.js HTML - (org-reveal-export-to-html) - - ;; Move to httpd directory - (let ((exported-file (concat (file-name-base) ".html"))) - (rename-file exported-file output-file t)) - - ;; Start httpd if not running - (unless (process-status "httpd") - (httpd-start) - (message "Started httpd server on port %d" httpd-port)) - - ;; Open in browser (fullscreen) - (let ((url (format "http://localhost:%d/%s" - httpd-port - (file-name-nondirectory output-file)))) - (browse-url url) - (message "Presenting: %s" url)))) - -(with-eval-after-load 'org - (define-key org-mode-map (kbd "C-c C-v p") #'cj/org-present-reveal)) -#+end_src - -*That's it.* 30 lines. One keybinding. Done. - -Test with a simple org file: -#+begin_src org -,#+TITLE: Test Presentation -,#+REVEAL_THEME: moon - -,* Slide 1 -Content here - -,* Slide 2 -More content -#+end_src - -Press =C-c C-v p= and you're presenting. - -*** TODO [#B] Set up difftastic integration :enhancement:usability: -:PROPERTIES: -:FILE: modules/vc-config.el or modules/magit-config.el -:EFFORT: 30 minutes -:END: - -Install difftastic: =sudo pacman -S difftastic= or =cargo install difftastic= - -Then add to magit: - -#+begin_src elisp -(use-package magit-difftastic - :ensure t - :after magit - :bind (:map magit-status-mode-map - ("C-c d" . magit-difftastic-show)) - :custom - (magit-difftastic-executable "difft")) -#+end_src - -Or for native integration: -#+begin_src elisp -(setq magit-diff-refine-hunk 'all) ;; Already have this probably - -(defun cj/magit-difftastic-diff (args files) - "Show diff using difftastic." - (interactive (magit-diff-arguments)) - (require 'magit) - (let ((command (concat "difft " (mapconcat #'identity args " ")))) - (magit-git-command command))) - -(transient-append-suffix 'magit-diff '(-1 -1) - '("D" "Difftastic diff" cj/magit-difftastic-diff)) -#+end_src - -Simple and useful. Just do it. - -*** TODO [#C] Implement local package development workflow :enhancement:architecture: -:PROPERTIES: -:FILES: modules/org-agenda-config.el, modules/mail-config.el -:EFFORT: 1 hour -:END: - -You maintain chime.el and org-msg. You need to switch between local dev and stable versions easily. - -*Clean pattern:* - -In =user-constants.el=: -#+begin_src elisp -(defcustom cj/development-packages '() - "List of packages to load from local development paths. -When a package symbol is in this list, load from ~/code/<package>. -Otherwise use stable version via vc-install or MELPA. - -Example: (setq cj/development-packages '(chime org-msg))" - :type '(repeat symbol) - :group 'cj) - -(defun cj/use-local-package-p (package) - "Return non-nil if PACKAGE should be loaded from local path." - (memq package cj/development-packages)) -#+end_src - -In your package configs: -#+begin_src elisp -;; Chime configuration -(if (cj/use-local-package-p 'chime) - ;; Development mode: use local path - (use-package chime - :load-path "~/code/chime.el" - :demand t - ;; ... rest of config ... - ) - ;; Production mode: use stable version - (use-package chime - :vc (:url "https://github.com/yourusername/chime.el") - :demand t - ;; ... rest of config ... - )) -#+end_src - -To switch modes, just toggle the symbol in =cj/development-packages= and restart Emacs. - -*Benefits:* -- Clear intent: variable name says what it does -- Easy to toggle: add/remove symbols from list -- Version controlled: your config documents which packages you're actively developing -- No path duplication: define path logic once - -** TIER 4: Maybe/Someday (Proceed with Caution) -*Time investment: Unknown, possibly infinite* -*Status: Only do if you have concrete use case* - -*** HOLD [#D] Investigate code-maat/code-compass in Emacs :enhancement:tooling: -:PROPERTIES: -:REASON: Complexity vs value unclear -:END: - -*Hard truth:* This is a big project. Code-maat is a mature Java tool with significant functionality. - -*Questions before investing time:* -1. What would Emacs integration give you that CLI doesn't? -2. How often do you actually use code-maat? -3. Is this solving a real problem or is it intellectually interesting? - -*Recommendation:* Use the CLI tools. Call them from Emacs if needed: -#+begin_src elisp -(defun cj/code-maat-analyze (git-log-file) - "Run code-maat analysis on GIT-LOG-FILE." - (interactive "fGit log file: ") - (async-shell-command - (format "code-maat -l %s -c git2 -a coupling" git-log-file) - "*Code Maat Results*")) -#+end_src - -Don't reimplement it in elisp unless you have a very specific need the CLI can't meet. - -*** HOLD [#D] Fix LaTeX configuration :documentation:enhancement: -:PROPERTIES: -:REASON: No concrete use case yet -:END: - -You said: "Right now, it's a bunch of cut/paste snippets from everyone else's config and it doesn't work." - -*Hard truth:* LaTeX config is complex because LaTeX is complex. Don't invest time until you have a specific document you need to write. - -*When you do need it:* -1. Start with one concrete document (paper? letter? resume?) -2. Get that one document working -3. Extract the config that made it work -4. Then generalize - -*Don't* try to build a complete LaTeX environment speculatively. You'll waste time on edge cases you'll never hit. - -*Minimal starting point when you need it:* -#+begin_src elisp -(use-package auctex - :ensure t - :defer t - :mode ("\\.tex\\'" . LaTeX-mode) - :custom - (TeX-auto-save t) - (TeX-parse-self t) - (TeX-master nil)) -#+end_src - -That's it. Start there. Add more when you hit actual problems. - -*** HOLD [#D] Improve Elfeed dashboard UX :enhancement:ui: -:PROPERTIES: -:REASON: Need to clarify actual usage patterns -:END: - -You said: "it doesn't bring any joy to see a mostly blank screen when I check Elfeed." - -*Questions:* -1. Do you actually use RSS regularly, or is it aspirational? -2. If blank screen = no new items, is that a problem? -3. Would you use RSS more if the UX was better, or is RSS not part of your workflow? - -*Options:* -1. *If you don't use RSS:* Remove Elfeed. One less thing to maintain. -2. *If you want to use RSS more:* Start by curating better feeds, not fixing UI -3. *If blank screen bothers you:* Add a "no new items" message or show last N items regardless of read status - -Don't fix tooling for a workflow you're not sure you want. - -*** HOLD [#D] DWIM shell + eshell integration :enhancement:shell: -:PROPERTIES: -:REASON: Current solution works fine -:END: - -You have 815 lines of dwim-shell-config. It works. Leave it alone. - -*Hard truth:* This is a working system. "I wonder if I can leverage them better" is not a problem statement. - -Unless you have a specific workflow that's currently painful, don't touch this. - -*** HOLD [#D] Calibredb performance optimization :performance: -:PROPERTIES: -:REASON: Root cause is data volume, not code -:END: - -You correctly identified: "The right thing to do is to shrink the ebook inventory to < 1k or so books." - -*You already know the answer.* Don't optimize code when the problem is 14k+ files. - -Curate your library. Delete what you won't read. This is a data problem, not a code problem. - -*** HOLD [#D] Flyspell/abbrev UX improvements :enhancement:usability: -:PROPERTIES: -:REASON: Vague complaint, no specific pain points -:END: - -You said: "I'm not sure the UX is right. It could use some more attention." - -*This is not actionable.* Come back when you can say: -- "Pressing X does Y but I want Z" -- "It's too slow when..." -- "It conflicts with..." - -Until then, it's bikeshedding. - -*** HOLD [#D] Video/audio config sink testing :testing:enhancement: -:PROPERTIES: -:REASON: Nice to have, not blocking anything -:END: - -You said: "could use some testing around which sinks to attach to" - -*Questions:* -1. Does it currently work for your use case? -2. Do you switch audio sinks often? -3. Is this actually painful or just "would be nice"? - -If it works, leave it. If it doesn't work, describe the failure mode first. - -*** HOLD [#D] Build jumper package :project:packaging: -:PROPERTIES: -:REASON: Another package to maintain, unclear value -:END: - -You said: "I believe jumper should probably be my next package, but I haven't had the time." - -*Hard questions:* -1. What problem does jumper solve? -2. Do existing navigation packages (avy, ace-window, consult-line) not solve it? -3. Are you building this for yourself or for others? -4. How will you maintain it alongside chime and org-msg? - -If you're excited about it: Build it. But recognize it's a significant time investment. - -If you're not excited: Don't. You already maintain two packages. - -*** HOLD [#D] Extract local-repository creation logic :refactor:library: -:PROPERTIES: -:REASON: Unclear benefit -:END: - -You said: "extract the code... so I can unit test the hell out of it and use it as a library" - -*Questions:* -1. What would you use the library for beyond local-repository? -2. Is local-repository not working currently? -3. Are you planning to build other tools that need this? - -Don't extract libraries speculatively. Extract when you have two concrete use cases. - -*** HOLD [#D] Org-contacts + chime.el birthdays integration :enhancement:integration: -:PROPERTIES: -:REASON: Should be fixed in chime, not config -:END: - -You said: "I should either figure out sexp expressions and support them in chime, or disable them" - -*This is a chime.el issue, not a config issue.* - -File it as a chime.el feature request or bug. Your config shouldn't work around package limitations. - -** What NOT To Do - -*** DON'T: Optimize Before Profiling -You can't fix performance issues you can't measure. Build profiling infrastructure FIRST. - -*** DON'T: Build Speculative Libraries -Extract code into libraries when you have 2+ concrete use cases. Not before. - -*** DON'T: Fix Vague Complaints -"UX could be better" is not actionable. Come back with specific pain points. - -*** DON'T: Reimplement Mature CLI Tools -code-maat is Java. It works. Don't rewrite it in elisp. - -*** DON'T: Configure for Hypothetical Use Cases -LaTeX doesn't work? Come back when you need to write a specific LaTeX document. - -*** DON'T: Maintain More Packages Than You Can Support -You have chime and org-msg. That's plenty. Think hard before adding jumper. - -** Large Functions Analysis - -You asked me to identify large, complex functions that need breaking up. - -*Reality check:* Your functions are mostly fine. Here's what I found: - -*** Functions You Mentioned - -*1. dirvish-config.el:350-406 =cj/dired-copy-path-as-kill= (57 lines)* -- Has 6 responsibilities: validation, project detection, path formatting, org-link formatting, clipboard, feedback -- *Status:* Works fine, moderately complex -- *Recommendation:* Could extract helpers (=cj/--determine-path-type=, =cj/--format-path-for-display=), but not urgent -- *Priority:* [#C] - Refactor for testability when you have time - -*2. org-roam-config.el:250-298 =cj/move-org-branch-to-roam= (49 lines)* -- Already well-decomposed with helper functions! -- Uses =cj/org-link-get-description=, =cj/--generate-roam-slug=, =cj/--demote-org-subtree=, =cj/--format-roam-node= -- *Status:* Good design, clear logic flow -- *Recommendation:* Add integration test, but don't refactor further -- *Priority:* [#B] - Add tests, keep implementation - -*3. org-agenda-config.el:91-105 =cj/build-org-agenda-list= (15 lines)* -- Small function, but THIS IS YOUR PERFORMANCE BOTTLENECK -- Not because of size, but because it recursively scans directories without caching -- *Status:* CRITICAL ISSUE -- *Recommendation:* Profile it, add caching, add error handling -- *Priority:* [#A] - Profile and optimize ASAP - -*** Other Large Files - -*dwim-shell-config.el (815 lines)* -- Collection of DWIM commands -- *Status:* Works fine -- *Recommendation:* Leave it alone unless something breaks - -*custom-comments.el (636 lines)* -- Well-tested utility functions -- *Status:* No issues -- *Recommendation:* Keep as is - -*music-config.el (484 lines)* -- EMMS configuration -- *Recommendation:* Review when you mentioned video/audio sink testing - -*** The Real Issue - -Your problem isn't function length. Your problem is *performance observability*. - -You need: -1. Profiling infrastructure (debug-profiling.el) -2. Instrumentation in critical paths -3. Caching where appropriate -4. Error handling in file operations - -Focus on TIER 2 (observability infrastructure) before worrying about function decomposition. - -** Concrete Action Plan: This Weekend - -*** Saturday Morning (2-3 hours) - -1. *Delete network check* (15 min) - - Remove =cj/internet-up-p=, =cj/network-available=, all related code - - Test startup, should be 1+ seconds faster - -2. *Fix Priority A bugs* (45 min) - - Fix duplicate =cj/network-available= (if not deleted above) - - Implement =cj/goto-git-gutter-diff-hunks= or remove binding - - Fix =cj/log-silently= (move to config-utilities or use message) - -3. *Fix lorem error* (5 min) - - Find lorem config, suppress error message - -4. *Test everything still works* (15 min) - -*** Saturday Afternoon (2-3 hours) - -5. *Corfu migration* (2 hours) - - Remove company configuration - - Add corfu + corfu-popupinfo + cape - - Test completion in elisp, org, mu4e - - Verify mu4e still excludes completion - -6. *Switch to mood-line* (30 min) - - Remove doom-modeline - - Add mood-line configuration - - Restart, verify modeline works - -*** Sunday Morning (3-4 hours) - -7. *Create debug-profiling.el* (3 hours) - - Use the template I provided above - - Add to modules/ - - Configure in user-constants.el with =cj/debug-modules= - - Test basic commands work - -8. *Profile org-agenda* (30 min) - - Run =M-x cj/profile-agenda-rebuild= - - Run =M-x cj/profile-function RET cj/build-org-agenda-list= - - Document findings in *CJ Profile Log* - -9. *Write up findings* (30 min) - - What's slow? - - Why is it slow? - - What's the fix? - -*** Sunday Afternoon (Optional, 1-2 hours) - -10. *Reveal.js workflow* (2 hours) - - Implement =cj/org-present-reveal= - - Test with simple presentation - - Bind to key - -11. *Difftastic* (30 min) - - Install difftastic - - Configure magit integration - - Test on a real diff - -** Expected Outcomes - -After this weekend: -- ✅ Startup 1+ seconds faster (network check removed) -- ✅ No more broken keybindings or missing functions -- ✅ Completion is faster and more responsive (Corfu) -- ✅ Modeline updates are faster (mood-line) -- ✅ You have profiling infrastructure for all future optimization -- ✅ You understand why org-agenda is slow -- ✅ You have concrete data to guide optimization -- ✅ (Optional) You can present org files as reveal.js slides - -*** After This Weekend - -*Then stop and reassess.* - -- How do you feel about the changes? -- What's still bothering you? -- Is org-agenda faster after caching? (if you implement it) -- What's the next pain point? - -Don't commit to more work until you've lived with these changes for a week. - -** My Question for You - -Looking at your list of improvements, I see someone who: -- Likes building tools (jumper, code-maat integration, etc.) -- Gets excited by new technologies (difftastic, MCP, reveal.js) -- Has many "wouldn't it be cool if..." ideas - -But I also see someone who: -- Already has a working, sophisticated config -- Knows what's actually painful (org-agenda performance, mail attachments) -- Has limited time (maintaining chime, org-msg) - -*So here's my question:* - -*Are you building tools because they solve real problems, or because you enjoy building?* - -Both are valid answers! But the answer changes what you should prioritize: - -*If solving real problems:* -- Focus on TIER 1 & 2 (bugs, performance, observability) -- Ignore most of TIER 4 -- Build reveal.js workflow (concrete need) -- Fix mail attachments (daily pain) - -*If you enjoy building:* -- Pick ONE project (jumper? code-compass? profiling infrastructure?) -- Go deep on it -- Accept that other stuff will wait -- Be honest that it's for learning/fun, not necessity - -*My read:* You're mostly problem-solving, but you get distracted by shiny tools. - -*My advice:* Fix the pain points first (TIER 1 & 2). Then build one fun project. Then reassess. - -** gptel vs MCP: Specific Guidance - -You asked: "I need to either build a bunch of tools or install an MCP server. Which way to go?" - -*Answer: Build gptel functions. Here's why:* - -*** Why gptel + elisp Functions - -*Advantages:* -1. *Simpler:* Just elisp functions, no external server -2. *Integrated:* Native Emacs, shares your environment -3. *Debuggable:* Use edebug, step through code -4. *Maintainable:* You understand elisp, you wrote the config -5. *Flexible:* Can use any elisp API (org, dired, magit, etc.) -6. *Portable:* Works on any system with Emacs + gptel - -*What you can build:* -#+begin_src elisp -;; Example gptel tools -(defun cj/gptel-add-project-context () - "Add project README and key files to gptel context." - (interactive) - (let ((readme (expand-file-name "README.md" (project-root)))) - (when (file-exists-p readme) - (cj/gptel-add-file readme)))) - -(defun cj/gptel-analyze-function-at-point () - "Send function at point to LLM for analysis." - (interactive) - (let ((func (elisp--fnsym-in-current-sexp))) - (gptel-send (format "Analyze this function: %s" (symbol-function func))))) - -(defun cj/gptel-git-diff-review () - "Send current git diff to LLM for review." - (interactive) - (let ((diff (shell-command-to-string "git diff HEAD"))) - (gptel-send (concat "Review this diff:\n\n" diff)))) -#+end_src - -*This is much simpler than MCP and covers 90% of use cases.* - -*** When MCP Makes Sense - -*Only use MCP if:* -1. You want to share tools across multiple applications (VS Code + Emacs + CLI) -2. You need sandboxed execution for security -3. You're building reusable infrastructure for others -4. You want to use existing MCP servers someone else maintains - -*For Emacs-only workflows: Use gptel + elisp.* - -*** Recommended Approach - -1. Start with gptel + simple elisp functions -2. Build 3-5 useful commands (add-file, analyze-function, git-diff, org-context) -3. Use them for a month -4. If you find yourself wanting features MCP provides, reassess - -*Don't build MCP infrastructure speculatively.* Build it when gptel + elisp isn't enough. - -** Final Thoughts: Adjusted Priorities - -Your original document was thorough and well-analyzed. My additions: - -1. *Ruthless prioritization* - 4 tiers instead of flat list -2. *Observability first* - Can't optimize what you can't measure -3. *Reality checks* - Some "nice to haves" are rabbit holes -4. *Concrete implementations* - Not just "do this" but "here's the code" -5. *Hard questions* - Before building, ask "why?" and "for whom?" - -*Your config is good.* The issues are normal for a mature system. - -*Focus on:* -1. Fix the bugs (TIER 1) -2. Build profiling infrastructure (TIER 2) -3. Pick ONE fun project (reveal.js or difftastic) -4. Ignore everything else for now - -*Then live with it for a month* and see what still bothers you. - -You're thoughtful, skilled, and self-aware. You'll make good decisions. Just don't try to do everything at once. - -Good luck! 🚀 diff --git a/modules/auth-config.el b/modules/auth-config.el index 6b8a8ddb..2b52087e 100644 --- a/modules/auth-config.el +++ b/modules/auth-config.el @@ -24,9 +24,11 @@ :ensure nil ;; built in :demand t ;; load this package immediately :config - (setenv "GPG_AGENT_INFO" nil) ;; disassociate with external gpg agent - (setq auth-sources `(,authinfo-file)) ;; use authinfo.gpg (see user-constants.el) - (setq auth-source-debug t)) ;; echo debug info to Messages + ;; USE gpg-agent for passphrase caching (400-day cache from gpg-agent.conf) + ;; (setenv "GPG_AGENT_INFO" nil) ;; DISABLED: was preventing gpg-agent cache + (setq auth-sources `(,authinfo-file)) ;; use authinfo.gpg (see user-constants.el) + (setq auth-source-debug t) ;; echo debug info to Messages + (setq auth-source-cache-expiry 86400)) ;; cache decrypted credentials for 24 hours ;; ----------------------------- Easy PG Assistant ----------------------------- ;; Key management, cryptographic operations on regions and files, dired @@ -40,5 +42,79 @@ ;; (setq epa-pinentry-mode 'loopback) ;; emacs request passwords in minibuffer (setq epg-gpg-program "gpg2")) ;; force use gpg2 (not gpg v.1) +;; ---------------------------------- Plstore ---------------------------------- +;; Encrypted storage used by oauth2-auto for Google Calendar tokens. +;; CRITICAL: Enable passphrase caching to prevent password prompts every 10 min. + +(use-package plstore + :ensure nil ;; built-in + :demand t + :config + ;; Cache passphrase indefinitely (relies on gpg-agent for actual caching) + (setq plstore-cache-passphrase-for-symmetric-encryption t) + ;; Allow gpg-agent to cache the passphrase (400 days per gpg-agent.conf) + (setq plstore-encrypt-to nil)) ;; Use symmetric encryption, not key-based + +;; ------------------------ Authentication Reset Utility ----------------------- + +(defun cj/reset-auth-cache (&optional include-gpg-agent) + "Reset authentication caches when wrong password was entered. + +By default, only clears Emacs-side caches (auth-source, EPA file +handler) and leaves gpg-agent's long-term cache intact. This preserves +your 400-day cache for GPG and SSH passphrases. + +With prefix argument INCLUDE-GPG-AGENT (\\[universal-argument]), also +clears gpg-agent's password cache. Use this when gpg-agent itself has +cached an incorrect password. + +Clears: +1. auth-source cache (Emacs-level credential cache) +2. EPA file handler cache (encrypted file cache) +3. gpg-agent cache (only if INCLUDE-GPG-AGENT is non-nil) + +Use this when you see errors like: + - \"Bad session key\" + - \"Decryption failed\" + - GPG repeatedly using wrong cached password" + (interactive "P") + (message "Resetting authentication caches...") + + ;; Clear auth-source cache (Emacs credential cache) + (auth-source-forget-all-cached) + + ;; Clear EPA file handler cache + (when (fboundp 'epa-file-clear-cache) + (epa-file-clear-cache)) + + ;; Only clear gpg-agent cache if explicitly requested + (if include-gpg-agent + (let ((result (shell-command "echo RELOADAGENT | gpg-connect-agent"))) + (if (zerop result) + (message "✓ Emacs and gpg-agent caches cleared. Next access will prompt for password.") + (message "⚠ Warning: Failed to clear gpg-agent cache"))) + (message "✓ Emacs caches cleared. GPG/SSH passphrases preserved for session."))) + +(defun cj/kill-gpg-agent () + "Force kill gpg-agent (it will restart automatically on next use). + +This is a more aggressive reset than `cj/reset-auth-cache'. Use this +when gpg-agent is stuck or behaving incorrectly. + +The gpg-agent will automatically restart on the next GPG operation." + (interactive) + (let ((result (shell-command "gpgconf --kill gpg-agent"))) + (if (zerop result) + (message "✓ gpg-agent killed. It will restart automatically on next use.") + (message "⚠ Warning: Failed to kill gpg-agent")))) + +;; Keybindings +(with-eval-after-load 'keybindings + (keymap-set cj/custom-keymap "A" #'cj/reset-auth-cache)) + +(with-eval-after-load 'which-key + (which-key-add-key-based-replacements + "C-; A" "reset auth cache")) + (provide 'auth-config) ;;; auth-config.el ends here. diff --git a/modules/config-utilities.el b/modules/config-utilities.el index 32018371..2af3effa 100644 --- a/modules/config-utilities.el +++ b/modules/config-utilities.el @@ -33,8 +33,7 @@ "C-c d i b" "info build" "C-c d i p" "info packages" "C-c d i f" "info features" - "C-c d r" "reload init" - "C-c d a" "reset auth cache")) + "C-c d r" "reload init")) ;;; --------------------------------- Profiling --------------------------------- @@ -283,15 +282,6 @@ Recompile natively when supported, otherwise fall back to byte compilation." (load-file user-init-file)) (keymap-set cj/debug-config-keymap "r" 'cj/reload-init-file) -;; ----------------------------- Reset-Auth-Sources ---------------------------- - -(defun cj/reset-auth-cache () - "Clear Emacs auth-source cache." - (interactive) - (auth-source-forget-all-cached) - (message "Emacs auth-source cache cleared.")) -(keymap-set cj/debug-config-keymap "a" 'cj/reset-auth-cache) - ;; ------------------------ Validate Org Agenda Entries ------------------------ (defun cj/validate-org-agenda-timestamps () diff --git a/modules/dirvish-config.el b/modules/dirvish-config.el index 441ff61b..b10c97f0 100644 --- a/modules/dirvish-config.el +++ b/modules/dirvish-config.el @@ -8,15 +8,15 @@ ;; ediff, playlist creation, path copying, and external file manager integration. ;; ;; Key Bindings: -;; - d: Duplicate file at point (adds "-copy" before extension) -;; - D: Delete marked files immediately (dired-do-delete) +;; - d: Delete marked files (dired-do-delete) +;; - D: Duplicate file at point (adds "-copy" before extension) ;; - g: Quick access menu (jump to predefined directories) ;; - G: Search with deadgrep in current directory ;; - f: Open system file manager in current directory ;; - o/O: Open file with xdg-open/custom command ;; - l: Copy file path (project-relative or home-relative) ;; - L: Copy absolute file path -;; - P: Create M3U playlist from marked audio files +;; - P: Copy file path (same as 'l', replaces dired-do-print) ;; - M-D: DWIM menu (context actions for files) ;; - TAB: Toggle subtree expansion ;; - F11: Toggle sidebar view @@ -120,9 +120,9 @@ Filters for audio files, prompts for the playlist name, and saves the resulting (setq dired-listing-switches "-l --almost-all --human-readable --group-directories-first") (setq dired-dwim-target t) (setq dired-clean-up-buffers-too t) ;; offer to kill buffers associated deleted files and dirs - (setq dired-clean-confirm-killing-deleted-buffers t) ;; don't ask; just kill buffers associated with deleted files - (setq dired-recursive-copies (quote always)) ;; “always” means no asking - (setq dired-recursive-deletes (quote top))) ;; “top” means ask once + (setq dired-clean-confirm-killing-deleted-buffers nil) ;; don't ask; just kill buffers associated with deleted files + (setq dired-recursive-copies (quote always)) ;; "always" means no asking + (setq dired-recursive-deletes (quote top))) ;; "top" means ask once ;; note: disabled as it prevents marking and moving files to another directory ;; (setq dired-kill-when-opening-new-dired-buffer t) ;; don't litter by leaving buffers when navigating directories @@ -322,13 +322,14 @@ regardless of what file or subdirectory the point is on." ("M-p" . dirvish-peek-toggle) ("M-s" . dirvish-setup-menu) ("TAB" . dirvish-subtree-toggle) - ("d" . cj/dirvish-duplicate-file) + ("d" . dired-do-delete) + ("D" . cj/dirvish-duplicate-file) ("f" . cj/dirvish-open-file-manager-here) ("g" . dirvish-quick-access) ("o" . cj/xdg-open) ("O" . cj/open-file-with-command) ; Prompts for command to run ("r" . dirvish-rsync) - ("P" . cj/dired-create-playlist-from-marked) + ("P" . cj/dired-copy-path-as-kill) ("s" . dirvish-quicksort) ("v" . dirvish-vc-menu) ("y" . dirvish-yank-menu))) diff --git a/modules/org-gcal-config.el b/modules/org-gcal-config.el index 28cc1933..97e8446a 100644 --- a/modules/org-gcal-config.el +++ b/modules/org-gcal-config.el @@ -165,9 +165,8 @@ Useful after changing `cj/org-gcal-sync-interval-minutes'." (setq org-gcal-managed-update-existing-mode "gcal") ;; GCal wins on conflicts :config - ;; Enable plstore passphrase caching after org-gcal loads - (require 'plstore) - (setq plstore-cache-passphrase-for-symmetric-encryption t) + ;; Plstore caching is now configured globally in auth-config.el + ;; to ensure it loads before org-gcal needs it ;; set org-gcal timezone based on system timezone (setq org-gcal-local-timezone (cj/detect-system-timezone)) diff --git a/modules/system-utils.el b/modules/system-utils.el index 2b39d862..e9686777 100644 --- a/modules/system-utils.el +++ b/modules/system-utils.el @@ -186,13 +186,22 @@ Logs output and exit code to buffer *external-open.log*." ;;; -------------------------- Scratch Buffer Happiness ------------------------- (defvar scratch-emacs-version-and-system - (concat ";; Emacs " emacs-version + (concat "# Emacs " emacs-version " on " system-configuration ".\n")) (defvar scratch-greet - (concat ";; Emacs ♥ you, " user-login-name ". Happy Hacking!\n\n")) + (concat "# Emacs ♥ you, " user-login-name ". Happy Hacking!\n\n")) (setopt initial-scratch-message (concat scratch-emacs-version-and-system scratch-greet)) +;; Set scratch buffer to org-mode +(setopt initial-major-mode 'org-mode) + +;; Move cursor to end of scratch buffer on startup +(add-hook 'emacs-startup-hook + (lambda () + (when (string= (buffer-name) "*scratch*") + (goto-char (point-max))))) + ;;; --------------------------------- Dictionary -------------------------------- (use-package quick-sdcv diff --git a/modules/transcription-config.el b/modules/transcription-config.el index 13b9bbce..fd2f4aaa 100644 --- a/modules/transcription-config.el +++ b/modules/transcription-config.el @@ -5,19 +5,23 @@ ;;; Commentary: ;; -;; Audio transcription workflow using OpenAI Whisper (API or local). +;; Audio transcription workflow with multiple backend options. ;; ;; USAGE: ;; In dired: Press `T` on an audio file to transcribe ;; Anywhere: M-x cj/transcribe-audio ;; View active: M-x cj/transcriptions-buffer +;; Switch backend: C-; T b (or M-x cj/transcription-switch-backend) ;; ;; OUTPUT FILES: ;; audio.m4a → audio.txt (transcript) ;; → audio.log (process logs, conditionally kept) ;; ;; BACKENDS: -;; - 'openai-api: Fast cloud transcription (requires OPENAI_API_KEY) +;; - 'openai-api: Fast cloud transcription +;; API key retrieved from authinfo.gpg (machine api.openai.com) +;; - 'assemblyai: Cloud transcription with speaker diarization +;; API key retrieved from authinfo.gpg (machine api.assemblyai.com) ;; - 'local-whisper: Local transcription (requires whisper installed) ;; ;; NOTIFICATIONS: @@ -33,12 +37,14 @@ (require 'dired) (require 'notifications) +(require 'auth-source) ;; ----------------------------- Configuration --------------------------------- -(defvar cj/transcribe-backend 'local-whisper +(defvar cj/transcribe-backend 'assemblyai "Transcription backend to use. - `openai-api': Fast cloud transcription via OpenAI API +- `assemblyai': Cloud transcription with speaker diarization via AssemblyAI - `local-whisper': Local transcription using installed Whisper") (defvar cj/transcription-keep-log-when-done nil @@ -83,9 +89,36 @@ SUCCESS-P indicates whether transcription succeeded." "Return absolute path to transcription script based on backend." (let ((script-name (pcase cj/transcribe-backend ('openai-api "oai-transcribe") + ('assemblyai "assemblyai-transcribe") ('local-whisper "local-whisper")))) (expand-file-name (concat "scripts/" script-name) user-emacs-directory))) +(defun cj/--get-openai-api-key () + "Retrieve OpenAI API key from authinfo.gpg. +Expects entry in authinfo.gpg: + machine api.openai.com login api password sk-... +Returns the API key string, or nil if not found." + (when-let* ((auth-info (car (auth-source-search + :host "api.openai.com" + :require '(:secret)))) + (secret (plist-get auth-info :secret))) + (if (functionp secret) + (funcall secret) + secret))) + +(defun cj/--get-assemblyai-api-key () + "Retrieve AssemblyAI API key from authinfo.gpg. +Expects entry in authinfo.gpg: + machine api.assemblyai.com login api password <key> +Returns the API key string, or nil if not found." + (when-let* ((auth-info (car (auth-source-search + :host "api.assemblyai.com" + :require '(:secret)))) + (secret (plist-get auth-info :secret))) + (if (functionp secret) + (funcall secret) + secret))) + ;; ---------------------------- Process Management ----------------------------- (defun cj/--notify (title message &optional urgency) @@ -125,14 +158,28 @@ Returns the process object." (format "Audio file: %s\n" audio-file) (format "Script: %s\n\n" script))) - ;; Start process - (let ((process (make-process - :name process-name - :buffer (get-buffer-create buffer-name) - :command (list script audio-file) - :sentinel (lambda (proc event) - (cj/--transcription-sentinel proc event audio-file txt-file log-file)) - :stderr log-file))) + ;; Start process with environment + (let* ((process-environment + ;; Add API key to environment based on backend + (pcase cj/transcribe-backend + ('openai-api + (if-let ((api-key (cj/--get-openai-api-key))) + (cons (format "OPENAI_API_KEY=%s" api-key) + process-environment) + (user-error "OpenAI API key not found in authinfo.gpg for host api.openai.com"))) + ('assemblyai + (if-let ((api-key (cj/--get-assemblyai-api-key))) + (cons (format "ASSEMBLYAI_API_KEY=%s" api-key) + process-environment) + (user-error "AssemblyAI API key not found in authinfo.gpg for host api.assemblyai.com"))) + (_ process-environment))) + (process (make-process + :name process-name + :buffer (get-buffer-create buffer-name) + :command (list script audio-file) + :sentinel (lambda (proc event) + (cj/--transcription-sentinel proc event audio-file txt-file log-file)) + :stderr log-file))) ;; Track transcription (push (list process audio-file (current-time) 'running) cj/transcriptions-list) @@ -298,6 +345,21 @@ Uses backend specified by `cj/transcribe-backend'." (kill-process process) (message "Killed transcription process"))) +;;;###autoload +(defun cj/transcription-switch-backend () + "Switch transcription backend. +Prompts with completing-read to select from available backends." + (interactive) + (let* ((backends '(("assemblyai" . assemblyai) + ("openai-api" . openai-api) + ("local-whisper" . local-whisper))) + (current (symbol-name cj/transcribe-backend)) + (prompt (format "Transcription backend (current: %s): " current)) + (choice (completing-read prompt backends nil t)) + (new-backend (alist-get choice backends nil nil #'string=))) + (setq cj/transcribe-backend new-backend) + (message "Transcription backend: %s" choice))) + ;; ------------------------------- Dired Integration --------------------------- (with-eval-after-load 'dired @@ -311,16 +373,18 @@ Uses backend specified by `cj/transcribe-backend'." (defvar-keymap cj/transcribe-map :doc "Keymap for transcription operations" "a" #'cj/transcribe-audio + "b" #'cj/transcription-switch-backend "v" #'cj/transcriptions-buffer "k" #'cj/transcription-kill) -(keymap-set cj/custom-keymap "t" cj/transcribe-map) +(keymap-set cj/custom-keymap "T" cj/transcribe-map) (with-eval-after-load 'which-key (which-key-add-key-based-replacements - "C-; t" "transcription menu" - "C-; t a" "transcribe audio" - "C-; t v" "view transcriptions" - "C-; t k" "kill transcription")) + "C-; T" "transcription menu" + "C-; T a" "transcribe audio" + "C-; T b" "switch backend" + "C-; T v" "view transcriptions" + "C-; T k" "kill transcription")) (provide 'transcription-config) ;;; transcription-config.el ends here diff --git a/modules/video-audio-recording.el b/modules/video-audio-recording.el index c714a0a6..45bab267 100644 --- a/modules/video-audio-recording.el +++ b/modules/video-audio-recording.el @@ -4,7 +4,7 @@ ;;; Commentary: ;; Use ffmpeg to record desktop video or just audio. ;; with audio from mic and audio from default audio sink -;; Also supports audio-only recording in Opus format. +;; Audio recordings use M4A/AAC format for best compatibility. ;; ;; Note: video-recordings-dir and audio-recordings-dir are defined ;; (and directory created) in user-constants.el @@ -311,16 +311,16 @@ Otherwise use the default location in `audio-recordings-dir'." (system-device (cdr devices)) (location (expand-file-name directory)) (name (format-time-string "%Y-%m-%d-%H-%M-%S")) - (filename (expand-file-name (concat name ".opus") location)) + (filename (expand-file-name (concat name ".m4a") location)) (ffmpeg-command (format (concat "ffmpeg " "-f pulse -i %s " "-ac 1 " "-f pulse -i %s " - "-ac 2 " - "-filter_complex \"[0:a]volume=%.1f[mic];[1:a]volume=%.1f[sys];[mic][sys]amerge=inputs=2\" " - "-c:a libopus " - "-b:a 96k " + "-ac 1 " + "-filter_complex \"[0:a]volume=%.1f[mic];[1:a]volume=%.1f[sys];[mic][sys]amerge=inputs=2[out];[out]pan=mono|c0=0.5*c0+0.5*c1\" " + "-c:a aac " + "-b:a 64k " "%s") mic-device system-device diff --git a/scripts/assemblyai-transcribe b/scripts/assemblyai-transcribe new file mode 100755 index 00000000..22cbf538 --- /dev/null +++ b/scripts/assemblyai-transcribe @@ -0,0 +1,134 @@ +#!/usr/bin/env bash +# assemblyai-transcribe - Transcribe audio files using AssemblyAI API with speaker diarization +# Usage: assemblyai-transcribe <audio-file> [language] +# +# Requires: ASSEMBLYAI_API_KEY environment variable +# Language: en, es, fr, etc. (default: en) +# Features: Speaker diarization (up to 50 speakers) + +set -euo pipefail + +# Parse arguments +AUDIO="${1:-}" +LANG="${2:-en}" + +# Validate arguments +if [[ -z "$AUDIO" ]]; then + echo "Usage: assemblyai-transcribe <audio-file> [language]" >&2 + echo "Example: assemblyai-transcribe meeting.m4a en" >&2 + exit 1 +fi + +if [[ ! -f "$AUDIO" ]]; then + echo "Error: Audio file not found: $AUDIO" >&2 + exit 1 +fi + +# Check API key is set +if [[ -z "${ASSEMBLYAI_API_KEY:-}" ]]; then + echo "Error: ASSEMBLYAI_API_KEY environment variable not set" >&2 + exit 1 +fi + +# Check curl is available +if ! command -v curl &> /dev/null; then + echo "Error: curl command not found" >&2 + exit 1 +fi + +# Check jq is available (for JSON parsing) +if ! command -v jq &> /dev/null; then + echo "Error: jq command not found (required for JSON parsing)" >&2 + echo "Install with: sudo pacman -S jq" >&2 + exit 1 +fi + +API_BASE="https://api.assemblyai.com/v2" + +# Step 1: Upload audio file +echo "Uploading audio file..." >&2 +UPLOAD_RESPONSE=$(curl -s -X POST "${API_BASE}/upload" \ + -H "Authorization: ${ASSEMBLYAI_API_KEY}" \ + --data-binary "@${AUDIO}") + +UPLOAD_URL=$(echo "$UPLOAD_RESPONSE" | jq -r '.upload_url') + +if [[ -z "$UPLOAD_URL" ]] || [[ "$UPLOAD_URL" == "null" ]]; then + echo "Error: Failed to upload audio file" >&2 + echo "$UPLOAD_RESPONSE" >&2 + exit 1 +fi + +echo "Upload complete. Submitting transcription..." >&2 + +# Step 2: Submit transcription request with speaker labels +TRANSCRIPT_REQUEST=$(cat <<EOF +{ + "audio_url": "${UPLOAD_URL}", + "language_code": "${LANG}", + "speech_model": "universal", + "speaker_labels": true +} +EOF +) + +TRANSCRIPT_RESPONSE=$(curl -s -X POST "${API_BASE}/transcript" \ + -H "Authorization: ${ASSEMBLYAI_API_KEY}" \ + -H "Content-Type: application/json" \ + -d "$TRANSCRIPT_REQUEST") + +TRANSCRIPT_ID=$(echo "$TRANSCRIPT_RESPONSE" | jq -r '.id') + +if [[ -z "$TRANSCRIPT_ID" ]] || [[ "$TRANSCRIPT_ID" == "null" ]]; then + echo "Error: Failed to submit transcription" >&2 + echo "$TRANSCRIPT_RESPONSE" >&2 + exit 1 +fi + +echo "Transcription job submitted (ID: ${TRANSCRIPT_ID})" >&2 +echo "Waiting for completion..." >&2 + +# Step 3: Poll for completion +STATUS="queued" +POLL_INTERVAL=3 +MAX_WAIT=1800 # 30 minutes +ELAPSED=0 + +while [[ "$STATUS" == "queued" ]] || [[ "$STATUS" == "processing" ]]; do + if [[ $ELAPSED -ge $MAX_WAIT ]]; then + echo "Error: Transcription timed out after ${MAX_WAIT} seconds" >&2 + exit 1 + fi + + sleep $POLL_INTERVAL + ELAPSED=$((ELAPSED + POLL_INTERVAL)) + + RESULT=$(curl -s -X GET "${API_BASE}/transcript/${TRANSCRIPT_ID}" \ + -H "Authorization: ${ASSEMBLYAI_API_KEY}") + + STATUS=$(echo "$RESULT" | jq -r '.status') + + if [[ "$STATUS" == "processing" ]]; then + echo "Processing... (${ELAPSED}s elapsed)" >&2 + fi +done + +# Check if transcription failed +if [[ "$STATUS" != "completed" ]]; then + ERROR_MSG=$(echo "$RESULT" | jq -r '.error // "Unknown error"') + echo "Error: Transcription failed with status: ${STATUS}" >&2 + echo "Error message: ${ERROR_MSG}" >&2 + exit 1 +fi + +echo "Transcription complete! (${ELAPSED}s total)" >&2 + +# Step 4: Format output with speaker labels +# Extract utterances and format as "Speaker A: text" +echo "$RESULT" | jq -r ' + if .utterances then + .utterances[] | "Speaker \(.speaker): \(.text)" + else + .text + end +' diff --git a/tests/test-transcription-config--transcription-script-path.el b/tests/test-transcription-config--transcription-script-path.el new file mode 100644 index 00000000..a56cb05c --- /dev/null +++ b/tests/test-transcription-config--transcription-script-path.el @@ -0,0 +1,106 @@ +;;; test-transcription-config--transcription-script-path.el --- Tests for cj/--transcription-script-path -*- lexical-binding: t; -*- + +;;; Commentary: +;; Tests for the cj/--transcription-script-path function from transcription-config.el +;; +;; This function returns the absolute path to the transcription script based on +;; the current value of cj/transcribe-backend. + +;;; Code: + +(require 'ert) + +;; Add modules directory to load path +(add-to-list 'load-path (expand-file-name "modules" user-emacs-directory)) + +;; Stub dependencies before loading the module +(defvar cj/custom-keymap (make-sparse-keymap) + "Stub keymap for testing.") + +;; Stub notification function +(unless (fboundp 'notifications-notify) + (defun notifications-notify (&rest _args) + "Stub notification function for testing." + nil)) + +;; Now load the actual production module +(require 'transcription-config) + +;;; Setup and Teardown + +(defun test-transcription-script-path-setup () + "Set up test environment." + ;; Save original backend setting + (setq test-transcription-original-backend cj/transcribe-backend)) + +(defun test-transcription-script-path-teardown () + "Clean up test environment." + ;; Restore original backend setting + (setq cj/transcribe-backend test-transcription-original-backend)) + +;;; Normal Cases + +(ert-deftest test-transcription-config--transcription-script-path-normal-openai-api-returns-oai-transcribe () + "Should return oai-transcribe script path for openai-api backend." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'openai-api) + (let ((result (cj/--transcription-script-path))) + (should (stringp result)) + (should (string-suffix-p "scripts/oai-transcribe" result)) + (should (string-prefix-p (expand-file-name user-emacs-directory) result)))) + (test-transcription-script-path-teardown))) + +(ert-deftest test-transcription-config--transcription-script-path-normal-assemblyai-returns-assemblyai-transcribe () + "Should return assemblyai-transcribe script path for assemblyai backend." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'assemblyai) + (let ((result (cj/--transcription-script-path))) + (should (stringp result)) + (should (string-suffix-p "scripts/assemblyai-transcribe" result)) + (should (string-prefix-p (expand-file-name user-emacs-directory) result)))) + (test-transcription-script-path-teardown))) + +(ert-deftest test-transcription-config--transcription-script-path-normal-local-whisper-returns-local-whisper () + "Should return local-whisper script path for local-whisper backend." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'local-whisper) + (let ((result (cj/--transcription-script-path))) + (should (stringp result)) + (should (string-suffix-p "scripts/local-whisper" result)) + (should (string-prefix-p (expand-file-name user-emacs-directory) result)))) + (test-transcription-script-path-teardown))) + +(ert-deftest test-transcription-config--transcription-script-path-normal-returns-absolute-path () + "Should return absolute path starting with user-emacs-directory." + (test-transcription-script-path-setup) + (unwind-protect + (progn + (setq cj/transcribe-backend 'openai-api) + (let ((result (cj/--transcription-script-path))) + (should (file-name-absolute-p result)) + (should (string-prefix-p "/" result)))) + (test-transcription-script-path-teardown))) + +;;; Boundary Cases + +(ert-deftest test-transcription-config--transcription-script-path-boundary-path-format-consistent () + "Should return paths in consistent format across backends." + (test-transcription-script-path-setup) + (unwind-protect + (let (paths) + (dolist (backend '(openai-api assemblyai local-whisper)) + (setq cj/transcribe-backend backend) + (push (cj/--transcription-script-path) paths)) + ;; All paths should have same structure: <emacs-dir>/scripts/<name> + (should (= (length paths) 3)) + (should (seq-every-p (lambda (p) (string-match-p "/scripts/[^/]+$" p)) paths))) + (test-transcription-script-path-teardown))) + +(provide 'test-transcription-config--transcription-script-path) +;;; test-transcription-config--transcription-script-path.el ends here @@ -17,7 +17,7 @@ If the answer is "no" to all five → DON'T ADD IT. V2MOM is located at: [[file:docs/emacs-config-v2mom.org][emacs-config-v2mom.org]] Research/ideas that don't serve vision: [[file:docs/someday-maybe.org][someday-maybe.org]] -* Method 1: Make Using Emacs Frictionless [7/13] +* Method 1: Make Using Emacs Frictionless [8/14] ** DONE [#A] Remove network check from startup (saves 1+ seconds) CLOSED: [2025-10-31 Fri] @@ -194,6 +194,52 @@ Generate transcript (once transcription workflow exists). Daily workflow improvement. +** TODO [#A] Fix Google Calendar password prompts every 15 minutes + +IRRITANT: gcal-sync triggers password prompts approximately every 15 minutes, +interrupting workflow and breaking focus. This defeats the purpose of having +passphrase caching configured. + +**Current Setup:** +- GPG agent configured with 400-day cache (gpg-agent.conf): + - default-cache-ttl 34560000 + - max-cache-ttl 34560000 + - allow-loopback-pinentry enabled +- Plstore caching enabled (auth-config.el:54): + - plstore-cache-passphrase-for-symmetric-encryption t + - plstore-encrypt-to nil (symmetric encryption) +- Auth-source cache: 24 hours (auth-config.el:31) +- Auto-sync interval: 30 minutes (org-gcal-config.el:50) + +**Problem:** +Despite proper GPG agent caching, oauth2-auto.plist prompts for passphrase +every ~15 minutes during gcal-sync operations. This suggests: +1. plstore may not be using GPG agent cache properly for symmetric encryption +2. oauth2-auto token refresh might be bypassing cache +3. EPinentry mode may need explicit configuration (currently commented out) + +**Goal:** +Passphrase should be entered ONCE per Emacs session, then cached until Emacs +closes. No interruptions during normal work. + +**Investigation Paths:** +1. Check if oauth2-auto respects plstore passphrase caching +2. Investigate plstore symmetric encryption cache behavior with GPG agent +3. Test enabling epa-pinentry-mode 'loopback (auth-config.el:42) +4. Check oauth2-auto token refresh cycle vs password prompt timing +5. Consider oauth2-auto configuration options for token persistence +6. Review org-gcal or oauth2-auto issues for similar problems + +**Files:** +- modules/auth-config.el (plstore and GPG configuration) +- modules/org-gcal-config.el (org-gcal and oauth2-auto setup) +- ~/.gnupg/gpg-agent.conf (GPG agent cache settings) +- oauth2-auto.plist (encrypted OAuth tokens - prompts every access?) + +**Related:** +This violates the "Frictionless" value - interruptions every 15 minutes during +calendar sync breaks concentration and workflow momentum. + ** TODO [#B] Optimize org-agenda performance using built-in profiler THE BOTTLENECK. Currently 30+ seconds, target < 5 seconds. @@ -295,13 +341,17 @@ CLOSED: [2025-11-03 Sun] Already using prescient with vertico. Extend to Corfu after migration. -* Method 3: Make *Fixing* Emacs Frictionless [0/5] +* Method 3: Make *Fixing* Emacs Frictionless [1/4] ** TODO [#B] Build debug-profiling.el module Reusable profiling infrastructure for any future performance work. -** TODO [#C] Document localrepo limitations (treesitter grammars not included) +** TODO [#C] Build localrepo out (package snapshot system) + +Repeatable installs and safe rollbacks. + +*** TODO [#C] Document localrepo limitations (treesitter grammars not included) .localrepo only contains packages from package.el archives. Treesitter grammars are downloaded separately by treesit-auto on first use. @@ -311,10 +361,6 @@ For true offline reproducibility, need to cache treesitter grammars separately. Complex workflow testing capability. -** TODO [#C] Build localrepo out (package snapshot system) - -Repeatable installs and safe rollbacks. - ** DONE [#C] Integrate difftastic (structural diffs) CLOSED: [2025-11-03 Sun] @@ -333,6 +379,29 @@ CLOSED: [2025-11-03 Sun] Result: Better diffs everywhere - ediff for interactive buffer comparison, difftastic for understanding git changes. +** TODO [#C] Remove orphaned dwim-shell-security tests and unused production code + +Why: 12 tests in test-dwim-shell-security.el fail because the functions they test +are inside a use-package :config block (dwim-shell-config.el:101-108) that only +loads when the dwim-shell-command package is available. During batch testing, +the package isn't loaded, so functions are never defined (void-function errors). + +These are PDF password protection and ZIP encryption functions that likely have +never been used in practice - they're placeholder code from initial setup. + +What to delete: +1. Test file: tests/test-dwim-shell-security.el (12 failing tests) +2. Production functions in modules/dwim-shell-config.el (lines ~302-347): + - cj/dwim-shell-commands-pdf-password-protect (lines 302-324) + - cj/dwim-shell-commands-pdf-password-unprotect (lines 326-347) + - cj/dwim-shell-commands-create-encrypted-zip (search for it) + - cj/dwim-shell-commands-remove-zip-encryption (search for it) + +After deletion: Run "make test-all" to confirm 18 failures → 6 failures +(only benchmark performance tests remain, which are environment-dependent). + +Aligns with: Reducing test failures from 18 to 6, cleaning up unused code. + * Method 4: Contribute to the Emacs Ecosystem [0/4] ** TODO [#C] Set up package-lint for elisp linting (chime, org-msg, wttrin) @@ -381,3 +450,4 @@ Review this inbox, cancel stale items, keep < 20 active. Track in calendar. Can't research next thing until current thing is implemented. * Emacs Config Inbox +** TODO cj/flyspell-then-abbrev loses keybinding in scratch org-mode buffer |