path: root/TESTING.org

#+TITLE: Chime Test Suite
#+AUTHOR: Craig Jennings
#+DATE: 2026-04-04

Quick reference for running and writing tests in the Chime project.

#+begin_quote
*Contents:* [[Quick start]] | [[Running tests]] | [[Writing tests]] | [[Test infrastructure]] | [[Key patterns]] | [[Important notes]] | [[Dependencies]] | [[Test inventory]]
#+end_quote

* Quick start

#+begin_src sh
# From the project root:
make test                              # Run all tests (unit + integration)
make test-unit                         # Run unit tests only
make test-integration                  # Run integration tests only

# From the tests/ directory (more options):
make test-file FILE=validate           # Run tests in one file (fuzzy match)
make test-one TEST=pilot               # Run a single test (fuzzy match)
make count                             # Count tests per file
#+end_src

* Running tests

All test logic lives in =tests/Makefile=. The root Makefile delegates to it, so all commands work from either the project root or the =tests/= directory.

| Command | Purpose |
|---------+---------|
| =make test= | Run all tests (unit + integration) |
| =make test-unit= | Run unit tests only |
| =make test-integration= | Run integration tests only (=test-integration-*.el=) |
| =make test-file FILE=overdue= | Run tests in one file (fuzzy match) |
| =make test-one TEST=pilot= | Run a single test (fuzzy match) |
| =make test-name TEST=pattern= | Run tests matching an ERT name pattern |
| =make count= | Count tests per file, sorted by count |
| =make list= | List all test names |
| =make validate= | Check parentheses balance in all files |
| =make lint= | Run elisp-lint on all files |
| =make check-deps= | Verify all dependencies are installed |
| =make clean= | Remove byte-compiled files and logs |
| =make help= | Show all available commands |

Each test file runs in its own Emacs process for isolation. Integration tests are identified by the =test-integration-= prefix. Dependencies are auto-detected from =~/.emacs.d/elpa/=. Override with =EMACS= or =ELPA_DIR= environment variables.

** Examples

#+begin_src sh
# Run one file by fuzzy match
make test-file FILE=validate-configuration
# Finds: test-chime-validate-configuration.el

# Run one test by fuzzy match
make test-one TEST=pilot
# Finds: test-chime-validate-configuration-normal-valid-config-returns-nil

# Run tests matching an ERT name pattern
make test-name TEST="test-chime-check-*"

# Use a specific Emacs version
make EMACS=emacs29 test
#+end_src

* Writing tests

** File structure

Every test file loads =test-bootstrap.el=, which sets up packages, dependencies, and chime itself:

#+begin_src elisp
;;; test-chime-FEATURE.el --- Tests for FEATURE -*- lexical-binding: t; -*-

;; Copyright (C) 2026 Craig Jennings
;; Author: Craig Jennings <c@cjennings.net>
;; License: GPL-3.0-or-later

;;; Commentary:
;; Unit tests for FEATURE.

;;; Code:

(require 'test-bootstrap (expand-file-name "test-bootstrap.el"))
(require 'testutil-time)     ; if using time utilities
(require 'testutil-events)   ; if using event utilities

;;; Normal Cases

(ert-deftest test-chime-FEATURE-normal-description ()
  "Descriptive docstring."
  (should (equal expected (function-under-test input))))

;;; Boundary Cases
;;; Error Cases

(provide 'test-chime-FEATURE)
;;; test-chime-FEATURE.el ends here
#+end_src

** Naming convention

#+begin_example
test-chime-FEATURE-CATEGORY-description
             |        |
             |        +-- normal, boundary, error
             +----------- function or module name
#+end_example

Examples:
- =test-chime-get-tags-normal-single-tag=
- =test-chime-extract-time-boundary-midnight=
- =test-chime-validate-configuration-error-missing-deps=

** Test categories

Tests are split into three categories:

- *Normal*: Standard inputs, expected use cases
- *Boundary*: Empty inputs, nil values, single-element lists, unicode, max values
- *Error*: Invalid inputs, missing dependencies, malformed data

* Test infrastructure

** test-bootstrap.el

Loads =ert=, =dash=, =alert=, =async=, =org-agenda=, and =chime.el= so test files don't repeat that boilerplate. For debug tests, set =chime-debug= before requiring:

#+begin_src elisp
(setq chime-debug t)
(require 'test-bootstrap (expand-file-name "test-bootstrap.el"))
#+end_src

** testutil-time.el --- Dynamic timestamps

Generates timestamps relative to "now" so tests never expire. The base time is always =current-time + 30 days= at =10:00 AM= for consistency.

*** Core functions

| Function | Purpose | Example |
|----------+---------+---------|
| =test-time-now= | Base time (today+30 at 10:00) | =(test-time-now)= |
| =test-time-at= | Relative to now (days, hours, min) | =(test-time-at 0 2 0)= -> 2h from now |
| =test-time-today-at= | Today at specific time | =(test-time-today-at 14 30)= -> 2:30 PM |
| =test-time-tomorrow-at= | Tomorrow at specific time | =(test-time-tomorrow-at 9 0)= |
| =test-time-yesterday-at= | Yesterday at specific time | =(test-time-yesterday-at 17 0)= |
| =test-time-days-from-now= | N days in future | =(test-time-days-from-now 3 14 0)= |
| =test-time-days-ago= | N days in past | =(test-time-days-ago 7)= |

*** Timestamp strings

| Function | Purpose | Output |
|----------+---------+--------|
| =test-timestamp-string= | Org timestamp | =<2026-05-04 Mon 14:00>= |
| =test-timestamp-string TIME t= | All-day timestamp | =<2026-05-04 Mon>= |
| =test-timestamp-range-string= | Date range | =<2026-05-04 Mon>--<2026-05-07 Thu>= |
| =test-timestamp-repeating= | Repeating event | =<2026-05-04 Mon +1w>= |

*** Mock macro

#+begin_src elisp
(with-test-time (test-time-now)
  ;; current-time is mocked to return the test base time
  (should (equal (current-time) (test-time-now))))
#+end_src

** testutil-events.el --- Event creation

Builds org events and event data structures for tests.

*** Creating org content

#+begin_src elisp
;; Single timed event
(test-create-org-event "Meeting" (test-time-now) t)
;; => "* TODO Meeting\nSCHEDULED: <2026-05-04 Mon 10:00>\n"

;; All-day event
(test-create-org-event "Birthday" (test-time-now) nil t)
;; => "* Birthday\n<2026-05-04 Mon>\n"

;; Multiple events
(test-create-org-events
 `(("Meeting" ,(test-time-at 0 2 0) t)
   ("Call" ,(test-time-at 0 4 0) t)))
#+end_src

*** Gathering events from content

#+begin_src elisp
;; Gather all events from org content string
(let ((events (test-gather-events-from-content content)))
  (should (= 2 (length events))))

;; Gather exactly one event (errors if != 1)
(let ((event (test-gather-single-event-from-content content)))
  (should (string= "Meeting" (cdr (assoc 'title event)))))
#+end_src

*** Creating event data directly

#+begin_src elisp
;; Simple event (title, time, optional interval/severity)
(test-make-simple-event "Call" (test-time-now) 5 'high)

;; Full control over data structure
(test-make-event-data "Meeting"
                      (list (cons (test-timestamp-string time) time))
                      '((10 . medium) (0 . high)))
#+end_src

*** Convenience macros

#+begin_src elisp
;; Temp file with auto-cleanup
(with-test-event-file (test-create-org-event "Meeting" (test-time-now))
  (with-current-buffer test-buffer
    (should (search-forward "Meeting" nil t))))

;; Gather events with auto-cleanup
(with-gathered-events (test-create-org-event "Call" (test-time-now))
                      events
  (should (= 1 (length events))))

;; Standard setup/teardown (test base dir lifecycle)
(with-test-setup
  (let ((file (chime-create-temp-test-file)))
    (should (file-exists-p file))))

;; Override chime config for a test
(with-chime-config
  chime-modeline-lookahead-minutes 1440
  chime-tooltip-lookahead-hours 24
  (should (= chime-modeline-lookahead-minutes 1440)))

;; Create org file from event specs
(with-org-event-file
    (("Meeting" event-time t)
     ("Birthday" bday-time nil t))
    org-file
  (setq org-agenda-files (list org-file)))
#+end_src

** testutil-general.el --- File system utilities

Manages test directories and temp files under =~/.temp-chime-tests/=.

| Function | Purpose |
|----------+---------|
| =chime-create-test-base-dir= | Create test root (idempotent) |
| =chime-delete-test-base-dir= | Recursively delete test root |
| =chime-create-temp-test-file= | Create uniquely named temp file |
| =chime-create-temp-test-file-with-content= | Temp file with content |
| =chime-create-test-subdirectory= | Create a subdirectory under test root |
| =chime-create-directory-or-file-ensuring-parents= | Create dir (trailing /) or file |

All paths are sandboxed under =chime-test-base-dir= -- attempts to escape are rejected with an error.

* Key patterns

** Mocking external dependencies

#+begin_src elisp
(cl-letf (((symbol-function 'file-exists-p) (lambda (_) t))
          ((symbol-function 'require) (lambda (_ &optional _ _) t)))
  ;; these functions are mocked only within this scope
  )
#+end_src

** Mocking time

#+begin_src elisp
(with-test-time (test-time-today-at 9 55)
  ;; current-time now returns 9:55 AM
  (should (chime--timestamp-within-interval-p timestamp 10)))
#+end_src

** Overriding config

#+begin_src elisp
(with-chime-config
  chime-sound-file nil
  chime-alert-intervals '((10 . medium))
  (chime--process-notifications events))
#+end_src

* Important notes

1. *Load path*: The Makefile automatically finds dependencies in =~/.emacs.d/elpa/=
2. *Fuzzy matching*: =test-file= and =test-one= support partial names
3. *Test logs*: Output saved to =test-output.log=, =test-file-output.log=, etc. in =tests/=
4. *Mock warnings*: "Redefining 'file-exists-p' might break native compilation" is normal and expected
5. *Dynamic timestamps*: Never hardcode dates -- use =testutil-time.el= functions
6. *Test isolation*: Each test file runs in its own Emacs process

* Dependencies

Required packages (auto-detected by Makefile):
- =dash= (list manipulation)
- =alert= (notifications)
- =async= (async processes)
- =org-agenda= (built-in, events source)

Use =make check-deps= (from =tests/=) to verify all dependencies are installed.

* Test inventory

645 tests across 53 files (as of 2026-04-04).

Top files by count:
| File | Tests |
|------+-------|
| test-convert-org-contacts-birthdays.el | 35 |
| test-chime-sanitize-title.el | 30 |
| test-chime-notification-text.el | 30 |
| test-chime-timestamp-parse.el | 26 |
| test-chime-12hour-format.el | 26 |
| test-chime-time-left.el | 24 |
| test-chime-modeline.el | 20 |
| test-chime-time-utilities.el | 19 |
| test-chime-all-day-events.el | 19 |
| test-chime-update-modeline.el | 18 |
| test-chime-has-timestamp.el | 18 |

Run =make count= from =tests/= for the full breakdown.