diff options
| author | Craig Jennings <c@cjennings.net> | 2026-05-31 13:57:40 -0500 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-05-31 13:57:40 -0500 |
| commit | ee903e4b63257573773e93d10612250e3634cae9 (patch) | |
| tree | 89cc6199b53f445a4c32252c6b982a6c86964e3b | |
| parent | 47ca509e69b6a1472a735a4b9521a952e7434491 (diff) | |
| download | rulesets-ee903e4b63257573773e93d10612250e3634cae9.tar.gz rulesets-ee903e4b63257573773e93d10612250e3634cae9.zip | |
feat(typescript): add coverage-summary to the TypeScript bundle
Last language in the coverage-summary fan-out, after Elisp, Python, and Go. Same kernel: count every source file on disk that's absent from the coverage report as 0% and weight the project number by file, so an untested file stays visible instead of being averaged away.
The script at languages/typescript/claude/scripts/coverage-summary.js parses an Istanbul json-summary report (the coverage-summary.json that c8, Vitest, and Jest all emit), takes per-file statements covered over total, and reports a file-weighted number plus the missing files. It walks the source dir for .ts/.js, skipping test files, declarations, and node_modules. Node built-ins only, so it runs via node with no install, and it doesn't reimplement the per-file table nyc already prints.
Tests are black-box, run with node's own test runner: a temp tree plus a json-summary report, the script invoked via node, output asserted. They cover missing-file detection, all-tracked, test-file and node_modules exclusion, and the missing-report error. make test gained a node --test discovery path for languages/*/tests, guarded so environments without Node skip it cleanly. As with Python, the TypeScript bundle had no gitignore-add.txt, which would have left the script un-gitignored on install, so I added one.
This finishes the fan-out: coverage-summary now ships in all four bundles, each parsing its own tool's report behind the same file-weighted, missing-as-0% kernel. I proved the Go and TypeScript scripts by running them (Go against a live profile, TS against a synthetic report and the CLI). Python and TypeScript weren't run against a live coverage tool, since neither coverage.py nor nyc is installed here, so the first adopter of each should check against a real report.
| -rw-r--r-- | Makefile | 9 | ||||
| -rw-r--r-- | languages/typescript/claude/rules/typescript-testing.md | 17 | ||||
| -rw-r--r-- | languages/typescript/claude/scripts/coverage-summary.js | 169 | ||||
| -rw-r--r-- | languages/typescript/coverage-makefile.txt | 45 | ||||
| -rw-r--r-- | languages/typescript/gitignore-add.txt | 11 | ||||
| -rw-r--r-- | languages/typescript/tests/coverage-summary.test.js | 90 | ||||
| -rw-r--r-- | scripts/tests/install-lang.bats | 10 | ||||
| -rw-r--r-- | todo.org | 13 |
8 files changed, 362 insertions, 2 deletions
@@ -477,6 +477,15 @@ test: ## Run all test suites (pytest + ERT + bats) else \ echo "go test: skipped (go not installed)"; \ fi + @if command -v node >/dev/null 2>&1; then \ + set -e; for d in languages/*/tests; do \ + ls "$$d"/*.test.js >/dev/null 2>&1 || continue; \ + echo "node test: $$d"; \ + ( cd "$$d" && node --test ); \ + done; \ + else \ + echo "node test: skipped (node not installed)"; \ + fi @set -e; for f in .ai/scripts/tests/test-*.el; do \ [ -e "$$f" ] || continue; \ echo "ert: $$(basename "$$f")"; \ diff --git a/languages/typescript/claude/rules/typescript-testing.md b/languages/typescript/claude/rules/typescript-testing.md index bd6933f..31c50fc 100644 --- a/languages/typescript/claude/rules/typescript-testing.md +++ b/languages/typescript/claude/rules/typescript-testing.md @@ -113,6 +113,23 @@ Workflow: invoke `/pairwise-tests` → get a PICT model + generated test matrix → paste the matrix into an `it.each` block. See `testing.md` § Combinatorial Coverage for the general rule and when to skip. +## Measuring Coverage — `make coverage-summary` + +The bundle ships a coverage summary at `.claude/scripts/coverage-summary.js` +and a Makefile fragment (`coverage-makefile.txt`) with `coverage` and +`coverage-summary` targets. After the suite runs under c8 (or Vitest/Jest with +the json-summary reporter) and writes `coverage/coverage-summary.json`, `make +coverage-summary` prints a file-weighted project number and the source files no +test imported. + +The number to watch is that missing-file count. A module no test imports never +appears in the Istanbul report, so a statement-weighted total skips it silently +and the suite looks healthier than it is. The summary counts every `.ts`/`.js` +under the source dir that's absent from the report as 0%, so an untested module +drags the project number down where you can see it. It doesn't reimplement the +per-file table — nyc/c8 already print that. Copy the fragment's targets into +your own Makefile to adopt it; the bundle never edits your Makefile. + ## Mocking Guidelines ### Mock these (external boundaries): diff --git a/languages/typescript/claude/scripts/coverage-summary.js b/languages/typescript/claude/scripts/coverage-summary.js new file mode 100644 index 0000000..72fa78d --- /dev/null +++ b/languages/typescript/claude/scripts/coverage-summary.js @@ -0,0 +1,169 @@ +#!/usr/bin/env node +/* + * Whole-project coverage summary from an Istanbul json-summary report. + * + * Batch helper for `make coverage-summary`. After the test run writes an + * Istanbul json-summary report (nyc / c8 `--reporter=json-summary`, or + * vitest/jest coverage), this prints a file-weighted project number and the + * source files present on disk but absent from the report. + * + * The value here is the missing-file detection: a module no test imports never + * appears in the report, so a statement-weighted total silently skips it and + * the suite looks healthier than it is. This counts such a file as 0% and + * weights the project number by file rather than by statement, so untested + * files stay visible. nyc's own text reporter prints the per-file table, so + * this deliberately doesn't reimplement that. + * + * Node built-ins only (fs, path), so it runs anywhere the bundle lands via + * `node coverage-summary.js`. The json-summary shape it reads is: + * { "total": {...}, "<abs-path>": { "statements": {"total", "covered"}, ... } } + * + * Usage: + * node coverage-summary.js REPORT SOURCE_DIR [PROJECT_ROOT] + * REPORT path to the Istanbul json-summary report + * SOURCE_DIR the directory to account for (walked recursively) + * PROJECT_ROOT defaults to the current directory + */ +"use strict"; + +const fs = require("node:fs"); +const path = require("node:path"); + +const SRC_EXT = new Set([".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs"]); +const SKIP_DIRS = new Set(["node_modules", "dist", "build", "coverage", ".claude", ".git"]); + +function die(msg) { + process.stderr.write(`coverage-summary: ${msg}\n`); + process.exit(1); +} + +function toSlash(p) { + return p.split(path.sep).join("/"); +} + +// parseReport reads an Istanbul json-summary into { absPath: [covered, total] }. +// Uses statements; falls back to lines when statements is absent. +function parseReport(reportPath) { + let raw; + try { + raw = fs.readFileSync(reportPath, "utf8"); + } catch { + die(`coverage report not found: ${reportPath}`); + } + let data; + try { + data = JSON.parse(raw); + } catch (e) { + die(`malformed coverage JSON in ${reportPath}: ${e.message}`); + } + const out = {}; + for (const [key, entry] of Object.entries(data)) { + if (key === "total" || !entry || typeof entry !== "object") continue; + const metric = entry.statements || entry.lines; + if (!metric) continue; + out[path.resolve(key)] = [Number(metric.covered) || 0, Number(metric.total) || 0]; + } + return out; +} + +function underDir(table, sourceRel) { + if (sourceRel === "." || sourceRel === "") return table; + const prefix = sourceRel + "/"; + const out = {}; + for (const [p, v] of Object.entries(table)) { + if (p === sourceRel || p.startsWith(prefix)) out[p] = v; + } + return out; +} + +function isSource(name) { + if (/\.(test|spec)\.[^.]+$/.test(name)) return false; + if (name.endsWith(".d.ts")) return false; + return SRC_EXT.has(path.extname(name)); +} + +// sourceFiles returns project-relative source files under sourceDir (recursive), +// skipping test files, declarations, and non-source directories. +function sourceFiles(sourceDir, root) { + const out = []; + const walk = (dir) => { + let entries; + try { + entries = fs.readdirSync(dir, { withFileTypes: true }); + } catch { + return; + } + for (const e of entries) { + const full = path.join(dir, e.name); + if (e.isDirectory()) { + if (!SKIP_DIRS.has(e.name)) walk(full); + } else if (e.isFile() && isSource(e.name)) { + out.push(toSlash(path.relative(root, full))); + } + } + }; + walk(sourceDir); + return out.sort(); +} + +function missing(tracked, sourceDir, root) { + const have = new Set(Object.keys(tracked)); + return sourceFiles(sourceDir, root).filter((f) => !have.has(f)); +} + +function filePct(covered, total) { + return total > 0 ? (100 * covered) / total : 100; +} + +function projectPct(tracked, miss) { + const total = Object.keys(tracked).length + miss.length; + if (total === 0) return 0; + let score = 0; + for (const [c, t] of Object.values(tracked)) score += filePct(c, t); + return score / total; +} + +function summaryText(tracked, miss, sourceRel) { + const trackedCount = Object.keys(tracked).length; + const total = trackedCount + miss.length; + const pct = projectPct(tracked, miss); + const lines = [ + `Coverage summary for ${sourceRel}`, + "", + `Project coverage: ${pct.toFixed(1)}% (${trackedCount} tracked, ${miss.length} missing, ` + + `${total} total; missing files count as 0%)`, + "", + `Not in coverage report: ${miss.length} file${miss.length === 1 ? "" : "s"}`, + ]; + if (miss.length > 0) { + lines.push("These files had no coverage entry; they count as 0% in project coverage."); + for (const m of [...miss].sort()) lines.push(` ${m}`); + } else { + lines.push("Every source file appears in the coverage report."); + } + lines.push("", "(Per-file table: run nyc's text reporter.)"); + return lines.join("\n") + "\n"; +} + +function main(argv) { + if (argv.length < 2 || argv.length > 3) { + process.stderr.write("usage: coverage-summary.js REPORT SOURCE_DIR [PROJECT_ROOT]\n"); + process.exit(2); + } + const [report, sourceDir] = argv; + const root = path.resolve(argv[2] || "."); + const absSource = path.resolve(sourceDir); + const sourceRel = toSlash(path.relative(root, absSource)) || "."; + + const table = parseReport(report); + // Re-key the report to project-relative paths for comparison with disk files. + const rel = {}; + for (const [abs, v] of Object.entries(table)) rel[toSlash(path.relative(root, abs))] = v; + + const tracked = underDir(rel, sourceRel); + const miss = missing(tracked, absSource, root); + + process.stdout.write("\n" + summaryText(tracked, miss, sourceRel)); +} + +main(process.argv.slice(2)); diff --git a/languages/typescript/coverage-makefile.txt b/languages/typescript/coverage-makefile.txt new file mode 100644 index 0000000..64c7fcd --- /dev/null +++ b/languages/typescript/coverage-makefile.txt @@ -0,0 +1,45 @@ +# TypeScript / JavaScript coverage — Makefile fragment + setup recommendation +# +# This file is owned by the project, not the rulesets bundle. The bundle never +# edits your Makefile. Copy the two targets below into your own Makefile (and +# adjust the variables at the top), then delete this file or keep it as a note. +# +# What you get: +# make coverage runs the suite under c8 and writes an Istanbul +# json-summary report plus c8's own text table +# make coverage-summary prints a file-weighted project number and every +# source file on disk absent from the report, at 0%. +# +# Why the summary matters: a module no test imports never appears in the +# Istanbul report, so a statement-weighted total silently skips it. The summary +# weights by file and counts a missing file as 0%, so untested files stay +# visible. It does not reimplement the per-file table — nyc/c8 already print it. +# +# --------------------------------------------------------------------------- +# Prerequisite: an Istanbul json-summary report. +# +# c8: npx c8 --reporter=json-summary --reporter=text npm test +# vitest: vitest run --coverage --coverage.reporter=json-summary +# jest: jest --coverage --coverageReporters=json-summary +# +# All three write coverage/coverage-summary.json. The summary script needs +# nothing beyond Node's standard library — it parses the JSON. +# --------------------------------------------------------------------------- + +# Variables — adjust to your layout. +NODE ?= node +SOURCE_DIR ?= src +COVERAGE_FILE ?= coverage/coverage-summary.json +# The summary script ships with the bundle under .claude/scripts/ (gitignored). +COVERAGE_SUMMARY ?= .claude/scripts/coverage-summary.js + +coverage: + @npx c8 --reporter=json-summary --reporter=text npm test + @$(MAKE) coverage-summary + +coverage-summary: + @if [ ! -f $(COVERAGE_FILE) ]; then \ + echo "[!] No coverage file at $(COVERAGE_FILE). Run 'make coverage' first."; \ + exit 1; \ + fi + @$(NODE) $(COVERAGE_SUMMARY) $(COVERAGE_FILE) $(SOURCE_DIR) . diff --git a/languages/typescript/gitignore-add.txt b/languages/typescript/gitignore-add.txt new file mode 100644 index 0000000..b8d700a --- /dev/null +++ b/languages/typescript/gitignore-add.txt @@ -0,0 +1,11 @@ +# Claude Code — local tooling, delivered by install/sync, not committed +.claude/ +CLAUDE.md +githooks/ + +# Node + coverage build artifacts (generated) +node_modules/ +coverage/ +dist/ +build/ +*.tsbuildinfo diff --git a/languages/typescript/tests/coverage-summary.test.js b/languages/typescript/tests/coverage-summary.test.js new file mode 100644 index 0000000..b2b6ea3 --- /dev/null +++ b/languages/typescript/tests/coverage-summary.test.js @@ -0,0 +1,90 @@ +// Black-box tests for the TypeScript bundle coverage-summary script. +// +// The script ships into a project's .claude/scripts/ and runs via `node`, so +// these tests exercise the real CLI rather than importing it: build a temp tree +// (source files + an istanbul json-summary report), run the script against it, +// and assert on stdout. Run with `node --test`. +// +// Normal / Boundary / Error coverage at the behavior level: missing-file +// detection, the file-weighted number, all-tracked, ignoring test files and +// node_modules, and a missing-report error. +const { test } = require("node:test"); +const assert = require("node:assert"); +const { execFileSync } = require("node:child_process"); +const fs = require("node:fs"); +const os = require("node:os"); +const path = require("node:path"); + +const SCRIPT = path.resolve(__dirname, "..", "claude", "scripts", "coverage-summary.js"); + +// fixture builds a temp project. sources maps relpath -> contents. report is the +// istanbul json-summary object, with per-file keys given as relpaths that get +// rewritten to absolute paths (matching how istanbul records them). +function fixture(sources, reportRel) { + const root = fs.mkdtempSync(path.join(os.tmpdir(), "tscov-")); + for (const [rel, body] of Object.entries(sources)) { + const full = path.join(root, rel); + fs.mkdirSync(path.dirname(full), { recursive: true }); + fs.writeFileSync(full, body); + } + const report = { total: {} }; + for (const [rel, stmts] of Object.entries(reportRel)) { + report[path.join(root, rel)] = { + statements: { total: stmts.total, covered: stmts.covered, pct: 0 }, + lines: { total: stmts.total, covered: stmts.covered, pct: 0 }, + }; + } + const reportPath = path.join(root, "coverage-summary.json"); + fs.writeFileSync(reportPath, JSON.stringify(report)); + return { root, reportPath }; +} + +function run(reportPath, sourceDir, root) { + return execFileSync("node", [SCRIPT, reportPath, sourceDir, root], { encoding: "utf8" }); +} + +test("missing file is surfaced and counted 0%", () => { + const { root, reportPath } = fixture( + { "src/a.ts": "export const a = 1;\n", "src/untested.ts": "export const u = 2;\n" }, + { "src/a.ts": { total: 2, covered: 2 } }, // a.ts 100%; untested.ts absent + ); + const out = run(reportPath, path.join(root, "src"), root); + assert.match(out, /src\/untested\.ts/); + assert.match(out, /0%/); + assert.match(out, /50\.0%/); // a=100, untested missing=0 -> 50 +}); + +test("all tracked, nothing missing", () => { + const { root, reportPath } = fixture( + { "src/a.ts": "export const a = 1;\n" }, + { "src/a.ts": { total: 2, covered: 2 } }, + ); + const out = run(reportPath, path.join(root, "src"), root); + assert.match(out, /Not in coverage report: 0 file/); + assert.match(out, /100\.0%/); +}); + +test("test files and node_modules are not counted as source", () => { + const { root, reportPath } = fixture( + { + "src/a.ts": "export const a = 1;\n", + "src/a.test.ts": "import {a} from './a';\n", + "node_modules/dep/index.js": "module.exports = 1;\n", + }, + { "src/a.ts": { total: 2, covered: 2 } }, + ); + const out = run(reportPath, path.join(root, "src"), root); + assert.doesNotMatch(out, /a\.test\.ts/); + assert.doesNotMatch(out, /node_modules/); + assert.match(out, /100\.0%/); +}); + +test("missing report exits non-zero", () => { + const { root } = fixture({ "src/a.ts": "export const a = 1;\n" }, {}); + assert.throws(() => + execFileSync("node", [SCRIPT, path.join(root, "nope.json"), path.join(root, "src"), root], { + encoding: "utf8", + stdio: "pipe", + }), + ); +}); diff --git a/scripts/tests/install-lang.bats b/scripts/tests/install-lang.bats index a9f3bfe..0fb83e3 100644 --- a/scripts/tests/install-lang.bats +++ b/scripts/tests/install-lang.bats @@ -69,6 +69,16 @@ teardown() { grep -qxF "coverage.json" "$PROJECT/.gitignore" } +@test "install-lang typescript: lands the coverage script in the gitignored footprint" { + run bash "$INSTALL_LANG" typescript "$PROJECT" + + [ "$status" -eq 0 ] + [ -f "$PROJECT/.claude/scripts/coverage-summary.js" ] + [ -f "$PROJECT/coverage-makefile.txt" ] + grep -qxF ".claude/" "$PROJECT/.gitignore" + grep -qxF "coverage/" "$PROJECT/.gitignore" +} + @test "install-lang go: coverage-only slice lands without a CLAUDE.md" { run bash "$INSTALL_LANG" go "$PROJECT" @@ -1156,12 +1156,21 @@ Reference (dotemacs): =scripts/coverage-summary.el=, =modules/coverage-core.el=, Origin: handoff from the .emacs.d session, 2026-05-25. -** TODO [#C] Fan out coverage-summary to the TypeScript bundle :feature: +** DONE [#C] Fan out coverage-summary across all language bundles :feature: +CLOSED: [2026-05-31 Sun] :PROPERTIES: :CREATED: [2026-05-31 Sun] :END: -The Elisp pilot proved the pattern; Python and Go followed (all three DONE above). The plumbing is generic: =install-lang.sh= seeds the fragment, and =make test= now discovers ERT (=test-*.el=), pytest (=test_*.py=), and =go test= (=*_test.go=) under =languages/*/tests/=. TypeScript is the last one. +Done 2026-05-31: coverage-summary now ships in all four bundles. Elisp pilot, then Python, Go, and TypeScript. Each parses its tool's report (SimpleCov / coverage.py JSON / Go cover.out / Istanbul json-summary), counts on-disk source files absent from the report as 0%, and file-weights the project number. The plumbing proved generic: =install-lang.sh= seeds the project-owned =coverage-makefile.txt= and ships the script into the gitignored =.claude/scripts/=; =make test= discovers ERT (=test-*.el=), pytest (=test_*.py=), =go test= (=*_test.go=), and =node --test= (=*.test.js=) under =languages/*/tests/=, each guarded on its toolchain. TypeScript and Go scripts were dogfooded (Go against a live profile, TS against the CLI); Python and TS weren't run against a live coverage tool (coverage.py / nyc not installed) — proven against faithful fixtures matching each tool's stable schema. + +Remaining follow-ups (not blockers): +- Go is a coverage-only slice — =languages/go/= has no rule file, so =sync-language-bundle.sh= can't fingerprint it and won't sync-maintain the script. Build out the real Go bundle (=go.md= / =go-testing.md= + =CLAUDE.md=) to close that. +- First real adopters of the Python and TS scripts should sanity-check against a live =coverage json= / nyc =coverage-summary.json= run. + +Original notes retained below for the next person. + +The Elisp pilot proved the pattern; Python and Go followed. The plumbing is generic: =install-lang.sh= seeds the fragment, and =make test= now discovers ERT (=test-*.el=), pytest (=test_*.py=), and =go test= (=*_test.go=) under =languages/*/tests/=. TypeScript is the last one. - TypeScript/JS: nyc/Istanbul =coverage-final.json= / =coverage-summary.json=. Same kernel: file-weighted project number, on-disk =*.ts=/=*.js= absent from the report counted as 0%. nyc prints its own table, so the script focuses on the missing-file list and the number. Needs a vitest/jest (or =node --test=) discovery path in =make test=, mirroring the go-test block. |
