aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2026-05-31 13:57:40 -0500
committerCraig Jennings <c@cjennings.net>2026-05-31 13:57:40 -0500
commitee903e4b63257573773e93d10612250e3634cae9 (patch)
tree89cc6199b53f445a4c32252c6b982a6c86964e3b
parent47ca509e69b6a1472a735a4b9521a952e7434491 (diff)
downloadrulesets-ee903e4b63257573773e93d10612250e3634cae9.tar.gz
rulesets-ee903e4b63257573773e93d10612250e3634cae9.zip
feat(typescript): add coverage-summary to the TypeScript bundle
Last language in the coverage-summary fan-out, after Elisp, Python, and Go. Same kernel: count every source file on disk that's absent from the coverage report as 0% and weight the project number by file, so an untested file stays visible instead of being averaged away. The script at languages/typescript/claude/scripts/coverage-summary.js parses an Istanbul json-summary report (the coverage-summary.json that c8, Vitest, and Jest all emit), takes per-file statements covered over total, and reports a file-weighted number plus the missing files. It walks the source dir for .ts/.js, skipping test files, declarations, and node_modules. Node built-ins only, so it runs via node with no install, and it doesn't reimplement the per-file table nyc already prints. Tests are black-box, run with node's own test runner: a temp tree plus a json-summary report, the script invoked via node, output asserted. They cover missing-file detection, all-tracked, test-file and node_modules exclusion, and the missing-report error. make test gained a node --test discovery path for languages/*/tests, guarded so environments without Node skip it cleanly. As with Python, the TypeScript bundle had no gitignore-add.txt, which would have left the script un-gitignored on install, so I added one. This finishes the fan-out: coverage-summary now ships in all four bundles, each parsing its own tool's report behind the same file-weighted, missing-as-0% kernel. I proved the Go and TypeScript scripts by running them (Go against a live profile, TS against a synthetic report and the CLI). Python and TypeScript weren't run against a live coverage tool, since neither coverage.py nor nyc is installed here, so the first adopter of each should check against a real report.
-rw-r--r--Makefile9
-rw-r--r--languages/typescript/claude/rules/typescript-testing.md17
-rw-r--r--languages/typescript/claude/scripts/coverage-summary.js169
-rw-r--r--languages/typescript/coverage-makefile.txt45
-rw-r--r--languages/typescript/gitignore-add.txt11
-rw-r--r--languages/typescript/tests/coverage-summary.test.js90
-rw-r--r--scripts/tests/install-lang.bats10
-rw-r--r--todo.org13
8 files changed, 362 insertions, 2 deletions
diff --git a/Makefile b/Makefile
index b8f34cb..33c582f 100644
--- a/Makefile
+++ b/Makefile
@@ -477,6 +477,15 @@ test: ## Run all test suites (pytest + ERT + bats)
else \
echo "go test: skipped (go not installed)"; \
fi
+ @if command -v node >/dev/null 2>&1; then \
+ set -e; for d in languages/*/tests; do \
+ ls "$$d"/*.test.js >/dev/null 2>&1 || continue; \
+ echo "node test: $$d"; \
+ ( cd "$$d" && node --test ); \
+ done; \
+ else \
+ echo "node test: skipped (node not installed)"; \
+ fi
@set -e; for f in .ai/scripts/tests/test-*.el; do \
[ -e "$$f" ] || continue; \
echo "ert: $$(basename "$$f")"; \
diff --git a/languages/typescript/claude/rules/typescript-testing.md b/languages/typescript/claude/rules/typescript-testing.md
index bd6933f..31c50fc 100644
--- a/languages/typescript/claude/rules/typescript-testing.md
+++ b/languages/typescript/claude/rules/typescript-testing.md
@@ -113,6 +113,23 @@ Workflow: invoke `/pairwise-tests` → get a PICT model + generated test matrix
→ paste the matrix into an `it.each` block. See `testing.md` § Combinatorial
Coverage for the general rule and when to skip.
+## Measuring Coverage — `make coverage-summary`
+
+The bundle ships a coverage summary at `.claude/scripts/coverage-summary.js`
+and a Makefile fragment (`coverage-makefile.txt`) with `coverage` and
+`coverage-summary` targets. After the suite runs under c8 (or Vitest/Jest with
+the json-summary reporter) and writes `coverage/coverage-summary.json`, `make
+coverage-summary` prints a file-weighted project number and the source files no
+test imported.
+
+The number to watch is that missing-file count. A module no test imports never
+appears in the Istanbul report, so a statement-weighted total skips it silently
+and the suite looks healthier than it is. The summary counts every `.ts`/`.js`
+under the source dir that's absent from the report as 0%, so an untested module
+drags the project number down where you can see it. It doesn't reimplement the
+per-file table — nyc/c8 already print that. Copy the fragment's targets into
+your own Makefile to adopt it; the bundle never edits your Makefile.
+
## Mocking Guidelines
### Mock these (external boundaries):
diff --git a/languages/typescript/claude/scripts/coverage-summary.js b/languages/typescript/claude/scripts/coverage-summary.js
new file mode 100644
index 0000000..72fa78d
--- /dev/null
+++ b/languages/typescript/claude/scripts/coverage-summary.js
@@ -0,0 +1,169 @@
+#!/usr/bin/env node
+/*
+ * Whole-project coverage summary from an Istanbul json-summary report.
+ *
+ * Batch helper for `make coverage-summary`. After the test run writes an
+ * Istanbul json-summary report (nyc / c8 `--reporter=json-summary`, or
+ * vitest/jest coverage), this prints a file-weighted project number and the
+ * source files present on disk but absent from the report.
+ *
+ * The value here is the missing-file detection: a module no test imports never
+ * appears in the report, so a statement-weighted total silently skips it and
+ * the suite looks healthier than it is. This counts such a file as 0% and
+ * weights the project number by file rather than by statement, so untested
+ * files stay visible. nyc's own text reporter prints the per-file table, so
+ * this deliberately doesn't reimplement that.
+ *
+ * Node built-ins only (fs, path), so it runs anywhere the bundle lands via
+ * `node coverage-summary.js`. The json-summary shape it reads is:
+ * { "total": {...}, "<abs-path>": { "statements": {"total", "covered"}, ... } }
+ *
+ * Usage:
+ * node coverage-summary.js REPORT SOURCE_DIR [PROJECT_ROOT]
+ * REPORT path to the Istanbul json-summary report
+ * SOURCE_DIR the directory to account for (walked recursively)
+ * PROJECT_ROOT defaults to the current directory
+ */
+"use strict";
+
+const fs = require("node:fs");
+const path = require("node:path");
+
+const SRC_EXT = new Set([".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs"]);
+const SKIP_DIRS = new Set(["node_modules", "dist", "build", "coverage", ".claude", ".git"]);
+
+function die(msg) {
+ process.stderr.write(`coverage-summary: ${msg}\n`);
+ process.exit(1);
+}
+
+function toSlash(p) {
+ return p.split(path.sep).join("/");
+}
+
+// parseReport reads an Istanbul json-summary into { absPath: [covered, total] }.
+// Uses statements; falls back to lines when statements is absent.
+function parseReport(reportPath) {
+ let raw;
+ try {
+ raw = fs.readFileSync(reportPath, "utf8");
+ } catch {
+ die(`coverage report not found: ${reportPath}`);
+ }
+ let data;
+ try {
+ data = JSON.parse(raw);
+ } catch (e) {
+ die(`malformed coverage JSON in ${reportPath}: ${e.message}`);
+ }
+ const out = {};
+ for (const [key, entry] of Object.entries(data)) {
+ if (key === "total" || !entry || typeof entry !== "object") continue;
+ const metric = entry.statements || entry.lines;
+ if (!metric) continue;
+ out[path.resolve(key)] = [Number(metric.covered) || 0, Number(metric.total) || 0];
+ }
+ return out;
+}
+
+function underDir(table, sourceRel) {
+ if (sourceRel === "." || sourceRel === "") return table;
+ const prefix = sourceRel + "/";
+ const out = {};
+ for (const [p, v] of Object.entries(table)) {
+ if (p === sourceRel || p.startsWith(prefix)) out[p] = v;
+ }
+ return out;
+}
+
+function isSource(name) {
+ if (/\.(test|spec)\.[^.]+$/.test(name)) return false;
+ if (name.endsWith(".d.ts")) return false;
+ return SRC_EXT.has(path.extname(name));
+}
+
+// sourceFiles returns project-relative source files under sourceDir (recursive),
+// skipping test files, declarations, and non-source directories.
+function sourceFiles(sourceDir, root) {
+ const out = [];
+ const walk = (dir) => {
+ let entries;
+ try {
+ entries = fs.readdirSync(dir, { withFileTypes: true });
+ } catch {
+ return;
+ }
+ for (const e of entries) {
+ const full = path.join(dir, e.name);
+ if (e.isDirectory()) {
+ if (!SKIP_DIRS.has(e.name)) walk(full);
+ } else if (e.isFile() && isSource(e.name)) {
+ out.push(toSlash(path.relative(root, full)));
+ }
+ }
+ };
+ walk(sourceDir);
+ return out.sort();
+}
+
+function missing(tracked, sourceDir, root) {
+ const have = new Set(Object.keys(tracked));
+ return sourceFiles(sourceDir, root).filter((f) => !have.has(f));
+}
+
+function filePct(covered, total) {
+ return total > 0 ? (100 * covered) / total : 100;
+}
+
+function projectPct(tracked, miss) {
+ const total = Object.keys(tracked).length + miss.length;
+ if (total === 0) return 0;
+ let score = 0;
+ for (const [c, t] of Object.values(tracked)) score += filePct(c, t);
+ return score / total;
+}
+
+function summaryText(tracked, miss, sourceRel) {
+ const trackedCount = Object.keys(tracked).length;
+ const total = trackedCount + miss.length;
+ const pct = projectPct(tracked, miss);
+ const lines = [
+ `Coverage summary for ${sourceRel}`,
+ "",
+ `Project coverage: ${pct.toFixed(1)}% (${trackedCount} tracked, ${miss.length} missing, ` +
+ `${total} total; missing files count as 0%)`,
+ "",
+ `Not in coverage report: ${miss.length} file${miss.length === 1 ? "" : "s"}`,
+ ];
+ if (miss.length > 0) {
+ lines.push("These files had no coverage entry; they count as 0% in project coverage.");
+ for (const m of [...miss].sort()) lines.push(` ${m}`);
+ } else {
+ lines.push("Every source file appears in the coverage report.");
+ }
+ lines.push("", "(Per-file table: run nyc's text reporter.)");
+ return lines.join("\n") + "\n";
+}
+
+function main(argv) {
+ if (argv.length < 2 || argv.length > 3) {
+ process.stderr.write("usage: coverage-summary.js REPORT SOURCE_DIR [PROJECT_ROOT]\n");
+ process.exit(2);
+ }
+ const [report, sourceDir] = argv;
+ const root = path.resolve(argv[2] || ".");
+ const absSource = path.resolve(sourceDir);
+ const sourceRel = toSlash(path.relative(root, absSource)) || ".";
+
+ const table = parseReport(report);
+ // Re-key the report to project-relative paths for comparison with disk files.
+ const rel = {};
+ for (const [abs, v] of Object.entries(table)) rel[toSlash(path.relative(root, abs))] = v;
+
+ const tracked = underDir(rel, sourceRel);
+ const miss = missing(tracked, absSource, root);
+
+ process.stdout.write("\n" + summaryText(tracked, miss, sourceRel));
+}
+
+main(process.argv.slice(2));
diff --git a/languages/typescript/coverage-makefile.txt b/languages/typescript/coverage-makefile.txt
new file mode 100644
index 0000000..64c7fcd
--- /dev/null
+++ b/languages/typescript/coverage-makefile.txt
@@ -0,0 +1,45 @@
+# TypeScript / JavaScript coverage — Makefile fragment + setup recommendation
+#
+# This file is owned by the project, not the rulesets bundle. The bundle never
+# edits your Makefile. Copy the two targets below into your own Makefile (and
+# adjust the variables at the top), then delete this file or keep it as a note.
+#
+# What you get:
+# make coverage runs the suite under c8 and writes an Istanbul
+# json-summary report plus c8's own text table
+# make coverage-summary prints a file-weighted project number and every
+# source file on disk absent from the report, at 0%.
+#
+# Why the summary matters: a module no test imports never appears in the
+# Istanbul report, so a statement-weighted total silently skips it. The summary
+# weights by file and counts a missing file as 0%, so untested files stay
+# visible. It does not reimplement the per-file table — nyc/c8 already print it.
+#
+# ---------------------------------------------------------------------------
+# Prerequisite: an Istanbul json-summary report.
+#
+# c8: npx c8 --reporter=json-summary --reporter=text npm test
+# vitest: vitest run --coverage --coverage.reporter=json-summary
+# jest: jest --coverage --coverageReporters=json-summary
+#
+# All three write coverage/coverage-summary.json. The summary script needs
+# nothing beyond Node's standard library — it parses the JSON.
+# ---------------------------------------------------------------------------
+
+# Variables — adjust to your layout.
+NODE ?= node
+SOURCE_DIR ?= src
+COVERAGE_FILE ?= coverage/coverage-summary.json
+# The summary script ships with the bundle under .claude/scripts/ (gitignored).
+COVERAGE_SUMMARY ?= .claude/scripts/coverage-summary.js
+
+coverage:
+ @npx c8 --reporter=json-summary --reporter=text npm test
+ @$(MAKE) coverage-summary
+
+coverage-summary:
+ @if [ ! -f $(COVERAGE_FILE) ]; then \
+ echo "[!] No coverage file at $(COVERAGE_FILE). Run 'make coverage' first."; \
+ exit 1; \
+ fi
+ @$(NODE) $(COVERAGE_SUMMARY) $(COVERAGE_FILE) $(SOURCE_DIR) .
diff --git a/languages/typescript/gitignore-add.txt b/languages/typescript/gitignore-add.txt
new file mode 100644
index 0000000..b8d700a
--- /dev/null
+++ b/languages/typescript/gitignore-add.txt
@@ -0,0 +1,11 @@
+# Claude Code — local tooling, delivered by install/sync, not committed
+.claude/
+CLAUDE.md
+githooks/
+
+# Node + coverage build artifacts (generated)
+node_modules/
+coverage/
+dist/
+build/
+*.tsbuildinfo
diff --git a/languages/typescript/tests/coverage-summary.test.js b/languages/typescript/tests/coverage-summary.test.js
new file mode 100644
index 0000000..b2b6ea3
--- /dev/null
+++ b/languages/typescript/tests/coverage-summary.test.js
@@ -0,0 +1,90 @@
+// Black-box tests for the TypeScript bundle coverage-summary script.
+//
+// The script ships into a project's .claude/scripts/ and runs via `node`, so
+// these tests exercise the real CLI rather than importing it: build a temp tree
+// (source files + an istanbul json-summary report), run the script against it,
+// and assert on stdout. Run with `node --test`.
+//
+// Normal / Boundary / Error coverage at the behavior level: missing-file
+// detection, the file-weighted number, all-tracked, ignoring test files and
+// node_modules, and a missing-report error.
+const { test } = require("node:test");
+const assert = require("node:assert");
+const { execFileSync } = require("node:child_process");
+const fs = require("node:fs");
+const os = require("node:os");
+const path = require("node:path");
+
+const SCRIPT = path.resolve(__dirname, "..", "claude", "scripts", "coverage-summary.js");
+
+// fixture builds a temp project. sources maps relpath -> contents. report is the
+// istanbul json-summary object, with per-file keys given as relpaths that get
+// rewritten to absolute paths (matching how istanbul records them).
+function fixture(sources, reportRel) {
+ const root = fs.mkdtempSync(path.join(os.tmpdir(), "tscov-"));
+ for (const [rel, body] of Object.entries(sources)) {
+ const full = path.join(root, rel);
+ fs.mkdirSync(path.dirname(full), { recursive: true });
+ fs.writeFileSync(full, body);
+ }
+ const report = { total: {} };
+ for (const [rel, stmts] of Object.entries(reportRel)) {
+ report[path.join(root, rel)] = {
+ statements: { total: stmts.total, covered: stmts.covered, pct: 0 },
+ lines: { total: stmts.total, covered: stmts.covered, pct: 0 },
+ };
+ }
+ const reportPath = path.join(root, "coverage-summary.json");
+ fs.writeFileSync(reportPath, JSON.stringify(report));
+ return { root, reportPath };
+}
+
+function run(reportPath, sourceDir, root) {
+ return execFileSync("node", [SCRIPT, reportPath, sourceDir, root], { encoding: "utf8" });
+}
+
+test("missing file is surfaced and counted 0%", () => {
+ const { root, reportPath } = fixture(
+ { "src/a.ts": "export const a = 1;\n", "src/untested.ts": "export const u = 2;\n" },
+ { "src/a.ts": { total: 2, covered: 2 } }, // a.ts 100%; untested.ts absent
+ );
+ const out = run(reportPath, path.join(root, "src"), root);
+ assert.match(out, /src\/untested\.ts/);
+ assert.match(out, /0%/);
+ assert.match(out, /50\.0%/); // a=100, untested missing=0 -> 50
+});
+
+test("all tracked, nothing missing", () => {
+ const { root, reportPath } = fixture(
+ { "src/a.ts": "export const a = 1;\n" },
+ { "src/a.ts": { total: 2, covered: 2 } },
+ );
+ const out = run(reportPath, path.join(root, "src"), root);
+ assert.match(out, /Not in coverage report: 0 file/);
+ assert.match(out, /100\.0%/);
+});
+
+test("test files and node_modules are not counted as source", () => {
+ const { root, reportPath } = fixture(
+ {
+ "src/a.ts": "export const a = 1;\n",
+ "src/a.test.ts": "import {a} from './a';\n",
+ "node_modules/dep/index.js": "module.exports = 1;\n",
+ },
+ { "src/a.ts": { total: 2, covered: 2 } },
+ );
+ const out = run(reportPath, path.join(root, "src"), root);
+ assert.doesNotMatch(out, /a\.test\.ts/);
+ assert.doesNotMatch(out, /node_modules/);
+ assert.match(out, /100\.0%/);
+});
+
+test("missing report exits non-zero", () => {
+ const { root } = fixture({ "src/a.ts": "export const a = 1;\n" }, {});
+ assert.throws(() =>
+ execFileSync("node", [SCRIPT, path.join(root, "nope.json"), path.join(root, "src"), root], {
+ encoding: "utf8",
+ stdio: "pipe",
+ }),
+ );
+});
diff --git a/scripts/tests/install-lang.bats b/scripts/tests/install-lang.bats
index a9f3bfe..0fb83e3 100644
--- a/scripts/tests/install-lang.bats
+++ b/scripts/tests/install-lang.bats
@@ -69,6 +69,16 @@ teardown() {
grep -qxF "coverage.json" "$PROJECT/.gitignore"
}
+@test "install-lang typescript: lands the coverage script in the gitignored footprint" {
+ run bash "$INSTALL_LANG" typescript "$PROJECT"
+
+ [ "$status" -eq 0 ]
+ [ -f "$PROJECT/.claude/scripts/coverage-summary.js" ]
+ [ -f "$PROJECT/coverage-makefile.txt" ]
+ grep -qxF ".claude/" "$PROJECT/.gitignore"
+ grep -qxF "coverage/" "$PROJECT/.gitignore"
+}
+
@test "install-lang go: coverage-only slice lands without a CLAUDE.md" {
run bash "$INSTALL_LANG" go "$PROJECT"
diff --git a/todo.org b/todo.org
index b6f28b8..9594a87 100644
--- a/todo.org
+++ b/todo.org
@@ -1156,12 +1156,21 @@ Reference (dotemacs): =scripts/coverage-summary.el=, =modules/coverage-core.el=,
Origin: handoff from the .emacs.d session, 2026-05-25.
-** TODO [#C] Fan out coverage-summary to the TypeScript bundle :feature:
+** DONE [#C] Fan out coverage-summary across all language bundles :feature:
+CLOSED: [2026-05-31 Sun]
:PROPERTIES:
:CREATED: [2026-05-31 Sun]
:END:
-The Elisp pilot proved the pattern; Python and Go followed (all three DONE above). The plumbing is generic: =install-lang.sh= seeds the fragment, and =make test= now discovers ERT (=test-*.el=), pytest (=test_*.py=), and =go test= (=*_test.go=) under =languages/*/tests/=. TypeScript is the last one.
+Done 2026-05-31: coverage-summary now ships in all four bundles. Elisp pilot, then Python, Go, and TypeScript. Each parses its tool's report (SimpleCov / coverage.py JSON / Go cover.out / Istanbul json-summary), counts on-disk source files absent from the report as 0%, and file-weights the project number. The plumbing proved generic: =install-lang.sh= seeds the project-owned =coverage-makefile.txt= and ships the script into the gitignored =.claude/scripts/=; =make test= discovers ERT (=test-*.el=), pytest (=test_*.py=), =go test= (=*_test.go=), and =node --test= (=*.test.js=) under =languages/*/tests/=, each guarded on its toolchain. TypeScript and Go scripts were dogfooded (Go against a live profile, TS against the CLI); Python and TS weren't run against a live coverage tool (coverage.py / nyc not installed) — proven against faithful fixtures matching each tool's stable schema.
+
+Remaining follow-ups (not blockers):
+- Go is a coverage-only slice — =languages/go/= has no rule file, so =sync-language-bundle.sh= can't fingerprint it and won't sync-maintain the script. Build out the real Go bundle (=go.md= / =go-testing.md= + =CLAUDE.md=) to close that.
+- First real adopters of the Python and TS scripts should sanity-check against a live =coverage json= / nyc =coverage-summary.json= run.
+
+Original notes retained below for the next person.
+
+The Elisp pilot proved the pattern; Python and Go followed. The plumbing is generic: =install-lang.sh= seeds the fragment, and =make test= now discovers ERT (=test-*.el=), pytest (=test_*.py=), and =go test= (=*_test.go=) under =languages/*/tests/=. TypeScript is the last one.
- TypeScript/JS: nyc/Istanbul =coverage-final.json= / =coverage-summary.json=. Same kernel: file-weighted project number, on-disk =*.ts=/=*.js= absent from the report counted as 0%. nyc prints its own table, so the script focuses on the missing-file list and the number. Needs a vitest/jest (or =node --test=) discovery path in =make test=, mirroring the go-test block.