diff options
| author | Craig Jennings <c@cjennings.net> | 2026-07-24 12:28:18 -0500 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-07-24 12:28:18 -0500 |
| commit | f0c1bc40708615d5b423922c08c1f27e6cf96259 (patch) | |
| tree | 5fc8ebe7fde6381f4cd91b5de15fe4a2521889c8 /working/hook-fail-open/test-validate-el-hook.bats.superseded | |
| parent | 781fa0786e2096797e630dcb81ffbc62ed98460b (diff) | |
| download | rulesets-f0c1bc40708615d5b423922c08c1f27e6cf96259.tar.gz rulesets-f0c1bc40708615d5b423922c08c1f27e6cf96259.zip | |
fix(hooks): the secret scan no longer passes when git fails
Every bundle built its scan input as `git diff --cached ... | grep ... || true`. The `|| true` has to stay, since grep exits 1 when it matches nothing and that is the ordinary case. But with no pipefail it also swallowed a failure of git itself, so an empty result made the scan search nothing, find nothing, and report clean with a real credential staged. A gate that passes without having looked.
.emacs.d found it in elisp. It was in all five: bash, elisp, go, python and typescript, eleven sites once each bundle's staged-file list is counted. Two of those bundles are ones I wrote yesterday by copying bash, so I propagated it while closing an unrelated gap in the same file. Each site now reads the diff on its own and aborts if git fails, leaving the greps their `|| true`.
Verified per bundle on three axes: refuses when the diff cannot be read, still blocks a real staged secret, still passes a clean commit.
The cross-bundle test suite had its own version of the same disease. Its VARIANTS list read "elisp bash go" while python and typescript also shipped hooks, so every "in every variant" assertion had quietly skipped two bundles since the day they were added. VARIANTS is now discovered from the tree. Two fail-closed assertions join it, and .emacs.d's elisp suite is adopted here beside the canonical hook, because a test living in the consuming project cannot fail when the canonical regresses.
Also removes the validate-el auto-test cap, Craig's call. Above 20 matching test files the runner skipped everything and exited 0 with no output. The premise was speed and it did not hold: a whole family runs in about a second. The cap was also hiding a real cross-test pollution bug that only surfaces when a family runs in one process.
Diffstat (limited to 'working/hook-fail-open/test-validate-el-hook.bats.superseded')
| -rw-r--r-- | working/hook-fail-open/test-validate-el-hook.bats.superseded | 73 |
1 files changed, 0 insertions, 73 deletions
diff --git a/working/hook-fail-open/test-validate-el-hook.bats.superseded b/working/hook-fail-open/test-validate-el-hook.bats.superseded deleted file mode 100644 index 2dbcae7..0000000 --- a/working/hook-fail-open/test-validate-el-hook.bats.superseded +++ /dev/null @@ -1,73 +0,0 @@ -#!/usr/bin/env bats -# Tests for .claude/hooks/validate-el.sh — specifically the auto-test cap. -# -# The hook runs the tests matching an edited file, but only when the match -# count is between 1 and MAX_AUTO_TEST_FILES. Above the cap the whole block -# was skipped with no else branch: nothing printed, exit 0, indistinguishable -# from a passing run. That is live for the three largest families here -# (calendar-sync 63 test files, music 45, ai-term 35), so every edit to those -# modules ran parens and byte-compile and zero tests, silently. -# -# The cap itself is fine — running 63 files per keystroke is not wanted. The -# defect is the silence, so these tests assert the skip announces itself and -# names what to run. -# -# Each test builds a synthetic project in BATS_TEST_TMPDIR and points -# CLAUDE_PROJECT_DIR at it, so nothing runs against the real tree. - -setup() { - HOOK="${BATS_TEST_DIRNAME}/../.claude/hooks/validate-el.sh" - PROJ="${BATS_TEST_TMPDIR}/proj" - mkdir -p "$PROJ/modules" "$PROJ/tests" - export CLAUDE_PROJECT_DIR="$PROJ" - printf '(provide (quote widget))\n' > "$PROJ/modules/widget.el" -} - -# Create N test files matching the widget stem. Each is trivially green so a -# run below the cap succeeds and the only variable is the count. -make_tests() { - local n="$1" i - for ((i = 1; i <= n; i++)); do - printf '(require (quote ert))\n(ert-deftest test-widget-%d () (should t))\n' \ - "$i" > "$PROJ/tests/test-widget-${i}.el" - done -} - -hook_input() { - printf '{"tool_input":{"file_path":"%s"}}' "$PROJ/modules/widget.el" -} - -# ------------------------------- Normal cases ------------------------------- - -@test "under the cap: runs the tests and stays quiet on success" { - make_tests 3 - run bash -c "$(printf '%q' "$HOOK") <<< '$(hook_input)'" - [ "$status" -eq 0 ] - [[ "${output,,}" != *"skipped"* ]] -} - -# ------------------------------ Boundary cases ------------------------------ - -@test "exactly at the cap: still runs the tests" { - make_tests 20 - run bash -c "$(printf '%q' "$HOOK") <<< '$(hook_input)'" - [ "$status" -eq 0 ] - [[ "${output,,}" != *"skipped"* ]] -} - -# -------------------------------- Error cases ------------------------------- - -@test "over the cap: says it skipped rather than exiting silently" { - make_tests 21 - run bash -c "$(printf '%q' "$HOOK") <<< '$(hook_input)'" - # Must not fail the edit — the cap is deliberate, the silence is not. - [ "$status" -eq 0 ] - [[ "${output,,}" == *"skipped"* ]] -} - -@test "over the cap: names the count and how to run them" { - make_tests 21 - run bash -c "$(printf '%q' "$HOOK") <<< '$(hook_input)'" - [[ "$output" == *"21"* ]] - [[ "$output" == *"make test-file"* ]] -} |
