diff options
Diffstat (limited to 'languages/python')
| -rw-r--r-- | languages/python/CLAUDE.md | 80 | ||||
| -rwxr-xr-x | languages/python/claude/hooks/validate-python.sh | 95 | ||||
| -rw-r--r-- | languages/python/claude/settings.json | 79 | ||||
| -rwxr-xr-x | languages/python/githooks/pre-commit | 79 | ||||
| -rw-r--r-- | languages/python/tests/pre-commit.bats | 138 | ||||
| -rw-r--r-- | languages/python/tests/validate-python.bats | 117 |
6 files changed, 588 insertions, 0 deletions
diff --git a/languages/python/CLAUDE.md b/languages/python/CLAUDE.md new file mode 100644 index 0000000..a2d0a82 --- /dev/null +++ b/languages/python/CLAUDE.md @@ -0,0 +1,80 @@ +# CLAUDE.md + +## Project + +Python project. Customize this section with your own description, layout, +and conventions. + +**Typical layout:** +- `src/<package>/` or a top-level package directory — importable code +- `tests/` — pytest tests mirroring the package layout +- `pyproject.toml` — dependencies, tool config (ruff, pytest, coverage) + +## Build & Test Commands + +If the project has a Makefile, document targets here. Common pattern: + +```bash +make test # run the pytest suite +make test FILE=tests/x.py # one file +make coverage # suite + coverage report +make lint # ruff across the tree +make typecheck # mypy (if the project adopts it) +make fmt # ruff format / black +``` + +Direct equivalents: `python3 -m pytest`, `pytest tests/test_x.py::test_name`, +`ruff check .`, `ruff format --diff .`, `mypy src/`. + +## Language Rules + +See rule files in `.claude/rules/`: +- `python-testing.md` — pytest conventions and fixture discipline +- `verification.md` — verify-before-claim-done discipline + +## Git Workflow + +Commit conventions: see `.claude/rules/commits.md` (author identity, +no AI attribution, message format). + +Pre-commit hook in `githooks/` scans for secrets, syntax-checks staged Python, +and runs `ruff` when it's installed. Activate on a fresh clone with +`git config core.hooksPath githooks`. + +## Problem-Solving Approach + +Investigate before fixing. When diagnosing a bug: +1. Read the relevant module and trace what actually happens +2. Identify the root cause, not a surface symptom +3. Write a failing test that captures the correct behavior +4. Fix, then re-run tests + +## Testing Discipline + +TDD is the default: write a failing test before any implementation. If you can't +write the test, you don't yet understand the change. Details in +`.claude/rules/python-testing.md`. + +## Editing Discipline + +A PostToolUse hook syntax-checks every Python file after Edit/Write/MultiEdit +and blocks on a parse error, then runs `ruff` when it's installed. The hook +covers `.py`, `.pyi`, and extensionless files with a python shebang. + +Type checking is not enforced by the hook — it needs the whole package and its +dependencies resolved, which is a build-scale operation rather than a +per-keystroke one. Run it via `make typecheck`. + +Formatting is likewise not enforced: a project picks its own line length and +quote style, so blocking on an unconfigured default would impose a contested +choice. Adopt one per project in `pyproject.toml`. + +## What Not to Do + +- Don't add features beyond what was asked +- Don't refactor surrounding code when fixing a bug +- Don't use a bare `except:` or swallow an exception without handling it +- Don't use a mutable default argument (`def f(xs=[])`) +- Don't add comments to code you didn't change +- Don't commit `.env` files, credentials, or API keys — the pre-commit hook + catches common patterns but isn't a substitute for care diff --git a/languages/python/claude/hooks/validate-python.sh b/languages/python/claude/hooks/validate-python.sh new file mode 100755 index 0000000..e43ad77 --- /dev/null +++ b/languages/python/claude/hooks/validate-python.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# Validate Python files after Edit/Write/MultiEdit. +# PostToolUse hook: receives tool-call JSON on stdin. +# +# On success: exit 0 silent. +# On failure: emit JSON with hookSpecificOutput.additionalContext so Claude +# sees a structured error in its context, THEN exit 2 to block the tool +# pipeline. stderr still echoes the error for terminal visibility. +# +# Phase 1: syntax — python3 compiles the file. Always available wherever this +# hook can meaningfully run, so it's the floor rather than an optional +# gate: a file that doesn't parse is never worth passing on. +# Phase 2: ruff — lint, when installed. Catches undefined names, unused +# imports, and the rest of the pyflakes set. Absent ruff doesn't block +# the edit, matching how the bash bundle treats shellcheck. +# +# Formatters (black, ruff format) are deliberately NOT enforced here. A project +# picks its own line length and quote style, so blocking on an unconfigured +# default would impose a contested choice. python.md recommends a formatter; +# this hook enforces correctness. +# +# Type checking (mypy, pyright) is also out: it needs the whole package and its +# dependencies resolved, which is a build-scale operation, not a per-keystroke +# one. Run it via `make lint` / `make typecheck`. +# +# Scope: .py and .pyi files, plus extensionless files whose first line is a +# python shebang (the CLI tools that fill a script-heavy repo carry no +# extension). + +set -u + +# Emit a JSON failure payload and exit 2. Arguments: +# $1 — short failure type (e.g. "PYTHON SYNTAX ERROR") +# $2 — file path +# $3 — tool output (error body) +fail_json() { + local ctx + ctx="$(printf '%s: %s\n\n%s\n\nFix before proceeding.' "$1" "$2" "$3" \ + | jq -Rs .)" + cat <<EOF +{"hookSpecificOutput": {"hookEventName": "PostToolUse", "additionalContext": $ctx}} +EOF + printf '%s: %s\n%s\n' "$1" "$2" "$3" >&2 + exit 2 +} + +f="$(jq -r '.tool_input.file_path // .tool_response.filePath // empty')" +[ -z "$f" ] && exit 0 +[ -f "$f" ] || exit 0 + +# Is this a Python file? By extension, or by shebang when it has no extension. +# Match on the basename, not the full path — a temp/parent dir can carry a dot +# (e.g. my.project/) and misfire the "*.*" extension test. +is_python=0 +base="${f##*/}" +case "$base" in + *.py | *.pyi) is_python=1 ;; + *.*) is_python=0 ;; # some other extension — not ours + *) + # No extension: sniff the shebang. + if head -1 "$f" 2>/dev/null | grep -qE '^#!.*\bpython[0-9.]*\b'; then + is_python=1 + fi + ;; +esac +[ "$is_python" -eq 1 ] || exit 0 + +# No python3 on this machine — nothing to validate, don't block the edit. +command -v python3 >/dev/null 2>&1 || exit 0 + +# --- Phase 1: syntax --- +# compile() rather than py_compile so no __pycache__ lands beside the source; +# the hook is a checker and must not leave build artifacts in the tree. +if ! out="$(python3 -c ' +import sys +p = sys.argv[1] +with open(p, "rb") as fh: + src = fh.read() +try: + compile(src, p, "exec") +except SyntaxError as e: + print(f"{e.msg} ({p}, line {e.lineno})", file=sys.stderr) + sys.exit(1) +' "$f" 2>&1)"; then + fail_json "PYTHON SYNTAX ERROR" "$f" "$out" +fi + +# --- Phase 2: lint (optional) --- +command -v ruff >/dev/null 2>&1 || exit 0 + +if ! out="$(ruff check "$f" 2>&1)"; then + fail_json "RUFF FAILED" "$f" "$out" +fi + +exit 0 diff --git a/languages/python/claude/settings.json b/languages/python/claude/settings.json new file mode 100644 index 0000000..9c6b2a9 --- /dev/null +++ b/languages/python/claude/settings.json @@ -0,0 +1,79 @@ +{ + "attribution": { + "commit": "", + "pr": "" + }, + "permissions": { + "allow": [ + "Bash(make)", + "Bash(make help)", + "Bash(make targets)", + "Bash(make test)", + "Bash(make test *)", + "Bash(make lint)", + "Bash(make fmt)", + "Bash(make coverage)", + "Bash(make coverage-summary)", + "Bash(make typecheck)", + "Bash(pytest)", + "Bash(pytest *)", + "Bash(python3 -m pytest *)", + "Bash(ruff check *)", + "Bash(ruff format --diff *)", + "Bash(black --check *)", + "Bash(black --diff *)", + "Bash(mypy *)", + "Bash(python3 -m py_compile *)", + "Bash(python3 --version)", + "Bash(pip list)", + "Bash(pip show *)", + "Bash(git status)", + "Bash(git status *)", + "Bash(git diff)", + "Bash(git diff *)", + "Bash(git log)", + "Bash(git log *)", + "Bash(git show)", + "Bash(git show *)", + "Bash(git blame *)", + "Bash(git branch)", + "Bash(git branch -v)", + "Bash(git branch -a)", + "Bash(git branch --list *)", + "Bash(git remote)", + "Bash(git remote -v)", + "Bash(git remote show *)", + "Bash(git ls-files *)", + "Bash(git rev-parse *)", + "Bash(git cat-file *)", + "Bash(git stash list)", + "Bash(git stash show *)", + "Bash(jq *)", + "Bash(date)", + "Bash(date *)", + "Bash(which *)", + "Bash(file *)", + "Bash(ls)", + "Bash(ls *)", + "Bash(wc *)", + "Bash(du *)", + "Bash(readlink *)", + "Bash(realpath *)", + "Bash(basename *)", + "Bash(dirname *)" + ] + }, + "hooks": { + "PostToolUse": [ + { + "matcher": "Edit|Write|MultiEdit", + "hooks": [ + { + "type": "command", + "command": "$CLAUDE_PROJECT_DIR/.claude/hooks/validate-python.sh" + } + ] + } + ] + } +} diff --git a/languages/python/githooks/pre-commit b/languages/python/githooks/pre-commit new file mode 100755 index 0000000..0a578b4 --- /dev/null +++ b/languages/python/githooks/pre-commit @@ -0,0 +1,79 @@ +#!/usr/bin/env bash +# Pre-commit hook: secret scan + syntax/lint check on staged Python files. +# Use `git commit --no-verify` to bypass for confirmed false positives. + +set -u + +REPO_ROOT="$(git rev-parse --show-toplevel)" +cd "$REPO_ROOT" || exit 1 + +# --- 1. Secret scan --- +# Patterns for common credentials. Scans only added lines in the staged diff. +# +# Two passes because case-sensitivity differs. AWS keys are uppercase, sk- keys +# lowercase, PEM headers fixed, so those match case-SENSITIVELY: under -i, +# AKIA[0-9A-Z]{16} matches any mixed-case 20-char run, which random base64 in an +# embedded image blob hits ~6% of the time per 100KB and blocks real commits. +# Only the keyword=value patterns need -i. +SECRET_PATTERNS_CS='(AKIA[0-9A-Z]{16}|sk-[a-zA-Z0-9_-]{20,}|-----BEGIN (RSA|DSA|EC|OPENSSH|PGP)( PRIVATE)?( KEY| KEY BLOCK)?-----)' +SECRET_PATTERNS_CI='(api[_-]?key|api[_-]?secret|auth[_-]?token|secret[_-]?key|bearer[_-]?token|access[_-]?token|password)[[:space:]]*[:=][[:space:]]*["'"'"'][^"'"'"']{16,}["'"'"']' + +added_lines="$(git diff --cached -U0 --diff-filter=AM \ + | grep '^+' | grep -v '^+++' || true)" + +cs_hits="$(printf '%s\n' "$added_lines" | grep -nE "$SECRET_PATTERNS_CS" || true)" +ci_hits="$(printf '%s\n' "$added_lines" | grep -niE "$SECRET_PATTERNS_CI" || true)" +# awk dedupes lines both passes matched, keeping first-seen order. +secret_hits="$(printf '%s\n%s' "$cs_hits" "$ci_hits" \ + | grep -v '^[[:space:]]*$' | awk '!seen[$0]++' || true)" + +if [ -n "$secret_hits" ]; then + echo "pre-commit: potential secret in staged changes:" >&2 + echo "$secret_hits" >&2 + echo "" >&2 + echo "Review the lines above. If this is a false positive (test fixture, documentation)," >&2 + echo "bypass with: git commit --no-verify" >&2 + exit 1 +fi + +# --- 2. Syntax check on staged Python files --- +staged_py="$(git diff --cached --name-only --diff-filter=AM \ + | grep -E '\.pyi?$' || true)" + +if [ -n "$staged_py" ] && command -v python3 >/dev/null 2>&1; then + failed="" + while IFS= read -r f; do + [ -z "$f" ] && continue + [ -f "$f" ] || continue + # compile() rather than py_compile so no __pycache__ lands in the tree. + if ! python3 -c 'import sys; compile(open(sys.argv[1], "rb").read(), sys.argv[1], "exec")' "$f" >/dev/null 2>&1; then + failed="${failed}${f}"$'\n' + fi + done <<< "$staged_py" + + if [ -n "$failed" ]; then + printf 'pre-commit: Python syntax errors in staged files:\n\n%s\n' "$failed" >&2 + echo "Run: python3 -m py_compile <file> to see the error, then re-stage." >&2 + exit 1 + fi +fi + +# --- 3. ruff on staged Python files (when installed) --- +if [ -n "$staged_py" ] && command -v ruff >/dev/null 2>&1; then + failed="" + while IFS= read -r f; do + [ -z "$f" ] && continue + [ -f "$f" ] || continue + if ! ruff check "$f" >/dev/null 2>&1; then + failed="${failed}${f}"$'\n' + fi + done <<< "$staged_py" + + if [ -n "$failed" ]; then + printf 'pre-commit: ruff failed on staged files:\n\n%s\n' "$failed" >&2 + echo "Run: ruff check <file> and fix the findings, then re-stage." >&2 + exit 1 + fi +fi + +exit 0 diff --git a/languages/python/tests/pre-commit.bats b/languages/python/tests/pre-commit.bats new file mode 100644 index 0000000..1ac82ee --- /dev/null +++ b/languages/python/tests/pre-commit.bats @@ -0,0 +1,138 @@ +#!/usr/bin/env bats +# +# Tests for languages/python/githooks/pre-commit — the secret scan plus +# syntax/lint gate that runs on staged Python files. +# +# The secret scan is the security-critical half and is language-independent, so +# it gets the same coverage here as in the bash bundle: a real key blocks, a +# clean diff passes, and the case-sensitivity split that keeps base64 blobs from +# false-positiving is exercised directly. +# +# Each test builds a throwaway git repo, stages content, and runs the hook from +# inside it — the hook reads `git diff --cached`, so a real index is required. + +HOOK="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)/githooks/pre-commit" + +setup() { + TEST_DIR="$(mktemp -d -t pre-commit-py-bats.XXXXXX)" + cd "$TEST_DIR" || exit 1 + git init -q . + git config user.email t@example.com + git config user.name Test + # A base commit so `git diff --cached` has a parent to diff against. + echo "seed" > seed.txt + git add seed.txt + git commit -qm seed +} + +teardown() { + cd / || true + rm -rf "$TEST_DIR" +} + +# ---- Normal ---------------------------------------------------------- + +@test "pre-commit(py): a clean staged Python file passes (exit 0)" { + printf 'def f(x):\n return x + 1\n' > ok.py + git add ok.py + run bash "$HOOK" + [ "$status" -eq 0 ] +} + +@test "pre-commit(py): an empty staging area passes (exit 0)" { + run bash "$HOOK" + [ "$status" -eq 0 ] +} + +# ---- Error: the secret scan ------------------------------------------ + +@test "pre-commit(py): an AWS key in a staged file blocks (exit 1)" { + printf 'KEY = "AKIAIOSFODNN7EXAMPLE"\n' > conf.py + git add conf.py + run bash "$HOOK" + [ "$status" -eq 1 ] + [[ "$output" == *"potential secret"* ]] +} + +@test "pre-commit(py): an sk- style token blocks (exit 1)" { + printf 'TOKEN = "sk-abcdefghijklmnopqrstuvwxyz0123"\n' > conf.py + git add conf.py + run bash "$HOOK" + [ "$status" -eq 1 ] +} + +@test "pre-commit(py): a quoted api_key assignment blocks (exit 1)" { + printf 'api_key = "abcdefghijklmnopqrstuvwxyz"\n' > conf.py + git add conf.py + run bash "$HOOK" + [ "$status" -eq 1 ] +} + +@test "pre-commit(py): a private-key header blocks (exit 1)" { + printf 'PEM = """-----BEGIN RSA PRIVATE KEY-----"""\n' > conf.py + git add conf.py + run bash "$HOOK" + [ "$status" -eq 1 ] +} + +# ---- Boundary: the case-sensitivity split ---------------------------- + +@test "pre-commit(py): a mixed-case base64 blob does NOT false-positive" { + # The AWS pattern is uppercase-only by design. Under -i it would match any + # 20-char mixed-case run, which random base64 hits often enough to block + # real commits. This is the regression test for that split. + printf 'BLOB = "AKIAbcdefGHIJklmnOPqr0123456789abcdefGHIJ"\n' > data.py + git add data.py + run bash "$HOOK" + [ "$status" -eq 0 ] +} + +@test "pre-commit(py): a short quoted password value does NOT block" { + # The keyword patterns require 16+ chars, so a placeholder stays quiet. + printf 'password = "short"\n' > conf.py + git add conf.py + run bash "$HOOK" + [ "$status" -eq 0 ] +} + +@test "pre-commit(py): a secret only in a REMOVED line does not block" { + printf 'KEY = "AKIAIOSFODNN7EXAMPLE"\n' > conf.py + git add conf.py + git commit -qm "add key" + rm conf.py + git add -A + run bash "$HOOK" + [ "$status" -eq 0 ] +} + +# ---- Error: the syntax gate ------------------------------------------ + +@test "pre-commit(py): a staged Python syntax error blocks (exit 1)" { + printf 'def f(:\n return 1\n' > bad.py + git add bad.py + run bash "$HOOK" + [ "$status" -eq 1 ] + [[ "$output" == *"syntax"* ]] +} + +@test "pre-commit(py): a .pyi stub with a syntax error blocks (exit 1)" { + printf 'def f( -> int: ...\n' > bad.pyi + git add bad.pyi + run bash "$HOOK" + [ "$status" -eq 1 ] +} + +@test "pre-commit(py): a broken NON-Python file does not trip the syntax gate" { + printf 'this is (((not python\n' > notes.txt + git add notes.txt + run bash "$HOOK" + [ "$status" -eq 0 ] +} + +@test "pre-commit(py): the syntax gate leaves no __pycache__ in the repo" { + printf 'def f():\n return 1\n' > ok.py + git add ok.py + run bash "$HOOK" + [ "$status" -eq 0 ] + [ ! -d __pycache__ ] +} diff --git a/languages/python/tests/validate-python.bats b/languages/python/tests/validate-python.bats new file mode 100644 index 0000000..b5e4957 --- /dev/null +++ b/languages/python/tests/validate-python.bats @@ -0,0 +1,117 @@ +#!/usr/bin/env bats +# +# Tests for languages/python/claude/hooks/validate-python.sh — the PostToolUse +# hook that syntax-checks edited Python files and blocks on a violation. +# +# The hook reads tool-call JSON on stdin and extracts the file path, so each +# test pipes a JSON payload naming a real file it wrote into a temp dir. +# +# The syntax gate is python3's own compiler, which is present wherever the hook +# can meaningfully run, so those tests never skip. The lint gate (ruff) is +# optional and its tests skip when it's absent, matching the bash bundle's +# treatment of shellcheck. + +HOOK="$(cd "$(dirname "$BATS_TEST_FILENAME")/.." && pwd)/claude/hooks/validate-python.sh" + +setup() { + TEST_DIR="$(mktemp -d -t validate-python-bats.XXXXXX)" +} + +teardown() { + rm -rf "$TEST_DIR" +} + +payload() { + printf '{"tool_input": {"file_path": "%s"}}' "$1" +} + +# ---- Normal ---------------------------------------------------------- + +@test "validate-python: a clean .py file passes silently (exit 0)" { + printf 'def f(x):\n return x + 1\n' > "$TEST_DIR/clean.py" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/clean.py")" + [ "$status" -eq 0 ] + [ -z "$output" ] +} + +@test "validate-python: a .pyi stub is validated too" { + printf 'def f(x: int) -> int: ...\n' > "$TEST_DIR/clean.pyi" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/clean.pyi")" + [ "$status" -eq 0 ] +} + +# ---- Error ----------------------------------------------------------- + +@test "validate-python: a syntax error blocks (exit 2, names the failure)" { + printf 'def f(:\n return 1\n' > "$TEST_DIR/bad.py" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/bad.py")" + [ "$status" -eq 2 ] + [[ "$output" == *"SYNTAX"* ]] +} + +@test "validate-python: the block payload is valid JSON carrying the context" { + printf 'def f(:\n' > "$TEST_DIR/bad.py" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/bad.py")" + [ "$status" -eq 2 ] + # The first line of stdout must parse as JSON and carry the hook event name. + echo "$output" | head -1 | jq -e '.hookSpecificOutput.hookEventName == "PostToolUse"' +} + +@test "validate-python: a ruff violation blocks when ruff is installed" { + command -v ruff >/dev/null 2>&1 || skip "ruff not installed" + # F821: reference to an undefined name — syntactically valid, lint-caught. + printf 'def f():\n return undefined_name\n' > "$TEST_DIR/lint.py" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/lint.py")" + [ "$status" -eq 2 ] + [[ "$output" == *"RUFF"* ]] +} + +# ---- Boundary -------------------------------------------------------- + +@test "validate-python: a non-Python file is ignored (exit 0)" { + printf 'not python at all (((\n' > "$TEST_DIR/notes.txt" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/notes.txt")" + [ "$status" -eq 0 ] +} + +@test "validate-python: an extensionless file with a python shebang is validated" { + printf '#!/usr/bin/env python3\ndef f(:\n' > "$TEST_DIR/cli-tool" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/cli-tool")" + [ "$status" -eq 2 ] +} + +@test "validate-python: an extensionless non-python file is ignored (exit 0)" { + printf '#!/usr/bin/env bash\necho hi\n' > "$TEST_DIR/shell-tool" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/shell-tool")" + [ "$status" -eq 0 ] +} + +@test "validate-python: a dotted parent directory does not misfire the extension test" { + mkdir -p "$TEST_DIR/my.project" + printf '#!/usr/bin/env python3\ndef f(:\n' > "$TEST_DIR/my.project/cli-tool" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/my.project/cli-tool")" + [ "$status" -eq 2 ] +} + +@test "validate-python: empty file_path is a no-op (exit 0)" { + run bash "$HOOK" <<< '{"tool_input": {}}' + [ "$status" -eq 0 ] +} + +@test "validate-python: a missing file is a no-op (exit 0)" { + run bash "$HOOK" <<< "$(payload "$TEST_DIR/does-not-exist.py")" + [ "$status" -eq 0 ] +} + +@test "validate-python: an empty .py file passes (valid, compiles to nothing)" { + : > "$TEST_DIR/empty.py" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/empty.py")" + [ "$status" -eq 0 ] +} + +@test "validate-python: compiling leaves no __pycache__ beside the file" { + printf 'def f():\n return 1\n' > "$TEST_DIR/clean.py" + run bash "$HOOK" <<< "$(payload "$TEST_DIR/clean.py")" + [ "$status" -eq 0 ] + [ ! -d "$TEST_DIR/__pycache__" ] +} |