archsetup/scripts/testing/run-test.sh, branch claude/hungry-franklin-09fffc

refactor(test): share runner and scenario contracts

2026-07-25T20:00:44+00:00

fix(test): harden four VM-framework paths from the S5 audit

2026-07-21T04:40:56+00:00

init_vm_paths suffixed the disk and NVRAM by FS_PROFILE but left PID_FILE, MONITOR_SOCK, and SERIAL_LOG shared. A concurrent btrfs and zfs run read each other's PID file, so the second run's stop logic could kill the first run's VM. All runtime paths now carry the suffix. kill_qemu sent kill -9 and returned without waiting, so the force-kill fallback could run qemu-img snapshot against a qcow2 the dying qemu still held locked (the restore failed silently behind || true, leaving the base image dirty). It now reaps or polls the process to death before returning. debug-vm.sh hardcoded the btrfs base-disk name, so FS_PROFILE=zfs booted the wrong base or fatalled. It now takes DISK_PATH from init_vm_paths. Both runners reported a completion-marker grep as "ArchSetup Exit Code," but the installer runs detached with set -e off, so its real exit status was never captured -- a run that errored mid-way and still reached its last line reported exit 0. The flag is now ARCHSETUP_COMPLETED with the report labeled honestly; Testinfra stays the pass/fail authority.

test(archsetup): add FS_PROFILE selector for ZFS VM coverage

2026-06-25T16:21:15+00:00

The VM harness only built one btrfs base image, so every ZFS-conditional check in the Testinfra suite skipped and the ZFS install path went untested in automation. I added an FS_PROFILE selector (btrfs default, zfs) so `make test FS_PROFILE=zfs` can target a ZFS root. init_vm_paths derives the image name from FS_PROFILE and validates it. btrfs keeps the legacy unsuffixed archsetup-base.qcow2 so existing images and invocations are untouched. The zfs profile gets archsetup-base-zfs.qcow2. create-base-vm.sh picks archsetup-test.conf vs the new archsetup-test-zfs.conf (FILESYSTEM=zfs, NO_ENCRYPT=yes for an unattended install), and the Makefile resolves the matching image for its base-VM check. The archsetup run config stays shared. archsetup reads no filesystem key. It detects ZFS from the live root via is_zfs_root, so the ZFS branch fires on its own once the base image is ZFS. The design doc is reconciled to that: no separate archsetup-vm-zfs.conf, and the non-ZFS profile is btrfs, not ext4. Building the ZFS base image and running the ZFS sweep green is next.

fix(testing): raise the install monitor timeout to 150 minutes

2026-06-25T12:35:28+00:00

A full archsetup install with heavy AUR builds (vagrant and its git-cloned installers) can run past the old 90-minute monitor cap on a slow mirror. When that happened the run stopped monitoring mid-install and validated a half-installed system, producing spurious late-step failures. Raise MAX_POLLS from 180 to 300 (90 -> 150 minutes) so a slow-but-healthy install completes.

test(archsetup): make Testinfra the authoritative validator (P3 cutover)

2026-06-25T07:39:10+00:00

run-test.sh no longer runs the shell run_all_validations sweep; the Testinfra pytest sweep now drives the run's pass/fail. run_testinfra_validation returns pytest's exit code (and treats "could not run" as a failure, not a silent pass), surfaces the pass/skip/fail counts through the shared VALIDATION_* counters, and parses the attribution file so generate_issue_report still buckets failures into archsetup / base_install / unknown. The shell-sweep functions stay in validation.sh for now because run-test-baremetal.sh still calls them; removing them (after migrating the bare-metal runner) is filed as a follow-up.

fix(testing): authorize a root key so make test survives sshd hardening

2026-06-25T05:24:33+00:00

The VM test SSHes into the guest as root with a password for the whole run. archsetup hardens sshd to PermitRootLogin prohibit-password and reloads it partway through the install, so every SSH after that step failed with "Permission denied" and the run aborted before any validation — make test had been silently broken since the hardening landed. inject_root_key authorizes a throwaway root key right after the first SSH (before archsetup runs) and the ssh/scp helpers now add -i via SSH_KEY_OPT. prohibit-password still allows root key auth, so the harness survives the very hardening it validates. Password stays as the fallback, so the change is additive.

test(archsetup): scaffold Testinfra post-install validation (P1)

2026-06-25T04:54:53+00:00

Stand up the Testinfra/pytest harness alongside the existing shell sweep so the two can be compared for parity before pytest takes over. Adds scripts/testing/tests/ (conftest with failure attribution markers, a report hook, and a target_user fixture, plus three parity checks: user, ufw, dotfiles) and scripts/testing/lib/testinfra.sh, which injects a throwaway SSH key into the VM and runs pytest over SSH. The sweep is advisory here (RUN_TESTINFRA toggle, non-fatal) and does not yet affect pass/fail. Pulls python-pytest and python-pytest-testinfra into make deps. Verified on the host: py_compile clean, pytest --collect-only green, bash -n and shellcheck clean. The sweep running against a real VM is verified by the next make test run.

chore: open-source release-prep (udev flag, SPDX headers, boolean style)

2026-06-24T23:44:29+00:00

Three release cleanups, all behavior-preserving for my machines: - Gated the Logitech BRIO udev rule behind INSTALL_DEVICE_UDEV_RULES (default yes, opt-out), so the device-specific rule is off for anyone without that hardware. Added the config read, validation, and a conf.example entry. - Added a GPL-3.0-or-later SPDX-License-Identifier header after the shebang of all 24 shell scripts in the repo. - Standardized boolean conditionals on the explicit [ "$var" = "true" ] form, replacing the bare `if $var` idiom. The STEPS function-dispatch is left alone, since it runs a function name rather than testing a boolean.

refactor(testing): clone the dotfiles repo into the VM test

2026-05-22T20:52:24+00:00

The dotfiles live in a separate repo now, so the VM test clones that source, bundles it, and clones from the bundle into /tmp/dotfiles-test on the VM. archsetup's in-VM clone of DOTFILES_REPO then resolves against the local path with no network. DOTFILES_SOURCE overrides the source for testing a local checkout instead of the published repo. The cleanup trap tracks the host-side bundle and clone, so a mid-run abort leaves nothing behind.

fix: correct a POSIX-sh bashism in init and document two SC2034 cases

2026-05-20T17:02:47+00:00

init runs under #!/bin/sh but used $(