From 3cac3b3dfcd432395201a309920c2491ee9caf01 Mon Sep 17 00:00:00 2001 From: Craig Jennings Date: Thu, 25 Jun 2026 01:12:35 -0400 Subject: test(archsetup): port full shell validation sweep to Testinfra (P2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Port all ~26 post-install checks from validation.sh to pytest/Testinfra, reaching parity before the cutover. Adds test_users, test_packages, test_services, test_desktop, test_boot, test_keyring, and test_archsetup (88 tests after parametrizing groups, services, timers, tools, and configs), plus shared conftest fixtures for ZFS/NVMe/compositor/networking gating. The shell sweep's three outcomes map cleanly: hard failures become assertions, advisory warnings and unmet preconditions (headless compositor, slirp networking, optional services, non-ZFS/non-NVMe hosts) become skips. One correctness fix vs the shell sweep: check awww, not swww — archsetup installs awww (swww's successor) and `pacman -Q swww` no longer matches. Verified on the host: py_compile clean, pytest --collect-only green (88 tests). The sweep against a real VM is verified by the make test run that follows. --- todo.org | 2 ++ 1 file changed, 2 insertions(+) (limited to 'todo.org') diff --git a/todo.org b/todo.org index 16b607e..e0da8e6 100644 --- a/todo.org +++ b/todo.org @@ -540,6 +540,8 @@ Done 2026-06-25: added a =backup_system_file = helper next to =safe_rm_rf= Reviewed against the existing harness: =scripts/testing/lib/validation.sh= already runs ~14 post-install checks (=run_all_validations=), so this isn't net-new capability — it's porting that shell validation to Testinfra/pytest for better expressiveness + reporting, then growing coverage. Craig's call (prioritizes test investment over feature speed): do the port and expand. Starting with a design doc in =docs/design/= per the task's own "design doc not yet written" note. Stale slice to drop/rescope: the X11/startx end-to-end tests (fleet is Wayland/Hyprland now). *** 2026-06-25 Thu @ 00:54:22 -0400 P1 scaffold landed (advisory, alongside shell sweep) Built the Testinfra harness skeleton: =scripts/testing/tests/= (conftest.py with the attribution marker + report hook + =target_user= fixture; 3 parity checks — user exists/shell, ufw enabled, dotfiles stowed+readable), =scripts/testing/lib/testinfra.sh= (=run_testinfra_validation=: ephemeral-key injection, ssh-config, pytest-over-SSH; advisory + non-fatal, =RUN_TESTINFRA= toggle), wired into run-test.sh after the shell sweep, and added =python-pytest python-pytest-testinfra= to =make deps=. Verified on host: py_compile clean, =pytest --collect-only= green in a throwaway venv (4 tests, fixtures resolve), =bash -n= + shellcheck clean, unit suite still green. Integration (the pytest sweep actually running against a VM) is unverified here — needs a =make test= run. Decisions locked: inject test key; run both through parity; full expansion (P4) in this task after the P3 cutover. +*** 2026-06-25 Thu @ 01:12:09 -0400 P2 full parity port (88 tests) +Ported the whole shell sweep to pytest: test_users (exists/shell/15 groups parametrized), test_packages (yay+functional, pacman, terminus-font, emacs+config readable, git, 5 dev tools), test_services (required enabled/active, enabled-only, timers, optional skip-if-absent, DoT drop-in, fail2ban/nmcli responds, log-cleanup cron, syncthing lingering, DNS/mDNS/docker skips), test_desktop (Hyprland tools+configs+portal+socket gated on install/compositor, DWM suckless, autologin), test_boot (grub, mkinitcpio hooks branched on zfs_root, console-font-in-initramfs, nvme gated, zfs/sanoid), test_keyring (dir 700/owner/default=login), test_archsetup (log no Error:, ≥12 state markers). conftest fixtures: target_user/home/zfs_root/has_nvme/hyprland_installed/dwm_installed/compositor_running/on_slirp. 88 tests collected, py_compile clean. Correctness fix vs the shell sweep: check =awww= not the stale =swww=. Installed python-pytest-testinfra on velox so the harness gate passes. Next: VM run to diff pytest vs shell sweep for parity. Create comprehensive integration tests using Testinfra (Python + pytest) to validate archsetup installations Tests should cover: -- cgit v1.2.3