From 06626390e75db998164f56fb968059c56502a18b Mon Sep 17 00:00:00 2001 From: Craig Jennings Date: Mon, 13 Apr 2026 03:35:00 -0400 Subject: docs: document bats unit tests + sync stale README bits testing-strategy.org: - Frame the two test layers (bats unit / VM integration) in Overview - Add 'Unit Tests (bats)' section: what's covered, what's deliberately not, how to run, how to install, and the extract-then-test pattern - Fix stale Makefile Targets table (bats row, test row now lint+bats) README.org: - Rename 'Testing with VMs' to 'Testing' and add the two-layer framing - Surface make test / make bats / make lint in the bulleted list - Project Structure tree: drop zfs.sh (deleted last session), add raid.sh with description, add tests/unit/ entry - Point archzfs link to GitHub Releases (archzfs.com was abandoned mid-2025; url updated in code last session) - Fix #testing-with-vms internal link to match the renamed heading --- testing-strategy.org | 64 +++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 61 insertions(+), 3 deletions(-) (limited to 'testing-strategy.org') diff --git a/testing-strategy.org b/testing-strategy.org index 6ae0e6f..4b948fd 100644 --- a/testing-strategy.org +++ b/testing-strategy.org @@ -4,8 +4,13 @@ * Overview -This document describes the testing strategy for the archzfs installer project, -including automated VM testing and the rationale for key technical decisions. +This document describes the testing strategy for the archangel installer +project, including automated VM testing and the rationale for key technical +decisions. + +Testing has two layers: fast [[#unit-tests-bats][bats unit tests]] for pure logic in +=installer/lib/*.sh=, and slower [[#running-tests][VM integration tests]] that exercise the +full install path against real block devices. * Running Tests @@ -19,8 +24,9 @@ including automated VM testing and the rationale for key technical decisions. | =make test-multi3= | Boot ISO in a 3-disk VM for raidz1 testing | | =make test-boot= | Boot from installed disk (after running install in VM) | | =make test-clean= | Remove VM disks and OVMF vars, start fresh | +| =make bats= | Run bats unit tests only (tests/unit/) | | =make lint= | Run shellcheck on all scripts | -| =make test= | Run lint (alias) | +| =make test= | Run lint + bats (fast; no VMs) | ** Running a Single Automated Test @@ -40,6 +46,58 @@ including automated VM testing and the rationale for key technical decisions. ./scripts/test-install.sh --list #+end_src +* Unit Tests (bats) +:PROPERTIES: +:CUSTOM_ID: unit-tests-bats +:END: + +** What bats covers + +The bats test suite exercises pure logic in =installer/lib/*.sh= — helpers +that can run without root, without block devices, and without a chroot. +It runs in under a second and is the fast feedback loop for every commit. + +Current coverage lives in =tests/unit/=: + +| File | What it covers | +|------+----------------| +| =test_common.bats= | =command_exists=, =require_command=, =info=/=warn=/=error=, =enable_color=, =log=, =prompt_password=, =pacstrap_packages= | +| =test_config.bats= | =parse_args=, =load_config=, =validate_config=, =check_config= | +| =test_raid.bats= | =raid_valid_levels_for_count=, =raid_is_valid=, =raid_usable_bytes=, =raid_fault_tolerance= | + +** What bats does NOT cover (deliberately) + +Anything that shells out to =mkfs=, =cryptsetup=, =zpool create=, +=pacstrap=, =arch-chroot=, =grub-install=, or needs root. Those behaviors +only mean anything against real partitions on real (virtual) hardware and +belong in the VM integration tests below. + +** Running + +#+begin_src bash +make bats # bats only +make test # lint + bats (pre-commit check) +bats tests/unit/ # direct invocation, same result +#+end_src + +** Installing bats + +#+begin_src bash +sudo pacman -S bats # Arch +brew install bats-core # macOS +apt install bats # Debian/Ubuntu +#+end_src + +The =make bats= target prints an install hint if =bats= isn't on =PATH=. + +** The pattern for adding coverage + +When a refactor extracts pure logic out of a monolithic installer +function into a =lib/*.sh= helper, add bats cases for the helper in the +same commit. Don't try to write bats tests against the monolith +directly — extract, then test. That's how =raid.sh=, =pacstrap_packages=, +and =prompt_password= got covered. + * Test Infrastructure ** Test Scripts -- cgit v1.2.3