diff options
| -rw-r--r-- | README.org | 34 | ||||
| -rw-r--r-- | testing-strategy.org | 64 |
2 files changed, 80 insertions, 18 deletions
@@ -57,7 +57,7 @@ The build script will report if you're missing any of these in a preflight check ** Basic Build -The makefile is your primary entry point for building, [[#testing-with-vms][testing]], and cleaning the project ISO. +The makefile is your primary entry point for building, [[#testing][testing]], and cleaning the project ISO. #+BEGIN_SRC bash make build @@ -303,16 +303,19 @@ installation and can decline, or set ~ENABLE_SSH=no~ in a config file. *Important*: Harden SSH after installation — switch to key-based authentication and consider installing ~fail2ban~. -* Testing with VMs +* Testing -QEMU VMs validate that the ISO boots correctly and that installations complete -successfully across all supported configurations (ZFS/Btrfs, single/multi-disk, -encrypted/unencrypted). Run these after modifying the installer or build scripts -to catch regressions. +Two test layers: ~make test~ runs shellcheck + bats unit tests for the pure +logic in ~installer/lib/*.sh~ (fast, no VMs). ~make test-install~ runs the full +VM integration suite against a freshly built ISO. Use the former every commit; +the latter before releases or when touching install paths. -See [[file:testing-strategy.org][testing-strategy.org]] for the full testing strategy, -including how to add new tests and technical details on encryption testing. +See [[file:testing-strategy.org][testing-strategy.org]] for the full testing strategy, unit-test coverage, +how to add new tests, and technical details on encryption testing. +- ~make test~ — Run shellcheck + bats unit tests (~1s, no VMs needed). +- ~make bats~ — Run bats unit tests only. +- ~make lint~ — Run shellcheck on all scripts. - ~make test-vm~ — Create a 50GB virtual disk and boot the ISO in a single-disk QEMU VM. Opens a GTK window with the VM console. SSH in with ~ssh -p 2222 root@localhost~ (password: ~archangel~). @@ -328,7 +331,6 @@ including how to add new tests and technical details on encryption testing. first). Runs 12 unattended installs covering ZFS/Btrfs, single/multi-disk, encrypted/unencrypted, and custom locale. Each test installs, reboots from disk, verifies boot, and checks rollback. -- ~make lint~ — Run shellcheck on all scripts. * Project Structure @@ -340,14 +342,16 @@ archangel/ │ ├── archangel # Interactive installation script │ ├── archangel.conf.example # Example config for unattended install │ ├── lib/ # Modular installer components -│ │ ├── common.sh # Shared utilities -│ │ ├── config.sh # Configuration handling -│ │ ├── disk.sh # Disk partitioning -│ │ ├── zfs.sh # ZFS-specific functions -│ │ └── btrfs.sh # Btrfs-specific functions +│ │ ├── common.sh # Shared utilities, password prompt, pacstrap list +│ │ ├── config.sh # Argument parsing + config-file loading + validation +│ │ ├── disk.sh # Disk partitioning and EFI formatting +│ │ ├── btrfs.sh # Btrfs-specific functions (LUKS, subvolumes, GRUB) +│ │ └── raid.sh # Pure RAID-level logic (levels, validation, usable space) │ ├── zfssnapshot # ZFS snapshot utility │ ├── zfsrollback # ZFS rollback utility │ └── RESCUE-GUIDE.txt # Recovery tools documentation +├── tests/ +│ └── unit/ # Bats unit tests for installer/lib/*.sh ├── scripts/ │ ├── test-vm.sh # QEMU test VM launcher │ ├── test-install.sh # Automated install tests @@ -403,7 +407,7 @@ The ISO includes DKMS-built ZFS modules. If modules fail to load: * Links -- [[https://archzfs.com][archzfs Repository]] - ZFS packages for Arch Linux +- [[https://github.com/archzfs/archzfs][archzfs Repository]] - ZFS packages for Arch Linux (GitHub Releases; archzfs.com was abandoned mid-2025) - [[https://openzfs.github.io/openzfs-docs/][OpenZFS Documentation]] - Official ZFS documentation - [[https://get.zfsbootmenu.org][ZFSBootMenu]] - ZFS boot manager - [[https://wiki.archlinux.org/title/Btrfs][Arch Wiki - Btrfs]] - Btrfs information diff --git a/testing-strategy.org b/testing-strategy.org index 6ae0e6f..4b948fd 100644 --- a/testing-strategy.org +++ b/testing-strategy.org @@ -4,8 +4,13 @@ * Overview -This document describes the testing strategy for the archzfs installer project, -including automated VM testing and the rationale for key technical decisions. +This document describes the testing strategy for the archangel installer +project, including automated VM testing and the rationale for key technical +decisions. + +Testing has two layers: fast [[#unit-tests-bats][bats unit tests]] for pure logic in +=installer/lib/*.sh=, and slower [[#running-tests][VM integration tests]] that exercise the +full install path against real block devices. * Running Tests @@ -19,8 +24,9 @@ including automated VM testing and the rationale for key technical decisions. | =make test-multi3= | Boot ISO in a 3-disk VM for raidz1 testing | | =make test-boot= | Boot from installed disk (after running install in VM) | | =make test-clean= | Remove VM disks and OVMF vars, start fresh | +| =make bats= | Run bats unit tests only (tests/unit/) | | =make lint= | Run shellcheck on all scripts | -| =make test= | Run lint (alias) | +| =make test= | Run lint + bats (fast; no VMs) | ** Running a Single Automated Test @@ -40,6 +46,58 @@ including automated VM testing and the rationale for key technical decisions. ./scripts/test-install.sh --list #+end_src +* Unit Tests (bats) +:PROPERTIES: +:CUSTOM_ID: unit-tests-bats +:END: + +** What bats covers + +The bats test suite exercises pure logic in =installer/lib/*.sh= — helpers +that can run without root, without block devices, and without a chroot. +It runs in under a second and is the fast feedback loop for every commit. + +Current coverage lives in =tests/unit/=: + +| File | What it covers | +|------+----------------| +| =test_common.bats= | =command_exists=, =require_command=, =info=/=warn=/=error=, =enable_color=, =log=, =prompt_password=, =pacstrap_packages= | +| =test_config.bats= | =parse_args=, =load_config=, =validate_config=, =check_config= | +| =test_raid.bats= | =raid_valid_levels_for_count=, =raid_is_valid=, =raid_usable_bytes=, =raid_fault_tolerance= | + +** What bats does NOT cover (deliberately) + +Anything that shells out to =mkfs=, =cryptsetup=, =zpool create=, +=pacstrap=, =arch-chroot=, =grub-install=, or needs root. Those behaviors +only mean anything against real partitions on real (virtual) hardware and +belong in the VM integration tests below. + +** Running + +#+begin_src bash +make bats # bats only +make test # lint + bats (pre-commit check) +bats tests/unit/ # direct invocation, same result +#+end_src + +** Installing bats + +#+begin_src bash +sudo pacman -S bats # Arch +brew install bats-core # macOS +apt install bats # Debian/Ubuntu +#+end_src + +The =make bats= target prints an install hint if =bats= isn't on =PATH=. + +** The pattern for adding coverage + +When a refactor extracts pure logic out of a monolithic installer +function into a =lib/*.sh= helper, add bats cases for the helper in the +same commit. Don't try to write bats tests against the monolith +directly — extract, then test. That's how =raid.sh=, =pacstrap_packages=, +and =prompt_password= got covered. + * Test Infrastructure ** Test Scripts |