diff options
Diffstat (limited to 'scripts/testing/README.org')
| -rw-r--r-- | scripts/testing/README.org | 379 |
1 files changed, 83 insertions, 296 deletions
diff --git a/scripts/testing/README.org b/scripts/testing/README.org index a52dbea..e58a89e 100644 --- a/scripts/testing/README.org +++ b/scripts/testing/README.org @@ -10,20 +10,23 @@ This directory contains the complete testing infrastructure for archsetup, built *Realism over speed.* We use full VMs (not containers) to test everything archsetup does: user creation, systemd services, X11/DWM, hardware drivers, and boot process. +** Architecture + +Uses direct QEMU with user-mode networking and SSH port forwarding. No libvirt/virsh dependency. Base VM creation is fully automated via the archangel ISO (unattended Arch Linux installer). + * Quick Start ** One-Time Setup #+begin_src bash -# Install required packages and configure environment +# Install required packages (qemu-full, sshpass, edk2-ovmf, socat) ./scripts/testing/setup-testing-env.sh -# Log out and back in (for libvirt group membership) +# Copy an archangel ISO to vm-images/ +cp /path/to/archzfs-*.iso vm-images/ -# Create the base VM (minimal Arch installation) +# Create the base VM (fully automated, no manual steps) ./scripts/testing/create-base-vm.sh -# Follow on-screen instructions to complete installation -# Run finalize-base-vm.sh when done #+end_src ** Run a Test @@ -42,14 +45,11 @@ This directory contains the complete testing infrastructure for archsetup, built ** Debug Interactively #+begin_src bash -# Clone base VM for debugging -./scripts/testing/debug-vm.sh - -# Use existing test disk -./scripts/testing/debug-vm.sh test-results/20251108-143000/test.qcow2 - -# Use base VM (read-only) +# Launch base VM with graphical display (copy-on-write overlay, safe) ./scripts/testing/debug-vm.sh --base + +# Use existing test disk directly +./scripts/testing/debug-vm.sh vm-images/some-disk.qcow2 #+end_src ** Clean Up @@ -72,82 +72,44 @@ This directory contains the complete testing infrastructure for archsetup, built *Purpose:* One-time setup of testing infrastructure *What it does:* -- Installs QEMU/KVM, libvirt, and related tools -- Configures libvirt networking -- Adds user to libvirt group +- Installs QEMU/KVM, sshpass, OVMF firmware, and socat - Verifies KVM support - Creates directories for artifacts *When to run:* Once per development machine -*Usage:* -#+begin_src bash -./scripts/testing/setup-testing-env.sh -#+end_src - ** create-base-vm.sh *Purpose:* Create the "golden image" minimal Arch VM *What it does:* -- Downloads latest Arch ISO -- Creates VM and boots from ISO -- Opens virt-viewer for you to complete installation manually +- Boots an archangel ISO in QEMU with SSH port forwarding +- Waits for SSH to become available on the live ISO +- Copies archangel config file (=archsetup-test.conf=) into the VM +- Runs =archangel --config-file= for fully unattended installation +- Boots the installed system and verifies SSH/services +- Creates a "clean-install" qemu-img snapshot *When to run:* Once (or when you want to refresh base image) -*Usage:* -#+begin_src bash -./scripts/testing/create-base-vm.sh -#+end_src - -*Process:* -1. Script creates VM and boots from Arch ISO -2. virt-viewer opens automatically showing VM display -3. You complete installation manually using archinstall: - - Login as root (no password) - - Run =archinstall= - - Configure: hostname=archsetup-test, root password=archsetup - - Install packages: openssh git vim sudo - - Enable services: sshd, dhcpcd - - Configure SSH to allow root login - - Poweroff when done -4. Run =./scripts/testing/finalize-base-vm.sh= to complete - -*See also:* [[file:../../docs/base-vm-installation-checklist.org][Base VM Installation Checklist]] - -*Result:* Base VM image at =vm-images/archsetup-base.qcow2= +*Result:* Base VM image at =vm-images/archsetup-base.qcow2= with "clean-install" snapshot ** run-test.sh *Purpose:* Execute a full test run of archsetup *What it does:* -- Clones base VM (copy-on-write, fast) -- Starts test VM -- Transfers archsetup script and dotfiles +- Reverts base VM to clean-install snapshot +- Boots the VM via QEMU +- Transfers archsetup script and dotfiles via git bundle - Executes archsetup inside VM - Captures logs and results -- Runs validation checks +- Runs comprehensive validation checks - Generates test report -- Cleans up (unless =--keep=) +- Reverts to clean snapshot (unless =--keep=) *When to run:* Every time you want to test archsetup -*Usage:* -#+begin_src bash -# Test current archsetup -./scripts/testing/run-test.sh - -# Test specific version -./scripts/testing/run-test.sh --script /path/to/archsetup - -# Keep VM for debugging -./scripts/testing/run-test.sh --keep -#+end_src - -*Time:* 30-60 minutes (mostly downloading packages) - *Results:* Saved to =test-results/TIMESTAMP/= - =test.log= - Complete log output - =test-report.txt= - Summary of results @@ -159,51 +121,34 @@ This directory contains the complete testing infrastructure for archsetup, built *Purpose:* Launch VM for interactive debugging *What it does:* -- Creates VM from base image or existing test disk -- Configures console and SSH access -- Provides connection instructions +- Creates a copy-on-write overlay disk (instant, protects base image) +- Boots QEMU with GTK graphical display +- Provides SSH connection info +- Cleans up overlay when the GTK window is closed *When to run:* When you need to manually test or debug -*Usage:* -#+begin_src bash -# Clone base VM for debugging -./scripts/testing/debug-vm.sh - -# Use existing test disk -./scripts/testing/debug-vm.sh vm-images/archsetup-test-20251108-143000.qcow2 - -# Use base VM (read-only) -./scripts/testing/debug-vm.sh --base -#+end_src - *Connect via:* -- Console: =virsh console archsetup-debug-TIMESTAMP= -- SSH: =ssh root@IP_ADDRESS= (password: archsetup) -- VNC: =virt-viewer archsetup-debug-TIMESTAMP= +- GTK window (opens automatically) +- SSH: =sshpass -p 'archsetup' ssh -p 2222 root@localhost= ** cleanup-tests.sh *Purpose:* Clean up old test VMs and artifacts *What it does:* -- Lists all test VMs and destroys them -- Removes test disk images +- Stops running QEMU processes +- Kills orphaned QEMU processes +- Removes temporary disk images (overlays, test clones) - Keeps last N test results, deletes rest *When to run:* Periodically to free disk space -*Usage:* -#+begin_src bash -# Interactive cleanup -./scripts/testing/cleanup-tests.sh +** archsetup-test.conf -# Keep last 3 test results -./scripts/testing/cleanup-tests.sh --keep 3 +*Purpose:* Archangel config file for automated base VM creation -# Force without prompts -./scripts/testing/cleanup-tests.sh --force -#+end_src +Defines the base system: btrfs filesystem, no encryption, SSH enabled, root password "archsetup". Used by =create-base-vm.sh=. * Directory Structure @@ -213,27 +158,30 @@ archsetup/ │ └── testing/ │ ├── README.org # This file │ ├── setup-testing-env.sh # Setup infrastructure -│ ├── create-base-vm.sh # Create base VM +│ ├── create-base-vm.sh # Create base VM (automated) │ ├── run-test.sh # Run tests +│ ├── run-test-baremetal.sh # Bare-metal testing │ ├── debug-vm.sh # Interactive debugging │ ├── cleanup-tests.sh # Clean up -│ ├── finalize-base-vm.sh # Finalize base (generated) -│ ├── archinstall-config.json # Archinstall config +│ ├── archsetup-test.conf # Archangel config for test VMs │ └── lib/ │ ├── logging.sh # Logging utilities -│ └── vm-utils.sh # VM management +│ ├── vm-utils.sh # QEMU VM management +│ ├── validation.sh # Test validation checks +│ └── network-diagnostics.sh # Network pre-flight checks ├── vm-images/ # VM disk images (gitignored) -│ ├── archsetup-base.qcow2 # Golden image -│ ├── arch-latest.iso # Arch ISO -│ └── archsetup-test-*.qcow2 # Test VMs +│ ├── archsetup-base.qcow2 # Golden image with snapshot +│ ├── archzfs-*.iso # Archangel ISO +│ ├── OVMF_VARS.fd # UEFI variables (per-VM) +│ └── debug-overlay-*.qcow2 # Temp debug overlays ├── test-results/ # Test results (gitignored) -│ ├── TIMESTAMP/ -│ │ ├── test.log -│ │ ├── test-report.txt -│ │ └── archsetup-*.log -│ └── latest -> TIMESTAMP/ # Symlink to latest +│ └── TIMESTAMP/ +│ ├── test.log +│ ├── test-report.txt +│ └── archsetup-*.log └── docs/ - └── testing-strategy.org # Complete strategy doc + ├── testing-strategy.org # Complete strategy doc + └── archangel-vm-testing-guide.org # Archangel integration guide #+end_example * Configuration @@ -241,30 +189,27 @@ archsetup/ ** VM Specifications All test VMs use: -- *CPUs:* 2 vCPUs -- *RAM:* 4GB (matches archsetup tmpfs build directory) +- *CPUs:* 4 vCPUs +- *RAM:* 4GB - *Disk:* 50GB (thin provisioned qcow2) -- *Network:* NAT via libvirt default network -- *Boot:* UEFI (systemd-boot bootloader) -- *Display:* Serial console + VNC +- *Network:* User-mode networking with SSH port forwarding (localhost:2222) +- *Boot:* UEFI (via OVMF firmware) +- *Display:* Headless (automated) or GTK (debug) Set environment variables to customize: #+begin_src bash -VM_CPUS=4 VM_RAM=8192 ./scripts/testing/run-test.sh +VM_CPUS=8 VM_RAM=8192 ./scripts/testing/run-test.sh #+end_src ** Base VM Specifications -The base VM contains a minimal Arch installation: -- Base system packages -- Linux kernel and firmware -- OpenSSH server (for automation) -- dhcpcd (for networking) -- git, vim, sudo (essentials) +The base VM is created automatically by archangel with: +- Btrfs filesystem (no encryption) +- GRUB bootloader +- OpenSSH server (root login enabled) +- NetworkManager - Root password: "archsetup" -This matches the documented prerequisites for archsetup. - * Validation Checks Each test run performs these validation checks: @@ -274,62 +219,37 @@ Each test run performs these validation checks: 2. User 'cjennings' was created 3. Dotfiles are stowed (symlinks exist) 4. yay (AUR helper) is installed -5. DWM is built and installed +5. DWM or Hyprland is configured -** Additional (Future) +** Additional - All expected packages installed -- X11 can start - systemd services enabled - Firewall configured +- Developer tools present * Troubleshooting ** VM fails to start -Check if libvirtd is running: -#+begin_src bash -sudo systemctl status libvirtd -sudo systemctl start libvirtd -#+end_src - -** Cannot get VM IP address - -The VM may not have booted completely or networking failed: +Check if KVM is available and port 2222 is free: #+begin_src bash -# Check VM status -virsh domstate VM_NAME - -# Connect to console to debug -virsh console VM_NAME - -# Check if dhcpcd is running in VM -systemctl status dhcpcd +ls -l /dev/kvm +ss -tln | grep 2222 #+end_src ** SSH connection refused -Wait longer for VM to boot, or check if sshd is enabled: +Wait longer for VM to boot, or check serial log: #+begin_src bash -virsh console VM_NAME -# Inside VM: -systemctl status sshd -systemctl start sshd +cat vm-images/qemu-serial.log #+end_src ** KVM not available Check if virtualization is enabled in BIOS and KVM modules loaded: #+begin_src bash -# Check for /dev/kvm ls -l /dev/kvm - -# Load KVM module (Intel) -sudo modprobe kvm-intel - -# Or for AMD -sudo modprobe kvm-amd - -# Verify +sudo modprobe kvm-amd # or kvm-intel lsmod | grep kvm #+end_src @@ -338,160 +258,27 @@ lsmod | grep kvm Clean up old tests: #+begin_src bash ./scripts/testing/cleanup-tests.sh --force -#+end_src - -Check disk usage: -#+begin_src bash du -sh vm-images/ test-results/ #+end_src -** Base VM Installation Issues - -*** Firewall blocking VM network - -If the VM cannot reach the internet (100% packet loss when pinging), check the host firewall: - -#+begin_src bash -# Check UFW status on host -sudo ufw status - -# Check libvirt NAT rules on host -sudo iptables -t nat -L -n -v | grep -i libvirt -#+end_src - -*Solution:* Temporarily disable UFW during base VM creation: -#+begin_src bash -sudo ufw disable -# Create base VM -sudo ufw enable -#+end_src - -Or add libvirt rules to UFW: -#+begin_src bash -sudo ufw allow in on virbr0 -sudo ufw allow out on virbr0 -#+end_src - -*** VM network not working after boot - -If =dhcpcd= isn't running or network isn't configured: - -#+begin_src bash -# In the VM - restart network from scratch -killall dhcpcd - -# Bring interface down and up (replace enp1s0 with your interface) -ip link set enp1s0 down -ip link set enp1s0 up - -# Start dhcpcd -dhcpcd enp1s0 - -# Wait and verify -sleep 3 -ip addr show enp1s0 -ip route - -# Test connectivity -ping -c 3 192.168.122.1 # Gateway -ping -c 3 8.8.8.8 # Google DNS -#+end_src - -*** DNS not working (127.0.0.53 in resolv.conf) - -The Live ISO uses systemd-resolved stub resolver which may not work: - -#+begin_src bash -# In the VM - set real DNS servers -echo "nameserver 8.8.8.8" > /etc/resolv.conf -echo "nameserver 1.1.1.1" >> /etc/resolv.conf - -# Test -ping -c 3 archlinux.org -#+end_src - -*** Cannot paste into virt-viewer terminal - -Clipboard integration doesn't work well with virt-viewer. Use HTTP server instead: - -#+begin_src bash -# On host - serve the installation script -cd vm-images -python -m http.server 8000 - -# In VM - download and run -curl http://192.168.122.1:8000/auto-install.sh | bash - -# Or download first to review -curl http://192.168.122.1:8000/auto-install.sh -o install.sh -cat install.sh -bash install.sh -#+end_src - -*** Partitions "in use" error - -If re-running installation after a failed attempt: +** Port 2222 already in use +Another QEMU instance may be running: #+begin_src bash -# In the VM - unmount and wipe partitions -mount | grep vda -umount /mnt/boot 2>/dev/null -umount /mnt 2>/dev/null -umount -l /mnt # Lazy unmount if still busy - -# Wipe partition table completely -wipefs -a /dev/vda - -# Run install script again -bash install.sh -#+end_src - -*** Alternative: Use archinstall - -Instead of the auto-install.sh script, you can use Arch's built-in installer: - -#+begin_src bash -# In the VM -archinstall -#+end_src - -*Recommended settings:* -- Disk: =/dev/vda= -- Filesystem: =ext4= -- Bootloader: =systemd-boot= -- Hostname: =archsetup-test= -- Root password: =archsetup= -- Profile: =minimal= -- Additional packages: =openssh dhcpcd git vim sudo= -- Network: =NetworkManager= or =systemd-networkd= - -*After installation, before rebooting:* -#+begin_src bash -# Chroot into new system -arch-chroot /mnt - -# Enable services -systemctl enable sshd -systemctl enable dhcpcd # or NetworkManager - -# Allow root SSH login -sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config -sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config +# Find what's using the port +ss -tlnp | grep 2222 -# Exit and poweroff -exit -poweroff +# Kill orphaned QEMU processes +pkill -f "qemu-system.*archsetup-test" #+end_src * Future Enhancements ** Planned Improvements -- [ ] Fully automated base VM creation (expect-based console automation) -- [ ] Parallel test execution (multiple VMs) -- [ ] Screenshot capture of X11 desktop +- [ ] Parallel test execution (multiple VMs on different ports) +- [ ] Screenshot capture of X11/Wayland desktop - [ ] CI/CD integration (GitHub Actions / GitLab CI) - [ ] Performance benchmarking over time -- [ ] Cloud-init based base image (faster provisioning) ** Test Scenarios - [ ] Idempotency test (run archsetup twice) @@ -502,7 +289,7 @@ poweroff * References - [[file:../../docs/testing-strategy.org][Testing Strategy Document]] -- [[https://wiki.archlinux.org/title/Libvirt][Arch Wiki: libvirt]] +- [[file:../../docs/archangel-vm-testing-guide.org][Archangel VM Testing Guide]] - [[https://wiki.archlinux.org/title/QEMU][Arch Wiki: QEMU]] - [[file:../../archsetup][Main archsetup script]] -- [[file:../../TODO.org][Project TODO]] +- [[file:../../todo.org][Project TODO]] |