1 files changed, 508 insertions, 0 deletions
diff --git a/scripts/testing/README.org b/scripts/testing/README.org
new file mode 100644
index 0000000..a52dbea
--- /dev/null
+++ b/scripts/testing/README.org
@@ -0,0 +1,508 @@
+#+TITLE: ArchSetup Testing Infrastructure
+#+AUTHOR: Craig Jennings
+#+DATE: 2025-11-08
+
+* Overview
+
+This directory contains the complete testing infrastructure for archsetup, built on QEMU/KVM virtualization. It provides automated, reproducible testing of the archsetup installation script in isolated virtual machines.
+
+** Philosophy
+
+*Realism over speed.* We use full VMs (not containers) to test everything archsetup does: user creation, systemd services, X11/DWM, hardware drivers, and boot process.
+
+* Quick Start
+
+** One-Time Setup
+
+#+begin_src bash
+# Install required packages and configure environment
+./scripts/testing/setup-testing-env.sh
+
+# Log out and back in (for libvirt group membership)
+
+# Create the base VM (minimal Arch installation)
+./scripts/testing/create-base-vm.sh
+# Follow on-screen instructions to complete installation
+# Run finalize-base-vm.sh when done
+#+end_src
+
+** Run a Test
+
+#+begin_src bash
+# Test the current archsetup script
+./scripts/testing/run-test.sh
+
+# Test a specific version
+./scripts/testing/run-test.sh --script /path/to/archsetup
+
+# Keep VM running for debugging
+./scripts/testing/run-test.sh --keep
+#+end_src
+
+** Debug Interactively
+
+#+begin_src bash
+# Clone base VM for debugging
+./scripts/testing/debug-vm.sh
+
+# Use existing test disk
+./scripts/testing/debug-vm.sh test-results/20251108-143000/test.qcow2
+
+# Use base VM (read-only)
+./scripts/testing/debug-vm.sh --base
+#+end_src
+
+** Clean Up
+
+#+begin_src bash
+# Interactive cleanup (prompts for confirmation)
+./scripts/testing/cleanup-tests.sh
+
+# Keep last 10 test results
+./scripts/testing/cleanup-tests.sh --keep 10
+
+# Force cleanup without prompts
+./scripts/testing/cleanup-tests.sh --force
+#+end_src
+
+* Scripts
+
+** setup-testing-env.sh
+
+*Purpose:* One-time setup of testing infrastructure
+
+*What it does:*
+- Installs QEMU/KVM, libvirt, and related tools
+- Configures libvirt networking
+- Adds user to libvirt group
+- Verifies KVM support
+- Creates directories for artifacts
+
+*When to run:* Once per development machine
+
+*Usage:*
+#+begin_src bash
+./scripts/testing/setup-testing-env.sh
+#+end_src
+
+** create-base-vm.sh
+
+*Purpose:* Create the "golden image" minimal Arch VM
+
+*What it does:*
+- Downloads latest Arch ISO
+- Creates VM and boots from ISO
+- Opens virt-viewer for you to complete installation manually
+
+*When to run:* Once (or when you want to refresh base image)
+
+*Usage:*
+#+begin_src bash
+./scripts/testing/create-base-vm.sh
+#+end_src
+
+*Process:*
+1. Script creates VM and boots from Arch ISO
+2. virt-viewer opens automatically showing VM display
+3. You complete installation manually using archinstall:
+   - Login as root (no password)
+   - Run =archinstall=
+   - Configure: hostname=archsetup-test, root password=archsetup
+   - Install packages: openssh git vim sudo
+   - Enable services: sshd, dhcpcd
+   - Configure SSH to allow root login
+   - Poweroff when done
+4. Run =./scripts/testing/finalize-base-vm.sh= to complete
+
+*See also:* [[file:../../docs/base-vm-installation-checklist.org][Base VM Installation Checklist]]
+
+*Result:* Base VM image at =vm-images/archsetup-base.qcow2=
+
+** run-test.sh
+
+*Purpose:* Execute a full test run of archsetup
+
+*What it does:*
+- Clones base VM (copy-on-write, fast)
+- Starts test VM
+- Transfers archsetup script and dotfiles
+- Executes archsetup inside VM
+- Captures logs and results
+- Runs validation checks
+- Generates test report
+- Cleans up (unless =--keep=)
+
+*When to run:* Every time you want to test archsetup
+
+*Usage:*
+#+begin_src bash
+# Test current archsetup
+./scripts/testing/run-test.sh
+
+# Test specific version
+./scripts/testing/run-test.sh --script /path/to/archsetup
+
+# Keep VM for debugging
+./scripts/testing/run-test.sh --keep
+#+end_src
+
+*Time:* 30-60 minutes (mostly downloading packages)
+
+*Results:* Saved to =test-results/TIMESTAMP/=
+- =test.log= - Complete log output
+- =test-report.txt= - Summary of results
+- =archsetup-*.log= - Log from archsetup script
+- =*.txt= - Package lists from VM
+
+** debug-vm.sh
+
+*Purpose:* Launch VM for interactive debugging
+
+*What it does:*
+- Creates VM from base image or existing test disk
+- Configures console and SSH access
+- Provides connection instructions
+
+*When to run:* When you need to manually test or debug
+
+*Usage:*
+#+begin_src bash
+# Clone base VM for debugging
+./scripts/testing/debug-vm.sh
+
+# Use existing test disk
+./scripts/testing/debug-vm.sh vm-images/archsetup-test-20251108-143000.qcow2
+
+# Use base VM (read-only)
+./scripts/testing/debug-vm.sh --base
+#+end_src
+
+*Connect via:*
+- Console: =virsh console archsetup-debug-TIMESTAMP=
+- SSH: =ssh root@IP_ADDRESS= (password: archsetup)
+- VNC: =virt-viewer archsetup-debug-TIMESTAMP=
+
+** cleanup-tests.sh
+
+*Purpose:* Clean up old test VMs and artifacts
+
+*What it does:*
+- Lists all test VMs and destroys them
+- Removes test disk images
+- Keeps last N test results, deletes rest
+
+*When to run:* Periodically to free disk space
+
+*Usage:*
+#+begin_src bash
+# Interactive cleanup
+./scripts/testing/cleanup-tests.sh
+
+# Keep last 3 test results
+./scripts/testing/cleanup-tests.sh --keep 3
+
+# Force without prompts
+./scripts/testing/cleanup-tests.sh --force
+#+end_src
+
+* Directory Structure
+
+#+begin_example
+archsetup/
+├── scripts/
+│   └── testing/
+│       ├── README.org                    # This file
+│       ├── setup-testing-env.sh          # Setup infrastructure
+│       ├── create-base-vm.sh             # Create base VM
+│       ├── run-test.sh                   # Run tests
+│       ├── debug-vm.sh                   # Interactive debugging
+│       ├── cleanup-tests.sh              # Clean up
+│       ├── finalize-base-vm.sh           # Finalize base (generated)
+│       ├── archinstall-config.json       # Archinstall config
+│       └── lib/
+│           ├── logging.sh                # Logging utilities
+│           └── vm-utils.sh               # VM management
+├── vm-images/                            # VM disk images (gitignored)
+│   ├── archsetup-base.qcow2             # Golden image
+│   ├── arch-latest.iso                   # Arch ISO
+│   └── archsetup-test-*.qcow2           # Test VMs
+├── test-results/                         # Test results (gitignored)
+│   ├── TIMESTAMP/
+│   │   ├── test.log
+│   │   ├── test-report.txt
+│   │   └── archsetup-*.log
+│   └── latest -> TIMESTAMP/              # Symlink to latest
+└── docs/
+    └── testing-strategy.org              # Complete strategy doc
+#+end_example
+
+* Configuration
+
+** VM Specifications
+
+All test VMs use:
+- *CPUs:* 2 vCPUs
+- *RAM:* 4GB (matches archsetup tmpfs build directory)
+- *Disk:* 50GB (thin provisioned qcow2)
+- *Network:* NAT via libvirt default network
+- *Boot:* UEFI (systemd-boot bootloader)
+- *Display:* Serial console + VNC
+
+Set environment variables to customize:
+#+begin_src bash
+VM_CPUS=4 VM_RAM=8192 ./scripts/testing/run-test.sh
+#+end_src
+
+** Base VM Specifications
+
+The base VM contains a minimal Arch installation:
+- Base system packages
+- Linux kernel and firmware
+- OpenSSH server (for automation)
+- dhcpcd (for networking)
+- git, vim, sudo (essentials)
+- Root password: "archsetup"
+
+This matches the documented prerequisites for archsetup.
+
+* Validation Checks
+
+Each test run performs these validation checks:
+
+** Critical (Must Pass)
+1. archsetup exits with code 0
+2. User 'cjennings' was created
+3. Dotfiles are stowed (symlinks exist)
+4. yay (AUR helper) is installed
+5. DWM is built and installed
+
+** Additional (Future)
+- All expected packages installed
+- X11 can start
+- systemd services enabled
+- Firewall configured
+
+* Troubleshooting
+
+** VM fails to start
+
+Check if libvirtd is running:
+#+begin_src bash
+sudo systemctl status libvirtd
+sudo systemctl start libvirtd
+#+end_src
+
+** Cannot get VM IP address
+
+The VM may not have booted completely or networking failed:
+#+begin_src bash
+# Check VM status
+virsh domstate VM_NAME
+
+# Connect to console to debug
+virsh console VM_NAME
+
+# Check if dhcpcd is running in VM
+systemctl status dhcpcd
+#+end_src
+
+** SSH connection refused
+
+Wait longer for VM to boot, or check if sshd is enabled:
+#+begin_src bash
+virsh console VM_NAME
+# Inside VM:
+systemctl status sshd
+systemctl start sshd
+#+end_src
+
+** KVM not available
+
+Check if virtualization is enabled in BIOS and KVM modules loaded:
+#+begin_src bash
+# Check for /dev/kvm
+ls -l /dev/kvm
+
+# Load KVM module (Intel)
+sudo modprobe kvm-intel
+
+# Or for AMD
+sudo modprobe kvm-amd
+
+# Verify
+lsmod | grep kvm
+#+end_src
+
+** Disk space issues
+
+Clean up old tests:
+#+begin_src bash
+./scripts/testing/cleanup-tests.sh --force
+#+end_src
+
+Check disk usage:
+#+begin_src bash
+du -sh vm-images/ test-results/
+#+end_src
+
+** Base VM Installation Issues
+
+*** Firewall blocking VM network
+
+If the VM cannot reach the internet (100% packet loss when pinging), check the host firewall:
+
+#+begin_src bash
+# Check UFW status on host
+sudo ufw status
+
+# Check libvirt NAT rules on host
+sudo iptables -t nat -L -n -v | grep -i libvirt
+#+end_src
+
+*Solution:* Temporarily disable UFW during base VM creation:
+#+begin_src bash
+sudo ufw disable
+# Create base VM
+sudo ufw enable
+#+end_src
+
+Or add libvirt rules to UFW:
+#+begin_src bash
+sudo ufw allow in on virbr0
+sudo ufw allow out on virbr0
+#+end_src
+
+*** VM network not working after boot
+
+If =dhcpcd= isn't running or network isn't configured:
+
+#+begin_src bash
+# In the VM - restart network from scratch
+killall dhcpcd
+
+# Bring interface down and up (replace enp1s0 with your interface)
+ip link set enp1s0 down
+ip link set enp1s0 up
+
+# Start dhcpcd
+dhcpcd enp1s0
+
+# Wait and verify
+sleep 3
+ip addr show enp1s0
+ip route
+
+# Test connectivity
+ping -c 3 192.168.122.1  # Gateway
+ping -c 3 8.8.8.8        # Google DNS
+#+end_src
+
+*** DNS not working (127.0.0.53 in resolv.conf)
+
+The Live ISO uses systemd-resolved stub resolver which may not work:
+
+#+begin_src bash
+# In the VM - set real DNS servers
+echo "nameserver 8.8.8.8" > /etc/resolv.conf
+echo "nameserver 1.1.1.1" >> /etc/resolv.conf
+
+# Test
+ping -c 3 archlinux.org
+#+end_src
+
+*** Cannot paste into virt-viewer terminal
+
+Clipboard integration doesn't work well with virt-viewer. Use HTTP server instead:
+
+#+begin_src bash
+# On host - serve the installation script
+cd vm-images
+python -m http.server 8000
+
+# In VM - download and run
+curl http://192.168.122.1:8000/auto-install.sh | bash
+
+# Or download first to review
+curl http://192.168.122.1:8000/auto-install.sh -o install.sh
+cat install.sh
+bash install.sh
+#+end_src
+
+*** Partitions "in use" error
+
+If re-running installation after a failed attempt:
+
+#+begin_src bash
+# In the VM - unmount and wipe partitions
+mount | grep vda
+umount /mnt/boot 2>/dev/null
+umount /mnt 2>/dev/null
+umount -l /mnt  # Lazy unmount if still busy
+
+# Wipe partition table completely
+wipefs -a /dev/vda
+
+# Run install script again
+bash install.sh
+#+end_src
+
+*** Alternative: Use archinstall
+
+Instead of the auto-install.sh script, you can use Arch's built-in installer:
+
+#+begin_src bash
+# In the VM
+archinstall
+#+end_src
+
+*Recommended settings:*
+- Disk: =/dev/vda=
+- Filesystem: =ext4=
+- Bootloader: =systemd-boot=
+- Hostname: =archsetup-test=
+- Root password: =archsetup=
+- Profile: =minimal=
+- Additional packages: =openssh dhcpcd git vim sudo=
+- Network: =NetworkManager= or =systemd-networkd=
+
+*After installation, before rebooting:*
+#+begin_src bash
+# Chroot into new system
+arch-chroot /mnt
+
+# Enable services
+systemctl enable sshd
+systemctl enable dhcpcd  # or NetworkManager
+
+# Allow root SSH login
+sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config
+sed -i 's/#PasswordAuthentication yes/PasswordAuthentication yes/' /etc/ssh/sshd_config
+
+# Exit and poweroff
+exit
+poweroff
+#+end_src
+
+* Future Enhancements
+
+** Planned Improvements
+- [ ] Fully automated base VM creation (expect-based console automation)
+- [ ] Parallel test execution (multiple VMs)
+- [ ] Screenshot capture of X11 desktop
+- [ ] CI/CD integration (GitHub Actions / GitLab CI)
+- [ ] Performance benchmarking over time
+- [ ] Cloud-init based base image (faster provisioning)
+
+** Test Scenarios
+- [ ] Idempotency test (run archsetup twice)
+- [ ] Network failure recovery
+- [ ] Offline installation (local package cache)
+- [ ] Different hardware profiles (CPU/RAM variations)
+
+* References
+
+- [[file:../../docs/testing-strategy.org][Testing Strategy Document]]
+- [[https://wiki.archlinux.org/title/Libvirt][Arch Wiki: libvirt]]
+- [[https://wiki.archlinux.org/title/QEMU][Arch Wiki: QEMU]]
+- [[file:../../archsetup][Main archsetup script]]
+- [[file:../../TODO.org][Project TODO]]