diff options
| author | Craig Jennings <c@cjennings.net> | 2026-02-12 16:10:05 -0600 |
|---|---|---|
| committer | Craig Jennings <c@cjennings.net> | 2026-02-12 16:10:05 -0600 |
| commit | f429b540fe14c88a07ad5685919d8e18d6ec0352 (patch) | |
| tree | 196b6f6bf6ba0849637b61f6541f6926be53fd2c /docs | |
| parent | 16df4c9b4d6f29111dc077618beaed75173552f8 (diff) | |
| download | archangel-f429b540fe14c88a07ad5685919d8e18d6ec0352.tar.gz archangel-f429b540fe14c88a07ad5685919d8e18d6ec0352.zip | |
session: rebuild ISO for linux-lts 6.12.70, fix install test bugs
Fixed archzfs GPG key hang and pgrep false positive in test suite.
Added local distribution targets to build-release. All tests passing.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/notes.org | 157 | ||||
| -rw-r--r-- | docs/previous-session-history.org | 119 |
2 files changed, 157 insertions, 119 deletions
diff --git a/docs/notes.org b/docs/notes.org index f52ab20..ec6e861 100644 --- a/docs/notes.org +++ b/docs/notes.org @@ -287,7 +287,7 @@ Implementation is ready - just need Craig's preference. ** Current Reminders -(None currently - will be added as needed) +- [2026-02-12] Verify TrueNAS ISO hash matches local: =d17351445e4110ed2cf7190c25dc5fa91ec7325bb34644bbca1515fcd876d251=. TrueNAS was unreachable at end of session. Push hash file to truenas isos directory after verifying. ** Instructions for This Section @@ -327,6 +327,43 @@ Each entry should use this format: ** Session Entries +*** 2026-02-12 Thu @ 08:23-16:08 -0600 + +*Status:* COMPLETE + +*What We Completed:* +- Rebuilt archangel ISO for linux-lts 6.12.70-1 kernel +- ISO: archangel-vmlinuz-6.12.70-lts-2026-02-12-x86_64.iso (2.3G) +- All tests passed: sanity (26/26), single-disk, mirror, raidz1 +- Fixed archzfs GPG key prompt hanging unattended installs (SigLevel → Never) +- Fixed pgrep false positive in full-test.sh (avahi matched hostname pattern) +- Bumped INSTALL_TIMEOUT from 900s to 1800s for DKMS builds +- Added local distribution to build-release (~/downloads/isos + archsetup inbox notification) +- Distributed ISO to ~/downloads/isos and truenas.local:/mnt/vault/isos +- Audited codebase for open-source readiness, added todo.org task with full checklist +- Dropped SSH access info and test VM rebuild notice in archsetup inbox + +*Key Decisions:* +- archzfs SigLevel changed to Never (HTTPS provides transport security; GPG key management kept breaking unattended installs) +- USB drives removed as distribution target +- build-release now handles ~/downloads/isos and archsetup inbox automatically + +*Bugs Found and Fixed:* +1. archzfs GPG key prompt: pacstrap -K creates empty keyring, pacman-key -r silently fails, pacman prompts interactively → changed SigLevel to Never in custom/archangel (2 locations) +2. Test pgrep false positive: pgrep -f 'archangel' matched avahi-daemon's "running [archangel.local]" → changed to pgrep -f '/usr/local/bin/archangel' +3. Install timeout: 15 min too short for DKMS compile in VM → bumped to 30 min + +*Files Modified:* +- [[file:../custom/archangel][custom/archangel]] — SigLevel fix (install_base + configure_system) +- [[file:../scripts/full-test.sh][scripts/full-test.sh]] — pgrep fix, timeout bump +- [[file:../scripts/build-release][scripts/build-release]] — local distribution + archsetup inbox +- [[file:../todo.org][todo.org]] — open-sourcing prep task + +*Next Steps:* +- Verify TrueNAS ISO hash (was unreachable at session end) +- Fix TrueNAS connectivity issues +- Continue with open-sourcing prep or other todo.org items + *** 2026-02-07 Sat @ 21:36 -0600 *Status:* COMPLETE @@ -520,122 +557,4 @@ Each entry should use this format: - Rebuild initramfs from live ISO - Verify fixes active after reboot -*** 2026-01-22 Thu @ 15:02 -0600 - -*Status:* COMPLETE - -*What We Completed:* -- Diagnosed and fixed ratio (Framework Desktop, AMD Strix Halo) boot failures -- Root cause: missing linux-firmware 20260110 caused amdgpu to freeze at boot -- Installed linux-firmware 20260110-1, fixed ZFS mountpoints, fixed hostid mismatch -- Configured kernel 6.15.2 as default (pinned), created clean GRUB menu -- Created retrospective workflow for continuous improvement -- Added PRINCIPLES.org with behavioral lessons learned -- Documented full troubleshooting session - -*Key Decisions:* -- linux-firmware version is critical for AMD Strix Halo (20260110+ required) -- ZFS rollback with separate /boot partition is dangerous - recommend ZFSBootMenu -- Established retrospective workflow for major problem-solving sessions -- Behavioral lessons go in PRINCIPLES.org, technical facts in session docs - -*Files Modified:* -- [[file:../custom/archangel][custom/install-archzfs]] - Fixed mkinitcpio configuration -- [[file:../todo.org][todo.org]] - Added ZFS rollback + /boot issue, ZFSBootMenu task -- [[file:PRINCIPLES.org][docs/PRINCIPLES.org]] - New file with behavioral lessons -- [[file:protocols.org][docs/protocols.org]] - Added PRINCIPLES.org to session startup -- [[file:retrospectives/2026-01-22-ratio-boot-fix.org][docs/retrospectives/]] - Retrospective for this session -- [[file:2026-01-22-ratio-boot-fix-session.org][docs/2026-01-22-ratio-boot-fix-session.org]] - Full technical session doc - -*Commits Made:* -- c46191c: Fix ratio boot issues: firmware, mkinitcpio, and document ZFS rollback dangers -- 9100517: Update ratio session doc: kernel 6.15.2 now default with clean GRUB menu -- e5aedfa: Add retrospective workflow and PRINCIPLES.org for continuous improvement - -*Next Steps:* -- Implement ZFSBootMenu on ratio to solve /boot rollback issue -- Consider adding ZFSBootMenu to install-archzfs as alternative to GRUB - -*** 2026-01-18 Sat @ 16:30 -0600 - -*Status:* COMPLETE - -*What We Completed:* -- Completed RESCUE-GUIDE.txt with all 8 sections fully documented -- Added final round of utility packages to ISO: - - Disk tools: ncdu, tree - - Hardware diagnostics: iotop - - Network: speedtest-cli, mosh, aria2, tmate, sshuttle - - Security: pass (password manager) -- Removed AUR-only packages that broke build: safecopy, ms-sys, dislocker, nwipe -- Successfully rebuilt ISO (5.1GB) -- Copied ISO to truenas.local:/mnt/vault/isos and ~/downloads/isos -- Wrote ISO to USB drives (/dev/sda 1TB, /dev/sdb 240GB) -- Ran all tests: - - zfs-snap-prune unit tests: 22/22 PASSED - - VM install test (single-disk): PASSED - - VM install test (mirror): PASSED - - VM install test (raidz1): PASSED -- Marked "Add common recovery tools" TODO as DONE - -*Commits Made:* -- 36aa130: Add utility tools and rescue guide documentation -- 6f4fd68: Remove AUR-only packages from ISO build - -*Files Modified:* -- [[file:../build.sh][build.sh]] - Added utility packages, removed AUR-only packages -- [[file:../custom/RESCUE-GUIDE.txt][custom/RESCUE-GUIDE.txt]] - Completed all 8 sections -- [[file:../TODO.org][TODO.org]] - Marked recovery tools task as DONE -- [[file:session-context.org][docs/session-context.org]] - Updated session state - -*Key Technical Notes:* -- AUR packages cannot be included in mkarchiso builds without custom AUR handling -- Documented AUR tools (safecopy, ms-sys, dislocker, nwipe) in RESCUE-GUIDE with install instructions -- ISO now doubles as a comprehensive rescue/recovery disk - -*Next Steps:* -- Test booting from physical USB drive on real hardware -- Consider CI/CD pipeline for automated ISO builds -- Consider adding ISO to GRUB boot menu for on-disk recovery - -*** 2026-01-17 Sat @ 17:10 -0600 - -*Status:* IN PROGRESS - -*What We Completed:* -- Fixed ZFS kernel module mismatch by switching to linux-lts + zfs-dkms -- Fixed bootloader to use linux-lts kernel (was defaulting to regular linux) -- Fixed broadcom-wl dependency (switched to broadcom-wl-dkms) -- Updated mkinitcpio preset for linux-lts with archiso config -- Fixed install-archzfs bugs: - - =[[ ]] && error= pattern causing early exit with =set -e= (changed to if/then) - - 50G reservation on 50G disk (now dynamic: 20% of pool, capped 5-20G) - - sanoid not in official repos (moved to archsetup) - - grub-mkfont needs freetype2 package (added to pacstrap) -- Removed sanoid/syncoid from install-archzfs (archsetup will handle) -- Created inbox item for archsetup with full sanoid/syncoid config -- ISO now 4.8G (was 5.4G) - only linux-lts kernel - -*Key Technical Insights:* -- =broadcom-wl= depends on =linux= kernel specifically - use =broadcom-wl-dkms= instead -- archiso releng profile has linux.preset in airootfs that needs renaming to linux-lts.preset -- With =set -e=, =[[ test ]] && command= returns exit code 1 if test is false, causing script exit -- =grub-mkfont= requires =freetype2= package (not installed by default with grub) - -*Files Modified:* -- [[file:../build.sh][build.sh]] - major updates for linux-lts, bootloader configs, mkinitcpio preset -- [[file:../custom/archangel][custom/install-archzfs]] - multiple bug fixes, removed sanoid/syncoid -- [[file:~/code/archsetup/inbox/zfs-sanoid-detection.txt][archsetup inbox]] - sanoid/syncoid config for archsetup to implement - -*Current State:* -- ISO builds successfully with linux-lts + zfs-dkms -- ZFS module loads correctly in live environment -- install-archzfs runs through most steps -- Last error: grub-mkfont missing freetype2 (now fixed, needs rebuild/test) - -*Next Steps:* -- Rebuild ISO with freetype2 fix -- Complete full install-archzfs test in VM -- Test booting the installed system -- Git commit all changes diff --git a/docs/previous-session-history.org b/docs/previous-session-history.org index 8a87058..2c9d377 100644 --- a/docs/previous-session-history.org +++ b/docs/previous-session-history.org @@ -10,6 +10,125 @@ Sessions are listed in reverse chronological order (most recent first). * Archived Sessions +*** 2026-01-22 Thu @ 15:02 -0600 + +*Status:* COMPLETE + +*What We Completed:* +- Diagnosed and fixed ratio (Framework Desktop, AMD Strix Halo) boot failures +- Root cause: missing linux-firmware 20260110 caused amdgpu to freeze at boot +- Installed linux-firmware 20260110-1, fixed ZFS mountpoints, fixed hostid mismatch +- Configured kernel 6.15.2 as default (pinned), created clean GRUB menu +- Created retrospective workflow for continuous improvement +- Added PRINCIPLES.org with behavioral lessons learned +- Documented full troubleshooting session + +*Key Decisions:* +- linux-firmware version is critical for AMD Strix Halo (20260110+ required) +- ZFS rollback with separate /boot partition is dangerous - recommend ZFSBootMenu +- Established retrospective workflow for major problem-solving sessions +- Behavioral lessons go in PRINCIPLES.org, technical facts in session docs + +*Files Modified:* +- [[file:../custom/archangel][custom/install-archzfs]] - Fixed mkinitcpio configuration +- [[file:../todo.org][todo.org]] - Added ZFS rollback + /boot issue, ZFSBootMenu task +- [[file:PRINCIPLES.org][docs/PRINCIPLES.org]] - New file with behavioral lessons +- [[file:protocols.org][docs/protocols.org]] - Added PRINCIPLES.org to session startup +- [[file:retrospectives/2026-01-22-ratio-boot-fix.org][docs/retrospectives/]] - Retrospective for this session +- [[file:2026-01-22-ratio-boot-fix-session.org][docs/2026-01-22-ratio-boot-fix-session.org]] - Full technical session doc + +*Commits Made:* +- c46191c: Fix ratio boot issues: firmware, mkinitcpio, and document ZFS rollback dangers +- 9100517: Update ratio session doc: kernel 6.15.2 now default with clean GRUB menu +- e5aedfa: Add retrospective workflow and PRINCIPLES.org for continuous improvement + +*Next Steps:* +- Implement ZFSBootMenu on ratio to solve /boot rollback issue +- Consider adding ZFSBootMenu to install-archzfs as alternative to GRUB + +*** 2026-01-18 Sat @ 16:30 -0600 + +*Status:* COMPLETE + +*What We Completed:* +- Completed RESCUE-GUIDE.txt with all 8 sections fully documented +- Added final round of utility packages to ISO: + - Disk tools: ncdu, tree + - Hardware diagnostics: iotop + - Network: speedtest-cli, mosh, aria2, tmate, sshuttle + - Security: pass (password manager) +- Removed AUR-only packages that broke build: safecopy, ms-sys, dislocker, nwipe +- Successfully rebuilt ISO (5.1GB) +- Copied ISO to truenas.local:/mnt/vault/isos and ~/downloads/isos +- Wrote ISO to USB drives (/dev/sda 1TB, /dev/sdb 240GB) +- Ran all tests: + - zfs-snap-prune unit tests: 22/22 PASSED + - VM install test (single-disk): PASSED + - VM install test (mirror): PASSED + - VM install test (raidz1): PASSED +- Marked "Add common recovery tools" TODO as DONE + +*Commits Made:* +- 36aa130: Add utility tools and rescue guide documentation +- 6f4fd68: Remove AUR-only packages from ISO build + +*Files Modified:* +- [[file:../build.sh][build.sh]] - Added utility packages, removed AUR-only packages +- [[file:../custom/RESCUE-GUIDE.txt][custom/RESCUE-GUIDE.txt]] - Completed all 8 sections +- [[file:../TODO.org][TODO.org]] - Marked recovery tools task as DONE +- [[file:session-context.org][docs/session-context.org]] - Updated session state + +*Key Technical Notes:* +- AUR packages cannot be included in mkarchiso builds without custom AUR handling +- Documented AUR tools (safecopy, ms-sys, dislocker, nwipe) in RESCUE-GUIDE with install instructions +- ISO now doubles as a comprehensive rescue/recovery disk + +*Next Steps:* +- Test booting from physical USB drive on real hardware +- Consider CI/CD pipeline for automated ISO builds +- Consider adding ISO to GRUB boot menu for on-disk recovery + +*** 2026-01-17 Sat @ 17:10 -0600 + +*Status:* IN PROGRESS + +*What We Completed:* +- Fixed ZFS kernel module mismatch by switching to linux-lts + zfs-dkms +- Fixed bootloader to use linux-lts kernel (was defaulting to regular linux) +- Fixed broadcom-wl dependency (switched to broadcom-wl-dkms) +- Updated mkinitcpio preset for linux-lts with archiso config +- Fixed install-archzfs bugs: + - =[[ ]] && error= pattern causing early exit with =set -e= (changed to if/then) + - 50G reservation on 50G disk (now dynamic: 20% of pool, capped 5-20G) + - sanoid not in official repos (moved to archsetup) + - grub-mkfont needs freetype2 package (added to pacstrap) +- Removed sanoid/syncoid from install-archzfs (archsetup will handle) +- Created inbox item for archsetup with full sanoid/syncoid config +- ISO now 4.8G (was 5.4G) - only linux-lts kernel + +*Key Technical Insights:* +- =broadcom-wl= depends on =linux= kernel specifically - use =broadcom-wl-dkms= instead +- archiso releng profile has linux.preset in airootfs that needs renaming to linux-lts.preset +- With =set -e=, =[[ test ]] && command= returns exit code 1 if test is false, causing script exit +- =grub-mkfont= requires =freetype2= package (not installed by default with grub) + +*Files Modified:* +- [[file:../build.sh][build.sh]] - major updates for linux-lts, bootloader configs, mkinitcpio preset +- [[file:../custom/archangel][custom/install-archzfs]] - multiple bug fixes, removed sanoid/syncoid +- [[file:~/code/archsetup/inbox/zfs-sanoid-detection.txt][archsetup inbox]] - sanoid/syncoid config for archsetup to implement + +*Current State:* +- ISO builds successfully with linux-lts + zfs-dkms +- ZFS module loads correctly in live environment +- install-archzfs runs through most steps +- Last error: grub-mkfont missing freetype2 (now fixed, needs rebuild/test) + +*Next Steps:* +- Rebuild ISO with freetype2 fix +- Complete full install-archzfs test in VM +- Test booting the installed system +- Git commit all changes + *** 2026-01-17 Sat @ 13:16 -0600 *Status:* COMPLETE (continued above) |
