diff options
Diffstat (limited to 'docs/workflows/strix-soak-watch.org')
| -rw-r--r-- | docs/workflows/strix-soak-watch.org | 144 |
1 files changed, 144 insertions, 0 deletions
diff --git a/docs/workflows/strix-soak-watch.org b/docs/workflows/strix-soak-watch.org new file mode 100644 index 0000000..b86921d --- /dev/null +++ b/docs/workflows/strix-soak-watch.org @@ -0,0 +1,144 @@ +#+TITLE: Strix Halo SMU Deadlock — Kernel Update Watch +#+AUTHOR: Craig Jennings +#+DATE: 2026-05-10 + +* Purpose + +Companion to =system-health-check.org=. Triggered when linux / linux-lts / linux-firmware land in =checkupdates= on ratio, this addendum checks whether the upstream side of the Strix Halo SMU deadlock has moved — so we know when to retire the locally-patched =linux-lts-strix= kernel and switch back to stock. + +This is a one-time hardware/kernel issue. Delete this file when the strix kernel is retired. + +* Current State (as of 2026-05-10) + +- *Boot kernel:* =linux-lts-strix 6.18.25-1= (custom AUR-style local build), GRUB default pinned +- *SMC firmware:* =100.6.0= (from =/sys/kernel/debug/dri/*/amdgpu_firmware_info= SMC line) +- *Cmdline workaround:* =amdgpu.no_vpe_idle_pg=1= active +- *Soak verdict:* HOLDING since 2026-04-30 — zero SMU/dcn35/VPE freeze signature hits (see the 2026-05-04 soak-check verdict, archived outside this repo) + +* Background (one-paragraph summary so future sessions don't have to reconstruct it) + +Strix Halo (gfx_v11_5_0, smu_v14_0_2 APU) has an SMU firmware deadlock triggered during VPE (Video Processing Engine) idle power-gating: hard freeze, no journal output, requires power-cycle. Crupi posted a workaround patch on the Framework community forum that disables VPE idle power-gating via =amdgpu.no_vpe_idle_pg=. Real fix has to come from AMD as a PMFW/SMU firmware update (delivered via =linux-firmware=). Until then, ratio runs a custom kernel build that carries the patch + two supporting amdgpu allocation-in-critical-path patches. Forum thread: https://community.frame.work/t/smu-deadlock-system-freeze-on-fedora-43/81795 + +* Local Patch Set + +Located at =~/build/linux-lts-patched/linux-lts/=: + +| File | Strix-relevant? | +|---------------------------------------------------------------------------------+-----------------| +| =0001-ZEN-Add-sysctl-and-CONFIG-to-disallow-unprivileged-CLONE_NEWUSER.patch= | No (security) | +| =0002-drm-amdgpu-avoid-memory-allocation-in-the-critical-code-path-v3.patch= | Yes | +| =0003-drm-amdgpu-use-GFP_ATOMIC-instead-of-NOWAIT-in-the-critical-path.patch= | Yes | +| =0004-drm-amdgpu-add-no_vpe_idle_pg-strix-halo-smu-deadlock-workaround.patch= | Yes (Crupi's) | + +* When This Watch Fires + +Phase 3 of =system-health-check.org= invokes this addendum when *any* of the following appear in =checkupdates=: + +- =linux= (any version bump) +- =linux-lts= (any version bump) +- =linux-firmware= (any version bump — this is the most likely vector for the real fix) +- =mesa= (major version only, e.g. 26.x.y → 27.x.y — display-stack timing can shift the deadlock window) + +* What To Check On Each Fire + +** 1. Stable kernel changelogs (per bumped version) + +Fetch the per-version changelog and search for the keyword set below. The 2026-05-10 baseline pass found *no matches* in 6.18.27, 6.18.28, 7.0.4, 7.0.5 — so anything that does match is news. + +#+begin_src +https://cdn.kernel.org/pub/linux/kernel/v6.x/ChangeLog-<NEW-LTS-VERSION> +https://cdn.kernel.org/pub/linux/kernel/v7.x/ChangeLog-<NEW-MAINLINE-VERSION> +#+end_src + +Keyword set (case-insensitive): +- =no_vpe_idle_pg=, =vpe_idle=, =VPE idle=, =VPE power gating= +- =Strix Halo=, =STRIX_HALO=, =strix_halo= +- =gfx_v11_5= (the GFX block id for Strix Halo) +- =smu_v14= (the SMU IP block driver, =drivers/gpu/drm/amd/pm/swsmu/smu14=) +- =dcn351= (Strix Halo display controller; =dcn35= is Phoenix-class — adjacent but not the same) +- =drm/amdgpu/vpe=, =drm/amdgpu/swsmu/smu14=, =amdgpu/pm/swsmu= +- =SMU deadlock=, =critical path= (combined with amdgpu/drm) + +The two adjacent allocation-fix patches (0002, 0003) may land separately from Crupi's patch. They have distinctive subject lines: =avoid memory allocation in the critical code path= and =use GFP_ATOMIC instead of NOWAIT in the critical path=. Search for those too. + +** 2. linux-firmware delta (if linux-firmware is the bumped package) + +The canonical fix arrives as a PMFW/SMU firmware update, NOT a kernel patch. + +#+begin_src bash +# Pre-bump SMC version (from running system, BEFORE topgrade) +sudo cat /sys/kernel/debug/dri/*/amdgpu_firmware_info | grep "SMC feature" + +# After topgrade installs the new linux-firmware, examine the delta. +# Watch for these files in the upgraded package: +# amdgpu/smu_14_0_2.bin — Strix Halo SMU firmware +# amdgpu/psp_14_0_*.bin — PSP/secure-os blobs +# amdgpu/strix* — anything Strix-tagged +pacman -Ql linux-firmware | grep -iE "smu_14_0_2|psp_14_0|strix" + +# Reboot is required to load the new firmware. Re-check SMC version post-reboot. +#+end_src + +A bump from =100.6.0= to anything higher is the canonical Strix-fix signal — that means AMD shipped new SMU firmware. Once that lands, the workaround kernel is no longer needed (in theory; see "Outcomes" below for the validation path). + +** 3. Crupi's forum thread + +#+begin_src +https://community.frame.work/t/smu-deadlock-system-freeze-on-fedora-43/81795 +#+end_src + +Skim recent posts for: +- "Merged upstream" / "Landed in 6.x.y" announcements +- AMD acknowledgements or workaround retirement notes +- Other affected users reporting fix verification on stock kernels + +** 4. amd-pm subsystem mailing list (optional, only if 1-3 are inconclusive) + +#+begin_src +https://lore.kernel.org/amd-gfx/?q=strix+halo+smu+deadlock +https://lore.kernel.org/amd-gfx/?q=no_vpe_idle_pg +#+end_src + +* Outcomes & Actions + +** Nothing Strix-adjacent in the bump (default — also the 2026-05-10 outcome) + +- Continue running =linux-lts-strix 6.18.25-1= +- Do *not* rebuild the strix kernel against the newer base purely for the version bump. Same four patches against a newer base = same workaround behavior + new risk of unrelated base-version regressions. Only rebuild when there's a security-pressing reason. +- Topgrade is safe; the new stock =linux-lts= installs to disk but isn't booted. + +** linux-firmware bumps SMC past =100.6.0= + +This is the most likely real-fix path. + +1. Note the new SMC version (capture pre and post). +2. Schedule a controlled test boot: + - Backup current =/etc/default/grub= GRUB_DEFAULT and =linux-lts-strix= package + - Drop =amdgpu.no_vpe_idle_pg=1= from the cmdline on a test boot of stock =linux-lts= + - Re-run the soak-check script for 1 week (same harness used for the original verdict) +3. If zero signature hits over the soak window → retire =linux-lts-strix=, switch GRUB default back to stock =linux-lts=, remove the custom build dir. +4. If any freeze recurs → restore strix-lts as default and re-engage the workaround. Note the recurrence in this file and on the forum thread. + +** Crupi's patch (or equivalent) lands in 6.18.x stable + +1. Check the merged form: is the =no_vpe_idle_pg= behavior the default-on, or still gated behind a cmdline? +2. If default-on: rebuild =linux-lts-strix= against the new base with patch 0004 dropped (keep 0002, 0003, 0001 as long as they remain non-redundant). +3. If still cmdline-gated: keep the strix kernel as-is until linux-firmware fix arrives — the in-tree path is equivalent to the local one. + +** Patches 0002/0003 land upstream but Crupi's doesn't + +Drop the matching patches from the PKGBUILD on next strix rebuild; keep 0004 until the SMU firmware fix or kernel merge. + +* Reference Artifacts + +- Soak verdict (archived): the 2026-05-04 soak-check verdict, archived outside this repo +- Working records for the VPE-patch deployment (2026-04-29), the soak fix (2026-05-03), and the verdict-holds check (2026-05-05) are archived outside this repo. +- Forum thread: https://community.frame.work/t/smu-deadlock-system-freeze-on-fedora-43/81795 +- Build dir: =~/build/linux-lts-patched/linux-lts/= +- Inventory: the =ratio - Desktop Workstation= node in =~/org/roam/hardware/= (resolve by =#+HOSTNAME: ratio=) + +* Out of Scope + +- Generalizing this to other AMD APUs. The deadlock signature is specific to Strix Halo (gfx_v11_5_0, smu_v14_0_2). Other Phoenix / Hawk Point / RDNA-N APUs are unaffected and shouldn't be checked here. +- Tracking AMD's internal bug tracker. Stick to public signals (kernel.org changelogs, lore.kernel.org, the Framework forum thread). +- This is not a soak-check runner. The soak-check harness (=at=-job + scripts) lives separately under =scripts/= or =inbox/=. This file watches for the *retirement signal* for that harness. |