diff options
| -rw-r--r-- | todo.org | 4 |
1 files changed, 3 insertions, 1 deletions
@@ -314,7 +314,9 @@ Recorded in [[file:docs/design/2026-07-13-runtime-portability-inventories.org][t *** TODO Local model floor evaluation The workflows assume long-context instruction-following (startup's multi-file read; the commits.md publish chain). Establish the minimum viable local tier (likely strong-70B+/MoE, 100k+ context), and what compensations a weaker model needs: shortened protocols, more checklist gates, more hook-level enforcement. Feeds the spec's "default local runtime/server" and "first supported local editing CLI" blocker decisions. -Environment inventory done 2026-07-13 (KB node "Local LLM inference inventory — daily drivers"): ratio is the inference box — Strix Halo iGPU + 125 GiB unified RAM, ollama 0.17.7 on demand (no systemd unit), llama3.1:8b + llama3.3 70B already in the 45 GB store. velox is out of scope (Iris Xe, 60 GiB, no ollama). So the floor evaluation can run today on ratio against the on-disk 70B: drive a scripted startup + a small publish-flow transcript through it and grade instruction-following. Remaining work is that evaluation, not procurement. +Environment inventory done 2026-07-13 (KB node "Local LLM inference inventory — daily drivers"): ratio is the inference box — Strix Halo iGPU + 125 GiB unified RAM. velox is out of scope (Iris Xe, 60 GiB, no ollama). + +Environment BUILT and verified, evening of 2026-07-13: ollama 0.31.2 as a systemd service, Vulkan/RADV backend forced via systemd override (ROCm loaded a 61 GB model at <25 MB/s — 40+ min, unusable; Vulkan loads it in 44 s), models swapped to the MoE shape this platform wants: gpt-oss:120b (44 s load, 37.9 tok/s, 100% GPU) and qwen3-coder:30b (5 s load, 76 tok/s). All four legacy dense models dropped; the orphaned ~/.ollama user store purged. Gotcha for the eval harness: pass num_ctx 8-32K — gpt-oss's native 128K default balloons the load. Remaining work is now purely the evaluation: drive a scripted startup read + a publish-flow transcript through gpt-oss:120b (and qwen3-coder as the fast comparator), grade instruction-following against the protocol stack, and answer the spec's default-local-runtime + first-supported-CLI blockers. ** TODO [#C] Docs-lifecycle convention — manual validation :test: The human-eyes half of the docs-lifecycle acceptance surface. The convention shipped IMPLEMENTED 2026-07-04 (spec [[id:80b0787b-4a60-4c82-8a16-b383d3e3c8f2][docs-lifecycle]]); these checks confirm the human-facing behavior. A failed check promotes to a bug. |