From ee1b5552552b231cec88114a1ceaa6cf344d6551 Mon Sep 17 00:00:00 2001 From: Craig Jennings Date: Tue, 7 Jul 2026 22:00:49 -0500 Subject: feat: Phase 4 threshold keys (logs, memory, power) --- configs/maintenance-thresholds.toml | 12 ++++++++++++ todo.org | 4 ++-- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/configs/maintenance-thresholds.toml b/configs/maintenance-thresholds.toml index 898f47a..b754814 100644 --- a/configs/maintenance-thresholds.toml +++ b/configs/maintenance-thresholds.toml @@ -63,10 +63,22 @@ expected_timers_extra = [] [logs] app_log_warn_days = 7 # log-cleanup cron should hold ~7 days journal_digest_top = 10 # groups shown in the journal digest +journal_disk_warn_gb = 2 # journald footprint before vacuum suggested +coredump_window_days = 14 # dumps older than this age out of the count + +[memory] +used_warn_pct = 90 # 1 - available/total (reclaimable-aware) +used_crit_pct = 97 +oom_window_days = 7 # kernel-journal OOM lookback [power] battery_low_pct = 15 # feeds the diagnostic state (glyph red) charge_limit_pct = 80 # the SET 80% longevity cap on velox +cpu_temp_warn_c = 90 # k10temp Tctl / coretemp package +gpu_temp_warn_c = 95 # amdgpu edge +battery_health_warn_pct = 80 # charge_full vs design floor +unclean_warn_pct = 25 # unclean-shutdown rate over recent boots +unclean_boot_sample = 10 # boots assessed for the rate [backups] rsyncshot_daily_crit_hours = 48 # daily backup older than this is a fire diff --git a/todo.org b/todo.org index 52da13e..c094911 100644 --- a/todo.org +++ b/todo.org @@ -36,8 +36,8 @@ All Phase 2 collectors TDD'd against the fake-tool harness (argv-matched cases a *** 2026-07-07 Tue @ 21:30:59 -0500 Built Phase 3 — packages, security, systemd collectors (dotfiles 3eaab6e) All Phase 3 collectors TDD'd: orphans name+size with [curation.kept_orphans], pacnew safe-delete vs needs-merge, keyring freshness, reboot-required (uname -r vs modules dir), -Qkk on =maint scan --slow= with a noise split (unverifiable-as-user reads, mtime-only, [curation.qkk_known] paths tallied but never counted — an unfiltered live run sat permanently red at 68 "altered" files; classified, ratio shows 10 real findings); =maint scan --net= writes checkupdates/yay/arch-audit/fwupd caches (yay rc-1-silent = AUR-clean zero; failures never clobber good cache; malformed payloads degrade to unprobed), readers grade with age + stale bump; is-system-running with state-aware cause, failed-unit roster since/exit/journalctl hint, maintenance-timers meta-metric (capability-derived expected set, @-prefix instance match), taint letters decoded. Prereq done: arch-audit added to archsetup deps + installed both hosts. Review subagent (live-verified) found 2 Important (yay AUR-clean rc quirk, cache-shape envelope kill) + 2 minor — all fixed red-first; archsetup now enables btrfs-scrub@-.timer on btrfs installs (was ratio-only out-of-band, the meta-metric expects it). maint suites 118→169; full make test green. Live on ratio: 23-metric envelope, worst=warn from real findings (10.75GB cache, 8 orphans, mirrorlist.pacnew, 713 repo updates, 32 CVEs, 1 firmware). -*** TODO Phase 4 — collectors: logs, memory, power :solo: -Journal digest (identifier groups, noise layers), coredumps by binary, kernel/hw events; memory + top-5, OOM, swap/zram, temps, throttle, EPP read, battery health + charge threshold read, unclean-shutdown rate. Verify: unit green + live read-only on ratio. +*** 2026-07-07 Tue @ 21:59:27 -0500 Built Phase 4 — logs + mem·pwr collectors (dotfiles 40c448b) +All Phase 4 collectors TDD'd: journal error digest (identifier groups w/ first/last seen + journalctl hint, known-noise curation tallied but never counted, byte-array MESSAGE decoded), coredumps by binary over a TOML window, kernel/hw event classes (MCE/EDAC/IO/thermal/GPU — generic driver noise excluded), journald disk usage, app-log staleness mirroring the log-cleanup cron's filename-date rules; memory pressure + top-5 RSS, OOM kills (kill-verdict lines only — one event spans several matching lines), swap/zram, temps via sensors -j (k10temp Tctl / coretemp package / amdgpu edge), Intel thermal-throttle count, EPP read, battery capacity/health/charge-cap, unclean-shutdown rate from wtmp (last -x pairing). Throttle/EPP/battery self-gate on sysfs and drop out of the envelope. Review subagent (live-verified) found 1 Blocking — windowed-empty coredumpctl exits 1 (my earlier --since-now pin was wrong), which left the metric permanently unprobed on healthy hosts — plus OOM ~3× overcount and a malformed-curation-entry envelope crash; all fixed red-first, re-review Approve. maint suites 169→227; full make test green. Live read-only both hosts: ratio 34 metrics (telega-server crashing 3× in window — real find), velox 38 (genuine 75% battery-health warn; charge_control_end_threshold ABSENT on velox's cros_ec hardware → Phase 9's SET 80% lever must gate on the file). TOML +journal_disk_warn_gb/coredump_window_days, new [memory], [power] temp/health/unclean keys; installed both hosts. *** TODO Phase 5 — collectors: network posture, services :solo: DNS/NM, firewall + public-bind exposure, listeners digest + expected merge, tailscale, fail2ban, NTP offset; rsyncshot freshness, docker df + containers + expected layer, libvirt, cron drift. Curation read side complete. Verify: unit green + live read-only on ratio; velox presence sweep for the tool set (tailscale ssh). -- cgit v1.2.3