aboutsummaryrefslogtreecommitdiff
path: root/docs/design
diff options
context:
space:
mode:
Diffstat (limited to 'docs/design')
-rw-r--r--docs/design/2026-07-20-signal-pager-runbook.org113
-rw-r--r--docs/design/2026-08-19-agent-text-relay-fallback-decision.org155
2 files changed, 237 insertions, 31 deletions
diff --git a/docs/design/2026-07-20-signal-pager-runbook.org b/docs/design/2026-07-20-signal-pager-runbook.org
index f31ed18..393c1c6 100644
--- a/docs/design/2026-07-20-signal-pager-runbook.org
+++ b/docs/design/2026-07-20-signal-pager-runbook.org
@@ -7,26 +7,42 @@ phone, how his replies come back, how the account stays healthy, and the
signal-cli setup behind it. This is the Signal successor to the retired ntfy
runbook. Canonical home is rulesets because the pager is cross-machine tooling.
-* What the pager is
+* SUPERSEDED 2026-08-19 — the topology below is out of date
+
+velox's 2026-08-13 reinstall wiped its signal-cli registration, so everything
+this runbook says about velox holding the primary is now false. I am flagging
+it here rather than rewriting the whole document. Treat everything below as
+predating the 2026-08-19 change except where a dated note says otherwise: the
+send, reply-reading and setup sections all name velox specifically, and I have
+marked each one inline rather than asking you to remember this banner four
+screens later.
+
+Current, and stated as a derivation rather than a snapshot, since the snapshot
+form is exactly what rotted: *any* machine whose local signal-cli holds the
+identity sends directly, and a machine without it ssh-relays to the first
+listed host that does. Check with =signal-cli listAccounts=. As of 2026-08-19
+that is ratio only. See
+[[file:2026-08-19-agent-text-relay-fallback-decision.org]] for what broke, why
+re-registering velox was not the fix, and the measurements behind the current
+=agent-text= dispatch.
-One Signal identity, =+15045173983=, registered in *velox's* signal-cli
-(account file 465310, velox is the primary device). It is a dedicated pager
-number, not Craig's personal Signal. Pages go *from* that identity *to* Craig's
-own Signal account, which fires a normal mobile push on his phone.
+* What the pager is
-As of 2026-07-20 the identity spans two devices: velox (primary) and ratio
-(linked device "ratio-pager"). Any machine holding the account sends directly;
-a machine that doesn't relays to velox over the tailnet.
+One Signal identity, =+15045173983=. It is a dedicated pager number, not
+Craig's personal Signal. Pages go *from* that identity *to* Craig's own Signal
+account, which fires a normal mobile push on his phone.
Two constants the tooling depends on:
-- Pager account: =+15045173983= (primary on velox, linked on ratio).
+- Pager account: =+15045173983=.
- Recipient: Craig's Signal account UUID =b1b5601e-6126-47f8-afaa-0a59f5188fde=.
His phone *number* reads as unregistered in Signal's directory — always
target the UUID, never the number.
-velox is the laptop that travels with Craig, so the pager account rides with
-him; ratio holds it too, so a page still lands when velox is down.
+Which machines hold it is runtime state, not a constant — see the superseded
+note above. Historical record: as of 2026-07-20 the identity spanned velox
+(primary, account file 465310) and ratio (linked device "ratio-pager"). velox's
+half is gone.
* Choosing a channel
@@ -63,16 +79,26 @@ local signal-cli:
desktop fallback line and exits non-zero, so a caller can tell the page did not
land.
-The raw command it runs, for reference or a manual send from velox:
+*Superseded 2026-08-19.* The relay and failure bullets above (the last two)
+describe the single-host dispatch that was replaced. =agent-text= now walks an ordered list (=AGENT_TEXT_RELAYS=,
+ratio first), skips any candidate matching =uname -n=, and advances past a host
+that is reachable but does not hold the account. The failure line no longer
+blames velox by name and distinguishes a local send failure from relay
+exhaustion.
+
+The raw command it runs, for reference or a manual send from a machine that
+holds the account:
#+begin_src bash
signal-cli -a +15045173983 send -m "your message" b1b5601e-6126-47f8-afaa-0a59f5188fde
#+end_src
-From another machine, the same send relayed over the tailnet:
+From another machine, the same send relayed over the tailnet — *substitute a
+host that actually holds the account* (=signal-cli listAccounts=); as of
+2026-08-19 velox does not, so the example below is written against ratio:
#+begin_src bash
-ssh velox.tailf3bb8c.ts.net \
+ssh ratio.tailf3bb8c.ts.net \
"signal-cli -a +15045173983 send -m 'your message' b1b5601e-6126-47f8-afaa-0a59f5188fde"
#+end_src
@@ -87,11 +113,15 @@ queue until something receives it.
Drain the queue and read what is there:
+*Superseded 2026-08-19:* these named velox, which no longer holds the account.
+Run the drain on a machine that does — =signal-cli listAccounts= names it; as of
+2026-08-19 that is ratio.
+
#+begin_src bash
-# On velox:
+# On the machine holding the account:
signal-cli -a +15045173983 receive --timeout 10
-# From another machine:
-ssh velox.tailf3bb8c.ts.net "signal-cli -a +15045173983 receive --timeout 10"
+# From any other machine:
+ssh ratio.tailf3bb8c.ts.net "signal-cli -a +15045173983 receive --timeout 10"
#+end_src
=receive= prints every queued envelope and exits 0 once the queue drains or the
@@ -116,10 +146,21 @@ account is a reliability risk on the one channel that reaches Craig when he is
away.
The fix mirrors roam-sync: a systemd user timer that drains the queue on a
-cadence, keeping the account warm and, as a bonus, picking up async replies. With
-the account linked on both machines, each device wants its own regular receive,
-so the timer runs on *both* velox and ratio (the shared =common= dotfiles
-package, same home as roam-sync).
+cadence, keeping the account warm and, as a bonus, picking up async replies. Each
+device *holding* the account wants its own regular receive, so the timer runs on
+every daily driver (the shared =common= dotfiles package, same home as
+roam-sync). It no-ops cleanly where the account is *genuinely* absent, which is why velox
+running it since 2026-08-13 is harmless rather than an error — the 2026-07-20
+rationale here said "linked on both machines", which no longer holds even though
+the arrangement it justifies still does.
+
+*Do not read that no-op as proof of health.* The guard is
+=signal-cli listAccounts 2>/dev/null | grep -q=, which cannot tell "the account
+is not on this machine" from "listAccounts failed" — the redirect discards the
+error and grep simply finds nothing either way. On 2026-08-19 that was masking a
+real outage on ratio, the only machine holding the account: a PATH version skew
+made the timer run signal-cli 0.14.5 against a database 0.14.7 had upgraded, so
+every 15-minute fire reported success while receiving nothing, for 17 days.
- Script: =scripts/signal-receive.sh= (rulesets, so both machines get it on
=git pull=). It no-ops cleanly on a machine that lacks the account.
@@ -140,25 +181,35 @@ systemctl --user status signal-receive.service # confirm a clean receive
* signal-cli setup notes
- *Version:* signal-cli 0.14.5 on velox (2026-07-20).
-- *Accounts:* velox's signal-cli holds the pager identity =+15045173983= as the
- registered primary (account file 465310). ratio's signal-cli holds two
+- *Accounts:* /as of 2026-07-20; velox's half is gone — its 2026-08-13 reinstall
+ wiped the registration./ velox's signal-cli holds the pager identity
+ =+15045173983= as the registered primary (account file 465310). ratio's signal-cli holds two
accounts: Craig's personal number =+15103169357= (its own primary,
note-to-self only — no phone push) and the pager identity as a *linked device*
(Device 2, "ratio-pager", linked 2026-07-20). Both accounts coexist; target
the pager with =-a +15045173983=. A future daily driver joins the same way.
-- *signal-mcp:* on velox, Claude sessions may also expose a =signal-mcp= tool
- (=send_message_to_user=, same pager identity) configured in velox's global
- =~/.claude.json=. It works there but is invisible from any other machine and
- from non-Claude runtimes, so =agent-text= is the portable habit. The old
+- *signal-mcp:* /superseded 2026-08-19 — this said the tool is configured in
+ velox's global Claude config and "works there". It is not registered there any
+ more; that config now lists only =linear= and =slack-deepsat=./ Treat
+ =signal-mcp= as a per-machine nicety that may or may not be present, and use
+ =agent-text= as the portable habit. The old
=page-signal= shell script was removed 2026-06-12 — do not resurrect it.
-- *Linking a device:* to add a second signal-cli as a linked device of the pager
- account (see the open decision below), provision it from the new machine and
- approve the link from the account holder:
+- *Linking a device:* /superseded 2026-08-19 — nobody can run this today./ The
+ recipe below needs the *primary* to approve the link, and the primary lived on
+ velox whose keys the reinstall destroyed. A linked device cannot authorize
+ another, so ratio cannot approve one either. Until the number is re-registered
+ there is no machine that can execute this, and re-registering is not free — it
+ mints a new primary, drops ratio's existing link, and needs an SMS or voice
+ code delivered to the pager number, which is unrecoverable if that code cannot
+ be received. Read
+ [[file:2026-08-19-agent-text-relay-fallback-decision.org]] before attempting
+ it; it carries that tradeoff in full. Kept here as the procedure to follow
+ *once* a primary exists again:
#+begin_src bash
# On the new machine — prints a tsdevice:/ URI (render as QR to approve):
signal-cli link -n "ratio-pager"
- # Approve from velox (the primary device):
+ # Approve from the machine holding the PRIMARY registration:
signal-cli -a +15045173983 addDevice --uri "tsdevice:/?uuid=…"
#+end_src
diff --git a/docs/design/2026-08-19-agent-text-relay-fallback-decision.org b/docs/design/2026-08-19-agent-text-relay-fallback-decision.org
new file mode 100644
index 0000000..48dd572
--- /dev/null
+++ b/docs/design/2026-08-19-agent-text-relay-fallback-decision.org
@@ -0,0 +1,155 @@
+#+TITLE: agent-text relay fallback — decision record
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-08-19
+
+* What broke
+
+velox's 2026-08-13 reinstall wiped its =signal-cli= registration. Its account
+store came back empty.
+
+That alone would have cost velox one channel. =agent-text= made it cost the
+fleet, because the script hardcoded =VELOX_HOST= as its single ssh relay
+target. So a send from velox relayed to velox, hit the same empty store, and
+failed; and any third machine without a local account relayed into the same
+dead end. Only ratio still worked, and only because it holds the account as a
+linked device and takes the direct branch.
+
+The failure text said "velox down or unreachable?", which was actively
+misleading. velox was up and on the tailnet the whole time. It simply was not
+registered any more.
+
+* Why re-registering velox was not the fix
+
+velox held the *primary* registration, not a linked device, and its keys are
+gone. In Signal a linked device cannot authorize another device, so ratio
+(device 2) cannot link velox back. There is no QR to scan.
+
+Re-registering the number from velox would work, but it mints a new primary,
+drops ratio's link, and needs an SMS or voice code delivered to
++15045173983 — and if that code cannot be received, the identity is
+unrecoverable. That is real downside risk taken on to restore exactly the
+single-primary topology that had just proved fragile. I chose the relay
+repoint instead.
+
+* Two proposals, one chosen
+
+Both =.emacs.d= (2026-08-14) and archsetup (2026-08-16) diagnosed this
+independently and proposed replacing the constant with an ordered relay list.
+I took archsetup's, on one difference.
+
+=.emacs.d='s version tries each host in order and stops at the first success.
+On velox that works — but only because ratio happens to be listed first. It
+never skips *self*, so the failure it fixes is still reachable: a fleet whose
+list led with velox would relay velox to velox exactly as before. It repairs
+the instance rather than the defect.
+
+archsetup's adds the guard:
+
+: [ "${host%%.*}" = "$self" ] && continue
+
+Reaching the relay branch *means* the account is not local. So an ssh round
+trip to this machine lands on the same store that just came back empty and
+cannot succeed, whatever the list order. That is the defect; the ordering was
+the symptom.
+
+archsetup's also makes the list overridable (=AGENT_TEXT_RELAYS=), which is a
+genuine improvement over both the old constant and a fixed array — the fleet
+assumption can be stepped around per invocation without editing the script.
+
+* The question neither proposal had answered
+
+=.emacs.d= raised the sharpest point in either handoff and left it open: a
+relay host that is *reachable but unregistered* looks identical to success
+from the caller's side unless the exit code is checked. The loop breaks on
+=rc -eq 0=, so if =signal-cli send= exits zero against an empty account store,
+the loop reports success for a message that went nowhere — the worst possible
+outcome for a channel whose entire job is reaching me when I am away.
+
+Answered empirically before applying, on velox, which is itself the
+reachable-but-unregistered case:
+
+: $ signal-cli -a +15045173983 send -m ... <uuid>
+: User +15045173983 is not registered.
+: exit code: 1
+
+So the failure is clean and the loop advances correctly. Recorded in
+=protocols.org= so the next reader does not have to re-derive it.
+
+* What else was stale
+
+Fixing the script exposed that =protocols.org= "Reaching Craig" asserted the
+same dead fact — the account "is registered on velox (primary) and ratio
+(linked device)". Rewritten to state the *derivation* (any machine whose local
+=signal-cli= holds it sends directly; check with =signal-cli listAccounts=)
+rather than the snapshot, per the host-identity rule. The snapshot form is
+what rotted; a second snapshot would rot the same way at the next rebuild.
+
+The same section's =signal-mcp= claim was also false by then: velox's global
+=~/.claude.json= carries only =linear= and =slack-deepsat= after the reinstall.
+Softened to "may or may not be registered on the machine you're on".
+
+* What the review caught
+
+The guard I chose this proposal *for* had the same shape of flaw I rejected the
+other proposal over, and I did not see it. It read:
+
+: [ "${host%%.*}" = "$self" ] && continue
+
+That strips the domain off the candidate host but not off =uname -n=. Both
+machines return a short nodename today, so it worked — and nodename is just
+whatever =/etc/hostname= holds. One =hostnamectl set-hostname= to an FQDN on
+any machine and the guard silently stops firing, restoring the 2026-08-13 bug
+on the away channel with nothing to announce it. So the defect fix was itself
+instance-dependent, which is the exact criticism I had made of .emacs.d's
+version one section above. Both sides are stripped now, and the test that used
+to assert the direction that already worked was repointed at the direction that
+was broken.
+
+Three more, all correct and all fixed here:
+
+- The failure message named the relay list on *both* branches, so a local
+ =signal-cli= failure on the machine that actually holds the account — ratio,
+ the fleet's only working sender — would send a debugger chasing the tailnet
+ while the fault sat on the box in front of them. That is the same misdirection
+ as the old "velox down or unreachable?" text that this change exists to
+ remove. It now carries a per-branch reason.
+- The header's "Verified end to end ... 2026-08-16 (velox -> ratio, relay)" was
+ archsetup's verification, not mine, sitting in a list of two I could vouch for
+ and inheriting their credibility. Their handoff reports running the exact
+ relay command from velox and getting a Signal timestamp with rc 0. That is
+ real, and it is on report — the header now says so.
+- =protocols.org= and the script header both point at the runbook in
+ =docs/design/= as authoritative, and that runbook still asserted the retired
+ topology in four places. Correcting two surfaces and leaving the one they
+ route to is worse than not having started: the contradiction reads as a
+ disagreement between sources rather than one uniformly old story. The runbook
+ now opens with a dated superseded note, and =signal-receive.sh='s header
+ comment moved to the derivation form.
+
+* Verification
+
+- =bash -n= and =shellcheck -S warning= clean.
+- =agent-text.bats= 5 tests → 10, all green. The suite now stubs =uname= so the
+ self-skip is exercised deterministically instead of depending on which
+ machine runs it.
+- Mutation-proven rather than assumed. Deleting the self-relay guard reds tests
+ 3, 4 and 9; making the loop break unconditionally reds test 5; seeding =rc=0=
+ instead of =1= reds test 9. Each mutation is caught by a distinct test, so
+ the suite guards the behavior rather than merely covering the lines.
+- Full suite: only the pre-existing =ai-launcher-runtime.bats= codex failure
+ remains, which is a velox environment gap tracked separately.
+- The two review fixes are mutation-proven the same way rather than assumed:
+ reverting the self-side strip reds the FQDN test, and making the failure
+ message branch-blind again reds the local-failure test. Suite is 11 tests.
+
+* Provenance of the relay verification
+
+Recorded here because the working directory was filed away and this is the only
+surviving copy. archsetup's 2026-08-16 handoff states: "Verified from velox
+before proposing: signal-cli is at /usr/bin/signal-cli on ratio's
+non-interactive PATH, and the exact relay command agent-text would run returned
+a Signal timestamp with rc 0. Direct send from ratio also verified. bash -n and
+shellcheck -S warning are both clean on the attached file."
+
+That is the source for the header's 2026-08-16 line. I did not re-run a live
+relay send, because doing so delivers a real message to Craig's phone.