aboutsummaryrefslogtreecommitdiff
path: root/working/context-engineering-rightsizing/proposals.org
diff options
context:
space:
mode:
authorCraig Jennings <c@cjennings.net>2026-07-27 13:15:14 -0500
committerCraig Jennings <c@cjennings.net>2026-07-27 13:15:14 -0500
commit79ed3b09a9ee2a63fb55d2354aa0c77ea24c6efa (patch)
tree928273e02ea3d40a2aee2bdde4a8f922aec9762f /working/context-engineering-rightsizing/proposals.org
parent7ea1d7b1402eb68a13479f8073b84819c1d59ec8 (diff)
downloadrulesets-79ed3b09a9ee2a63fb55d2354aa0c77ea24c6efa.tar.gz
rulesets-79ed3b09a9ee2a63fb55d2354aa0c77ea24c6efa.zip
docs: add context-engineering rightsizing analysis and rollout
I wrote three working documents while reviewing the Claude 5 context-engineering post, the Opus 5 prompting guide, and the Fable field guide against what this repo ships downstream. In proposals.org I measured the always-loaded surface at 32,123 words and ranked six changes, plus two places where a post contradicts something we arrived at deliberately. I also reread the rules as your prompts rather than as agent context, which is where the sharper finding is: 41 execution and hygiene workflows against 6 discovery and design ones, on a system whose bottleneck has moved. In rollout.org I phased the work and named the seven decisions that gate it. My lead finding is that the harness system prompt already carries most of what the Opus 5 guide recommends adding, so the posts' value here is subtractive. Applying them additively would make the duplication worse. In metrics.org I split the posts into separable claims and marked which are testable here and which are judgment calls, rather than inventing a metric for the ones an eval harness would be needed to settle. I also stated the pilot's stop conditions in advance, including a zero-tolerance threshold on the load-bearing rules, so I can't renegotiate that threshold later under pressure to make a phase succeed.
Diffstat (limited to 'working/context-engineering-rightsizing/proposals.org')
-rw-r--r--working/context-engineering-rightsizing/proposals.org319
1 files changed, 319 insertions, 0 deletions
diff --git a/working/context-engineering-rightsizing/proposals.org b/working/context-engineering-rightsizing/proposals.org
new file mode 100644
index 0000000..40bfcf8
--- /dev/null
+++ b/working/context-engineering-rightsizing/proposals.org
@@ -0,0 +1,319 @@
+#+TITLE: Context-Engineering Rightsizing — Proposals
+#+AUTHOR: Craig Jennings
+#+DATE: 2026-07-27
+
+* Status
+
+Proposals only. Nothing here is applied. Source: three Anthropic posts Craig
+supplied 2026-07-27 — the Claude 5 context-engineering post (2026-07-24), the
+Opus 5 prompting guide, and the Fable 5 field guide (2026-07-06).
+
+* The measurement
+
+Every session in every project loads this before the user's first word:
+
+| Surface | Words |
+|--------------------------+--------|
+| =claude-rules/= (20 files) | 25,386 |
+|--------------------------+--------|
+| =protocols.org= | 6,620 |
+|--------------------------+--------|
+| Project =CLAUDE.md= | 117 |
+|--------------------------+--------|
+| *Total* | 32,123 |
+|--------------------------+--------|
+
+Roughly 40,000 tokens of always-on context. The context-engineering post reports
+Anthropic cut over 80% of Claude Code's *system prompt* with no measurable loss
+on their coding evals. That's their number on their surface, not a target for
+this one, but the direction is the finding: this system is built in the "then"
+style the post retired.
+
+The content isn't the problem. The post says skills are best when they "encode
+particular opinions, knowledge, or best practices particular to you" — which is
+exactly what these rules are. The problem is *load timing*: all of it is
+resident on every turn, including =commits.md= (5,561 words) in a session that
+never commits, and =todo-format.md= (4,494 words) in a session that never opens
+an org file.
+
+* Proposals
+
+Ranked by value. Each carries my confidence and what I think the real risk is.
+
+** P1 — Progressive disclosure for =claude-rules/= [high value, highest risk]
+
+*Change.* Split the twenty rule files into two tiers.
+
+- *Always-loaded core* (target under 3,000 words): the genuine invariants that
+ must fire without being summoned. The no-AI-attribution rule, the
+ cross-project boundary stop, the no-popup-menus and no-reverse-video output
+ constraints, the =date=-before-timestamps rule, and a short index naming
+ which skill covers what.
+- *On-demand tier*: everything procedural, converted to skills whose
+ descriptions trigger them. =commits.md= becomes a publish skill that loads
+ when a commit or PR is in play. =todo-format.md= loads when an org todo file
+ is touched. =testing.md=, =working-files.md=, =docs-lifecycle.md=,
+ =org-tables.md=, =keybinding-display.md= likewise.
+
+*Why.* This is the post's central move, and the token math is the argument.
+
+*The real risk, stated plainly.* A rule that isn't loaded can't fire. Skill
+triggering is probabilistic in a way that always-resident text isn't. The
+failure mode is silent: a session commits without the voice pass because the
+publish skill didn't trigger, and nothing announces the miss. That's the same
+silent-failure shape as the two probe defects fixed this morning.
+
+*Mitigation.* Anything whose violation is expensive and hard to reverse stays
+in the always-loaded core, whatever its length. The split is by *blast radius*,
+not by word count. And the migration goes one file at a time with a live trial,
+not as a single cutover.
+
+*Confidence.* High that the direction is right. Medium on where exactly each
+line falls — that's a judgment call per rule, and worth walking together.
+
+** P2 — Stop pre-filtering review findings [high value, low risk]
+
+*Change.* =review-code/SKILL.md:251= says "Drop Low-confidence issues before
+the final report." Line 434 repeats it. Replace with: report every finding
+carrying an explicit confidence label, then filter in a named second pass.
+
+*Why.* The Opus 5 guide is specific about this: "If your review prompt says
+'only report high-severity issues' or 'be conservative,' the model may follow
+that instruction literally and report less; ask it to report everything and
+filter in a separate pass instead." The guide also reports that on this model
+the extra findings are mostly real rather than false positives, which is the
+premise the drop-rule was written against.
+
+*Confidence.* High. This is the most directly-actionable finding in the three
+posts, and it names the exact pattern the skill implements.
+
+** P3 — Add the unknowns-discovery practices [medium value, low risk]
+
+The field guide describes eight practices. The sweep found these absent:
+=unknown unknowns= framing (0 files), =implementation notes= (0), =quiz= (0).
+Present but thin: =blind spot= (1 file), =interview= (2), =pitch= (1).
+=brainstorm= appears in 5 and =references= is well covered by
+=ui-prototyping.md=, which already runs ahead of the post.
+
+*Change.* Add a blind-spot-pass practice and an interview practice, and an
+implementation-notes convention for long builds (a scratch file logging
+deviations from plan, which then feeds the retrospective). Add the quiz pattern
+to the review or wrap surface.
+
+*Placement matters.* These go in the on-demand tier from P1, never the
+always-loaded core — otherwise this proposal fights the one above it.
+
+*Confidence.* Medium-high on the practices being useful. Lower on the quiz,
+which may not fit how you actually work.
+
+** P4 — Effort calibration on the unattended loops [medium value, low risk]
+
+*Change.* Sentry, work-the-backlog, and the no-approvals speedrun run many
+passes at default effort. The Opus 5 guide says to use =low= and =medium=
+liberally as the primary cost control wherever quality holds, stepping up only
+for demanding work.
+
+*Why.* Sentry fires hourly. Effort is the lever with the largest cost
+multiple, and most sentry passes are mechanical sweeps.
+
+*Confidence.* Medium. The guide says to re-run an effort sweep on your own
+evals rather than trusting carried-over defaults, and we have no evals — so
+this wants a trial and a judgment, not a blanket change.
+
+** P5 — Positive framing over prohibition [low value, low risk, slow]
+
+*Change.* =commits.md= carries 41 prohibition markers (NEVER / DO NOT /
+MANDATORY / CRITICAL) across 5,561 words. =todo-format.md= 21,
+=testing.md= 19. Rewrite the ones that aren't hard invariants as positive
+descriptions of the wanted behavior.
+
+*Why.* The Opus 5 guide: "Positive examples of the communication style you want
+tend to be more effective than instructions about what not to do."
+
+*Keep as prohibitions:* the AI-attribution rule and anything else where the
+worst case is genuinely unacceptable. Those earn their emphasis.
+
+*Confidence.* Medium. Real but diffuse, and it competes with P1 for the same
+editing effort. Probably rides along with P1 rather than standing alone.
+
+** P6 — Deduplicate the two always-loaded surfaces [medium value, low risk]
+
+=protocols.org= restates rules that also live in =claude-rules/=: the
+cross-project boundary, the working-files convention, the AI-attribution ban,
+inbox cadence. Both are loaded every session, so each duplicated rule is paid
+for twice, and the two copies can drift apart.
+
+*Change.* One home per rule. =protocols.org= keeps the pointer, the rule file
+keeps the content — or the reverse, but not both.
+
+*Confidence.* High on the duplication being real, medium on which surface
+should own each rule.
+
+* Conflicts — your call, not mine to inherit
+
+Two places where a post contradicts something this system arrived at
+deliberately. I'm flagging rather than adopting.
+
+** C1 — =verification.md= versus the over-verification warning
+
+The Opus 5 guide says: "If your prompt contains explicit verification
+instructions ('include a final verification step for any non-trivial task',
+'use a subagent to verify'), remove them: instructions like these cause
+over-verification on Claude Opus 5, and removing them reduces wasted tokens
+with no loss in quality."
+
+=verification.md= is 1,486 always-loaded words of exactly that shape. But the
+two aren't the same thing, and the distinction decides the answer:
+
+- Its *honesty core* — don't claim tests pass without running them, "unable to
+ verify" is a required outcome, replace beliefs with evidence — is about
+ truthful reporting, not about adding verification steps. The guide doesn't
+ argue against it.
+- Its *process injection* — green baseline before starting, full suite as its
+ own step before every commit — is the shape the guide names.
+
+*My read:* keep the honesty core, shorten it, and let the process injection
+move into the publish skill where it fires only when publishing. *My
+confidence: medium*, and this is the one I'd most want you to overrule if it
+feels wrong. It's also the rule closest to your standing "never guess, always
+check" direction, so the guide's advice and your stated preference genuinely
+pull against each other here.
+
+** C2 — =subagents.md= versus the delegation warning
+
+The guide says "do not use subagents to verify or double-check your own work."
+=subagents.md= has a review-gate cadence and a rule to dispatch a *fix*
+subagent rather than repairing in the orchestrator's context.
+
+*My read:* the fix-subagent rule is about context pollution, not verification,
+so it survives. The review-gate cadence is closer to the flagged pattern and
+wants a look. Most of =subagents.md= already matches the guide's advice — it
+argues against spawning for small work and against letting the agent pick its
+own scope, which is what the guide asks for. *Confidence: medium-high.*
+
+* One thing the posts would change about this document
+
+Both the context-engineering post and the field guide argue that a rich
+reference beats a prose description — an HTML artifact, a test suite, source
+code. This proposal document is prose in org, which is your reading format and
+the right call for a decision doc. Worth noting the tension rather than
+silently ignoring it: for the *next* artifact in this line of work, an HTML
+comparison of the before and after rule tree would likely beat another org
+file.
+
+* Scope proposal for the consistency sweep
+
+The audit behind this document was targeted, not exhaustive — I checked the
+claims the posts made and measured the surface. A real inconsistency sweep over
+20 rule files, 47 workflows, and the skills is its own pass.
+
+Proposed scope, in order:
+
+1. *Contradictions between always-loaded surfaces* — the P6 duplication set,
+ read side by side for drift rather than just counted.
+2. *Stale facts* — assertions about tools, paths, and behavior that were true
+ when written. The spot check found the =agent-page= to =agent-text= rename
+ correctly handled, so this may be in better shape than expected.
+3. *Instructions that contradict each other across files* — the failure the
+ context-engineering post opens with. This is the expensive one and the most
+ valuable.
+
+Sizing: item 1 is an afternoon, item 2 is mechanical, item 3 is the real work.
+
+* From your side of the desk
+
+Everything above treats these files as the agent's context, to be rightsized.
+That was the smaller question. Read as *your prompts* — the map you hand every
+project — the finding is different, and it's the one worth acting on.
+
+** The bottleneck this system was built for has moved
+
+Counting the workflows by what they're for: 41 are execution and hygiene
+(publish, task grading, inbox routing, session archiving, calendar, email,
+sync), 6 are discovery and design (the spec trio, retrospective, code-quality,
+readability-audit). Roughly seven to one.
+
+That ratio was correct when the risk was the model doing things wrong. The
+field guide's claim is that the risk moved: "Claude Fable is the first model
+where I find the quality of the work is bottlenecked by my ability to clarify
+its unknowns." If that's true here too, the system is heavily invested in the
+half of the problem that got easier and thin on the half that didn't.
+
+Not an argument to delete the execution machinery. It's load-bearing, and
+hygiene that runs itself is exactly what you want automated. The argument is
+that *the growth has all been on one side*, and the next increment of quality
+probably comes from the other one.
+
+** The instructions don't practice what they demand
+
+Three concrete cases, all checkable:
+
+- =commits.md= says "Brief. Terse is preferred. A one-sentence body beats a
+ paragraph saying the same thing." It is 5,561 words, the longest file in the
+ set.
+- =interaction.md= bans bold and code spans in chat output because they render
+ as reverse video. The rule files carry 591 bold markers.
+- =testing.md= mandates TDD as "non-negotiable" and follows with a table of
+ eight rationalizations to refuse. That's the repeat-yourself-and-overconstrain
+ shape the context-engineering post retired, applied to a rule the model no
+ longer needs argued into.
+
+This matters beyond tidiness. An instruction whose own form contradicts its
+content is a mixed signal of exactly the kind the post opens with — the
+model spends effort reconciling "be terse" against a source that isn't.
+
+** Over-specification has a cost you're paying, not me
+
+The field guide: "If you are too specific, Claude will follow your instructions
+even when a pivot may be more appropriate. If you are too vague, Claude will
+often make choices and assumptions based on industry best practices that may
+not be a fit."
+
+The workflows are phase-numbered and gate-heavy. For repeatable operations —
+wrap-up, publish, inbox — that precision is right and should stay. For the
+creative surfaces it's a straitjacket: a spec workflow that prescribes its own
+phases forecloses the pivot that a brainstorm was supposed to surface. The
+system doesn't currently distinguish those two classes, and it should. Same
+file format, opposite optimal specificity.
+
+** Which gates are guardrails and which are preference
+
+The approval gates (publish, inbox shared-asset, spec) were written when the
+worst case was an agent shipping something wrong. Some of them you'd want
+regardless, because you want to be in the loop on what goes out under your
+name. Those are preference and should never be cut.
+
+Others are pure guardrail, and the context-engineering post's argument applies:
+"these constraints were once needed to avoid worst case scenarios, we have
+since found we can delete many of them."
+
+I can't tell the two apart from the files — they read identically. Only you
+know which gates you'd keep if the agent were perfectly reliable. That
+separation is the highest-leverage thing you could tell me, and it's not
+something I should guess at.
+
+** One map, many territories
+
+Every project loads the same 32,000 words. The finances project loads the TDD
+mandate and the full publish machinery. =.emacs.d= loads the org-table
+standard. Some of that is right (the universal rules genuinely are universal)
+and some is the map/territory mismatch the field guide names.
+
+The language-bundle mechanism already solves this for =languages/=. Nothing
+equivalent exists for the rules layer.
+
+** What's already right
+
+Worth saying, because the list above is all deficit:
+
+- =ui-prototyping.md= is the field guide's brainstorm-and-prototype practice,
+ written down before the post and in more detail than the post gives it.
+- The spec spine (decisions plus implementation phases) is the guide's
+ implementation-plan pattern.
+- =session-context.org= is the implementation-notes pattern, and it already
+ captures deviations and dead ends.
+- =retrospective.org= exists.
+
+So the practices are partly here. What's missing is that they're framed as
+*process compliance* rather than as *unknowns discovery*, and none of them fire
+at the moment the guide says they pay off — before scope is set.