← all features · evolve-loop explainers

Token telemetry

The measurement-first campaign: know what every phase of every loop spends before optimizing any of it.

How it works today

Every LLM launch flows through one chokepoint — Engine.Launch in go/internal/bridge/engine.go — where a token resolver extracts usage (input, output, cache-read, cache-write) from the driver transcript and appends a per-attempt record to llm-calls.ndjson in the cycle’s run dir:

tmux transcriptclaude/codex/agy session
TokenResolverengine.go Launch hook
llm-calls.ndjsonper attempt, per phase
rollupsdossier · cache-hit ratio
evolve tokens reportper-phase totals

The canonical counts type is cyclestate.TokenUsage, wire-pinned by test so no writer can drift. Provenance rides every record (which transcript, which resolver), so the report can say source=transcript rather than guessed. Slices S1–S8 built this chain: transcript scanner → fidelity parsing → launch instrumentation → terminal projection → advisor/swarm attribution → dossier rollups → the report CLI → fleet-level shadow join.

How it got built — and the outage that almost hid it

Cycle(s)ChangeStory
S1–S7 (earlier batches)Scanner, fidelity chain, launch hook, projection, attribution, rollup schema, evolve tokens reportPlan-approved campaign, loop-implemented slice by slice
incidentFirst full-chain batch measured: all zerosDeps.TokenResolver was wired nowhere — a fail-open nil silently disabled the whole chain. The 0.96 defect recording this was then consumed from the queue without landing
705, 722, 741 FAIL ×3token-telemetry-s6-rollups-dossier — the useful failureRollups kept failing with nothing to roll up; its persistence was the only alarm that led back to the lost resolver defect
742 → 745 PASSThe 8-line composition-root fix + loud boot WARN on nil resolver (commit d9ceaa43)Re-filed 00:10, picked within one wave, landed within two
754/756 PASSFallback resolution chain in the scanner (commit 2abbaa77)Hardening so a single resolver miss degrades gracefully instead of zeroing
Lesson — fail-open must be loud. All-zero telemetry looked identical to "nothing measured yet"; the outage had no witness except a downstream task that refused to stop failing. The boot WARN and the landing-note gate (see queue integrity) both exist so this class of silence cannot recur.

Acceptance for the whole feature: the next batch’s evolve tokens report --last 8 showing non-zero, per-phase, transcript-sourced numbers — which then re-ranks seven queued optimization items by measured spend instead of guesses.

Generated by /evo:explain · sources: loop logs, cycle dossiers, ship commits, campaign plans.