Decide under constraints
Selection optimises within the eligible set. A cheap but incompatible alternative is not a valid option.
Links containing ?t= open the video at a specific second.
The ideas to retain
Policy starts with what it cannot choose
Before optimizing quality, cost, or latency, a request needs a set of eligible candidates.
Model routing chooses before the first attempt
Once E(x) has been filtered, a router can select the candidate with the highest expected utility. One conceptual formulation, not a standard, is:
Static routing, explicit rules, and learned routers solve different problems
Jump directly to a section
Read the reviewed transcript
This video has no narration. This transcript reproduces its on-screen text; it does not invent a spoken track.
Eligibility comes first
The router has three candidate models. One lacks the required tool contract.
That model is excluded before price or latency are compared.
Selection optimises within the eligible set. A cheap but incompatible alternative is not a valid option.
Routing is not recovery
Routing chooses the first model before the request executes.
Fallback starts after a failure or a defined degradation condition.
The trace must separate decision, attempt and recovery. Do not attribute every outcome to the first model.
The deadline does not reset
The user has one thousand milliseconds. The first attempt spends four hundred, and backoff spends another hundred.
Five hundred remain for recovery, including network, processing and delivery.
A second attempt expected to take six hundred does not fit. Switching models creates no new budget.
Visible output is committed
The user has already received part of an answer when the stream fails.
Appending a fresh answer can duplicate or contradict what was shown.
The product needs an explicit interruption or replacement policy. This is not a transparent retry.
Two different forms of reuse
Response caching returns a validated answer without a new complete inference.
Prefix caching reuses some input state but still needs output generation.
Treating their hits as the same saving mixes different work boundaries.
Similarity is not equivalence
Two almost identical questions refer to different policy revisions.
Semantic proximity can retrieve an old answer after the current deadline has changed.
The cache must respect versions, permissions and truth conditions. A nearby neighbour can still be wrong.
