Voice architectures
The same turn crosses every stage. The runtime must coordinate their results.
Links containing ?t= open the video at a specific second.
The ideas to retain
Three modality architectures
1. Full cascade: audio → STT → LLM → TTS → audio
A conventional full cascade exposes a contract at every stage:
2. Audio-native input with text output + external TTS
Half-cascade is not a formal standard, but the term is now used in production frameworks. LiveKit, for example, defines a half-cascade as a realtime model that understands speech and…
Jump directly to a section
Read the reviewed transcript
This video has no narration. This transcript reproduces its on-screen text; it does not invent a spoken track.
Where text enters the path
Audio passes through transcription, decision and synthesis. Each boundary changes its representation.
Text makes stages inspectable and replaceable, but does not retain every acoustic signal.
The same turn crosses every stage. The runtime must coordinate their results.
Hear audio. Return text.
An audio-native model receives speech directly, without requiring a transcript first.
When it returns text only, an external voice can synthesize the response.
Input prosody does not automatically reach TTS. That boundary needs an explicit contract.
Fewer boundaries. The same controls.
Speech-to-speech connects input and output audio without a mandatory text boundary.
The acoustic path becomes simpler; tools and their permissions remain outside the model.
The application validates the action. A speaking model does not gain business authority.
Two directions. A separate axis.
Accepting and generating audio does not imply listening and speaking simultaneously.
With full-duplex, input and output can coexist. The system decides how to handle overlap.
Modality and turn control are different decisions. S2S does not guarantee full-duplex.
Who owns each boundary
A browser can send media directly to a provider while the backend keeps control.
Removing a framework does not remove authentication, state, tools or recovery.
Draw the actual path and give every transition an owner.
The requirement before the ranking
An explicit transcript favors a cascade; an external voice needs its own boundary.
When acoustic information matters, test audio-native input on the same task.
Compare task success, voice and reliability under equal conditions. No architecture wins universally.
