What the voice layer runs
Conversational dynamics
Turn-taking, pause handling, interruption and backchannels. Higher is better; bar length is relative to 100%.
The correction
What the voice layer actually owns
The intent is written in the code, not inferred. From apps/server/src/circe/Services/CirceLiveVoice.ts:
"The live model only owns the spoken conversation and the decision to ask the backend for help. Every project, task, and provider decision stays with the deterministic Circe Director behind the delegation."
The instruction builder backs that up. It forbids the voice model from holding tools, guessing results, or inventing names and statuses, and it says so repeatedly: "Delegate before giving an answer that depends on backend work", "Never invent project names, task names, statuses, or outcomes", "Delegate instead". The backend supplies the quick actions (weather, time, open website) as tools the model delegates to, not tools the model runs.
The only things it answers itself are bounded, factual overview questions, and it can only do that because the client hands it a capped snapshot: connected nodes, current project, current task, known projects with aliases, available providers, the default agent model, running task count, and up to eight recent tasks. That is the whole direct-answer surface.
So the voice model does three jobs: run the spoken conversation (turn-taking, backchannels, interruption, tone), decide when to hand the utterance to the backend, and read back short facts from the snapshot. The tools, the grounding, the disambiguation, the provider selection, and the coding work are all downstream of the delegation. Your read is correct.
Score the right thing
The axes that matter, and the one that doesn't
τ-Voice is "proportion of replica customer service scenarios resolved while acting as a customer support agent" with domain tools and a policy document. It measures a model holding the tools and completing the task. Circe tells its voice model to do the opposite. Scoring the standard Gemini tier on τ-Voice is like scoring a receptionist on their welding.
The axes Circe's voice layer does own are speech reasoning (can it understand spoken requests, including reasoning delivered by voice), conversational dynamics (turn-taking, pauses, interruption, backchannels), and task success (when it does produce a call, is it the right one). On those, the cheap Gemini tier is competitive or better.
| Model | Speech reasoning | Conv. dynamics | Task success | First audio | Cost / hr input audio |
|---|---|---|---|---|---|
| Gemini 3.8 Live | 92% | 96.1% | 93.2% | 1.18 s | $0.84 |
| GPT-Live-1 (Sol, low) | 89% | 97.3% | 90.9% | 1.24 s | $4.47 |
| GPT-Live-1 (Astra, medium) | 90% | 94.9% | 87.4% | 1.34 s | $5.83 |
| Gemini 3.8 Live Ext. Thinking | 98% | 91.9% | 89.1% | 1.35 s | $3.50 |
| Gemini 3.1 Flash Live High | 97% | 74.3% | 71.8% | 2.99 s | $1.75 |
What the voice layer does not run
τ-Voice, kept here for contrast only
Tool-using scenario completion. Circe hands this to the Director, so this gap is not a defect in the product. Bar length is relative to 70%.
Money
The voice layer is the only cost this changes
With client delegation, the voice model does not bill a backend text model. The coding work runs on the user's provider subscription (Codex, Claude, Cursor, Grok, OpenCode), selected by the Director. So the front-end rate is the entire delta for this decision. Gemini 3.8 Live is measured at $0.84 per hour of input audio against $4.47 for GPT-Live-1 (Sol) and $5.83 for (Astra). Roughly a fifth of the price on the line that actually moves.
Two caveats still apply.
Long sessions. Gemini Live bills per token and re-bills accumulated audio context every turn, so a session gets more expensive as it runs unless context compression is tuned. GPT-Live-1's $0.05 per minute is flat per second and stays linear. The $0.84 figure is measured on a 40-question subset, not on Circe's session shape, so if the goal is longer sessions, measure that before committing.
Preview pricing. Both Gemini 3.8 Live tiers are preview. Gemini's 3.8 Flash text price doubles on 1 January 2027, and a preview voice price can move the same way. The comparison is good today and not contractual.
The real blocker
A swap is a rewrite, not a config change
There is no VoiceProvider, VoiceDriver or realtime adapter in the repo. ProviderAdapterShape has no audio methods. The live voice path is hardcoded to GPT-Live-1 down to the URL, the request body, the event names and the data-channel label.
What ships today
- WebRTC peer connection with an oai-events data channel.
- POST /v1/live/sessions with delegation: client and an SDP offer.
- Server events: session.started, session.delegation.created, transcript deltas, session.closed.
- Model speech commands: commentary.append and thinking.append, each with a delegation_id.
- Close through /v1/live/sessions/{id}/hangup, mirrored by the cloud relay.
- One secret: circe-live-voice-openai-api-key.
What Gemini forces you to change
- Transport. Gemini Live is WebSocket only and server-to-server. Google tells you to proxy through an app server and points at LiveKit or Pipecat for WebRTC.
- Session model. setup / setupComplete and ephemeral tokens instead of a minted WebRTC session.
- Delegation. Function calling with toolCall / toolResponse, not a client-delegation event the model creates.
- Events and close. serverContent, goAway, session resumption. Different everything.
- The relay. It currently fronts OpenAI session create and hangup. It needs a second provider path.
- Settings and contracts. Provider, model, voice, endpoint, auth, and the create-result shape all change.
The relevant files: apps/server/src/circe/Services/CirceLiveVoice.ts, packages/contracts/src/circeLiveVoice.ts, apps/web/src/components/circe/CirceLiveVoice.logic.ts, and packages/circe-client-runtime/src/circe/liveVoice.ts. Adopting Gemini means a new voice-provider seam plus a second implementation behind it, not a branch inside these files.
Third candidate
StepFun: fits the architecture, fails the dependability test
StepFun's StepAudio 3 Realtime is the closest architectural match to Circe of anything on the board. It is a full-duplex WebSocket speech-to-speech model with built-in ASR, VAD and context management, natural interruption and backchannels, and a Voice Agent that calls tools and runs backend tasks while the conversation continues. That is the Circe shape exactly. StepFun even ships a terminal voice-coding assistant, Step-Realtime-CLI, which is the same product category as this repo.
On paper it should be the front-runner. Four facts stop it from being one today.
What works in its favor
- Full-duplex with interruption and backchannel handling, the exact conversational behavior Circe needs.
- Built-in ASR, VAD, transcripts and context management, so there is less client work than with Gemini.
- Tool calling and background tasks, which maps directly onto client delegation.
- Leading speech reasoning and conversational dynamics on the public board (100% and 98.9%).
- StepFun builds the voice-coding use case itself, so the scenario is not hypothetical.
What blocks it today
- Preview id, no price. It ships as stepaudio-3-realtime-preview on a free trial. StepFun says the preview "will be retired and a formal paid version added." You cannot depend on a model id and a price the vendor has told you will change.
- The reachable tier may train on your data. The terms reserve the right to use Usage Data, including prompts and outputs, to train and fine-tune models, and the free-trial / Startup Program license states it explicitly. That is the only tier available right now.
- Latency is unproven and looks bad. The only independent measurement is 8.83 s time to first audio, against 1.18 s for Gemini 3.8 Live and 1.24 to 1.34 s for GPT-Live-1. StepFun's docs claim low latency, so this may be a reasoning-path measurement. It is still the number you have.
- No agentic or arena data. No τ-Voice, task success or arena score, so delegation-trigger behavior, the one thing you actually need, is entirely unmeasured.
Open weights do not rescue it. The full Step-Audio-Chat model is 130B and needs roughly 265 GB of GPU memory, which is not a per-node deployment. The Apache-2.0 Step-Audio-TTS-3B is self-hostable at 8 GB, but it is text-to-speech only, so it replaces the synthesis half and gives you none of the conversation, interruption or delegation loop.
There is a quieter second option. Step-Audio R1.1 (Realtime) is measured at 1.53 s first audio with published rates near $0.06 per hour of audio input and $1.69 per hour of audio output, which is far cheaper than anything else here. But it has no conversational dynamics, arena, task-success or agentic score at all, so it is less verified than the flagship, not more.
Verdict: a spike, not a commitment. The architecture is a better fit than Gemini's. Revisit when the paid version ships with a fixed id, a published price, and a DPA-covered tier that does not train on usage. Then measure first-audio latency on your own network before believing either 8.83 s or "very low latency".
Recommendation
Standard Gemini 3.8 Live is a real contender. Build the seam, then pilot it.
1. Drop τ-Voice as the gate. The voice model is a conversational front end plus a delegation trigger. Score it on speech reasoning, conversational dynamics, latency and cost. Revision 1 scored it on tools it is explicitly told never to hold.
2. Standard Gemini 3.8 Live is now a legitimate primary candidate on merit and cost. It leads GPT-Live-1 on speech reasoning (92% vs 89-90%) and task success (93.2% vs 87.4-90.9%), ties it on conversational dynamics (96.1%), responds slightly faster (1.18 s vs 1.24-1.34 s), and costs about a fifth as much. The one number it loses is τ-Voice, which maps to the Director, not the voice model.
3. Skip Extended Thinking. Its two extras, multi-step reasoning and spoken narration of progress, duplicate the Director and the existing note()/speak() commentary. It costs more than the standard tier and scores lower on conversational dynamics (91.9% vs 96.1%). You were right that it is the wrong choice.
4. The blocker is the seam, not the model. A VoiceProvider interface and a Gemini adapter are the prerequisite. That is the work. Everything else is choosing which model id to pass.
5. Measure the two things AA does not cover. First, delegation-trigger reliability under the Circe policy: does the model hand off when it should, rather than chatting or answering from the snapshot. The client already has a safety net (a tool-refusal regex that auto-delegates the last utterance, plus transcript-based capture), but it should be validated against Gemini. Second, long-session cost with context compression, since that is the exact shape more live time will produce.
6. Treat StepFun as a spike, not a swap. StepAudio 3 Realtime fits the architecture better than Gemini does, which is the strongest argument for it. But it is a preview id with no price, the reachable tier can train on your data, and its only independent latency figure is 8.83 s. Revisit at paid GA with a non-training tier, then measure latency yourself.
7. One unrelated check. The default voice is marin. GPT-Live-1 shipped twelve voices (Quartz, Ripple, Vesper, Willow, Stone, Gleam, Meridian, Bossa, Tempo, Beacon, Delta, Cinder). Verify marin is still accepted rather than silently falling back.
Sources
Where the numbers come from
- Artificial Analysis, Speech to Speech leaderboard, captured 16 September 2026. Reasoning, dynamics, τ-Voice, task success, latency and cost per hour. artificialanalysis.ai/speech-to-speech
- OpenAI, "Build more natural voice experiences with GPT-Live-1 in the API", 10 September 2026. $0.05 per minute, billed per second, backend billed separately. openai.com and developers.openai.com/api/docs/guides/live
- Google, Gemini 3.8 Live and 3.8 Live Extended Thinking, 15 September 2026, preview. Coverage: OrcaRouter, Gemini API pricing
- Google, Live API billing. Accumulated audio tokens re-billed per turn. Google AI Developers Forum and Live API best practices
- StepFun, StepAudio 3 Realtime and StepAudio 3 Chat docs, preview-trial notice, and audio models list. platform.stepfun.ai
- StepFun overseas Terms of Service, DPA and privacy policy: SPARKLING AI PTE. LTD. and STEPFUN TECHNOLOGY (HK) LIMITED, with SCCs. terms and DPA. Mainland platform is a separate PRC-law contract at platform.stepfun.com.
- Step-Audio open weights: Apache-2.0 code, 130B chat model at roughly 265 GB GPU memory, TTS-3B at 8 GB. github.com/stepfun-ai/Step-Audio
- This repository at Jarvis: the voice-model ownership comment and delegation policy in CirceLiveVoice.ts, the bounded reference snapshot in CirceLiveVoice.logic.ts, and the absence of a voice-provider abstraction.