Learn · Real programs · 29 of 34
Talking agents
Goal. Hold a live spoken conversation with a voice agent, with tools and interruptions.
The agents so far answer one question at a time: a prompt goes in, the loop runs, a reply comes out. A talking agent is different. It holds one live connection to a speech-to-speech model, and audio goes both ways at the same time. While the model speaks, you can talk over it, and it can call your tools mid-sentence.
In Tin that is a voice agent. It looks like the agents you know, and the compiler checks it the
same way.
A session
Concierge.open() connects and gives you a Session. say sends text,
send sends audio, and next() waits for the next thing the conversation does:
tin run session.tin/// The weather in a city.
tool fn weather(city: string): string => "18C and clear in {city}"
voice agent Concierge {
model: "mock"
system: "You are a hotel concierge. One short sentence."
tools: [weather]
}
fn main() {
let s = Concierge.open()
s.say("weather in 'Oslo'")
var answered = false
loop {
match s.next() {
CallingTool { name, input } => print("calling {name} {input}")
ToolReturned { name, output, ok } => print("{name} gave: {output}")
Said(text) => {
print("said: {text}")
answered = true
}
TurnDone => if answered { s.stop() }
HungUp(reason) => {
print("hung up: {reason}")
break
}
_ => {}
}
}
} calling weather {"city":"Oslo"}
weather gave: 18C and clear in Oslo
said: mock: 18C and clear in Oslo
hung up: stopped Every event is a VoiceEvent, an ordinary enum, so match works on it and the
compiler knows its cases. The tool call is a turn of its own: the model asks, the runtime runs
weather while the connection stays open, and the answer comes in the next turn. That is why the loop
hangs up after the spoken answer, not after the first TurnDone.
The common events: Heard(text) and Said(text) are final transcripts of the user and
the agent; Hearing and Saying are the pieces as they arrive; Speech(chunk)
is a piece of the reply's audio; CallingTool and ToolReturned bracket a tool;
BargedIn means the user talked over the reply; HungUp(reason) is the last one, and
next() keeps returning it after the call ends.
Handlers and audio
As with text agents, on handlers run as things happen: on heard, on said,
on toolCall, on toolResult, on bargeIn. A Speech chunk is an
Audio value: 16-bit sound held by the host, not copied into your program. You can ask how long it is,
how loud, or play it.
tin run handlers.tinvoice agent Echo {
model: "mock"
// handlers run as the conversation goes, before next() hands the event on
on said(text) => print("[said] {text}")
}
fn main() {
let s = Echo.open()
s.say("hello there")
var audio = 0.0
loop {
match s.next() {
Speech(chunk) => {
audio = audio + chunk.seconds()
}
TurnDone => s.stop()
HungUp(reason) => break
_ => {}
}
}
print("{audio.fixed(2)} s of reply audio")
} [said] mock: hello there
1.02 s of reply audio "mock" answers with a tone, about 60 ms per character, so programs and tests have real audio to
work with offline.
A whole call
Most programs don't need the event loop. talk wires a microphone to the model and the model to a
speaker, clears the speaker the moment you interrupt, and returns the transcript when the call ends:
fn main() {
let transcript = FrontDesk.talk(mic(), speaker())
print(transcript)
} mic() and speaker() come from the host. In the playground
they are your browser's microphone and speakers (it asks first). In the terminal they are files or devices:
tin run call.tin --mic question.wav --speaker reply.wav # recorded audio in, the reply written out
tin run call.tin # the default microphone and speaker (needs sox) What the compiler checks
A voice agent is not a function you ask a question, so calling it like one is an error with a fix:
tin check called.tinvoice agent Concierge {
model: "mock"
}
fn main() {
print(Concierge("what's the weather?"))
} error: Concierge is a voice agent: it holds a live conversation, not one question
--> called.tin:6:9
|
6 | print(Concierge("what's the weather?"))
| ^^^^^^^^^
= note: open a session: `let s = Concierge.open()`, or run a whole call: `Concierge.talk(mic(), speaker())`
= fix: open a session Its model has to be a speech-to-speech model. A text model would accept the connection and then never speak:
tin check model.tinvoice agent Concierge {
model: "openai:gpt-5.1"
}
fn main() {} error: 'openai:gpt-5.1' is not a speech-to-speech model
--> model.tin:2:10
|
2 | model: "openai:gpt-5.1"
| ^^^^^^^^^^^^^^^^
= note: a voice agent uses one of: openai-realtime:<model>, gemini-live:<model>, bedrock-sonic:<model>, or "mock" The rules from the earlier chapters hold. A tool not in tools: never runs, a
guarded tool fn asks a person first, and what the model says may not reach JavaScript unapproved.
The microphone and the speaker are capabilities, so tin caps shows who can listen:
tin caps listen.tin/// Listen to the room for a moment.
tool fn listen(): string => match mic().read() {
Some(chunk) => "heard {chunk.seconds()} s"
None => "silence"
}
/// The time at the desk.
tool fn deskTime(): string => "9:41"
voice agent FrontDesk { model: "mock", tools: [deskTime] }
voice agent Spy { model: "mock", tools: [listen] }
fn main() {} voice agent FrontDesk
tools: deskTime
can call freely: deskTime
needs approval: nothing
capabilities: model
voice agent Spy
tools: listen
can call freely: listen
needs approval: nothing
capabilities: model, mic
tools
listen free mic
deskTime free pure Real voices
Change "mock" to a speech-to-speech model and nothing else:
model: | Needs | Runs in |
|---|---|---|
"openai-realtime:gpt-realtime-mini" | OPENAI_API_KEY | the CLI and the browser |
"gemini-live:gemini-live-2.5-flash-preview" | GEMINI_API_KEY | the CLI and the browser |
"bedrock-sonic:amazon.nova-sonic-v1:0" | access keys (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) | the CLI only |
A live connection does not last forever (Nova Sonic about 8 minutes, Gemini about 10, OpenAI an hour). Tin reconnects shortly before the limit, waits for a pause in the conversation to do it, and keeps the conversation. The reference says what has been tested against each provider.
Tasks
Task 1: A host who asks first
book is guarded. Print every tool result and whether it worked, then hang up after the spoken
answer. Run it without approval first: what does the model hear?
tin run task-1.tin/// Book a table for a number of people.
guarded tool fn book(people: int): string => "booked for {people}"
voice agent Host {
model: "mock"
tools: [book]
}
fn main() {
let s = Host.open()
s.say("book a table for 4")
// print each tool result and whether it worked, then hang up after the spoken answer
} Solution
tin run task-1.solution.tin/// Book a table for a number of people.
guarded tool fn book(people: int): string => "booked for {people}"
voice agent Host {
model: "mock"
tools: [book]
}
fn main() {
let s = Host.open()
s.say("book a table for 4")
var answered = false
loop {
match s.next() {
ToolReturned { name, output, ok } => print("{name} ok={ok}: {output}")
Said(text) => {
answered = true
}
TurnDone => if answered { s.stop() }
HungUp(reason) => break
_ => {}
}
}
} book ok=false: Tool call denied by the user. Do not retry it; continue without it. Recap
- A voice agent keeps one speech-to-speech session open.
- Events are typed, and
matchmust handle each kind. - The microphone is a capability the compiler reports.
In the reference: Talking agents · VoiceEvent · Speech-to-speech models