<!-- Talking agents · Tin · https://tin.ad/docs/learn/voice/ · generated from the built page -->

[Learn](https://tin.ad/docs/learn/) · Real programs · 29 of 34

# Talking agents

**Goal.** Hold a live spoken conversation with a `voice agent`, with tools and interruptions.

The agents so far answer one question at a time: a prompt goes in, the loop runs, a reply comes out. A
talking agent is different. It holds one live connection to a speech-to-speech model, and audio goes both ways
at the same time. While the model speaks, you can talk over it, and it can call your tools mid-sentence.

In Tin that is a `voice agent`. It looks like the agents you know, and the compiler checks it the
same way.

## A session

`Concierge.open()` connects and gives you a `Session`. `say` sends text,
`send` sends audio, and `next()` waits for the next thing the conversation does:

*session.tin · tin run session.tin*

```tin
/// The weather in a city.
tool fn weather(city: string): string => "18C and clear in {city}"

voice agent Concierge {
  model: "mock"
  system: "You are a hotel concierge. One short sentence."
  tools: [weather]
}

fn main() {
  let s = Concierge.open()
  s.say("weather in 'Oslo'")
  var answered = false
  loop {
    match s.next() {
      CallingTool { name, input } => print("calling {name} {input}")
      ToolReturned { name, output, ok } => print("{name} gave: {output}")
      Said(text) => {
        print("said: {text}")
        answered = true
      }
      TurnDone => if answered { s.stop() }
      HungUp(reason) => {
        print("hung up: {reason}")
        break
      }
      _ => {}
    }
  }
}
```

<!-- output -->
```text
calling weather {"city":"Oslo"}
weather gave: 18C and clear in Oslo
said: mock: 18C and clear in Oslo
hung up: stopped
```

Every event is a `VoiceEvent`, an ordinary enum, so `match` works on it and the
compiler knows its cases. The tool call is a turn of its own: the model asks, the runtime runs
`weather` while the connection stays open, and the answer comes in the next turn. That is why the loop
hangs up after the spoken answer, not after the first `TurnDone`.

The common events: `Heard(text)` and `Said(text)` are final transcripts of the user and
the agent; `Hearing` and `Saying` are the pieces as they arrive; `Speech(chunk)`
is a piece of the reply's audio; `CallingTool` and `ToolReturned` bracket a tool;
`BargedIn` means the user talked over the reply; `HungUp(reason)` is the last one, and
`next()` keeps returning it after the call ends.

## Handlers and audio

As with text agents, `on` handlers run as things happen: `on heard`, `on said`,
`on toolCall`, `on toolResult`, `on bargeIn`. A `Speech` chunk is an
`Audio` value: 16-bit sound held by the host, not copied into your program. You can ask how long it is,
how loud, or play it.

*handlers.tin · tin run handlers.tin*

```tin
voice agent Echo {
  model: "mock"
  // handlers run as the conversation goes, before next() hands the event on
  on said(text) => print("[said] {text}")
}

fn main() {
  let s = Echo.open()
  s.say("hello there")
  var audio = 0.0
  loop {
    match s.next() {
      Speech(chunk) => {
        audio = audio + chunk.seconds()
      }
      TurnDone => s.stop()
      HungUp(reason) => break
      _ => {}
    }
  }
  print("{audio.fixed(2)} s of reply audio")
}
```

<!-- output -->
```text
[said] mock: hello there
1.02 s of reply audio
```

`"mock"` answers with a tone, about 60 ms per character, so programs and tests have real audio to
work with offline.

## A whole call

Most programs don't need the event loop. `talk` wires a microphone to the model and the model to a
speaker, clears the speaker the moment you interrupt, and returns the transcript when the call ends:

```tin
fn main() {
  let transcript = FrontDesk.talk(mic(), speaker())
  print(transcript)
}
```

 `mic()` and `speaker()` come from the host. In the [playground](https://tin.ad/docs/play/#ex=agents%2F14-call)
they are your browser's microphone and speakers (it asks first). In the terminal they are files or devices:

```tin
tin run call.tin --mic question.wav --speaker reply.wav     # recorded audio in, the reply written out
tin run call.tin                                           # the default microphone and speaker (needs sox)
```

## What the compiler checks

A voice agent is not a function you ask a question, so calling it like one is an error with a fix:

*called.tin · tin check called.tin*

```tin
voice agent Concierge {
  model: "mock"
}

fn main() {
  print(Concierge("what's the weather?"))
}
```

<!-- compiler output -->
```text
error: Concierge is a voice agent: it holds a live conversation, not one question
 --> called.tin:6:9
  |
6 |   print(Concierge("what's the weather?"))
  |         ^^^^^^^^^
  = note: open a session: `let s = Concierge.open()`, or run a whole call: `Concierge.talk(mic(), speaker())`
  = fix: open a session
```

Its model has to be a speech-to-speech model. A text model would accept the connection and then never speak:

*model.tin · tin check model.tin*

```tin
voice agent Concierge {
  model: "openai:gpt-5.1"
}

fn main() {}
```

<!-- compiler output -->
```text
error: 'openai:gpt-5.1' is not a speech-to-speech model
 --> model.tin:2:10
  |
2 |   model: "openai:gpt-5.1"
  |          ^^^^^^^^^^^^^^^^
  = note: a voice agent uses one of: openai-realtime:<model>, gemini-live:<model>, bedrock-sonic:<model>, or "mock"
```

The rules from the earlier chapters hold. A tool not in `tools:` never runs, a
`guarded tool fn` asks a person first, and what the model says may not reach JavaScript unapproved.
The microphone and the speaker are capabilities, so `tin caps` shows who can listen:

*listen.tin · tin caps listen.tin*

```tin
/// Listen to the room for a moment.
tool fn listen(): string => match mic().read() {
  Some(chunk) => "heard {chunk.seconds()} s"
  None => "silence"
}

/// The time at the desk.
tool fn deskTime(): string => "9:41"

voice agent FrontDesk { model: "mock", tools: [deskTime] }
voice agent Spy { model: "mock", tools: [listen] }

fn main() {}
```

<!-- output -->
```text
voice agent FrontDesk
  tools:           deskTime
  can call freely: deskTime
  needs approval:  nothing
  capabilities:    model

voice agent Spy
  tools:           listen
  can call freely: listen
  needs approval:  nothing
  capabilities:    model, mic

tools
  listen    free     mic
  deskTime  free     pure
```

## Real voices

Change `"mock"` to a speech-to-speech model and nothing else:

| `model:` | Needs | Runs in |
| --- | --- | --- |
| `"openai-realtime:gpt-realtime-mini"` | `OPENAI_API_KEY` | the CLI and the browser |
| `"gemini-live:gemini-live-2.5-flash-preview"` | `GEMINI_API_KEY` | the CLI and the browser |
| `"bedrock-sonic:amazon.nova-sonic-v1:0"` | access keys (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`) | the CLI only |

 A live connection does not last forever (Nova Sonic about 8 minutes, Gemini about 10, OpenAI an hour). Tin
reconnects shortly before the limit, waits for a pause in the conversation to do it, and keeps the conversation.
The [reference](https://tin.ad/docs/ref/bidi/#providers) says what has been tested against each provider.

## Tasks

### Task 1: A host who asks first

`book` is guarded. Print every tool result and whether it worked, then hang up after the spoken
answer. Run it without approval first: what does the model hear?

*task-1.tin · tin run task-1.tin*

```tin
/// Book a table for a number of people.
guarded tool fn book(people: int): string => "booked for {people}"

voice agent Host {
  model: "mock"
  tools: [book]
}

fn main() {
  let s = Host.open()
  s.say("book a table for 4")
  // print each tool result and whether it worked, then hang up after the spoken answer
}
```

Your answer   Checking needs JavaScript; the solution below works without it.

**Solution

*task-1.solution.tin · tin run task-1.solution.tin*

```tin
/// Book a table for a number of people.
guarded tool fn book(people: int): string => "booked for {people}"

voice agent Host {
  model: "mock"
  tools: [book]
}

fn main() {
  let s = Host.open()
  s.say("book a table for 4")
  var answered = false
  loop {
    match s.next() {
      ToolReturned { name, output, ok } => print("{name} ok={ok}: {output}")
      Said(text) => {
        answered = true
      }
      TurnDone => if answered { s.stop() }
      HungUp(reason) => break
      _ => {}
    }
  }
}
```

<!-- output -->
```text
book ok=false: Tool call denied by the user. Do not retry it; continue without it.
```

## Recap

- A voice agent keeps one speech-to-speech session open.
- Events are typed, and `match` must handle each kind.
- The microphone is a capability the compiler reports.

In the reference: [Talking agents](https://tin.ad/docs/ref/bidi/) · [VoiceEvent](https://tin.ad/docs/ref/bidi/#events) · [Speech-to-speech models](https://tin.ad/docs/ref/bidi/#providers)
