Tin

Learn · Real programs · 31 of 34

Seeing what an agent did

Goal. See each agent question, model call and tool call as an OpenTelemetry trace, and add spans of your own.

An agent that works in a test can still be slow, expensive or wrong in production. To see why, you need a record of what it did: which model it asked, how many tokens each turn took, which tools it called and what failed. Tin writes that record as OpenTelemetry traces, the format every observability tool reads, without a library or a line of setup code.

One variable

Nothing is recorded until an OTEL_* variable asks for it. Then every agent question becomes a tree: invoke_agent for the question, a chat span for each model call, an execute_tool span for each tool call. Print it to see it:

$ OTEL_TRACES_EXPORTER=console tin run spans.tin
[otel] 3ae43db1 d56dd9230fab912c ← 3bd3868e3cca87bd chat mock 0.6ms gen_ai.operation.name="chat" ...
[otel] 3ae43db1 54cb3a7c2586eca8 ← 3bd3868e3cca87bd execute_tool add 0.4ms gen_ai.tool.name="add" ...
[otel] 3ae43db1 3bd3868e3cca87bd ← 851a6f8016e32c5c invoke_agent Calc 2.1ms team="billing" ...

Each line is a span: its trace, its own id, its parent, its name, how long it took and its attributes. Set OTEL_EXPORTER_OTLP_ENDPOINT instead and the same spans go to Jaeger, Grafana or any OTLP collector.

Spans of your own

span("name") { ... } wraps a block in a span and gives back the block's value. The agent call inside becomes its child, so a slow request shows which step was slow. spanAttr labels the innermost span; a key that starts with # is a number. The program prints the same thing with telemetry on or off:

spans.tintin run spans.tin
/// Add two integers.
tool fn add(a: int, b: int): int => a + b

agent Calc {
  model: "mock"
  tools: [add]
  telemetry: { attributes: { team: "billing" } }
}

fn total(orders: int[]): int {
  span("total") {
    spanAttr("#orders", "{orders.len}")
    var sum = 0
    for o in orders { sum += o }
    sum
  }
}

fn main() {
  let t = span("load", () => total([3, 4, 5]))
  print("total {t}")
  span("ask") {
    print(Calc("add 2 and 3"))
  }
}
total 12
mock: 5

The telemetry: field on Calc puts team="billing" on every span the agent makes. It can also keep only a share of traces (sample: 0.1) and turn content capture on or off for that agent alone.

What is not recorded

Prompts and replies stay out of traces unless you set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true: they can hold personal data. API keys and other secrets are never exported, even then: their values are replaced with [redacted] before a span leaves the program.

Tasks

Task 1: Trace a checkout

Write a Shop agent with a price tool. Ask it for the price of tea inside a span named checkout that records the customer's name, and give the agent the attribute shop="corner". Run it with OTEL_TRACES_EXPORTER=console and find the execute_tool price span under checkout.

Checking needs JavaScript; the solution below works without it.

Solution
task-1.solution.tintin run task-1.solution.tin
/// Look up a price.
tool fn price(item: string): int => if item == "tea" { 3 } else { 5 }

agent Shop {
  model: "mock"
  tools: [price]
  telemetry: { attributes: { shop: "corner" }, sample: 1 }
}

fn main() {
  span("checkout") {
    spanAttr("customer", "ada")
    print(Shop("price of 'tea'"))
  }
}
mock: 3

Recap

  • One OTEL_* variable turns tracing on; without it nothing is recorded.
  • span("name") { ... } nests the agent calls inside it.
  • Prompts are recorded only when you opt in, and secrets never.

In the reference: Telemetry · The spans · Content and secrets