<!-- Telemetry · Tin · https://tin.ad/docs/ref/telemetry/ · generated from the built page -->

[Reference](https://tin.ad/docs/ref/)

-  [The language](https://tin.ad/docs/ref/language/)
-  [Agents and tools](https://tin.ad/docs/ref/agents/)
-  [Model providers](https://tin.ad/docs/ref/providers/)
-  [The Anthropic API](https://tin.ad/docs/ref/anthropic/)
-  [Talking agents](https://tin.ad/docs/ref/bidi/)
-  [Standard library](https://tin.ad/docs/ref/stdlib/)
-  [Glossary](https://tin.ad/docs/ref/glossary/)
-  [Package docs](https://tin.ad/docs/ref/api/)
-  [Telemetry](https://tin.ad/docs/ref/telemetry/)
- [Turning it on](#turn-on)
- [The spans](#spans)
- [Metrics](#metrics)
- [Trace context](#context)
- [Your own spans](#user)
- [The telemetry field](#field)
- [Content and secrets](#content)
- [Hosts](#hosts)
- [tin.ad](#tinad)
- [Environment variables](#env)
- [Not supported](#limits)
-  [Robots](https://tin.ad/docs/ref/robot/)
-  [Servers](https://tin.ad/docs/ref/server/)
-  [Deploying](https://tin.ad/docs/ref/deploy/)
-  [tin.ad](https://tin.ad/docs/ref/cloud/)
-  [The tin command](https://tin.ad/docs/ref/cli/)
-  [How it works](https://tin.ad/docs/ref/internals/)

**Reference · Telemetry

- [The language](https://tin.ad/docs/ref/language/)
- [Agents and tools](https://tin.ad/docs/ref/agents/)
- [Model providers](https://tin.ad/docs/ref/providers/)
- [The Anthropic API](https://tin.ad/docs/ref/anthropic/)
- [Talking agents](https://tin.ad/docs/ref/bidi/)
- [Standard library](https://tin.ad/docs/ref/stdlib/)
- [Glossary](https://tin.ad/docs/ref/glossary/)
- [Package docs](https://tin.ad/docs/ref/api/)
- [Telemetry](https://tin.ad/docs/ref/telemetry/)
- [Robots](https://tin.ad/docs/ref/robot/)
- [Servers](https://tin.ad/docs/ref/server/)
- [Deploying](https://tin.ad/docs/ref/deploy/)
- [tin.ad](https://tin.ad/docs/ref/cloud/)
- [The tin command](https://tin.ad/docs/ref/cli/)
- [How it works](https://tin.ad/docs/ref/internals/)

On this page

- [Turning it on](#turn-on)
- [The spans](#spans)
- [Metrics](#metrics)
- [Trace context](#context)
- [Your own spans](#user)
- [The telemetry field](#field)
- [Content and secrets](#content)
- [Hosts](#hosts)
- [tin.ad](#tinad)
- [Environment variables](#env)
- [Not supported](#limits)

# Telemetry

OpenTelemetry built in: every agent question, model call and tool call is a span by the GenAI conventions, with metrics and trace context, exported over OTLP from every host.

Tin has OpenTelemetry built in. Every agent question, model call and tool call becomes a span named and
attributed by the [OpenTelemetry GenAI
conventions](https://github.com/open-telemetry/semantic-conventions-genai), so any OTLP backend (Jaeger, Grafana Tempo, Honeycomb, Datadog, an OpenTelemetry Collector) shows
an agent's run as a tree. There is nothing to install and nothing to import. With no `OTEL_*` variable
set it is off and costs nothing: no span object is made and the agent loop runs the same code as before. The
tutorial chapter is [Seeing what an agent did](https://tin.ad/docs/learn/telemetry/).

## Turning it on

Point the standard variable at a collector, or print spans to stderr:

<!-- output -->
```text
$ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 tin run app.tin
$ OTEL_TRACES_EXPORTER=console tin run app.tin
```

 The OTLP exporter speaks OTLP/HTTP: protobuf by default (`http/protobuf`), JSON with
`OTEL_EXPORTER_OTLP_PROTOCOL=http/json`. Spans are batched (every
`OTEL_BSP_SCHEDULE_DELAY`, default 5 s, or 512 at a time) and the rest are sent when the program
exits; a collector that is down delays the exit by at most 5 s and is reported once on stderr. Checked against
the official `otelcol-contrib` 0.165.0 in both encodings.

## The spans

| When | Span | Kind | Main attributes |
| --- | --- | --- | --- |
| an agent is asked (`A(q)`, `A.ask`, `A.try`) | `invoke_agent A` | internal | `gen_ai.agent.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.output.type`, `gen_ai.conversation.id`, summed `gen_ai.usage.*`, `tin.agent.model_calls`, `tin.agent.tool_calls`, `error.type` |
| each model call | `chat <model>` | client | `gen_ai.request.max_tokens`, `gen_ai.request.stream`, `gen_ai.response.id`, `gen_ai.response.model`, `gen_ai.response.finish_reasons`, `gen_ai.usage.input_tokens` / `output_tokens` / `cache_read.input_tokens` / `cache_write.input_tokens` / `reasoning.output_tokens` |
| each tool call | `execute_tool name` | internal | `gen_ai.tool.name`, `gen_ai.tool.call.id`, `gen_ai.tool.type`, `gen_ai.tool.description`; a guarded tool gets a `tin.approve` event (allowed or denied) |
| a sub-agent offered as a tool | `execute_tool Sub` with `invoke_agent Sub` inside | internal | `gen_ai.transfer.mode` = `return_to_caller`, `gen_ai.transfer.target.name` |
| a request to a `server` | `GET /hello/:name` | server | `http.request.method`, `http.route`, `url.path`, `http.response.status_code` (5xx is an error); an event-stream reply ends its span when the stream ends |

 A failure (a provider error, a turn limit, a refusal) sets the span's status to error with
`error.type` and an `exception` event; a tool that returns an error marks only its own span.
Successful spans leave the status unset, as the trace API asks.

## Metrics

With an OTLP endpoint set, metrics go to `/v1/metrics` every
`OTEL_METRIC_EXPORT_INTERVAL` (60 s) and at exit, cumulative by default:
`gen_ai.client.operation.duration` and `gen_ai.client.token.usage` (by
`gen_ai.token.type`), the newer `gen_ai.client.inference.usage.*` token counters,
`gen_ai.invoke_agent.duration`, `gen_ai.invoke_agent.inference_calls`,
`gen_ai.invoke_agent.tool_calls` and `gen_ai.execute_tool.duration`, with the
conventions' bucket boundaries. `OTEL_METRICS_EXPORTER=none` turns them off.

## Trace context

W3C `traceparent` and `tracestate` travel both ways. A `server` request that
carries a `traceparent` continues the caller's trace. Outgoing `fetch()` and
`http()` calls and the model requests of the providers written in Tin carry the current span, so a
model gateway or your own service joins the same trace. `tin run` under WASI or native also reads
`TRACEPARENT` from the environment, as CI systems set it.

## Your own spans

`span(name) { ... }` runs a block inside a span and returns what the block returns; agent calls,
tool calls and `fetch()` inside become its children, and tasks spawned inside inherit it.
`spanAttr(key, value)` sets an attribute on the innermost one (a key written `"#name"`
is an integer). A panic inside ends the span as an error. Without telemetry both just run the code.

```tin
fn total(orders: int[]): int {
  span("total") {
    spanAttr("#orders", "{orders.len}")
    var sum = 0
    for o in orders { sum += o }
    sum
  }
}

fn main() {
  let t = span("load", () => total([3, 4, 5]))
  print("total {t}")
}
```

<!-- output -->
```text
total 12
```

## The telemetry field

Agents and servers take a `telemetry:` field. On an agent: attributes for its spans (any string
expression, `env("X")` included), the share of its new traces kept (`sample`, 0 to 1; a
caller's decision always wins), and whether its prompts and replies are recorded (`content`,
overriding the environment). On a server: constant attributes and `sample` for its request spans.
Providers never see this field, and tapes don't record it.

```tin
agent Support {
  model: "mock"
  telemetry: { attributes: { team: "support", region: "eu" }, sample: 0.25, content: false }
}

server Api grants [net.listen, model] {
  telemetry: { attributes: { "deployment.environment.name": "prod" }, sample: 1 }
  post "/ask" (req) { Support(req.body) }
}
```

## Content and secrets

Prompts, replies, system prompts, tool definitions, tool arguments and results are **not**
recorded unless `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true` (or an agent's
`content: true`). When they are, they use the conventions' message shape
(`gen_ai.input.messages`, `gen_ai.output.messages`, ...) and the text the model wrote is
listed in `tin.tainted`, the same taint the compiler tracks. Secrets are never exported: before a span
leaves, every value of a secret-looking variable (names with KEY, TOKEN, SECRET, PASSWORD, AUTH, ... and the OTLP
headers) that the host holds or the program read with `env()` is replaced with
`[redacted]`.

## Hosts

| Host | Traces | Metrics | Notes |
| --- | --- | --- | --- |
| `tin run`, `tin serve` (Node) | OTLP protobuf / JSON, console | yes | everything on this page |
| the browser (playground) | OTLP JSON over fetch, console in the trace pane | yes (OTLP) | `?otel=console`, or an endpoint in `localStorage['tin.otel.endpoint']`; the collector must allow CORS |
| WASI 0.2 and native | OTLP JSON, console | no | the tracer is Tin code compiled into the program; OTLP is compiled in when the program reaches the network anyway; one span stack per program |
| tin.ad | to the console, and to your collector | no | see below |

## tin.ad

Every deployment's spans appear in the console at `api.tin.ad/console`. The collector must be
`https://` (a deployment may not reach plain http or private addresses). To send them to your own
backend as well, set the usual variables as secrets:

<!-- output -->
```text
$ tin secrets set OTEL_EXPORTER_OTLP_ENDPOINT --app my-app    # asks for the value: https://otlp.example.com
$ tin secrets set OTEL_EXPORTER_OTLP_HEADERS --app my-app     # authorization=Bearer%20...
```

 Each request continues the platform's `traceparent`, and the secrets themselves are redacted from
what is exported.

## Environment variables

`OTEL_SDK_DISABLED`, `OTEL_TRACES_EXPORTER` (otlp, console, none),
`OTEL_METRICS_EXPORTER`, `OTEL_EXPORTER_OTLP_ENDPOINT`,
`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` / `_METRICS_ENDPOINT` (used as given),
`OTEL_EXPORTER_OTLP_[TRACES_|METRICS_]HEADERS`, `_PROTOCOL`, `_TIMEOUT`,
`OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE`, `OTEL_SERVICE_NAME` (default: the
program's file name; WASI and native: `unknown_service:tin`), `OTEL_RESOURCE_ATTRIBUTES`,
`OTEL_TRACES_SAMPLER` (always_on, always_off, traceidratio, parentbased_*) and
`_SAMPLER_ARG`, `OTEL_BSP_SCHEDULE_DELAY`, `OTEL_BSP_MAX_EXPORT_BATCH_SIZE`,
`OTEL_BSP_MAX_QUEUE_SIZE`, `OTEL_METRIC_EXPORT_INTERVAL`,
`OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT`.

## Not supported

- Logs (the OTLP logs signal) and exemplars.
- WASI and native: no metrics, no server request span, no content capture, OTLP as JSON only, and model
requests do not carry `traceparent` yet. Tasks that interleave share one span stack there.

- The five providers still written in TypeScript (Mistral, llama.cpp, LiteLLM, Writer, Vercel) do not send
`traceparent`.

- Voice agents make no spans yet.
