Reference · Telemetry
- The language
- Agents and tools
- Model providers
- The Anthropic API
- Talking agents
- Standard library
- Glossary
- Package docs
- Telemetry
- Robots
- Servers
- Deploying
- tin.ad
- The tin command
- How it works
On this page
Telemetry
OpenTelemetry built in: every agent question, model call and tool call is a span by the GenAI conventions, with metrics and trace context, exported over OTLP from every host.
Tin has OpenTelemetry built in. Every agent question, model call and tool call becomes a span named and
attributed by the OpenTelemetry GenAI
conventions, so any OTLP backend (Jaeger, Grafana Tempo, Honeycomb, Datadog, an OpenTelemetry Collector) shows
an agent's run as a tree. There is nothing to install and nothing to import. With no OTEL_* variable
set it is off and costs nothing: no span object is made and the agent loop runs the same code as before. The
tutorial chapter is Seeing what an agent did.
Turning it on
Point the standard variable at a collector, or print spans to stderr:
$ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 tin run app.tin
$ OTEL_TRACES_EXPORTER=console tin run app.tin The OTLP exporter speaks OTLP/HTTP: protobuf by default (http/protobuf), JSON with
OTEL_EXPORTER_OTLP_PROTOCOL=http/json. Spans are batched (every
OTEL_BSP_SCHEDULE_DELAY, default 5 s, or 512 at a time) and the rest are sent when the program
exits; a collector that is down delays the exit by at most 5 s and is reported once on stderr. Checked against
the official otelcol-contrib 0.165.0 in both encodings.
The spans
| When | Span | Kind | Main attributes |
|---|---|---|---|
an agent is asked (A(q), A.ask, A.try) | invoke_agent A | internal | gen_ai.agent.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.output.type, gen_ai.conversation.id, summed gen_ai.usage.*, tin.agent.model_calls, tin.agent.tool_calls, error.type |
| each model call | chat <model> | client | gen_ai.request.max_tokens, gen_ai.request.stream, gen_ai.response.id, gen_ai.response.model, gen_ai.response.finish_reasons, gen_ai.usage.input_tokens / output_tokens / cache_read.input_tokens / cache_write.input_tokens / reasoning.output_tokens |
| each tool call | execute_tool name | internal | gen_ai.tool.name, gen_ai.tool.call.id, gen_ai.tool.type, gen_ai.tool.description; a guarded tool gets a tin.approve event (allowed or denied) |
| a sub-agent offered as a tool | execute_tool Sub with invoke_agent Sub inside | internal | gen_ai.transfer.mode = return_to_caller, gen_ai.transfer.target.name |
a request to a server | GET /hello/:name | server | http.request.method, http.route, url.path, http.response.status_code (5xx is an error); an event-stream reply ends its span when the stream ends |
A failure (a provider error, a turn limit, a refusal) sets the span's status to error with
error.type and an exception event; a tool that returns an error marks only its own span.
Successful spans leave the status unset, as the trace API asks.
Metrics
With an OTLP endpoint set, metrics go to /v1/metrics every
OTEL_METRIC_EXPORT_INTERVAL (60 s) and at exit, cumulative by default:
gen_ai.client.operation.duration and gen_ai.client.token.usage (by
gen_ai.token.type), the newer gen_ai.client.inference.usage.* token counters,
gen_ai.invoke_agent.duration, gen_ai.invoke_agent.inference_calls,
gen_ai.invoke_agent.tool_calls and gen_ai.execute_tool.duration, with the
conventions' bucket boundaries. OTEL_METRICS_EXPORTER=none turns them off.
Trace context
W3C traceparent and tracestate travel both ways. A server request that
carries a traceparent continues the caller's trace. Outgoing fetch() and
http() calls and the model requests of the providers written in Tin carry the current span, so a
model gateway or your own service joins the same trace. tin run under WASI or native also reads
TRACEPARENT from the environment, as CI systems set it.
Your own spans
span(name) { ... } runs a block inside a span and returns what the block returns; agent calls,
tool calls and fetch() inside become its children, and tasks spawned inside inherit it.
spanAttr(key, value) sets an attribute on the innermost one (a key written "#name"
is an integer). A panic inside ends the span as an error. Without telemetry both just run the code.
fn total(orders: int[]): int {
span("total") {
spanAttr("#orders", "{orders.len}")
var sum = 0
for o in orders { sum += o }
sum
}
}
fn main() {
let t = span("load", () => total([3, 4, 5]))
print("total {t}")
} total 12 The telemetry field
Agents and servers take a telemetry: field. On an agent: attributes for its spans (any string
expression, env("X") included), the share of its new traces kept (sample, 0 to 1; a
caller's decision always wins), and whether its prompts and replies are recorded (content,
overriding the environment). On a server: constant attributes and sample for its request spans.
Providers never see this field, and tapes don't record it.
agent Support {
model: "mock"
telemetry: { attributes: { team: "support", region: "eu" }, sample: 0.25, content: false }
}
server Api grants [net.listen, model] {
telemetry: { attributes: { "deployment.environment.name": "prod" }, sample: 1 }
post "/ask" (req) { Support(req.body) }
} Content and secrets
Prompts, replies, system prompts, tool definitions, tool arguments and results are not
recorded unless OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true (or an agent's
content: true). When they are, they use the conventions' message shape
(gen_ai.input.messages, gen_ai.output.messages, ...) and the text the model wrote is
listed in tin.tainted, the same taint the compiler tracks. Secrets are never exported: before a span
leaves, every value of a secret-looking variable (names with KEY, TOKEN, SECRET, PASSWORD, AUTH, ... and the OTLP
headers) that the host holds or the program read with env() is replaced with
[redacted].
Hosts
| Host | Traces | Metrics | Notes |
|---|---|---|---|
tin run, tin serve (Node) | OTLP protobuf / JSON, console | yes | everything on this page |
| the browser (playground) | OTLP JSON over fetch, console in the trace pane | yes (OTLP) | ?otel=console, or an endpoint in localStorage['tin.otel.endpoint']; the collector must allow CORS |
| WASI 0.2 and native | OTLP JSON, console | no | the tracer is Tin code compiled into the program; OTLP is compiled in when the program reaches the network anyway; one span stack per program |
| tin.ad | to the console, and to your collector | no | see below |
tin.ad
Every deployment's spans appear in the console at api.tin.ad/console. The collector must be
https:// (a deployment may not reach plain http or private addresses). To send them to your own
backend as well, set the usual variables as secrets:
$ tin secrets set OTEL_EXPORTER_OTLP_ENDPOINT --app my-app # asks for the value: https://otlp.example.com
$ tin secrets set OTEL_EXPORTER_OTLP_HEADERS --app my-app # authorization=Bearer%20... Each request continues the platform's traceparent, and the secrets themselves are redacted from
what is exported.
Environment variables
OTEL_SDK_DISABLED, OTEL_TRACES_EXPORTER (otlp, console, none),
OTEL_METRICS_EXPORTER, OTEL_EXPORTER_OTLP_ENDPOINT,
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT / _METRICS_ENDPOINT (used as given),
OTEL_EXPORTER_OTLP_[TRACES_|METRICS_]HEADERS, _PROTOCOL, _TIMEOUT,
OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE, OTEL_SERVICE_NAME (default: the
program's file name; WASI and native: unknown_service:tin), OTEL_RESOURCE_ATTRIBUTES,
OTEL_TRACES_SAMPLER (always_on, always_off, traceidratio, parentbased_*) and
_SAMPLER_ARG, OTEL_BSP_SCHEDULE_DELAY, OTEL_BSP_MAX_EXPORT_BATCH_SIZE,
OTEL_BSP_MAX_QUEUE_SIZE, OTEL_METRIC_EXPORT_INTERVAL,
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT.
Not supported
- Logs (the OTLP logs signal) and exemplars.
- WASI and native: no metrics, no server request span, no content capture, OTLP as JSON only, and model
requests do not carry
traceparentyet. Tasks that interleave share one span stack there. - The five providers still written in TypeScript (Mistral, llama.cpp, LiteLLM, Writer, Vercel) do not send
traceparent. - Voice agents make no spans yet.