Tin
Reference · Telemetry

On this page

Telemetry

OpenTelemetry built in: every agent question, model call and tool call is a span by the GenAI conventions, with metrics and trace context, exported over OTLP from every host.

Tin has OpenTelemetry built in. Every agent question, model call and tool call becomes a span named and attributed by the OpenTelemetry GenAI conventions, so any OTLP backend (Jaeger, Grafana Tempo, Honeycomb, Datadog, an OpenTelemetry Collector) shows an agent's run as a tree. There is nothing to install and nothing to import. With no OTEL_* variable set it is off and costs nothing: no span object is made and the agent loop runs the same code as before. The tutorial chapter is Seeing what an agent did.

Turning it on

Point the standard variable at a collector, or print spans to stderr:

$ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 tin run app.tin
$ OTEL_TRACES_EXPORTER=console tin run app.tin

The OTLP exporter speaks OTLP/HTTP: protobuf by default (http/protobuf), JSON with OTEL_EXPORTER_OTLP_PROTOCOL=http/json. Spans are batched (every OTEL_BSP_SCHEDULE_DELAY, default 5 s, or 512 at a time) and the rest are sent when the program exits; a collector that is down delays the exit by at most 5 s and is reported once on stderr. Checked against the official otelcol-contrib 0.165.0 in both encodings.

The spans

WhenSpanKindMain attributes
an agent is asked (A(q), A.ask, A.try)invoke_agent Ainternalgen_ai.agent.name, gen_ai.provider.name, gen_ai.request.model, gen_ai.output.type, gen_ai.conversation.id, summed gen_ai.usage.*, tin.agent.model_calls, tin.agent.tool_calls, error.type
each model callchat <model>clientgen_ai.request.max_tokens, gen_ai.request.stream, gen_ai.response.id, gen_ai.response.model, gen_ai.response.finish_reasons, gen_ai.usage.input_tokens / output_tokens / cache_read.input_tokens / cache_write.input_tokens / reasoning.output_tokens
each tool callexecute_tool nameinternalgen_ai.tool.name, gen_ai.tool.call.id, gen_ai.tool.type, gen_ai.tool.description; a guarded tool gets a tin.approve event (allowed or denied)
a sub-agent offered as a toolexecute_tool Sub with invoke_agent Sub insideinternalgen_ai.transfer.mode = return_to_caller, gen_ai.transfer.target.name
a request to a serverGET /hello/:nameserverhttp.request.method, http.route, url.path, http.response.status_code (5xx is an error); an event-stream reply ends its span when the stream ends

A failure (a provider error, a turn limit, a refusal) sets the span's status to error with error.type and an exception event; a tool that returns an error marks only its own span. Successful spans leave the status unset, as the trace API asks.

Metrics

With an OTLP endpoint set, metrics go to /v1/metrics every OTEL_METRIC_EXPORT_INTERVAL (60 s) and at exit, cumulative by default: gen_ai.client.operation.duration and gen_ai.client.token.usage (by gen_ai.token.type), the newer gen_ai.client.inference.usage.* token counters, gen_ai.invoke_agent.duration, gen_ai.invoke_agent.inference_calls, gen_ai.invoke_agent.tool_calls and gen_ai.execute_tool.duration, with the conventions' bucket boundaries. OTEL_METRICS_EXPORTER=none turns them off.

Trace context

W3C traceparent and tracestate travel both ways. A server request that carries a traceparent continues the caller's trace. Outgoing fetch() and http() calls and the model requests of the providers written in Tin carry the current span, so a model gateway or your own service joins the same trace. tin run under WASI or native also reads TRACEPARENT from the environment, as CI systems set it.

Your own spans

span(name) { ... } runs a block inside a span and returns what the block returns; agent calls, tool calls and fetch() inside become its children, and tasks spawned inside inherit it. spanAttr(key, value) sets an attribute on the innermost one (a key written "#name" is an integer). A panic inside ends the span as an error. Without telemetry both just run the code.

fn total(orders: int[]): int {
  span("total") {
    spanAttr("#orders", "{orders.len}")
    var sum = 0
    for o in orders { sum += o }
    sum
  }
}

fn main() {
  let t = span("load", () => total([3, 4, 5]))
  print("total {t}")
}
total 12

The telemetry field

Agents and servers take a telemetry: field. On an agent: attributes for its spans (any string expression, env("X") included), the share of its new traces kept (sample, 0 to 1; a caller's decision always wins), and whether its prompts and replies are recorded (content, overriding the environment). On a server: constant attributes and sample for its request spans. Providers never see this field, and tapes don't record it.

agent Support {
  model: "mock"
  telemetry: { attributes: { team: "support", region: "eu" }, sample: 0.25, content: false }
}

server Api grants [net.listen, model] {
  telemetry: { attributes: { "deployment.environment.name": "prod" }, sample: 1 }
  post "/ask" (req) { Support(req.body) }
}

Content and secrets

Prompts, replies, system prompts, tool definitions, tool arguments and results are not recorded unless OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true (or an agent's content: true). When they are, they use the conventions' message shape (gen_ai.input.messages, gen_ai.output.messages, ...) and the text the model wrote is listed in tin.tainted, the same taint the compiler tracks. Secrets are never exported: before a span leaves, every value of a secret-looking variable (names with KEY, TOKEN, SECRET, PASSWORD, AUTH, ... and the OTLP headers) that the host holds or the program read with env() is replaced with [redacted].

Hosts

HostTracesMetricsNotes
tin run, tin serve (Node)OTLP protobuf / JSON, consoleyeseverything on this page
the browser (playground)OTLP JSON over fetch, console in the trace paneyes (OTLP)?otel=console, or an endpoint in localStorage['tin.otel.endpoint']; the collector must allow CORS
WASI 0.2 and nativeOTLP JSON, consolenothe tracer is Tin code compiled into the program; OTLP is compiled in when the program reaches the network anyway; one span stack per program
tin.adto the console, and to your collectornosee below

tin.ad

Every deployment's spans appear in the console at api.tin.ad/console. The collector must be https:// (a deployment may not reach plain http or private addresses). To send them to your own backend as well, set the usual variables as secrets:

$ tin secrets set OTEL_EXPORTER_OTLP_ENDPOINT --app my-app    # asks for the value: https://otlp.example.com
$ tin secrets set OTEL_EXPORTER_OTLP_HEADERS --app my-app     # authorization=Bearer%20...

Each request continues the platform's traceparent, and the secrets themselves are redacted from what is exported.

Environment variables

OTEL_SDK_DISABLED, OTEL_TRACES_EXPORTER (otlp, console, none), OTEL_METRICS_EXPORTER, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_TRACES_ENDPOINT / _METRICS_ENDPOINT (used as given), OTEL_EXPORTER_OTLP_[TRACES_|METRICS_]HEADERS, _PROTOCOL, _TIMEOUT, OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE, OTEL_SERVICE_NAME (default: the program's file name; WASI and native: unknown_service:tin), OTEL_RESOURCE_ATTRIBUTES, OTEL_TRACES_SAMPLER (always_on, always_off, traceidratio, parentbased_*) and _SAMPLER_ARG, OTEL_BSP_SCHEDULE_DELAY, OTEL_BSP_MAX_EXPORT_BATCH_SIZE, OTEL_BSP_MAX_QUEUE_SIZE, OTEL_METRIC_EXPORT_INTERVAL, OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT.

Not supported

  • Logs (the OTLP logs signal) and exemplars.
  • WASI and native: no metrics, no server request span, no content capture, OTLP as JSON only, and model requests do not carry traceparent yet. Tasks that interleave share one span stack there.
  • The five providers still written in TypeScript (Mistral, llama.cpp, LiteLLM, Writer, Vercel) do not send traceparent.
  • Voice agents make no spans yet.