<!-- Policies and the control loop · Tin · https://tin.ad/docs/learn/policies/ · generated from the built page -->

[Learn](https://tin.ad/docs/learn/) · Robots · 33 of 34

# Policies and the control loop

**Goal.** Run a policy in a control loop, read what the run did, and replay it exactly.

A policy decides the next action from what the robot sees. The loop around it, how often it runs, what
happens when it has nothing to say, how a run ends, is the same for every policy, so Tin writes it once.

## Run a policy

`arm.run(wave, 5s, 50Hz)` ticks round(seconds &times; hz) times: observe, ask the policy, send its
action, let one period pass. In the built-in world the clock is virtual, so a five-second run takes no wall time
and gives the same numbers every time:

*wave.tin · tin run wave.tin*

```tin
robot Arm = so101

fn wave(o: Observation): Option<Pose> => Some(Pose { shoulder_pan: 0.5rad * sin(o.t / 1s * 2.0) })

fn main() {
  var arm = Arm()
  let r = arm.run(wave, 5s, 50Hz)
  print("{r.steps} steps, {r.holds} holds, stopped by {r.stoppedBy}")
}
```

<!-- output -->
```text
250 steps, 0 holds, stopped by steps
```

The duration and the rate are different units, so the arguments cannot be swapped.

## Policy forms

`policy(f)` wraps any function from an `Observation` to `Option<Pose>`.
`sequence` ramps through waypoints, `chunked` plays a list of actions one per tick, and
`perturb` adds noise fixed by a seed. A policy that returns `None` holds: nothing is sent,
and the run counts it, so a slow decision is never mistaken for a missed tick:

*forms.tin · tin run forms.tin*

```tin
robot Arm = so101

fn main() {
  var arm = Arm()
  let pick = sequence([
    step(Pose { shoulder_lift: 20deg, elbow_flex: 40deg }, 1s),
    step(Pose { gripper: 0.8rad }, 300ms),
    step(Pose { gripper: 0rad }, 300ms),
  ])
  print(arm.run(pick, 2s, 50Hz).final.at("elbow_flex").deg().fixed(1))

  // None is a hold: nothing is written, and the run counts it
  let wait = policy(o => if o.t < 3s { None } else { Some(Pose { gripper: 0.2rad }) })
  let r = arm.run(wait, 1s, 10Hz)
  print("{r.steps} steps, {r.holds} holds")

  let shaky = perturb(policy(o => Some(Pose { wrist_roll: 0rad })), 7, 2deg)
  print(arm.run(shaky, 1s, 100Hz).final.at("wrist_roll").deg().fixed(3))
}
```

<!-- output -->
```text
39.2
10 steps, 10 holds
-0.203
```

## Tapes

Every run keeps a tape: each tick's action and where every joint was after it. `replay(tape)` turns
it back into a policy, and `verify` replays it and says where the joints first differ, so a change to
a controller can be checked against yesterday's run exactly. Tapes save to JSON with `tape.save`.

*tape.tin · tin run tape.tin*

```tin
robot Arm = so101

fn main() {
  var arm = Arm()
  let r = arm.run(perturb(policy(o => Some(Pose { shoulder_pan: 1rad })), 3, 0.1rad), 1s, 20Hz)
  print("{r.tape.q.len} ticks recorded")

  var again = Arm()
  print(again.verify(r.tape))

  var moved = Arm()
  moved.act(Pose { elbow_flex: 0.1rad })
  print(moved.verify(r.tape))
}
```

<!-- output -->
```text
20 ticks recorded
None
Some("the replay differs from the tape at start: elbow_flex 0.03934693402873665 vs 0.0")
```

## Tasks

### Task 1: A nod that stops early

Make a three-waypoint `sequence` that tips the wrist to 30&deg;, to &minus;30&deg; and back to
0&deg;, half a second each. Run it for up to 3 s at 50 Hz with `runUntil`, stopping once the robot's
clock passes 1.2 s, and print the steps and why it stopped.

Your answer   Checking needs JavaScript; the solution below works without it.

**Solution

*task-1.solution.tin · tin run task-1.solution.tin*

```tin
robot Arm = so101

fn main() {
  var arm = Arm()
  let nod = sequence([
    step(Pose { wrist_flex: 30deg }, 500ms),
    step(Pose { wrist_flex: -30deg }, 500ms),
    step(Pose { wrist_flex: 0deg }, 500ms),
  ])
  let r = arm.runUntil(nod, 3s, 50Hz, o => o.t > 1200ms)
  print("{r.steps} steps, stopped by {r.stoppedBy}")
}
```

<!-- output -->
```text
61 steps, stopped by stopWhen
```

## Recap

- `arm.run(p, 5s, 50Hz)` runs round(seconds × hz) ticks on a virtual clock.
- A policy that returns None holds: counted, never a miss.
- Every run has a tape; `verify` checks a replay lands on the same joints.

In the reference: [arm.run](https://tin.ad/docs/ref/robot/#run) · [Policy forms](https://tin.ad/docs/ref/robot/#policies) · [Tapes](https://tin.ad/docs/ref/robot/#tapes)
