Tin

Learn · Robots · 33 of 34

Policies and the control loop

Goal. Run a policy in a control loop, read what the run did, and replay it exactly.

A policy decides the next action from what the robot sees. The loop around it, how often it runs, what happens when it has nothing to say, how a run ends, is the same for every policy, so Tin writes it once.

Run a policy

arm.run(wave, 5s, 50Hz) ticks round(seconds × hz) times: observe, ask the policy, send its action, let one period pass. In the built-in world the clock is virtual, so a five-second run takes no wall time and gives the same numbers every time:

wave.tintin run wave.tin
robot Arm = so101

fn wave(o: Observation): Option<Pose> => Some(Pose { shoulder_pan: 0.5rad * sin(o.t / 1s * 2.0) })

fn main() {
  var arm = Arm()
  let r = arm.run(wave, 5s, 50Hz)
  print("{r.steps} steps, {r.holds} holds, stopped by {r.stoppedBy}")
}
250 steps, 0 holds, stopped by steps

The duration and the rate are different units, so the arguments cannot be swapped.

Policy forms

policy(f) wraps any function from an Observation to Option<Pose>. sequence ramps through waypoints, chunked plays a list of actions one per tick, and perturb adds noise fixed by a seed. A policy that returns None holds: nothing is sent, and the run counts it, so a slow decision is never mistaken for a missed tick:

forms.tintin run forms.tin
robot Arm = so101

fn main() {
  var arm = Arm()
  let pick = sequence([
    step(Pose { shoulder_lift: 20deg, elbow_flex: 40deg }, 1s),
    step(Pose { gripper: 0.8rad }, 300ms),
    step(Pose { gripper: 0rad }, 300ms),
  ])
  print(arm.run(pick, 2s, 50Hz).final.at("elbow_flex").deg().fixed(1))

  // None is a hold: nothing is written, and the run counts it
  let wait = policy(o => if o.t < 3s { None } else { Some(Pose { gripper: 0.2rad }) })
  let r = arm.run(wait, 1s, 10Hz)
  print("{r.steps} steps, {r.holds} holds")

  let shaky = perturb(policy(o => Some(Pose { wrist_roll: 0rad })), 7, 2deg)
  print(arm.run(shaky, 1s, 100Hz).final.at("wrist_roll").deg().fixed(3))
}
39.2
10 steps, 10 holds
-0.203

Tapes

Every run keeps a tape: each tick's action and where every joint was after it. replay(tape) turns it back into a policy, and verify replays it and says where the joints first differ, so a change to a controller can be checked against yesterday's run exactly. Tapes save to JSON with tape.save.

tape.tintin run tape.tin
robot Arm = so101

fn main() {
  var arm = Arm()
  let r = arm.run(perturb(policy(o => Some(Pose { shoulder_pan: 1rad })), 3, 0.1rad), 1s, 20Hz)
  print("{r.tape.q.len} ticks recorded")

  var again = Arm()
  print(again.verify(r.tape))

  var moved = Arm()
  moved.act(Pose { elbow_flex: 0.1rad })
  print(moved.verify(r.tape))
}
20 ticks recorded
None
Some("the replay differs from the tape at start: elbow_flex 0.03934693402873665 vs 0.0")

Tasks

Task 1: A nod that stops early

Make a three-waypoint sequence that tips the wrist to 30°, to −30° and back to 0°, half a second each. Run it for up to 3 s at 50 Hz with runUntil, stopping once the robot's clock passes 1.2 s, and print the steps and why it stopped.

Checking needs JavaScript; the solution below works without it.

Solution
task-1.solution.tintin run task-1.solution.tin
robot Arm = so101

fn main() {
  var arm = Arm()
  let nod = sequence([
    step(Pose { wrist_flex: 30deg }, 500ms),
    step(Pose { wrist_flex: -30deg }, 500ms),
    step(Pose { wrist_flex: 0deg }, 500ms),
  ])
  let r = arm.runUntil(nod, 3s, 50Hz, o => o.t > 1200ms)
  print("{r.steps} steps, stopped by {r.stoppedBy}")
}
61 steps, stopped by stopWhen

Recap

  • arm.run(p, 5s, 50Hz) runs round(seconds × hz) ticks on a virtual clock.
  • A policy that returns None holds: counted, never a miss.
  • Every run has a tape; verify checks a replay lands on the same joints.

In the reference: arm.run · Policy forms · Tapes