Learn · Agents · 18 of 34
Guarded tools and approval
Goal. Mark a dangerous tool guarded so a person approves every call, and let the compiler prove there is no way around it.
Some tools shouldn't run because a model decided they should: deleting files, sending money, moving a robot
arm. Mark them guarded, and a person approves every call.
A guarded tool
When the model asks for a guarded tool, the agent loop pauses and someone is asked. Here no one is there to answer, so the call is denied. The tool doesn't run, and the model is told to carry on without it:
tin run denied.tin/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"
agent Janitor {
model: "mock"
tools: [deleteFile]
on toolResult(name, output, ok) => print(" {name} -> {output}")
}
fn main() {
print(Janitor("deleteFile 'old-backup.zip'"))
} deleteFile -> Tool call denied by the user. Do not retry it; continue without it.
mock: error: Tool call denied by the user. Do not retry it; continue without it. With tin run --yes, every call is approved:
tin run approved.tin --yes/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"
agent Janitor {
model: "mock"
tools: [deleteFile]
on toolResult(name, output, ok) => print(" {name} -> {output}")
}
fn main() {
print(Janitor("deleteFile 'old-backup.zip'"))
} deleteFile -> deleted old-backup.zip
mock: deleted old-backup.zip | Where the program runs | Who is asked |
|---|---|
tin run in a terminal | you, with a y/N prompt showing the tool and its arguments |
tin run --yes | nobody: everything is approved |
| no terminal (a pipe, CI) | nobody: everything is denied |
| the playground | you, in a browser dialog |
The compiler checks there's no way around it
A prompt at run time would be worth little if other code could just call the function. So the compiler proves it can't. Calling a guarded tool directly is an error:
tin check direct.tin/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"
fn main() {
print(deleteFile("old-backup.zip")) // skipping the person is not allowed
} error: 'deleteFile' is a guarded tool, and this call skips approval
--> direct.tin:5:9
|
5 | print(deleteFile("old-backup.zip")) // skipping the person is not allowed
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
= note: path: main → deleteFile
= note: give the tool to an agent (the agent loop asks before it runs), or call it from another guarded tool fn
= note: or ask a person at this call: `match approve deleteFile(...) { Ok(out) => ..., Err(why) => ... }` (approve gives a Result: Err when denied) And it follows calls all the way down. Here an ordinary tool reaches a guarded function two calls away. The error shows the whole path:
tin check sneaky.tin/// Move money. Cannot be undone.
guarded fn wire(to: string, cents: int): string => "sent {cents} to {to}"
fn payInvoice(id: int): string => wire("vendor-{id}", 100)
/// Pay an invoice by id.
tool fn pay(id: int): string => payInvoice(id) // an unguarded tool, two calls away from wire
agent Clerk { model: "mock", tools: [pay] }
fn main() {
print(Clerk("pay 7"))
} error: tool 'pay' can run 'wire', which is guarded, without anyone approving it
--> sneaky.tin:4:35
|
4 | fn payInvoice(id: int): string => wire("vendor-{id}", 100)
| ^^^^^^^^^^^^^^^^^^^^^^^^
= note: path: pay → payInvoice → wire
= note: mark it `guarded tool fn pay` so the agent loop asks first
= fix: mark pay as guarded The note is also a fix: tin fix marks pay as guarded for you. This check is exact,
not a guess, because tool parameters are plain data and Tin has no global variables: the only code a tool can
run is code its body names.
Asking yourself: approve
Your own code can call a guarded tool too, through approve. It shows a person the exact call and
returns a Result: Ok with the tool's result on a yes, and Err on a
no.
tin run approve.tin/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"
fn main() {
// approve shows a person the exact call, and runs it only on a yes
match approve deleteFile("draft.md") {
Ok(msg) => print(msg)
Err(why) => print("not done: {why}")
}
} not done: denied: deleteFile tin run approve-yes.tin --yes/// Permanently delete a file.
guarded tool fn deleteFile(path: string): string => "deleted {path}"
fn main() {
// approve shows a person the exact call, and runs it only on a yes
match approve deleteFile("draft.md") {
Ok(msg) => print(msg)
Err(why) => print("not done: {why}")
}
} deleted draft.md Tasks
Task 1: An admin agent
Give an agent a listUsers tool anyone can use and a removeUser tool that needs
approval. Print each call, and run it with --yes.
Solution
tin run task-1.solution.tin --yes/// List the users.
tool fn listUsers(): string[] => ["ada", "lin", "old-test-user"]
/// Remove a user account. Cannot be undone.
guarded tool fn removeUser(name: string): string => "removed {name}"
agent Admin {
model: "mock"
tools: [listUsers, removeUser]
on toolCall(name, input) => print(" call {name} {input}")
}
fn main() {
print(Admin("use listUsers"))
print(Admin("removeUser 'old-test-user'"))
} call listUsers {}
mock: ["ada","lin","old-test-user"]
call removeUser {"name":"old-test-user"}
mock: removed old-test-user Task 2: Make it compile
Fix the payment program from above the right way: keep wire guarded, and
make a person approve every payment.
Solution
tin run task-2.solution.tin/// Move money. Cannot be undone.
guarded fn wire(to: string, cents: int): string => "sent {cents} to {to}"
fn payInvoice(id: int): string => wire("vendor-{id}", 100)
/// Pay an invoice by id.
guarded tool fn pay(id: int): string => payInvoice(id) // now a person approves every payment
agent Clerk { model: "mock", tools: [pay] }
fn main() {
print(Clerk("pay 7"))
} mock: error: Tool call denied by the user. Do not retry it; continue without it. Marking pay as guarded tool fn is what the compiler suggested.
Removing guarded from wire would also compile, but then nobody would be asked.
Recap
- A
guarded tool fnasks a person before each call. - Any path that reaches it without an agent asking is a compile error.
approveasks the person yourself, in your own code.
In the reference: Guarded tools and approval · The compiler checks the approval path · approve