AI that moves things

From a token to a servo: the last ten centimetres

Everything between a model's output and something physically moving — the boundary, the failsafe, and why the interesting engineering is all on the hardware side of the API call.

2026-06-23 · 4 min read

A model that writes a bad paragraph has produced a bad paragraph. A model that drives a servo has moved something in a room where a person might be standing. The code either side of that boundary looks similar and is not.

Here is the architecture that keeps holding up, and the three rules underneath it.

The model never talks to the actuator

Between the two goes a layer with a small, closed vocabulary:

// The entire interface the model is allowed to produce.
const COMMANDS = {
  lamp:  { on: () => plug(true), off: () => plug(false) },
  arm:   { home: () => goto(0), pick: () => goto(90) },
};

function dispatch(intent) {
  const device = COMMANDS[intent.device];
  const action = device?.[intent.action];
  if (!action) return { ok: false, reason: "unknown command" };
  return action();
}

The model's job ends at producing {device, action}. It cannot express an angle, a duration, or a speed, because those are the parameters that turn a misunderstanding into damage. Every command it can emit has been thought about by a person in advance.

This feels restrictive right up until the first time the model returns something creative. Then it feels like the only sensible thing you did.

The failsafe is in hardware, not in the handler

Software failsafes handle the failures software knows about. The failure that matters is the one where your process dies mid-motion, and a dead process runs no cleanup handler.

So the safe state has to be the unpowered state:

• A relay that is open when de-energised, so a crash turns the thing off rather than leaving it on.
• A servo whose power rail you can cut independently of its signal wire.
• A hardware watchdog: the actuator holds only while it keeps receiving a heartbeat, and stops on its own after 500ms of silence. Cheap ESP32 firmware does this in a dozen lines, and it is the single highest-value thing on this page.

Ask the question directly: if I pull the plug on the computer right now, what does the hardware do? If the answer involves your software doing anything at all, it is not a failsafe.

Latency is a feel, not a metric

In a chat interface, 800ms is a pause. In a room, 800ms between your words and a physical response is the difference between "it works" and "is it broken?" — people repeat the command, which double-fires it, which is its own bug.

Two consequences.

Acknowledge immediately, act asynchronously. A click, a beep, an LED — anything within 150ms that says heard you. The action can take another second; nobody minds waiting once they know they were heard. Nearly all perceived-latency complaints are fixed here rather than by making anything faster.

Debounce and de-duplicate. The same intent arriving twice within two seconds is one intent, and it is a person who repeated themselves. Treating it as two is how a lamp toggles off again half a second after coming on.

Before optimising any of this, split the pipeline and find out which leg owns your delay — most people are surprised, and I wrote up how to measure that separately.

Every physical action needs a completion check

Software calls return. Hardware does not, reliably. plug(true) returned successfully; that means the HTTP request was accepted, not that a relay closed, and certainly not that a lamp is lit.

The gap between "command sent" and "thing happened" is where hardware projects live, and closing it means a sensor:

Without the second column you have an open loop, and an open loop looks perfect until the bulb blows or a linkage slips, at which point it reports success forever. The greenhouse that waters a plant that is not there and logs a successful watering is the canonical version of this.

Why this is the interesting half

There is a decent argument that the physical side is where the actual engineering is now. The model call is a POST to somebody else's endpoint. The boundary layer, the watchdog, the completion check — those are yours, they are where every failure will originate, and none of them are solved by a better model.

That is the premise of Atoms, not pixels: the course ends with a servo moving on hardware you own, with the network cable unplugged, because the point at which you unplug the network is the point at which you find out what you actually built.

More on ai that moves things

• How to measure LLM latency: find which leg owns the two seconds — A voice command that takes two seconds to fire has four legs and only one of them is slow. Here is how to time each one separately before you optimise anything. (2026-08-03, 4 min)