The Control Room

When I was a kid, seven of us lived in a three-bedroom apartment. I shared one room with my brothers. After we were supposed to be asleep, my father would come in, sit at the computer, and start playing Red Alert 2.

Staying awake until 2 a.m. was hard, but I usually did it anyway. None of my friends had a father who did anything like this. To me, it was the coolest thing in the world.

My father managed soldiers, tanks, drones, and strange machines against an enemy force much larger than his. One person commanding an entire war. He clicked the mouse rapidly, sending units across the map. The machines moved and fought on their own until they needed another command. He watched the whole field and decided where to intervene.

Looking back, I can see why that stayed with me: one person watching the whole game, deciding where to step in.

Years later, I built the same kind of room for real vehicles and robots. Today I build it for AI agents doing legal and financial work.

Phantom Auto operations center, Atlanta. Built to route remote operators into vehicles and robots at the moment the machine couldn't decide on its own.

At Phantom

At Phantom, we put operators in a room where they could see the vehicles and take over when needed.

Starting in 2017, I ran this as a company. Phantom Auto had six operator cockpits in Mountain View, each with a steering wheel, brake, and accelerator. Bonded cellular ran across AT&T, Verizon, and T-Mobile, with cameras covering every angle on the vehicle, so a remote operator in California could drive a Lincoln MKZ through traffic in Las Vegas, 540 miles away, in real time. The New York Times and Wired covered it; WSJ filmed it.

We pivoted in 2019 from highway autonomous vehicles to logistics: forklifts, yard trucks, and sidewalk delivery robots, for customers that included Maersk, CJ Logistics, ArcBest, and Serve Robotics. The operators were no longer driving on public roads; they were steering warehouse equipment across multiple facilities from a single room thousands of miles away. One person can drive one forklift, and one person can watch several.

Phantom Auto remote operators, 2024. Each operator managed multiple vehicles simultaneously across different customer sites. Wired covered this shift to remote physical labor as a broader pattern.

We shut Phantom down in March 2024, after seven years and $95 million raised.

Yet the need for people to help machines through difficult situations remained. Serve Robotics acquired the IP and assets in April 2025 to support its sidewalk delivery fleet. Waymo calls its remote assistance fleet response and runs it from a staffed room. Tesla has remote operators for its Cybercabs.

An intervention can also become a training example. An operator steering around a construction zone or correcting a robot's warehouse route gives us a decision to examine later. The same is true of an attorney correcting a proposed contract clause. We still have to decide which corrections belong in the model's training data.

When a mistake stops the work

In Red Alert a mistake cost you the game; with real systems it can stop the whole deployment.

On June 12, three days after Fable 5 launched, Anthropic was ordered to cut off foreign-national access to Fable 5 and Mythos 5, and because the company could not guarantee compliance by nationality in real time, it disabled both models for all customers. Two weeks later, Mythos 5 began returning for a small set of approved organizations. Fable 5 remained blocked.

Anthropic disputed the finding and warned that if the same standard were applied across the industry, it "would essentially halt all new model deployments for all frontier model providers." One risk, described only at a high level, was enough to take a frontier model offline for hundreds of millions of people in an afternoon.

Human judgment built these models. Reinforcement learning from human feedback, RLHF, is people sitting in the loop during post-training, ranking outputs and teaching the model what good looks like. That need does not end with training. It moves to wherever the consequences become real: a model writing a first draft can run wide open, and a model sending the email, moving the money, signing the contract, or steering the car is working where a wrong answer is expensive and hard to take back. The amount of review a system needs is set by the cost of being wrong.

In March 2018 an Uber self-driving car killed a pedestrian in Tempe, Arizona. Uber suspended its entire autonomous testing program within days. One death, and a whole fleet stopped. After that, the AV industry became much more careful about the handoff between machine and human.

Software agents now pay invoices, file documents, and change customer records. After a serious failure, customers will want to know who could have stopped it, and why they didn't.

Who gets called

Before an agent starts work, I want to know who gets called when it cannot safely continue. That person needs authority to stop the task and a way to see what the agent has already done.

A model may be able to prepare a contract or a financial action without being ready to commit someone to it, and those are separate permissions in the workflow.

At mixus, a step can require human approval before it proceeds, and the system waits until someone signs off; the review routes to the person with the right expertise, and if it stalls, it escalates. Let it prepare the work, and before it commits someone to a contract or moves their money, get approval. Every decision is logged.

The reviewer's queue

A review that takes an hour becomes a queue, and people route around queues. The handoff has to reach the right person with enough context to decide, and every approval has to carry a name. If routine decisions keep interrupting the expert, the difficult ones end up waiting too.

How do we give the reviewer enough context to make a good decision without making them redo the work?

---

Shai Magzimof

Related: Let the Sand Think, We're Merging

---