One agent drafts a customer promise. Another updates the runway model. A third prepares a regulatory filing. Each piece of work is plausible; together, they describe three different companies. Sales has promised a feature product deliberately deferred, finance has counted a renewal procurement has not approved, and the filing exists as a polished draft that nobody has submitted.
The problem is not that the models are insufficiently intelligent. The company has made local action cheap without building a way to keep those actions coherent. More agents will produce more work, more quickly, and increase the surface area for contradiction, review and recovery.
An AI-run company therefore needs a different definition of scale. The useful measure is not tasks automated or documents produced. It is the number of consequential decisions the company can make independently without losing its intent, evidence or ability to correct course.
Scale judgement, not founder presence
Early companies often run on proximity. The founders hear the same customer calls, remember why an exception was made and can repair a misunderstanding in real time. “Use your judgement” works because the relevant context is shared, even when it is not written down.
That arrangement fails as decisions multiply. The answer is not to turn every founder preference into a policy. It is to make the few things that should travel explicit: what the company is trying to change, the trade-offs it will consistently make, who owns which outcome and what evidence can overturn the current view. Claire Hughes Johnson's operating work is strongest on this point: self-awareness becomes a company capability only when other people can use it to make a better decision.
Principles should resolve trade-offs
Most company principles are too broad to guide an unfamiliar decision. “Customer first” does not tell an agent whether to promise a custom integration, delay a launch or protect product reliability. A useful principle names the tension and gives one side a deliberate advantage: preserve trust over closing this quarter; prefer reversible tests over confident debate; never represent prepared work as completed work.
The same test applies to objectives. “Improve growth” invites local optimisation. “Increase paid activation among founder-led UK teams without worsening 60-day retention” gives the agent a result, a customer and a guardrail. It can compare actions instead of merely completing a task.
Write a principle only after a recurring trade-off appears. Attach the decision that made it necessary, the evidence behind it and the condition under which it should be reconsidered. The principle is then a compact piece of company judgement, not decoration.
Put decision rights where the evidence lives
Jensen and Meckling's distinction between general and specific knowledge explains why central approval breaks at scale. Some knowledge is cheap to transmit; other knowledge is expensive to detach from the customer conversation, operational system or person who holds it. A company can either move that knowledge towards the decision, or move the right to decide towards the knowledge.
Agents make that choice concrete because they can retrieve evidence and act in the same workflow. Give autonomy according to the decision, not a vague role. An agent can act when the relevant truth is clear, the move is reversible and the consequence is bounded. It should prepare a recommendation when the evidence is strong but the action changes an external commitment. It should escalate when objectives conflict, sources disagree or recovery would be costly.
- Act: reconcile a known record, schedule approved work or run a bounded test.
- Prepare: draft a contract, filing, payment or customer promise for the accountable person.
- Escalate: surface the conflict, likely consequence and smallest decision required.
This is a decision envelope. Its boundary should be defined by evidence, reversibility and consequence, not by whether a task sounds administrative. Sending one wrong email can be harder to recover from than re-running a large internal analysis.
Design the hand-off between people and agents
A long prompt is not an operating model. The durable interface is a responsibility brief: the objective, accepted sources, decision envelope, expected evidence and stop conditions. It should also say how the work returns. The recipient needs the decision, the reasons it matters and the evidence required to approve or correct it, not a diary of everything the agent did.
This protects a scarce resource. Herbert Simon observed that abundant information consumes the attention of its recipients. AI makes the production side dramatically cheaper, but it does not make executive attention abundant. A company that asks a founder to read every generated artifact has automated production and preserved the bottleneck.
Do not leave people only the rare, confusing failures. Bainbridge's classic automation warning still applies: when the system handles the routine work, the human may be least prepared precisely when an abnormal condition demands intervention. Keep accountable people close enough to consequential work that they retain context, can spot drift and can challenge the operating rule, not merely click approve.
Make the operating system learn
Most agent workflows end with an output. Company work should end with evidence. A sent message returns a delivery record and reply. A filing returns an authority receipt and accepted document. A pricing test returns observed behaviour, not a confident summary. That evidence must update the company's accepted truth before the next action begins.
There are two kinds of correction. The first improves the action: fix the number, change the message, retry through the approved path. The second changes the rule that produced it: revise the target customer, narrow the agent's authority or replace an assumption the company had treated as fact. Argyris called the latter double-loop learning. It is how the operating system becomes more accurate rather than merely more efficient.
This is where a company-aware agent can contribute more than a generic assistant. With persistent access to the company's objectives, accepted records, decisions and work receipts, Cady can notice a risk before it appears in a dashboard: a customer promise diverging from product intent, an obligation with no current owner, or repeated exceptions showing that the rule itself no longer fits. The useful soft signal is often the distance between what the company says matters and what its work is actually doing.
Widen autonomy only after evidence
Confidence is not a reason to widen an agent's authority. A reliable outcome, inspectable evidence and successful recovery are. Start with one recurring responsibility whose sources and consequences are clear. Let the agent prepare the work. Review both the result and the path that produced it. Then widen the envelope one decision at a time.
- Did the agent use the authoritative record rather than the most convenient one?
- Did it know when evidence conflicted and return a focused decision?
- Did the action produce a receipt another person can inspect?
- Did the result update company memory before the next run?
- Did review reveal a bad action, or a bad operating assumption?
Make that review continuous rather than ceremonial. NIST's AI risk framework is useful here because it treats governance, measurement and monitoring as recurring work, with explicit roles for human oversight. For an operating company, the practical translation is simple: name the owner, keep the source and outcome visible, review exceptions, and narrow authority when the evidence deteriorates.
The goal is not a company where humans disappear from decisions. It is a company where judgement travels, attention goes to genuine tension and every completed action leaves the organisation easier to operate. That is how AI increases management capacity rather than creating management debt.
Continue with Which growth channel should you test first?, or see how the operating layer develops on the public roadmap.