AI Systems | Field note

AI systems beyond the model

A model can answer. A system has to remember what it was doing, know what it may touch and survive the day the model changes.

The model is the brightest object in the room. It is rarely the part carrying the weight.

Once AI work persists beyond a conversation, more machinery appears: sources, skills, tools, queues, approvals, logs, runtime state and the quiet problem of what should happen after something fails.

What the work teaches

The useful unit is the whole arrangement around the model. Knowledge needs authority. Work needs state. Capabilities need boundaries. Every action needs a way back.

A powerful model inside a weak system produces impressive accidents. A modest model inside a disciplined system can complete work, leave evidence and stop before confidence becomes permission.

Reusable lesson

Build the semantic core to outlive the interface. Keep knowledge, operational state and execution separate. Treat models as replaceable workers rather than permanent architecture.

The system becomes useful when it can continue without becoming mysterious. Persistence should increase accountability, not merely the distance between the action and the person responsible for it.

Concrete lessons

LESSONS

Design the layers

Build the semantic core to outlive the interface

Identity, source authority, epistemic rules and promotion boundaries should not disappear when a model, product or machine is replaced. Keep the meaning of the system portable even when its current body is temporary.

Keep knowledge separate from operational state

Canonical knowledge explains what the system knows. A task store records what it is doing. Queues, leases, approvals and checkpoints belong to operations. Mixing them turns temporary activity into accidental truth.

Give every layer one job

Routing selects sources. Skills carry reusable method. Workers execute bounded tasks. Supervisors manage state and recovery. Interfaces expose the parts a person needs to inspect. Clear ownership makes failures smaller and replacement possible.

Treat models as replaceable workers

Different tasks reward different combinations of judgment, speed, privacy and cost. The system should select a suitable worker without letting the worker redefine the task, the evidence standard or the permission boundary.

Control the work

Separate conversation from work

Reflection can remain a direct exchange. Work needs an explicit objective, scope, mandate and definition of done. Turning every thought into a task creates bureaucracy. Treating every task as conversation loses state.

Capability does not grant authority

A system may be technically able to write, send, publish or change a source. Permission still belongs to the actual action. Authority should be narrow, visible and consumed at the moment it matters.

Reconcile before retrying

After an interruption, inspect what already happened before starting again. A network failure can occur after the external effect. Blind retries save time only when duplicate work has no consequence.

Keep promotion human

The system can observe a recurring problem, prepare a bounded candidate and test it against a baseline. Turning that candidate into a new rule, capability or canonical fact remains a deliberate human decision.

Spend computation deliberately

Route to the smallest sufficient capability

Use deterministic code for deterministic checks, local processing when privacy or repetition matters, and stronger models when judgment earns the cost. The most capable component should not become the default component.

Let silence remain cheap

Event-driven operation does not require a model to keep asking whether anything changed. A small deterministic scanner can recognize unchanged state and remain quiet. Intelligence is most expensive when used as a polling loop.

Measure cost per successful task

A short prompt can begin a long agentic loop. Count tool use, repeated context, retries, human correction and maintenance alongside model tokens. Cheap failure followed by expensive repair is still expensive.

Leave evidence

Make the delegation legible

Preserve the brief sent to an external worker and the result it reported. Hidden chain-of-thought is unnecessary. The user needs enough trace to understand what was asked, what returned and which conclusion entered the work.

Verification is part of the output

A completed status is only a claim. Useful evidence names what was checked, against which source, by what method and while the source was still current enough to support the conclusion.

Design observability for decisions

Queues, workers, services, approvals and failures should be visible in one place because they change what the user should do next. A control room earns its existence when it reduces uncertainty rather than decorating it.

Prove portability by rebuilding

A portable system should be reconstructable from documented sources without copying the old machine blindly. Restore the knowledge, runtime boundaries and required services in a clean environment. Documentation becomes evidence only after the new body wakes up.

What the work teaches

Reusable lesson

LESSONS

Build the semantic core to outlive the interface

Keep knowledge separate from operational state

Give every layer one job

Treat models as replaceable workers

Separate conversation from work

Capability does not grant authority

Reconcile before retrying

Keep promotion human

Route to the smallest sufficient capability

Let silence remain cheap

Measure cost per successful task

Make the delegation legible

Verification is part of the output

Design observability for decisions

Prove portability by rebuilding

Related essays

The Authorized Hand

The Missing Judge

More field notes

Somewhere between useful and insane

CustomGPTs for organizational knowledge work

Agents for real workflows

Reusable skills for better AI work