From demos to production: the missing layer that turns AI into operations

Eight pilots went into the year. One reached production. The other seven were not bad pilots — they were demos that nobody figured out how to run. The model worked. The notebook ran. The slide deck closed the room. Then the question landed: who watches it on a Tuesday at 3 a.m., who fixes it when the input format drifts, who signs off on the borderline cases, who proves to audit that the call was the right one. Nobody had an answer. The pilot got renamed “phase two” and quietly died. This pattern is not a talent problem. It is a layer problem.

TL;DR

• Most enterprise AI fails not at the model layer but at the operating layer — the workflow, the evidence, the human gate, the recovery path.
• The unit of deployment is not a model. It is a governed workflow with explainability, identity context and HITL built into the contract.
• An operating framework turns AI from a demo asset into a production capability with the same rigor as an ERP module.

The graveyard of pilots

Walk into any enterprise that has spent the last two years on AI and you will find the same exhibits. A document extraction prototype that pulls 92 percent of fields correctly on the test set and sits in a Jupyter notebook nobody owns. A support classifier that routes tickets beautifully on a Friday demo and gets quietly turned off three weeks later because nobody knew what to do with the low-confidence bucket. A summarization tool wired to a Slack channel that two people use and forty-seven people ignore. A copilot embedded in the CRM that generates email drafts the sales team finds “kind of useful” and never quite trusts. Each of these is a real piece of work. None of them is in production.

The friction is rarely about model quality. The friction is about everything around the model. Where does the input come from when there is no human pasting it into a textbox. What happens when the input is malformed, when the third-party API times out, when the model hallucinates a field that does not exist in the downstream system. Who reviews the borderline cases, and through what surface. How do those reviews get back into the next training pass, or into the rules that override the model. When the auditor shows up six months later and asks why a specific refund was approved, can anyone reconstruct the chain of evidence — input, rule applied, model output, human override, system effect.

The honest answer in most pilots is: nobody built that. The team built the smart part. The boring part — the framing, the routing, the persistence, the gating, the trace — was supposed to come later. Later never came, because later turned out to be the actual product.

The graveyard of pilots is not a story about bad models. It is a story about missing layers.

What gets shipped to production is never just the inference. It is the full operating envelope around it: ingestion, normalization, governance, fallback, human review, audit trail, recovery. That envelope is what makes a piece of intelligence durable. Without it, every smart demo becomes a one-off party trick, impressive in isolation, impossible to run on a Tuesday morning when the team is dealing with everything else.

The teams that did get something into production usually built that envelope by hand, on top of a queue, a database, a custom admin panel and a lot of duct tape. It worked. It was also brittle, undocumented and known only to the two engineers who wrote it. The third pilot died not because the model was wrong but because nobody wanted to write the envelope a third time.

The unit of deployment is wrong

Most teams ship a model. The operation needs a workflow. Those are not the same object, and the confusion between them is what keeps pilots stuck.

A model is a function. Input goes in, output comes out, sometimes the output is good, sometimes it is not, and the team measures quality on a held-out set. That is a useful artifact for research. It is not a useful artifact for an operation that has to handle a million events a month, route exceptions, prove its decisions, and recover from failures it has never seen before. An operation does not need a function. It needs a workflow with evidence — a typed sequence of steps that knows what came in, what was decided, what was done, by whom, and under which policy.

The shift sounds small. It is not. When the unit of deployment is a model, the team thinks in terms of accuracy, latency and prompt versions. When the unit is a workflow, the team thinks in terms of states, transitions, gates, retries, idempotency keys, dead-letter behavior, and the human surfaces that touch each transition. The first vocabulary belongs to a research lab. The second vocabulary belongs to a system of record. Production lives in the second vocabulary, which is why production keeps rejecting things built in the first.

This reframing is not anti-model. It is the opposite. A model that lives inside a workflow gets to be much smaller, much more focused, much more replaceable. It is one node in a graph. It can be A/B tested against a heuristic. It can be wrapped in a confidence threshold that routes uncertain cases to a human. It can be swapped out for a different provider without rewriting the surrounding business logic. The model is treated like any other operational component — important, but not load-bearing on its own.

Once the unit of deployment is a workflow, the operating questions become tractable. Logging is a property of the node, not a TODO. Retry policy is a property of the node, not a hope. Authorization is a property of the workflow, not a custom check in a service. Versioning is built in, because a workflow has a definition and an execution, and the execution carries the version of the definition it ran against. Six months later, an auditor can ask “what did this workflow look like when it processed this record” and the answer is a single query.

A model is the smartest part of a workflow. A workflow is the smallest unit you can actually run.

The pilots that survive are the ones that figure this out, usually the hard way. The ones that stall are the ones still trying to deploy a model.

What an operating layer changes

Once the unit is right, the next question is what the operating layer actually has to do. Not “what would be nice to have”, but what is non-negotiable for a piece of AI to behave like the rest of the operation. There are five shifts, and each of them is concrete.

First, execution is deterministic by default. The workflow runs the same way every time. Given the same input and the same definition version, the path through the graph is reproducible. The model call inside the workflow is non-deterministic by nature, but the framing around it is not — the policy lookup happened, the timeout fired at the same point, the next node was selected by the same rule. Operators can ship changes with confidence because they are changing one node, not the entire fabric. Workflows carry version, state and trace. Demos do not.

Second, governance is built into the workflow, not bolted on. Who can run this. Whose data can it touch. What rate limit applies. What is the cost ceiling per run. These are not annotations added during a security review three weeks before launch. They are declared on the workflow definition, enforced by the runtime, and visible in the Command Center when an operator inspects what is available to them. Governance becomes a property of the artifact, not a manual checklist.

Third, human-in-the-loop is a contract, not a fallback. Tasks are typed nodes in the workflow. The workflow pauses, a human reviews on a surface designed for the decision, and the run resumes with the evidence captured. The human is not interrupted by a vague Slack message asking for help. They get a structured artifact, with the context that produced it, the options the workflow expects back, and the SLA the workflow is tracking. When the human acts, the action is part of the same run. HITL becomes an asset, not a regression.

Fourth, explainability is a byproduct, not a project. Because every node logs actor, input, policy, output and outcome, Explainability is just a different read on the same data. There is no separate “explainability initiative”. A run can be replayed in the UI, stepped through node by node, and the rule that fired at each step is one click away. When a customer disputes an outcome, the team does not go hunting through six services. They open the run.

Fifth, knowledge is a first-class input. Policies, rule sets, eligibility criteria, escalation matrices — the things that change frequently and need to stay close to the people who own them — live in the Knowledge Center. The workflow reads from there. When the policy changes, the change is versioned, published and immediately visible to every run that follows. The model is not asked to memorize the policy. The model is asked to apply the policy that was retrieved for this run. That separation is the difference between a one-off prompt and an operational system.

Each shift is small on its own. Together they are the difference between a demo and a production capability.

The cognitive layer pattern in practice

“Cognitive layer” is easy to say and hard to mean. The honest version of what it means in practice is unglamorous: one place where signals come in, rules apply, agents act, humans approve, and evidence accumulates. The glamour is in what becomes possible once that place exists.

Signals come in through triggers. A webhook fires from the CRM. A row appears in a queue. A scheduled run kicks off. A user submits something through an Operational App. The signal is normalized into a workflow input, and the workflow starts. Nothing about this is novel — every integration platform does it. What is different is that the signal carries its identity context from the first node, and that context follows it through every subsequent step.

Rules apply through the workflow definition itself. The graph encodes the order of operations. Which checks happen first. Which extraction runs before which classification. Which path is taken when a value is missing, when a customer is in a sanctioned country, when a refund exceeds a threshold. These are not buried in the prompt. They are visible in the graph, modifiable by people who do not write code, and versioned.

Agents act within bounded surfaces. An agent in this layer is not a free-roaming entity given a goal and a credit card. It is a node in a workflow with an explicit set of tools, a cost ceiling, a time ceiling, and an expected output shape. The agent can be creative inside its sandbox. It cannot wander. When it produces output, the output flows into the next node — which is often a validator, often a confidence check, often a human gate.

Humans approve through Tasks. The same surface, the same SLA model, the same evidence trail. A queue exists. Reviewers know what is theirs. Decisions feed back into the workflow without anyone having to copy a result from one system into another.

Evidence accumulates because every step writes its trace. Omnisearch makes that trace discoverable. The team can find a run by customer, by date, by outcome, by the rule that fired, by the human who approved. The trace is not exhaust. The trace is the system of record for what the AI did.

That is the organizational cognitive layer. Not an idea, a substrate. The interesting work happens on top of it — but the substrate is what makes the interesting work possible.

What this looks like: a refund triage flow

Take a concrete case. The operation receives refund requests through a customer portal. Volume is a few thousand a week. Today, every request lands in a generic support queue and a human reads it, hunts for the order, checks the policy, decides, and writes a reply. The team spends most of its day on the eighty percent of requests that are obvious — policy says yes, policy says no — and not enough time on the twenty percent that require judgment.

The workflow gets four real steps.

Step one: webhook trigger. A submission from the portal hits a webhook node. The payload is normalized, validated against a schema and tagged with the customer’s identity context. Bad payloads go to a dead-letter queue with a Task for engineering. Good payloads continue. The workflow definition pins the schema version, so when the portal team changes a field, the change is detected and surfaced rather than silently breaking things downstream.

Step two: policy lookup. A node reads from the Knowledge Center, pulling the current refund policy, the regional overrides, and the customer’s tier-specific rules. The policy is not embedded in a prompt. It is fetched at execution time, with its version stamped onto the run. When finance updates the policy on Monday, every run starting Tuesday morning uses the new version. Old runs remain replayable against the version that was in force when they ran.

Step three: agent classification. A classification Agent receives the normalized request and the resolved policy. Its job is narrow: decide whether the request falls into “auto-approve”, “auto-deny”, or “needs human review”, and produce a short rationale citing the specific policy clause it applied. The agent has access to two tools — an order lookup against the ERP, and an entitlement check. It has a cost ceiling per call and a confidence threshold. Below the threshold, the output is forced to “needs human review” regardless of what the model said.

Step four: HITL gate or auto-execute. Auto-approve and auto-deny paths execute a refund action against the payment system and notify the customer. The “needs human review” path creates a Task, assigned to the appropriate queue based on amount and region. The reviewer sees the request, the policy that was applied, the agent’s rationale, and the recommended action. They approve, modify or reject. The workflow resumes from that point and executes the same downstream actions, with the human’s decision and reasoning attached to the run.

Through the whole thing, the Command Center shows operators what is running, what is blocked, what is overdue. Omnisearch lets a support lead find every refund touched by a specific policy clause this quarter. Explainability lets an auditor replay any single run, see the policy version that applied, the agent’s rationale, and the human’s decision. Nodes that get reused — the policy lookup, the order lookup, the entitlement check — come from the Node Marketplace rather than being rebuilt per workflow.

None of this is novel as ideas. What is novel is that it all lives in one operating layer instead of seven Jira tickets across four systems.

What it changes for the buyer

For a COO or a CTO, the calculus shifts in a way that a model API alone cannot deliver. The first thing that changes is the unit of risk. The risk is no longer “did we pick the right model”. The risk is “does our workflow handle the edge cases”, which is a much more tractable conversation. Models change every quarter. Workflows that are well-defined survive model swaps.

The second thing that changes is the unit of staffing. A team that ships workflows does not need to be a research team. It needs operators, business analysts and a few engineers — people who understand the operation and can encode it. The expertise is in the domain, not in the model. That is the staffing pattern of every other operational system in the enterprise, and it is the only staffing pattern that scales past a single pilot.

The third thing that changes is the unit of audit. Auditors get a system that is built for them. Every run has an actor, a policy, a decision and an outcome. There is no need to reverse-engineer the audit trail from logs scattered across services. The trail is the system.

The fourth thing is the unit of governance. New use cases do not start from zero. They inherit the policies, the identity model, the rate limits and the recovery patterns of the operating layer. A new workflow is a new graph, not a new platform. That is what makes the second use case cheaper than the first, and the tenth cheaper than the second.

A model API is a component. An operating layer is a capability. Buyers eventually realize they need the second, and they realize it most often after the third pilot dies.

Closing

The end of the pilot era is not about better models. The models are already better than the operations around them. The end of the pilot era is about admitting that production AI is an operational discipline, not a research output, and choosing the layer that makes that discipline cheap to practice. That layer is what Sommatic is. It is in production today, it is opening to more customers, and it is built on the observation that what stalled the last cycle of AI was never the intelligence — it was the framing, the gating, the trace and the recovery path. Get those right and the model gets to do its job. Skip them and the model gets to do another demo. If your team has a pilot that should be in production by now, bring it.

→ Talk to us