Human-in-the-Loop Doesn’t Scale at Machine Speed

The irony in all of the AI discussion is that the more AI you deploy, the more humans you seem to need around it. That may sound completely at odds with all the job-loss predictions floating around the media, but the reality inside the enterprise is quite a bit different. The AI productivity boom is forcing organizations to add humans to check the work of the machines they deployed to reduce human workload in the first place. There is probably a philosophical rant in there somewhere, but I’ll leave that one for another article.

I am hearing variations of the same problem across organizations, from code review and QA teams through to governance, risk and compliance. The issue becomes much more serious in regulated industries where AI is moving into workflows that carry legal, financial and operational consequences. QA costs are climbing, review teams are expanding, and in some cases there simply are not enough people available to inspect what increasingly autonomous AI systems are producing and doing. That is not exactly the productivity curve everyone had in mind.

So why is this happening if AI is such a productivity beast?

A lot of this comes down to distrust in the outcome … and some of that distrust is completely justified. I use AI every day across multiple models and tools, and most of the time I am planted in front of a keyboard prompting, re-prompting, correcting, copying, pasting and doing it again until the result is usable. You may be doing exactly the same thing. However, in practical terms, we are both acting as the “eyes-and-ears” for the AI and that is the core of the QA function. This is the awkward reality for technology that is supposed to be automating our work but you can see the ramifications everyday in news where firm after firm gets caught publishing AI generated hallucinations and slop.

Humans have been good at quality assuring human produced work. And that was fine and human-speed. With AI production, it’s now machine-speed and humans can’t keep up.

It’s an Automation Game

Forget AI for a moment, along with some of the increasingly creative rebranding around it. Think about this as an automation problem where AI is simply one of the technologies driving the assembly line. Automation has always required human oversight, so the presence of humans is not the problem … the architecture around when and why they intervene is the problem. A human absolutely should be involved when an AI system encounters bad data, recognizes a conflict in facts, is provided low confidence information, attempts a high-consequence action, crosses a defined risk threshold, or begins behaving materially differently from its established profile.

What does not scale is asking humans to continuously inspect activity happening at machine-speed simply because the system cannot decide when intervention is actually required. That is not control, and it will never scale. It is brute-force QA wrapped around automation, with very expensive humans sitting at the beginning to start the machine, peeking in the middle to see if it’s still spinning, and then waiting at the end for the machine to pop-out its result for intensive review.

This is also where finance teams start asking uncomfortable questions about ROI. If you accelerate one part of an operational process by 10x but triple the cost of supervision, review and exception handling around it, you have not really transformed the economics of the workflow. You have simply moved the bottleneck down the assembly line.

Now consider what happens when agents number in the hundreds or thousands and collectively execute millions of tasks every month across the organization. They will generate code, move money, process transactions, accept and reject contracts, perform AML and KYC work, interact with sensitive data and make increasingly consequential decisions across the enterprise. At that scale, the question is no longer whether a human should remain involved; the real question is how many humans would actually be required if every automated workflow had to be manually reviewed.

That real question is being asked today in Board rooms that are waking up to a new reality.

Getting the Automation Architecture Right

Automation itself is not new, and neither is automated AML, KYC, transaction processing or rules-based decisioning. What is different is the breadth of activity being automated and the degree of functional decisioning being handed to probabilistic systems. We are no longer automating one predefined step in a tightly constrained process. We are increasingly allowing software to interpret context, choose among tools, sequence actions and decide what comes next. That’s fundamentally different than an automated AML/KYC ML/rule-based check bounded by humans.

From an architecture perspective, this is where real-time monitoring and real-time control have to be integrated. These functions are traditionally separated, and they may still operate on separate channels, but in a real-time AI control system they have to be synchronized. I’d say aligned in this case as well but I hate using the word alignment given what the frontier AI safety pundits have done to it … but … monitoring and control have to remain aligned around the actual operating state of the agent.

The major difference is what you are monitoring for. This is not traditional infrastructure monitoring looking primarily at health, availability or performance. You are monitoring decisions, behaviour, judgment, authority and outcomes. That creates a completely different architectural requirement because the telemetry is no longer just something a human reviews later on a dashboard; it becomes an input into the control decision itself.

Monitoring (the act of watching) tells you what an agent is doing and how its behaviour is changing. Control determines what it is allowed to do, under which conditions, with which authority and when the system has to stop or bring a human into the loop. The policies can still be deterministic, but their enforcement increasingly depends on real-time state. Monitoring therefore becomes part of the control loop, continuously feeding the evidence required to make the next control decision.

This is effectively where the Control Plane architecture comes into play. You need the Control Plane to be well aware of state throughout the entire agentic execution. This requires a concrete wrapper around every single agent and agentic call. Agentic flows inevitably contain opaque or partially observable execution paths, and those need to be contained through deterministic controls rather than probabilistic ‘safe’ and ‘feel good’ guardrails. The wrappers are feeding real-time instrumentation signals to the control plane throughout execution. These signals form part of the intelligent state, state change control infrastructure.

Instrumentation vs Telemetry

The industry refers to this instrumentation and signalling as telemetry and it is a valid reference with OpenTelemetry representing a good standards-based approach. However, telemetry and instrumentation signalling are different fundamental approaches on implementation. Telemetry can be captured as logging and can capture different aspects of code functions. But that isn’t necessarily real-time instrumentation and signalling for the purposes of monitoring and control.

The key point here is that tracking code execution in today’s world is still primarily designed for diagnostics and after-the-fact investigation, which eventually gets you to the ugly task of log stitching. Even with real-time dashboards, most telemetry is structured around system health, network activity, performance or intrusion monitoring. Unfortunately, that same approach has largely carried over into AI observability. It’s easy to dump logs, put a dashboard on top and worry about reconstructing everything later.

Case in point, I don’t envy Anthropic having to work through enormous volumes of historical evidence following recent agent-driven security incidents, and OpenAI is dealing with many of the same after-action challenges. When the answers arrive weeks or months later, that may be useful for investigation, but it is not real-time control. Regulators and auditors are going to balk at being told a control worked when the evidence only emerges months after the fact. Most enterprises cannot operationally or financially support that kind of forensic reconstruction every time something goes sideways.

And to be clear, at CharliAI we use logs as well. We just use them for a different purpose. They are generally diagnostic evidence once the control plane has already detected the state change, exception or control event and pointed the operator toward the issue.

One of the important distinctions in the image above is that the control plane is watching state changes as well as real-time exceptions from agents. This is real-time signal interpretation and not something you get from after-the-fact logging or system health checks. That real-time signal informs real-time decisions and pulls operators in to immediately intervene. In this case it was fairly simple token exhaustion on a request that exceeded the allowed credit limit. Easy enough, but instantly trapped and notified.

So Where’s the Human in All This

The objective is not to remove the human from the loop. That is an absolute requirement and I’m in no way jaded by the AI pundits hyping the intelligence in these systems. The intelligence is entirely on the human factor and that’s why QA needs automation as much as the pipeline. The goal is to stop wasting human judgment on activity the system can safely assess, enforce and document on its own, while bringing a human into the loop immediately when real judgment is actually required. It’s as simple as “don’t waste my time with the simple stuff.”

The previous screenshot is a good example. Among thousands of workflows and potentially millions of individual tasks, the operators are alerted to what matters, provided with the surrounding context and pointed directly to the cause. No hunting through dashboards and no waiting for somebody to stitch together logs after the fact. That is what scalable human-in-the-loop looks like, but it requires considerably more underneath the surface than simply throwing an approval screen in front of an agent.

At the architecture level, that means every agent needs to operate through a controlled handler that normalizes signalling, execution and enforcement regardless of which agent framework or model sits underneath it. Agent-native telemetry incorporated by the developer is useful, but handler-level instrumentation is critical because it provides a consistent control surface across the environment. From there you need full tracing across agent actions, handlers, filters and network activity, along with continuous awareness of state changes within and across workflows.

The control plane also needs to understand context, not simply events. That includes ontology awareness, data ingress and egress, movement into and out of data stores and scratchpads, and the relationship between instructions, actions and outcomes. On top of that you can apply deterministic neurosymbolic checks, threshold controls against probabilistic outcomes, adversarial validation such as FISCAL and other “counter” models, consensus across diverse model processing, and layered control over instructions and sub-instructions as reasoning progresses.

Do all of that and you’ve got it nailed. Almost. 🫠 The architecture still has to allow the human production line to scale at the same rate as the automated workflows. I’d like to say that’s all there is to it, but the next and arguably most important step is bringing everything together in a console where humans can watch, inspect, interrogate, verify, approve and release without having to jump between systems to figure out what is going on.

The AI-to-human interface becomes a critical linchpin in that architecture because intervention by exception only works if the human can understand the exception quickly enough to act on it. Context, evidence, state, confidence, controls and the underlying outcome all need to come together in one place. Chunk that experience up, fragment it across bespoke tooling and dashboards, and you have simply created another bottleneck in the assembly line you were trying to automate in the first place.

Beyond the system administrator functions, each and every outcome may need to be inspected, investigated and sometimes approved before release. So that AI-to-Human interface doesn’t stop at the administrator, it also needs to allow business operators to inspect, approve, stop and restart. The scale factor in real-time means that the business operators are now on the factory floor. They can no longer be isolated as back-room functions or up in the ivory tower. Real-time operations and automation means that business users, executives, are becoming true operators.

There is another important consequence to this architecture. Every intervention should leave behind evidence. If an operator is asked to approve, stop, modify or release an AI outcome, the system should already know why that person was brought into the loop, what the agent attempted, what controls were triggered, which authority was in effect and what ultimately happened. That turns human intervention from an expensive QA activity into a governed decision with a forensic record that can be handed to an auditor, regulator, risk team or insurer without spending three weeks stitching logs together afterward. Hint: Everything is a workflow and everything is tracked and traced.

The world of AI has genuinely changed how business is being transformed, but the early days of chatbot productivity are giving way to real operational automation. Human-in-the-loop now has to scale alongside the workflows being automated, and that means the enterprise control architecture has to evolve before the agents outrun the people responsible for them. AI can scale only if the oversight architecture scales with it.