“What a cheeky bugger,” some might say. That may give away my roots in the Scottish Lowlands. 🏴
The title is a bit of clickbait … but not really. It is meant to draw attention to something that is getting lost in all the noisy, endless discussions surrounding AI agents.
Agents today are just engineered software systems. They’re entirely dependent on their code, prompts, configurations, and packages for proper functioning on the use case they’ve been given and for maintaining that facade of intelligence.
They do use and rely on an LLM for part of what they do, including helping predict what to try next in a task.
However, the intelligence is not in the LLM, it’s almost entirely in the scaffolding around it.
The LLM is just a component
If you are impressed by the amp’d-up headlines surrounding OpenAI and Anthropic breaking out of their sandboxes and breaking into real companies. Don’t be.
After a lengthy argument and a ridiculous amount of nitpicking over terminology, I finally got ChatGPT to produce this:
The LLM supplies probabilistic inference. The agent architecture supplies the operational intelligence that turns inference into reliable action.
That was after a lot of back-and-forth and statements from ChatGPT, such as:
“an older model reached the same conclusion.”
My argument to that was these models reached no conclusion whatsoever and that the statement is entirely misleading.
The model simply classified and produced an outcome based entirely on the context fed by the agent’s scaffolding, constructed by its scripts, code, system prompts, and scribbles in the scratchpad. The agent’s code or script needed to determine how to act based on what was produced. No magic, and not even the model in the decisioning.
At one point, ChatGPT even said the agent can learn:
“Toolformer similarly demonstrated that a token-predicting model can learn …”
After shaking my head vigorously, I challenged that one too. These agents do not learn while they’re running a mile a minute. They simply capture and use context. That “learning” as you say is simply the model reacting to updated context within the agent and how the agent’s code decides how to apply the context next time around.
It’s Just a Loop
We have a word for it now. An overloaded word. It’s ReAct.
Reason and Act. It sounds wonderful until you poke your head under the covers and discover it’s nothing more than a loop with some code … err scaffolding.
As a basic human you (and I do mean you) do this every single day with your chatbot. You are the one prompting. You are the one guiding. You are the one looping. You are the one correcting. You are the one figuring out what to use and what to throw away until you get the essay you need. One big loop with your AI.
I did it for this article while trying desperately to retain some basic, and yes, sarcastically simple language.
Agents are just repeating what you and I do at a much faster clip.
Traditional Coding Masquerading as Intelligence
These are purpose-designed agents with a lot of traditional constructs including code, scripts, packages, definitions, scratchpads, checklists, and so on. The LLM is an important component, and there is a dependency upon how good that component is for outcomes. But it’s not entirely different from how important good search results would be when you are trying to figure out how to proceed.
In simple terms, you can think of it as a glorified cheat sheet used by the code to determine how to break out and then break in. Interestingly, ChatGPT suggested I use the term “highly capable probabilistic cheat sheet” … I didn’t go with that.
Do not under any circumstance think of this as a semi-sentient agent that can shake off its surroundings. This type of code has been seen in many forms over the years, with different terms including viruses, worms, bots, malware, and exploit kits. They could all do damage. They could act autonomously. What we have now is a different type of hacker, with different componentry.
And yes, they are getting more capable, that is for certain. Because they now have access to that “highly capable probabilistic cheat sheet.” 🤦♂️
My Recent Debate
At CharliAI Labs we get into healthy debates about the problems we’re solving and techniques we use to solve them. In one recent debate, it was argued that not all our workflows were agentic. Instead, we apparently had workflows containing agents that were agentic. I was a bit taken aback. We debated it for a good 40 minutes before our scientists finally realized that, sure enough, all our workflows were agentic.
Hence the title: My Agent is Better Than Your Agent.
It depends on your definition. Are you using the latest and greatest tools and models? Vanity matters to techies when they walk into a room, so terms such as ReAct and MCP matter.
For an old-school, been around the block, grey-haired Scotsman , you better prove you’ve dragged yourself through the mud and earned your stripes.
The debate came down to how you construct your agent, what you think an agent really is, and what you expect it to do. Concepts and architecture matter more than terminology.
And to this point, my next article is going to explain the difference between “macro workflows” designed for business, versus “micro workflows” that are really just scaffolding inside an agent. Hint, businesses need macro workflows that involve tens of agents, systems, and humans participating in an end-to-end workflow to get a job done.
Not one so-called “God” agent coded to get out of a sandbox with Python, ReAct and MCP … because much of that is just updated terminology for s*t that’s been going on for decades.
You Need Better Visibility & Controls
The capabilities of agents are far more democratized, and that is the real risk. The agents are getting better, but it is the democratization of those capabilities that has dramatically increased the blast radius.
Powerful tools in the wrong hands, even those with good intentions, create risk.
Agents are not sentient. They are not magically breaking out or breaking in. They are moving at machine speed to expose weaknesses in your infrastructure.
What is needed?
Better visibility and better controls.