Every wave of software has eventually produced its own runtime. Web apps got the application server. Distributed services got Kubernetes. Microservices got the service mesh. Agent systems are creating a similar need for shared operational infrastructure.
This post traces how we got here. The route runs from prompting, through context engineering, into the agent harness, and finally to the agent mesh. Each step was a response to a real limit of the previous one. None of it is theoretical. The pieces exist, the patterns are converging, and the practical implications for any organization trying to become “AI-first” are sharper than the marketing makes them out to be.
Each stage absorbs the previous one and adds the abstraction that the previous one was missing.
Prompting
The first competence anyone developed was writing prompts. Tell the model what you want, then iterate until it does it. Most of the early craft was conventions for structure: XML tags, markdown headers, in-context examples, role framing, separating instructions from data. Anthropic’s guidance was explicit about a lot of this and most of it has aged well.
Prompting is still the foundation. It hasn’t gone away, it has been absorbed. In a production system, the prompt is part of a larger set of instructions, context, tools, and runtime behavior.
Context engineering
The moment people started building anything beyond a single-turn assistant, the prompt stopped being the artifact. What mattered was the context, meaning everything the model could see at the moment it generated a response. Retrieved documents, tool results, prior turns, system instructions, memory. The prompt became the smallest piece of a larger, dynamically assembled payload.
Context engineering is the discipline of getting that payload right. Larger context windows do not remove the need to select and organize information; they can make that task harder. Models suffer from what Drew Breunig usefully named context rot, and it fails in four reliably observable ways:
Context poisoning. A hallucination or error lands in context and gets referenced repeatedly, compounding with each turn.
Context distraction. The context grows so large the model over-indexes on its history and stops drawing on what it learned in training.
Context confusion. Superfluous content, such as overlapping tool descriptions or irrelevant retrieved chunks, pulls the model toward low-quality responses.
Context clash. Pieces of the context contradict each other, usually because they came from independent sources that don’t know about each other.
The counter-moves are by now well understood: tight retrieval, tool loadouts, compaction, partitioning, isolation between subtasks. The point isn’t to list techniques. The point is that filling the context window is not a substitute for deciding which information a task needs.
Three things that had to happen first
Before the agent harness was buildable as anything other than a one-off, three things had to settle. They settled faster than most of us expected, and together they are the precondition for everything that follows.
The first was a working definition of what an agent actually is. After a couple of years of marketing taxonomies, the field converged on a boring and useful answer: an agent is an LLM that runs in a loop, calling tools, until a task is done. That’s it. Once that definition stabilized, you could stop arguing about whether something was “really” an agent and start engineering one.
The second was the Model Context Protocol. MCP gave the industry a shared way to expose tools, data sources, and capabilities to a model. Before MCP, every agent framework reinvented its own tool interface, which meant tools weren’t portable, integrations didn’t compose, and every team rebuilt the same connectors. MCP is the USB-C moment for agentic tooling, and like USB-C, its value is not the elegance of the spec but the network effect of everyone using it.
The third was Agent-to-Agent communication, or A2A. Once individual agents became useful, the obvious next move was composing them, which required a protocol for agents to discover and call each other rather than each framework inventing its own message bus. A2A is doing for inter-agent communication what MCP did for tool use.
Sitting alongside these, the idea of an agent skill emerged as a portable unit of know-how. A skill bundles instructions, references, and small bits of code into something the agent can load on demand. Skills are what let a generalist agent acquire specialist competence without bloating its base prompt, and they fit naturally on top of MCP and the loop-calling-tools definition.
None of this is glamorous, but it matters. Standards are what turn one-off systems into ecosystems. MCP, A2A, and skills can reduce bespoke integration work. A shared operational layer is possible without adopting every one of them, but common interfaces make components easier to combine.
The agent harness
Context engineering deals with what the model sees. The agent harness deals with what the model can do.
Wrap a model in the loop, give it tools (over MCP), file access, a shell, code execution, persistent memory, and a set of skills that describe how to use any of it, and you have an agent. The harness is everything around the loop: the orchestration, the tool layer, the memory subsystem, the I/O contracts, the guardrails. Anthropic’s recent work on agent skills and Claude Code is the cleanest reference implementation I know of for what this looks like in practice. Frameworks like Mem0 sit inside the harness; agentic memory is an active subfield, but a tangent here.
An agent is an LLM in a loop. Everything else is the harness around it.
The harness is where agentic systems became actually useful. It’s also where the per-team cost of building agents got expensive enough to matter. If every team in your organization needs to design its own agent loop, ship its own MCP servers, manage its own memory layer, instrument its own telemetry, and deploy the whole thing as a microservice on Kubernetes, you risk duplicating operational work across teams and making adoption depend on each team’s infrastructure capacity.
The agent mesh
The agent mesh is one way to organize these shared responsibilities. The story rhymes with the move from raw VMs to Kubernetes, except the unit of orchestration is no longer a container, it’s an agent.
A useful way to think about the mesh is as three concerns layered on top of the harness.
An artifact model and a registry. An agent is not one thing, it’s a composition. An MCP server here, a prompt template there, a set of skills, a model binding, an agent definition that ties it all together. Once those become versioned, addressable artifacts with a lifecycle, you can have a registry. A Docker Hub for agentic components. Discovery, provenance, signing, version pinning, curation, the same problems package registries solved twenty years ago, applied to a new set of building blocks. The good registries already pair with the gateway, so what gets published can be governed, scanned, and observed end to end.
An agent gateway. The mesh needs a communication layer, and it turns out to be load-bearing in more directions than people initially expect. It carries A2A traffic between agents and MCP traffic from agents to tools. It is the unified entry point for LLM calls, which is where you put failover, cost accounting, rate limiting, and policy. It is where authentication and authorization live, because you do not want each agent re-implementing OAuth flows against ten different SaaS connectors. The service mesh analogy is exact: the gateway is the data plane.
A runtime. This is the part that does for agents what Kubernetes does for containers. Declare desired state, let the engine reconcile. You stop thinking about “an agent” as something a team writes and operates by hand, and start thinking about a declarative definition that the runtime is responsible for scheduling, scaling, restarting, and observing. Solo.io’s open-source stack (kagent, agentgateway, agentregistry) is the reference architecture I keep returning to, and it is useful for exploring these patterns. Suitability for a production deployment needs to be assessed against its requirements.
The mesh: registry above, gateway in the middle as the data plane, and runtime managing lifecycle. The gateway’s position is also why it’s the natural place to enforce agent IAM.
Why this matters operationally
Stating it as a control plane sounds like architecture astronaut territory. It isn’t. The reason the mesh matters is operational, not aesthetic.
When agents become declarative artifacts managed by a runtime, teams can reduce the repeated infrastructure work needed to ship each one. The size of that reduction depends on the platform and the workload. An AI engineer can define an agent (its skills, MCP dependencies, model binding, and policies) and deploy it without owning a Kubernetes cluster or negotiating with the platform team for a new namespace each time. The platform team is still essential; somebody has to run the mesh, set the policies, curate the registry, and manage the gateway. But the daily work of building agents stops being a DevOps project for each team and becomes something closer to writing a deployment manifest.
This is the same shift Kubernetes produced for services. Before Kubernetes, deploying a service meant negotiating infrastructure. After Kubernetes, deploying a service meant declaring intent. The agent mesh is doing the same thing one abstraction layer up, with a similar aim: reducing the infrastructure decisions each application team must repeat.
The agent IAM problem
There is one more reason the mesh is going to matter, and it’s the reason most organizations underestimate at first: agent identity and access management.
Giving an agent a service account is only the start of its access design. You still need to limit its permissions, account for prompt injection, and record whose authority it is using. A broad credential and a vague claim that the agent acts on behalf of a user do not answer those questions.
Agent IAM is its own problem and it doesn’t reduce to traditional service-account IAM. An agent acts on behalf of a user, but with delegated, scoped, time-bound, often dynamically negotiated authority. It calls tools that themselves require credentials, sometimes from systems with their own identity models that predate any of this. It composes with other agents under A2A, each of which has its own provenance, its own permissions, and its own audit trail. Token exchange, fine-grained delegation, capability scoping, and meaningful logging must work together across the agents and services involved. Existing identity mechanisms provide building blocks; integrating them into an agent workflow remains a design responsibility.
A mesh can provide a useful place to coordinate these controls. A gateway sees the tool calls, model requests, and cross-agent messages routed through it. The registry knows what every agent is composed of and where it came from. The runtime knows what is running, on whose behalf, and under what policy. If you want to enforce that a particular agent can only call a particular MCP server when invoked on behalf of a particular user, with a token that expires in fifteen minutes and is audit-logged with the originating user’s identity, the mesh can coordinate that enforcement. It still depends on correct identity propagation and authorization in the tools and services themselves, including any paths that bypass the gateway.
I expect shared platforms to play an important role in agent IAM. The useful question for an organization is where it can enforce policy consistently and preserve the context needed to make those decisions.
Where this leaves us
The path from prompting to the agent mesh is not a story about ever-larger context windows or ever-smarter models. It’s a story about the same pattern every previous wave of software has gone through: figure out the primitive, agree on the protocols, build the runtime, then standardize the operational layer that lets organizations run it at scale without each team rebuilding the world.
For organizations operating agents across several teams, the agent mesh offers a way to share deployment, discovery, and governance responsibilities. It is worth evaluating when duplicated infrastructure and fragmented controls become an obstacle. The architecture should earn its place by reducing that burden.