What I'm Learning Building Multi-Agent Systems (And What's Breaking)

I'm building a multi-agent system. I'm hitting walls I don't have answers to.

So instead of pretending I'm an expert, I'm going to learn in public.

Here's what I've been studying, what's breaking, and the numbers that changed how I think about agent architecture.

The Numbers That Changed My Thinking

1,300+ PRs per week

Stripe's "Minions" system merges over 1,300 pull requests weekly that are 100% AI-generated. Zero human-written code. On a codebase that processes $1.9 trillion annually.

That's not a prototype. That's production at scale.

3ms boot time

Browser Use runs millions of agents using micro-VMs (Unikraft) that boot in under 100ms, with some configurations hitting 3ms. Standard QEMU containers take ~40ms.

When you're spinning up thousands of sandboxed agents, that difference compounds fast.

~500 MCP tools

Stripe doesn't stuff all tools into context. They built an internal "Toolshed" with around 500 MCP tools and route dynamically based on the task. The agent gets only what it needs.

2 CI retries maximum

Stripe's agents get a maximum of 2 CI test rounds before escalating to a human. Unbounded retry loops burn tokens and compute. Hard caps save money and prevent runaway agents.

Five Patterns I'm Studying

1. Sandbox as a Tool, Not Agent in a Sandbox

The traditional approach: run your agent inside a Docker container.

The better approach: give your agent a sandbox as a callable tool. The agent decides when to execute code, the sandbox returns results, and the agent stays in control.

This separation matters for:

Parallelism: One orchestrator can spawn multiple sandboxes
Isolation: Each sandbox is disposable
Security: The agent never has persistent access to the execution environment

2. OS-Level Isolation is Non-Negotiable

NVIDIA's AI Red Team published guidance that prompts are not security boundaries. Application-level controls are insufficient. Attackers can use indirection to bypass allowlists.

You need:

Micro-VMs or Kata containers (not standard Docker)
Explicit secret injection with short-lived credentials
Network isolation by default

3. Control Plane Architecture

When you have 500+ tools, you can't load them all into context. You need a control plane that:

Routes to the right tools dynamically
Manages tool permissions per task
Handles tool versioning and deprecation

Think of it like a service mesh for AI tools.

4. Hard Limits on Everything

Every unbounded loop is a potential runaway:

Retries: Cap at 2-3 attempts, then escalate
Token budget: Set per-task limits
Execution time: Hard timeouts on all operations
Parallelism: Limit concurrent agents per user

5. Verification Budgets

Stripe allocates explicit "verification budgets" for critical changes. The more risky the change, the more automated and human verification it gets before merge.

This is the inverse of "move fast and break things." It's "move fast with proportional verification."

What I'm Building (And What's Failing)

I'm working on a multi-agent orchestration system for a B2B startup. Here's what's actually happening:

What's working:

Basic tool calling and response handling
Simple sequential workflows
Local sandbox execution

What's breaking:

Context bloat: Loading too many tools kills performance
State management: Agents lose track of what they've already done
Error recovery: When something fails, starting over is expensive
Parallel coordination: Getting multiple agents to collaborate is hard

Questions I don't have answers to:

How do you handle partial failures in multi-step workflows?
What's the right granularity for agent specialization?
How do you debug agent behavior at scale?
When should an agent ask for help vs. retry?

People I'm Learning From

Proper shoutouts to the people publishing their learnings:

Alistair Gray (Stripe) - Published the "Minions" series documenting Stripe's agent infrastructure
Larsen Cundric (Browser Use) - Documented micro-VM architecture for agent sandboxing
Rich Harang (NVIDIA AI Red Team) - Published security guidance on agent isolation
Sivasathivel Kandasamy - Research on "Control Plane as a Tool" patterns

What's Next

I'm documenting this journey in a LinkedIn group: Agentic Engineering & Orchestration.

I'll be sharing:

Prompts and patterns I'm experimenting with
What's breaking and why
Resources from people doing this at scale
Questions for people with more experience

If you're building agents or want to follow along, come learn with me.

This isn't a tutorial. It's a learning log. I'll update it as I figure things out and break new things.