Scaling Your Pokémon Team: When Your AI System Outgrows Its Original Design

When Your AI System Outgrows Its Original Design

Kourtney Meiss’s personal AI orchestration system uses Metagross to route tasks to specialized agents. After rapid expansion, the shared agent journal grew to over 240KB of prose, consuming half the context window before work even began.

Why This Matters

Ideal models assume static scope and fixed workloads, but real-world AI systems inevitably outgrow their initial design. The naive shared-journal approach became a liability, wasting tokens and degrading performance. Without infrastructure changes like capability registries and archival scripts, routing decisions turn fuzzy, handoffs lose information, and token budgets are exhausted on overhead rather than actual tasks.

Key Insights

• Fact: The shared agent journal reached 240KB of prose after months of operation, forcing agents to read the entire file each session (source: author’s experience, 2026).
• Concept: A capability registry (registry.json) serves as source of truth for what each agent can do—Metagross consults it instead of guessing from memory when tasks are ambiguous.
• Tool: query-journal.sh allows agents to retrieve only relevant entries by keyword or date range instead of reading the full journal, cutting context load dramatically.
• Concept: Skills directory (skills/) stores domain knowledge in focused files separate from system prompts—any agent can load relevant skills without bloating its prompt.
• Tool: LLM-as-judge rubrics in evals/ test output quality across five dimensions (e.g., research completeness, writing style) with pass thresholds averaging ≥3.5/5.

Working Examples

Example entry in registry.json defining Genesect’s capabilities for Metagross to route tasks correctly.

{
"name": "genesect",
"description": "Coding agent",
"capabilities": ["write_code", "debug", "explain_code", "refactor", "code_review", "stack_trace_analysis"],
"taskTypes": ["coding"]
}

Example entry in state.json showing shared facts that multiple agents need without re-deriving them.

"current_sdk_version": {
"value": "0.23.6941",
"source": "agent-journal.md — 2026-04-23T09:50-mew",
"last_updated": "2026-04-23T09:50-05:00",
"updated_by": "mew"
}

Practical Applications

• {Use case} Implementing a capability registry to dynamically route tasks based on agent descriptions instead of hardcoded logic — {Pitfall} Relying solely on orchestrator memory leads to fuzzy routing decisions when agent rosters expand beyond a few members.
• {Use case} Providing agents with a query-journal.sh script to retrieve only relevant historical entries — {Pitfall} Allowing a shared journal to grow unbounded wastes context window space and degrades performance.
• {Use case} Using a skills directory (skills/) for domain knowledge loaded on demand — {Pitfall} Stuffing all best practices into system prompts makes them unmaintainable and forces updates across multiple agents.
• {Use case} Setting a delegation depth limit (delegationDepth >= 3) to prevent runaway chains — {Pitfall} Without limits, agents can spiral into deep recursion wasting tokens and time before surfacing blockers.

References: