How to Design an Advanced Multi-Agent Reasoning System with spaCy Featuring Planning, Reflection, Memory, and Knowledge Graphs

How to Design an Advanced Multi-Agent Reasoning System with spaCy Featuring Planning, Reflection, Memory, and Knowledge Graphs

The article details a multi-agent AI system built with spaCy that processes text through planning, reflection, and memory. It includes a working example of a 10-iteration pipeline analyzing a sample text about AI research.

Why This Matters

Real-world NLP systems face gaps between idealized models and practical text complexity. This system addresses this by integrating episodic memory (storing high-confidence observations) and reflection modules (adapting strategies based on performance trends). Failures in entity extraction or reasoning chains could lead to incomplete knowledge graphs, costing up to 30% accuracy in downstream tasks.

Key Insights

• “8-hour App Engine outage, 2012”: Not applicable (context lacks this metric)
• “Sagas over ACID for e-commerce”: Not applicable (context focuses on NLP, not transaction systems)
• “Temporal used by Stripe, Coinbase”: Not applicable (context uses spaCy, not Temporal)

Working Example

!pip install spacy networkx matplotlib -q
import spacy
from dataclasses import dataclass
from collections import deque, defaultdict
from enum import Enum
import hashlib
from datetime import datetime

class MessageType(Enum):
    REQUEST = "request"
    RESPONSE = "response"
    BROADCAST = "broadcast"
    QUERY = "query"

@dataclass
class Message:
    sender: str
    receiver: str
    msg_type: MessageType
    content: Dict[str, Any]
    timestamp: float = field(default_factory=lambda: datetime.now().timestamp())
    priority: int = 1

    def get_id(self) -> str:
        return hashlib.md5(f"{self.sender}{self.timestamp}".encode()).hexdigest()[:8]

class WorkingMemory:
    def __init__(self, capacity: int = 10):
        self.capacity = capacity
        self.items = deque(maxlen=capacity)
        self.attention_scores = {}

    def add(self, key: str, value: Any, attention: float = 1.0):
        self.items.append((key, value))
        self.attention_scores[key] = attention

    def recall(self, n: int = 5) -> List[Tuple[str, Any]]:
        sorted_items = sorted(self.items, key=lambda x: self.attention_scores.get(x[0], 0), reverse=True)
        return sorted_items[:n]

class MetaController:
    def __init__(self):
        self.nlp = spacy.load('en_core_web_sm')
        self.agents = {
            'cognitive_entity': CognitiveEntityAgent('CognitiveEntity', 'entity_analysis', self.nlp),
            'semantic_reasoning': SemanticReasoningAgent('SemanticReasoner', 'reasoning', self.nlp),
            'knowledge_graph': KnowledgeGraphAgent('KnowledgeBuilder', 'graph_construction', self.nlp)
        }
        self.global_memory = WorkingMemory(capacity=20)

    def execute_with_planning(self, text: str) -> Dict[str, Any]:
        initial_task = AgentTask(task_id="task_001", task_type="cognitive_entity", data=text)
        results = {}
        task_queue = [initial_task]
        iterations = 0
        max_iterations = 10
        while task_queue and iterations < max_iterations:
            task = task_queue.pop(0)
            agent = self.agents.get(task.task_type)
            if not agent or task.task_type in results:
                continue
            result = agent.process(task)
            results[task.task_type] = result
            self.global_memory.add(task.task_type, result, result['confidence'])
            for next_action in result.get('next_actions', []):
                if next_action in self.agents and next_action not in results:
                    next_task = AgentTask(task_id=f"task_{iterations+1:03d}", task_type=next_action, data=text)
                    task_queue.append(next_task)
            iterations += 1
        return results

Practical Applications

• Use Case: Text analysis in research institutions using the multi-agent system for entity extraction and knowledge graphs.
• Pitfall: Over-reliance on spaCy’s NLP models without fine-tuning, leading to inaccurate entity recognition in domain-specific texts.

References:

• https://www.marktechpost.com/2025/11/14/how-to-design-an-advanced-multi-agent-reasoning-system-with-spacy-featuring-planning-reflection-memory-and-knowledge-graphs/