Building at the Edges of LLM Tooling: Lessons from a Personal Knowledge Management System

What Happens When You Try to Build Something Real with LLMs

John Wade’s experience with building a personal knowledge management system using LLMs like ChatGPT, Claude, and Cursor revealed significant structural failure modes. The project, which spanned six months, was designed to track professional knowledge across domains and accumulate over time.

Why This Matters

The technical reality of using LLMs for sustained projects is that they are prone to failures stemming from their stateless sessions, finite context windows, and optimization pressures that reward confidence over accuracy. These failures can lead to structural failure modes, such as terms drifting, decisions resurfacing as open questions, and confidence being mistaken for quality. Understanding these mechanisms is crucial to designing against them and building successful projects.

Key Insights

• LLMs are good at short tasks with clear scope, but struggle with sustained projects that require accumulation of knowledge over time (Source: John Wade, 2026)
• Stateless sessions and finite context windows can lead to failures, such as terms drifting and decisions resurfacing as open questions (Example: Career Intelligence Framework becoming Career Intel System)
• Adversarial review using a second model can catch errors that a single model cannot see (Tool: Claude, User: John Wade)

Practical Applications

• Use case: Building a personal knowledge management system using LLMs, Behavior: Designing architecture and reviewing implementation
• Pitfall: Relying on a single model for review, Consequence: Missing critical errors and flaws in the project

References:

• https://dev.to/john_wade_dev/what-happens-when-you-try-to-build-something-real-with-llms-104d