Extracting Emergent Structural Knowledge from LLMs through Sideways Questioning

What If You Could Ask an AI the Question It Doesn’t Know It Knows the Answer To?

Sean Trifero of Strife Technologies investigates methods to extract structural patterns from LLMs that emerge from aggregate data rather than single sources. Large language models compress billions of human-generated parameters into a statistical map of interconnected knowledge.

Why This Matters

Technical reality shows that LLM outputs are often limited by the specific queries humans know how to ask, even though internal activations contain deeper cross-domain relationships. While research like Eliciting Latent Knowledge (ELK) focuses on AI safety and truthfulness, there is a significant gap in systematically surfacing structural patterns that exist across the model’s entire training set but have never been explicitly articulated by any single human author.

Key Insights

• Eliciting Latent Knowledge (ELK) focuses on extracting internal truths from models that may differ from their verbalized outputs.
• Sideways Questioning utilizes cross-domain prompts to force the collapse of domain-specific knowledge into fundamental underlying structures.
• Latent knowledge can be identified by ‘Convergence,’ where unrelated angles point to the same structural pattern without explicit instruction.
• Root access to model internals, such as activation states and logit lens analysis, allows researchers to observe gradient markers during construction vs. retrieval.

Practical Applications

• Use case: Probing latent programming knowledge to identify tacit developer behaviors like naming drift and instinctive technical debt pricing. Pitfall: Relying on self-directed extraction where the model’s weights limit its ability to generate truly unpredictable prompts.
• Use case: Utilizing human-in-the-loop lateral jumps to bypass the model’s predictable statistical patterns and surface new structural insights. Pitfall: Treating model outputs as simple retrieval rather than complex construction, leading to a failure to recognize emerging domain wall collapses.

References:

• https://dev.to/strifero/what-if-you-could-ask-an-ai-the-question-it-doesnt-know-it-knows-the-answer-to-512c