The Path Reuse and Compression Theory of LLM Hallucination

What are Path Reuse and Path Compression in LLMs?

Path Reuse and Path Compression are two structural mechanisms that cause language models to hallucinate at different stages of training.

Researchers at Michigan State University, led by Xinnan Dai, Kai Yang, Cheng Luo, Shenglai Zeng, Kai Guo and Jiliang Tang, published findings on arXiv on April 4, 2026, showing that LLM hallucinations aren't random glitches. They follow predictable patterns tied to how neural networks organize knowledge during training. The team used graph theory to map these patterns, treating the model's internal knowledge as a network of connected paths.

Path Reuse appears in early training. When a model encounters a query, it activates paths it memorized from training data instead of following the context provided in the prompt. Path Compression emerges in later training, when frequently traveled multi-step reasoning routes collapse into direct shortcuts. The model jumps from premise to conclusion without the intermediate steps.

Both mechanisms are structural. They're baked into the model's weights during pretraining, which means surface-level fixes like better prompting or basic fine-tuning can't fully eliminate them.

Why does ChatGPT ignore the context you give it?

Path Reuse causes the model to prefer its memorized training data over the specific information you provide in your prompt.

You paste a document into ChatGPT and ask a specific question about it. The model answers from training data you never provided. It's the most persistent complaint about production RAG deployments. The Michigan State team's graph analysis shows why: the model's internal knowledge paths are so well-worn from billions of training examples that new context can't override them.

Think of it like asking directions from someone who's driven the same route for 20 years. Even if you show them a map with a new faster route, they'll default to the one they know. The model does the same thing, but with facts. When training data says X and your prompt says Y, Path Reuse means the model often picks X.

The researchers tested this on Erdős-Rényi random graphs with 10 nodes, 0.4 edge probability, and 225,808 subgraphs. They trained models to learn graph structures and then presented contradictory information. Models consistently favored memorized paths over contextual evidence during early and mid-training checkpoints.

What is the reversal curse and how does Path Reuse explain it?

The reversal curse is a known failure where models trained on "A is B" can't answer "B is ?" Path Reuse shows it's a directional memory problem.

Ask a language model "Who starred in Forrest Gump?" and it gets it right. Flip the direction — "What film did Tom Hanks star in?" — and the same model fails. Same facts, opposite direction, completely different result.

Berglund et al. documented this pattern in 2023. The Michigan State team's Path Reuse analysis explains the mechanism: the memorized path through the model's weight structure runs from query pattern A to answer B, with no reciprocal path from B back to A. The graph is one-way. Building the reverse path requires separate training examples that explicitly establish the B-to-A connection.

Volume alone won't fix this. You need training data that explicitly covers both directions, or architectural changes that force bidirectional path formation.

How does Path Compression cause reasoning failures?

Path Compression collapses multi-step logic into shortcuts, making models skip intermediate steps and produce wrong answers on complex questions.

Give a language model a multi-hop question, one requiring A to B before it can reach C, and it often skips B entirely. It still lands on C. Sometimes correctly, sometimes not. The middle step was compressed out during training, not skipped at inference time.

When a model processes thousands of reasoning chains where A leads to B leads to C, it eventually shortcuts directly from A to C. The intermediate step disappears. That works when the shortcut is valid. It fails when a new problem requires taking a different path through B.

Path Compression becomes dominant in later training steps. At that point, the model has seen enough examples that common reasoning chains crystallize into direct mappings. The result: the model answers confidently and quickly, but it's using a shortcut that doesn't apply to the specific question.

This connects to Anthropic's finding that chain-of-thought traces are often unfaithful, documented by Chen et al. in 2025. The model appears to show its reasoning step-by-step, but the actual computation skips those steps entirely. The chain-of-thought is a post-hoc rationalization for a compressed path the model already traveled.

RLHF helps. It doesn't fix the foundation.

Reinforcement learning helps more than supervised fine-tuning, but neither method can fully undo structural patterns formed during pretraining.

The Michigan State team tested both PPO (Proximal Policy Optimization) and SFT (Supervised Fine-Tuning) as recovery methods. PPO outperformed SFT at reducing hallucinations from both Path Reuse and Path Compression. But the improvements were partial. The structural patterns formed during pretraining leave lasting imprints in the model's weight structure.

This has significant implications for the industry's approach to AI safety. Most alignment work focuses on post-training interventions: RLHF, constitutional AI and system prompt engineering. The Path Reuse framework suggests these interventions are working against structural patterns that formed much earlier in the training pipeline, during pretraining itself.

The paper aligns with a broader research trend. Ouyang et al.'s original InstructGPT paper established RLHF as the standard alignment approach. But recent work keeps finding failure modes that post-training can't fully address. Path Reuse and Path Compression add two more entries to that list.

Does this explain why bigger models still hallucinate?

Yes. Both mechanisms are structural consequences of how transformers learn, regardless of model size. Scaling parameters doesn't solve the underlying graph problem.

This is arguably the paper's most important implication. The AI industry has spent billions on the assumption that bigger models produce fewer errors. GPT-4 has fewer hallucinations than GPT-3, which seemed to validate that assumption. But the Michigan State analysis shows the mechanisms causing hallucinations are baked into the transformer training process itself.

Larger models have more parameters to form more paths, but they also develop more shortcuts through Path Compression. They have richer memorized knowledge through Path Reuse, but that makes the override problem worse, not better. The model becomes more confident in its memorized answers, making it harder for new context to break through.

This matters for anyone evaluating AI tools for enterprise use. A model that scores well on benchmarks may still hallucinate on your specific documents because Path Reuse causes it to favor training data over your context. Retrieval-augmented generation (RAG) helps but doesn't eliminate the problem, because the model's internal paths still compete with the retrieved context.

225,808 subgraphs, two phase transitions

Hallucination rates tracked across every checkpoint reveal two clear phase transitions: Path Reuse peaks early, Path Compression rises late and their overlap produces the highest error rates.

The experimental setup used Llama-2 architecture trained on two types of synthetic graphs. Erdős-Rényi random graphs provided controlled environments with N=10 nodes and 0.4 edge probability, producing 225,808 subgraphs — yielding a total of 3,982,697 training samples — for analysis. Stochastic Block Model graphs with 1,000 nodes tested whether the patterns held at larger scale.

Tasks included shortest path prediction, reachability queries and path existence determination. At each training checkpoint, the researchers measured how often the model chose memorized paths over contextually correct ones (Path Reuse) and how often it skipped intermediate reasoning steps (Path Compression).

The results showed clear phase transitions. Path Reuse hallucinations peak early and then gradually decline as the model learns to balance memory with context. Path Compression hallucinations follow the opposite curve, rising as training progresses and common reasoning routes get compressed. The overlap period, where both mechanisms are active, produces the highest hallucination rates. This suggests there's an optimal training duration for each task, beyond which additional training increases certain types of errors.

How will this research change AI development in 2026?

The unified framework gives engineers specific intervention targets at known training stages, replacing the trial-and-error approach to hallucination reduction.

Before this paper, hallucination mitigation was largely reactive. Engineers would test models, find errors, and apply patches through fine-tuning or prompt engineering. The Path Reuse and Path Compression framework converts this into a proactive process. If you know Path Reuse peaks in early training and Path Compression dominates in later training, you can design training schedules that intervene at those exact points.

Potential interventions include curriculum learning strategies that vary the training data distribution at critical phases, architectural modifications that maintain separate pathways for memorized and contextual knowledge and training-time regularization that penalizes shortcut formation.

For enterprise AI teams, the practical takeaway is straightforward: don't trust any model's output on tasks where your context contradicts its training data. RAG systems help, but the model's internal paths will still compete with retrieved context. Build verification layers that flag when the model's answer contradicts the provided documents.