Why Your RAG Architecture Is Lying to You About Code Dependencies

Enterprise teams adopting RAG for code modernisation typically inherit an implicit assumption: that vector similarity is a reliable proxy for what the model needs to see. Two converging research findings published in 2026 suggest that assumption is structurally unsound. The retrieval layer powering your migration tooling may be returning plausible-looking context while silently discarding the relational and measurement information that determines whether the generated code actually compiles.

Companion piece to our broader work on production code migration. See Why Coding Agents Fail at Real Migration Work for how benchmark blindness compounds these retrieval failures at the repository level.

The Structural Problem with Chunk-Based Retrieval

Standard RAG pipelines split source code into text chunks and retrieve the chunks whose embeddings sit closest to the query vector. That design works acceptably when the relevant information is self-contained within a passage. Code dependencies are not self-contained.

Inheritance chains, interface contracts, and parent-child class relationships are topological properties of a codebase. They exist across files, across modules, and across layers of abstraction. When a retrieval system fetches isolated chunks, it severs those chains at the retrieval boundary before the LLM ever sees them.

The practical consequence is that the model generates code against an incomplete picture of the system. It produces output that is syntactically coherent but architecturally broken, because the structural context it needed was ranked below the similarity threshold and never retrieved.

What the Research Shows About Dependency Loss

Jaiswal et al. (arXiv 2026) ran a direct comparison between standard vector RAG and a graph-based alternative on an enterprise code migration task. The results quantify the structural failure precisely. API hallucination rates under standard RAG reached 56.4%, and Dependency Resolution Quality sat at 34.8%. These are not edge-case failures. They represent the baseline behaviour of a chunk-retrieval system operating on relational code.

The same study found that CodeBLEU, one of the most widely used automated metrics for evaluating code generation, scored 91% for both methods. That single finding should give any engineering leader pause. A metric that cannot distinguish between a system with 56% API hallucination and one with 16% is not measuring what matters in production.

The implication is that standard evaluation pipelines will not surface this class of failure. Teams running automated benchmarks on their migration tooling may be receiving confident quality signals from a fundamentally misleading measurement instrument.

How Embedding Models Handle Measurement Context

The dependency problem is compounded by a separate failure mode at the embedding level. Opitz and Michail (arXiv 2026) examined how well embedding models represent physical measurements across 24 models. Their finding is direct: physical measurement relationships are only weakly captured in embedding space, and the patterns that do emerge are driven largely by superficial string similarity rather than semantic equivalence.

The practical consequence for code retrieval is that measurement-heavy contexts, unit conversions, tolerance thresholds, API versioning numbers, configuration constants, are retrieved based on how the strings look rather than what they mean. A query for code handling 100-millisecond timeouts may retrieve chunks about 100-unit quantities of an unrelated type, while missing the semantically relevant timeout logic expressed differently.

Recalibration of similarity scores does not substantially correct this, according to Opitz and Michail (arXiv 2026). The misalignment is not a tuning problem. It reflects how these models were trained and what information their representations actually encode.

Graph RAG as a Structural Alternative

The HCRG Approach

Jaiswal et al. (arXiv 2026) propose a Hierarchical Context-Resident Graph methodology that addresses the topological failure directly. The pipeline uses tree-sitter to extract Abstract Syntax Trees, maps architectural relationships into a property graph, and serialises that structure into context in dependency order. Parent nodes are translated before children, preserving the inheritance chain the LLM needs to reason correctly.

Under this approach, API hallucination dropped from 56.4% to 16.2%, and Dependency Resolution Quality improved from 34.8% to 65.9%. Those are meaningful shifts in the metrics that reflect whether generated code will actually function in a production system.

The Trade-offs That Come With It

Graph RAG is not a cost-free upgrade. Jaiswal et al. (arXiv 2026) document a specific failure mode introduced by the denser context: the LLM exhibits defensive over-engineering, adding complexity that was not present in the source. Cyclomatic Complexity Consistency dropped from 71.6% to 46.7%, and Docstring Preservation declined slightly. Teams adopting graph-based retrieval are trading one class of failure for another, and that trade-off needs to be evaluated deliberately rather than assumed away.

The right framing is not that graph RAG is correct and vector RAG is wrong. It is that different retrieval architectures fail in different ways, and the failure mode of standard RAG in relational code contexts is more damaging to downstream correctness than the over-engineering tendency introduced by graph context.

Building an Evaluation Framework That Catches These Failures

The CodeBLEU masking effect identified by Jaiswal et al. (arXiv 2026) points to a broader evaluation design problem. Text-overlap metrics were designed to measure surface similarity. They will not detect severed inheritance chains, hallucinated APIs, or measurement context retrieved on string pattern rather than semantic equivalence.

An evaluation framework for enterprise code migration needs metrics that operate at the structural level. Dependency resolution accuracy, API reference validity, compilation success rate, and parent-child consistency are the signals that reflect whether the retrieval layer is doing its job. These require purpose-built test harnesses, not off-the-shelf benchmarks.

Engineering leaders evaluating a RAG stack for migration work should treat the evaluation framework as a first-class architectural decision. The retrieval architecture you choose determines which failure modes you are exposed to. The evaluation framework you choose determines whether you can see them.