Retrieval-Augmented Generation in High-Stakes Domains: Lessons from Surgical Scene Captioning That Apply Across Healthcare AI

RAG has become a reliable architectural pattern for grounding language model outputs in external knowledge, and most enterprise implementations work well enough for the use cases they were designed for. The problem is that most were designed for low-stakes retrieval: finding a policy document, surfacing a product FAQ, summarising a support ticket. Healthcare AI operates under different constraints. The retrieved context carries clinical meaning, the errors carry clinical consequence, and the people who need to trust the system outputs are rarely in a position to interrogate the retrieval logs. Surgical scene captioning makes these constraints unusually visible, and the engineering decisions it forces are worth examining carefully by anyone building RAG-based clinical applications.

Why Document Retrieval Is the Wrong Mental Model for Clinical RAG

Most RAG implementations treat retrieval as a search problem: embed a query, find the nearest chunks in a vector store, concatenate them as context. This works when the underlying knowledge is document-shaped, meaning relatively self-contained, semantically dense, and tolerably forgiving of partial matches.

Clinical knowledge is rarely document-shaped. A surgical scene involves a set of instruments, a set of anatomical targets, and a set of actions connecting them. The meaningful unit of retrieval is not a text passage but a relational structure: which instrument is acting on which tissue, in what sequence, in what anatomical region. Retrieving the wrong relational context produces outputs that are linguistically plausible but factually incorrect, and in a clinical setting that distinction matters considerably.

The GRAC framework addresses this by representing surgical scenes as graphs and retrieving graph-caption pairs rather than text chunks (Dhakal et al., arXiv 2026). The system then compares instrument-tissue interaction patterns between the retrieved examples and the input scene before conditioning the caption generator. The result is a retrieval mechanism that is sensitive to relational structure rather than surface similarity, and the performance improvement over transformer-based baselines is substantial enough to suggest this is not a marginal refinement.

Constructing a Domain Knowledge Graph That Captures Clinical Relationships

The quality of a graph-based retrieval system is determined upstream, at the point where the knowledge graph is constructed. In surgical captioning, the nodes are instruments, anatomical structures, and actions; the edges encode directional relationships between them. Getting those edges right requires domain expertise that cannot be approximated from general medical text.

This is a recurring pattern in healthcare AI: the representation layer encodes assumptions that determine what the model can and cannot distinguish. A knowledge graph that conflates anatomically adjacent structures, or that uses inconsistent action terminology across annotators, will produce retrieval errors that are invisible to standard accuracy metrics but clinically significant. Annotation protocols need to be designed with the same rigour applied to label quality in supervised learning.

The practical implication is that knowledge graph construction is not an engineering task that can be handed off without clinical input. It requires a structured collaboration between domain experts and ML engineers, with explicit agreement on the ontology before annotation begins. Retrofitting ontological consistency after the fact is expensive and frequently incomplete.

Context Retrieval Quality as a Patient Safety Variable

In a standard enterprise RAG system, retrieval quality affects response usefulness. In a clinical system, it affects whether the output can be safely acted upon. That distinction changes the engineering calculus around retrieval evaluation.

Triplet-set accuracy, the metric used in GRAC to measure exact instrument-action-target agreement, is a more clinically meaningful measure than BLEU or ROUGE scores (Dhakal et al., arXiv 2026). A caption that scores well on n-gram overlap but misidentifies the anatomical target of an action is not a near-miss; it is a factual error about a clinical event. Evaluation frameworks for clinical RAG systems need at least one metric that tests factual agreement at the entity level, not just linguistic similarity.

Retrieval failure modes in clinical contexts also tend to cluster in ways that matter. Edge cases in surgical scenes, unusual instrument combinations, anatomically complex regions, or rare procedural variants, are precisely the cases where retrieval is most likely to surface a poor match. These are also the cases where accurate documentation is most clinically important. A retrieval system that performs well on common cases but degrades on rare ones is not a system that can be deployed without careful scope constraints.

Hallucination Failure Modes Specific to Medical Language

General-purpose language models hallucinate in ways that are often detectable: invented citations, implausible facts, obvious contradictions. Medical language hallucinations are harder to detect because the outputs remain within the plausible vocabulary of clinical documentation. A model that generates a caption describing the wrong instrument acting on the correct tissue produces text that reads as entirely credible to a non-specialist reviewer.

This is why retrieval conditioning matters beyond accuracy metrics. When the generator is conditioned on retrieved relational context, as in the GRAC architecture, the failure mode shifts from unconstrained confabulation toward retrieval-conditioned error. The latter is more tractable: it can be audited by examining what was retrieved and why, rather than trying to characterise the generative model's internal representations. Explainability in clinical AI is not primarily a regulatory checkbox; it is a mechanism for catching systematic errors before they propagate.

The implication for system design is that the retrieval log needs to be a first-class artefact, not a debugging tool. Clinical reviewers need to be able to see what context was retrieved for a given output, in a form they can interpret. Building that interface is an engineering investment, but it is the investment that makes the system auditable.

Evaluation Frameworks That Carry Clinical Weight

Evaluation for clinical RAG needs to operate at multiple levels simultaneously. Linguistic quality metrics confirm that the outputs are coherent and grammatically appropriate. Entity-level accuracy metrics confirm that the clinical facts are correct. And coverage metrics confirm that no clinically significant event in the scene has been omitted from the description.

None of these alone is sufficient. A system that scores well on entity accuracy but produces awkward or ambiguous phrasing creates downstream interpretation risk. A system that produces fluent text but omits key procedural steps fails the documentation purpose entirely. The evaluation framework needs to be designed to surface failure on any of these dimensions independently, so that improvements in one area do not mask regressions in another.

The harder evaluation question is how to validate performance on the distribution of cases the system will actually encounter in deployment, rather than on the benchmark distribution used during development. This is a problem common to all clinical AI systems, and the answer is the same: prospective evaluation on held-out data that reflects the target deployment environment, with subgroup analysis across procedure types, instrument sets, and anatomical regions. Benchmark performance on EndoVis-18 or TORS is a useful signal, but it is not a deployment readiness assessment.