Transfer Learning in Medical Imaging: What Healthcare AI Teams Get Wrong Before They Write a Single Line of Code

Most healthcare AI teams arrive at transfer learning with a reasonable instinct and a flawed execution plan. The instinct is correct: pre-trained weights accelerate training, reduce labelled data requirements, and give models a head start on low-level feature extraction. The execution fails because those teams import assumptions from natural image benchmarks without examining whether those assumptions hold for clinical datasets. Mammography is the clearest case study for why that matters. The structural properties of mammographic data - class imbalance at population scale, subtle inter-class differences invisible to ImageNet-trained features, and label noise introduced by radiologist variability - break standard transfer learning pipelines in ways that only surface at validation time, not during development.

Companion piece to our broader work on diagnostic imaging AI. See Building AI for Diagnostic Imaging: What Works, What Breaks, and What Regulators Will Ask for a full treatment of the engineering and regulatory gap between benchmark performance and clinical deployment.

The Assumption Gap Between ImageNet and Clinical Cohorts

Transfer learning from ImageNet-pretrained models rests on a specific premise: that low-level features learned from natural images generalise to the target domain. For many vision tasks, that premise holds well enough. For mammography, it holds partially and fails where it matters most.

Natural images contain high-contrast edges, colour variation, and object boundaries that make class separation relatively straightforward. Mammographic images are greyscale, high-resolution, and dominated by subtle tissue density differences where the diagnostic signal is often a marginal variation in parenchymal texture. The features that make ImageNet representations useful - colour histograms, object contours, spatial layout - contribute little to breast density classification or lesion characterisation.

The more consequential failure is distributional. ImageNet class distributions are roughly balanced by design. Clinical screening datasets are not. In population-level mammography cohorts, the distribution of breast density categories is inherently skewed, and malignant findings are rare relative to benign or normal cases. A model initialised on balanced natural image weights and fine-tuned with standard cross-entropy loss will learn to optimise for majority-class accuracy, producing metrics that look acceptable in aggregate while performing poorly on the clinically important minority classes.

Why Dataset Difficulty Signals Should Drive Model Selection

The conventional response to class imbalance in medical imaging is focal loss: down-weight easy examples, up-weight hard ones, tune the gamma hyperparameter until validation metrics improve. This approach works, but it introduces a problem that compounds at scale. The hyperparameter that produces good performance on one dataset does not transfer reliably to another, and clinical datasets vary substantially in their difficulty distributions across sites, scanners, and patient populations.

A more principled approach treats dataset difficulty as a signal to be measured rather than a parameter to be tuned. Panambur et al. (arXiv 2026) demonstrate this with the Dataset-Informed Transfer Learning (DITL) framework applied to mammography classification. DITL estimates per-sample difficulty by measuring label purity in the k-nearest neighbourhood of each sample within a self-supervised feature space. Samples surrounded by neighbours with inconsistent labels are treated as inherently ambiguous and weighted accordingly during training, without requiring manual hyperparameter specification.

The practical implication for engineering teams is significant. Difficulty-weighted training that derives weights from the dataset's own geometry generalises across cohorts in a way that fixed focal loss parameters do not. It also surfaces label noise as a first-class signal rather than treating it as irreducible variance, which matters when your training labels come from retrospective radiologist reads with known inter-observer variability.

Neighbourhood-Informed Representation Learning at Population Scale

Standard triplet loss formulations enforce a fixed margin between positive and negative pairs in embedding space. That fixed margin assumes a consistent notion of inter-class distance across the dataset, which does not hold when the dataset spans multiple acquisition sites, scanner manufacturers, or patient demographics. Margins calibrated on one subset of the data will be either too tight or too loose for other subsets.

DITL addresses this with an adaptive neighbourhood representation triplet objective that learns the margin jointly with the model weights (Panambur et al., arXiv 2026). The margin adapts to the local geometry of each training batch, enforcing intra-class compactness and inter-class separation in proportion to the actual difficulty of the examples being processed. On the VinDR-Mammo dataset - a large-scale clinical cohort with the distributional properties of a real screening programme - this approach produced statistically significant improvements in accuracy, F1, and AUC compared to standard transfer learning baselines.

The engineering lesson is not that DITL is the only valid approach. It is that representation learning objectives designed for population-level clinical data behave differently from objectives designed for balanced benchmark datasets, and that the difference is not marginal. Teams that evaluate model selection on small, curated validation sets and then deploy to population-scale cohorts are comparing performance under conditions that do not match.

Infrastructure Choices That Determine Generalisation

Model architecture and loss function choices are visible decisions that get documented and debated. Infrastructure choices are quieter but often more consequential for whether a diagnostic AI system generalises beyond its training set.

The first infrastructure decision is feature space construction for difficulty estimation. Methods like DITL require a self-supervised feature space in which neighbourhood relationships are meaningful. That feature space needs to be constructed from representations that capture clinically relevant variation, not just low-level image statistics. Teams that use generic ImageNet features as the basis for neighbourhood computation will produce difficulty estimates that reflect natural image similarity rather than clinical similarity. The feature extraction backbone used for difficulty estimation should be pretrained on mammographic or at minimum medical imaging data.

The second decision is validation dataset design. Internal validation on a held-out split of the training cohort measures memorisation avoidance, not generalisation. Genuine generalisation testing requires an external cohort with different scanner hardware, different acquisition protocols, or a different patient demographic. The gap between internal and external validation performance is consistently larger for imaging AI than for other clinical AI categories, and it is largely driven by distribution shift that internal splits cannot capture.

The third decision is label quality infrastructure. Difficulty-weighted training methods amplify the signal from hard examples. If hard examples are hard because of genuine clinical ambiguity, that is appropriate. If they are hard because of labelling errors, inconsistent annotation protocols, or single-reader labels without adjudication, the weighting mechanism will amplify noise rather than signal. Label quality assurance is not a preprocessing step that can be deferred. It determines whether the difficulty signal is informative or misleading.

What a Dataset-Aware Architecture Looks Like in Practice

A dataset-aware transfer learning pipeline for mammography classification starts with source model selection informed by domain proximity, not benchmark rank. A model pretrained on a large medical imaging corpus will produce more clinically relevant initial representations than an ImageNet champion, even if its ImageNet accuracy is lower.

Training objective design follows from dataset characterisation, not from defaults. Before writing training code, the engineering team should measure the class distribution of the training cohort, estimate label noise through inter-annotator agreement analysis where multiple reads are available, and construct the self-supervised feature space needed for neighbourhood-based difficulty estimation. These measurements determine whether standard cross-entropy is adequate, whether difficulty weighting is necessary, and what margin behaviour the representation learning objective should target.

Validation architecture should be specified before training begins. The external validation cohort, the subgroup analyses covering relevant demographic and scanner variables, and the performance thresholds that define clinical adequacy should all be fixed in advance. Post-hoc validation design, where the evaluation methodology is adjusted after seeing model performance, produces results that regulators will scrutinise and that will not survive external replication. The dataset-aware approach applies to evaluation as much as to training.