Why Expanding Your Voice AI to New Languages Costs You the Old Ones

Most enterprise teams approaching multilingual ASR expansion treat it as an additive problem: add a new language, run fine-tuning, ship. What the research and production evidence increasingly show is that this framing is wrong. Expanding a deployed voice AI system to a new language market routinely degrades performance in the languages already live, often without triggering any alerting, because the degradation is gradual and the affected users are not the ones being measured during the rollout.

The Architecture That Makes This Problem Worse

The systems most at risk are connector-based Speech LLMs, which have become a common architecture for enterprise voice AI. In these systems, a trainable connector maps speech representations into a frozen large language model. The LLM itself does not move, which makes the architecture appear stable. The connector, however, carries the full burden of multilingual adaptation.

The problem is that the connector is small relative to the task it is being asked to perform. When you fine-tune it on a new target language, it overwrites the parameter space that was previously encoding source-language acoustic and phonological patterns. This is the mechanism behind catastrophic forgetting in this specific architecture, and it is more acute here than in full-model fine-tuning because there is less redundancy to absorb the update.

Research confirms this is not a theoretical concern. Target-only fine-tuning and even standard TIES merging both cause severe source-language forgetting when applied to the connector in isolation (Zhang et al., arXiv 2026). The implication for engineering leaders is that a vendor claiming multilingual coverage through sequential fine-tuning should be asked directly how they are measuring source-language retention after each expansion cycle.

Why Replay-Based Fixes Are Not Always Available

The standard mitigation for catastrophic forgetting in continual learning is replay: interleave source-language data during the new fine-tuning run to keep the model anchored to its prior performance. This works, and in contexts where you control the training pipeline and retain the original data, it remains a sound approach.

In practice, many enterprise deployments cannot rely on replay. The original training data may be subject to data residency constraints, licensing terms, or simply have not been retained by the vendor who sold you the system. In regulated industries, re-using customer voice data for model updates introduces compliance exposure that legal teams will not accept.

This is the constraint that makes the research on replay-free adaptation directly relevant to procurement decisions. If your vendor's multilingual expansion path depends on replay and you cannot guarantee replay access across all language markets, the expansion strategy has a structural dependency that needs to be surfaced before any contract is signed.

Model Merging as a Structural Alternative

The more technically interesting approaches involve merging separately trained parameter updates rather than sequentially applying them to a shared model. The intuition is that if you train a source-preserving update and a target-acquisition update independently, you may be able to combine them in weight space in a way that retains both objectives.

SSU-Wanda and TIES

TIES merging addresses one of the core failure modes of naive parameter averaging, which is that conflicting updates across different fine-tuning runs tend to cancel each other out in ways that hurt both objectives. SSU-Wanda adds a structured sparsification step before merging, which reduces the interference between updates by isolating the weight regions most relevant to each language. The combination of SSU-Wanda with TIES produces the best source-preserving trade-off in single-target expansion scenarios (Zhang et al., arXiv 2026).

Multi-Target Expansion and Order Sensitivity

Multi-target expansion, where you are adding several languages simultaneously or in rapid succession, introduces a further complication. Update merging across multiple target languages provides stable source retention, which is the safer default. Recursive expansion, where each new language is added on top of the previous expansion, can improve target-language adaptation but is highly sensitive to the order in which languages are introduced (Zhang et al., arXiv 2026). This means that an expansion sequence optimised for one regional rollout plan may perform materially differently if the business changes the order of market entry.

What This Means for Sequencing Language Rollouts

The order-sensitivity finding has a direct operational implication. If your roadmap has six language markets planned over eighteen months, the sequence in which you expand is not just a project management question. It affects the final model quality in each language and the degree of source-language degradation you accumulate along the way.

Engineering leaders should be asking for empirical evidence of source-language performance across each expansion step, not just target-language benchmark scores at launch. A vendor who can only show you word error rate on the new language at go-live, without showing you what happened to the existing languages during that training run, is not giving you the information you need to manage quality at scale.

The practical recommendation is to treat language expansion as a model architecture decision, not a fine-tuning task. That means evaluating whether your current connector-based system has the structural capacity to support the number of languages on your roadmap, and whether the merging strategy your vendor or internal team is using has been validated against source-language retention at each step.

Evaluating Vendor Claims About Multilingual Coverage

Vendor marketing around multilingual voice AI tends to lead with the number of supported languages. That number is meaningful only if it comes with a retention guarantee: that adding language N did not degrade languages one through N minus one below an agreed threshold.

The research on source-aware adaptation makes clear that this guarantee is technically non-trivial to provide. It requires either replay access, a validated merging strategy, or both. A vendor who cannot describe their catastrophic forgetting mitigation approach in specific architectural terms is likely not measuring it systematically.

The questions worth asking in any procurement or architecture review are specific: What connector architecture are you using? How do you measure source-language performance after each expansion run? What merging strategy do you apply when adding new languages, and what is the empirical trade-off profile across your current language set? The answers will distinguish teams who have genuinely solved this problem from those who have not yet encountered it at the scale where it becomes visible.