Cross-lingual knowledge transfer is essential for multilingual models when target-language data is scarce, but existing methods require large parallel corpora, translation systems, auxiliary models, or additional training stages that are unavailable for many languages.

LINK is a data-level intervention applied during pretraining: in a portion of the high-resource English corpus, randomly selected words are replaced with word-level translations according to a replacement ratio, using only a bilingual vocabulary. The intervention requires no additional model training and no parallel data or translation systems; a bilingual vocabulary can be obtained at near-zero cost for virtually any language. The method improves knowledge transfer for downstream tasks involving scientific reasoning, commonsense inference, and world knowledge. Evaluation across eight languages and five model sizes shows notable improvements on target-language downstream tasks, including up to a 2x speedup in training to reach equivalent performance. The work is by Anastasiia Sedova, Natalie Schluter, Skyler Seto, and Maartje ter Hoeve.