Building accurate translation-tailored large language models with language-aware instruction tuning

• A two-stage fine-tuning algorithm with instruction-conflicting samples and unlikelihood loss reduces off-target translations in zero-shot directions. • Up to 62.4 percentage points reduction in off-target translation ratio on LLaMA 2 and LLaMA 3 across 16 zero-shot directions. • Up to +9.7 BLEU improvement in translation quality compared to a competitive translation-finetuned baseline. • The method preserves model performance on supervised translation and general tasks, ensuring no degradation of other capabilities.