Building accurate translation-tailored large language models with language-aware instruction tuning

Abstract

Large language models (LLMs) exhibit remarkable capabilities in various natural language processing tasks, such as machine translation. However, the large number of LLM parameters incurs significant costs during inference. Previous studies have attempted to train translation-tailored LLMs with moderately sized models by fine-tuning them on the translation data. Nevertheless, when performing translations in zero-shot directions that are absent from the fine-tuning data, the problem of ignoring instructions and thus producing translations in the wrong language (i.e., the off-target translation issue) remains unresolved. In this work, we design a two-stage fine-tuning algorithm to improve the instruction-following ability of translation-tailored LLMs, particularly for maintaining accurate translation directions. We first fine-tune LLMs on the translation data to elicit basic translation capabilities. At the second stage, we construct instruction-conflicting samples by randomly replacing the instructions with the incorrect ones. Then, we introduce an extra unlikelihood loss to reduce the probability assigned to those samples. Experiments on two benchmarks using the LLaMA 2 and LLaMA 3 models, spanning 16 zero-shot directions, demonstrate that, compared to the competitive baseline—translation-finetuned LLaMA, our method could effectively reduce the off-target translation ratio (up to −62.4 percentage points), thus improving translation quality (up to +9.7 bilingual evaluation understudy). Analysis shows that our method can preserve the model’s performance on other tasks, such as supervised translation and general tasks. Code is released at https://github.com/alphadl/LanguageAware_Tuning.

1. Introduction

Large language models (LLMs) (Min et al., 2023; Li B et al., 2024) have demonstrated excellent performance on a wide range of natural language processing (NLP) tasks, including reasoning (Wei et al., 2023), summarization (Wang YM et al., 2023; Huang et al., 2024), translation (Hendy et al., 2023), understanding (Zhang HB et al., 2022; Zhong et al., 2023), and evaluation (Lu et al., 2024). LLMs exemplified by GPT-3 (Brown et al., 2020), OPT (Zhang SS et al., 2022), LLaMA (Touvron et al., 2023a), and LLaMA 2 (Touvron et al., 2023b), leverage large-scale monolingual data through pre-training with the causal language modeling task, and exhibit strong zero-shot capabilities with a few demonstration examples. Instruction tuning (Mishra et al., 2022; Wei et al., 2022) further elicits the capacity of LLMs to address general tasks directly with proper guidance, such as task definition. However, due to the significant cost to call state-of-the-art proprietary LLMs, such as GPT-4 (OpenAI, 2024), it is attractive to explore strategies for effectively fitting suitably sized LLMs into specific tasks, such as machine translation (Fu et al., 2023; Xu HR et al., 2024b).

In zero-shot translation (ZST) (Gu et al., 2019; Chen et al., 2023; Zan et al., 2023), the objective is to translate sentences from a source language to a target language, where there is either a lack of direct mapping between the source and target languages in the training data, or the target or source languages are absent during training. Addressing the ZST problem is both vital and challenging, especially for low-resource languages. Recent research demonstrates that building translation-tailored LLMs by fine-tuning translation data can achieve superior translation performance (Liu YJ et al., 2023; Xu HR et al., 2024b; Zeng et al., 2024). However, as illustrated in Fig. 1, our preliminary study shows that when tackling zero-shot directions, translation-tailored LLMs often encounter the off-target translation problem, where the generated translations are in the wrong languages.