Large language models have revolutionized natural language processing, with recent models like Tower catering to translation tasks in 10 languages. Developed by researchers at Unbabel, SARDINE Lab, and MICS Lab, Tower outperforms other open-source models and offers features like automatic post-editing and named-entity recognition. The researchers aim to release TowerEval for evaluating language models against Tower’s standards.
“`html
Unbabel Releases Tower: A Multilingual 7B Parameter Large Language Model (LLM) Optimized for Translation-Related Tasks
With the growth of large language models, natural language processing has been revolutionized. Many LLMs, like GPT-3.5, LLaMA, and Mixtral, came up last year, which helped tackle diverse language tasks. However, open-source models have lacked reliable models for translation tasks.
Introducing Tower: A Multilingual Translation Solution
A collaboration between researchers of Unbabel, the SARDINE Lab at Instituto Superior Técnico, and the researchers of the MICS lab at CentraleSupélec, University of Paris-Saclay, has created a new multilingual model Tower. This Llama 2-based multilingual LLM has 7B parameters specifically designed for translation-related tasks. The main highlight of this model is that, unlike other open-source models, which are predominantly built with English data, Tower supports 10 languages.
In addition to multilingual translation, Tower also has capabilities for pre-translation activities, like grammar improvement, to translation assessment jobs, like machine translation and automatic post-editing. The researchers found that this model performed better than the state-of-the-art counterparts in translation and better than alternative open-source solutions.
How Tower Was Formulated
The researchers used two stages to formulate Tower: extended pre-training and instruction tuning. They used continued pre-training to enhance LLaMA2’s proficiency in non-English languages, while instruction tuning improved its performance in addressing particular problems without prior experience.
The second step of instruction tuning enhanced the model’s ability to handle specific tasks at a higher level in a 0-shot fashion. They developed a dataset named TowerBlocks for supervised fine-tuning. This dataset helped the model to maintain competency across various translation-related tasks by providing prompts for all tasks, including zero and few-shot templates.
Practical Applications and Future Developments
TowerInstruct can be a significant step in multilingual machine translation as it outperforms other models. Its features, including automatic post-edition, named-entity recognition, or source error correction, can be very helpful in this domain. The researchers are also looking forward to the release of TowerEval, an evaluation repository focused on machine translation and related tasks.
For more information, visit the Model and Reference Blog.
AI Solutions for Middle Managers
If you want to evolve your company with AI, stay competitive, and use AI to your advantage, consider Unbabel’s Tower for multilingual translation tasks. AI can redefine your way of work by identifying automation opportunities, defining KPIs, selecting AI solutions, and implementing them gradually.
Spotlight on a Practical AI Solution: AI Sales Bot
Consider the AI Sales Bot from itinai.com/aisalesbot designed to automate customer engagement 24/7 and manage interactions across all customer journey stages.
For AI KPI management advice, connect with us at hello@itinai.com. For continuous insights into leveraging AI, stay tuned on our Telegram channel or Twitter.
Discover how AI can redefine your sales processes and customer engagement. Explore solutions at itinai.com.
“`
