TRANSMI: A Machine Learning Framework to Create Baseline Models Adapted for Transliterated Data from Existing Multilingual Pretrained Language Models mPLMs without Any Training

TRANSMI: A Machine Learning Framework to Create Baseline Models Adapted for Transliterated Data from Existing Multilingual Pretrained Language Models mPLMs without Any Training

The Challenge in Multilingual NLP

The increasing availability of digital text in diverse languages and scripts presents a significant challenge for natural language processing (NLP). Multilingual pre-trained language models (mPLMs) often struggle to handle transliterated data effectively, leading to performance degradation.

Current Limitations

Models like XLM-R and Glot500 perform well with text in their original scripts but struggle significantly with transliterated text due to ambiguities and tokenization issues. These limitations degrade their performance in cross-lingual tasks, making them less effective when handling text converted into a common script such as Latin.

Introducing TRANSMI Framework

TRANSMI is a framework designed to enhance mPLMs for transliterated data without requiring additional training. It modifies existing mPLMs using three merge modes—Min-Merge, Average-Merge, and Max-Merge—to incorporate transliterated subwords into their vocabularies, thereby addressing transliteration ambiguities and improving cross-lingual task performance.

Performance and Results

TRANSMI-modified models outperform their original versions in various tasks like sentence retrieval, text classification, and sequence labeling. This modification ensures that models retain their original capabilities while adapting to the nuances of transliterated text, thus enhancing their overall performance in multilingual NLP applications.

Impact and Future Potential

TRANSMI offers a practical and innovative solution to a complex problem, providing a strong foundation for further advancements in multilingual NLP and improving global communication and information processing.

Evolving with AI Solutions

If you want to evolve your company with AI, stay competitive, and use TRANSMI to create baseline models adapted for transliterated data from existing mPLMs. Discover how AI can redefine your way of work, identify automation opportunities, define KPIs, select an AI solution, and implement gradually.

Spotlight on a Practical AI Solution:

Consider the AI Sales Bot from designed to automate customer engagement 24/7 and manage interactions across all customer journey stages. Explore how AI can redefine your sales processes and customer engagement at

For AI KPI management advice, connect with us at And for continuous insights into leveraging AI, stay tuned on our Telegram or Twitter @itinaicom.

List of Useful Links:

AI Products for Business or Try Custom Development

AI Sales Bot

Welcome AI Sales Bot, your 24/7 teammate! Engaging customers in natural language across all channels and learning from your materials, it’s a step towards efficient, enriched customer interactions and sales

AI Document Assistant

Unlock insights and drive decisions with our AI Insights Suite. Indexing your documents and data, it provides smart, AI-driven decision support, enhancing your productivity and decision-making.

AI Customer Support

Upgrade your support with our AI Assistant, reducing response times and personalizing interactions by analyzing documents and past engagements. Boost your team and customer satisfaction

AI Scrum Bot

Enhance agile management with our AI Scrum Bot, it helps to organize retrospectives. It answers queries and boosts collaboration and efficiency in your scrum processes.