Can I use LLMs to translate or localize conversational AI experiences?

🌈 Abstract

The article discusses the importance of engaging professional translators when expanding conversational AI experiences to multiple languages, rather than relying solely on language models (LLMs) for translation. It highlights the various aspects of bot translation that require careful consideration, such as translating the bot's name, local/regional expressions, buttons, regex patterns, NLU setup, and accounting for linguistic and cultural differences. The article provides evidence from research studies demonstrating the limitations of machine-only translations and the potential risks they pose. It then outlines the best practice of hiring a full team of language experts to ensure a high-quality translation, or at least engaging an expert translator as a workaround. The article also suggests alternative solutions, such as providing customers with direct access to language-specific support, if the resources for a full translation are not available.

🙋 Q&A

[01] The Importance of Engaging Translators

1. What are the key aspects of bot translation that require careful consideration beyond just translating the words?

• The bot's name, which needs to capture the essence of the name in the target language
• Local/regional expressions and idioms that need to be translated to preserve the persona and tone
• Buttons and other interface elements that need to be translated within character limits
• Regex patterns and NLU setup to ensure the bot understands the user correctly in the target language
• Accounting for linguistic and cultural differences, such as inclusive vs. exclusive "we" in Tamil

2. What are the limitations of using machine-only translations, as demonstrated by research studies?

• Machine translations can be highly inaccurate, with accuracy varying greatly by language (e.g., Spanish 94%, Tagalog 90%, Korean 82.5%, Chinese 81.7%, Farsi 67.5%, Armenian 55%)
• Machines struggle to handle context and ambiguity, leading to translations that can be nonsensical or even dangerous (e.g., translating "ibuprofen" to "anti-tank missile")
• Machine-translated text may not be natural or clear in the target language, making it difficult for users to understand

3. Why is it important to have a well-thought-out strategy for handling multiple languages in a conversational AI system?

• If the system is not designed to properly handle language switching, it may refuse to speak the requested language or provide responses that are not appropriate for the language (as seen in the example of the bot that refused to speak Hindi)
• Proper language handling is crucial for providing a great customer experience and avoiding frustration or confusion for users

[02] Best Practices for Translating Conversational AI

1. What is the ideal practice for translating an existing bot to another language?

• Hire a full team of language experts, including a conversation designer, bot tuner, developers, testers, and conversation analysts, in the target language

2. If the ideal practice is not feasible, what is the recommended workaround?

• Hire an expert translator and educate them on conversation design, bot tuning, internal testing, user testing, and post-launch analysis, iteration, and maintenance
• The translator can use LLMs to create a first draft translation and then polish it with their expertise

3. What are some alternative solutions if the resources for a full translation are not available?

• Provide customers with a direct way to connect to language-specific support, such as a button to connect to a Spanish-speaking agent
• Design the conversational AI system to elegantly handle language switching and direct users to appropriate language support