3 notes
#translation #ai #localization
Please see the following slide deck for a description of the process and potential benefits of using LLMs to generate and localize translator’s notes for the rest of the Bible.
#translation #ai
I’ve been thinking about how to improve LLM performance for Bible translation, especially for low-resource languages. The current approaches aren’t great - LLMs struggle with the nuances of these languages and often produce poor translations. This is a big problem since we need translations that respect both cultural and linguistic aspects.
The first thing I tried was combining traditional rule-based grammars with modern LLMs. Rule-based grammars are pretty rigid on their own, but they give us a solid structure to work with. When you mix this with an LLM’s ability to understand context, you get something much more useful. The grammar provides guardrails while the LLM handles the nuanced language understanding.
Here’s an interesting twist - we can actually use LLMs to help write grammars for low-resource languages. Instead of just using them as translators, we’re using them to build the linguistic frameworks we need. I think this could be a game-changer for working with lesser-known languages.
The cool part is seeing how these old-school grammar rules work with modern LLM capabilities. What used to be seen as outdated becomes really powerful when combined with the way LLMs process context and meaning.
I’ve been looking into using context-free grammars (CFGs) like GBNF to control LLM outputs. This approach, called grammar-constrained decoding (GCD), works particularly well for structured content.
What does this actually look like? The best example I’ve come across so far is the GBNF grammar approach found in the llama.cpp library. They give the following example of a formal ‘grammar’ for specifying that the only outputs that an LLM can give must be Japanese characters:
# A probably incorrect grammar for Japanese
root ::= jp-char+ ([ \t\n] jp-char+)*
jp-char ::= hiragana | katakana | punctuation | cjk
hiragana ::= [ぁ-ゟ]
katakana ::= [ァ-ヿ]
punctuation ::= [、-〾]
cjk ::= [一-鿿]Research has revealed several key advantages of using grammar constraints:
However, there are some challenges to consider:
This hybrid approach isn’t just a theoretical exercise; it’s a practical solution to a real-world problem. By harnessing the strengths of both rule-based grammars and the adaptive capabilities of LLMs, we can make significant strides in Bible translation for low-resource languages. It’s a step towards leveraging powerful probabilistic models in a way that does not clobber much rarer and not-yet-digitized languages.
In conclusion, the integration of rule-based grammars with LLMs presents a promising avenue for enhancing Bible translation into low-resource languages. It’s a testament to the power of combining traditional methods with cutting-edge technology – a blend of the old and the new to achieve something truly remarkable in the realm of linguistic translation.
I’ll post some updates when I finally get around to applying the OpenText 2.0 grammar definition to an LLM. I’m excited to see what happens!
The landscape of grammar constraints has evolved significantly since this article was first written. Some key developments include:
Rather than just relying on explicit grammar rules, newer models like OpenAI’s o-series demonstrate the ability to perform structured reasoning during inference time. This allows for more naturally constrained outputs without requiring formal grammar definitions.
Modern constraint systems like LMQL now combine:
This makes it much more practical to implement grammar constraints while maintaining the flexibility of modern LLMs.
Labs are increasingly using synthetic data to bake structural constraints directly into model training. As noted in the Phi-4 technical report, this creates more direct relationships between tokens and helps models learn cleaner reasoning patterns.
Recent research has introduced more sophisticated approaches to grammar constraints:
#translation #ai
For any machine translation technique to succeed requires lots of data. When it comes to low-resource languages, you simply don’t have lots of data. For this reason, we need to explore other methods and techniques that are somewhat orthogonal to the traditional approach of training a model on a large corpus of data.
Many AI-based translation approaches are based on the idea of transfer learning. This is where you train a model on a large corpus of data, and then fine-tune it on a smaller corpus of data. The idea is that the model will have learned some generalizable features from the large corpus, and then it can learn the specific features of the smaller corpus. This is a great approach, but it still requires a large corpus of data to train on.
Ultimately, we’re going to need something quite different to achieve high-quality translation for low-resource languages.
One of the biggest challenges with AI-based translation is that it works like a black box. You feed in some data, and you get some data out. But you don’t really know what’s going on inside the black box. You don’t know what the model is learning, or how it’s learning it. You don’t know what features it’s learning, or how it’s combining them to produce the output. This should rightly make translators somewhat nervous. How can you trust the output of a model that you don’t understand?
My vision for translation assistance with AI is thus to pry open the black box by using LLM-based predictions, rather than simple sequence-to-sequence predictions. This is a fundamentally different approach, and it relies on the concept of in-context learning, rather than a fine-tuning or transfer learning approach.
In short, in-context learning involves providing “training data” to the model on the fly, in the prompt. (Read more from Stanford AI Lab).
This is a fundamental difference that is really critical for translation assistance. It means we don’t need to take a bunch of data (e.g., a complete New Testament translation) in order to fine-tune a model for generating more data (e.g., a complete Old Testament translation). Instead, we can take a small amount of data (e.g., a few verses of the New Testament) and use it to generate more data (e.g., the next few verses of the New Testament, or some similar verses being drafted in the Old Testament).
We get two big benefits from this approach:
To leverage in-context learning, we need to be able to draft new translations using few-shot examples (e.g., use a few already-translated verses to draft a new verse). We also need to be able to evaluate those translations using few-shot examples (e.g., use a few already-translated verses to evaluate a new verse). Optionally, we might want or need a back-translation in order to enable third party evaluation (such as a translation consultant).
Working backwards, then:
valid_tokens (see Token matching evaluation)valid_tokensTranslation assistance can leverage LLMs and in-context learning to open up the black box of AI-based translation. This approach can help us bootstrap our way to a complete translation in a manner that cannot be achieved using traditional machine translation and typical AI-based approaches such as sequence-based model fine-tuning, and it can accept instant feedback from the translator, which means that every single correction or improvement made by the translator gets leveraged in the very next verse or chunk being drafted.
Note: you can also use an approach like this for a multi-agent simulation. See also the Social Approach to Low-Resource Language Translation.