#methodology

In a room of around forty consultants, one I respect raised five objections to “AI in Bible translation” in the space of about ten minutes: it can’t use the Greek and Hebrew, it has no dictionary, it doesn’t know genre, you can’t give it a translator’s prompt, and you can’t feed it small, controllable batches of text. Every one of those is a fair complaint. None of them is about the tool we’re actually using.

They’re objections to Google Translate.

Seq2seq machine translation—the architecture behind Google Translate and its era-mates—really did work that way. It mapped strings to strings with no access to source-language grammar as an object you could reason about, no lexical resources beyond whatever got baked into training, no sense of genre or register beyond statistical correlation, and no mechanism for a human to say “translate this the way we discussed last week” and have it stick. If that’s the AI you have in mind, the objections aren’t just fair, they’re the whole story. I’ve spent enough time watching seq2seq output wander off-register mid-sentence to know the scar tissue is earned.

But a modern LLM can read the Hebrew, hold a conversation about a difficult word’s semantic range, look up a lexicon entry you hand it in context, take a translator’s brief as a literal system prompt, and work in exactly the batch size a team wants—a verse, a paragraph, a pericope. It can be shown the genre and asked to translate accordingly. None of this is hypothetical; it’s the ordinary operating mode of the tools I use most weeks. So when I hear the same five objections raised against “AI,” I want to redirect: every objection I hear to “AI in Bible translation” is actually an objection to Google Translate. Which means the objection is sound, but the target has moved.

I don’t think this makes the skepticism illegitimate. Consultants built their caution around real failure modes, over years of watching tools get chosen from the top down and pushed into low-resource languages even after the field had already seen them produce fluent-sounding nonsense. That caution is a feature of the field, not a bug to route around. The useful move isn’t to say “you’re wrong, AI is great now”—it’s to ask which specific objection still holds against a system that can read source text, hold a controllable-length conversation, and be handed a prompt like any other collaborator. Some of them, updated for the new architecture, still land. Most of the five I listed above don’t survive contact with what an LLM can actually do, though—which is a separate question from whether the output is good, a question I’ve tried to be more precise about before.

That precision question is where I get more cautious, not less. There’s a pull toward settling all of this with a rigorous benchmark—controlled comparison, blinded reviewers, speed and quality scored against a fixed rubric. I understand the appeal; it’s the responsible-sounding thing to do. But I’ve come to distrust it for a specific reason: by the time you finish the rigorous benchmark, the thing you benchmarked is obsolete. The workflow, the model, the prompting approach—all of it will have shifted meaningfully in the months a careful study takes to design, run, and write up. What you publish is a precise measurement of a tool nobody is using anymore, dressed up in the authority of a number. I’d rather have a translator’s rough, unrigorous sense of “this draft used to take me a day and a half, now it takes an afternoon” than a beautifully controlled study of last year’s pipeline. It’s less defensible on paper and more true in practice—false precision is still false.

That same instinct toward “faster at what, exactly” applies to the bigger debate hovering over all of this: is AI-assisted translation faster than the traditional process? I’ve started to think that’s the wrong question, or at least an underspecified one. Traditional translation and AI-assisted translation quite often aren’t racing toward the same finish line. A team producing a digital-only translation for a community that needs Scripture in an app this year has a different goal than a team producing a fully checked, print-ready Bible meant to anchor a language’s translation for a generation. Comparing their speed head-to-head is a bit like timing a sprinter against a marathoner and declaring one of them slow. “Is AI faster than traditional translation?” is the wrong question. Faster at what—and toward whose goal? This is the same move I made in thinking through Occam’s razor and translation: the interesting question usually isn’t which process is objectively better, it’s what inputs and outputs a given goal actually requires, and whether the process in front of you is shaped for that goal or just inherited from a different one.

I don’t think any of this settles the underlying argument about whether AI belongs in Bible translation workflows at all—that’s a longer conversation, and a more contested one than I’m trying to resolve here. But I’d like the argument to be about the tool that’s actually in the room. Google Translate earned its objections. The thing translators are using now is a different animal, and it deserves to be evaluated as one—carefully, skeptically, and on its own terms, rather than refuted by proxy.

Every demo I give starts with the same reflex from the audience: they want to see the model translate a hard verse well. That’s the “sparkle button” moment, and I used to think it was the whole pitch. Increasingly I don’t. A lot of the friction in a real translation project has nothing to do with translation quality at all — it’s project management. Who’s done Genesis. Who’s approved Exodus. Which reviewer has the current draft of Leviticus, and has anyone told them Numbers is ready. If we can point AI at that, it’s arguably more valuable than a better draft of any single verse, because it’s friction every project pays regardless of how good the translation itself is.

I don’t have a rigorous number for how large that share is, so I want to be careful here rather than confident. The figure I keep coming back to — heard from Reinier, who built Paratext, the software most of the field’s translation project management already runs on — puts project-management overhead at somewhere around 36% of total translation friction. I’m citing that because it matches everything I’ve seen firsthand, not because I’ve independently audited it, and I’d rather flag that than dress up a remembered conversation as a verified statistic.

Whatever the precise number, the shape of the claim matches what I’d expect from watching real projects. Translators lose time to status-chasing, handoff ambiguity, and duplicate work far more than they lose time to a model producing a mediocre first draft — a bad draft gets fixed in minutes; a lost handoff can stall a book for weeks. If that’s roughly right, it reframes where the leverage is. The sparkle button is a nice demo. The unglamorous version — an agent that knows who has what, what’s blocking what, and what needs a nudge — is the one that actually compounds across a whole Bible.

I have one real data point that makes this concrete rather than theoretical, and I want to present it more conservatively than the raw comparison suggests, because the comparison is easy to oversell. A team of two translators completed a full Bible translation in about a year. Sagamore Institute’s September 2022 report, A Study of Cost and Use of Funds in Bible Translation — commissioned by the Maclellan Foundation for illuminNations Resource Partners, based on self-reported data validated against audited financials — put the field average at roughly 15.8 years and $937,446 per complete Bible.

Even being conservative about it, that’s something like two, three, or four times faster than the field average, not fifteen times faster, and I think the smaller number is the honest one. The comparison isn’t apples to apples in several ways that matter: this was a high-resource language with existing reference translations to draw on, the two translators were already professionals, and “done” here means digital-ready, not print-ready — print typically adds another one and a half to two and a half years of typesetting, formatting, and final checks on top. A fifteen-year average almost certainly includes projects in far lower-resource languages, with far less existing scaffolding, and I don’t want to imply this team solved a problem that’s actually much harder in most of the places Bible translation happens.

What I do think the comparison shows is that the acceleration is real and worth taking seriously, even after you strip out every favorable condition you can find. And notably, neither of the two speed levers here — the model’s draft quality, or the reduced project-management overhead — did all the work alone. The team wasn’t just generating faster; per the pattern I described in Steering, Not Batch-and-Correct, they were correcting in context, which is itself a project-management behavior as much as a translation one — it’s about how work flows from one verse to the next, not just how good any single prediction is. I’d guess if I could actually decompose their year into “time saved on drafting” versus “time saved on not losing track of where things stood,” the second number would surprise people who assume this is purely a model-quality story.

None of this is a claim that Bible translation project management is a solved problem, or that 36% (or any other number) is the right way to carve it up. It’s more that I keep being drawn back to the boring part of the workflow as the place with the most unclaimed leverage, in a field where everyone — myself included, most days — wants to talk about the model instead.

I’ve started thinking about AI-assisted translation the way I think about flying a plane. Not takeoff and landing — the boring middle part, where the whole job is thousands of tiny course corrections. A pilot doesn’t fly a thousand miles off heading and then swing back. They nudge, constantly, and each nudge changes the trajectory for everything that follows. That’s steering. The alternative — fly wherever the plane happens to go, then fix the path after landing — isn’t really an alternative, it’s a category error. But it’s more or less how most people still do AI-assisted translation: generate a draft, then post-edit it.

The generate-then-fix pattern is everywhere in machine translation tooling, and for good reason — it’s simple, it maps onto existing editorial workflows, and for a lot of content it’s fine. But I’ve become convinced it’s the wrong default for anything that depends on in-context learning, which is most of what I work on. Here’s the distinction I keep coming back to: when you correct a translation in context, before the model has moved on, that correction becomes part of the context for every subsequent prediction. When you batch-generate first and correct afterward, the correction lands in a document, not in the model’s working context for the next verse. It’s informative to a human reviewer. It’s close to invisible to the model.

Concretely: imagine you finish Genesis 1:1, and you corrected three things — a term choice, a syntactic pattern, an idiom. Am I gonna have to correct those same three things in every other verse where they show up? That would be crazy. If the correction is made in-context and the model sees it before generating Genesis 1:2, it doesn’t have to be. If it’s made afterward, in a separate editing pass, it does — you’ll be fixing the same idiom for the two-hundredth time in Deuteronomy, wondering why the model “isn’t learning.” It isn’t that the model can’t learn. It’s that batch editing never gave it the chance.

Whether this matters depends a lot on the stakes. For low-stakes content — app UI copy, a button label, something a user skims and moves past — generate-and-fix is genuinely fine. Nobody’s faith or family history hinges on a “Submit” button. But for high-trust, educational, or scriptural content, the calculus flips, because a fluent wrong answer is a different kind of failure than an obviously bad one. A garbled, clunky translation gets caught — it looks wrong, so someone checks it. A confidently fluent wrong translation doesn’t announce itself; it reads naturally and gets trusted. I’ve come to think a confident wrong answer is worse than no answer at all, and steering is partly a response to that asymmetry: it’s a way of keeping a human correction in the loop before fluency gets a chance to disguise an error, rather than trying to audit fluency after the fact. This is the same instinct behind treating translation quality as something that has to be actively converged on rather than assumed — I wrote about the social, consensus-driven version of that idea in Consensus Translation.

I have a real case that convinced me this isn’t just theory. On a full Bible translation project in Portuguese, we saw both patterns play out on the same underlying system. One workflow involved continuous correction — a translator reviewing and correcting essentially every prediction as it was generated, feeding straight back into the next one. Another workflow, used by some translators on the same team, was to batch-draft a chunk of verses first and edit them afterward, document-style. The batch-and-edit group came away convinced the AI wasn’t learning from their corrections — the same errors kept resurfacing chapter after chapter. But the model wasn’t failing to learn; it was never shown the correction at a moment when it could use it. It’s like flying a thousand miles and then trying to correct it, instead of correcting as you go — the destination doesn’t move just because you finally noticed you were off heading.

That case study is what pushed me toward a rougher framework I’ve been using since, mostly as a sequencing heuristic rather than anything rigorous: golden path and silver path. Golden path is for when someone’s actually watching — a human reviewing and correcting every single prediction in real time. Because that attention is expensive, you spend it on the material most worth learning from: the most distinctive, highest-coverage constructions, the ones whose corrections will generalize furthest downstream, translated first. Silver path is for when nobody’s watching as closely — the model runs ahead on material that’s comparatively easy or low-risk, and anything that looks uncertain gets flagged for a human to review later rather than corrected live. Golden path when someone’s watching, silver path when they’re not.

I don’t think this framework is settled — it’s closer to a rule of thumb I’ve been testing against real projects than a methodology I’d defend in the abstract, and I’d guess the boundary between “distinctive enough to deserve golden-path attention” and “safe enough for silver path” needs to get a lot more precise before this generalizes past the projects I’ve watched it work on. But it’s already changed how I sequence work, in the same spirit as the simplification argument in Occam’s Razor and Bible Translation: before adding more review infrastructure, ask whether the correction you’re about to make is actually reaching the next prediction, or just the next document.

I ran into something last week that unsettled me more than expected. I was testing whether a model could translate a passage using only its own reasoning, so I hid the source text behind a simple substitution cipher before handing it over — the idea being that if the model could still produce a faithful translation, it would have to be doing something closer to genuine cross-lingual reasoning than pattern completion. It decrypted the cipher and recited the verse anyway. Correctly. Immediately. The Bible is so memorized by frontier models that hiding the source text doesn’t work — the model just decrypts and recites it anyway.

That’s stranger than it first sounds. This wasn’t a model translating an encrypted string. It was a model recognizing that a scrambled string of characters was shaped like a passage it already had memorized verbatim, discarding the cipher as a distraction, and reciting from memory. That’s a different capability than translation, and it means a huge chunk of what looks like “the model translated this well” for biblical text is really “the model already knew this text in every language I asked for.” You can’t use Scripture to test whether a model can translate, because for Scripture specifically, the model usually doesn’t need to.

This is a genuine methodological problem, not just a fun fact. If I want to evaluate — or train — a system’s ability to reason from meaning to meaning rather than reciting a memorized target string, I need source material the model hasn’t swallowed whole. Obfuscating the surface form doesn’t work, as I just found out the hard way. What might actually work is changing what the “source” is in the first place.

Somewhere around 2010, deep into a master’s degree in Ancient Greek linguistics, I was trying to build word2vec models over a Greek corpus using Gensim, which was — and largely still is — the workhorse library for this kind of thing. Gensim’s whole pitch was memory efficiency: instead of loading a corpus into RAM and training a model on the full thing in one pass, you streamed documents through it one at a time. The model updated incrementally as the data arrived, rather than requiring the entire corpus to exist in memory simultaneously before training could start.

At the time this felt like a plumbing detail, not an idea. But it planted a question I didn’t know I was carrying around for the next decade and a half: why train a model? Why don’t we figure out a streaming approach where we just give it the data it needs on the fly? Ancient Greek is a low-resource language by any modern standard — there wasn’t enough data to justify pretraining a dedicated model in the first place, and streaming was mostly a practical concession to a laptop that couldn’t hold the corpus in memory. But the concession turned out to be more interesting than the workaround it was standing in for.

That question is, more or less, how the translation tool I’ve been building handles languages it has never seen. It doesn’t train a per-language model. There’s no fine-tuning step, no “we support 40 languages, request the 41st.” Instead, at prediction time, it streams relevant translated examples to the model as context and asks it to produce a translation using those examples as its only local evidence for how the language behaves. The model isn’t learning the language in any durable sense; it’s being handed exactly the data it needs, on the fly, for this one prediction, and then that context evaporates. This is the same move as Gensim’s streaming corpus reader, just moved from “how do I fit training data in memory” to “how do I give a frozen model enough signal to be useful in a language it’s never encountered.”

The practical difference is enormous. Training or fine-tuning a model per language — even a small one — is the kind of thing that will bog your computer down for hours and then crash, especially if you’re iterating on ultra-low-resource languages where you don’t have the corpus size to justify the cost in the first place. Streaming relevant examples at inference time is orders of magnitude faster, because you’ve swapped a training job for a retrieval-and-context-assembly step. It’s the same logic I was chasing in Minimal Translation Memory for AI Agents — find the smallest, most generalizable set of examples that covers the most ground, and hand the model only that, rather than everything you have.

I want to be honest about where this breaks down, though, because it’s a real limitation and not a rounding error. The streaming approach works today because a human is validating each prediction live — approving, correcting, or rejecting the model’s output as it’s generated. That human-in-the-loop step is quietly doing a lot of work: it’s the error-correction mechanism that a trained model would otherwise have baked in through gradient descent over many examples. Remove the human, and you no longer have a guarantee that streamed context is sufficient on its own to produce a reliable translation. Whether streaming-plus-retrieval can substitute for training without a validator in the loop is, as far as I can tell, an open research problem — not a solved one I’m just being modest about. I’d put it in the same bucket as the clustering evaluation problem I never resolved in Contextual Vectors for Lexical Meaning: a piece I know is missing, not a piece I’ve quietly decided doesn’t matter.

What convinces me this is a real pattern, rather than a translation-specific hack I’ve talked myself into, is that I’ve watched the identical lesson surface somewhere completely unrelated. Years ago, I was on the other side of a job interview where a candidate was asked to process seventy million XML records. The naive approach — load everything, validate everything — didn’t just run slowly, it fell over. The fix was to stream: validate records as they arrived, discard what you didn’t need to keep, never hold the whole seventy million in memory at once. It’s the same shape as few-shot prompting, just wearing different clothes: you don’t dump your entire knowledge base into a context window and hope the model sorts it out, you stream the handful of examples that are actually relevant to this prediction and let the rest stay where it lives.

That’s one of maybe four lessons a developer has to learn, and I seem to have learned it twice, fifteen years apart, in two fields that have nothing to do with each other. I don’t think that’s a coincidence so much as a sign that “stream the relevant slice instead of loading the whole thing” is one of those ideas that’s basically fractal — it shows up at the level of a corpus reader, a translation model, an XML validator, and a prompt, and it’s probably going to keep showing up wherever the next constraint is memory, time, or data scarcity.

#translation

Three months ago, my team was discussing ways to gather translation data for new, undocumented languages that would be helpful for Bible translation, but which sidestepped the intimidation factor that accompanies the prospect of translating the Bible itself.

One idea that came to mind was to create a short novel that would cover the major genres of the Bible in a minimal form.

Here’s an example outline for a short novel that would cover the major genres of the Bible in a minimal form:

#translation

#translation

This idea was co-ideated with Ben Scholtens

Double-Entry Translation, or Ledger Translation (LT), is a framework that adapts the principles of double-entry bookkeeping to ensure accuracy, consistency, and a clear mapping between the source and target texts in the Bible translation process.

The framework consists of the following components:

1. Translation Accounts:

   • Source Text (ST): The original text, treated as an asset.
   • Translated Text (TT): The new translation, treated as a liability.
   • Translation Mapping (TM): The mapping between ST and TT, representing the transactions or “translation-actions” for translation-units at multiple levels of granularity (e.g., meaning-based alignments, sliding-window ngram alignments, etc.).

2. Translation Journal: A chronological record of all translation entries, including the source text, multiple translated texts, and their corresponding translation mappings.

3. Translation Ledger: A collection of “posted” translations and their reconciled mappings, perhaps organized by book, chapter, and verse, representing the accepted translations and alignments.

4. Translation Process:

   • One or more translators create journal entries for each translation unit, including the source text (ST), multiple translated texts (TT1, TT2, etc.), and their respective translation mappings (TM1, TM2, etc.).
   • Each translation mapping “debits” the source text and “credits” its corresponding translated text, ensuring a balanced and accurate representation of the translation process. Anything credited to the translation can be debited from the source text, and vice versa for full accountability.

5. Reconciliation and Review:

   • Editors review the translation journal entries to ensure accuracy and consistency between the source text, multiple translated texts, and their mappings.
   • Discrepancies or issues are discussed and resolved, with the goal of creating a reconciled translation—there can be more than one accepted version for any unit, though a published version might be expected to arrive at a single preferred version for a given project—and mapping for each translation unit.
   • The reconciled translation and mapping are “posted” to the translation ledger, linking to the relevant journal entries.
   • The ledger entry represents a balanced translation that accounts for the credits, debits by way of the linking or mapping between the source and target texts.

6. Reporting and Iterative Publishing:

   • Regular reports summarize translation progress, including the number of translation units processed, reviewed, reconciled, and posted to the ledger.
   • These reports can be used as a mechanism for iterative publishing, allowing the translation team to share their work in progress and gather feedback without the pressure of producing a finished product.
   • Iterative publishing helps maintain transparency, engage stakeholders, encourage believers, and refine the translation over time.

7. Breaking Down Translation Units and Machine Translation:

   • The LT Framework enables the translation process to be broken down into smaller, manageable units, such as phrases or sentences, rather than focusing on larger segments like paragraphs or chapters.
   • As the translation progresses, the growing dataset of source texts, translated texts, and their mappings can be used to train and refine statistical and neural machine translation models.
   • These models can provide suggestions and assistance to translators, improving efficiency and consistency as the project grows.

By allowing for multiple translations and mappings to be submitted via the journal, and then reconciling them into balanced ledger entries, the LT Framework promotes collaboration and ensures that the final translation is a result of careful consideration and consensus.

The linking of ledger entries to their corresponding journal entries maintains a clear “audit trail” and allows for a comprehensive understanding of the translation process. This approach enhances transparency, accountability, and the overall quality of the translated text. More intriguingly, the LT Framework is a decentralized, collaborative, and iterative framework that does not rely on external expertise, and that can adapt to various translation projects, languages, and contexts, making it a valuable tool for emerging Bible translation projects and teams around the world.

#translation #ai

In the context of a translation project, where we aim to utilize only the resources at hand, let’s consider a simplified approach to the evaluation stage. This approach would involve the following steps:

1. Start by taking the draft translation and identify the translation pairs that are most similar to the paired prediction.
2. Evaluate the draft translation for any potential misuse of words.
   1. One possible method could be to use a rules-based matcher that identifies any tokens in the source+prediction pair that only occur in EITHER the source OR the target in any of the similar examples.
   2. It could also be beneficial to examine whether any words seem to be mistranslated based on the gold standard examples.
   3. To aid this process, aligning these examples by phonetics, orthography, and apparent semantic similarities could be helpful.
3. Once the evaluation is complete, pass the task back to the translation bot, providing special notes about any words that appear to be mistranslated.

In essence, the process can be summarized as: [ most similar examples ] —> [ prediction ] —> [ most similar examples ]

The goal is to identify the most similar examples to the prediction, and then use those examples to evaluate the prediction. This process can be repeated iteratively, with the bot learning from the evaluation and then generating a new prediction and a new set of examples that will help the initial bot improve its output.

#translation

Linguistic structures are the formal patterns that realize linguistic meanings. In the process of translation, it is the meanings that are translated; the structures are left behind by design. Semantics are the priority in translation.

Note: between two languages, the semantic systems are not isomorphic. This mismatch means that we cannot assume that the precise semantic categories of one language may be transferred into another. Comparing the lexicogrammatical systems of gender in two languages should suffice to illustrate this point. However, the general semantic categories seem to be fairly universal (one would have to compare every language to actually establish this claim).

This fundamental observation must be factored into the translation process as we consider how AI tools might be leveraged for translation drafting.

For instance, rather than trying to account for every word in the source by translating a word in the target, we should attempt to account for the things the words are being used to realise, namely the entities, processes, traits, collocations, logical sets, speech acts, and more.

It is definitively better to traffic in semantic units such as these, since accounting for such meanings is what makes a translation good or bad.

By contrast, imagine if someone claimed that you needed to account for every character in the source text (all the alphas, omegas, deltas, and iotas). If you did somehow manage to do this, you would not be translating, but copying.

#translation #ai

In the realm of machine translation, achieving the highest level of accuracy is paramount. One approach to improve the precision of translations is through a method called “Token Matching Evaluation”. This method centres around the use of valid tokens, which are words or phrases that have appeared in sample translation pairs.

#translation

(A Dynamic Equivalence Alignment is something like “Essentially Aligned Bible Translations”)

Token to token mapping is problematic, despite its ubiquity. It is theoretically the same as character to character mapping (a token is, with some exceptions, a set of characters delimited by whitespace). There is a fundamental distinction between structural units like characters, tokens, phrases, clusters, wordings, sets, etc., where the defining characteristic is composition (“what is this thing built out of?”) and semantic or functional units like entities, processes, predications, speech acts, definitions, scenarios, etc., where the defining characteristic is role or function (“what does this thing do in its immediate context?”). 

With regard to alignment, the question driving any alignment is “what does span A have in common with span B?

The answer depends on what type of spans you are trying to align.