#nlp

10 notes

Translate the Meaning, Not the Words

2026-07-03 · 6 min read

I ran into something last week that unsettled me more than expected. I was testing whether a model could translate a passage using only its own reasoning, so I hid the source text behind a simple substitution cipher before handing it over — the idea being that if the model could still produce a faithful translation, it would have to be doing something closer to genuine cross-lingual reasoning than pattern completion. It decrypted the cipher and recited the verse anyway. Correctly. Immediately. The Bible is so memorized by frontier models that hiding the source text doesn’t work — the model just decrypts and recites it anyway.

That’s stranger than it first sounds. This wasn’t a model translating an encrypted string. It was a model recognizing that a scrambled string of characters was shaped like a passage it already had memorized verbatim, discarding the cipher as a distraction, and reciting from memory. That’s a different capability than translation, and it means a huge chunk of what looks like “the model translated this well” for biblical text is really “the model already knew this text in every language I asked for.” You can’t use Scripture to test whether a model can translate, because for Scripture specifically, the model usually doesn’t need to.

This is a genuine methodological problem, not just a fun fact. If I want to evaluate — or train — a system’s ability to reason from meaning to meaning rather than reciting a memorized target string, I need source material the model hasn’t swallowed whole. Obfuscating the surface form doesn’t work, as I just found out the hard way. What might actually work is changing what the “source” is in the first place.

Acting the scene instead of laundering the words

There’s a practice-level version of this same insight that I’ve been leaning on for a while, mostly with human translators and increasingly as an instruction to models: stop translating the words. Act the scene. Then say what just happened in your own language.

The reasoning is the same as in prioritizing semantics over structures: a word is not a translation unit, it’s a residue left behind by a meaning that already happened. Translating the residue is really just copying with extra steps — the failure mode the cipher experiment exposed at scale. A model (or a person) staring at a string of tokens finds the shortest path to an output resembling those tokens, and if it’s already memorized the “right answer,” the shortest path is recall, not comprehension. Acting the scene forces a detour around that shortcut. You have to know who is speaking, to whom, what they want, what’s at stake, before you’re allowed to say anything at all. Only then do you say what happened, in whatever language you’re working in.

The more rigorous version: don’t feed it the words at all

The practice-level fix is good discipline, but discipline is not architecture, and I’d rather not depend on a model or a translator “resisting the urge” to recall a memorized string. The more rigorous fix, and the one I actually want to build, is to stop handing the model literal source text as the thing to be translated at all.

Instead, represent the passage as a language-agnostic semantic description: pericopes broken down into nested units of who is speaking, to whom, and what is being communicated — speech acts nested inside speech acts, the way narrative and dialogue actually nest inside each other. This is the same move I described in the case for dynamic equivalence alignments — form → meaning → form instead of form → form — pushed one step further upstream. Instead of asking a model to align a translation back to source words after the fact, you never give it the source words as the object to reproduce in the first place. Its job becomes representing the meaning of a discourse unit faithfully, not pattern-matching against a corpus it already has memorized down to the verse numbers. If there’s no recognizable surface string to recite, “recite the memorized answer” stops being a viable shortcut, and the model actually has to do the thing you’re asking it to do. This is really an extension of the point I made in beyond word alignments: the word was never the right unit, and now I have a concrete reason — memorization contamination — to stop treating it as one even operationally.

Where this is honestly still rough

I don’t want to oversell this, because I’ve already tried a related methodology and it only sort of worked. I built a multimodal process I’ve been calling Familiarize → Internalize → Articulate — acting out a passage, drawing the scene, studying the characters and key terms — before ever producing a rendering, as a more embodied version of the “act the scene” instinct above. In practice it can shunt between layers of context in ways that are only sometimes helpful; moving between acting, drawing, and terminology study doesn’t reliably converge on a clearer semantic representation, it sometimes just adds noise. And the supporting resources for it — the character studies, the term glossaries, the scaffolding that’s supposed to make internalization tractable rather than vibes-based — are still unfinished.

So this isn’t a solved problem, it’s a diagnosis with a promising direction attached. The cipher experiment told me the failure mode is real and larger than I assumed. The semantic-description architecture is my best current guess at a fix that doesn’t rely on willpower. Whether nested speaker/addressee/content units are actually the right granularity, and whether a model can be made to represent meaning at that level reliably, is still very much open.

Permalink →

Streaming the Model Instead of Pretraining It

2026-07-01 · 5 min read

Somewhere around 2010, deep into a master’s degree in Ancient Greek linguistics, I was trying to build word2vec models over a Greek corpus using Gensim, which was — and largely still is — the workhorse library for this kind of thing. Gensim’s whole pitch was memory efficiency: instead of loading a corpus into RAM and training a model on the full thing in one pass, you streamed documents through it one at a time. The model updated incrementally as the data arrived, rather than requiring the entire corpus to exist in memory simultaneously before training could start.

At the time this felt like a plumbing detail, not an idea. But it planted a question I didn’t know I was carrying around for the next decade and a half: why train a model? Why don’t we figure out a streaming approach where we just give it the data it needs on the fly? Ancient Greek is a low-resource language by any modern standard — there wasn’t enough data to justify pretraining a dedicated model in the first place, and streaming was mostly a practical concession to a laptop that couldn’t hold the corpus in memory. But the concession turned out to be more interesting than the workaround it was standing in for.

That question is, more or less, how the translation tool I’ve been building handles languages it has never seen. It doesn’t train a per-language model. There’s no fine-tuning step, no “we support 40 languages, request the 41st.” Instead, at prediction time, it streams relevant translated examples to the model as context and asks it to produce a translation using those examples as its only local evidence for how the language behaves. The model isn’t learning the language in any durable sense; it’s being handed exactly the data it needs, on the fly, for this one prediction, and then that context evaporates. This is the same move as Gensim’s streaming corpus reader, just moved from “how do I fit training data in memory” to “how do I give a frozen model enough signal to be useful in a language it’s never encountered.”

The practical difference is enormous. Training or fine-tuning a model per language — even a small one — is the kind of thing that will bog your computer down for hours and then crash, especially if you’re iterating on ultra-low-resource languages where you don’t have the corpus size to justify the cost in the first place. Streaming relevant examples at inference time is orders of magnitude faster, because you’ve swapped a training job for a retrieval-and-context-assembly step. It’s the same logic I was chasing in Minimal Translation Memory for AI Agents — find the smallest, most generalizable set of examples that covers the most ground, and hand the model only that, rather than everything you have.

I want to be honest about where this breaks down, though, because it’s a real limitation and not a rounding error. The streaming approach works today because a human is validating each prediction live — approving, correcting, or rejecting the model’s output as it’s generated. That human-in-the-loop step is quietly doing a lot of work: it’s the error-correction mechanism that a trained model would otherwise have baked in through gradient descent over many examples. Remove the human, and you no longer have a guarantee that streamed context is sufficient on its own to produce a reliable translation. Whether streaming-plus-retrieval can substitute for training without a validator in the loop is, as far as I can tell, an open research problem — not a solved one I’m just being modest about. I’d put it in the same bucket as the clustering evaluation problem I never resolved in Contextual Vectors for Lexical Meaning: a piece I know is missing, not a piece I’ve quietly decided doesn’t matter.

What convinces me this is a real pattern, rather than a translation-specific hack I’ve talked myself into, is that I’ve watched the identical lesson surface somewhere completely unrelated. Years ago, I was on the other side of a job interview where a candidate was asked to process seventy million XML records. The naive approach — load everything, validate everything — didn’t just run slowly, it fell over. The fix was to stream: validate records as they arrived, discard what you didn’t need to keep, never hold the whole seventy million in memory at once. It’s the same shape as few-shot prompting, just wearing different clothes: you don’t dump your entire knowledge base into a context window and hope the model sorts it out, you stream the handful of examples that are actually relevant to this prediction and let the rest stay where it lives.

That’s one of maybe four lessons a developer has to learn, and I seem to have learned it twice, fifteen years apart, in two fields that have nothing to do with each other. I don’t think that’s a coincidence so much as a sign that “stream the relevant slice instead of loading the whole thing” is one of those ideas that’s basically fractal — it shows up at the level of a corpus reader, a translation model, an XML validator, and a prompt, and it’s probably going to keep showing up wherever the next constraint is memory, time, or data scarcity.

Permalink →

MT Pipeline for Low-Resource Languages

2024-09-26 · 3 min read

#translation #nlp #machine-learning

Machine translation for languages with limited resources is a difficult challenge. Here’s a straightforward pipeline that could yield some interesting results:

Accuracy First: Start with an interlinear-like, literalistic rendering. This step uses statistical glossing or high-granularity predictions to create a word-for-word translation. It’s not pretty, but it’s “accurate” in the sense that it is a representation of the source text.
Naturalness Post-Edit: Here’s where it gets interesting. Take your literal translation and try to iteratively make it sound natural:
- Gather as much unstructured text in the target language as you can.
- Measure the likelihood of your output from step 1 using different slices of this data (think sliding window n-grams or similar techniques). (This is kind of like measuring perplexity of a model.)
- Systematically swap and rearrange parts of the translation.
- Rank the different versions based on their ‘naturalness’ (as defined by the likelihood measurements).

This approach seems deceptively simple, but it could be quite promising. By separating the accuracy and naturalness concerns, we’re able to leverage the strengths of both statistical and corpus-based methods.

The beauty of this pipeline is its potential to work with limited resources. Even if we don’t have parallel corpora (expensive to get), or extensive training data (requires model fine-tuning, quickly gets out of date, and likely will be inaccurate until some critical mass of data is reached), we can still produce a reasonable translation by focusing first on word-level accuracy and then on phrase-level naturalness.

You could also add additional steps to the pipeline, such as:

Back-Translation: Translate the output of the previous step (now in the target language) back to the source language for comparison.
Gloss at Lower Granularity: Use the output from the previous step and attempt to re-gloss the text at a lower granularity (e.g., complete phrases).

Of course, there are challenges to consider. I suspect such an approach would work best for languages that are similar to the source language, perhaps from the same language family. The potential here lies in the use of more traditional, statistical approaches to machine translation, which are typically much faster and can be iterated many times with minimal resources and more easily measured for success.

Permalink →

LangSim: Language Similarity Metrics for Translation

2024-06-20 · 2 min read

#ai

Comparing languages often requires extensive datasets and complex algorithms. But what if we could gain insights with just a handful of translation pairs? Enter langsim (“language similarity”), an experimental project that aims to do just that.

langsim is a Python library that compares languages using a variety of metrics, even with limited data. From lexical similarity to morphological complexity, it provides a multi-faceted view of language relationships.

Key Features

Compares languages using 11 different metrics
Works with small datasets (even just a few translation pairs - this is huge for Bible translation into ultra-low resource languages)
Language-agnostic approach
Easy to install and use

The Road Ahead

It’s important to note that langsim is very much a work in progress. It’s not recommended for production use yet, and there’s plenty of room for improvement. The current metrics are just a starting point, and we’re actively seeking feedback and contributions.

One exciting direction for langsim’s future is leveraging Large Language Models (LLMs) to evolve its language comparison capabilities. Inspired by Stanford’s fascinating DrEureka project, we’re exploring ways to use LLMs to fine-tune the weights of our comparison metrics, potentially leading to more accurate and nuanced language comparisons.

Get Involved

If you’re interested in language comparison, computational linguistics, or just enjoy tinkering with code, we’d love your input. Try out langsim, open issues with your impressions, or contribute to its development.

Let’s push the boundaries of what’s possible in language comparison, one metric at a time!

Links

Permalink →

Linguistic Anomaly Detection

2024-01-01 · 3 min read

#translation

Linguistic Anomaly Detection is a technique for similarity scoring (compression-based, embedding-based, or traditional NLP-based) between source-source and target-target pairs. This involves comparing the similarity of source texts and their corresponding target texts. Rather than being scored against a baseline, however, it has the essential feature of being relativized to the current state of the translation project, such that only the verses a human has already translated/edited serve as the gold standard. Here’s an idea for how LAD might work in a translation project:

Suppose you have a source1-target1 pair that has been manually vetted and is considered the gold standard.
You also have a source2-target2 pair that has not been vetted yet.
Compare the similarity of source1 and source2. This could involve comparing the compression results or similarity between the sources and between the targets, doing TF-IDF intersection scoring against your current project, or some other method.
If the similarity between source1 and source2 is 0.8, then the similarity between target1 and target2 should be within a standard deviation of 0.8. (This is my suspicion at least.)
Provide Scores or Reports on Estimated Consistency: The above technique (or a variation thereof) could be used to provide scores or reports on the estimated consistency for the unvetted target text (target2). This allows for a quantitative measure of the translation’s internal consistency, and by implication it should indicate something about its overall quality.

There’s a lot of room for variation here, but the basic idea is to use a partially complete translation to evaluate the consistency of the rest of the translation. This is a very different approach from the traditional BLEU score, which compares the translation to a baseline (e.g., a human translation or a machine translation). Traditional approaches require you to have a baseline and hypothesis (i.e., two separate versions) for the specific text you are translating. This approach only requires you to have a single target rendering. Instead, this approach compares the translation to itself, which, in my view, is a much more useful metric for translation quality assessment in the first place, since it doesn’t necessarily prioritize word-for-word equivalence, which can be deceptive as a quantitative metric.

Permalink →

Grammar Constraints on LLMs

2023-12-22 · 5 min read

#translation #ai

I’ve been thinking about how to improve LLM performance for Bible translation, especially for low-resource languages. The current approaches aren’t great - LLMs struggle with the nuances of these languages and often produce poor translations. This is a big problem since we need translations that respect both cultural and linguistic aspects.

Combining Old and New Approaches

Rule-Based Grammars + Context-Aware LLMs

The first thing I tried was combining traditional rule-based grammars with modern LLMs. Rule-based grammars are pretty rigid on their own, but they give us a solid structure to work with. When you mix this with an LLM’s ability to understand context, you get something much more useful. The grammar provides guardrails while the LLM handles the nuanced language understanding.

Using LLMs to Build Grammars

Here’s an interesting twist - we can actually use LLMs to help write grammars for low-resource languages. Instead of just using them as translators, we’re using them to build the linguistic frameworks we need. I think this could be a game-changer for working with lesser-known languages.

How It All Works Together

The cool part is seeing how these old-school grammar rules work with modern LLM capabilities. What used to be seen as outdated becomes really powerful when combined with the way LLMs process context and meaning.

Grammar-Constrained Decoding

I’ve been looking into using context-free grammars (CFGs) like GBNF to control LLM outputs. This approach, called grammar-constrained decoding (GCD), works particularly well for structured content.

What does this actually look like? The best example I’ve come across so far is the GBNF grammar approach found in the llama.cpp library. They give the following example of a formal ‘grammar’ for specifying that the only outputs that an LLM can give must be Japanese characters:

# A probably incorrect grammar for Japanese
root ::= jp-char+ ([ \t\n] jp-char+)*
jp-char ::= hiragana | katakana | punctuation | cjk
hiragana ::= [ぁ-ゟ]
katakana ::= [ァ-ヿ]
punctuation ::= [、-〾]
cjk ::= [一-鿿]

Benefits and Challenges

Research has revealed several key advantages of using grammar constraints:

Improved Accuracy: Studies show dramatically reduced error rates when using grammar-based constraints, especially for longer sequences
Efficient Learning: LLMs can learn from fewer examples - in some cases, needing only about 16.5% of the usual training samples
Domain Knowledge Integration: Grammars provide a way to inject external knowledge and domain-specific constraints into the generation process

However, there are some challenges to consider:

Distribution Distortion: Some grammar constraint techniques can distort the LLM’s natural distribution, potentially affecting output quality
Implementation Complexity: Effective grammar constraints often require specialized frameworks or algorithms

The Road Ahead

This hybrid approach isn’t just a theoretical exercise; it’s a practical solution to a real-world problem. By harnessing the strengths of both rule-based grammars and the adaptive capabilities of LLMs, we can make significant strides in Bible translation for low-resource languages. It’s a step towards leveraging powerful probabilistic models in a way that does not clobber much rarer and not-yet-digitized languages.

In conclusion, the integration of rule-based grammars with LLMs presents a promising avenue for enhancing Bible translation into low-resource languages. It’s a testament to the power of combining traditional methods with cutting-edge technology – a blend of the old and the new to achieve something truly remarkable in the realm of linguistic translation.

I’ll post some updates when I finally get around to applying the OpenText 2.0 grammar definition to an LLM. I’m excited to see what happens!

Recent Developments in 2024

The landscape of grammar constraints has evolved significantly since this article was first written. Some key developments include:

Inference-Time Reasoning

Rather than just relying on explicit grammar rules, newer models like OpenAI’s o-series demonstrate the ability to perform structured reasoning during inference time. This allows for more naturally constrained outputs without requiring formal grammar definitions.

Hybrid Approaches

Modern constraint systems like LMQL now combine:

Token-level masking with eager validation
High-level text constraints that abstract away tokenization details
Automatic translation between constraint definitions and implementation

This makes it much more practical to implement grammar constraints while maintaining the flexibility of modern LLMs.

Synthetic Data for Structure

Labs are increasingly using synthetic data to bake structural constraints directly into model training. As noted in the Phi-4 technical report, this creates more direct relationships between tokens and helps models learn cleaner reasoning patterns.

Advanced Constraint Techniques

Recent research has introduced more sophisticated approaches to grammar constraints:

Adaptive Sampling: New methods like ASAp (Adaptive Sampling with Approximate Expected Futures) help maintain output quality while enforcing grammar rules
Specialized Frameworks: Tools like YieldLang are making it easier to implement effective grammar constraints
Distribution-Aware Constraints: Researchers are developing methods to maintain the LLM’s natural distribution while enforcing grammatical rules

Permalink →

Reranked Translation Suggestions

2023-12-22 · 4 min read

#translation #ai

In the field of machine translation, ensuring the accuracy and context-relevance of translations is critical. One approach to address this challenge is through the use of semantic similarity between source verses as a tool for suggesting a disambiguated rendering from multiple possible options. (example below)

This idea, which I will refer to as “Reranked Translation Suggestions”, can be particularly effective when dealing with semantic units, but it can also be implemented with words only.

A Practical Example

Consider two translation units:

“Jesus walked on the water” -> “Jesus walked on the big blue wet thing”
“Jesus said, give me some water to drink” -> “Jesus said, give me some drinking stuff”
“Peter walked on the water” -> “Peter walked on the ?”

Let’s imagine that in the target language, the source word ‘water’ is translated as ‘big blue wet thing’ in the first verse and as ‘drinking stuff’ in the second verse.

In this scenario, a dictionary is updated to note that there are two possible renderings of the ‘water’ entity (this is a semantic unit that is not a process, not a trait, etc.—see below) in the source, and it notes the verse context for each rendering.

Later, when a rendering for ‘water’ is needed, the method checks the new source verse and determines whether the new verse is most similar to the verse that uses ‘big blue wet thing’ or the verse that uses ‘drinking stuff’. The most likely option from the project dictionary is then suggested accordingly.

Put even more simply, the method uses semantic similarity between the original verses to determine which of the two renderings for ‘water’ is most likely to be the correct rendering in the current verse.

“Peter walked on the water” is a new verse that has not yet been translated. The method can suggest the two possible renderings for ‘water’ based on the two previous verses, and then the translator can choose the most appropriate rendering for the current verse. In this case, the translator would likely choose ‘big blue wet thing’ as the most appropriate rendering for ‘water’ in this verse.

By leveraging semantic similarity and understanding the context of the source verses, this method can significantly improve the accuracy and relevance of translation suggestions, leading to more precise and contextually appropriate translations.

In short, translation suggestions can be “reranked” based on semantic similarity between the source verses.

Special Note on Semantic Units

Semantic units are often deeply nested. For example, “Jesus” is an entity, “Christ” is an entity, and “Jesus Christ” is an entity as well. When working with glosses for semantic units, each glosses should be assumed to refer to the largest entity in nested entities and all children until there is a separate case where there’s only partial overlap between the source and target. Previous glosses apply to these semantic units, further refining the translation suggestions.

For example, assume I have a semantic dictionary in which I know the “Jesus” has gloss A, “Christ” has gloss B, and thus “Jesus Christ” cannot be simply assigned either gloss A or B. On the contrary, if “automobile” has gloss C, then the high-level entity “the automobile” might be assigned gloss C as well, since it has no inherent conflict in its constituent children (assuming ‘the’ is simply a grammatical marker that shouldn’t always be glossed).

This seems a bit abstract at the moment, so I’ll give a more concrete example in the future. See also Multi-Agent Translation Simulations for another approach to improving translation accuracy.

Permalink →

Three Approaches to Tokenizing Out-of-Training Languages

2023-12-22 · 5 min read

#ai

In the realm of natural language processing (NLP), tokenization is the fundamental process of breaking down text into smaller, manageable units called tokens. This process, however, becomes increasingly complex and error-prone when dealing with out-of-training or low-resource languages. I’ll explore the three approaches to this challenge I’ve come across that seem most promising, drawing from a recent presentation (slide deck here) I gave at the SIL/Microsoft Hackathon and learnings shared in the Partnership for Applied Biblical NLP meetings.

1. Adding New Tokens to Large Language Models (LLMs)

The Problem: LLMs like the GPT models are trained on vast datasets, predominantly in high-resource languages. When encountering a low-resource language, these models struggle due to a lack of appropriate tokens and training data.

The Solution: Expanding the LLM’s vocabulary by training a tokenizer on new, low-resource language text and adding these tokens to the model. This approach has been exemplified by the SIL AI team’s experiments with the NLLB (No Language Left Behind) model (led by Michael Martin, details shared by Bethany Moore and Matthew Shannon as part of the presentation linked above). I can’t actually track down any links to their work here, but I suspect they will share more public information at some point. Some of the solution was based on forum discussions like this one.

How It Works:

Train a tokenizer (e.g., a SentencePiece model) on new text from a low-resource language.
Integrate these new tokens into an existing multilingual model. You need to make sure the model prioritizes these new tokens.
Fine-tune the model with the new training data (in SIL’s case, use an existing New Testament translation in the target language for fine tuning).
Predict some new text, such as the Old Testament.

This method apparently has improved the base NLLB results quite a bit, though I don’t have any numbers to share. I’m not sure if they’ve tried this approach with other models, but I suspect it would work with any multilingual model.

2. Cipher-Based Approach

The Problem: Standard tokenization methods may not capture the linguistic nuances of low-resource languages, leading to inaccurate or nonsensical translations, or the model might get stuck outputting some unknown token over and over.

The Solution: Utilizing a cipher-based approach to mask inputs and outputs, encouraging the model to rely on linguistic patterns via in-context learning (since the model is simply observing character level patterns rather than inappropriately associating the text in question with tokens mistakenly “recognized” from other languages in the training data). This technique can prevent dominant language probabilities or unknown-token nonsense from overshadowing the target language’s unique characteristics.

How It Works:

Masking the text of a low-resource language using a cipher.
Training the model to recognize and decode these patterns.
Allowing the model to generate outputs based on learned patterns rather than defaulting to high-resource language structures.

This approach is particularly useful for languages that share syntactic or phonetic similarities with high-resource languages, reducing the risk of misinterpretation.

See the early experiments by Chris Priebe. I suspect there is a lot more that can be done on this approach, and the use of in-context learning seems like a huge win from my perspective (cf. this post).

3. Logits Warping

The Problem: LLMs may favor more common tokens from high-resource languages, leading to poor representation of low-resource languages.

The Solution: Logits warping involves adjusting the probabilities of certain tokens to bias the model towards a specific language or grammatical structure.

How It Works:

Whether on the fly, or via a grammar definition, adjust the model’s logits, which are the model’s output predictions, to favor tokens from the target low-resource language.
This biasing can force the model to generate text in the intended language, adhering to its unique grammatical rules.

This technique is particularly effective for languages with distinct grammatical structures or non-latin but in-training scripts, ensuring that the model’s output aligns more closely with the target language’s syntax and orthography.

I’ve done a few experiments with this approach, and it seems incredibly promising. I will probably write some more about how it might be combined with a system network in the flavour of Systemic Functional Linguistics (SFL) in the future.

Here’s a short video clip showing how you can specify a list of tokens (I just passed it [token1, token2, token3] as the possible outputs). The model then generates text that only includes those tokens.

Here’s a second short clip where I force the model to only output Japanese characters.

Conclusion

Tokenizing low-resource languages presents unique challenges in the field of NLP. However, by employing innovative approaches like adding new tokens, cipher-based methods, and logits warping, we can significantly improve the representation and understanding of these languages in LLMs. This not only enhances the accuracy of language models but also promotes linguistic diversity and inclusivity in the digital space. As we continue to advance in NLP, it’s crucial to keep exploring and refining these techniques to better serve the global community of diverse language speakers.

Permalink →

Contextual Vectors for Lexical Meaning

2023-07-04 · 5 min read

In Natural Language Processing (NLP), understanding the contextual subtleties that modulate a word’s meaning is a significant challenge. The traditional approach, representing words as multiple discrete senses, generally fails to capture the range of complex meaning variation exhibited by words in context (which is as varied as the contexts, as a rule), specifically because that variation is arbitrarily parsed into a discrete set sub-meanings (semantic domain models also fall prey to this issue, when they allocate some lexemes to multiple semantic domains—in essence the domains predefine the set of meanings any given lexeme may fall into).

Transformer models do a good job of modelling contextual meanings (both distant and immediate context), but such models have the distinct disadvantage of being mostly opaque. Global vector models are better in this regard, yet they only model global context (or all the contexts a target shows up in in the corpus).

To tackle this problem, I will introduce a technique that combines attention mechanisms with various clustering techniques, aiming to represent words in a monosemic manner with explicit representations of contextual modulation.

For some very sketchy and prone-to-change implementation details, see my draft notebook on contextual vectors.

1. Unraveling Context with Attention Mechanisms

The process begins by transforming the text data into global vectors, and then into contextual vectors using attention mechanisms. These mechanisms, a cornerstone of modern transformer models, allow us to capture both the inherent meaning of a word and the rich context surrounding it in the form of contextual embeddings.

What I have tried so far is the following:

Start with global word2vec or fastText vectors
For each unique lexeme in corpus
- Gather all sentences this lexeme occurs in
- For each gathered sentence
  - Calculate the cosign similarity between the target lexeme and every other lexeme in the sentence
  - Weight each similarity score by the distance from the target lexeme (further away = less similar; this is obviously a step that is amenable to multiple possible configurations)
- Store all of the resulting contextual vectors (or “attention” vectors) in a dictionary with the lexeme as the key

2. Identifying Contextual Patterns with Clustering

The next step involves clustering these contextual vectors. Using unsupervised machine learning methods such as K-means, DBSCAN, or hierarchical clustering, I group together similar vectors. These clusters form centroids that represent typical patterns of contextual variation, rather than separate, isolated word senses.

What I have tried so far:

For each lexeme
- Cluster all associated contextual vectors using some clustering technique
- Identify centroid vectors within each cluster
- Allow the centroid vectors to represent the ‘word senses’

3. Strengthening Clustering with Ensemble Techniques

To enhance the reliability of the clusters, I implement ensemble clustering techniques. By merging the results of multiple clustering algorithms, one could create a consensus on the cluster assignments. This approach capitalizes on the strengths of each algorithm, providing a more resilient understanding of contextual modulation.

In reality, I haven’t yet worked through any means of evaluating the resulting clusters. The trickiest stage in any clustering process (in my experience) is determining how many clusters you need. This is always a lossy process, so there seem to be drawbacks no matter how you slice up the semantic space.

4. Advanced Techniques: Neighbour-Aware and Consensus Clustering

I further refine the clustering process with two more techniques: neighbour-aware and consensus clustering. Neighbour-aware clustering considers the nearest neighbours of each vector, providing a better representation of local contextual variation. In consensus clustering, I amalgamate the outcomes from several neighbour-aware clustering methods to determine the most accurate cluster assignments.

Conclusion

Through the synergistic combination of attention mechanisms and multiple clustering techniques, I am developing a method to represent lexical meanings in a monosemic manner that transparently accounts for the role of contextual modulation in arriving at ‘typical’ word senses.

I’ve always been prone to challenge the common assumption of multiple word senses being inherent features of lexemes (rather than emergent features of larger contexts). This work in progress offers one attempt at representing and analyzing text data to this end. This opens up new possibilities for text understanding and semantic analysis in NLP, paving the way for more accurate and contextually-aware models of lexical semantics that recognize decontextualized lexemes as meaning potentials, and contextualized lexemes as specifications of those potentials.

Permalink →

Why pretrained models should use introspection fact sheets

No date · 2 min read

Pretrained models are powerful tools for natural language processing, but they often lack the ability to answer basic questions about themselves, such as who created them, what is their purpose, and what model architecture they use.

These questions are not only relevant for curious users, but also for researchers and developers who want to understand the strengths and limitations of different models.

One way to address this gap is to use introspection fact sheets, which are short texts that provide essential information about a pretrained model. These fact sheets can be added to the training data as answers to a set of common introspection questions, such as:

Who created you?
What is your purpose?
What model is this?
What data did you train on?
How do you handle bias and fairness?
How do you deal with uncertainty and errors?

By adding these fact sheets to the training data, we can help the pretrained models learn to answer these questions more accurately and consistently. This can improve the transparency and trustworthiness of the models, as well as their usability and interoperability.

However, it is not clear what the best mechanism is for giving foundation models this information aside from a shallow fine-tuning. For example, how should we balance the fact sheet data with the original training data? How should we format and structure the fact sheet texts? How should we evaluate the quality and usefulness of the fact sheet answers? These are open questions that require further research and experimentation.

Permalink →