Exploring three promising approaches to tokenizing low-resource languages: adding new tokens to LLMs, cipher-based methods, and logits warping.
Ryder.Dev