#development

6 notes

#design

Here’s a quick reference list of questions to ask when designing a product, when you want to get to the core of your product, trim the fat, sort out the signal from the noise, etc.

VISCERAL IMPACT & DESTINY

  1. “What would make someone’s eyes light up when they first see this?”
  2. “How does this change people’s relationship with [the domain]?”
  3. “What limitation are people accepting as ‘just how it is’ that we could eliminate?”
  4. “If this succeeds wildly, what industry becomes obsolete?”

EXPERIENCE PERFECTIONISM

  1. “What moment in using this feels even slightly awkward or inelegant?”
  2. “Where are we making the user do something the computer should know?”
  3. “What detail would seem obsessive to include but users would love once they discover it?”
  4. “Which part of this experience isn’t bringing joy yet?”

PRODUCT TRUTH

  1. “What’s the ‘one thing’ this product is about that everything else serves?”
  2. “Which feature are we keeping because we’re afraid to remove it?”
  3. “What would this look like if we had to remove half the features?”
  4. “Where are we compromising on excellence to accommodate edge cases?”

REVOLUTIONARY POTENTIAL

  1. “How could this be 10x better, not just 2x better?”
  2. “What crazy idea for this product would be dismissed as impossible but would be magical if it worked?”
  3. “What would make the mainstream mock this at first but later seem obvious?”
  4. “How does this contribute to making technology more human?”
Permalink →

Multimodal Language Server Protocol

2024-03-04 · 3 min read

#translation

The Multimodal Language Server Protocol (MLSP) is a protocol, based on the Language Server Protocol for communication between a language server and a client. It is designed to support the development of language servers that can process non-textual data.

Imagine being able to open an MP3 file in VS Code (or some other app) and have a server in the background identify, on the fly,

  • the language(s) spoken in the audio
  • characteristics of the speaker(s), their emotion, inflection, etc.
  • the words spoken
  • byte-based positions where you have spliced in edits
  • milestones or transition markers for navigation
  • etc.

And then, imagine if your language server could automatically clean out background noise on the fly, either for the whole file, or for byte-ranges you select in your editor using a waveform.

Key benefits

The benefits of an MLSP are the same as the LSP:

  • pushing heavy processes out of the client and into the background on a separate process
  • enabling the same server (whether local or remote) to be used by multiple clients

These twin benefits are especially important for multimodal language servers, which may require significant computational resources to process non-textual data. By breaking them out into separate processes, we keep the client responsive.

Major features

Like the Language Server Protocol, the MLSP would be designed to do two things:

  1. Publish diagnostics (error, warning, info, hint levels) for a given byte position range in the file
  2. Update the content for a given byte position range

Separation of concerns is of the utmost importance. These diagnostics and content updates are completely agnostic to what you are trying to do with your multimodal language server. All that matters is the kinds of client procedures that can be called remotely from the server, and the kinds of server procedures that can be called remotely from the client.

Adapation of LSP

The goal is to leverage the existing LSP as much as possible, only extending (or disabling/ignoring) features where it is absolutely necessary.

Conclusion

In conclusion, the Multimodal Language Server Protocol (MLSP) represents a significant leap forward in the development of language servers. By extending the capabilities of the traditional Language Server Protocol to handle non-textual data, MLSP opens up new possibilities for developers and users alike. Whether it’s processing spoken language in audio files, identifying speaker characteristics, or automatically cleaning background noise, MLSP has the potential to revolutionize the way we interact with digital content. I’m excited to continue exploring and expanding the boundaries of what language servers can do.

Reach out to me at ryderwishart at gmail dot com if you’re interested in contributing to the project.

Permalink →

#ai #translation

Building an AI-native translation app

We’re exploring the idea of using AI agents, via simple user interfaces, as a way to orchestrate Bible translation tasks both within an app and by leveraging outside tools and APIs.

While we’re still in the early stages of this exploration, we’ve already found that it’s possible to build a Bible translation app using VS Code as a base.

Our proof-of-concept plugin has one simple task: when the user says “the right way to say X is Y”, the plugin will discern the user intent and update the lexicon.txt file (the project’s dictionary).

What we want to do next is to use the AI agent to orchestrate the translation process, to take the next step of suggesting updates on any verses that are implicated in the lexicon change.

This is where VS Code comes in.

We realized we can use VS Code as a base for a Bible translation app. It’s open source and has a lot of features that you would need for a Bible translation app. It’s also very extensible, and operates at the highest echelons of enterprise software, and can be localized into many languages of wider communication (e.g., the Chinese localization has over 24 million downloads!).

How feasible is it to fork VS Code?

Providentially, we have an example of someone forking VS Code and building a fully functional AI-native app. A huge congratulations to the folks at Anysphere/Cursor. This app is awesome.

The craziest thing about this is that they did it in a matter of months, with a small team of developers (I think it was 2-3 or so people). They forked it in March 2023, and I tried the app in July, and it rocked.

I think there are some learnings to gather here for the Bible translation world: VS Code provides a great base for building an AI-native app, and it seems feasible to fork it and build something that is useful for translators by leveraging the fact that VS code has done all of the hard stuff already (like mature source control, tested localization, an impressively stable architecture with over 51K published plugins, live sharing for instant remote collaboration, and more).

Adding translation functionality suddenly seems like a small task in comparison.

Permalink →

Originally posted: 2021-05-19

You got this GraphQL error because you have two fields with the same value

The error

The relation field [fieldname] must specify a `@relation` directive: `@relation(name: "MyRelation")`

Why you got this error

You have two fields that have the same value, for example:

user {
    id: ID
    name: String
    currentCompany: UserCompany
    previousCompany: UserCompany
}

This set of graphql query fields has two fields that have the value UserCompany. You have to either indicate a relation between user companies and users, or else create a subtype so they don’t reside at the same level.

Permalink →

Originally posted: 2020-02-07

There’s just something about a backend server trying to talk to a convoluted frontend editor that kills all productivity.

TLDR: A JAMStack site crushes Wordpress because one of these scales, and the other doesn’t.

Completely jammed up, lagged out, flat out not working. 😖

Ever opened your frontend editor, whether that’s Squarespace, Wix, a premium Wordpress plugin—or whatever—and experience it?

If I ever break my femur, I’ll try to do a squat, just to see which is more painful.

Just kidding? 😅

Maybe you’re making an “utimate guide” for some topic as part of your content marketing strategy, and you hit a critical mass of content and the browser just can’t seem to handle what you are putting it through.

Heck, I even experience this problem with Microsoft Word.

After a certain number of pages (usually a couple hundred, not uncommon for a manuscript), it just stops working.

A fellow PhD student told me that in the final days of his dissertation revisions it would take over a minute for Word to just save his file. It’s painfully frustrating.

Well, this frontend log-jam bogged me down this morning.

The checklist I never finish writing

I’ve been working on this ultimate SEO checklist for months now.

I keep putting it off, and then later I come back to it, thinking,

Why did I stop working on that?”

Of course, five minutes into using my Wordpress frontend editor and it all came back to me: it’s just impossible to get anything done!

Now I am seriously considering migrating my digital marketing business’s website to Gatsby.js.

You know it. I know it. JAMStack is It

I’ve made a few JAMStack sites for clients recently using some Gatsby starters. I deploy them using Netlify, and all I can say is, “wow.”

JAMStack sites use static HTML files, so there is no PHP involved, no backend, and in the case of Netlify, continuous deployment and no server! 🤩

It’s ultimate security, ultimate speed (seriously, you get local speeds), and, best of all, you can do all your editing in a text editor.

Using a text editor really seals the deal as far as I’m concerned. What could be simpler, or more reliable?

The Plan

I’m going to migrate my business site, truemarketing.ca to a JAMStack site.

Of course, I’ll be using Gatsby. While I could try to use a headless CMS, I think I want to keep my workflow completely local.

I will probably use MDX (markdown with JSX—which in turn is extended JavaScript), but I’ve got a few options I’ll explore.

Either way, I am pumped ✊ to ditch my clunky Wordpress frontend.

If you’re pumped about Gatsby.js too, share this post to Twitter with the #gatsbyjs hashtag!

Here’s a nify quote you can use, too:

TLDR: A JAMStack site crushes Wordpress because one of these scales, and the other doesn’t.

Permalink →

A thought experiment. (credit to Ben Scholtens for asking the obvious question: Why not fork VS Code ourselves?)

2023-09-19 #ai #translation


Your Translation Project is a lot like a Software Project. Why not use a World-Class Text Editor?

🔍 The Insight: Bible translation, at its core, is a text-editing endeavor, akin in many ways to software development. It’s about collaborating, maintaining versions, and ensuring quality – just like developing a codebase.

❓ The Question: Why start from scratch when there’s a mature, battle-tested platform that offers vast capabilities tailored to these tasks?


VS Code: The Unsung Hero for Bible Translation

📖 No Loss in Functionality: At first glance, Paratext appears irreplaceable. But dive deeper, and you’ll find that the VS Code ecosystem can replicate its core functions, and it’s highly stable, scalable, and useable. From textual analysis, collaboration, to project management, there’s an extension for that. Oh, and it comes with rock solid AI integration.

🛠 Superior Toolset:

  • Stability & Text Editing: VS Code’s origin is text editing. Its advanced features ensure seamless, bug-free experiences.
  • Plug-ins: With tens of thousands of extensions, there’s a plugin for almost every need, and creating new ones is straightforward, whether you’re a software firm or an SIL programmer.
  • Git Integration: Mature version control is baked in, making collaboration and version tracking effortless.
  • File Management: Navigate complex projects with ease, and leverage the advanced file system capabilities.
  • LiveShare: Work on the same file across devices. Real-time collaboration made simple.
  • Customization: Connect to linting systems, render previews, automate tasks, and more.
  • Localization: Tailor the interface to your linguistic needs. It’s globally friendly.
  • Extensibility: VS Code’s architecture is primed for expansion, ensuring it’ll grow with your needs.
  • AI Integration: Benefit from native AI tools for smarter, quicker leveraging of all of your project resources and work to date.
  • Support & Documentation: A vast community, extensive documentation, forums, videos, and guides stand ready to assist.

The Implication:

While Bible translation has unique domain requirements, it shouldn’t reinvent the wheel for tasks common to many projects. By leveraging VS Code, we tap into a decade of optimization, a vast ecosystem, and avoid the pitfalls of building from the ground up. Imagine cutting down development time by 10 or 20 years (how long would it take a team to accomplish 115K commits, on thousands of branches, with as good a result as VS Code?), ensuring greater stability, and having a world-class UI — all while keeping focus on the unique needs of translation.

Some of the most advanced and complex features that translators or consultants could only dream of (like having complete project version control on an enterprise level with automated actions and quality check processes, or enabling 4 people in four geographic locations to collaborate on the exact same file in real time) are already built, tested, stable, and being used by hundreds of thousands of VS Code users.

Why would we put our time and effort into recreating all of this, when its already free and completely open source (MIT license)? In conclusion, the complexities of tool development are vast, but they’ve been met and tamed by VS Code. Rather than venturing into the wilderness to forge a new path, let’s stand on the shoulders of giants, repurpose what’s proven, and channel our energies solely into the specialized challenges of Bible translation.


🚀 The Call to Action: Before investing further in new tools, let’s explore the vast potential of VS Code. It promises a foundation that’s both solid and flexible, ensuring our Bible translation endeavors are as efficient, effective, and impactful as possible.


Minor update: VS Code fork seems feasible

Permalink →