Speech To Text Accessibility

Minecraft 版本： 1.21.10 1.21.11

平台： fabric

标签： social technology utility

This mod aims to plug the communicative gaps between people who are capable of speaking and people who are incapable of hearing i.e. hard-hearing/deaf/disabled people. By relying on JNI wrapper for whisper.cpp, which is a C++ port of OpenAI's Whisper, as the intermediary base ASR model in translating what people speak into words, the equal level of communication can be established, in which people who speak don't have to stop moving to type, and can avoid typing in heated moments, while the disabled people will be able to read everything, what was spoken, fully.

This project was inspired by my personal experiences when trying to communicate with others in Minecraft, who were slow at typing and preferred to speak instead, additionally having to repeat in text on what they just said, which made it tedious, as well as this mod by Jaffe2718.

It is a client-side mod entirely, so it does not depend on any server nor uses any functionality that relies on a server, which means you can use it anywhere!! :3c

Features

• Choose any available Whisper model to download, record and use to translate speech to text!
• Client-side entirely!
• JSON config that remembers your previous choices!
• Roll your own Whisper model and use it even!
• Speaker emoji that indicates you are using the accessibility mod!
• With multilingual models, even translate into other languages or into English!

More on our Codeberg repo page!! https://codeberg.org/lunaposs/Minecraft-STTA-Mod

FAQ

Will this require a powerful GPU...?

A: Absolutely not!! In fact, you only need a CPU, Whisper doesn't have to depend on a GPU. Also, Whisper models can even run pretty well on potato machines!!

In one instance, our tester reported that the base model took only a couple of seconds to finish on a quite low-end computer :3c

Do the models require lots of memory?

A: Not quite! The smallest model available, tiny, according to whisper.cpp's GitHub page, only takes about 273 megabytes of memory. And the largest one available through this mod, the small model? 852 megabytes!

Can I use it in public v1.21+ servers?

A: Yes! Since it's entirely client-side and it does not tamper with any of server-related stuff, you should be able to without issues!

In fact, having that mod is a big boon since you only need to hold the button to speak while doing anything, compared to tedious nature of typing. :3c