Speaking Villagers - ChatGPT and TTS

Minecraft 版本： 1.21

平台： fabric

标签： adventure mobs social

Speaking Villagers – AI Conversations, Friendships & Quests

Talk to villagers in Minecraft with AI-generated responses, build friendships, and receive dynamic quests!

Now with Mistral API support! Free to use with your own API key ? Now with player speech recognition!

Read the full description or check the in-game ModMenu for setup instructions!

Video is kinda old, mod is better now than whats shown there.

Overview

Speaking Villagers enhances villager interactions in Minecraft by introducing AI-generated conversations, dynamic villager-player friendships and quests.
Villagers now recognize their surroundings, react to your actions, and remember past interactions.

Each villager is assigned one of 8000+ distinct personalities, making every interaction feel even more unique.

Some features (TTS & voice recognition) only work on Windows PCs.

Highly recommend to use with ModMenu!

? Version 0.6 – Another Major Update!

• Player Voice Input: Talk to villagers directly with your microphone using speech-recognition. (setup below)
• 23 Unique Voices: English, German, and Multilingual voices, selectable via ModMenu.
• Overhauled Quest System: 176 different gather quests and 210 profession-based rewards with improved balance and scaling.
• Expanded Personality Engine: Villagers now have over 8,000 unique personality combinations.
• Improved Gift System: Villagers now drop gifted items on death and only pick up items dropped by players.
• Experimental Language Settings: Villagers can auto-detect the player's language or use a set preferred language.
• Redesigned ModMenu Config: Many new options to control TTS, memory, speech models, awareness, and more.
• No more environment variables required, as API keys can be set via ModMenu
• Important Bug Fixes: No more /speak crashes when API keys are missing, along with general stability improvements.

? Full changelog under the file!

Features

✅ AI Conversations – Talk to villagers with smart, contextual responses
✅ Context Awareness – Villagers respond to:

• Your player name, armor, held items, hotbar, health and hunger
• Biome, weather, and time of day
• Nearby mobs, blocks, and structures
• Active potion effects and game mode
• Reputation and friendship level
• Whether they're indoors, in bed, swimming, or riding

✅ Friendship System – Befriend villagers and they’ll follow you
✅ Quest System – Villagers offer procedurally generated quests
✅ Gift System – Villagers pick up items and comment on them (must be activated via ModMenu)
✅ 8000+ different personalities – Each villager feels unique
✅ Windows Voice Support – Villagers speak aloud using Windows Speech API
✅ Works alongside the Talking Mobs Mod

?️ Fully customizable via ModMenu: Choose AI model, set limits, toggle systems, and more!

API Setup (Mistral or OpenAI)

? Free Option – Mistral

• Create a free account at mistral.ai
• Generate an API key - you can find it under "La plateforme" on their site when you are logged in.
• Add this to your system as an environment variable: MISTRAL_API_KEY=your_api_key
• From version 0.6 on you don't need to set API keys as environment variables anymore, alternatively you can now set them via ModMenu!

Mistral usage is limited to 1.000.000.000 tokens/month, which you will not hit by only using this mod

? Paid Option – OpenAI

• Requires an OpenAI account and API key
• Add this to your system as: OPENAI_API_KEY=your_api_key
• From version 0.6 on you don't need to set API keys as environment variables anymore, alternatively you can now set it via ModMenu!
• Billed per token

No API key = No AI responses = No charges. You decide which one to use via ModMenu.

Restart your PC after setting environment variables to take effect. Alternatively set 'em via ModMenu (no restart needed)

Installation

1. ? Install the Mod
   Drop the .jar into your mods folder.

2. ? Install Natural Voice SAPI Adapter for better villager speech (only works on Windows)
   Without it, voices will sound robotic.
   ? Download Adapter

3. ? Set up your API key
   Choose OpenAI or/and Mistral — both supported. From version 0.6 on you don't need to set API keys as environment variables anymore, alternatively you can now set them via ModMenu!

4. ? Setup Voice recognition (optional)

• Run the mod once and enable voice recognition in ModMenu.
• When you attempt to speak, the mod will display the required model path (usually under ./AppData/Local/SpeakingVillagers/vosk_model).
• Download a Vosk model (e.g., from alphacephei.com/vosk/models) and extract its contents into the shown folder.
• Restart the game and try speaking to a villager again.

? Download Vosk models (recommended: vosk-model-small-en-us-0.15).

Larger models require high-end hardware.

Commands

• /speak <message> – Say something to a nearby villager
• /befriend – Manually start a friendship with the nearest villager
• /unfriend – Manually end a friendship with the nearest villager
• /quests - shows your active quests
• /villagertraits - shows the personality traits of all villagers nearby
• /tokens - shows how many tokens you have used, mainly for debugging/info, not relevant for gameplay

Compatibility

✅ Compatible with most other mods
⚠ Some features require Windows (uses Windows Speech API for TTS, Vosk natives for speech recognition)
⚠ Tested in Singleplayer only — should work on servers when installed on both server and client, but full multiplayer optimization is still pending.
⚠ Java 21 is required. Minecraft 1.20.x users must manually set the Java Executable in their launcher settings to point to Java 21 (javaw.exe), as it defaults to Java 17 otherwise and the game may crash on launch. 

? Need help? Check the wiki!

☕ Wanna support this project? Buy me a coffee :)

? Now go chat with your villagers and have fun!

Planned features for future updates

• Full multiplayer compatibility
• Support for other TTS APIs e.g. ElevenLabs - would bring support TTS support for more operating systems, more voices and languages.
• GUI and visual improvements
• Translating settings to more languages