A new GTA 5 mod adds AI characters you can talk to with a microphone

A veteran modder has released a mod for Grand Theft Auto V which gives NPCs the ability to talk to players using AI.

Sentient Streets is a new story-based download for the game and is created by Bloc, a modder who previously released a similar mod in Mount & Blade II: Bannerlord.

According to Bloc’s description for the mod, the NPCs in Sentient Streets “are powered by 30+ AI models that let players have open-ended conversations in real-time with eccentric cult members, cunning police officers, and unsuspecting civilians.”

Bloc demonstrates this in a video showing the mod in action. After introducing how the mod works and showing how to launch it, Bloc shows how the mod starts with the player controlling a police officer, standing in a street with three other officers.

The first task in the story involves choosing which officer will be the player’s partner. They do this by approaching each officer, pressing T to get their attention, then holding the N key and speaking into their microphone to ask them a question.

The officer will then use AI to interpret the question, generate an answer and reply to the player.  Each officer has also been given a different personality – one is friendly, one is tough and one is flirty.

The AI in the mod has been created using Inworld AI, a character engine designed to power AI-driven characters. The NPCs’ voices are then generated with text-to-speech software created by voice tech research company ElevenLabs.

https://youtu.be/cu4g743ONoo?t=178

Bloc’s mod is just one of many projects attempting to make use of AI tech to let players have fluid and improvised conversations with NPCs, rather than preset dialogue.

Earlier this year, Nvidia introduced Nvidia Ace, a new tool for developers that “aims to transform games by bringing intelligence to NPCs through AI-powered natural language interactions”.

Nvidia Ace includes a number of “optimised AI foundation models”, which help developers create backstories and personalities for their AI NPCs, implement speech recognition and text-to-speech, and generate facial animations from the generated speech.