Jack Dorsey's Block releases voice AI that learns only from sound

Block, the financial services firm led by Jack Dorsey, has released a new artificial intelligence model that processes audio without converting it to text first.

The system, named Dessa, was trained exclusively on human conversation. Ragavan Thurairatnam, who leads the project, shared a sample recording where the model initially mistakes a user for a person named Ronaldo. The exchange subsequently proceeds with mostly natural timing and flow.

Additional demos on the project's website show the model handling varied topics. In one clip, the conversation shifts from Christopher Nolan films to Bryan Singer's Superman movies. In another, a user describes a surprise trip to Paris, and the model responds with what the website describes as teary gratitude. A separate test pairs Dessa with OpenAI's GPT-Live-1 voice model, resulting in a more natural sounding interaction.

The developers describe Dessa as a true speech-to-speech model. They state that the system can run locally on a MacBook Pro M4 Max or better hardware. This allows the AI to process voice data on the user's device rather than sending it to a central server.

The team outlined three main reasons for building the model: curiosity, improved human-AI interaction, and safety. They compared the training process to a baby learning to speak without seeing written words. The developers noted that while this method is very hard, it is possible.

They argued that current voice assistants reduce speech to text, which slows down the interaction. The team believes speech is the most natural interface for intelligence. They claimed that existing systems often miss the emotional tone in a user's voice. They described Dessa as a step toward fixing that gap in understanding.

Block plans to give early access to a small group of AI safety researchers. The company has not announced a date for a wider public release.