Voice AI pioneer ElevenLabs has unveiled an AI-powered voice dubbing tool that can translate spoken audio into other languages in minutes while retaining the original speaker's voice. This new feature aims to increase content accessibility across linguistic barriers.
ElevenLabs' AI Dubbing can convert speech to text, translate it into over 20 languages such as Spanish, Hindi, and Arabic, then synthesize the translated text back into speech in the original speaker's voice. This creates a more authentic dubbing effect than traditional methods reliant on hiring voice actors.
ElevenLabs' co-founder and CEO Mati Staniszewski says that AI Dubbing represents the company's biggest advancement towards eliminating language barriers to date. He emphasizes how it will help creators easily reach global audiences and allow consumers to enjoy content in their native tongue.
The voice cloning technology clones the timbre, emotion, and speaking style of the original audio. This ensures that dubbed translations sound natural while preserving actors' distinct voices across multiple characters. ElevenLabs' research also filters out background noise and separates speech from other audio elements.
By automating dubbing, ElevenLabs makes producing multilingual content far simpler and more cost-effective for creators. This helps address the issue that only 6% of people worldwide speak English natively, while 75% speak no English at all.
ElevenLabs' launch follows Spotify's announcement last month that is will begin piloting using AI to translate podcast episodes into other languages while mimicking hosts' voices. Spotify's solution also matches speech patterns like cadence and tone to replicate podcasters' distinct voices in translated versions. However, it is currently limited to a few languages and exclusive to select popular podcasters.
With today's announcement, ElevenLabs continues to expand its catalog of AI voice solutions in speech synthesis, translation, and voice cloning. Dubbed voice translation represents a massive massive step towards dismantling linguistic barriers in the digital world and furthers ElevenLabs' goal of achieving universal accessibility for all media, regardless of native language.