YouTube is going to help its creators reach an international audience as the platform plans on introducing a new AI-powered dubbing tool for translating videos into other languages.
Announced at VidCon 2023, the goal of this latest endeavor is to provide a quick and easy way for creators to translate “at no cost” their content into languages they don’t speak. This can help out smaller channels as they may not have the resources to hire a human translator. To make this all possible, Amjad Hanif, vice president of Creator Products at YouTube, revealed the tool will utilize the Google-created Aloud plus the platform will be bringing over the team behind the AI from Area 120, a division of the parent company that frequently works on experimental tech.
Easy translation
The way the translation system works, according to the official Aloud website, is the AI will first transcribe a video into a script. You then edit the transcription to get rid of any errors, make clarifications, or highlight text “where timing is critical.” From there, you give the edited script back to Aloud where it will automatically translate your video into the language of your choice. Once done, you can publish the newly dubbed content by uploading any new audio tracks onto their original video.
A Google representative told us “creators do not have to [actually] understand any of the languages that they are dubbing into.” Aloud will handle all of the heavy lifting surrounding complex tasks like “translation, timing, and speech synthesis.” Again, all you have to do is double-check the transcription.
Future changes
It’s unknown when the Aloud update will launch. However, YouTube is already working on expanding the AI beyond what it’s currently possible. Right now, Aloud can only translate English content to either Spanish or Portuguese. But there are plans to expand into other languages from Hindi to Indonesian plus support for different dialects.
Later down the line, the platform will introduce a variety of features such as “voice preservation, better emotion transfer, and even lip reanimation” to improve enunciation. Additionally, YouTube is going to build in some safeguards ensuring only the creators can “dub their own content”.
The same Google representative from earlier also told us the platform is testing the Aloud AI with “hundreds of [YouTube] creators” with plans to add more over time. As of June 2023, over 10,000 videos have been dubbed in over 70 languages.
You can join the early access program by filling out the official Google Docs form. If you want to know what an Aloud dub sounds like, go watch the channel trailer for the Amoeba Sisters channel on YouTube. Click the gear icon, go to Audio Track, then select Spanish. The robotic voice you’ll hear is what the AI will create.
English
Arabic
Belarusian
Bengali
Bosnian
Bulgarian
Chinese (Simplified)
Croatian
Czech
Danish
Dutch
Estonian
Filipino
Finnish
French
Georgian
German
Greek
Gujarati
Hausa
Hebrew
Hindi
Hmong
Hungarian
Igbo
Indonesian
Italian
Japanese
Javanese
Kazakh
Korean
Kurdish (Kurmanji)
Kyrgyz
Lao
Latvian
Lithuanian
Macedonian
Malagasy
Norwegian
Persian
Polish
Portuguese
Punjabi
Romanian
Russian
Serbian
Slovak
Slovenian
Spanish
Swedish
Thai
Turkish
Ukrainian
Vietnamese
Yoruba