YouTube now has a sound effect captioning system that can automatically identify music, sound, and laughter in videos.
It's a small but significant addition to the accessibility features of the internet's go-to video streaming site, a Google subsidiary. YouTube has offered automatic captions for dialogue tracks based on Google's voice recognition since 2009. But captioning sound effects is a much harder task, according to Google engineers, and one that's only feasible with the advancements in machine learning that have taken place over the past couple of years.
The problem is not so much a computer's ability to detect and classify things (products like Google Photos already have detection capabilities for images, for instance), but instead the lack of a significant database of sound effects to use when training the neural network that would identify them.
"While labeled ambient sound information is difficult to come by, we were able to generate a large enough dataset for training using weakly labeled data," Google engineer Sourish Chaudhuri wrote in a blog post. The team decided to focus on music, sound, and laughter first since they add meaningful context to a video's dialogue for people who are deaf or hard of hearing.
Related
- How to Download YouTube VideosHow to Download YouTube Videos
After processing thousands of hours of videos, YouTube now has a trained artificial intelligence algorithm for sound effects, which you can check out in videos like this clip from America's Got Talent (click the CC button to activate captions). The work is not yet done, however, according to Google engineer Noah Wang.
"Future challenges might include adding other common sound classes like ringing, barking and knocking, which present particular problems—for example, with ringing we need to be able to decipher if this is an alarm clock, a door or a phone," he wrote.
Its AI can detect laughter, applause and music for the deaf or hard of hearing.
AT&T and Verizon are no longer advertising on Google's video platform after discovering their ads may have appeared next to horrible, offensive content.
The DRM-free digital games store brings its software out of beta and ensures you'll never lose save games again.
Modern smartphones and cloud photo services want to automatically upload every single photo you take to the cloud. This ensures all those photos you take are safely backed up somewhere, but it isn’t ideal for every single photo. Unfortunately, companies like Apple and Google haven’t gotten that
Relax YouTube Adds Automatic Sound Effect Captions stories
AT&T and Verizon are no longer advertising on Google's video platform after discovering their ads may have appeared next to horrible, offensive content.
Google is making good on a promise to more closely monitor advertisements that appear alongside YouTube videos and give brands more control over where their ads appear.
Google said it could do better to ensure that its advertisers' content doesn't appear alongside videos with extremist and other objectionable content.
Everything you say over voice chat can be converted to text in real-time, and everything you type can also be spoken.
A night out with a delivery driver.
Launching the app will be as easy as saying 'YouTube' into the X1 voice remote.
Clearly aimed at competing with Microsoft's Skype and Google's Hangouts.
She's edgy, funny and 100 percent authentic.
Lopez's hard-luck origins and step-by-step rise to enormous success is an inspiring story that thrills entrepreneurs.
The reason why should be obvious, but many fell for PewDiePie's stunt.
People's 'Sexiest Man Alive,' who, oh yeah, is also the highest-paid actor in Hollywood, has launched a new YouTube series called 'Seven Bucks Moment.'
Tons of footage is uploaded to YouTube every minute. But what if you want to download a video? Here's how.
Seagate's 4TB Backup Plus Desktop Drive offers fast speeds and ample storage space for its price.
Its color changes when it's stretched.
The judge ruled that the mother was not only well within her rights, but her action was 'almost required by law.'
Where you can safely fly a drone, or shoot one down, is still a gray area.
While other manufacturers are ditching curved TVs, Samsung is embracing them as sales are set to peak this year.
This counts as the first public US Prime Air autonomous drone delivery, and the FAA helped make it happen. Are home deliveries far behind?
Its AI can detect laughter, applause and music for the deaf or hard of hearing.
From the bank, rather than from investors.
Filmmakers used algorithms to hijack random video content.
This counts as the first public US Prime Air autonomous drone delivery, and the FAA helped make it happen.
