3 ms·

one of cool google projects that does not take off because its so hard to run and integrate with other cool things...

by netdur 4y ago

philjohn 4y ago
If we take the less cynical take, it'll end up in Youtube as audio description for people with visual impairments.
- moritonal 4y ago
  It'll become a standard option in CCTV and as an extension a tool for overly authoritive bosses. "the employee is talking to X, the employee is working, the employee is going to the toilet"
- martyvis 4y ago
  It is perfect for Audio Description [1]. Certain broadcast TV programs have had this here in Australia to allow those with poor vision to be able to follow along - it allows them to appreciate content that has originally be produced as audio-visual. https://en.wikipedia.org/wiki/Audio_description?wprov=sfla1 https://en.wikipedia.org/wiki/Audio_description?wprov=sfla1
TaylorAlexander 4y ago
This isn’t really a “google project” in the way I think about that term, but it’s a research project. Google’s research is constantly advancing and when things get far enough along they do tend to get used in production. Individual research papers are just a step along the way. This research seems useful for training video generation systems like transformers and especially multi modal systems. Imagine you have a robot that needs to understand the world around it. It needs to interpret text input (likely as voice) but it also needs to understand complex scenes around it. If you can get a system to accurately describe YouTube videos (a captive data set) then it should also be able to understand a live video feed on a robot. That’s an important part of a robot. But it is not in itself a product or notable project.
- deleted 4y ago
  [deleted]
notatoad 4y ago
I suspect this or something like it has already “taken off”, but only internally at google. They seem to be a lot better at searching the contents of YouTube videos than simple transcripts or image detection would give.

one of cool google projects that does not take off because its so hard to run and integrate with other cool things...

by netdur 4y ago

one of cool google projects that does not take off because its so hard to run and integrate with other cool things...

philjohn 4y ago

If we take the less cynical take, it'll end up in Youtube as audio description for people with visual impairments.

moritonal 4y ago
It'll become a standard option in CCTV and as an extension a tool for overly authoritive bosses. "the employee is talking to X, the employee is working, the employee is going to the toilet"
martyvis 4y ago
It is perfect for Audio Description [1]. Certain broadcast TV programs have had this here in Australia to allow those with poor vision to be able to follow along - it allows them to appreciate content that has originally be produced as audio-visual. https://en.wikipedia.org/wiki/Audio_description?wprov=sfla1 https://en.wikipedia.org/wiki/Audio_description?wprov=sfla1

TaylorAlexander 4y ago

This isn’t really a “google project” in the way I think about that term, but it’s a research project. Google’s research is constantly advancing and when things get far enough along they do tend to get used in production. Individual research papers are just a step along the way. This research seems useful for training video generation systems like transformers and especially multi modal systems. Imagine you have a robot that needs to understand the world around it. It needs to interpret text input (likely as voice) but it also needs to understand complex scenes around it. If you can get a system to accurately describe YouTube videos (a captive data set) then it should also be able to understand a live video feed on a robot. That’s an important part of a robot. But it is not in itself a product or notable project.

deleted 4y ago
[deleted]

notatoad 4y ago

I suspect this or something like it has already “taken off”, but only internally at google. They seem to be a lot better at searching the contents of YouTube videos than simple transcripts or image detection would give.