4 ms·

Advancing Research on Video Understanding with the YouTube-BoundingBoxes Dataset

54 points by olivercameron 10y ago

jalk 10y ago
Wondering who and how they manually boxed/labeled the objects. Sounds like an amazing job :)
- 0x00000000 10y ago
  Probably done with the new type of captcha (select all boxes with ____)
  - maxmcd 10y ago
    The bounding boxes look more exact than that. I have also only seen that new type of captcha with images that look like they are from streetview.
- Guyag 10y ago
  Their method is detailed in the whitepaper [1], mainly section 3.2. TL;DR mturk for high level annotations (frame contains/does not contain object in class), and internal annotators for bounding boxes. [1] https://arxiv.org/pdf/1702.00824.pdf https://arxiv.org/pdf/1702.00824.pdf
AndrewKemendo 10y ago
210,000 annotated videos is amazingly large, and as others asked, I really wonder how they did their training - probably offloaded it to users more than likely. Really glad to see these data sets going out to the world. It is a huge problem that Google/Apple/Facebook/Amazon have the bulk of the data gathering ability, which just means they have 100x the ability to iterate machine learning capabilities and applications. For them it's a huge win and exactly what I would be doing if I were them, but it leaves startups and smaller machine learning companies like us out there scrambling for new data sources that aren't touched by those (hint: there basically are none). edit: This data also really helps us start teaching our machine vision systems mechanical and physical processes, as described here: https://arxiv.org/abs/1603.01312 https://arxiv.org/abs/1603.01312
pmalynin 10y ago
The official site ( https://research.google.com/youtube-bb/explore.html https://research.google.com/youtube-bb/explore.html ), has interesting examples. One of them is a guy drinking what appears to be urine from a toilet with a spoon.
teLeopardthy 10y ago
It seems odd that only one object of each class is tracked in a video segment. I'd imagine that it would be a limitation for algorithms that generate bounding boxes for each class, as they might be penalized for correctly finding the non-tracked instances of a given class. Is only tracking a single instance of a class standard for this kind of dataset?
rasz_pl 10y ago
All this amazing automation and deep learning and Google still spams me with pewdiepie and other meme lord YT drama garbage suggestions despite clicking "not interested" hundreds of times.

54 points by olivercameron 10y ago

jalk 10y ago

Wondering who and how they manually boxed/labeled the objects. Sounds like an amazing job :)

0x00000000 10y ago
Probably done with the new type of captcha (select all boxes with ____)
- maxmcd 10y ago
  The bounding boxes look more exact than that. I have also only seen that new type of captcha with images that look like they are from streetview.
Guyag 10y ago
Their method is detailed in the whitepaper [1], mainly section 3.2. TL;DR mturk for high level annotations (frame contains/does not contain object in class), and internal annotators for bounding boxes. [1] https://arxiv.org/pdf/1702.00824.pdf https://arxiv.org/pdf/1702.00824.pdf

AndrewKemendo 10y ago

210,000 annotated videos is amazingly large, and as others asked, I really wonder how they did their training - probably offloaded it to users more than likely. Really glad to see these data sets going out to the world. It is a huge problem that Google/Apple/Facebook/Amazon have the bulk of the data gathering ability, which just means they have 100x the ability to iterate machine learning capabilities and applications. For them it's a huge win and exactly what I would be doing if I were them, but it leaves startups and smaller machine learning companies like us out there scrambling for new data sources that aren't touched by those (hint: there basically are none). edit: This data also really helps us start teaching our machine vision systems mechanical and physical processes, as described here: https://arxiv.org/abs/1603.01312 https://arxiv.org/abs/1603.01312

pmalynin 10y ago

The official site ( https://research.google.com/youtube-bb/explore.html https://research.google.com/youtube-bb/explore.html ), has interesting examples. One of them is a guy drinking what appears to be urine from a toilet with a spoon.

teLeopardthy 10y ago

It seems odd that only one object of each class is tracked in a video segment. I'd imagine that it would be a limitation for algorithms that generate bounding boxes for each class, as they might be penalized for correctly finding the non-tracked instances of a given class. Is only tracking a single instance of a class standard for this kind of dataset?

rasz_pl 10y ago

All this amazing automation and deep learning and Google still spams me with pewdiepie and other meme lord YT drama garbage suggestions despite clicking "not interested" hundreds of times.