3 ms·

AVA: The Art and Science of Image Discovery at Netflix

89 points by trueduke 8y ago

aub3bhat 8y ago
You can setup AVA style video processing pipeline using my project https://www.deepvideoanalytics.com/ https://www.deepvideoanalytics.com/ It comes built in with face detection/recognition, object detection, OCR, Open Images tagger etc.
- rememberlenny 8y ago
  This is an incredible product. I have come across the github repo's authors work a number of times now, and I am continually impressed by the quality of documentation, examples, and immediately usable code.
invalidusernam3 8y ago
I love this niche type of project, sounds like it would be such a fun project to work on.
m3kw9 8y ago
Why don’t they Kaggle it?
- tedsanders 8y ago
  Netflix cancelled its second million-dollar Netflix Prize in 2010 due to privacy lawsuits. Netflix has not outsourced its data science since. https://www.wired.com/2010/03/netflix-cancels-contest/ https://www.wired.com/2010/03/netflix-cancels-contest/
bwang29 8y ago
I see one potential downside of this, as a Netflix subscriber I do find the frequent shuffle and changes of cover art confusing sometimes as it makes you feel there is a new film coming out but in fact you've already watched it.
- dtien 8y ago
  Actually, this might be one of the intended effects of this feature. What better way to make it appear their catalog is wider and ever-changing then to cycle through these static stills every so often. It may not be the most user friendly design pattern to 'trick' your user into thinking there's a bigger catalog, but I can understand the desire for it from their perspective. And to be perfectly honest, sometimes those changing stills have actually made me watch a movie that I had previously glossed over because the image wasn't as interesting. So from personal experience I can say their primary intent was reached w/ the periodic cycling of stills.
nmstoker 8y ago
Maybe they're not telling about some of the detail (and they've clearly given it plenty of thought!) but it seems wasteful to fully process all frames when in several cases many of them will never be selected. eg if it fails due to poor Visual Metadata (such as strong motion blur or very low brightness) you'd want to abandon all frames like that before running the other processing.
- mkl 8y ago
  They probably do, for efficiency. You have to process a frame to some extent to detect motion blur or darkness (how else could you possibly know?), but that doesn't mean you can't abort the processing as soon as you detect the frame is totally unsuitable. I expect whole groups of frames may be able to be discarded more-or-less together, e.g. a dark key frame followed by frames with minimal difference, or where the video encoding's motion estimation says there's a lot moving.

89 points by trueduke 8y ago

You can setup AVA style video processing pipeline using my project https://www.deepvideoanalytics.com/ https://www.deepvideoanalytics.com/ It comes built in with face detection/recognition, object detection, OCR, Open Images tagger etc.

rememberlenny 8y ago
This is an incredible product. I have come across the github repo's authors work a number of times now, and I am continually impressed by the quality of documentation, examples, and immediately usable code.

invalidusernam3 8y ago

I love this niche type of project, sounds like it would be such a fun project to work on.

m3kw9 8y ago

Why don’t they Kaggle it?

tedsanders 8y ago
Netflix cancelled its second million-dollar Netflix Prize in 2010 due to privacy lawsuits. Netflix has not outsourced its data science since. https://www.wired.com/2010/03/netflix-cancels-contest/ https://www.wired.com/2010/03/netflix-cancels-contest/

bwang29 8y ago

I see one potential downside of this, as a Netflix subscriber I do find the frequent shuffle and changes of cover art confusing sometimes as it makes you feel there is a new film coming out but in fact you've already watched it.

dtien 8y ago
Actually, this might be one of the intended effects of this feature. What better way to make it appear their catalog is wider and ever-changing then to cycle through these static stills every so often. It may not be the most user friendly design pattern to 'trick' your user into thinking there's a bigger catalog, but I can understand the desire for it from their perspective. And to be perfectly honest, sometimes those changing stills have actually made me watch a movie that I had previously glossed over because the image wasn't as interesting. So from personal experience I can say their primary intent was reached w/ the periodic cycling of stills.

nmstoker 8y ago

Maybe they're not telling about some of the detail (and they've clearly given it plenty of thought!) but it seems wasteful to fully process all frames when in several cases many of them will never be selected. eg if it fails due to poor Visual Metadata (such as strong motion blur or very low brightness) you'd want to abandon all frames like that before running the other processing.

mkl 8y ago
They probably do, for efficiency. You have to process a frame to some extent to detect motion blur or darkness (how else could you possibly know?), but that doesn't mean you can't abort the processing as soon as you detect the frame is totally unsuitable. I expect whole groups of frames may be able to be discarded more-or-less together, e.g. a dark key frame followed by frames with minimal difference, or where the video encoding's motion estimation says there's a lot moving.