3 ms·

"By detecting the subjects of interest, AutoFlip is able to avoid cropping off important visual content." And they manage to demo it with a scene where a lady

"By detecting the subjects of interest, AutoFlip is able to avoid cropping off important visual content." And they manage to demo it with a scene where a lady talks to a guy in a checkered shirt in front of a yellow background. The algorithm cuts off the lady completely and only keeps the guy. I guess she mustn't have been important. Next time she should try to look more like "interesting, salient content".

aloknnikhil 7y ago
> AutoFlip’s configuration graph provides settings for either best-effort or required reframing. If it becomes infeasible to cover all the required regions (for example, when they are too spread out on the frame), the pipeline will automatically switch to a less aggressive strategy by applying a letterbox effect, padding the image to fill the frame. Looks like that was particularly demoing the "best effort" mode. If you see the demo near that paragraph, it highlights the possibility of having certain faces cut-off unless you ask for all faces to be required.
ZeroGravitas 7y ago
They cover this on the what's next section: > Like any machine learning algorithm, AutoFlip can benefit from an improved ability to detect objects relevant to the intent of the video, such as speaker detection for interviews
rcthompson 7y ago
Yeah, I noticed that. While it's possible this is the result of the machine learning algorithm implicitly picking up a male bias in the data set that causes it to rank men more highly in terms of importance, I don't know how likely that is in practice. Maybe the explanation is as simple as the man taking up more pixels, or a bias toward lighter colors (like the man's white & blue shirt) while black colors (like the woman's clothing) are more likely to be considered unimportant background. Maybe the algorithm considered both approximately equally important and the result was essentially a coin flip. Regardless, I agree that it's not a great look when the caption on that example talks about how AutoFlip "detects the subjects of interest" and "avoids cropping off important visual content". It's definitely not sending the intended message.
- drusepth 7y ago
  I interpreted it as just picking one visual entity to focus on for that cropped clip for design reasons, since the letterboxes necessary to get both entities in that portrait orientation would be huge and visually unappealing [1]. [1] https://i.imgur.com/yxxEnGe.png https://i.imgur.com/yxxEnGe.png

aloknnikhil 7y ago

> AutoFlip’s configuration graph provides settings for either best-effort or required reframing. If it becomes infeasible to cover all the required regions (for example, when they are too spread out on the frame), the pipeline will automatically switch to a less aggressive strategy by applying a letterbox effect, padding the image to fill the frame. Looks like that was particularly demoing the "best effort" mode. If you see the demo near that paragraph, it highlights the possibility of having certain faces cut-off unless you ask for all faces to be required.

ZeroGravitas 7y ago

They cover this on the what's next section: > Like any machine learning algorithm, AutoFlip can benefit from an improved ability to detect objects relevant to the intent of the video, such as speaker detection for interviews

rcthompson 7y ago

Yeah, I noticed that. While it's possible this is the result of the machine learning algorithm implicitly picking up a male bias in the data set that causes it to rank men more highly in terms of importance, I don't know how likely that is in practice. Maybe the explanation is as simple as the man taking up more pixels, or a bias toward lighter colors (like the man's white & blue shirt) while black colors (like the woman's clothing) are more likely to be considered unimportant background. Maybe the algorithm considered both approximately equally important and the result was essentially a coin flip. Regardless, I agree that it's not a great look when the caption on that example talks about how AutoFlip "detects the subjects of interest" and "avoids cropping off important visual content". It's definitely not sending the intended message.

drusepth 7y ago
I interpreted it as just picking one visual entity to focus on for that cropped clip for design reasons, since the letterboxes necessary to get both entities in that portrait orientation would be huge and visually unappealing [1]. [1] https://i.imgur.com/yxxEnGe.png https://i.imgur.com/yxxEnGe.png