5 ms·

This seems ridiculous. Based on the output of an automated tool developed by 3M that produces a saliency map based on an image, I'm not sure how we can conclude

by gfaure 7y ago

This seems ridiculous. Based on the output of an automated tool developed by 3M that produces a saliency map based on an image, I'm not sure how we can conclude that people ignore skyscrapers.

sansnomme 7y ago
How does the software work? Seam carving style "energy functions" or neural networks? If it's neural networks how did they train it? Eye tracker?
- gfaure 7y ago
  My guess is a CNN trained to produce a mask that matches the output of some eye tracker on a specifically solicited task. As another poster observed, strapping someone in and asking them to "act natural" probably introduces some unwanted bias.
- SiempreViernes 7y ago
  It seems to be old fashioned image processing using expert created feature extractors and such. https://web.archive.org/web/20180604223618/http://solutions.3m.com/wps/portal/3M/en_US/VAS_NA/Home/How2/ https://web.archive.org/web/20180604223618/http://solutions.... Validation data is eye tracking http://multimedia.3m.com/mws/media/1006827O/3msm-visual-attention-software-vas-validation-study.pdf http://multimedia.3m.com/mws/media/1006827O/3msm-visual-atte...
praptak 7y ago
I don't believe the article makes this claim. It's not about people actually ignoring the skyscrapers, but rather the low-level layers of the brain tagging them as not worth looking at. This sounds plausible - glassy skyscrapers look a lot like the sky, which isn't very interesting to look at.
- brador 7y ago
  They strapped people with a headset that tracks their eyes and told them to look at something. The brain will naturally then try to find the special thing the wearer thinks the researchers want them to look at to get a good result. Probably the sky, not the boring buildings.
  - fxtentacle 7y ago
    no. they used a smartphone app that is trying to predict where people would look.
    - SiempreViernes 7y ago
      It's a smartphone app in the same sense that any app frontend is a smartphone app: a not very relevant sense. You can find some more details in their validation study: http://multimedia.3m.com/mws/media/1006827O/3msm-visual-attention-software-vas-validation-study.pdf http://multimedia.3m.com/mws/media/1006827O/3msm-visual-atte...
- gfaure 7y ago
  No, this is wrong. In reality there's no link between the claim in the article and the way that the human visual system processes its input. The article is trying to make a leap between the output of some algorithm, and what the human visual system does. Meaningless.
  - praptak 7y ago
    I guess there is no disagreement. The evidence is indeed worthless. Still the hypothesis is interesting and plausible. Maybe there is a way to test it more rigorously, this time based on actual eye tracking.
  - SiempreViernes 7y ago
    3M put out a validation study[1] of their algorithm, an algorithm explicitly created to simulate what the eye does, and it seems to do ok compared to real eye tracking data in the example shown. Where is your evidence that it was all completely faked? [1] http://multimedia.3m.com/mws/media/1006827O/3msm-visual-attention-software-vas-validation-study.pdf http://multimedia.3m.com/mws/media/1006827O/3msm-visual-atte...
fxtentacle 7y ago
I fully agree. The author seems to confuse "this AI predicts" with "the human brain does". Looking at the quality of image recognition AI, for example the famous guacamole/cat or panda/gun mis-predictions, that's a very far stretch. https://www.labsix.org/physical-objects-that-fool-neural-nets/ https://www.labsix.org/physical-objects-that-fool-neural-net... I used to work on training a saliency-predicting NN and apart from "color and texture are great" it didn't learn much. So I would bet that taking a picture of the same building with a colorful sunset in your back, so that the sun's colors are reflected on the building but not visible in the sky, would generate a hugely different result.

by gfaure 7y ago

This seems ridiculous. Based on the output of an automated tool developed by 3M that produces a saliency map based on an image, I'm not sure how we can conclude that people ignore skyscrapers.

sansnomme 7y ago

How does the software work? Seam carving style "energy functions" or neural networks? If it's neural networks how did they train it? Eye tracker?

gfaure 7y ago
My guess is a CNN trained to produce a mask that matches the output of some eye tracker on a specifically solicited task. As another poster observed, strapping someone in and asking them to "act natural" probably introduces some unwanted bias.
SiempreViernes 7y ago
It seems to be old fashioned image processing using expert created feature extractors and such. https://web.archive.org/web/20180604223618/http://solutions.3m.com/wps/portal/3M/en_US/VAS_NA/Home/How2/ https://web.archive.org/web/20180604223618/http://solutions.... Validation data is eye tracking http://multimedia.3m.com/mws/media/1006827O/3msm-visual-attention-software-vas-validation-study.pdf http://multimedia.3m.com/mws/media/1006827O/3msm-visual-atte...

praptak 7y ago

I don't believe the article makes this claim. It's not about people actually ignoring the skyscrapers, but rather the low-level layers of the brain tagging them as not worth looking at. This sounds plausible - glassy skyscrapers look a lot like the sky, which isn't very interesting to look at.

brador 7y ago
They strapped people with a headset that tracks their eyes and told them to look at something. The brain will naturally then try to find the special thing the wearer thinks the researchers want them to look at to get a good result. Probably the sky, not the boring buildings.
- fxtentacle 7y ago
  no. they used a smartphone app that is trying to predict where people would look.
  - SiempreViernes 7y ago
    It's a smartphone app in the same sense that any app frontend is a smartphone app: a not very relevant sense. You can find some more details in their validation study: http://multimedia.3m.com/mws/media/1006827O/3msm-visual-attention-software-vas-validation-study.pdf http://multimedia.3m.com/mws/media/1006827O/3msm-visual-atte...
gfaure 7y ago
No, this is wrong. In reality there's no link between the claim in the article and the way that the human visual system processes its input. The article is trying to make a leap between the output of some algorithm, and what the human visual system does. Meaningless.
- praptak 7y ago
  I guess there is no disagreement. The evidence is indeed worthless. Still the hypothesis is interesting and plausible. Maybe there is a way to test it more rigorously, this time based on actual eye tracking.
- SiempreViernes 7y ago
  3M put out a validation study[1] of their algorithm, an algorithm explicitly created to simulate what the eye does, and it seems to do ok compared to real eye tracking data in the example shown. Where is your evidence that it was all completely faked? [1] http://multimedia.3m.com/mws/media/1006827O/3msm-visual-attention-software-vas-validation-study.pdf http://multimedia.3m.com/mws/media/1006827O/3msm-visual-atte...

fxtentacle 7y ago

I fully agree. The author seems to confuse "this AI predicts" with "the human brain does". Looking at the quality of image recognition AI, for example the famous guacamole/cat or panda/gun mis-predictions, that's a very far stretch. https://www.labsix.org/physical-objects-that-fool-neural-nets/ https://www.labsix.org/physical-objects-that-fool-neural-net... I used to work on training a saliency-predicting NN and apart from "color and texture are great" it didn't learn much. So I would bet that taking a picture of the same building with a colorful sunset in your back, so that the sun's colors are reflected on the building but not visible in the sky, would generate a hugely different result.