7 ms·

A Horrible Experiment

87 points by marvindanig 6y ago

daenz 6y ago
Isn't Mitch McConnell in the news right now far more "intensely" than Obama? If I was writing a system to pick a thumbnail, I would pick the thumbnail that is most relevant to current tweets, which is definitely McConnell. A quick way to test this would be to use two well known figures, where the PoC is definitely more in the news than the other.
pityJuke 6y ago
Some more info: - This apparently done via a NN [1] - There are mixed results, with various factors playing a factor [2]. Worth digging into this thread, and the original thread where this was figured out. It's intriguing. [1]: https://blog.twitter.com/engineering/en_us/topics/infrastructure/2018/Smart-Auto-Cropping-of-Images.html https://blog.twitter.com/engineering/en_us/topics/infrastruc... [2]: https://mobile.twitter.com/NotAFile/status/1307337294249103361 https://mobile.twitter.com/NotAFile/status/13073372942491033...
DanBC 6y ago
See also Zoom backgrounds: https://twitter.com/colinmadland/status/1307111816250748933?s=20 https://twitter.com/colinmadland/status/1307111816250748933?...
greenduck 6y ago
This is pretty egregious. For those of us working on machine learning, does anybody know of datasets that have decent representation of different colors of skin. That seems like the cause here and thinking back, my datasets aren't the most diverse out there.
- TazeTSchnitzel 6y ago
  ML inherits the biases of the data it is trained on. If you want “unbiased” ML models, you must train them on data that is carefully constructed to avoid the biases deemed undesirable.
  - marvindanig 6y ago
    The way HN’s topic floatation algorithm is behaving on this thread right now makes me suspicious about this forum as well. I am not sure if it’s an ML model issue here or just some quiet downvotes by the you-know-who.
    - Lammy 6y ago
      I don’t know “who”. Please tell me. I’m sorry to sound flippant but I mean this sincerely since I’ve never observed any correlation between downvote-used-as-disagree and any particular beliefs.
      - marvindanig 6y ago
        > "never observed any correlation between downvote-used-as-disagree and any particular beliefs." Quick question: Why would anyone downvote or upvote anything if they didn't believe in something? Your line of query isn't flippant, I agree. It's a reaction to a privilege at worst or a denial of why this thread was downvoted by the you-do-know-who in the first place. Edit: I wasn't certain earlier, but now thanks for proving the point.
        Lammy 6y ago
        I 'vouch'ed for your comment, for what it's worth. I still don't know 'who'.
        marvindanig 6y ago
        Sure. Well, maybe you should ask this question to the mods then. Just like you I'm all for outing the 'who' here.
  - GuB-42 6y ago
    I remember I took an introduction class in machine learning when I was a student. I was surprised how important bias removal was. Preparing the dataset was more than half of the work. And the class was more about statistics than anything else. It wasn't about racial bias, ethics, or anything like that. More like if you feed too many "5" in your handwriting recognition algorithm, it will will start seeing 5 everywhere.
- Lammy 6y ago
  No it isn’t. Look at the massive empty space this person had to construct between the two content areas of those tall images to get it to behave like this. No reasonable person would ever create something like that accidentally in a way that would create this behavior.
- svrb 6y ago
  The dataset linked from this thread on Twitter may serve your needs too?: https://twitter.com/vinayprabhu/status/1307497736191635458?s=19 https://twitter.com/vinayprabhu/status/1307497736191635458?s...
kingosticks 6y ago
There's a whole lot of random people chiming in and guessing on Twitter (shocker) so it is worth pointing out that: > The algorithm does not do face detection at all (it actually replaced a previous algorithm which did). From https://twitter.com/ZehanWang/status/1307461285811032066?s=20 https://twitter.com/ZehanWang/status/1307461285811032066?s=2... Not to say there is or isn't a problem to be fixed, more work is needed to understand. Such as the analysis at https://twitter.com/vinayprabhu/status/1307460502017028096?s=20 https://twitter.com/vinayprabhu/status/1307460502017028096?s...
- 8fGTBjZxBcHq 6y ago
  Why is it worth pointing that out? Like you said, the problem is there either way. What are lay people supposed to gain from hearing that the algorithm's designers don't consider it to be based on face detection?
  - kingosticks 6y ago
    Because it has totally derailed many of the twitter threads. Thankfully we get less conspiracy theories here but I figured we could try and avoid it altogether.
Gys 6y ago
It seems this needs a twitter account to see? Maybe someone can post a screenshot?
- acqq 6y ago
  And for those of us who need a textual description to get it, maybe someone can post an explanation or description of what is expected and what's an observed behavior? Anybody, please?
  - Nzen 6y ago
    Twitter crops rather than rescales posted images to some 'interesting' subset of an image that doesn't fit the preferred aspect ratio. So, the person behind this account created an antagonistic situation: made a very tall, thin image with Mitch Mconnell on the top and Barak Obama on the bottom, and another with them swapped between top and bottom. In the initial post, the cropped thumbnail shows Mitch Mconnell for both. The thread has two more situations, one with the ties (red/blue) flipped and the colors inverted. The only time it 'prefers' Obama's end of the antagonistic image is with inverted colors and him on top.
  - yc-kraln 6y ago
    Twitter has an algorithm to pick which part of oblong images to showcase in the feed. People have been posting examples where, given the "choice" between a white person and a person of color, the white person is always picked, regardless of orientation or where they are in the picture. This was noticed in a thread about how Zoom's background detection seems to not work great for people of color, and how the WebEx cross-talk protection seems to favor men over women.
fr2null 6y ago
Interestingly, swapping the glasses does seem to make the thumbnail Obama. [1] 1. https://twitter.com/SergioSemJ/status/1307493041742254080?s=19 https://twitter.com/SergioSemJ/status/1307493041742254080?s=...
- nichos 6y ago
  So does the smile: https://twitter.com/MASCARPOWN/status/1307454405059452928?s=09 https://twitter.com/MASCARPOWN/status/1307454405059452928?s=... And actually swapping the smile: https://twitter.com/DrWhen2/status/1307497434872905728?s=19 https://twitter.com/DrWhen2/status/1307497434872905728?s=19 Twitter is nothing if not full of victims and people talking about things they know nothing about.
dazbradbury 6y ago
https://twitter.com/SergioSemJ/status/1307493041742254080?s=19 https://twitter.com/SergioSemJ/status/1307493041742254080?s=... One theory is it's actually the glasses increasing the likelihood of correctly identifying a face (or "feature" element in the photo). Interesting looking through the various tests in the thread, as clearly lots of variables here, but no way this will be exhaustive. Suspect this might kick off some serious arguments at Twitter HQ for allowing users to pick the cropping vs. an algorithm regardless of any bias found one way or not.
- DanBC 6y ago
  Here's an example where neither person is wearing glasses. https://twitter.com/LOVEMUNY/status/1307789227535499266 https://twitter.com/LOVEMUNY/status/1307789227535499266 It also includes an example where the black man is wearing glasses and the white man isn't.
nailer 6y ago
Misleading - it seems to be looking for contrast https://twitter.com/kim/status/1307548258491801600?s=19 https://twitter.com/kim/status/1307548258491801600?s=19 UPDATE: someone wrote a boy to test the conspiracy and properly debunked it, source code is online: https://twitter.com/vinayprabhu/status/1307497736191635458?s=19 https://twitter.com/vinayprabhu/status/1307497736191635458?s...
noxer 6y ago
The real problem I see here is the default assumption from many people that these is racial bias in the algorithm with no evidence thereof whatsoever. Thats how twitter works, shitstorm first, research later, evidence or corrections for false assumptions - never.
- matthewmacleod 6y ago
  I mean “no evidence whatsoever” is a pretty big stretch. There’s the initial example, a bunch of other examples of the same effect posted in the thread, and it’s already a really well-known and obvious effect that we have seen occur many times before.
  - noxer 6y ago
    That called "anecdotal" not an evidence.
    - odonnellryan 6y ago
      Anecdotal evidence is still evidence. These are not anecdotes however, but probably not very significant.
      - noxer 6y ago
        lok what. "Anecdotal evidence" specifically described a false evidence. Just because "evidence" is part of the term doesn't mean it a valid evidence.
        411111111111111 6y ago
        That's incorrect. Anecdotes aren't necessarily wrong/incorrect. They're just Stories which may or may not be coincidences, falsehoods or representative of the reality. An anecdote is just a single data point, so no conclusion can be drawn from it alone Its true that it's often used to discredit someone, often even in bad faith... But that doesn't mean that it's inherintly derogatory
Kednicma 6y ago
Humans can't make unbiased computers. Every computer which humans make is indelibly human as well. This is the anthropic bias.
- noxer 6y ago
  It's not a biased if isn't a result of a human bias. If the algo picks the spot with the highest contrast for example, that can not be biased it may lead to unwanted results but its not biased.
  - Aeronwen 6y ago
    Someone decided the algo was producing usable results when it was picking the highest contrast areas.
    - noxer 6y ago
      Because it did. I'm sure they made tests and decided it worked good enough. It wasn't made only for faces/persons especially not for images with 2 faced far apart. It also never was meant to decide between 2 possibly "image centers". Its not biased because you can craft an image where it procedure unwanted results. The only persons biased are the people who see a bias in everything.
TazeTSchnitzel 6y ago
Wonderful example of the ethics of ML. The people whose eyes they tracked gravitated towards certain kinds of faces (not necessarily just based on skin colour), so now Twitter's model crops other faces out of pictures, among other problems (now it likes cleavage apparently). The old approach may have been a bit mechanical but it avoided the non-neutrality of unconscious human perception. The assumption that what human eyes gravitate to is what an image should be cropped to is a very big and questionable one. The first thing I look at in a picture is not always the most important part, and the image as a whole has value.
dmitriid 6y ago
It's funny how these big companies willingly convert a non-problem into massive problems with hundreds of hours of engineering effort and huge social implications. Non-problem: don't mess with the images, and display them as is. Problem: see Twitter
t0astbread 6y ago
Why does Twitter even try so hard to make preview crops "meaningful"? Sure, it produces interesting results sometimes (sometimes funny, sometimes less so) but isn't it a lot of work for not a lot of benefit?
- noxer 6y ago
  Agree, resize would in most cases be way more useful.
- Lammy 6y ago
  I used to get annoyed by this too but came around to accepting it after building a Web (Jekyll) photo processing pipeline of my own. Art direction is very difficult to do well when you have to be able to handle any user-uploaded photo that might be in any aspect ratio. Desktops favor displaying wide photos, and handsets favor displaying tall ones. Cropping everything down to a square lets you handle any combination of photo and display aspect ratios with minimal Badness. I’ve never seen the Twitter-dot-com codebase but would be willing to bet it’s probably using VIPS’ smartcrop feature (:crop => :attention) https://www.rubydoc.info/gems/ruby-vips/Vips/Interesting https://www.rubydoc.info/gems/ruby-vips/Vips/Interesting
- matthewmacleod 6y ago
  Cropping well is quite a hard problem! I know it seems on the surface like “just resize it” is a good approach, but it’s not super uncommon to see quite extreme aspect images, and even for normal use there are lots of different constraints in different systems that mean a really simple system can produce undesirable results. I used to get this all the time back when I built custom content management systems. Users would upload images which were automatically cropped into various sizes for use in particular bits of content; really long or wide images would look awful when scaled, and automatic fixed-size crops would often end up cutting out the actual interesting bit of the image. That’s what various “automatically generate interesting thumbnails” algorithms try to avoid. In the end for me the best tool was auto-crop-with-preview-and-manual-override, and I’m beyond infuriated every time I try to upload an image to Twitter and it doesn’t allow this obvious and simple way to avoid the issue.
rambojazz 6y ago
I'm out of the loop. What's happening here? All I see is a picture.
- s_gourichon 6y ago
  Glad you asked. There are two pictures in each tweet, shown left and right. You can click on each to see the full picture. Each picture is actually two portraits (Mitch McConnell and Barack Obama), one above one below, separated by a big gap. Since apparently Twitter automatically chooses to crop pictures in default tweet display, the OP posted different cases to see where Twitter would crop the image. And it happens that Twitter chose one of the portrait always, until the OP reversed color. Some people wonder if it is because Barack Obama is black. Whether that's the reason or not, the observable fact remains.

87 points by marvindanig 6y ago

Isn't Mitch McConnell in the news right now far more "intensely" than Obama? If I was writing a system to pick a thumbnail, I would pick the thumbnail that is most relevant to current tweets, which is definitely McConnell. A quick way to test this would be to use two well known figures, where the PoC is definitely more in the news than the other.

pityJuke 6y ago

Some more info: - This apparently done via a NN [1] - There are mixed results, with various factors playing a factor [2]. Worth digging into this thread, and the original thread where this was figured out. It's intriguing. [1]: https://blog.twitter.com/engineering/en_us/topics/infrastructure/2018/Smart-Auto-Cropping-of-Images.html https://blog.twitter.com/engineering/en_us/topics/infrastruc... [2]: https://mobile.twitter.com/NotAFile/status/1307337294249103361 https://mobile.twitter.com/NotAFile/status/13073372942491033...

DanBC 6y ago

See also Zoom backgrounds: https://twitter.com/colinmadland/status/1307111816250748933?s=20 https://twitter.com/colinmadland/status/1307111816250748933?...

greenduck 6y ago

This is pretty egregious. For those of us working on machine learning, does anybody know of datasets that have decent representation of different colors of skin. That seems like the cause here and thinking back, my datasets aren't the most diverse out there.

TazeTSchnitzel 6y ago
ML inherits the biases of the data it is trained on. If you want “unbiased” ML models, you must train them on data that is carefully constructed to avoid the biases deemed undesirable.
- marvindanig 6y ago
  The way HN’s topic floatation algorithm is behaving on this thread right now makes me suspicious about this forum as well. I am not sure if it’s an ML model issue here or just some quiet downvotes by the you-know-who.
  - Lammy 6y ago
    I don’t know “who”. Please tell me. I’m sorry to sound flippant but I mean this sincerely since I’ve never observed any correlation between downvote-used-as-disagree and any particular beliefs.
    - marvindanig 6y ago
      > "never observed any correlation between downvote-used-as-disagree and any particular beliefs." Quick question: Why would anyone downvote or upvote anything if they didn't believe in something? Your line of query isn't flippant, I agree. It's a reaction to a privilege at worst or a denial of why this thread was downvoted by the you-do-know-who in the first place. Edit: I wasn't certain earlier, but now thanks for proving the point.
      - Lammy 6y ago
        I 'vouch'ed for your comment, for what it's worth. I still don't know 'who'.
        marvindanig 6y ago
        Sure. Well, maybe you should ask this question to the mods then. Just like you I'm all for outing the 'who' here.
- GuB-42 6y ago
  I remember I took an introduction class in machine learning when I was a student. I was surprised how important bias removal was. Preparing the dataset was more than half of the work. And the class was more about statistics than anything else. It wasn't about racial bias, ethics, or anything like that. More like if you feed too many "5" in your handwriting recognition algorithm, it will will start seeing 5 everywhere.
Lammy 6y ago
No it isn’t. Look at the massive empty space this person had to construct between the two content areas of those tall images to get it to behave like this. No reasonable person would ever create something like that accidentally in a way that would create this behavior.
svrb 6y ago
The dataset linked from this thread on Twitter may serve your needs too?: https://twitter.com/vinayprabhu/status/1307497736191635458?s=19 https://twitter.com/vinayprabhu/status/1307497736191635458?s...

kingosticks 6y ago

There's a whole lot of random people chiming in and guessing on Twitter (shocker) so it is worth pointing out that: > The algorithm does not do face detection at all (it actually replaced a previous algorithm which did). From https://twitter.com/ZehanWang/status/1307461285811032066?s=20 https://twitter.com/ZehanWang/status/1307461285811032066?s=2... Not to say there is or isn't a problem to be fixed, more work is needed to understand. Such as the analysis at https://twitter.com/vinayprabhu/status/1307460502017028096?s=20 https://twitter.com/vinayprabhu/status/1307460502017028096?s...

8fGTBjZxBcHq 6y ago
Why is it worth pointing that out? Like you said, the problem is there either way. What are lay people supposed to gain from hearing that the algorithm's designers don't consider it to be based on face detection?
- kingosticks 6y ago
  Because it has totally derailed many of the twitter threads. Thankfully we get less conspiracy theories here but I figured we could try and avoid it altogether.

Gys 6y ago

It seems this needs a twitter account to see? Maybe someone can post a screenshot?

acqq 6y ago
And for those of us who need a textual description to get it, maybe someone can post an explanation or description of what is expected and what's an observed behavior? Anybody, please?
- Nzen 6y ago
  Twitter crops rather than rescales posted images to some 'interesting' subset of an image that doesn't fit the preferred aspect ratio. So, the person behind this account created an antagonistic situation: made a very tall, thin image with Mitch Mconnell on the top and Barak Obama on the bottom, and another with them swapped between top and bottom. In the initial post, the cropped thumbnail shows Mitch Mconnell for both. The thread has two more situations, one with the ties (red/blue) flipped and the colors inverted. The only time it 'prefers' Obama's end of the antagonistic image is with inverted colors and him on top.
- yc-kraln 6y ago
  Twitter has an algorithm to pick which part of oblong images to showcase in the feed. People have been posting examples where, given the "choice" between a white person and a person of color, the white person is always picked, regardless of orientation or where they are in the picture. This was noticed in a thread about how Zoom's background detection seems to not work great for people of color, and how the WebEx cross-talk protection seems to favor men over women.

fr2null 6y ago

Interestingly, swapping the glasses does seem to make the thumbnail Obama. [1] 1. https://twitter.com/SergioSemJ/status/1307493041742254080?s=19 https://twitter.com/SergioSemJ/status/1307493041742254080?s=...

nichos 6y ago
So does the smile: https://twitter.com/MASCARPOWN/status/1307454405059452928?s=09 https://twitter.com/MASCARPOWN/status/1307454405059452928?s=... And actually swapping the smile: https://twitter.com/DrWhen2/status/1307497434872905728?s=19 https://twitter.com/DrWhen2/status/1307497434872905728?s=19 Twitter is nothing if not full of victims and people talking about things they know nothing about.

dazbradbury 6y ago

https://twitter.com/SergioSemJ/status/1307493041742254080?s=19 https://twitter.com/SergioSemJ/status/1307493041742254080?s=... One theory is it's actually the glasses increasing the likelihood of correctly identifying a face (or "feature" element in the photo). Interesting looking through the various tests in the thread, as clearly lots of variables here, but no way this will be exhaustive. Suspect this might kick off some serious arguments at Twitter HQ for allowing users to pick the cropping vs. an algorithm regardless of any bias found one way or not.

DanBC 6y ago
Here's an example where neither person is wearing glasses. https://twitter.com/LOVEMUNY/status/1307789227535499266 https://twitter.com/LOVEMUNY/status/1307789227535499266 It also includes an example where the black man is wearing glasses and the white man isn't.

nailer 6y ago

Misleading - it seems to be looking for contrast https://twitter.com/kim/status/1307548258491801600?s=19 https://twitter.com/kim/status/1307548258491801600?s=19 UPDATE: someone wrote a boy to test the conspiracy and properly debunked it, source code is online: https://twitter.com/vinayprabhu/status/1307497736191635458?s=19 https://twitter.com/vinayprabhu/status/1307497736191635458?s...

noxer 6y ago

The real problem I see here is the default assumption from many people that these is racial bias in the algorithm with no evidence thereof whatsoever. Thats how twitter works, shitstorm first, research later, evidence or corrections for false assumptions - never.

matthewmacleod 6y ago
I mean “no evidence whatsoever” is a pretty big stretch. There’s the initial example, a bunch of other examples of the same effect posted in the thread, and it’s already a really well-known and obvious effect that we have seen occur many times before.
- noxer 6y ago
  That called "anecdotal" not an evidence.
  - odonnellryan 6y ago
    Anecdotal evidence is still evidence. These are not anecdotes however, but probably not very significant.
    - noxer 6y ago
      lok what. "Anecdotal evidence" specifically described a false evidence. Just because "evidence" is part of the term doesn't mean it a valid evidence.
      - 411111111111111 6y ago
        That's incorrect. Anecdotes aren't necessarily wrong/incorrect. They're just Stories which may or may not be coincidences, falsehoods or representative of the reality. An anecdote is just a single data point, so no conclusion can be drawn from it alone Its true that it's often used to discredit someone, often even in bad faith... But that doesn't mean that it's inherintly derogatory

Kednicma 6y ago

Humans can't make unbiased computers. Every computer which humans make is indelibly human as well. This is the anthropic bias.

noxer 6y ago
It's not a biased if isn't a result of a human bias. If the algo picks the spot with the highest contrast for example, that can not be biased it may lead to unwanted results but its not biased.
- Aeronwen 6y ago
  Someone decided the algo was producing usable results when it was picking the highest contrast areas.
  - noxer 6y ago
    Because it did. I'm sure they made tests and decided it worked good enough. It wasn't made only for faces/persons especially not for images with 2 faced far apart. It also never was meant to decide between 2 possibly "image centers". Its not biased because you can craft an image where it procedure unwanted results. The only persons biased are the people who see a bias in everything.

TazeTSchnitzel 6y ago

Wonderful example of the ethics of ML. The people whose eyes they tracked gravitated towards certain kinds of faces (not necessarily just based on skin colour), so now Twitter's model crops other faces out of pictures, among other problems (now it likes cleavage apparently). The old approach may have been a bit mechanical but it avoided the non-neutrality of unconscious human perception. The assumption that what human eyes gravitate to is what an image should be cropped to is a very big and questionable one. The first thing I look at in a picture is not always the most important part, and the image as a whole has value.

dmitriid 6y ago

It's funny how these big companies willingly convert a non-problem into massive problems with hundreds of hours of engineering effort and huge social implications. Non-problem: don't mess with the images, and display them as is. Problem: see Twitter

t0astbread 6y ago

Why does Twitter even try so hard to make preview crops "meaningful"? Sure, it produces interesting results sometimes (sometimes funny, sometimes less so) but isn't it a lot of work for not a lot of benefit?

noxer 6y ago
Agree, resize would in most cases be way more useful.
Lammy 6y ago
I used to get annoyed by this too but came around to accepting it after building a Web (Jekyll) photo processing pipeline of my own. Art direction is very difficult to do well when you have to be able to handle any user-uploaded photo that might be in any aspect ratio. Desktops favor displaying wide photos, and handsets favor displaying tall ones. Cropping everything down to a square lets you handle any combination of photo and display aspect ratios with minimal Badness. I’ve never seen the Twitter-dot-com codebase but would be willing to bet it’s probably using VIPS’ smartcrop feature (:crop => :attention) https://www.rubydoc.info/gems/ruby-vips/Vips/Interesting https://www.rubydoc.info/gems/ruby-vips/Vips/Interesting
matthewmacleod 6y ago
Cropping well is quite a hard problem! I know it seems on the surface like “just resize it” is a good approach, but it’s not super uncommon to see quite extreme aspect images, and even for normal use there are lots of different constraints in different systems that mean a really simple system can produce undesirable results. I used to get this all the time back when I built custom content management systems. Users would upload images which were automatically cropped into various sizes for use in particular bits of content; really long or wide images would look awful when scaled, and automatic fixed-size crops would often end up cutting out the actual interesting bit of the image. That’s what various “automatically generate interesting thumbnails” algorithms try to avoid. In the end for me the best tool was auto-crop-with-preview-and-manual-override, and I’m beyond infuriated every time I try to upload an image to Twitter and it doesn’t allow this obvious and simple way to avoid the issue.

rambojazz 6y ago

I'm out of the loop. What's happening here? All I see is a picture.

s_gourichon 6y ago
Glad you asked. There are two pictures in each tweet, shown left and right. You can click on each to see the full picture. Each picture is actually two portraits (Mitch McConnell and Barack Obama), one above one below, separated by a big gap. Since apparently Twitter automatically chooses to crop pictures in default tweet display, the OP posted different cases to see where Twitter would crop the image. And it happens that Twitter chose one of the portrait always, until the OP reversed color. Some people wonder if it is because Barack Obama is black. Whether that's the reason or not, the observable fact remains.