12 ms·
Algorithmic bias bounty challenge
- pipthepixie 5y ago> We want to take this work a step further by inviting and incentivizing the community to help identify potential harms of this algorithm beyond what we identified ourselves. What more needs to be done here? Twitter got caught out with bias in their image cropping algorithm, and now they want people to further elaborate on how/why this is? > For this challenge, we are re-sharing our saliency model and the code used to generate a crop of an image given a predicted maximally salient point and asking participants to build their own assessment. Seems like a very exhaustive endeavour just to point out algorithmic bias and re-iterate what went wrong, but in more detail.
- SpicyLemonZest 5y agoThey want to develop a fuller understanding of what kinds of biases they need to be aware of. "Caught out with bias" isn't a good way to understand the situation - this isn't the kind of problem that can be solved by just debugging to find the biased functions and using unbiased ones instead.
- vineyardmike 5y ago> Seems like a very exhaustive endeavour just to point out algorithmic bias and re-iterate what went wrong, but in more detail. They want everyone to help them find additional bias, and other points it could go wrong, with the presumed purpose of them fixing it. They want to find bias they can solve and remove, not just get more people to tell them why the bias they found is wrong!
- stolenmerch 5y ago"Although we find no evidence the saliency model explicitly encodes male gaze, in cases where the model crops out a woman’s head in favor of their body due to a jersey or lettering, the chosen crop still runs the risk of representational harm for women. In these instances, it is important to remember that users are unaware of the details of the saliency model and the cropping algorithm. Regardless of the underlying mechanism, when an image cropped to a woman’s body area is viewed on social media, due to the historical hyper-sexualization and objectification of women’s bodies, the image runs this risk of being instilled with the notion of male gaze." "Point multipliers are applied for those harms that particularly affect marginalized communities since Twitter’s goal is to responsibly and equitably serve the public conversation." I'm sure Twitter is just responding to the controversy from Fall of 2020 and doing due diligence to address the problem. However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? Genuine question here and one I'm always curious about. Seems difficult if not impossible.
- cryptoz 5y agoThe quote you cited says "no evidence the saliency model explicitly encodes male gaze" emphasis mine. Your re-phrasing however changes the meaning in your question, "historical biases not inherent in the model". Explicitly encodes != inherent in. The difference between the two might answer your question. You can solve for something that is inherent in a system (ie, this), but was not explicitly encoded. They are trying to say it isn't their fault specifically, they find no evidence this was done on purpose or in a targeted way, but that it happens as an accident / lack of testing on the current model.
- stolenmerch 5y agoThey are very clear about 'intentional' vs. 'unintentional' harm in their judging metric[1], so I get that I think. However, there is no possible way a saliency model could crop an image without someone reading bias into it. E.g. only cropping a jersey, only cropping a face, not cropping at all, etc. can all be signs of historical bias. At some point it just becomes a rorschach test for the reviewer. Not saying discrimination isn't happening or can't happen, just saying this very tenuous and weak connection to "historical bias" interpretation lacks a certain rigor in a bug bounty program or model optimization. [1] https://hackerone.com/twitter-algorithmic-bias https://hackerone.com/twitter-algorithmic-bias
- basseq 5y agoFor anyone else wondering about "the controversy from Fall 2020": https://www.theguardian.com/technology/2020/sep/21/twitter-apologises-for-racist-image-cropping-algorithm https://www.theguardian.com/technology/2020/sep/21/twitter-a...
- deleted 5y ago[deleted]
- bastawhiz 5y ago> However, how do you award a bounty for addressing "risk of representational harm" due to historical biases not inherent in the model? I think it's pretty straightforward. Most sensible, considerate humans would avoid cropping an image of a woman to her boobs simply because it's insensitive to do so. Just because the machine is trained to highlight text or other visual features doesn't preclude it from ALSO understanding human concepts which are difficult to express to the computer in a straightforward way. There are plenty of ways the model can be improved (e.g., not preferring white faces over Black faces), and they're certainly difficult. If they weren't difficult, Twitter would have simply fixed the problem and there wouldn't be a competition. Arguably, though, if the job can reasonably accomplished manually by a human then a computer should be able to do a similarly effective job. Figuring out how is why it's a challenge. And if we can't figure out how, that's another interesting point in the ongoing conversation about ML.
- slg 5y agoI never really pay attention to these bounty challenges. Are those rewards reasonable? They seem incredibly low compared to the work involved. I have seen locally sponsored hackathons with higher total prizes.
- noxer 5y agoIt's PR nonsense. They dont want anyone not working for them to work for them.
- potatoman22 5y ago3 grand don't seem like nonsense to me.
- codyb 5y agoI figure it's kind of like a musician busking on the street corner. It's a no risk way to add a few constraints to the practicing you're probably already doing and if you're lucky maybe you'll earn a few bucks too.
- ryankupyn 5y agoI think the challenge is that if the rewards were high, Twitter employees (with the advantage of inside information) might be tempted to "tip off" an outsider in exchange for a cut of the reward, rather than just reporting the issue internally. At the same time, there isn't much of an outside market for algorithmic bias info in the same way there for security vulnerabilities. Probably the biggest effect of this reward will be to pull some grad students who were going to study algorithmic bias anyways towards studying Twitter specifically - after all, there aren't any rewards for studying the algorithmic bias of other companies!
- hatchnyc 5y agoI am increasing convinced that all of the harms of social media can be traced back the ML models used to sort and filter user content. The world of pure deterministic feeds ordered by date descending never had these problems--things would still "go viral" I guess, but it was humans sharing them deliberately to people they knew. I do not believe this problem can be fixed, nor do the companies involved have the incentive to do so beyond whatever minimal amount is necessary for PR purposes.
- rumblerock 5y agoTo take it one step further I'm convinced at this point that the social conundrum presented by Twitter and other social media platforms has extended far beyond the algorithm and the platforms themselves, and into how people interact in general. The reductionist, reactionary mode of thinking now rears its head in my real-world interactions. I wouldn't care as much if people kept that stuff on these social platforms that I spend next to zero time on, but I can't help but roll my eyes when my friends speak in "hot takes" as if I'm handing out certificates for internet points.
- jancsika 5y ago> To take it one step further I'm convinced at this point that the social conundrum presented by Twitter and other social media platforms has extended far beyond the algorithm and the platforms themselves, and into how people interact in general. A cryptographer once mentioned attacking the PGP WoT by uploading a shadow WoT that has different keys with the same emails/names and signing-graph structure. Similar to what you note-- the real problems come when some bona fide WoT user fucks up and accidentally signs a key from the shadow graph. Even if it's just a single accidental signature before the shadow WoT is spotted and cleaned up, the admin would probably want to quickly go out of band to actually fix the original WoT at that point-- otherwise perhaps the shadow admin got to the user first. :) (And of course if the shadow graph were uploaded incrementally over a wider period of time, the problems probably become a bigger pain in the ass.) The problem you mention is in the same category. But rather than being hypothetical, it is real and probably as bad an instance as someone could design given current technology. To make the WoT attack above comparable, I'm imagining a Debian user accidentally signing a shadow key, then receiving messages from that key that convince them Debian keyring is in fact the shadow key ring and they should use their permissions to destroy Debian. Edit: and I guess in keeping with the analogy, the purpose of the propaganda emails is to generate as much outrage in the recipient as possible, so that they may spend more time reading the entire email on the off chance they also spend one or two seconds extra looking at the ad in the email signature about a special on a nice pair of designer pants.
- gotoeleven 5y agoI applaud twitter's efforts to solve the problem of men wanting to look at boobs.
- thepangolino 5y agoI would have given to way more than that to find out what trick did the 2-3 top anti-Trump account used to always show up on top of the comments of each of Trump's tweets.
- only_as_i_fall 5y agoDoes Twitter not order tweets based on number of likes?
- noxer 5y agoNo, that would be unbiased^^
- debrice 5y agoI have an ethic issue with these kind of challenges and rewards. When a company spends millions of dollars, probably tens of millions on a project… $3,500 reward for, in a way, successfully contributing to that project feels off. Don’t get me wrong, the subject and goals of the work are definitely good but not entirely philanthropic. I feed like helping to find and fix broken parts of a billion dollars for profit industry should generate significantly more wealth than few grands
- tayo42 5y ago3500 is a weeks work of pay if you got 180k a year. about a weeks worth of a paid engineers time. How much time do spend on this?
- bitbckt 5y agoThis reminds of the time at Twitter when all Japanese users were (accidentally) inferred to be female after a model changed.
- bequanna 5y agoWouldn’t a better solution be to optimize the photo cropping for the specific viewer? Or, for whatever ‘group’ the viewer falls in?
- acituan 5y agoHere’s my qualitative submission; bounty is not necessary. Image cropping is not representative of the most important AI biases within Twitter, and making only that salient is the biggest meta-bias. Differential impact and harm analyses are required for engagement maximizing recommendations of the tweets. Also, no one cares about your cropping algorithms, open the data that trained the model if you dare. Without it you don’t get to posture transparency or ethicality.
- crummybowley 5y agoIt seems like we should just stop using ML for feeds. My twitter feed is always hate. I don't actually comment, or engage other tan reading. And I am continuously being suggested post that are clearly bias to somebodies agenda. I also fallow CNN breaking news. And the comments are complete trash. What is worse, is some small town thing happens, and now you have millions across the US and the world upset and outraged about it. While the issue was tragic, I am not sure it is healthy for society as a whole to be so connected to things that play nearly zero in their life. I know, I know. Somebody is going to say "but they do play a role in their life". And I am going to reply. The only role they play is in that they saw a tweet or a reedit post or a 12 second clip with a loaded title that tries to convey that this bad thing happening is "status quo" and it is so outrageous that it happens and we all need to stand up to do something about it. When in reality we are dealing with a world with ~8 billion people in it, where if 0.0000001% (8 people) of the population experience something bad to them in a given day, that we have systems in place to ensure that the vast majority know about it and get outraged, and are made to feel as if this is some huge problem that everybody needs to be involved in. Listen folks, shitty things are going to happen, and they are going to keep happening, and it is going to seem like the problem is going to get worse and worse as the population grows. But we will never have zero bad things happening, and being outraged over and over about something that simply won't ever be corrected is a huge burden on society as a whole. I challenge folks, especially the ones who get outraged at twitter post, reddit post, or stupid things said on HN to take a month off from social media. You will find the world is a much better place than you think it is currently.
- ekianjo 5y agoThere's no algorithmic bias without raw data bias in the first place.
- usaar333 5y agoMaybe? I've always found the ideas on algorithmic bias lacking formalization. I'm not really sure what data or algorithmic bias really means. I think "data bias" means non-representative data and "algorithm bias" indicates classifier accuracy differing between two differing groups, but in such a definition algorithm bias can exist without data bias. Example: I might have a representative set of faces, of which perhaps 1% are wearing sunglasses. There's no particular reason to believe that a face detection classifier optimized over this set performs as well over sunglass wearing faces as those without, given the low frequency of sunglasses. (I might get a better ROC if I only consider faces with eyes visible) In a sense the data isn't biased, but there is an algorithm bias between the groups.
- ekianjo 5y agoin your example you can easily avoid this issue with k fold sampling. its not an algorithmic bias rather a training data bias.
- usaar333 5y agoI'm not following why cross validation helps here. One group (sunglasses) is simply much rarer so my algorithm may globally perform better on the overall set by accepting lower recall on the rarer subgroup (allowing for higher precision on the much larger group). This isn't a hypothetical example; many years back I improved the F-score of a face detector I was working on by using eye detection and skin color detection in the classification pipeline. I assume automated systems could produce similar effects, tanking recall on a small subgroup to buy better precision on the larger one.
- ekianjo 5y ago> One group (sunglasses) is simply much rarer so my algorithm may globally perform better on the overall set by accepting lower recall on the rarer subgroup (allowing for higher precision on the much larger group). I see what you mean. Well then my answer is still the same. If you have an unbalanced input data, this is what is causing the bias at the classifier level - not the classifier model/algorithm in itself.
- techbio 5y ago$7000.00? How can they afford it?
- bo1024 5y agoThey announce the challenge starting today and it ends 6 days from now? I feel like that will really limit the entries.
- dehrmann 5y agoYeah. The time to do it is pretty short, and the prize pool is small. I'd say it's just for PR, but you think someone would call out how off those numbers are.
- Zababa 5y agoHow about removing automatic cropping and offering to users a way to control the cropping of what they post? No bias but I guess that would drive down engagement.
- deleted 5y ago[deleted]
- deleted 5y ago[deleted]
- vmception 5y agoThe prizes need to be large voting share blocs of the social media company's equity, and long dated options to purchase more at a discount. Accidentally replace the ownership with people who are conscious of what should be changed.
- josephcsible 5y agoWhy not just either (1) let the person posting the image choose how to crop it, or (2) scale the image down instead of cropping it?
- ThinkBeat 5y agoIf you could successfully solve the problem at hand, You develop an algorithmic way to do this accurately and reliably it would be worth billions. Handing it over for $3500 does not really seem like a good deal for anyone but Twitter (as expected). And one other thing "Winners will be announced at the DEF CON AI Village workshop hosted by Twitter" Wow Defcon must be an entirely different event and crowd now than when I was attending