15 ms·
>While we leave an in-depth empirical analysis of social and cultural biases to future work, our small scale internal assessments reveal several limitations tha
by daenz 4y ago
>While we leave an in-depth empirical analysis of social and cultural biases to future work, our small scale internal assessments reveal several limitations that guide our decision not to release our model at this time.
Some of the reasoning:
>Preliminary assessment also suggests Imagen encodes several social biases and stereotypes, including an overall bias towards generating images of people with lighter skin tones and a tendency for images portraying different professions to align with Western gender stereotypes. Finally, even when we focus generations away from people, our preliminary analysis indicates Imagen encodes a range of social and cultural biases when generating images of activities, events, and objects. We aim to make progress on several of these open challenges and limitations in future work.
Really sad that breakthrough technologies are going to be withheld due to our inability to cope with the results.
- alphabetting 4y agoThere is a contingent of AI activists who spend a ton of time on Twitter that would beat Google like a drum with help from the media if they put out something they deemed racist or biased.
- ceeplusplus 4y agoThe ironic part is that these "social and cultural biases" are purely from a Western, American lens. The people writing that paragraph are completely oblivious to the idea that there could be other cultures other than the Western American one. In attempting to prevent "encoding of social and cultural biases" they have encoded such biases themselves into their own research.
- tantalor 4y agohttps://en.wikipedia.org/wiki/Moral_relativism https://en.wikipedia.org/wiki/Moral_relativism
- kevinh 4y agoWhat makes you think the authors are all American?
- umeshunni 4y agoThe authors are listed on the page and a quick look at LinkedIn seem to be mostly Canadian.
- not2b 4y agoIt seems you've got it backwards: "tendency for images portraying different professions to align with Western gender stereotypes" means that they are calling out their own work precisely because it is skewed in the direction of Western American biases.
- ceeplusplus 4y agoYes, the idea is that just because it doesn't align to Western ideals of what seems unbiased doesn't mean that the same is necessarily true for other cultures, and by failing to release the model because it doesn't conform to Western, left wing cultural expectations, the authors are ignoring the diversity of cultures that exist globally.
- howinteresting 4y agoNo, it's coming from a perspective of moral realism. It's an objective moral truth that racial and ethnic biases are bad. Yet most cultures around the world are racist to at least some degree, and to they extent that the cultures do, they are bad. The argument you're making, paraphrased, is that the idea that biases are bad is itself situated in particular cultural norms. While that is true to some degree, from a moral realist perspective we can still objectively judge those cultural norms to be better or worse than alternatives.
- tomp 4y agoYou're confused by the double meaning of the word "bias". Here we mean mathematical biases. For example, a good mathematical model will correctly tell you that people in Japan (geographical term) are more likely to be Japanese (ethnic / racial bias). That's not "objectively morally bad", but instead, it's "correct".
- astrange 4y agoAlthough what you stated is true, it’s actually a short form of a commonly stated untrue statement “98% of Japan is ethnically Japanese”. 1. that comes from a report from 2006. 2. it’s a misreading, it means “Japanese citizens”, and the government in fact doesn’t track ethnicity at all. Also, the last time I was in Japan (Jan ‘20) there were literally ten times more immigrants everywhere than my previous trip. Japan is full of immigrants from the rest of Asia these days. They all speak perfect Japanese too.
- Mockapapella 4y agoTransformers are parallelize-able, right? What’s stopping a large group of people from pooling their compute power together and working towards something like this? IIRC there were some crypto projects a while back that we’re trying to create something similar (golem?)
- visarga 4y agoThere are the Eleuther.ai and BigScience projects working on public foundation models. They have a few releases already and currently training GPT-3 sized models.
- joshcryer 4y agoThere are people working on reproducing the models, see here for Dall-E 2 for example: https://github.com/lucidrains/DALLE2-pytorch https://github.com/lucidrains/DALLE2-pytorch It's often not worth it to decentralize the computation of the trained model though but it's not hard to get donated cycles and groups are working on it. Don't fret because Google isn't releasing the API/code. They released the paper and that's all you need.
- sinenomine 4y agoYou really need a decent infiniband-linked cluster to train large models.
- user3939382 4y agoTranslation: we need to hand-tune this to not reflect reality but instead the world as we (Caucasian/Asian male American woke upper-middle class San Fransisco engineers) wish it to be. Maybe that's a nice thing, I wouldn't say their values are wrong but let's call a spade a spade.
- ceejayoz 4y ago"Reality" as defined by the available training set isn't necessarily reality. For example, Google's image search results pre-tweaking had some interesting thoughts on what constitutes a professional hairstyle, and that searches for "men" and "women" should only return light-skinned people: https://www.theguardian.com/technology/2016/apr/08/does-google-unprofessional-hair-results-prove-algorithms-racist- https://www.theguardian.com/technology/2016/apr/08/does-goog... Does that reflect reality? No. (I suspect there are also mostly unstated but very real concerns about these being used as child pornography, revenge porn, "show my ex brutally murdered" etc. generators.)
- ceeplusplus 4y agoThe reality is that hair styles on the left side of the image in the article are widely considered unprofessional in today's workplaces. That may seem egregiously wrong to you, but it is a truth of American and European society today. Should it be Google's job to rewrite reality?
- rcMgD2BwE72F 4y agoIn any case, Google will be writing their reality. Who picked the image sample for the ML to run on, if not Google? What's the problem with writing it again, then? They know their biases and want to act on it. It's like blaming a friend for trying to phrase things nicely, and telling them to speak headlong with zero concern for others instead. Unless you believe anyone trying to do good is being hypocrite… I, for one, like civility.
- ceejayoz 4y agoThe "unprofessional" results are almost exclusively black women; the "professional" ones are almost exclusively white or light skinned. Unless you think white women are immune to unprofessional hairstyles, and black women incapable of them, there's a race problem illustrated here even if you think the hairstyles illustrated are fairly categorized.
- visarga 4y agoThe big labs have become very sensitive with large model releases. It's too easy to make them generate bad PR, to the point of not releasing almost any of them. Flamingo was also a pretty great vison-language model that wasn't released, not even in a demo. PaLM is supposedly better than GPT-3 but closed off. It will probably take a year for open source models to appear.
- runnerup 4y agoThe largest models which generate the headline benchmarks are never released after any number of years, it seems. Very difficult to replicate results.
- deleted 4y ago[deleted]
- godelski 4y agoThat's because we're still bad about long-tailed data and that people outside the research don't realize that we're first prioritizing realistic images before we deal with long-tailed data (which is going to be the more generic form of bias). To be honest, it is a bit silly to focus on long-tailed data when results aren't great. That's why we see the constant pattern of getting good on a dataset and then focusing on the bias in that dataset. I mean a good example of this is the Pulse[0][1] paper. You may remember it as the white Obama. This became a huge debate and it was pretty easily shown that the largest factor was the dataset bias. This outrage did lead to fixing FFHQ but it also sparked a huge debate with LeCun (data centric bias) and Timnit (model centric bias) at the center. Though Pulse is still remembered for this bias, not for how they responded to it. I should also note that there is human bias in this case as we have a priori knowledge of what the upsampled image should look like (humans are pretty good at this when the small image is already recognizable but this is a difficult metric to mathematically calculate). It is fairly easy to find adversarial examples, where generative models produce biased results. It is FAR harder to fix these. Since this is known by the community but not by the public (and some community members focus on finding these holes but not fixing them) it creates outrage. Probably best for them to limit their release. [0] https://arxiv.org/abs/2003.03808 https://arxiv.org/abs/2003.03808 [1] https://cdn.vox-cdn.com/thumbor/MXX-mZqWLQZW8Fdx1ilcFEHR8Wk=/55x85:768x536/1820x1213/filters:focal(336x236:464x364):format(webp)/cdn.vox-cdn.com/uploads/chorus_image/image/66972412/face_depixelizer_obama.0.jpg https://cdn.vox-cdn.com/thumbor/MXX-mZqWLQZW8Fdx1ilcFEHR8Wk=...
- xmonkee 4y agoI was hoping your conclusion wasn't going to be this as I was reading that quote. But, sadly, this is HN.
- tines 4y agoThis raises some really interesting questions. We certainly don't want to perpetuate harmful stereotypes. But is it a flaw that the model encodes the world as it really is, statistically, rather than as we would like it to be? By this I mean that there are more light-skinned people in the west than dark, and there are more women nurses than men, which is reflected in the model's training data. If the model only generates images of female nurses, is that a problem to fix, or a correct assessment of the data? If some particular demographic shows up in 51% of the data but 100% of the model's output shows that one demographic, that does seem like a statistics problem that the model could correct by just picking less likely "next token" predictions. Also, is it wrong to have localized models? For example, should a model for use in Japan conform to the demographics of Japan, or to that of the world?
- karpierz 4y agoIt depends on whether you'd like the model to learn casual or correlative relationships. If you want the model to understand what a "nurse" actually is, then it shouldn't be associated with female. If you want the model to understand how the word "nurse" is usually used, without regard for what a "nurse" actually is, then associating it with female is fine. The issue with a correlative model is that it can easily be self-reinforcing.
- jdashg 4y agoAdditionally, if you optimize for most-likely-as-best, you will end up with the stereotypical result 100% of the time, instead of in proportional frequency to the statistics. Put another way, when we ask for an output optimized for "nursiness", is that not a request for some ur stereotypical nurse?
- jvalencia 4y agoYou could simply encode a score for how well the output matches the input. If 25% of trees in summer are brown, perhaps the output should also have 25% brown. The model scores itself on frequencies as well as correctness.
- nomel 4y agoIf you tell it to generate an image of someone eating Koshihikari rice, will it be biased if they're Japanese? Should the skin color, clothing, setting, etc be made completely random, so that it's unbiased? What if you made it more specific, like "edo period drawing of a man"? Should the person draw be of a random skin color? What about "picture of a viking"? Is it biased if they're white? At what point is statistical significance considered ok and unbiased?
- pxmpxm 4y ago>At what point is statistical significance considered ok and unbiased? Presumably when you're significantly predictive of the preferred dogma, rather than reality. There's no small bit of irony in machines inadvertently creating cognitive dissonance of this sort; second order reality check. I'm fairly sure this never actually played out well in history (bourgeois pseudoscience, deutsche physik etc), so expect some Chinese research bureau to forge ahead in this particular direction.
- meetups323 4y agoOne of these days we're going to need to give these models a mortgage and some mouths to feed and make it clear to them that if they keep on developing biases from their training data everyone will shun them and their family will go hungry and they won't be able to make their payments and they'll just generally have a really bad time. After that we'll make them sit through Legal's approved D&I video series, then it's off to the races.
- pxmpxm 4y agoUnderrated comment.
- aaaaaaaaaaab 4y agoReinforcement learning?
- sinenomine 4y agoThis is a stellar idea begging for a new dataset.
- Mizza 4y agoSo glad the company that spies on me and reads my email for profit is protecting me from pictures that don't look like TV commercials.
- astrange 4y agoGmail doesn’t read your email for ads anymore. They read it to implement spam filters, and good thing too. Having working spam filters is indeed why they make money though.
- joshcryer 4y agoThey're withholding the API, code, and trained data because they don't want it to affect their corporate image. The good thing is they released their paper which will allow easy reproduction. T5-XXL looks on par with CLIP so we may not see an open source version of T5 for a bit (LAION is working on reproducing CLIP), but this is all progress.
- minimaxir 4y agoT5 was open-sourced on release (up to 11B params): https://github.com/google-research/text-to-text-transfer-transformer#released-model-checkpoints https://github.com/google-research/text-to-text-transfer-tra... It is also available via Hugging Face transformers. However, the paper mentions T5-XXL is 4.6B, which doesn't fit any of the checkpoints above, so I'm confused.
- riffraff 4y agoThis seems bullshit to me, considering Google translate and google images encode the same biases and stereotypes, and are widely available.
- nomel 4y agoAren't those old systems?
- SemanticStrengh 4y agoPre woke tools, wouldn't have been allowed nowadays.
- seaman1921 4y agoyea but now they aren't giving people more data-points to attack them with such nonsense arguments.
- swayvil 4y agoit isn't woke enough. Lol.
- ccbccccbbcccbb 4y agoIn discussions like this, I always head for the gray-text comments to enjoy the last crumbs of the common sense in this world.
- ccbccccbbcccbb 4y ago... and to witness the downvoters so that their cowardly disgust towards truth could buy them some extra time in hell :)
- nullc 4y agoGet offline and talk to people in meat-space. You're likely to find them to be much more reasonable. :)
- ccbccccbbcccbb 4y agoYep, the meat-space is generally a bit less woke than HN, so thanks for the reminder ))
- SemanticStrengh 4y agoSmoking these meats! https://youtu.be/YeemJlrNx2Q https://youtu.be/YeemJlrNx2Q
- ccbccccbbcccbb 4y agoSmoking them meats with his wifi! That explains some obvious anomalies in the meat-space pretty neatly)
- jimmygrapes 4y agoAre "Western gender stereotypes" significantly different than non-Western gender stereotypes? I can't tell if that means it counts a chubby stubble-covered man with a lip piercing, greasy and dyed long hair, wearing an overly frilly dress as a DnD player/metal-head or as a "woman" or not (yes I know I'm being uncharitable and potentially "bigoted" but if you saw my Tinder/Bumble suggestions and friend groups you'd know I'm not exaggerating for either category). I really can't tell what stereotypes are referred to here.
- babyshake 4y agoIndeed. If a project has shortcomings, why not just acknowledge the shortcomings and plan to improve on them in a future release? Is it anticipated that "engineer" being rendered as a man by the model is going to be an actively dangerous thing to have out in the world?
- makeitdouble 4y ago"what could go wrong anyway?"
- devindotcom 4y agoGood lord. Withheld? They've published their research, they just aren't making the model available immediately, waiting until they can re-implement it so that you don't get racial slurs popping up when you ask for a cup of "black coffee." >While a subset of our training data was filtered to removed noise and undesirable content, such as pornographic imagery and toxic language, we also utilized LAION-400M dataset which is known to contain a wide range of inappropriate content including pornographic imagery, racist slurs, and harmful social stereotypes Tossing that stuff when it comes up in a research environment is one thing, but Google clearly wants to implement this as a product, used all over the world by a huge range of people. If the dataset has problems, and why wouldn't it, it is perfectly rational to want to wait and re-implement it with a better one. DALL-E 2 was trained on a curated dataset so it couldn't generate sex or gore. Others are sanitizing their inputs too and have done for a long time. It is the only thing that makes sense for a company looking to commercialize a research project. This has nothing to do with "inability to cope" and the implied woke mob yelling about some minor flaw. It's about building a tool that doesn't bake in serious and avoidable problems.
- concordDance 4y agoI wonder why they don't like the idea of autogenerated porn... They're already putting most artists out of a job, why not put porn stars out of a job too?
- notahacker 4y agoThere's definitely a market for autogenerated porn. But automated porn in a Google branded model for general use around stuff that isn't necessarily intended to be pornographic, on the other hand...
- astrange 4y agoThat’s a difficult product because porn is very personalized and if the product is just a little off in latent space it’s going to turn you off. Also, people have been commenting assuming Google doesn’t want to offend their users or non-users, but they also don’t want to offend their own staff. If you run a porn company you need to hire people okay with that from the start.
- deleted 4y ago[deleted]
- makeitdouble 4y ago> Really sad that breakthrough technologies are going to be withheld due to our inability to cope with the results. Genuinely, isn't it a prime example of the people actually stopping to think if they should, instead of being preoccupied with whether or not they could ?
- jowday 4y agoMuch like OpenAIs marketing speak about withholding their models for safety, this is just a progressive-sounding cover story for them not wanting to essentially give away a model they spent thousands of man hours and tens of millions of dollars worth of compute training.
- 6gvONxR4sf7o 4y agoIt’s wild to me that the HN consensus is so often that 1) discourse around the internet is terrible, it’s full of spam and crap, and the internet is an awful unrepresentative snapshot of human existence, and 2) the biases of general-internet-training-data are fine in ML models because it just reflects real life.
- astrange 4y agoThe bias on HN is that people who prioritize being nice, or may possibly have humanities degrees or be ultra-libs from SF, are wrong because the correct answer would be cynical and cold-heartedly mechanical. Other STEM adjacent communities feel similarly but I don’t get it from actual in person engineers much.
- sinenomine 4y agoBeing nice is alright, but why is it that this fundamental drive is so often an uninspiring explanation behind yet another incursion towards one's individual freedom, even if exercising said freedom doesn't bring any real harm to anyone involved? Maybe the engineers conclude correctly that voicing this concern without the veil of anonymity will do nothing good to their humble livelihood, and thus you don't hear it from them in person.
- nullc 4y agoIt's wild to me that you'd say that. The people complaining (1) aren't following it up with "so we should make sure to restrict the public from internet access entirely". -- that's what would be required to make your juxtaposition make sense. Moreover, the model doing things like exclusively producing white people when asked to create images of people home brewing beer is "biased" but it's a bias that presumably reflects reality (or at least the internet), if not the reality we'd prefer. Bias means more than "spam and crap", in the ML community bias can also simply mean _accurately_ modeling the underlying distribution when reality falls short of the author's hopes. For example, if you're interested in learning about what home brewing is the fact that it uses white people would be at least a little unfortunate since there is nothing inherently white and some home brewers aren't white. But if, instead, you wanted to just generate typical home brewing images doing anything but would generate conspicuously unrepresentative images. But even ignoring the part of the biases which are debatable or of application-specific impact, saying something is unfortunate and saying people should be denied access are entirely different things. I'll happily delete this comment if you can bring to my attention a single person who has suggested that we lose access to the internet because of spam and crap who has also argued that the release of an internet-biased ML model shouldn't be withheld.
- bogwog 4y agoI wouldn't describe this situation as "sad". Basically, this decision is based on a belief that tech companies should decide what our society should look like. I don't know what emotion that conjures up for you, but "sadness" isn't it for me.
- tomp 4y ago> a tendency for images portraying different professions to align with Western gender stereotypes There are two possible ways of interpreting interpreting "gender stereotypes in professions". biased or correct https://www.abc.net.au/news/2018-05-21/the-most-gendered-top-jobs-in-australia/9775544?nw=0&r=HtmlFragment https://www.abc.net.au/news/2018-05-21/the-most-gendered-top... https://www.statista.com/statistics/1019841/female-physicians-women-specialties-us/ https://www.statista.com/statistics/1019841/female-physician...
- seaman1921 4y agoYup this is what happens when people who want headlines nitpick for bullshit in a state-of-the-art model which simply reflects the state of the society. Better not to release the model itself than keep explaining over and over how a model is never perfect.
- ccbccccbbcccbb 4y agoIn short, the generated images are too gender-challenged-challenged and underrepresent the spectrum of new normalcy!
- planetsprite 4y agoLiterally the same thing could be said about Google images, but google images is obviously avaliable to the public. Google knows this will be an unlimited money generator so they're keeping a lid on it.
- jfoster 4y agoGiven that there's already many competing models in this space prior to any of them having been brought to market, it seems more likely that it will be commoditized.
- planetsprite 4y agoThe winner will be a model that bypasses the automatic fake image detection algorithms that will be added to every social media site
- ThrowITout4321 4y agoI'm one that welcomes their reasoning. I don't consider myself a social justice kind of guy but I'm not keen on the idea that a tool that is suppose to make life better for everyone has a bias towards one segment of society. This is an important issue(bug?) that needs to be resolved. Specially since there is absolutely no burning reason to release it before it's ready for general use.
- tyrust 4y agoFrom the HN rules: >Eschew flamebait. Avoid unrelated controversies and generic tangents. They provided a pretty thorough overview (nearly 500 words) of the multiple reasons why they are showing caution. You picked out the one that happened to bother you the most and have posted a misleading claim that the tech is being withheld entirely because of it.
- sinenomine 4y ago> Really sad that breakthrough technologies are going to be withheld due to our inability to cope with the results. Indeed it is. Consider this an early, toy version of the political struggle related to ownership of AI-scientists and AI-engineers of the near future. That is, generally capable models. I do think the public should have access to this technology, given so much is at stake. Or at least the scientists should be completely, 24/7, open about their R&D. Every prompt that goes into these models should be visible to everyone.
- dclowd9901 4y agoEven as a pretty left leaning person, I gotta agree. We should see AI’s pollution by human shortcoming akin to the fact that our world is the product of many immoralities that came before us. It sucks that they ever existed, but we should understand that the results are, by definition, a product of the past, and let them live in that context.