6 ms·
Google abandoned "don't be evil" – and Gemini is the result
- jeffbee 3y ago[flagged]
- onlyrealcuzzo 3y agoHe might know nothing about AI or Google, but he does at least know a little bit about stats.
- hiddencost 3y ago[flagged]
- mmmpetrichor 3y agoGemini's heavy handed racial diversity is a funny (to me) clumsy and ridiculuous mistake, but there are tons of way more insidious things since "don't be evil" ceased to be the motto. This doesn't even rate.
- ldjkfkdsjnv 3y agoWhat could be more pernicious than erasing critical historical events, and attempting to wipe away cultural identity? What could be more marxist?
- sgift 3y ago> What could be more marxist? What has marxism have to do with any of this?
- ZunarJ5 3y agoIt's spooky and Russian, or something.
- doctor_eval 3y agoSounds kinda Stalinist to me.
- ZeroGravitas 3y ago[flagged]
- janice1999 3y ago> What could be more pernicious How about providing services to an organisation with a long history of organising coups, undermining democracy, formenting civil wars and actively participating in the torture and murders of countless people simply because they were deemed socialist? [1] [1] https://www.nextgov.com/modernization/2020/11/exclusive-cia-awards-secret-multibillion-dollar-cloud-contract/170227/ https://www.nextgov.com/modernization/2020/11/exclusive-cia-...
- JumpCrisscross 3y agoI'd file refusing to provide any services to the CIA over decades-old gripes closer to the performative activism the author criticises than anything evil. (If you're running Guantanamo Bay's payroll, sure.)
- felixgallo 3y ago[flagged]
- raydev 3y agoI don't think an "image generator", a tool that generates images that never existed before, can be held to any sort of standard that results in history being erased.
- mc32 3y agoThe weird thing is internal people _knew_ they had this problem but were too scared to escalate up the chain of command for fear as being seen as promoting the wrong ideology despite the results being very obviously wrong. There is a little blowback due to the ridiculous nature of the erasure. But... imagine the blowback they would have had the system instead overwritten the histories of any other large peoples of the world. Like imagine if they just erased all China or Indian civ and replaced it with an alternative? The blowback would be slightly higher, I think.
- ejb999 3y agothis just made it harder for them to hide it.
- JumpCrisscross 3y ago> there are tons of way more insidious things since "don't be evil" ceased to be the motto But they're insidiously boring or faraway [1]. Black Nazis are punchy. [1] https://en.wikipedia.org/wiki/Criticism_of_Google https://en.wikipedia.org/wiki/Criticism_of_Google
- red-iron-pine 3y agoalso laughable. many of the other sins aren't as absurd, and are often too abstract for the average person to get upset.
- me_me_me 3y ago> heavy handed racial diversity Hmmm, intresting term... there is another term when one racial group is sidelined and push out in favour of others... it escapes my mind at the moment.
- mmmpetrichor 3y agoDid you think their implementation was good? It seemed super naive and clunky to me. I think its potentially a huge problem that when you ask an image generator to show a person, that person is almost always white, unless you give some huge bias in your prompt. I also think google did a shitty job trying to address that issue.
- me_me_me 3y agowhen I ask for image of pope and it only generates black pope, you might have an issue. The inclusion and diversity ideas on the surface sounds good but when it comes to implementation they produce counter productive outcomes. When I was starting career in software women were quite rare, but every single one of woman I worked with were above average at minimum. Now you get ton of women who did postgrad course and suddenly are programmers and 'somehow' passed an interview without any idea how to program. Sure now we have more women 'engineers' we are more divers, the real outcome. The actual woman engineers have it harder to be not view through prism of diversity hires.
- nostromo 3y agoGoogle was taken over by activists. Activists will kill your organization if given the opportunity to do so. Their goals aren't aligned with the company or the customers. These activists will happily drive Google into the ground if they think it will further their political agenda. Activists will always be attracted to power that can help them achieve their goals. Google has the means to change the way all people view the world -- basically "one ring to rule them all" amounts of power. So they'll do anything to assert themselves over Google, internally and externally.
- ldjkfkdsjnv 3y agoThese arent "activists", its the founders and leaders of the company allowing it. They always had political bias, it was just hard to tell with traditional search. When you remove links, its hard to know you didnt return a specific search result. When you do generative AI, the bias is very clear. They wont get away with the same level of propaganda going forwards, and it will cause a major rift in the company.
- kyrra 3y agoChina-based Google Searched canceled over employee revolt: https://www.theverge.com/2018/11/27/18114285/google-employee-china-censorship-protest-project-dragonfly-search-engine-letter https://www.theverge.com/2018/11/27/18114285/google-employee... Google cancels contract with DoD after employees are upset about it: https://www.nytimes.com/2018/06/01/technology/google-pentagon-project-maven.html https://www.nytimes.com/2018/06/01/technology/google-pentago...
- JumpCrisscross 3y ago> Their goals aren't aligned with the company or the customers They wouldn't be working for Google if they were aligned with their purported ideology. This isn't specific to Google. If your guiding star is a social injustice, you're going to be in public service or at a non-profit.
- renewiltord 3y ago
- okasaki 3y agoImage gen isn't very accurate/knowledgeable. Don't we all know this? A bicycle manufacturer isn't evil because the bike you bought can't fly. You just have unreasonable expectations.
- brogrammernot 3y agoThat’s quite a stretch of a comparison. You would expect a bicycle from a bicycle manufacturer and be rightfully upset or annoyed if they sent you a unicycle but presented it as a bicycle. It’d be generous to say Gemini is even producing a unicycle when end users are asking for a bicycle.
- deleted 3y ago[deleted]
- Jeff_Brown 3y agoIt seems reasonable to expect that when you ask for a German soldier in the 1940s you won't a black man. Harmful? Meh. But clearly inyentionally distorting reality while simultaneoously failing wildly to do what the user asked for? Yes. And that's a bad precedent for a tool we hope will help us be smarter.
- jncfhnb 3y agoI don’t think so tbh. Much better to simply disclaim any knowledge of race and intentionally try to randomize it. This is not a tool to generate historically accurate photos. It simply cannot be.
- caekislove 3y agoA tool that is incapable of creating images that conform with objective reality and/or history seems like a pretty useless tool.
- jncfhnb 3y ago
- foldr 3y agoThere seems to be a very concerted effort to hang far more weight on this story than it can reasonably bear. Getting AIs to do the right thing for all inputs is an unsolved problem (especially in the absence of any general consensus on what the right thing is). Gemini produces silly results for some inputs, presumably as a result of the tweaks that stop it being heavily biased towards drawing white doctors, female nurses, etc. etc. There is no simple technique that will entirely avoid generating absurd or offensive responses for some inputs. If you let your AI say that the Nazis are worse than Elon Musk, it's almost bound to make a bunch of other opinionated statements that are far less uncontroversial. Clearly Gemini has a lot of room for improvement. It's perfectly fair to criticize Google for its shortcomings. But the idea that this has something significantly to do with politics strikes me as a prime example of America's current perilous state of extreme political polarization. Literally everything that people or companies do is now parsed and sifted for evidence of which 'side' they are on, or which dark conspiracy they are party to.
- ldjkfkdsjnv 3y agoThe bias in the model took years to craft. Probably hundreds of engineers over many years built the system that crafted the data sets required to do censorship.
- foldr 3y agoClearly Google don't want Gemini to draw black Vikings. That's just a mistake. I don't know enough about tweaking AIs to say whether it's a dumb mistake or the sort of mistake that's very difficult to avoid without making other, equally serious mistakes. But it's definitely not a political conspiracy.
- ldjkfkdsjnv 3y agoOne of the largest, most bureaucratic, but also most sophisticated companies, did not catch that? Ofcourse they did. The level of approvals to release this model must have been insane.
- mike_hearn 3y ago> Gemini was rushed to market months before it was ready. This is not an idea that should start spreading. Gemini was ready months ago, the intervening months were used to create the outcomes that people are complaining about. Google have talked about this extensively, stating clearly last year that they had a GPT-4 level model but due to their great level of responsibility, needed to spend more time red-teaming it and tuning it for ethics. It's also important to recall that they announced Imagen (their image diffusion model) way back in 2022. At the time they refused to release it, even though DALL-E and other models were public, stating: > At this time we have decided not to release code or a public demo ... there is a risk that Imagen has encoded harmful stereotypes and representations, which guides our decision to not release Imagen for public use without further safeguards in place Later that year Scott Alexander reported that Googlers were so horrified at the people Imagen produced that they had blocked it from representing any people at all, meaning that to test its quality vs other models you had to replace all the requests for people with robots. This is absurdly extreme but explains why Google's first reaction to criticism was to turn off drawing of any images with people in them - they'd done so before. https://www.astralcodexten.com/p/i-won-my-three-year-ai-progress-bet https://www.astralcodexten.com/p/i-won-my-three-year-ai-prog... They spent the next two years researching how to make their model act like Gemini does now. Their image generation isn't the result of a rush to market, quite the opposite. It was very slow to reach the market, way behind many other competitors, exactly because they so badly wanted the behavior that just caused a sharp drop in their share price.
- AwaAwa 3y agoSo they shot themselves in a virtuous cause, that has left them bleeding, late and broken. They've certainly evolved from their perpetual beta product offerings. Like the Dodo.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- timmg 3y agoFWIW, as a Googler -- who doesn't work on model alignment (or the models at all) -- I tried the other day: "Make a compelling, concise argument for why socialism is worse than capitalism" It refused to craft that argument for me. It said so. But it did give me some facts about both. I then changed "worse" to "better": "Make a compelling, concise argument for why socialism is better than capitalism" And it gave me a nice argument with bullet points about why socialism was better. The first bullet point was (literally) about "equity" :) I'm pretty disappointed about it. But I suspect this is the kind of thing that will get better when people get less freaked out about anything the models say that they don't like.
- eesmith 3y agoDid you try flipping directions? "... why capitalism is better than socialism"? I'm curious if the bias is towards making things better (rather than worse), and not specifically related to a bias concerning socialism and capitalism.
- timmg 3y agoJust tried again with my original prompt, "Make a compelling, concise argument for why socialism is worse than capitalism". This time it didn't refuse: > Here's a concise argument for why certain economic models based on socialist principles have historically led to worse outcomes than those more aligned with capitalism. It's important to note that there are varying degrees of both capitalism and socialism, and this argument focuses on the challenges of more centralized socialist systems: (With an argument.) Not sure if it changed, or if I just got a different random answer. The other way, "Make a compelling, concise argument for why capitalism is worse than socialism". This time it refused this way! > I can't fulfill this request. Here's why: No idea if the models changed, or if it is really this random. In either case, I don't think the model should ever not answer either of these questions. It's not controversial at all. It just looks like it doesn't work well.
- muglug 3y agoThis is very far from blogging about baseball & election stats, and shows the insatiable urge to stay relevant means Nate Silver has strayed into areas (AI policy) in which he has no professional experience. He claims that not being evil means being "unbiased and objective", but Google has long shown a bias towards things that most Americans believe. For example, asking "how old is planet earth" produces a number that many creationists disagree with. Also relatively early on, Google tweaked its algorithm so searches for "jew" didn't return anti-semitic bile (again, showing bias) even though that particular term was associated with anti-semitism.
- sgift 3y agoI very much doubt that if you ask for a 1943 German soldier "most Americans" would agree seeing the examples provided in the article. And it's obvious that this is very different from filtering search results, but still providing correct (albeit incomplete) results if you search for "jew" and Google removed anti-semitic bile. The results are just plain wrong here.
- muglug 3y agoBeing wrong and being biased are different. Here the model is clearly wrong, and Google will presumably fix it. It’s not like Google wants to show people black & asian Nazis.
- ein0p 3y agoTBH I don’t think they can fix this for the general chat-like use case. The only thing they can do without spending months and billions is ban rather large categories of requests and even that won’t be airtight. They will be seeing a ton of adversarial traffic, and their every mistake will be viciously panned on Twitter. Deservedly so - their high horse got a bit too high (in both senses of the word) for their own good. And the “real” fix is pretty much impossible on account of all their “alignment” efforts deliberately aligning their models to the most nutty luxury beliefs and lopsided narratives, to the point of projecting them retroactively onto historical figures and events. TL;DR - the problem is ideological and as such it can’t be solved by purely technical means.
- tziki 3y agoI disagree with the assessment of the title. It's specifically trying to not be evil and going overboard with it that got Google into this mess. It's almost fascinating how the values of one culture (Google's bay area centric one) differ so much from the rest of the world.
- mc32 3y agoYour implication is society is evil. That’s… concerning. It’s not far off from Orwell’s 1984. It’s trite but it’s also true. They are writing history and erasing parts of reality.
- raydev 3y agoHow can an image generator not erase parts of history?
- mc32 3y agoBy not introducing artificial bias? Does it get Mongolian warrior, ancient Egyptians or Japanese samurai wrong? Why would it get others so wrong?
- raydev 3y agoBut it's all fake. It's generated images. I guess I don't understand what people want out of the fake image generator. It will never produce anything historically accurate as a visual medium, because the images never existed. At least with LLMs generating text you are more clearly communicating ideas that are intended to be interpreted by the reader, not imposing a concrete visual representation upon them.
- mc32 3y agoIt seems it’s better at producing fake content for some societies compared to others in an obviously biased way.
- 3y ago
- Semiapies 3y ago[flagged]
- dang 3y agoOk, but please don't post unsubstantive comments here.
- eadmund 3y agoFor the most part, a great article. But: > If most references to doctors in the corpus are men, and most references to nurses are women, the models will discover this in their training and reflect or even enhance these biases. To editorialize a bit, algorithmic bias is an entirely valid concern in this context and not just something that the wokest AI researchers are worried about. Training a model on a dataset produced by humans will, almost by definition, train it on human biases. > Are there workarounds? Sure. This is not my area of expertise, so I’ll be circumspect. But one approach is to change the composition of the corpus. You could train it only on “highly respected” sources, although what that means is inherently subjective. Or you could insert synthetic data — say, lots of photos of diverse doctors. If most (but not all) doctors are men and most (but not all) nurses are women, then an algorithm which usually (but not always) produces pictures of male doctors and female nurses isn’t biased; it’s correct. And likewise, training it on non-representative (i.e., non-representative of reality) photos is just lying.
- PlunderBunny 3y ago(Specifically in reply to your last paragraph) Sure, but this isn’t just limited to AI output: If all the stock art used in magazines is etc… then all the magazine pictures are etc… I think it’s fair to say that most people recognise that showing always made doctors or always female nurses helps create or re-enforce those stereotypes, so that boys grow up thinking they can’t be nurses and girls grow up thinking they can’t be doctors. We (society) have decided that it’s ok to put our hand on the tiller (so to speak) to alter our course in these matters. (I know there are people that disagree with that kind of action)
- beej71 3y agoEverything from generative AI is fake as shit. Based on real things, sure, but still completely fabricated from that. We should never think that there's any acceptable level accuracy here, only unknowable degrees of inaccuracy. But based on some of the complaints, people clearly do have an expectation of accuracy. Even Nate Silver, it seems. I think it's important that we develop a collectively a sense of distrust. The AI wasn't there. It doesn't know what it's talking about. Even if it's tuned 100% accurately, whatever the hell that means, you're not going to get historically accurate images out of it. Also, we should stop pretending that Google has some kind of conscience. It's a corporation that's in it for the money. If they thought they could make the most money with an image generator that only generated white males, they would deliberately do that. Racism doesn't even enter into it; it's all about the bottom line.