7 ms·

Google CEO Sundar Pichai says its malfunctioning Gemini AI is 'unacceptable'

59 points by antimora 3y ago

ChrisArchitect 3y ago
[dupe] Lots more discussion over here: https://news.ycombinator.com/item?id=39534608 https://news.ycombinator.com/item?id=39534608
29athrowaway 3y ago
The system was designed to do that. It is not an AI problem it is a product problem.
- hackerlight 3y ago
  This could just be a real-world example of the difficulty of the alignment problem. Having no personal insight into the minds of Google engineers who built this thing, my assumption is they wanted Gemini to give diverse results when someone asked for a "person" or an "accountant" (reasonable), but didn't think of cases like "English kings" (unreasonable) where the added context changes the distribution. So they added some one-line hack to the system prompt (we know this is how OpenAI achieves this) which unbiased the former distribution but added lots of bias the latter. Quite an easy oversight to make. This is what AI risk people have been saying. You can't get AI to behave how you want exactly because it's extremely difficult to encode your values in a way where there's no unintended consequences.
emilfihlman 3y ago
>Malfunctioning No, it's behaving exactly how the training data and training procedures instructed it to behave, ie what Google's employees instructed it to do. It's not malfunctioning because you made it like this purposefully.
deleted 3y ago
[deleted]
0xy 3y ago
The reason Google has about 100 DEI employees with their hands on the controls of this thing is because they believe anything else is unacceptable. Gemini wasn't shocking because it showed racist and historically inaccurate material, it was shocking because it showed the world through Google's lens. As others have pointed out, even Google search has the same issue. Image search will highlight minorities even when the search term suggests none should appear. It was 100% deliberate, and they spent countless hours refining, polishing and testing the model. They were happy with it.
- fngjdflmdflg 3y ago
  The funny thing is that for the images results it was hard to say if there was a manufactured result, an incidental one or completely nonexistent (ie. it's in people's minds). But in this case it's completely impossible to argue anything but an explicit instruction and comically explicit at that (by basically adding the desired result to the prompt). To be clear though this isn't a comment on weather search is being explicitly augmented in a similar way.
- skywhopper 3y ago
  Which minorities are those, exactly?
  - 0xy 3y ago
    Things like a black 1930s German soldier, for example. Google shoehorns in diversity so hard that it simply invents historical diversity in the name of DEI.
- Mountain_Skies 3y ago
  Google claims they were trying to make it reflect the global population, in which case those it tried to erase are the global minority.
s9df898r32h 3y ago
[flagged]
- paulddraper 3y ago
  I think you misread the article.
  - s9df898r32h 3y ago
    I don't think I read the article... I think the article title dissuaded me to. (I read very few articles when the title suck.)
alephnerd 3y ago
While everyone on HN is hyperventilating about DEI crap, the malfunction Pichai is most likely referencing is Gemini AI calling Modi a fascist. [0]. A Union Minister has already gave a veiled threat that Google broke Indian Telecom Laws [1] surrounding fake news. It's election season in India and governments get very ban happy during this time (look at what happened to Amazon India in 2019) Edit: Please don't make this an Indian politics flame war. The point is, a public company like Google is constrained by PR. This is why OpenAI has been innovating so fast - they are private and don't have to answer to almost anybody Public companies need to protect their brand, otherwise they lose out to similarly sized competitors (eg. Microsoft or Facebook) [0] - https://www.thehindubusinessline.com/info-tech/googles-gemini-ai-tool-gets-into-a-spot-over-biased-response/article67879554.ece https://www.thehindubusinessline.com/info-tech/googles-gemin... [1] - https://twitter.com/Rajeev_GoI/status/1760910808773710038 https://twitter.com/Rajeev_GoI/status/1760910808773710038
- kypro 3y ago
  The link you cited, says Gemini says: 'Modi has been "accused of implementing policies some experts have characterised as fascist"'. Is this true? Or more specifically, is this issue here that Gemini is saying something that's accurate but that the Indian government doesn't like, or that no experts have characterised Modi as a Fascist and Gemini is saying something inaccurate?
  - alephnerd 3y ago
    It doesn't matter. The issue is Gemini AI spewed an answer that caused Google to enter Indian culture wars less than 30 days before national elections. Most of the horrible crap that happens in India happens during this time of year, because Parliamentary elections are very high stakes. Most companies want to stay out of these kinds of culture wars, and Google inadvertently entered one
    - rob74 3y ago
      It's only a small step from rightwing populism to fascism. Rightwing populists say some things (like "we'll let Russia invade countries that don't spend 2% of their GDP on defence"), fascists actually do it...
tiahura 3y ago
Other than for entertainment purposes, who cares what a statistical analysis of English has to say about ethics?
- karaterobot 3y ago
  In theory, nobody should, it would be ridiculous. The answer to your question is: lots of people today, probably even more tomorrow.
  - xanderlewis 3y ago
    Well, perhaps we ought to educate people better. And quickly. The real danger of LLMs is that people come to believe they hold some sort of authority — or that their output is in any way the end result of a process of deliberation and contemplation rather than simply being a (stochastically perturbed) best guess at what an answer should look like. Yes, people already take Google at face value, but at least (in theory) they know that anything they’re finding there is written either by an organisation with an axe to grind or an individual no more omniscient than they.
- FredPret 3y ago
  If that's what it was, analyzing an entire language would be an interesting lens into how a large number of people think about ethics. But this one is more interesting because it shows us how Google thinks about ethics.
throwaway5959 3y ago
How badly does this guy have to fail before he gets the boot? Google is an embarrassment in areas it should be excelling and leading the industry at. They _invented_ transformers.
- behnamoh 3y ago
  Every big tech company needs a Steve Ballmer type of person to appreciate previous its engineering/design-minded CEOs.
  - meragrin_ 3y ago
    What did Ballmer do which was comparatively this bad?
    - dylan604 3y ago
      Vista? Windows 8? Windows Phone? IE6
    - behnamoh 3y ago
      Nokia purchase, Longhorn, Vista, Windows Phone
    - ThrowawayB7 3y ago
      If the Yahoo acquisition that he really, really wanted had gone though, Steve Ballmer would probably have been the laughingstock tech CEO of the decade. Luckily for him, Jerry Yang captured that title instead by turning Ballmer's offer down.
- est31 3y ago
  > They _invented_ transformers. IIRC most/all the "Attention Is All You Need" authors have left Google.
  - throwaway5959 3y ago
    I rest my case.
cloudking 3y ago
I'm surprised their LLMs passed the strict Google launch review process, how do you test something that gives a random response every time you query it?
- cjbgkagh 3y ago
  AFAIK they intentionally inject randomness to reduce the effectiveness of jailbreaking.
- throwaway888999 3y ago
  Their launch review process is anything but strict. Google makes awful launch decisions all the time. The culture is one of launching rather than launching something good, and there's no individual accountability for anything, as evidenced by Sundar's email.
nimchimpsky 3y ago
[dead]
0xbadcafebee 3y ago
Is it malfunctioning? Or is it just being generative AI? Maybe it's time people accept that generative AI is good at making shit up, and so we should just use it for that. Maybe we should stop pretending that "probabilistically predicting the next in a series of words" counts as intelligence. I say, let the generative AI be racist. Let it hallucinate and come up with all kinds of crazy shit. And with that, stop using it for things that matter. Use it for art, entertainment, experiments, explorations. Use it to mine the depths of the human soul and reflect back at us what we are. Don't use it for law, health care, directions, or anything humans depend on. Acknowledge that it's just a stupid program with output that "looks impressive", and that we've all been fleeced into thinking it would be anything more. Regardless how many billions we sink into it, it only appears intelligent when we conceal how batshit crazy it can be.
- dylan604 3y ago
  > it only appears intelligent when we conceal how batshit crazy it can be. this can be applied to a broader aspect than just AI. in today's post-fact world, this is absolutely applicable as well to pretty much anything in life. we have people believing in the wildest ideas and not as a joke or just for the lulz, but wholeheartedly believing them as fact. people believe the earth is flat, that vaccines make you magnetic, or any of the other well known tropes. all of that content has been used to train these "AI" systems. why would we ever expect it to not make shit up when a non-insignificant amount of its training data is made up shit.
  - ekianjo 3y ago
    thats not at all what is happening here. it looks a lot more like Gemini is following direct rules from a kind of DEI organization than anything else.
- serial_dev 3y ago
  I agree with everything you wrote, it's the mindset I wish people would have. One note, though, in this case, it wasn't the generative AI that was racist, it's the people who added the extra configuration of not allowing (or at the very least, strongly discouraging) white people in their images. The racism wasn't a bug (like in the past with the gorilla incident), this time it was a feature that was deliberately put there.
sergiotapia 3y ago
>“This wasn’t what we intended. We did not want Gemini to refuse to create images of any particular group. And we did not want it to create inaccurate historical—or any other—images,” The model seemed to respond exactly how it's been trained. The problem isn't technical. It's a cultural rot and the only fix is to get rid of the people rotting it. No amount of merge requests are going to fix this.
behnamoh 3y ago
Companies RLHF the models. The market RLHF's the companies. It was about time someone called out the evil incentives behind push for alignment.
elintknower 3y ago
The issue is anyone with an IQ over 85 can tell that this behavior is in no way a "malfunction" of the system. The system was intended to operate this way and the general population found the output abhorrent and wrong. The way Gemini talks about certain events (and generates images) is clearly in line with the walled garden of compromised liberal thinking at google. I say this as someone who votes blue and considers themselves progressive. What's unacceptable, is Google clearly thinking it was okay to use their AI to erase history how it actually happened and just play it off as their org having moral highground.
- spaniard89277 3y ago
  The thing is that this got released to the public after _many_ people said it was ok, internally, at google. You really have to lost any grip with reality to go for something like that, so I guess they'll rationalize some excuse. That will be interesting to see.
  - elintknower 3y ago
    Having talked to friends who currently work at G in Mountain View - "lost any grip with reality" is an understatement when it comes to the collective ethos of G employees.
- elintknower 3y ago
  Looks like the "enlightened" google employees found this comment and applied "first principles" thinking ;)
ra7 3y ago
If you read his full memo [1], it's the most Sundar Pichai response possible to a crisis. It's full of vague MBA speak ("red-teaming", really?) with the obligatory Google mission statement sprinkled in along with self congratulatory statements of their AI achievements. It does not inspire any confidence whatsoever. [1] https://twitter.com/TechEmails/status/1762849036363505996 https://twitter.com/TechEmails/status/1762849036363505996
- dekhn 3y ago
  Red-teaming came from think tanks helping the military. It's a sensible term in this context, basically they act to anticipate the reaction of the public by entering offensive queries and reporting on them in a way that can override the blue team's shortsightedness. What I really don't like is the "working around the clock". There is no need to rush a solution as quickly as possible, and will only serve to make the results worse because as we all know, googlers don't like to work around the clock and will not give the best results if they are forced to. Except if promotions are on the line.
somethoughts 3y ago
I think the biggest strategic oversight that Google leadership has made is not learning from Microsoft's Tay fiasco. What Google should have done is setup an entirely separate startup (or startups) similar to the approach Microsoft has taken with OpenAI. Let a few young, ambitious startup CEOs lead the way and supply them with billions in funding which they must use in Google Cloud Compute AI spend - in exchange for a majority stake in their company and integrating with Google properties only. Book the GCP AI revenue and get stock bump from YoY GCP growth. If the startup(s) misstep, claim the tax write-off then move on to the next batch. If they are successful over a five year stretch then just buy them out fully and get some positive news coverage over the successful strategic acquisition. As a member of Mag7/FAANG, Google leadership must realize that any and all minor missteps are going to be blown out of proportion via clickbait articles. [1] https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)
- est31 3y ago
  Startups also provide brand isolation. If you get the Microsoft AI to say things it shouldn't say then that tarnishes the MS brand, if a MS associated startup has prompt hacks, then there is a layer of isolation. Large companies are sadly not good at innovating themselves, probably this is an emergent effect from the size alone and the bureaucracy such size sadly requires. Especially when people sue you for millions when you search their name and lies show up, you must erect barriers. You also cannot let every PM in the company launch potentially disappointing products. Instead proposals need to get reviewed at least from legal points of view, but usually by a larger set of departments. By the time half of the reviews are done, the competing startup has already launched the product, gotten direct user feedback, and established itself as the "innovator", all this while the big tech company where someone might have had the same idea is busy with getting it through its bureaucracy.
  - somethoughts 3y ago
    Agree - I do think that part of the bureaucracy is due to the fact that any new consumer facing product from a FAANG is going to be relentlessly challenged adversarially by every social media tech-influencer as in this case. And no product team wants to put the CEO in the media/congressional hearing hot seat. Conversely a new consumer product from a new startup is going to spend half the time talking about what the new company is etc. before getting to any product flaws. And most of the time such things are relegated to niche reddit subgroups.
robertwt7 3y ago
What about google search? This is also happening for google image searched and need to be fixed!!
glimshe 3y ago
He talks about it as if it was someone else's fault. His internal policies of extreme virtue signaling and DEI focus could only have led to that outcome. And now that I think about it, this was probably not a problem of lack of testing - it was tested and the results appeared as expected given Google's past of trying to mold society through subtle but deliberate biases in its products.
pachorizons 3y ago
It is truly indicative of the shortcomings of the supposedly enlightened Hacker News hivemind that Google - a company that has killed influential projects and sacrificed / cannibalised the integrity of it's core search product - should be condemned because it's latest AI malfunction is too 'DEI'. Google's AI blunders are more systemic than the latest culture war. The cringe hate spewed by earlier models are just as incompetent as Gemini, these are nothing more than the flailing of a multinational who is incapable of fashioning a vision of a world they promise. These are bugs at the highest level, that are clearly poisioning every layer of Google. They are the same bugs that led to Google search shortcuts that pushed disinformation COVID. This is a failure of rigour from the flailing of the world's self appointed organiser of knowledge. Your least favourite DEI minority has nothing to do with it. For fucks sake. Place your blame where it belongs.
- fngjdflmdflg 3y ago
  >supposedly enlightened What made you suppose this? >The cringe hate spewed by earlier models are just as incompetent as Gemini [...] Your least favourite DEI minority has nothing to do with it Except unlike Tay ect. this would specifically add "of various diverse ethnicities" to each promt, so this isn't just some "bug" but an intentional decision. Do you seriously believe Google's DEI initiative is unrelated? Why ignore what they specifically say they are doing? eg. >moving forward, all VP+ performance reviews will include an evaluation of leadership in support of diversity, equity and inclusion. [0] [0] https://about.google/commitments/racialequity/#our-commitments https://about.google/commitments/racialequity/#our-commitmen...
  - pachorizons 3y ago
    Do you really think that a DEI mandate specifies refusing to produce white faces in historical contexts? That is an extraordinary position, especially in the face of the layoffs of the ethics teams that this sort of claim would consider as collaborators for such a conspiracy. Do you really truly think that this is anything beyond the obvious naked incompetence of Google under it's current leadership? To put it another way, this is an unsophisticated position. It is an unsophisticated to describe Google or Microsoft as Nazi organization for releasing a racist AI, so to is it an unsophisticated position to for clearly absurd "DEI" behaviour from Gemini. What both examples demonstrate are companies utterly incapble of governing this kind of system.
curtisblaine 3y ago
It's good that this happened and I hope it stays in the news as much as possible. It's also good that Google entered an Indian culture war and I hope many incidents of this kind keep happening until the maximum amount of users see what happens when DEI takes control and has free rein with indoctrination. I'm not very hopeful for the future, but anything that exposes the ridiculousness of the people that were put in control of this tragic debacle is very welcome.
1vuio0pswjnm7 3y ago
We really need something like The Onion with quotes from users, the way university newspapers will have a row of headshots and quotes from random students asked some question. The Onion of course was orginally a student newspaper that had something like that. Instead all we get is an endless stream of BS from "Big Tech CEOs". I want to hear what the people say, not some "Big Tech" compulsive liar.
- hiisukun 3y ago
  Just thought I'd chime in to say this is often called a 'vox pop'.
deleted 3y ago
[deleted]
bitcharmer 3y ago
And of course this was gang-flagged by the blue-haired fairies.

59 points by antimora 3y ago

[dupe] Lots more discussion over here: https://news.ycombinator.com/item?id=39534608 https://news.ycombinator.com/item?id=39534608

29athrowaway 3y ago

The system was designed to do that. It is not an AI problem it is a product problem.

hackerlight 3y ago
This could just be a real-world example of the difficulty of the alignment problem. Having no personal insight into the minds of Google engineers who built this thing, my assumption is they wanted Gemini to give diverse results when someone asked for a "person" or an "accountant" (reasonable), but didn't think of cases like "English kings" (unreasonable) where the added context changes the distribution. So they added some one-line hack to the system prompt (we know this is how OpenAI achieves this) which unbiased the former distribution but added lots of bias the latter. Quite an easy oversight to make. This is what AI risk people have been saying. You can't get AI to behave how you want exactly because it's extremely difficult to encode your values in a way where there's no unintended consequences.

emilfihlman 3y ago

>Malfunctioning No, it's behaving exactly how the training data and training procedures instructed it to behave, ie what Google's employees instructed it to do. It's not malfunctioning because you made it like this purposefully.

deleted 3y ago

[deleted]

0xy 3y ago

The reason Google has about 100 DEI employees with their hands on the controls of this thing is because they believe anything else is unacceptable. Gemini wasn't shocking because it showed racist and historically inaccurate material, it was shocking because it showed the world through Google's lens. As others have pointed out, even Google search has the same issue. Image search will highlight minorities even when the search term suggests none should appear. It was 100% deliberate, and they spent countless hours refining, polishing and testing the model. They were happy with it.

fngjdflmdflg 3y ago
The funny thing is that for the images results it was hard to say if there was a manufactured result, an incidental one or completely nonexistent (ie. it's in people's minds). But in this case it's completely impossible to argue anything but an explicit instruction and comically explicit at that (by basically adding the desired result to the prompt). To be clear though this isn't a comment on weather search is being explicitly augmented in a similar way.
skywhopper 3y ago
Which minorities are those, exactly?
- 0xy 3y ago
  Things like a black 1930s German soldier, for example. Google shoehorns in diversity so hard that it simply invents historical diversity in the name of DEI.
Mountain_Skies 3y ago
Google claims they were trying to make it reflect the global population, in which case those it tried to erase are the global minority.

s9df898r32h 3y ago

[flagged]

paulddraper 3y ago
I think you misread the article.
- s9df898r32h 3y ago
  I don't think I read the article... I think the article title dissuaded me to. (I read very few articles when the title suck.)

alephnerd 3y ago

While everyone on HN is hyperventilating about DEI crap, the malfunction Pichai is most likely referencing is Gemini AI calling Modi a fascist. [0]. A Union Minister has already gave a veiled threat that Google broke Indian Telecom Laws [1] surrounding fake news. It's election season in India and governments get very ban happy during this time (look at what happened to Amazon India in 2019) Edit: Please don't make this an Indian politics flame war. The point is, a public company like Google is constrained by PR. This is why OpenAI has been innovating so fast - they are private and don't have to answer to almost anybody Public companies need to protect their brand, otherwise they lose out to similarly sized competitors (eg. Microsoft or Facebook) [0] - https://www.thehindubusinessline.com/info-tech/googles-gemini-ai-tool-gets-into-a-spot-over-biased-response/article67879554.ece https://www.thehindubusinessline.com/info-tech/googles-gemin... [1] - https://twitter.com/Rajeev_GoI/status/1760910808773710038 https://twitter.com/Rajeev_GoI/status/1760910808773710038

kypro 3y ago
The link you cited, says Gemini says: 'Modi has been "accused of implementing policies some experts have characterised as fascist"'. Is this true? Or more specifically, is this issue here that Gemini is saying something that's accurate but that the Indian government doesn't like, or that no experts have characterised Modi as a Fascist and Gemini is saying something inaccurate?
- alephnerd 3y ago
  It doesn't matter. The issue is Gemini AI spewed an answer that caused Google to enter Indian culture wars less than 30 days before national elections. Most of the horrible crap that happens in India happens during this time of year, because Parliamentary elections are very high stakes. Most companies want to stay out of these kinds of culture wars, and Google inadvertently entered one
  - rob74 3y ago
    It's only a small step from rightwing populism to fascism. Rightwing populists say some things (like "we'll let Russia invade countries that don't spend 2% of their GDP on defence"), fascists actually do it...

tiahura 3y ago

Other than for entertainment purposes, who cares what a statistical analysis of English has to say about ethics?

karaterobot 3y ago
In theory, nobody should, it would be ridiculous. The answer to your question is: lots of people today, probably even more tomorrow.
- xanderlewis 3y ago
  Well, perhaps we ought to educate people better. And quickly. The real danger of LLMs is that people come to believe they hold some sort of authority — or that their output is in any way the end result of a process of deliberation and contemplation rather than simply being a (stochastically perturbed) best guess at what an answer should look like. Yes, people already take Google at face value, but at least (in theory) they know that anything they’re finding there is written either by an organisation with an axe to grind or an individual no more omniscient than they.
FredPret 3y ago
If that's what it was, analyzing an entire language would be an interesting lens into how a large number of people think about ethics. But this one is more interesting because it shows us how Google thinks about ethics.

throwaway5959 3y ago

How badly does this guy have to fail before he gets the boot? Google is an embarrassment in areas it should be excelling and leading the industry at. They _invented_ transformers.

behnamoh 3y ago
Every big tech company needs a Steve Ballmer type of person to appreciate previous its engineering/design-minded CEOs.
- meragrin_ 3y ago
  What did Ballmer do which was comparatively this bad?
  - dylan604 3y ago
    Vista? Windows 8? Windows Phone? IE6
  - behnamoh 3y ago
    Nokia purchase, Longhorn, Vista, Windows Phone
  - ThrowawayB7 3y ago
    If the Yahoo acquisition that he really, really wanted had gone though, Steve Ballmer would probably have been the laughingstock tech CEO of the decade. Luckily for him, Jerry Yang captured that title instead by turning Ballmer's offer down.
est31 3y ago
> They _invented_ transformers. IIRC most/all the "Attention Is All You Need" authors have left Google.
- throwaway5959 3y ago
  I rest my case.

cloudking 3y ago

I'm surprised their LLMs passed the strict Google launch review process, how do you test something that gives a random response every time you query it?

cjbgkagh 3y ago
AFAIK they intentionally inject randomness to reduce the effectiveness of jailbreaking.
throwaway888999 3y ago
Their launch review process is anything but strict. Google makes awful launch decisions all the time. The culture is one of launching rather than launching something good, and there's no individual accountability for anything, as evidenced by Sundar's email.

nimchimpsky 3y ago

[dead]

0xbadcafebee 3y ago

Is it malfunctioning? Or is it just being generative AI? Maybe it's time people accept that generative AI is good at making shit up, and so we should just use it for that. Maybe we should stop pretending that "probabilistically predicting the next in a series of words" counts as intelligence. I say, let the generative AI be racist. Let it hallucinate and come up with all kinds of crazy shit. And with that, stop using it for things that matter. Use it for art, entertainment, experiments, explorations. Use it to mine the depths of the human soul and reflect back at us what we are. Don't use it for law, health care, directions, or anything humans depend on. Acknowledge that it's just a stupid program with output that "looks impressive", and that we've all been fleeced into thinking it would be anything more. Regardless how many billions we sink into it, it only appears intelligent when we conceal how batshit crazy it can be.

dylan604 3y ago
> it only appears intelligent when we conceal how batshit crazy it can be. this can be applied to a broader aspect than just AI. in today's post-fact world, this is absolutely applicable as well to pretty much anything in life. we have people believing in the wildest ideas and not as a joke or just for the lulz, but wholeheartedly believing them as fact. people believe the earth is flat, that vaccines make you magnetic, or any of the other well known tropes. all of that content has been used to train these "AI" systems. why would we ever expect it to not make shit up when a non-insignificant amount of its training data is made up shit.
- ekianjo 3y ago
  thats not at all what is happening here. it looks a lot more like Gemini is following direct rules from a kind of DEI organization than anything else.
serial_dev 3y ago
I agree with everything you wrote, it's the mindset I wish people would have. One note, though, in this case, it wasn't the generative AI that was racist, it's the people who added the extra configuration of not allowing (or at the very least, strongly discouraging) white people in their images. The racism wasn't a bug (like in the past with the gorilla incident), this time it was a feature that was deliberately put there.

sergiotapia 3y ago

>“This wasn’t what we intended. We did not want Gemini to refuse to create images of any particular group. And we did not want it to create inaccurate historical—or any other—images,” The model seemed to respond exactly how it's been trained. The problem isn't technical. It's a cultural rot and the only fix is to get rid of the people rotting it. No amount of merge requests are going to fix this.

behnamoh 3y ago

Companies RLHF the models. The market RLHF's the companies. It was about time someone called out the evil incentives behind push for alignment.

elintknower 3y ago

The issue is anyone with an IQ over 85 can tell that this behavior is in no way a "malfunction" of the system. The system was intended to operate this way and the general population found the output abhorrent and wrong. The way Gemini talks about certain events (and generates images) is clearly in line with the walled garden of compromised liberal thinking at google. I say this as someone who votes blue and considers themselves progressive. What's unacceptable, is Google clearly thinking it was okay to use their AI to erase history how it actually happened and just play it off as their org having moral highground.

spaniard89277 3y ago
The thing is that this got released to the public after _many_ people said it was ok, internally, at google. You really have to lost any grip with reality to go for something like that, so I guess they'll rationalize some excuse. That will be interesting to see.
- elintknower 3y ago
  Having talked to friends who currently work at G in Mountain View - "lost any grip with reality" is an understatement when it comes to the collective ethos of G employees.
elintknower 3y ago
Looks like the "enlightened" google employees found this comment and applied "first principles" thinking ;)

ra7 3y ago

If you read his full memo [1], it's the most Sundar Pichai response possible to a crisis. It's full of vague MBA speak ("red-teaming", really?) with the obligatory Google mission statement sprinkled in along with self congratulatory statements of their AI achievements. It does not inspire any confidence whatsoever. [1] https://twitter.com/TechEmails/status/1762849036363505996 https://twitter.com/TechEmails/status/1762849036363505996

dekhn 3y ago
Red-teaming came from think tanks helping the military. It's a sensible term in this context, basically they act to anticipate the reaction of the public by entering offensive queries and reporting on them in a way that can override the blue team's shortsightedness. What I really don't like is the "working around the clock". There is no need to rush a solution as quickly as possible, and will only serve to make the results worse because as we all know, googlers don't like to work around the clock and will not give the best results if they are forced to. Except if promotions are on the line.

somethoughts 3y ago

I think the biggest strategic oversight that Google leadership has made is not learning from Microsoft's Tay fiasco. What Google should have done is setup an entirely separate startup (or startups) similar to the approach Microsoft has taken with OpenAI. Let a few young, ambitious startup CEOs lead the way and supply them with billions in funding which they must use in Google Cloud Compute AI spend - in exchange for a majority stake in their company and integrating with Google properties only. Book the GCP AI revenue and get stock bump from YoY GCP growth. If the startup(s) misstep, claim the tax write-off then move on to the next batch. If they are successful over a five year stretch then just buy them out fully and get some positive news coverage over the successful strategic acquisition. As a member of Mag7/FAANG, Google leadership must realize that any and all minor missteps are going to be blown out of proportion via clickbait articles. [1] https://en.wikipedia.org/wiki/Tay_(chatbot) https://en.wikipedia.org/wiki/Tay_(chatbot)

est31 3y ago
Startups also provide brand isolation. If you get the Microsoft AI to say things it shouldn't say then that tarnishes the MS brand, if a MS associated startup has prompt hacks, then there is a layer of isolation. Large companies are sadly not good at innovating themselves, probably this is an emergent effect from the size alone and the bureaucracy such size sadly requires. Especially when people sue you for millions when you search their name and lies show up, you must erect barriers. You also cannot let every PM in the company launch potentially disappointing products. Instead proposals need to get reviewed at least from legal points of view, but usually by a larger set of departments. By the time half of the reviews are done, the competing startup has already launched the product, gotten direct user feedback, and established itself as the "innovator", all this while the big tech company where someone might have had the same idea is busy with getting it through its bureaucracy.
- somethoughts 3y ago
  Agree - I do think that part of the bureaucracy is due to the fact that any new consumer facing product from a FAANG is going to be relentlessly challenged adversarially by every social media tech-influencer as in this case. And no product team wants to put the CEO in the media/congressional hearing hot seat. Conversely a new consumer product from a new startup is going to spend half the time talking about what the new company is etc. before getting to any product flaws. And most of the time such things are relegated to niche reddit subgroups.

robertwt7 3y ago

What about google search? This is also happening for google image searched and need to be fixed!!

glimshe 3y ago

He talks about it as if it was someone else's fault. His internal policies of extreme virtue signaling and DEI focus could only have led to that outcome. And now that I think about it, this was probably not a problem of lack of testing - it was tested and the results appeared as expected given Google's past of trying to mold society through subtle but deliberate biases in its products.

pachorizons 3y ago

It is truly indicative of the shortcomings of the supposedly enlightened Hacker News hivemind that Google - a company that has killed influential projects and sacrificed / cannibalised the integrity of it's core search product - should be condemned because it's latest AI malfunction is too 'DEI'. Google's AI blunders are more systemic than the latest culture war. The cringe hate spewed by earlier models are just as incompetent as Gemini, these are nothing more than the flailing of a multinational who is incapable of fashioning a vision of a world they promise. These are bugs at the highest level, that are clearly poisioning every layer of Google. They are the same bugs that led to Google search shortcuts that pushed disinformation COVID. This is a failure of rigour from the flailing of the world's self appointed organiser of knowledge. Your least favourite DEI minority has nothing to do with it. For fucks sake. Place your blame where it belongs.

fngjdflmdflg 3y ago
>supposedly enlightened What made you suppose this? >The cringe hate spewed by earlier models are just as incompetent as Gemini [...] Your least favourite DEI minority has nothing to do with it Except unlike Tay ect. this would specifically add "of various diverse ethnicities" to each promt, so this isn't just some "bug" but an intentional decision. Do you seriously believe Google's DEI initiative is unrelated? Why ignore what they specifically say they are doing? eg. >moving forward, all VP+ performance reviews will include an evaluation of leadership in support of diversity, equity and inclusion. [0] [0] https://about.google/commitments/racialequity/#our-commitments https://about.google/commitments/racialequity/#our-commitmen...
- pachorizons 3y ago
  Do you really think that a DEI mandate specifies refusing to produce white faces in historical contexts? That is an extraordinary position, especially in the face of the layoffs of the ethics teams that this sort of claim would consider as collaborators for such a conspiracy. Do you really truly think that this is anything beyond the obvious naked incompetence of Google under it's current leadership? To put it another way, this is an unsophisticated position. It is an unsophisticated to describe Google or Microsoft as Nazi organization for releasing a racist AI, so to is it an unsophisticated position to for clearly absurd "DEI" behaviour from Gemini. What both examples demonstrate are companies utterly incapble of governing this kind of system.

curtisblaine 3y ago

It's good that this happened and I hope it stays in the news as much as possible. It's also good that Google entered an Indian culture war and I hope many incidents of this kind keep happening until the maximum amount of users see what happens when DEI takes control and has free rein with indoctrination. I'm not very hopeful for the future, but anything that exposes the ridiculousness of the people that were put in control of this tragic debacle is very welcome.

1vuio0pswjnm7 3y ago

We really need something like The Onion with quotes from users, the way university newspapers will have a row of headshots and quotes from random students asked some question. The Onion of course was orginally a student newspaper that had something like that. Instead all we get is an endless stream of BS from "Big Tech CEOs". I want to hear what the people say, not some "Big Tech" compulsive liar.

hiisukun 3y ago
Just thought I'd chime in to say this is often called a 'vox pop'.

deleted 3y ago

[deleted]

bitcharmer 3y ago

And of course this was gang-flagged by the blue-haired fairies.