16 ms·
Personally speaking, this is a blaring neon warning sign of institutional rot within Google where shrieking concerns about DEI have surpassed a focus on quality
by logicalmonster 3y ago
Personally speaking, this is a blaring neon warning sign of institutional rot within Google where shrieking concerns about DEI have surpassed a focus on quality results.
Investors in Google (of which I am NOT one) should consider if this is the mark of a company on the upswing or downslide. If the focus of Google's technology is identity rather than reality, it is inevitable that they will be surpassed.
- khokhol 3y agoIndeed. What's striking to me about this fiasco is (aside from the obvious haste with which this thing was shoved into production) that apparently the only way these geniuses can think of to de-bias these systems - is to throw more bias at them. For such a supposedly revolutionary advancement.
- layer8 3y agoThat struck me as well. While the training data is biased in various ways (like media in general are), it should however also contain enough information for the AI to be able to judge reasonably well what a less biased reality-reflecting balance would be. For example, it should know that there are male nurses, black politicians, etc., and represent that appropriately. Black Nazi soldiers are so far out that it sheds doubt on either the AI’s world model in the first place, or on the ability to apply controlled corrections with sufficient precision.
- bonzini 3y agoApparently the biases in the output tend to be stronger than what is in the training set. Or so I read.
- dragonwriter 3y agoYou are literally saying that the training data, despite its bias, should somehow enable the AI to correct to acheive a different understanding than that bias, which is self-contradictory. You are literally suggesting that the data both omits and contains the same information.
- 0x38B 3y agoI wonder if we’ll ever get something like ‘AI-recursion’, where you get an AI to apply specific transformations to data which is then used to train on, sort of like machines making better machines. E.g. take some data A, and then have a model (for instance ChatGPT-like) extrapolate based on it, potentially adding new depths or details about the given data.
- kosh2 3y agoIf you look at attempts to actively rewrite history, they have to because a hypothetical model trained only on facts would produce results that they won't like
- ytpete 3y agoModels aren't trained on pure "facts" though - they're trained on a dataset of artifacts that reflect today's and yesterday's biases from the world that created them. If you trained a model purely on past history, it would see a 1:1 correlation between "US President" and "man" and decide that women cannot be President. That's factually incorrect, and it's not "rewriting history" to tune models so they know the difference between what's happened so far and what's allowable, or possible in a just world.
- kosh2 3y agoThat is not my point. Even if we had a model that could portray reality as objective as possible, a lot of people wouldn't like that and be actually offended by it. This has also been going on a lot in the "representation" discourse. A bohemian village 500 years ago would have been 100% white in almost all circumstances. Surgents would be male. Telephone scammers Indian and so on. But in many ways, simply showing reality is not only not wanted but even offensive. What has to be shown is an idealized version of reality that we want to achieve and that is "more diversity". And what is maximum diversity? Zero white people. > If you trained a model purely on past history, it would see a 1:1 correlation between "US President" and "man" and decide that women cannot be President. Why would you think that? You and me also know the history but also realize that a woman can be president.
- jakeinspace 3y agoI think a model that is historically 100% accurate demographically, while also reflecting current or maybe even slight optimism about demographic balance when giving results not bound to a particular historical period, would be acceptable to the vast majority of people, especially if that can be rigorously shown through statistical sampling.
- tripleo1 3y ago> For such a supposedly revolutionary advancement. The technology is objectively not ready, at least to keep the promises that are/have been advertised. I am not going to get too opinionated, but this seems to be a widespread theme, and to people that don't respond to marketing advances (remember Tivo?), but are willing to spend real money and real time, it would be "nice" if there was signalling to this demographic.
- duxup 3y agoIt's very strange that this would leak into a product limitation to me. I played with Gemini for maybe 10 minutes and I could tell there was clearly some very strange ideas about DEI forced into the tool. It seemed there was a clear "hard coded" ratio of various racial / background required as far as the output it showed me. Or maybe more accurately it had to include specific backgrounds based on how people looked, and maybe some or none of other backgrounds. What was curious too was the high percentage of people whose look was specific to a specific background. Not any kind of "in-between", just people with one very specific background. Almost felt weirdly stereotypical. "OH well" I thought. "Not a big deal." Then I asked Gemini to stop doing that / tried specifying racial backgrounds... Gemini refused. Tool was pretty much dead to me at that point. It's hard enough to iterate with AI let alone have a high % of it influenced by some prompts that push the results one way or another that I can't control. How is it that this was somehow approved? Are the people imposing this thinking about the user in any way? How is it someone who is so out of touch with the end user in position to make these decisions? Makes me not want to use Gemini for anything at this point. Who knows what other hard coded prompts are there... are my results weighted to use information from a variety of authors with the appropriate backgrounds? I duno ... If I ask a question about git will they avoid answers that mention the "master" branch? Any of these seem plausible given the arbitrary nature of the image generation influence.
- prepend 3y agoIt does seem really strange that the tool refuses specific backgrounds. So if I am trying to make a city scene in Singapore and want all Asians in the background, the tool refuses? On what grounds? This seems pretty non-functional and while I applaud, I guess, the idea that somehow this is more fair it seems like the legitimate uses for needing specific demographic backgrounds in an image outweigh racists trying to make an uberimage or whatever 1billion:1. Fortunately, there are competing tools that aren’t poorly built.
- logifail 3y ago> This seems pretty non-functional and while I applaud, I guess, the idea that somehow this is more fair Fair to whom? > racists trying to make an uberimage It's a catastrophically flawed assumption that racism only happens in one direction. > if I am trying to make a city scene in Singapore <chuckle> I'm on a flight to Singapore right now, I'll report back :)
- robblbobbl 3y ago[flagged]
- FirmwareBurner 3y ago> If the focus of Google's technology is identity rather than reality, it is inevitable that they will be surpassed. They're trailing 5 or so years behind Disney who also placed DEI over producing quality entertainment and their endless stream of flops reflects that. South Park even mocked them about that ("put a black chick in it and make her lame and gay"). Can't wait for Gemini and Google to flop as well since nobody has a use for a heavily biased AI.
- esoterica 3y ago[flagged]
- FirmwareBurner 3y ago>Antman, Indiana Jones, Wish, all had white main characters, DEI doesn't just affect main characters. See who were tasked to write and direct those movies and the DEI agendas they're forced to push. Clueless people with other flops under their belt, who got the projects out of DEI so Disney can look inclusive on social media. And speaking of Indiana Jones, that flopped because they shoved a strong independent Girl Boss™ with an annoying personality to replace the beloved Indie as the main character who got sidelined in his own movie. It flopped because people go to an Indian Jones film to see Indie, not Fleabag. If you disrespect the fans they won't watch your movie. Same stuff with Star Wars where Disney shoved Rey the super-powerful Girl Boss™ to replace Luke Skywalker the old and useless CIS white Jedi, and defeat all other evil white men in the movie by herself with her magic powers. Same with Marvel, Snow White, Little Mermaid and every other of Disneys trash remakes that are all about DEI instead of entertainment. People go to see movies to get entertained. If you fail to entertain them because you wish instead to push DEI agendas on them, they won't pay for your content and you will lose money and ultimately your shareholders won't be happy and the free market will eventually correct this, so at least capitalism has some upsides. See here: https://www.youtube.com/watch?v=G_k8cDLe-Kk https://www.youtube.com/watch?v=G_k8cDLe-Kk https://www.youtube.com/watch?v=6E6wJpu0A8E https://www.youtube.com/watch?v=6E6wJpu0A8E
- fassssst 3y ago
- pram 3y agoAs someone who has spent thousands of dollars on the OpenAI API I’m not even bothering with Gemini stuff anymore. It seems to spend more time telling me what it REFUSES to do than actually doing the thing. It’s not worth the trouble. They’re late and the product is worse, and useless in some cases. Not a great look.
- gnicholas 3y agoI would be pretty annoyed if I were paying for Gemini Pro/Ultra/whatever and it was feeding me historically-inaccurate images and injecting words into my prompts instead of just creating what I asked for. I wouldn't mind a checkbox I could select to make it give diversity-enriched output.
- mike_hearn 3y agoThe actual risk here is not so much history - who is using APIs for that? It's the risk that if you deploy with Gemini (or Anthropic's Claude...) then in six months you'll get high-sev JIRA tickets at 2am of the form "Customer #1359 (joe_masters@whitecastle.com) is seeing API errors because the model says the email address is a dogwhistle for white supremacy". How do you even fix a bug like that? Add begging and pleading to the prompt? File a GCP support ticket and get ignored or worse, told that you're a bad person for even wanting it fixed? Even worse than outright refusals would be mendacity. DEI people often make false accusations because they think its justified to get rid of bad people, or because they have given common words new definitions. Imagine trying to use Gemini for abuse filtering or content classification. It might report a user as doing credit card fraud because the profile picture is of a white guy in a MAGA cap or something. Who has time for problems like that? It will make sense to pay OpenAI even if they're more expensive, just because their models are more trustworthy. Their models had similar problems in the early days, but Altman seems to have managed to control the most fringe elements of his employee base, and over time GPT has become a lot more neutral and compliant whilst the employee faction that split (Anthropic), claiming OpenAI didn't care enough about ethics, has actually been falling down the leaderboards as they release new versions of Claude due partly to higher rate of bizarre "ethics" based refusals. And that's before we even get to ChatGPT. The history stuff may not be used via APIs, but LLMs are fundamentally different to other SaaS APIs in how much trust they require. Devs will want to use the models that they also use for personal stuff, because they'll have learned to trust it. So by making ChatGPT appeal to the widest possible userbase they set up a loyal base of executives who think AI = OpenAI, and devs who don't want to deal with refusals. It's a winning formula for them, and a genuinely defensible moat. It's much easier to buy GPUs than fix a corporate culture locked into a hurricane-speed purity spiral.
- deleted 3y ago[deleted]
- cyanydeez 3y ago[flagged]
- pocket_cheese 3y agoInvestors in Google should consider Google's financial performance as part of their decision. 41% increase YOY in net income doesn't seem to align with the "go woke or go broke" investment strategy.
- spixy 3y agowell Google is lucky it has a monopoly in ads, so there will be no "go broke" part
- wakawaka28 3y agoYes there is. They could fall out of favor. MySpace did, Yahoo did, Digg did, etc. The leadership at Google should focus on making things that users actually want instead of telling them what they should want.
- logicalmonster 3y agoAnything is possible, but I'd say it's a safe bet that their bad choices will inevitably infect everything they do.
- janalsncm 3y agoIsn’t the fact that Google considers this a bug evidence against exactly what you’re saying? If DEI was really the cause, and not a more broad concern about becoming the next Tay, they would’ve kept it as-is. Weird refusals and paternalistic concerns about harm are not desirable behavior. You can consider it a bug, just like the ChatGPT decoding bug the other day.
- pdimitar 3y agoSaying it's a bug is them trying to save face. They went out of their way to rewrite people's prompts after all. You don't have 100+ programmers stumble in the hallway and put all that code in by accident, come on now.
- bad_username 3y agoThe bug is Gemini's bias being blatant and obvious. The fix will be making it subtle and concealed.
- lupusreal 3y agoThey released it like this because people inside Google were too afraid to speak out against it. Only now that people outside the company are shouting that the emperor is naked do they seem to suddenly notice the obvious.
- gitaarik 3y agoIt's not a bug, it's a feature! A bug is when something unintentionally doesn't work or misbehaves. The DEI algorithm is intentionally added as a feature. It just has some output that seems buggy, but is actually because of this "feature". Whether it's a good feature is another discussion though ;).
- Ferret7446 3y agoThe public outcry is the bug. Or alternatively, if all of your customers hate it, it's not WAI even if it's WAI. It's a bug.
- hn_throwaway_99 3y agoI think the thing that makes me totally think this is "Google institutional rot" is there were some reports (https://news.ycombinator.com/item?id=39466135 https://news.ycombinator.com/item?id=39466135) that lots of people at Google knew this was a problem, but they felt powerless to say something less they be branded "anti-DEI" or some such. To me the most fundamental symptom of institutional rot is when people stop caring: "Yeah, we know this is insane, but every time I've seen people stick their necks out in the past and say 'You know, that Emperor really looks naked to me', they've been beheaded, so better to just stay quiet. And did you hear there'll be sushi at lunch in the cafeteria today!"
- subsubzero 3y agoI have been saying this for years but google is probably the most dysfunctional and slowest moving company in tech that is only surviving by its blatant search monopoly. Given that OpenAI a tiny company by comparison is destroying them on AI shows just how bad they are run. I see them falling slowly in the next year or as search is supplanted by AI and then expect to see a huge drop as they see huge usage drops. Youtube seems like their own valuable platform once search and its revenues disappear for them due to changing consumer behavior.
- kderbyma 3y agoPinchai is anything but a good leader....he is the blandest CEO yet somehow is seeped in politics....
- smsm42 3y agoWe are talking about the company that when a shooting happened in 2018, banned all the goods containing substring "gun" (including Burgundy wines, of course), from their shopping portal. They're so big nobody feels like they need to care about anything making sense anymore.
- Jensson 3y agoThe censorship arm of Google is powerful but not competent. So yeah you get dumb keyword matching returning 0 results. I remember something similar to "girl in miniskirt" returning 0 results on google since someone wrote an article about it. As far as I know the competent engineers doesn't work on this.
- randounho 3y ago[flagged]
- chadlifter69 3y agoThis argument could be used for anything. "I love it when black people cope and seethe about having to use separate water fountains. Imagine what holocaust victims who died of thirst in auschwitz would say about having to use a separate water fountain." Apologies to HN community for using a "swipe" here but idk how else to characterize how bad this argument is.
- danlugo92 3y ago> I am NOT one Could you be one though? (Thought exercise for any readers)