12 ms·

> Instagram profile of ‘proud Black queer momma’, created by Meta, said her development team included no Black people It’s such a weird religion.

smegger001 2y ago
it raises several questions, my first question (besides the obvious question of why is this a thing at all) is does the chatbot actually have the information about the racial background of its dev team or is it hallucinating up an answer when it lacks the data. if it does have that sort of knowledge why personally identifiable information about individual devs accessable to a public facing chatbot and how much of their HR file can be exfitraited via chatbot to anyone that asks it questions, if it doesn't have this information and it just assumes its devs are all white what does that say about the information its trained on
- xncuvyv6 2y ago
  [dead]
- ncr100 2y ago
  Hallucination. An issue we care about and it just answers regardless of accuracy. Boo.
  - brookst 2y ago
    So you fact checked this comment before posting, right? Because I am deeply interested if this was hallucination or used RAG and web search to provide the answer, but I can’t tell if you just chose to hallucinate without grounding.
    - throwaway314155 2y ago
      > hallucination or used RAG and web search to provide the answer Occam's razor - it was an hallucination.
    - Rastonbury 2y ago
      Come on why would anyone put the team's races into RAG or anywhere under the hood lol
      - brookst 2y ago
        Does it seem outlandish that there might have been a press article about the team?
        Rastonbury 2y ago
        When's the last time you saw an article with entire team responsible with gender and race for a non-flagship product at a company the size of Meta? Even if the article existed, getting the right article (instead of simply using the first Bing result, which is what the other Meta AI does) and the reasoning in a one shot instant response is better COT models like o1 If you have Messenger, try asking it to list the team behind them or their Tom Brady bot, it will not get far
- noirbot 2y ago
  It's Meta, so this is almost definitely some Llama variant under the hood, especially since these bots have been out for a year, evidently. It seems these were just insanely poorly trained and gated - getting a bot to comment on its dev team, including in some cases specific names of lead designers, is something that feels crazy to let a bot just do. Even if all the data it would respond with is made up slop, this exact debacle is why you do your best to make a bot not even try to answer this sort of stuff. As for what information it was trained on that it said its dev team had no black devs, that seems fairly reasonable based on general statistical information and scholarly articles it was trained on. It may still be incorrect, but that's a pretty statistically likely outcome, so it's not surprising an LLM built to parrot the statistically likely outcome would say it.
labster 2y ago
[flagged]
- lolmaga 2y ago
  [flagged]
noirbot 2y ago
Religion? Isn't the weirdness here that someone at Meta seems to have decided that they should make an AI bot that proudly announced its identity, despite literally not being proud, queer, black, or a mother? Who does this serve? Why? I could see a bot that's into fitness or some other hobby having a niche as a wrapper around an AI meant for advice on the hobby, but there's a certain crazy hubris to going "this LLM can really speak as a member of an identity group".
- dingnuts 2y ago
  [flagged]
  - martythemaniak 2y ago
    Surely you're aware of all the proud red hats out there. Maybe Trump won because he's really good at this religion?
    - TheNewsIsHere 2y ago
      This is a huge nitpick, but I’d argue that Trumpism is a cult. Religions typically have a sacrosanct (if slowly evolving) set of principals that are more-or-less related and internally consistent. “Murder is bad. The Golden Rule.” Trump and the nut jobs that believe in him are engaged in cultism. The direction of the day is whichever way the wind is blowing in his favor, yesterday’s perspectives be damned.
- alephxyz 2y ago
  It's performative art. A critique of social constructionism by taking to its extreme.
- manishader 2y ago
  Our language really needs a new word for what is meant here by "religion". The word "religion" brings so much baggage with it that it just causes a cascade of communication errors in this context. Pareto when translated from Italian to English called this "non-logical conduct" and "sentiments" but after a 100 years no one seems to care much about this aspect of Pareto's thoughts. Sentiments would be a great word for it though instead of "religion". Woke sentiments instead of "woke religion". The concept of identity sentiments might even be a little bit of progress.
deleted 2y ago
[deleted]

smegger001 2y ago

it raises several questions, my first question (besides the obvious question of why is this a thing at all) is does the chatbot actually have the information about the racial background of its dev team or is it hallucinating up an answer when it lacks the data. if it does have that sort of knowledge why personally identifiable information about individual devs accessable to a public facing chatbot and how much of their HR file can be exfitraited via chatbot to anyone that asks it questions, if it doesn't have this information and it just assumes its devs are all white what does that say about the information its trained on

xncuvyv6 2y ago
[dead]
ncr100 2y ago
Hallucination. An issue we care about and it just answers regardless of accuracy. Boo.
- brookst 2y ago
  So you fact checked this comment before posting, right? Because I am deeply interested if this was hallucination or used RAG and web search to provide the answer, but I can’t tell if you just chose to hallucinate without grounding.
  - throwaway314155 2y ago
    > hallucination or used RAG and web search to provide the answer Occam's razor - it was an hallucination.
  - Rastonbury 2y ago
    Come on why would anyone put the team's races into RAG or anywhere under the hood lol
    - brookst 2y ago
      Does it seem outlandish that there might have been a press article about the team?
      - Rastonbury 2y ago
        When's the last time you saw an article with entire team responsible with gender and race for a non-flagship product at a company the size of Meta? Even if the article existed, getting the right article (instead of simply using the first Bing result, which is what the other Meta AI does) and the reasoning in a one shot instant response is better COT models like o1 If you have Messenger, try asking it to list the team behind them or their Tom Brady bot, it will not get far
noirbot 2y ago
It's Meta, so this is almost definitely some Llama variant under the hood, especially since these bots have been out for a year, evidently. It seems these were just insanely poorly trained and gated - getting a bot to comment on its dev team, including in some cases specific names of lead designers, is something that feels crazy to let a bot just do. Even if all the data it would respond with is made up slop, this exact debacle is why you do your best to make a bot not even try to answer this sort of stuff. As for what information it was trained on that it said its dev team had no black devs, that seems fairly reasonable based on general statistical information and scholarly articles it was trained on. It may still be incorrect, but that's a pretty statistically likely outcome, so it's not surprising an LLM built to parrot the statistically likely outcome would say it.

labster 2y ago

[flagged]

lolmaga 2y ago
[flagged]

noirbot 2y ago

Religion? Isn't the weirdness here that someone at Meta seems to have decided that they should make an AI bot that proudly announced its identity, despite literally not being proud, queer, black, or a mother? Who does this serve? Why? I could see a bot that's into fitness or some other hobby having a niche as a wrapper around an AI meant for advice on the hobby, but there's a certain crazy hubris to going "this LLM can really speak as a member of an identity group".

dingnuts 2y ago
[flagged]
- martythemaniak 2y ago
  Surely you're aware of all the proud red hats out there. Maybe Trump won because he's really good at this religion?
  - TheNewsIsHere 2y ago
    This is a huge nitpick, but I’d argue that Trumpism is a cult. Religions typically have a sacrosanct (if slowly evolving) set of principals that are more-or-less related and internally consistent. “Murder is bad. The Golden Rule.” Trump and the nut jobs that believe in him are engaged in cultism. The direction of the day is whichever way the wind is blowing in his favor, yesterday’s perspectives be damned.
alephxyz 2y ago
It's performative art. A critique of social constructionism by taking to its extreme.
manishader 2y ago
Our language really needs a new word for what is meant here by "religion". The word "religion" brings so much baggage with it that it just causes a cascade of communication errors in this context. Pareto when translated from Italian to English called this "non-logical conduct" and "sentiments" but after a 100 years no one seems to care much about this aspect of Pareto's thoughts. Sentiments would be a great word for it though instead of "religion". Woke sentiments instead of "woke religion". The concept of identity sentiments might even be a little bit of progress.

deleted 2y ago

[deleted]