7 ms·

New UNESCO report warns that Generative AI threatens Holocaust memory

36 points by t0bia_s 2y ago

hammyhavoc 2y ago
Absolutely. This can be applied to anything and everything in its output, and this is why I take a very dim view of how quickly this has been trotted out, yet how little understanding the average person has of the maths behind it. You should try asking LLMs about politics, specifically policies of each party, ask it to help you decide who to vote for et cetera. You'll very quickly see it spout absolute bollocks, especially if you're talking politics beyond the USA or Canada. Extra nonsensical if you start using languages other than English. Even asking it questions about COVID can yield some nonsensical responses in regards to politics. Having it gaslight you about the NHS and policies of each party is also amazing, especially when you tell it that it's wrong and it tells you that it isn't. With elections looming, these things are the next Brexit bus or Cambridge Analytica waiting to happen.
- TheCapeGreek 2y ago
  >especially if you're talking politics beyond the USA or Canada I think globally we've very quickly hit LLM limits because they were simply not trained on as much information as we think. Or at least, not nearly as much region-specific information in many countries. I've hit the same issues trying to ask any LLM to give me tax advice for tax law in specific countries, and even beyond the "ask a professional" type restrictions eg OpenAI has, it's clear that it just does not have the information. Sometimes that's just because of the US-centric view of these companies training the models, and sometimes it's also because information sources are tucked away on botched government Wordpress sites and random PDFs. This is why you have to do things like RAG to make a "talk to your doc" kind of bot even if you're feeding it public info.
- KingMob 2y ago
  The extra danger is when we don't know it comes from an LLM. If we think a human wrote it, we're less skeptical.
  - hammyhavoc 2y ago
    I've had a few less tech-savvy friends tell me dubious things this past year and asked where they heard it—usually "this neat app I found called ChatGPT". eyeroll
boxed 2y ago
Asking a large language model about facts is like asking an image model for photos. That's not what it's for, or what it does. It's a bit weird that we're building tools that make the large scale pollution of the information space approach a marginal cost of zero. I get that it's a bit useful for artists, and for helping dyslexics write emails or whatever, but it's highly doubtful this will be worth it. We already have huge issues with humans being stupid, why would we insert this not-even-stupid noise into the information space?
- seydor 2y ago
  > That's not what it's for, or what it does. Then again, it has been marketed as such, and does not tell people that it's lying. Image models don't have that problem because they never claimed to answer truthfully.
  - intended 2y ago
    I can’t wait to see what happens as the term info pollution gains transaction. We already know that generative performance degrades when they learn from generated data (don’t remember the exact findings).
  - boxed 2y ago
    I think even calling it lying is too kind. It just has no relation to truth.
- ADeerAppeared 2y ago
  > why would we insert this not-even-stupid noise into the information space? Because US tech would "collapse" otherwise. Big Tech's stock price is entirely propped up by AI hype. Metaverse didn't go anywhere so they need a new bubble. US tech startups are even worse. The VC culture is completely incapable of building useful companies, AI is the last hype. Just a rat-race to grab the last funding and get bought out before the end of ZIRP kills the startup scene entirely. There's no real demand for any of this shit. Even the web spam renaissance is mostly hustle-bros losing money. The writing is already on the wall with investment slowing down, and AI product after AI product bombing.
seydor 2y ago
[flagged]
- aredox 2y ago
  Oct. 7th 2023 was the greatest massacre of Jews since the Holocaust. Who threatens who, already?
  - fabioborellini 2y ago
    [flagged]
    - aredox 2y ago
      [flagged]
  - megous 2y ago
    [flagged]
    - aredox 2y ago
      >Even one of their prime ministers headed a terror organization that killed > 200 jews itself in one incident. https://en.wikipedia.org/wiki/Patria_disaster https://en.wikipedia.org/wiki/Patria_disaster Did Hamas kill over 1000 jews by accident? Did they parade their bodies by accident too? Do you have an actual argument or do you intend to whatabout in more and more stupid ways?
      - megous 2y ago
        Yes, it's possible the scale of killings was unintended by the leadership, and the focus was on other things. Hard to say without independent investigation. https://www.wsj.com/world/middle-east/gaza-chiefs-brutal-calculation-civilian-bloodshed-will-help-hamas-626720e7 https://www.wsj.com/world/middle-east/gaza-chiefs-brutal-cal... > Though Sinwar planned and greenlighted the Oct. 7 attacks, early messages to cease-fire negotiators show he seemed surprised by the brutality of Hamas’s armed wing and other Palestinians, and how easily they committed civilian atrocities. “Things went out of control,” Sinwar said in one of his messages, referring to gangs taking civilian women and children as hostages. “People got caught up in this, and that should not have happened.” And you will not win an argument about who's threat to who in this particular conflict in favor of Israeli Jews by comapring numbers of victims.
hruzgar 2y ago
[flagged]
- indigo945 2y ago
  [flagged]
  - ta-295235423 2y ago
    [flagged]
    - wiseowise 2y ago
      [flagged]
      - ta-295235423 2y ago
        [flagged]
    - Propelloni 2y ago
      Ah, moving the goal post. Caring about the Nazi genocide of the Jews does not exclude caring about other genocides. But you, of course, knew that. You just used an unfortunate phrase, didn't you?
      - ta-295235423 2y ago
        [flagged]
        Propelloni 2y ago
        Dito. "who has the power and money" is your answer to your question "why is UNESCO [sic] specifically concerned about LLMs diluting Holocaust facts, and not the significantly more important, current genocides taking place?" It is not the only answer, and you know it. As other posters have already pointed out, the Holocaust is a pretty special event in history which makes it an excellent example of the potential dangers of revisionism through LLMs.
roenxi 2y ago
I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness. Which UNESCO demonstrates. The problem is the Holocaust was the first time industrial techniques were applied to killing large groups of citizens. The unique evil is the tools, bureaucracy and mundanity of mass murder. The horror was kids in the wrong family tree being shipped off to well engineered gas chambers - not the details of ideologies and personalities. We're already forgetting the more important lessons of not designating internal citizen groups as the out-group (if we ever learned it) and not building up those registers labelling for one bad government to use. > The Historical Figures App allowed users to chat with prominent Nazis such as Adolf Hitler and Joseph Goebbels, and falsely claimed that individuals such as Goebbels were not intentionally involved in the Holocaust and had tried to prevent violence against Jews. I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).
- surfingdino 2y ago
  > I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness. Misinformation is a real, serious threat. > I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed). I was born and raised in a city where one of the largest concentration camps was built and run by Germans. Members of my family were murdered by Germans following orders of Hitler and told that it was necessary to do so by Goebbels. Then Russians came and repurposed the camp to imprison people who fought Germans and ruled the place for close to 50 years by proxies. I have no desire to see genAI or "works of art" showing Hitler or Goebbels as innocent or even likeable characters. Holocaust was a real thing. The fact that other genocides happened before and after does not change the reality or the legacy of it. Hitler puppets saying he was innocent are not just denying the facts, they are prepping people for another round of persecutions, and not just Jews. If you think it's ok to think Hitler or Goebbels are irrelevant, or that what they did doesn't matter then you are paving the way you will be sent down on to a camp yourself.
oefrha 2y ago
You don’t need ChatGPT to rewrite WWII history. I choke every time I read “pacifist Japan militarize in response to expansionist neighbor” in American media.
blueflow 2y ago
This article gives me "What an odd thing to say" vibes. I guess the focus on antisemitism and holocaust memory is because Published in partnership with the World Jewish Congress Which is still an odd choice when it comes to AI. I think this article should be more framed as "World Jewish Congress considers AI dangerous".
- boxed 2y ago
  I mean.. everyone who can string two words together does...
oldpersonintx 2y ago
[dead]
propaganja 2y ago
What the actual fuck? People think the earth is flat—does Generative AI threaten our core understanding of astrophysics?
- KingMob 2y ago
  If flat-earthers had the numbers and motivation of revisionist neo-nazis, they might very well be able to decrease popular (not academic) understanding of astrophysics.
aaron695 2y ago
[dead]
helij 2y ago
History rewriting has been going on for awhile now. No need for LLM and "malicious actors to seed disinformation and hate-fueled narratives".
deleted 2y ago
[deleted]

36 points by t0bia_s 2y ago

Absolutely. This can be applied to anything and everything in its output, and this is why I take a very dim view of how quickly this has been trotted out, yet how little understanding the average person has of the maths behind it. You should try asking LLMs about politics, specifically policies of each party, ask it to help you decide who to vote for et cetera. You'll very quickly see it spout absolute bollocks, especially if you're talking politics beyond the USA or Canada. Extra nonsensical if you start using languages other than English. Even asking it questions about COVID can yield some nonsensical responses in regards to politics. Having it gaslight you about the NHS and policies of each party is also amazing, especially when you tell it that it's wrong and it tells you that it isn't. With elections looming, these things are the next Brexit bus or Cambridge Analytica waiting to happen.

TheCapeGreek 2y ago
>especially if you're talking politics beyond the USA or Canada I think globally we've very quickly hit LLM limits because they were simply not trained on as much information as we think. Or at least, not nearly as much region-specific information in many countries. I've hit the same issues trying to ask any LLM to give me tax advice for tax law in specific countries, and even beyond the "ask a professional" type restrictions eg OpenAI has, it's clear that it just does not have the information. Sometimes that's just because of the US-centric view of these companies training the models, and sometimes it's also because information sources are tucked away on botched government Wordpress sites and random PDFs. This is why you have to do things like RAG to make a "talk to your doc" kind of bot even if you're feeding it public info.
KingMob 2y ago
The extra danger is when we don't know it comes from an LLM. If we think a human wrote it, we're less skeptical.
- hammyhavoc 2y ago
  I've had a few less tech-savvy friends tell me dubious things this past year and asked where they heard it—usually "this neat app I found called ChatGPT". eyeroll

boxed 2y ago

Asking a large language model about facts is like asking an image model for photos. That's not what it's for, or what it does. It's a bit weird that we're building tools that make the large scale pollution of the information space approach a marginal cost of zero. I get that it's a bit useful for artists, and for helping dyslexics write emails or whatever, but it's highly doubtful this will be worth it. We already have huge issues with humans being stupid, why would we insert this not-even-stupid noise into the information space?

seydor 2y ago
> That's not what it's for, or what it does. Then again, it has been marketed as such, and does not tell people that it's lying. Image models don't have that problem because they never claimed to answer truthfully.
- intended 2y ago
  I can’t wait to see what happens as the term info pollution gains transaction. We already know that generative performance degrades when they learn from generated data (don’t remember the exact findings).
- boxed 2y ago
  I think even calling it lying is too kind. It just has no relation to truth.
ADeerAppeared 2y ago
> why would we insert this not-even-stupid noise into the information space? Because US tech would "collapse" otherwise. Big Tech's stock price is entirely propped up by AI hype. Metaverse didn't go anywhere so they need a new bubble. US tech startups are even worse. The VC culture is completely incapable of building useful companies, AI is the last hype. Just a rat-race to grab the last funding and get bought out before the end of ZIRP kills the startup scene entirely. There's no real demand for any of this shit. Even the web spam renaissance is mostly hustle-bros losing money. The writing is already on the wall with investment slowing down, and AI product after AI product bombing.

seydor 2y ago

[flagged]

aredox 2y ago
Oct. 7th 2023 was the greatest massacre of Jews since the Holocaust. Who threatens who, already?
- fabioborellini 2y ago
  [flagged]
  - aredox 2y ago
    [flagged]
- megous 2y ago
  [flagged]
  - aredox 2y ago
    >Even one of their prime ministers headed a terror organization that killed > 200 jews itself in one incident. https://en.wikipedia.org/wiki/Patria_disaster https://en.wikipedia.org/wiki/Patria_disaster Did Hamas kill over 1000 jews by accident? Did they parade their bodies by accident too? Do you have an actual argument or do you intend to whatabout in more and more stupid ways?
    - megous 2y ago
      Yes, it's possible the scale of killings was unintended by the leadership, and the focus was on other things. Hard to say without independent investigation. https://www.wsj.com/world/middle-east/gaza-chiefs-brutal-calculation-civilian-bloodshed-will-help-hamas-626720e7 https://www.wsj.com/world/middle-east/gaza-chiefs-brutal-cal... > Though Sinwar planned and greenlighted the Oct. 7 attacks, early messages to cease-fire negotiators show he seemed surprised by the brutality of Hamas’s armed wing and other Palestinians, and how easily they committed civilian atrocities. “Things went out of control,” Sinwar said in one of his messages, referring to gangs taking civilian women and children as hostages. “People got caught up in this, and that should not have happened.” And you will not win an argument about who's threat to who in this particular conflict in favor of Israeli Jews by comapring numbers of victims.

hruzgar 2y ago

[flagged]

indigo945 2y ago
[flagged]
- ta-295235423 2y ago
  [flagged]
  - wiseowise 2y ago
    [flagged]
    - ta-295235423 2y ago
      [flagged]
  - Propelloni 2y ago
    Ah, moving the goal post. Caring about the Nazi genocide of the Jews does not exclude caring about other genocides. But you, of course, knew that. You just used an unfortunate phrase, didn't you?
    - ta-295235423 2y ago
      [flagged]
      - Propelloni 2y ago
        Dito. "who has the power and money" is your answer to your question "why is UNESCO [sic] specifically concerned about LLMs diluting Holocaust facts, and not the significantly more important, current genocides taking place?" It is not the only answer, and you know it. As other posters have already pointed out, the Holocaust is a pretty special event in history which makes it an excellent example of the potential dangers of revisionism through LLMs.

roenxi 2y ago

I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness. Which UNESCO demonstrates. The problem is the Holocaust was the first time industrial techniques were applied to killing large groups of citizens. The unique evil is the tools, bureaucracy and mundanity of mass murder. The horror was kids in the wrong family tree being shipped off to well engineered gas chambers - not the details of ideologies and personalities. We're already forgetting the more important lessons of not designating internal citizen groups as the out-group (if we ever learned it) and not building up those registers labelling for one bad government to use. > The Historical Figures App allowed users to chat with prominent Nazis such as Adolf Hitler and Joseph Goebbels, and falsely claimed that individuals such as Goebbels were not intentionally involved in the Holocaust and had tried to prevent violence against Jews. I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed).

surfingdino 2y ago
> I don't think it is reasonable to say Generative AI is a bigger threat than basic forgetfulness. Misinformation is a real, serious threat. > I mean really. This is a nonissue. Hitler and Goebbels aren't unique evils and there isn't any point remembering them as people; at this point they are more rhetorical symbols then people. Genocidal maniacs aren't at all uncommon in history (the records suggest Ghengis Khan literally erased entire countries if he felt they were inconvenient to him - and his title is "Great" because he won and enough time passed). I was born and raised in a city where one of the largest concentration camps was built and run by Germans. Members of my family were murdered by Germans following orders of Hitler and told that it was necessary to do so by Goebbels. Then Russians came and repurposed the camp to imprison people who fought Germans and ruled the place for close to 50 years by proxies. I have no desire to see genAI or "works of art" showing Hitler or Goebbels as innocent or even likeable characters. Holocaust was a real thing. The fact that other genocides happened before and after does not change the reality or the legacy of it. Hitler puppets saying he was innocent are not just denying the facts, they are prepping people for another round of persecutions, and not just Jews. If you think it's ok to think Hitler or Goebbels are irrelevant, or that what they did doesn't matter then you are paving the way you will be sent down on to a camp yourself.

oefrha 2y ago

You don’t need ChatGPT to rewrite WWII history. I choke every time I read “pacifist Japan militarize in response to expansionist neighbor” in American media.

blueflow 2y ago

This article gives me "What an odd thing to say" vibes. I guess the focus on antisemitism and holocaust memory is because Published in partnership with the World Jewish Congress Which is still an odd choice when it comes to AI. I think this article should be more framed as "World Jewish Congress considers AI dangerous".

boxed 2y ago
I mean.. everyone who can string two words together does...

oldpersonintx 2y ago

[dead]

propaganja 2y ago

What the actual fuck? People think the earth is flat—does Generative AI threaten our core understanding of astrophysics?

KingMob 2y ago
If flat-earthers had the numbers and motivation of revisionist neo-nazis, they might very well be able to decrease popular (not academic) understanding of astrophysics.

aaron695 2y ago

[dead]

helij 2y ago

History rewriting has been going on for awhile now. No need for LLM and "malicious actors to seed disinformation and hate-fueled narratives".

deleted 2y ago

[deleted]