6 ms·

'Model collapse'? An expert explains the rumours about an impending AI doom

46 points by ColinWright 2y ago

viraptor 2y ago
This could use a bit more nuance around training from AI. While the naive approaches will experience worse replies, there are many documented cases where the quality improves instead. Star, self-reflection, groups of agents, and likely others I don't know about all improve the results using only the same model's output.
antklan 2y ago
This can be replicated on a small scale by using two LLMs (which can be two instances of the same LLM). Start with a human prompt, then feed the answer of LLM-1 as the prompt to LLM-2, then feed that answer of LLM-2 to LLM-1 and so forth. The answers soon converge to some boring, bland repetition that isn't even logical. No intelligence here, which means that code emitting LLMs are just stealing human IP that they happened to have read.
- JKCalhoun 2y ago
  I too was trained stealing human "IP". It just took decades (perhaps is continuing still). Maybe slow, ongoing AI is the future.
  - lab03615 2y ago
    I would reject the implicit materialistic worldview, but anyone is free to have their own opinions of course. Aside from that, if a human creates a new work that builds on other people's work, it must have sufficient creativity and magnitude to qualify as new IP. A human also does not connect to the Internet and distributes the IP that they have read to millions of users for subscription fee.
    - ben_w 2y ago
      > A human also does not connect to the Internet and distributes the IP that they have read to millions of users for subscription fee. Patreon, substack, reaction videos, the news? "It's one of those irregular verbs, isn't it? I am inspired, you plagiarise, they are being prosecuted under Section 7 of the Copyright Act" - https://news.ycombinator.com/item?id=36830574 https://news.ycombinator.com/item?id=36830574
- viraptor 2y ago
  What do you think would happen if you did that with people who have no reason to talk, but are forced to give an answer each time? I'm sure at some point they'd just degrade to nonsense, but I don't think that says anything about intelligence. (Being stuck for a long time with a friend does occasionally degrade into nonsense and absurd injokes in my experience)
  - lab03615 2y ago
    In a political discussion people are forced to give an answer each time and yet the discussion does not converge to utter nonsense, provided that both participants are intelligent. For the younger people here who have only watched recent political discussions, search on YouTube for discussions before 2000, preferably before 1980. Same for a scientific discussion. By the way, Bing Copilot absolutely aborts conversations it does not like. LLMs could also change the subject, like humans do if they detect repetition.
    - viraptor 2y ago
      > In a political discussion That's a completely different scenario than OP stated. In a political discussion you have an agenda, reasons to engage, plan for progress, constraints on topics/time, context and experience, and you know when to withdraw. The scenario described is closer to "Do a political discussion." and being forced to continue forever.
  - thuuuomas 2y ago
    Can you link this idle observation directly to the operation of LLMs or is this more anthropomorphizing boosterism?
    - viraptor 2y ago
      I'm not sure where you saw anthropomorphizing. It's not in my message.
- whoitwas 2y ago
  I wouldn't call crawling public data stealing. LLMs are just lazy and bad like the people who implemented them.
- ndsipa_pomu 2y ago
  > No intelligence here, which means that code emitting LLMs are just stealing human IP that they happened to have read. Well, "stealing" is incorrect as they don't consume the original text - it's still there. "Plagiarising " would be more accurate.
JKCalhoun 2y ago
> To train GPT-3, OpenAI needed over 650 billion English words of text – about 200x more than the entire English Wikipedia. Since, I assume, humans don't need this much training (?) this area would seem to be ripe to explore — can you achieve similar training with a fraction of the data needed for GPT-3.
- mattnewton 2y ago
  It’s an apples to oranges comparison- humans are getting massive amounts of data from their nervous system far richer than the token stream of Wikipedia. The model has to learn everything about the world solely from statistical patterns of text whereas humans are seeing, hearing and touching the world in full resolution. Humans also take an active role in seaking novel stimuli that they can learn from. But that does I think shed light on why multi-modality is so promising; beyond additional use cases, it could make the models better at existing text tasks too when they train on all that video data. And I think it also reinforces why everyone is so hyped for reinforcement-learning techniques like the self play that alpha zero used to come LLMs.
  - mschuster91 2y ago
    > humans are getting massive amounts of data from their nervous system far richer than the token stream of Wikipedia. And their tokenizers are better - that's the primary reason why ChatGPT is so bad at math, its network literally doesn't (and can't!) comprehend numbers longer than the token size by definition. Additionally, humans can cross-reference information with information they already know at learning time. For example, an AI "reads" some climate change denier politician's speech - how is it supposed to know that the politician is a climate change denier and downrank their opinion on climate change as a result, at least without a human having pre-classified the data?
    - zarzavat 2y ago
      Humans tokenize numbers too. Which is easier to read? 100000000 or 100,000,000? You only need tokens for 000 to 999
botro 2y ago
In the model testing I've conducted, I've seen that LLMs from competing companies including GPT-4o, Gemini Flash 1.5, Llama 3.1 and Phi-3 all converge on the exact same joke. For a test of creativity this was alarming. They all tell slight variations of the same joke about ladders. I've posted about it here: https://news.ycombinator.com/item?id=41125309 https://news.ycombinator.com/item?id=41125309
KolmogorovComp 2y ago
> For instance, researchers found a 16% drop in activity on the coding website StackOverflow one year after the release of ChatGPT. This suggests AI assistance may already be reducing person-to-person interactions in some online communities. I don't get why this is a bad news. It meant that most likely these questions would have been duplicates and/or very easy, as ChatGPT could fix them. My issue with SO is on the opposite side of the spectrum. By the time I am ready to post a question after having done my homeworks, I know it is either an unfixable bug or out of reach for most if not all users, and the probability of getting an answer is very low. Most of the time I then answer my own questions a few months later.
- clausecker 2y ago
  > Most of the time I then answer my own questions a few months later. Thank you for doing that. This is very valuable.
- lifeisstillgood 2y ago
  Agreed - those who get the most out of SO seem to use it as a personal blogging tool - answering questions as a trigger for writing out what they know. I have always assumed that a good marketing ruse for FOSS would be to pay an intern to basically Q&A your entire docs into SO - if you do it right it would even build a better roadmap for your project.
- creesch 2y ago
  The article actually does explain this in the first few paragraphs. It is an issue because communities like StackOverflow are actually one of the sources of training material. LLM's are not capable of innovation, they can only do variations based on what is in their training material. People asking questions to other people about things does allow for actual new and fresh answer, where people might come up with solutions other people have not yet. And yes, you are right, some of these answers would be duplicates and easy to answer. But only to a certain point, there are a lot of answers on StackOverflow that are technically correct but severely outdated by this point. They have been superseeded by more modern ways of doing things or the APIs to do so have changed in newer versions of frameworks. With a human community you will have people leaving comments on older answers pointing this out and answering with the current answers on new questions. LLM's, again, are not capable of picking up this new information on their own. You can actually often see this in the answers you get from them, they are often ever so slightly dated already. When the communities that feed them die they will have less learning material so at the very least they will stagnate.
lifeisstillgood 2y ago
>>> 98% of collected data was rejected. I mean apart from wow ! 98% of the internet is shit (that’s higher than even I assumed) - how did they differentiate between good and shit? PageRank? Length? I can tell the difference between good writing and bad writing (and porn and erotica) but I have to read it using my brain based LLM - how did they do it ? This leads to the whole open source, show us your training data thing
- creesch 2y ago
  Very cheap human labor in Africa: - https://time.com/6247678/openai-chatgpt-kenya-workers/ https://time.com/6247678/openai-chatgpt-kenya-workers/ - https://www.theguardian.com/technology/2024/apr/16/techscape-ai-gadgest-humane-ai-pin-chatgpt https://www.theguardian.com/technology/2024/apr/16/techscape... Another hidden aspect of the current LLM landscape.
- Mashimo 2y ago
  The article says 90% of collected data. Not 98% of all internet. > discard as much as 90% of the data they initially collect for training models.
mathw 2y ago
Surely the only way you overcome this is to write a system which can actually generate something new, rather than regurgitate an incredibly complicated statistical reworking of things it got trained on. I did note how the author said that training models on other models doesn't have the ethical implications of training on stolen human data - except it does, because where did the first model get its training set from? This is why we make it illegal not just to steal but also to handle stolen goods.
DrScientist 2y ago
In the human space there is also 'model collapse' as demonstrated by group-think, or the dark arts of marketing/PR to sway opinion in the large. This is why the scientific method was developed. If you want an AI that truly learns then it has to get up of the sofa, and test the ideas in the real world, rather than learning to parrot hearsay.
- tho234i2342334 2y ago
  The "scientific method" doesn't fix anymore than praying to Isis does. It's not the way science is done any way.
  - DrScientist 2y ago
    Don't understand. How does doing real world experiments to test your hypothesis not help with the very common problem of being wrong? > It's not the way science is done any way. That's a very sweeping statement - especially since the scientific method is quite a catch-all phrase. In the end science is driven forward by new data, gathered either intentionally or incidentally. Self-driving cars can continue to learn because they are continually generating more data. Advertising algorithms can continue to learn and adapt as the campaigns they create are in effect experiments - and some are even well designed. I suppose large language models can continue to learn by chatting to humans - but they are potentially only learning how to please humans, not necessarily underlying truths.
ColinWright 2y ago
People flooding the 'net with LLM-generated crap are both eating their seed corn and poisoning the well. Steel manufactured before 1944[0] can be incredibly valuable[1] because it hasn't been tainted by the nuclear tests/bombs fallout, and maybe fairly soon any archive internet material written pre-2010 will be considered equally valuable. [0] https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel [1] https://interestingengineering.com/science/what-is-pre-war-steel https://interestingengineering.com/science/what-is-pre-war-s...
- seydor 2y ago
  I guess peopoe thought the same when photocopies were invented. Unlike steel, language is a human product
  - cinntaile 2y ago
    Steel is also a human product.
  - omnicognate 2y ago
    Not sure what point this is trying to make, but steel is a man-made alloy, not a naturally occurring substance (except possibly in tiny quantities by chance). The background radiation gets into it through the use of atmospheric air in the production process, so it seems rather a good analogy for background AI output entering the production process of internet "content" and rendering it less suitable for certain purposes.
    - seydor 2y ago
      Language is entirely human made , and it s affected by both communication and the medium. Every time a new mode of communication was invented, language was polluted with other people's language. AI is both a medium and communication which indirectly communicates using the "medial value content" of all other people on the internet. That content was itself already contaminated via too much communication over the past 30 years. So it's not really a fundamental shift , rather an acceleration of a process that s already happening
imtringued 2y ago
We don't need "more" data. We already have all the data we need in terms of quantity. We don't need more "synthetic" data for supervised learning. We need better training algorithms that go beyond minimizing token level loss.
openrisk 2y ago
We can speculate all day long how the current "AI" phenomenon will evolve but, alas, there isn't much solid on which to ground arguments. In the oral phase of human communication "AI models" were residing within human brains: the origin of any stream of messages was literally in front of you, in flesh and blood. In the print phase we have the first major decoupling. Whether in the form of a cuneiform tablet or a modern paperback, the provenance was no longer assured. We had to rely on "controlled" seals, trust the publishers and their distribution chains etc. Ultimately this worked because the relative difficulty of producing printed artifacts helped developed a legal/political apparatus to control the spread of the "fake" stuff. Enter the digital communications era and we have the second major decoupling. The amount (and increasingly the apparent veracity) of generating human oriented messaging is no longer a limiting factor. This has no precedent. You can create now plausibly create a fake Wikipedia [1] by just running a model. The signal-to-noise of digitally exchanged messages can experience catastrophic collapse. > A flood of synthetic content might not pose an existential threat to the progress of AI development, but it does threaten the digital public good of the (human) internet. Indeed, this is the real risk. I don't care about idiot "AI" feeding on itself but I do care about destroying any basis of sane digital communication between humans. Will we develop the legal/political apparatus to control the flood of algorithmic junk? Will it be remotely democratic? The best precedent of digital automation destroying communication channels and our apparent inability and/or unwillingness to do something about it is email spam [2]. Decades after it first appeared spam is still degrading our infosphere. The writing is on the wall. [1] https://en.wikipedia.org/wiki/Wikipedia:List_of_hoaxes_on_Wikipedia https://en.wikipedia.org/wiki/Wikipedia:List_of_hoaxes_on_Wi... [1] https://en.wikipedia.org/wiki/Email_spam https://en.wikipedia.org/wiki/Email_spam
pjc50 2y ago
See also https://www.alexirpan.com/2024/07/08/tragedies-of-reality.html https://www.alexirpan.com/2024/07/08/tragedies-of-reality.ht...

46 points by ColinWright 2y ago

This could use a bit more nuance around training from AI. While the naive approaches will experience worse replies, there are many documented cases where the quality improves instead. Star, self-reflection, groups of agents, and likely others I don't know about all improve the results using only the same model's output.

antklan 2y ago

This can be replicated on a small scale by using two LLMs (which can be two instances of the same LLM). Start with a human prompt, then feed the answer of LLM-1 as the prompt to LLM-2, then feed that answer of LLM-2 to LLM-1 and so forth. The answers soon converge to some boring, bland repetition that isn't even logical. No intelligence here, which means that code emitting LLMs are just stealing human IP that they happened to have read.

JKCalhoun 2y ago
I too was trained stealing human "IP". It just took decades (perhaps is continuing still). Maybe slow, ongoing AI is the future.
- lab03615 2y ago
  I would reject the implicit materialistic worldview, but anyone is free to have their own opinions of course. Aside from that, if a human creates a new work that builds on other people's work, it must have sufficient creativity and magnitude to qualify as new IP. A human also does not connect to the Internet and distributes the IP that they have read to millions of users for subscription fee.
  - ben_w 2y ago
    > A human also does not connect to the Internet and distributes the IP that they have read to millions of users for subscription fee. Patreon, substack, reaction videos, the news? "It's one of those irregular verbs, isn't it? I am inspired, you plagiarise, they are being prosecuted under Section 7 of the Copyright Act" - https://news.ycombinator.com/item?id=36830574 https://news.ycombinator.com/item?id=36830574
viraptor 2y ago
What do you think would happen if you did that with people who have no reason to talk, but are forced to give an answer each time? I'm sure at some point they'd just degrade to nonsense, but I don't think that says anything about intelligence. (Being stuck for a long time with a friend does occasionally degrade into nonsense and absurd injokes in my experience)
- lab03615 2y ago
  In a political discussion people are forced to give an answer each time and yet the discussion does not converge to utter nonsense, provided that both participants are intelligent. For the younger people here who have only watched recent political discussions, search on YouTube for discussions before 2000, preferably before 1980. Same for a scientific discussion. By the way, Bing Copilot absolutely aborts conversations it does not like. LLMs could also change the subject, like humans do if they detect repetition.
  - viraptor 2y ago
    > In a political discussion That's a completely different scenario than OP stated. In a political discussion you have an agenda, reasons to engage, plan for progress, constraints on topics/time, context and experience, and you know when to withdraw. The scenario described is closer to "Do a political discussion." and being forced to continue forever.
- thuuuomas 2y ago
  Can you link this idle observation directly to the operation of LLMs or is this more anthropomorphizing boosterism?
  - viraptor 2y ago
    I'm not sure where you saw anthropomorphizing. It's not in my message.
whoitwas 2y ago
I wouldn't call crawling public data stealing. LLMs are just lazy and bad like the people who implemented them.
ndsipa_pomu 2y ago
> No intelligence here, which means that code emitting LLMs are just stealing human IP that they happened to have read. Well, "stealing" is incorrect as they don't consume the original text - it's still there. "Plagiarising " would be more accurate.

JKCalhoun 2y ago

> To train GPT-3, OpenAI needed over 650 billion English words of text – about 200x more than the entire English Wikipedia. Since, I assume, humans don't need this much training (?) this area would seem to be ripe to explore — can you achieve similar training with a fraction of the data needed for GPT-3.

mattnewton 2y ago
It’s an apples to oranges comparison- humans are getting massive amounts of data from their nervous system far richer than the token stream of Wikipedia. The model has to learn everything about the world solely from statistical patterns of text whereas humans are seeing, hearing and touching the world in full resolution. Humans also take an active role in seaking novel stimuli that they can learn from. But that does I think shed light on why multi-modality is so promising; beyond additional use cases, it could make the models better at existing text tasks too when they train on all that video data. And I think it also reinforces why everyone is so hyped for reinforcement-learning techniques like the self play that alpha zero used to come LLMs.
- mschuster91 2y ago
  > humans are getting massive amounts of data from their nervous system far richer than the token stream of Wikipedia. And their tokenizers are better - that's the primary reason why ChatGPT is so bad at math, its network literally doesn't (and can't!) comprehend numbers longer than the token size by definition. Additionally, humans can cross-reference information with information they already know at learning time. For example, an AI "reads" some climate change denier politician's speech - how is it supposed to know that the politician is a climate change denier and downrank their opinion on climate change as a result, at least without a human having pre-classified the data?
  - zarzavat 2y ago
    Humans tokenize numbers too. Which is easier to read? 100000000 or 100,000,000? You only need tokens for 000 to 999

botro 2y ago

In the model testing I've conducted, I've seen that LLMs from competing companies including GPT-4o, Gemini Flash 1.5, Llama 3.1 and Phi-3 all converge on the exact same joke. For a test of creativity this was alarming. They all tell slight variations of the same joke about ladders. I've posted about it here: https://news.ycombinator.com/item?id=41125309 https://news.ycombinator.com/item?id=41125309

KolmogorovComp 2y ago

> For instance, researchers found a 16% drop in activity on the coding website StackOverflow one year after the release of ChatGPT. This suggests AI assistance may already be reducing person-to-person interactions in some online communities. I don't get why this is a bad news. It meant that most likely these questions would have been duplicates and/or very easy, as ChatGPT could fix them. My issue with SO is on the opposite side of the spectrum. By the time I am ready to post a question after having done my homeworks, I know it is either an unfixable bug or out of reach for most if not all users, and the probability of getting an answer is very low. Most of the time I then answer my own questions a few months later.

clausecker 2y ago
> Most of the time I then answer my own questions a few months later. Thank you for doing that. This is very valuable.
lifeisstillgood 2y ago
Agreed - those who get the most out of SO seem to use it as a personal blogging tool - answering questions as a trigger for writing out what they know. I have always assumed that a good marketing ruse for FOSS would be to pay an intern to basically Q&A your entire docs into SO - if you do it right it would even build a better roadmap for your project.
creesch 2y ago
The article actually does explain this in the first few paragraphs. It is an issue because communities like StackOverflow are actually one of the sources of training material. LLM's are not capable of innovation, they can only do variations based on what is in their training material. People asking questions to other people about things does allow for actual new and fresh answer, where people might come up with solutions other people have not yet. And yes, you are right, some of these answers would be duplicates and easy to answer. But only to a certain point, there are a lot of answers on StackOverflow that are technically correct but severely outdated by this point. They have been superseeded by more modern ways of doing things or the APIs to do so have changed in newer versions of frameworks. With a human community you will have people leaving comments on older answers pointing this out and answering with the current answers on new questions. LLM's, again, are not capable of picking up this new information on their own. You can actually often see this in the answers you get from them, they are often ever so slightly dated already. When the communities that feed them die they will have less learning material so at the very least they will stagnate.

lifeisstillgood 2y ago

>>> 98% of collected data was rejected. I mean apart from wow ! 98% of the internet is shit (that’s higher than even I assumed) - how did they differentiate between good and shit? PageRank? Length? I can tell the difference between good writing and bad writing (and porn and erotica) but I have to read it using my brain based LLM - how did they do it ? This leads to the whole open source, show us your training data thing

creesch 2y ago
Very cheap human labor in Africa: - https://time.com/6247678/openai-chatgpt-kenya-workers/ https://time.com/6247678/openai-chatgpt-kenya-workers/ - https://www.theguardian.com/technology/2024/apr/16/techscape-ai-gadgest-humane-ai-pin-chatgpt https://www.theguardian.com/technology/2024/apr/16/techscape... Another hidden aspect of the current LLM landscape.
Mashimo 2y ago
The article says 90% of collected data. Not 98% of all internet. > discard as much as 90% of the data they initially collect for training models.

mathw 2y ago

Surely the only way you overcome this is to write a system which can actually generate something new, rather than regurgitate an incredibly complicated statistical reworking of things it got trained on. I did note how the author said that training models on other models doesn't have the ethical implications of training on stolen human data - except it does, because where did the first model get its training set from? This is why we make it illegal not just to steal but also to handle stolen goods.

DrScientist 2y ago

In the human space there is also 'model collapse' as demonstrated by group-think, or the dark arts of marketing/PR to sway opinion in the large. This is why the scientific method was developed. If you want an AI that truly learns then it has to get up of the sofa, and test the ideas in the real world, rather than learning to parrot hearsay.

tho234i2342334 2y ago
The "scientific method" doesn't fix anymore than praying to Isis does. It's not the way science is done any way.
- DrScientist 2y ago
  Don't understand. How does doing real world experiments to test your hypothesis not help with the very common problem of being wrong? > It's not the way science is done any way. That's a very sweeping statement - especially since the scientific method is quite a catch-all phrase. In the end science is driven forward by new data, gathered either intentionally or incidentally. Self-driving cars can continue to learn because they are continually generating more data. Advertising algorithms can continue to learn and adapt as the campaigns they create are in effect experiments - and some are even well designed. I suppose large language models can continue to learn by chatting to humans - but they are potentially only learning how to please humans, not necessarily underlying truths.

ColinWright 2y ago

People flooding the 'net with LLM-generated crap are both eating their seed corn and poisoning the well. Steel manufactured before 1944[0] can be incredibly valuable[1] because it hasn't been tainted by the nuclear tests/bombs fallout, and maybe fairly soon any archive internet material written pre-2010 will be considered equally valuable. [0] https://en.wikipedia.org/wiki/Low-background_steel https://en.wikipedia.org/wiki/Low-background_steel [1] https://interestingengineering.com/science/what-is-pre-war-steel https://interestingengineering.com/science/what-is-pre-war-s...

seydor 2y ago
I guess peopoe thought the same when photocopies were invented. Unlike steel, language is a human product
- cinntaile 2y ago
  Steel is also a human product.
- omnicognate 2y ago
  Not sure what point this is trying to make, but steel is a man-made alloy, not a naturally occurring substance (except possibly in tiny quantities by chance). The background radiation gets into it through the use of atmospheric air in the production process, so it seems rather a good analogy for background AI output entering the production process of internet "content" and rendering it less suitable for certain purposes.
  - seydor 2y ago
    Language is entirely human made , and it s affected by both communication and the medium. Every time a new mode of communication was invented, language was polluted with other people's language. AI is both a medium and communication which indirectly communicates using the "medial value content" of all other people on the internet. That content was itself already contaminated via too much communication over the past 30 years. So it's not really a fundamental shift , rather an acceleration of a process that s already happening

imtringued 2y ago

We don't need "more" data. We already have all the data we need in terms of quantity. We don't need more "synthetic" data for supervised learning. We need better training algorithms that go beyond minimizing token level loss.

openrisk 2y ago

We can speculate all day long how the current "AI" phenomenon will evolve but, alas, there isn't much solid on which to ground arguments. In the oral phase of human communication "AI models" were residing within human brains: the origin of any stream of messages was literally in front of you, in flesh and blood. In the print phase we have the first major decoupling. Whether in the form of a cuneiform tablet or a modern paperback, the provenance was no longer assured. We had to rely on "controlled" seals, trust the publishers and their distribution chains etc. Ultimately this worked because the relative difficulty of producing printed artifacts helped developed a legal/political apparatus to control the spread of the "fake" stuff. Enter the digital communications era and we have the second major decoupling. The amount (and increasingly the apparent veracity) of generating human oriented messaging is no longer a limiting factor. This has no precedent. You can create now plausibly create a fake Wikipedia [1] by just running a model. The signal-to-noise of digitally exchanged messages can experience catastrophic collapse. > A flood of synthetic content might not pose an existential threat to the progress of AI development, but it does threaten the digital public good of the (human) internet. Indeed, this is the real risk. I don't care about idiot "AI" feeding on itself but I do care about destroying any basis of sane digital communication between humans. Will we develop the legal/political apparatus to control the flood of algorithmic junk? Will it be remotely democratic? The best precedent of digital automation destroying communication channels and our apparent inability and/or unwillingness to do something about it is email spam [2]. Decades after it first appeared spam is still degrading our infosphere. The writing is on the wall. [1] https://en.wikipedia.org/wiki/Wikipedia:List_of_hoaxes_on_Wikipedia https://en.wikipedia.org/wiki/Wikipedia:List_of_hoaxes_on_Wi... [1] https://en.wikipedia.org/wiki/Email_spam https://en.wikipedia.org/wiki/Email_spam

pjc50 2y ago

See also https://www.alexirpan.com/2024/07/08/tragedies-of-reality.html https://www.alexirpan.com/2024/07/08/tragedies-of-reality.ht...