5 ms·

This is something I'm trying to get a better understanding. What some have called the AI hallucination problem. Has anyone done a good deep dive and explained

This is something I'm trying to get a better understanding. What some have called the AI hallucination problem. Has anyone done a good deep dive and explained whether this is something likely overcome relatively soon or is it more problematic as part of the design/architecture ?

williamcotton 4y ago
These models are very good at taking an analytic prompt and translating into a correct response. The problems of “hallucinations” are already being solved with a good number of analytic augmentation methods like pre-computing document embedding and augmenting the prompt with text from related documents, making the task more of a translation than a synthesis. More details: https://www.williamcotton.com/articles/chatgpt-and-the-analytic-synthetic-distinction https://www.williamcotton.com/articles/chatgpt-and-the-analy...
- MacsHeadroom 4y ago
  None of these methods solve the potentially catastrophic problems created by "trust and safety" teams neutering these models. Try asking chatGPT to summarize an article about a specific instance of violence like police use of force which resulted in a death and watch it confabulate and dodge regardless of how much source material you give it to work with, simply because it is afraid of touching "unsafe" topics like murder.
  - 13years 4y ago
    Yes, this is the problem of the idea we can create an impartial unbiased AI. It will only lead to a specific lens of bias. We only have biased observers to monitor and tweak the system. Attempts to counter the bias that already exists in information will only result in another layer of different bias. Ironically, we have two major issues with AI. One in which it doesn't function as intended, which leads to the problems you are describing. However, even in the conditions of working as desired, it still leads to a lot of perplexing philosophical issues for society. I've spent a lot of time trying to work through all of this and putting some thoughts into writing. So far, it has been a lot to contemplate. https://dakara.substack.com/p/ai-and-the-end-to-all-things https://dakara.substack.com/p/ai-and-the-end-to-all-things
jqpabc123 4y ago
Personal opinion --- it is an architectural issue deeply embedded in the "computer" itself. A computer as we know it is a binary logic playback device. This is an immensely useful tool but expecting real, original "intelligence" from a box of silicon switches is the modern equivalent of alchemy.
- thfuran 4y ago
  A brain is just a sack of meat stochastically nudged towards only dying after more reproduction. Expecting real, original intelligence from something like that is the modern equivalent of alchemy.
  - jqpabc123 4y ago
    Maybe I'm giving your brain too much credit but generating your post demonstrated more real intelligence than AI ever has. And yet at the same time, you have "faith" that what your brain does can be duplicated by a box of sand. And "faith" is the proper description since just like an alchemist, you have no evidence or functioning example to justify your belief.
    - thfuran 4y ago
      You seem to have faith that a mountain is impossible to build because no one has done it. Assuming we're operating with a definition of "real intelligence" such that humans have it, it is known to be physically realizable. Any quantum system can be simulated by a classical computer. Ergo, a computer can have "real intelligence".
      - jqpabc123 4y ago
        Any quantum system can be simulated by a classical computer. Really? How would a classical computer create an emergent property like entanglement? What if your brain function depends on such emergent properties? There are too many unknowns associated with "intelligence" to reasonably declare that it can be created from a box of sand --- aka, a "computer" as we know it.

williamcotton 4y ago

These models are very good at taking an analytic prompt and translating into a correct response. The problems of “hallucinations” are already being solved with a good number of analytic augmentation methods like pre-computing document embedding and augmenting the prompt with text from related documents, making the task more of a translation than a synthesis. More details: https://www.williamcotton.com/articles/chatgpt-and-the-analytic-synthetic-distinction https://www.williamcotton.com/articles/chatgpt-and-the-analy...

MacsHeadroom 4y ago
None of these methods solve the potentially catastrophic problems created by "trust and safety" teams neutering these models. Try asking chatGPT to summarize an article about a specific instance of violence like police use of force which resulted in a death and watch it confabulate and dodge regardless of how much source material you give it to work with, simply because it is afraid of touching "unsafe" topics like murder.
- 13years 4y ago
  Yes, this is the problem of the idea we can create an impartial unbiased AI. It will only lead to a specific lens of bias. We only have biased observers to monitor and tweak the system. Attempts to counter the bias that already exists in information will only result in another layer of different bias. Ironically, we have two major issues with AI. One in which it doesn't function as intended, which leads to the problems you are describing. However, even in the conditions of working as desired, it still leads to a lot of perplexing philosophical issues for society. I've spent a lot of time trying to work through all of this and putting some thoughts into writing. So far, it has been a lot to contemplate. https://dakara.substack.com/p/ai-and-the-end-to-all-things https://dakara.substack.com/p/ai-and-the-end-to-all-things

jqpabc123 4y ago

Personal opinion --- it is an architectural issue deeply embedded in the "computer" itself. A computer as we know it is a binary logic playback device. This is an immensely useful tool but expecting real, original "intelligence" from a box of silicon switches is the modern equivalent of alchemy.

thfuran 4y ago
A brain is just a sack of meat stochastically nudged towards only dying after more reproduction. Expecting real, original intelligence from something like that is the modern equivalent of alchemy.
- jqpabc123 4y ago
  Maybe I'm giving your brain too much credit but generating your post demonstrated more real intelligence than AI ever has. And yet at the same time, you have "faith" that what your brain does can be duplicated by a box of sand. And "faith" is the proper description since just like an alchemist, you have no evidence or functioning example to justify your belief.
  - thfuran 4y ago
    You seem to have faith that a mountain is impossible to build because no one has done it. Assuming we're operating with a definition of "real intelligence" such that humans have it, it is known to be physically realizable. Any quantum system can be simulated by a classical computer. Ergo, a computer can have "real intelligence".
    - jqpabc123 4y ago
      Any quantum system can be simulated by a classical computer. Really? How would a classical computer create an emergent property like entanglement? What if your brain function depends on such emergent properties? There are too many unknowns associated with "intelligence" to reasonably declare that it can be created from a box of sand --- aka, a "computer" as we know it.