3 ms·

> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again This would be a world without generative AI ava

by nomel 1y ago

> in a sane world the folks that run ChatGPT would be falling over themselves to be sure it didn’t happen again This would be a world without generative AI available to the public, at the moment. Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists, which are both completely irrational, since many people are finding practical value in its current imperfect state. The current state of LLM is useful for what it's useful for, warnings of hallucinations are present on every official public interface, and its limitations are quickly understood with any real use. Nearly everyone in AI research is working on this problem, directly or indirectly.

gortok 1y ago
No one is “requiring perfection”, but hallucination is a major issue and is in the opposite direction of the “goal” of AGI. If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.
- nomel 1y ago
  > No one is “requiring perfection” > If “don’t hallucinate” is too much to ask then ethics flew out the window long ago. Those sentences aren't compatible. > but hallucination is a major issue Again, every official public AI interface has warnings/disclaimers for this issue. It's well known. It's not some secret. Every AI researcher is directly or indirectly working on this. > is in the opposite direction of the “goal” of AGI This isn't a logical statement, so it's difficult to respond to. Hallucination isn't a direction that's being headed towards, it's being actively, with intent and $$$, headed away from.
  - lucianbr 1y ago
    > Those sentences aren't compatible. My web browser isn't perfect, but it does not hallucinate inexistent webpages. It sometimes crashes, it sometimes renders wrong, it has bugs and errors. It does not invent plausible-looking information. There really is a lot middle gound between perfect and "accept anything we give you, no matter how huge the problems".
    - jeffhuys 1y ago
      Different tech, different failure modes. > it sometimes renders wrong Is close to equivalent.
    - tucnak 1y ago
      > It does not invent plausible-looking information. This is where your analogy is falling apart; of course web browsers do not "invent plausible-looking information" because they don't invent anything in the first place! Web browsers represent a distinct set of capabilities, and as you correctly pointed out, these are often riddled with bugs and errors. If I was making a browser analogy, I would point towards fingerprinting; most browsers reveal too much information about any given user and system, either via cross-site cookies, GPU prints, and whatnot. This is an actual example where "ethics flew out the window long ago." As the adjacent commenter pointed out: different software, different failure modes.
Velorivox 1y ago
> which are both completely irrational Really!? [0] https://i.imgur.com/ly5yk9h.png https://i.imgur.com/ly5yk9h.png
- tucnak 1y ago
  Your screenshot is conveniently omitting the disclaimer below: "AI responses may include mistakes. Learn more[1]" [1]: https://support.google.com/websearch/answer/14901683 https://support.google.com/websearch/answer/14901683
  - Velorivox 1y ago
    It isn't doing anything "conveniently", I was not shown the disclaimer (nor anything else, I assume it mostly failed to load). In any case, if you really believe a disclaimer makes it okay for Google to display blatant misinformation in a first-party capacity, we have little to discuss.
    - tucnak 1y ago
      https://www.google.com/search?q=is+all+of+oregon+north+of+new+york+city https://www.google.com/search?q=is+all+of+oregon+north+of+ne... Show more -> Disclaimer and the feedback buttons are shown at the end. If you bothered enough to read the full response, you would have seen the disclaimer, but you never did, so you haven't. For something to be considered "misinformation," in the very least the subject of speech has to be asserting its truthfulness—and, indeed, Google makes no such claims. The claim they're making is precisely that its search result-embedded "[..] responses may include mistakes." In this specific case, they are not asserting truthfulness. FWIW, Gemini 2.5 Pro answers the question correctly. The search hints are clearly a low-compute first approximation, which is probably correct for most trivial questions which is probably the majority of user queries, and it's not surprising that it fails in this specific instance. The application doesn't allow for reasoning due to scale; even Google cannot afford to run reasoning traces on every search question. I concur that there's very little to discuss: you seemingly made up your mind re: LLM technology, and I doubt you will appreciate the breaking-up of your semantics to begin with.
epidemian 1y ago
> Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists What?? What does AGI have to do with this? (If this was some kind of hyperbolic joke, sorry, i didn't get it.) But, more importantly, the GP only said that in a sane world, the ChatGPT creators should be the ones trying to fix this mistake on ChatGPT. After all, it's obviously a mistake on ChatGPT's part, right? That was the main point of the GP post. It was not about "requiring perfection" or something like that. So please let's not attack a straw man.
- nomel 1y ago
  > What does AGI have to do with this? Their requirement is no hallucinations [1], also stated as "be sure it didn't happen again" in the original comment. If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then you've placed a profound constraint on the system, requiring determinism. That requirement fundamentally, by the non-deterministic statistics that these run on, means you cannot use an LLM, as they exist today. They're not "truth" machines - use a database instead. Saying "I don't know", with determinism is only slightly different than saying "I know" with determinism, since it requires being fully aware of what you do know, not at a fact level, but at a conceptual/abstract level. Once you have a system that fully reasons about concepts, is self aware of its own knowledge, and can find the fundamental "truth" to answer a question with determinism, you have something indistinguishable from AGI. Of course, there's a terrible hell that lives between those two, in the form of: "Error: Question outside of known questions." I think a better alternative to this hell would be a breakthrough that allowed "confidence" to be quantified. So, accept that hallucinations will exist, but present uncertainty to the user. [1] https://news.ycombinator.com/item?id=44496098 https://news.ycombinator.com/item?id=44496098
  - penteract 1y ago
    You have a very strong definition of AGI. "Never being wrong" is something that humans fall far short of.

by nomel 1y ago

gortok 1y ago

No one is “requiring perfection”, but hallucination is a major issue and is in the opposite direction of the “goal” of AGI. If “don’t hallucinate” is too much to ask then ethics flew out the window long ago.

nomel 1y ago
> No one is “requiring perfection” > If “don’t hallucinate” is too much to ask then ethics flew out the window long ago. Those sentences aren't compatible. > but hallucination is a major issue Again, every official public AI interface has warnings/disclaimers for this issue. It's well known. It's not some secret. Every AI researcher is directly or indirectly working on this. > is in the opposite direction of the “goal” of AGI This isn't a logical statement, so it's difficult to respond to. Hallucination isn't a direction that's being headed towards, it's being actively, with intent and $$$, headed away from.
- lucianbr 1y ago
  > Those sentences aren't compatible. My web browser isn't perfect, but it does not hallucinate inexistent webpages. It sometimes crashes, it sometimes renders wrong, it has bugs and errors. It does not invent plausible-looking information. There really is a lot middle gound between perfect and "accept anything we give you, no matter how huge the problems".
  - jeffhuys 1y ago
    Different tech, different failure modes. > it sometimes renders wrong Is close to equivalent.
  - tucnak 1y ago
    > It does not invent plausible-looking information. This is where your analogy is falling apart; of course web browsers do not "invent plausible-looking information" because they don't invent anything in the first place! Web browsers represent a distinct set of capabilities, and as you correctly pointed out, these are often riddled with bugs and errors. If I was making a browser analogy, I would point towards fingerprinting; most browsers reveal too much information about any given user and system, either via cross-site cookies, GPU prints, and whatnot. This is an actual example where "ethics flew out the window long ago." As the adjacent commenter pointed out: different software, different failure modes.

Velorivox 1y ago

> which are both completely irrational Really!? [0] https://i.imgur.com/ly5yk9h.png https://i.imgur.com/ly5yk9h.png

tucnak 1y ago
Your screenshot is conveniently omitting the disclaimer below: "AI responses may include mistakes. Learn more[1]" [1]: https://support.google.com/websearch/answer/14901683 https://support.google.com/websearch/answer/14901683
- Velorivox 1y ago
  It isn't doing anything "conveniently", I was not shown the disclaimer (nor anything else, I assume it mostly failed to load). In any case, if you really believe a disclaimer makes it okay for Google to display blatant misinformation in a first-party capacity, we have little to discuss.
  - tucnak 1y ago
    https://www.google.com/search?q=is+all+of+oregon+north+of+new+york+city https://www.google.com/search?q=is+all+of+oregon+north+of+ne... Show more -> Disclaimer and the feedback buttons are shown at the end. If you bothered enough to read the full response, you would have seen the disclaimer, but you never did, so you haven't. For something to be considered "misinformation," in the very least the subject of speech has to be asserting its truthfulness—and, indeed, Google makes no such claims. The claim they're making is precisely that its search result-embedded "[..] responses may include mistakes." In this specific case, they are not asserting truthfulness. FWIW, Gemini 2.5 Pro answers the question correctly. The search hints are clearly a low-compute first approximation, which is probably correct for most trivial questions which is probably the majority of user queries, and it's not surprising that it fails in this specific instance. The application doesn't allow for reasoning due to scale; even Google cannot afford to run reasoning traces on every search question. I concur that there's very little to discuss: you seemingly made up your mind re: LLM technology, and I doubt you will appreciate the breaking-up of your semantics to begin with.

epidemian 1y ago

> Requiring perfection would either mean guardrails that would make it useless for most cases, or no LLM access until AGI exists What?? What does AGI have to do with this? (If this was some kind of hyperbolic joke, sorry, i didn't get it.) But, more importantly, the GP only said that in a sane world, the ChatGPT creators should be the ones trying to fix this mistake on ChatGPT. After all, it's obviously a mistake on ChatGPT's part, right? That was the main point of the GP post. It was not about "requiring perfection" or something like that. So please let's not attack a straw man.

nomel 1y ago
> What does AGI have to do with this? Their requirement is no hallucinations [1], also stated as "be sure it didn't happen again" in the original comment. If you define a hallucination as something that wasn't in the training data, directly or indirectly (indirectly being something like an "obvious" abstract concept), then you've placed a profound constraint on the system, requiring determinism. That requirement fundamentally, by the non-deterministic statistics that these run on, means you cannot use an LLM, as they exist today. They're not "truth" machines - use a database instead. Saying "I don't know", with determinism is only slightly different than saying "I know" with determinism, since it requires being fully aware of what you do know, not at a fact level, but at a conceptual/abstract level. Once you have a system that fully reasons about concepts, is self aware of its own knowledge, and can find the fundamental "truth" to answer a question with determinism, you have something indistinguishable from AGI. Of course, there's a terrible hell that lives between those two, in the form of: "Error: Question outside of known questions." I think a better alternative to this hell would be a breakthrough that allowed "confidence" to be quantified. So, accept that hallucinations will exist, but present uncertainty to the user. [1] https://news.ycombinator.com/item?id=44496098 https://news.ycombinator.com/item?id=44496098
- penteract 1y ago
  You have a very strong definition of AGI. "Never being wrong" is something that humans fall far short of.