8 ms·

> Yes, very good point. Self-driving maximalists who believe that self-driving will be solved with more data need to realize that ChatGPT was trained with ALL t

> Yes, very good point. Self-driving maximalists who believe that self-driving will be solved with more data need to realize that ChatGPT was trained with ALL the data possible and is still deficient. This defect is probably inherent to existing neural net models and a leap forward of some sort is necessary to solve this. This is the thing that bugs me about ChatGPT4 which everyone says is a lot better. Did they fix the underlying issues or does it just have more data? If it's the latter, that means if it's force to operate outside of its "domain" it's going to produce rubbish again - and heaven knows where the limits of its current domain are. These AIs need to not catastrophically fail if they are missing information. IMHO in order for AI to be truly useful, we need to be able to trust it. I can't trust something that produces rubbish wherever it's out of its depth instead of just saying "I don't know."

weaksauce 3y ago
> IMHO in order for AI to be truly useful, we need to be able to trust it. I can't trust something that produces rubbish wherever it's out of its depth instead of just saying "I don't know." I wholeheartedly agree. what we have now is a very capable and convincing liar.
- deleted 3y ago
  [deleted]
- ants_everywhere 3y ago
  > what we have now is a very capable and convincing liar. I think things might get even wilder once companies start allowing advertisers to influence chat results like they do with search. Imagine a capable and convincing liar who has an ulterior motive when it talks to you.
- zdragnar 3y ago
  It cannot tell the truth, because it does not have the context or understanding of what is true or incorrect. It is less a liar (who intends to mislead) and instead a fantastic bullshitter who just talks and sounds convincing.
ballenf 3y ago
If AI "lies" less than the top Google hit on the prompt, then it's progress.
- debaserab2 3y ago
  Google doesn’t really “lie” though, it gives you the source and allows you to make a decision about its authenticity instead of masking it.
  - noduerme 3y ago
    Moreover, Google doesn't cite false sources or obfuscate what link you're visiting, or claim a page says something it doesn't.
    - eppp 3y ago
      You forgot the sarcasm tag.
- dzamo_norton 3y ago
  We get multiple hits from Google (though not always ranked by merit). We can scan a few and we often find forum style threads containing valuable elaboration or criticism of the primary material.
FractalHQ 3y ago
I find GPT-4 to be very useful almost daily. I can often spot hallucinations quickly, and they are otherwise easy enough to verify. If I can get a single new perspective or piece of relevant information from an interaction with it, then that is very valuable. It would be significantly more useful if it were more grounded in reality though… I agree with you there.
- sanderjd 3y ago
  I think it's enormously useful as a tool paired with a human who has decent judgment. I think it would be useless on its own. I'm constantly impressed by how useful it is, but I'm also constantly mystified by people who claim to be getting this feeling of talking to a "real" intelligence; it doesn't feel that way to me at all.
- JasonFruit 3y ago
  How do you know you spot the hallucinations, and that you're not just catching the less-good ones while accepting convincing half-truths? It may be that your subject is just that clear-cut, and you've been careful — but what I worry about is that people won't be, and will just accept the pretty-much correct details that don't really matter that much, until they accrete into a mass of false knowledge, like the authoritative errors quoted in Isadore of Seville's Encyclopedia and similar medieval works.
- grumpyprole 3y ago
  On the contrary, the "hallucinations" are often very hard to spot without expert knowledge. The output is often plausible but wrong, as shown by Knuth's questions.
majormajor 3y ago
I used GPT-4 for an interview problem from leetcode out of curiosity. It got it right, very quickly, yay! Then I asked it to modify it by eliminating one of the constraints on the problem. It did a very convincing "Ah, if we need [that] we need to do [this] and output a new version... that didn't actually work right. I pointed out the specific edge case, it said "you are correct, for that sort of case we have to modify it" and then spit out exactly the same code as the last attempt. The most interesting thing to me there isn't that it got it wrong - it's that spitting out exactly the same output without realizing it, while saying that you are going to do something different, is the clearest demonstration I've seen from it that it doesn't "understand" in human-like ways. Extremely powerful and useful, but VERY important for users to know where it runs into the wall. Since it often won't tell you on its own.
- altmind 3y ago
  Cahtgpt is quite good for known problems before 2022, since the questions got into the training set. It's quite bad for new interview questions though.
- dotancohen 3y ago
  These models are designed to produce a _plausible_ text output for a given prompt. Nothing more. They are not designed to produce a _correct_ text output to a question or request, even if sometimes the output is correct. These proverbial stopped clocks might be correct more than twice a day, but that's just the huge training set speaking.
  - drdeca 3y ago
    Are you taking the RLHF into account when you say so?
    - alex_sf 3y ago
      Taking RLHF into account: it's not actually generating the most plausible completion, it's generating one that's worse.
    - 3y ago
PeterisP 3y ago
> Did they fix the underlying issues or does it just have more data? IIRC they do have slightly more data, but that's not the primary cause of improvement, the key factor is simply more parameters and more training. No significant actions have been taken "fix the underlying issues" - you should assume that any major differences between GPT-2 (which is horrible in comparison to GPT-3) and GPT-4 are emergent behavior from the model having more horsepower.
bodge5000 3y ago
> IMHO in order for AI to be truly useful, we need to be able to trust it. A common response to this by AI advocates is to point out that humans lie all the time, as long as the AI lies less than humans (debatable at this current point anyway) its an improvement. I think what that forgets is the importance of context. We all know humans are perfectly capable of lying, but we don't generally expect that of software. If your compiler lied about your code being valid, I doubt the general response would be "meh, its only done that once, I've lied far more than that"
- truthreplicator 3y ago
  I suspect he posited trust in juxtaposition to reliability, rather than veracity.
- cageface 3y ago
  The other difference is that over time we build up a network of people we consider to be knowledgeable and honest. Current LLMs can never match that because their output is controlled guessing.
- woeirua 3y ago
  More importantly humans have ways to detect deception from other humans, be it through body language or other cues. With only text it is very hard to determine whether the model is lying to you or not.
  - thayne 3y ago
    Even in text, there is more context. For example, I am more likely to trust the wikipedia article about a deeply technical topic than an article about politics or a celebrity, because the technical article is far more likely to only be edited by people who are actually very knowledgeable on the topic, and there is very little incentive to lie (in general, there are exceptions).
- dreamcompiler 3y ago
  > A common response to this by AI advocates is to point out that humans lie all the time, as long as the AI lies less than humans (debatable at this current point anyway) its an improvement. This is also Elon Musk's justification for self-driving cars: "They make fewer mistakes than humans and are therefore safer." It's true that self-driving cars avoid many of the mistakes of human drivers, but they also invent whole new categories of fatal mistakes that humans rarely make. And that's why Musk's argument is garbage.
nullsense 3y ago
Unfortunately trusting something with capabilities that generalize isn't an easy thing to do.
cageface 3y ago
I've been thinking about this lately and it seems to me that what these models are very good at is generating text that has the right structure, but of all the permutations with the right structure only a few actually contain useful and correct information and it only hits on those by chance. And, since the real value in communication is the information contained, that puts a fairly low ceiling on the value of their output. If it can't be trusted without careful review by someone that really understands the subject and can flag mistakes then it can never truly replace people in any role where correctness matters and that's most of the roles with a lot of economic value.
- int_19h 3y ago
  If that were the case, outputs would be consistently nonsense - the number of possible variations of text like "colorless green ideas sleep furiously" is so much larger than the meaningful subset, the probability of hitting the latter by chance would be zero for all practical purposes.
  - cageface 3y ago
    Only if the words were chosen simply at random in sequence and of course they're not this simplistic. They're constrained by the attention models so they do much better than this but they're still random. You can control the degree of randomness with the temperature knob.
    - int_19h 3y ago
      This part about "constrained by the attention model" is doing a lot of implicit work here to dodge the question why GPT-4 can verifiably reason about things in text.
      - cageface 3y ago
        It also demonstrably is either flat out wrong about a lot of things or completely invents things that don't exist. It's a random process that sometimes generates content with actual informational value but the randomness is inherent in the algorithm.
3np 3y ago
> IMHO in order for AI to be truly useful, we need to be able to trust it Disagree, but perhaps we have different ideas of "useful". I think automated systems including AI can be very useful but that executive decisions yielded by nondeterministic processes (such as AI) must be signed off by a human and that usage should be mindful of inherent limitations. This includes cross-checking factual claims with sources and verifying produced code works - just as you would (I hope) with a forum comment or Stackoverflow answer before publishing it as fact or pushing it to production. So I'd rather say: In order for AI to be truly useful, we need to be able to work with it with never trusting it. Let go of unsupervised execution.

weaksauce 3y ago

> IMHO in order for AI to be truly useful, we need to be able to trust it. I can't trust something that produces rubbish wherever it's out of its depth instead of just saying "I don't know." I wholeheartedly agree. what we have now is a very capable and convincing liar.

deleted 3y ago
[deleted]
ants_everywhere 3y ago
> what we have now is a very capable and convincing liar. I think things might get even wilder once companies start allowing advertisers to influence chat results like they do with search. Imagine a capable and convincing liar who has an ulterior motive when it talks to you.
zdragnar 3y ago
It cannot tell the truth, because it does not have the context or understanding of what is true or incorrect. It is less a liar (who intends to mislead) and instead a fantastic bullshitter who just talks and sounds convincing.

ballenf 3y ago

If AI "lies" less than the top Google hit on the prompt, then it's progress.

debaserab2 3y ago
Google doesn’t really “lie” though, it gives you the source and allows you to make a decision about its authenticity instead of masking it.
- noduerme 3y ago
  Moreover, Google doesn't cite false sources or obfuscate what link you're visiting, or claim a page says something it doesn't.
  - eppp 3y ago
    You forgot the sarcasm tag.
dzamo_norton 3y ago
We get multiple hits from Google (though not always ranked by merit). We can scan a few and we often find forum style threads containing valuable elaboration or criticism of the primary material.

FractalHQ 3y ago

I find GPT-4 to be very useful almost daily. I can often spot hallucinations quickly, and they are otherwise easy enough to verify. If I can get a single new perspective or piece of relevant information from an interaction with it, then that is very valuable. It would be significantly more useful if it were more grounded in reality though… I agree with you there.

sanderjd 3y ago
I think it's enormously useful as a tool paired with a human who has decent judgment. I think it would be useless on its own. I'm constantly impressed by how useful it is, but I'm also constantly mystified by people who claim to be getting this feeling of talking to a "real" intelligence; it doesn't feel that way to me at all.
JasonFruit 3y ago
How do you know you spot the hallucinations, and that you're not just catching the less-good ones while accepting convincing half-truths? It may be that your subject is just that clear-cut, and you've been careful — but what I worry about is that people won't be, and will just accept the pretty-much correct details that don't really matter that much, until they accrete into a mass of false knowledge, like the authoritative errors quoted in Isadore of Seville's Encyclopedia and similar medieval works.
grumpyprole 3y ago
On the contrary, the "hallucinations" are often very hard to spot without expert knowledge. The output is often plausible but wrong, as shown by Knuth's questions.

majormajor 3y ago

I used GPT-4 for an interview problem from leetcode out of curiosity. It got it right, very quickly, yay! Then I asked it to modify it by eliminating one of the constraints on the problem. It did a very convincing "Ah, if we need [that] we need to do [this] and output a new version... that didn't actually work right. I pointed out the specific edge case, it said "you are correct, for that sort of case we have to modify it" and then spit out exactly the same code as the last attempt. The most interesting thing to me there isn't that it got it wrong - it's that spitting out exactly the same output without realizing it, while saying that you are going to do something different, is the clearest demonstration I've seen from it that it doesn't "understand" in human-like ways. Extremely powerful and useful, but VERY important for users to know where it runs into the wall. Since it often won't tell you on its own.

altmind 3y ago
Cahtgpt is quite good for known problems before 2022, since the questions got into the training set. It's quite bad for new interview questions though.
dotancohen 3y ago
These models are designed to produce a _plausible_ text output for a given prompt. Nothing more. They are not designed to produce a _correct_ text output to a question or request, even if sometimes the output is correct. These proverbial stopped clocks might be correct more than twice a day, but that's just the huge training set speaking.
- drdeca 3y ago
  Are you taking the RLHF into account when you say so?
  - alex_sf 3y ago
    Taking RLHF into account: it's not actually generating the most plausible completion, it's generating one that's worse.
  - 3y ago

PeterisP 3y ago

> Did they fix the underlying issues or does it just have more data? IIRC they do have slightly more data, but that's not the primary cause of improvement, the key factor is simply more parameters and more training. No significant actions have been taken "fix the underlying issues" - you should assume that any major differences between GPT-2 (which is horrible in comparison to GPT-3) and GPT-4 are emergent behavior from the model having more horsepower.

bodge5000 3y ago

> IMHO in order for AI to be truly useful, we need to be able to trust it. A common response to this by AI advocates is to point out that humans lie all the time, as long as the AI lies less than humans (debatable at this current point anyway) its an improvement. I think what that forgets is the importance of context. We all know humans are perfectly capable of lying, but we don't generally expect that of software. If your compiler lied about your code being valid, I doubt the general response would be "meh, its only done that once, I've lied far more than that"

truthreplicator 3y ago
I suspect he posited trust in juxtaposition to reliability, rather than veracity.
cageface 3y ago
The other difference is that over time we build up a network of people we consider to be knowledgeable and honest. Current LLMs can never match that because their output is controlled guessing.
woeirua 3y ago
More importantly humans have ways to detect deception from other humans, be it through body language or other cues. With only text it is very hard to determine whether the model is lying to you or not.
- thayne 3y ago
  Even in text, there is more context. For example, I am more likely to trust the wikipedia article about a deeply technical topic than an article about politics or a celebrity, because the technical article is far more likely to only be edited by people who are actually very knowledgeable on the topic, and there is very little incentive to lie (in general, there are exceptions).
dreamcompiler 3y ago
> A common response to this by AI advocates is to point out that humans lie all the time, as long as the AI lies less than humans (debatable at this current point anyway) its an improvement. This is also Elon Musk's justification for self-driving cars: "They make fewer mistakes than humans and are therefore safer." It's true that self-driving cars avoid many of the mistakes of human drivers, but they also invent whole new categories of fatal mistakes that humans rarely make. And that's why Musk's argument is garbage.

nullsense 3y ago

Unfortunately trusting something with capabilities that generalize isn't an easy thing to do.

cageface 3y ago

I've been thinking about this lately and it seems to me that what these models are very good at is generating text that has the right structure, but of all the permutations with the right structure only a few actually contain useful and correct information and it only hits on those by chance. And, since the real value in communication is the information contained, that puts a fairly low ceiling on the value of their output. If it can't be trusted without careful review by someone that really understands the subject and can flag mistakes then it can never truly replace people in any role where correctness matters and that's most of the roles with a lot of economic value.

int_19h 3y ago
If that were the case, outputs would be consistently nonsense - the number of possible variations of text like "colorless green ideas sleep furiously" is so much larger than the meaningful subset, the probability of hitting the latter by chance would be zero for all practical purposes.
- cageface 3y ago
  Only if the words were chosen simply at random in sequence and of course they're not this simplistic. They're constrained by the attention models so they do much better than this but they're still random. You can control the degree of randomness with the temperature knob.
  - int_19h 3y ago
    This part about "constrained by the attention model" is doing a lot of implicit work here to dodge the question why GPT-4 can verifiably reason about things in text.
    - cageface 3y ago
      It also demonstrably is either flat out wrong about a lot of things or completely invents things that don't exist. It's a random process that sometimes generates content with actual informational value but the randomness is inherent in the algorithm.

3np 3y ago

> IMHO in order for AI to be truly useful, we need to be able to trust it Disagree, but perhaps we have different ideas of "useful". I think automated systems including AI can be very useful but that executive decisions yielded by nondeterministic processes (such as AI) must be signed off by a human and that usage should be mindful of inherent limitations. This includes cross-checking factual claims with sources and verifying produced code works - just as you would (I hope) with a forum comment or Stackoverflow answer before publishing it as fact or pushing it to production. So I'd rather say: In order for AI to be truly useful, we need to be able to work with it with never trusting it. Let go of unsupervised execution.