3 ms·

> They are not the answers. You can't possibly know that.

by p-e-w 4y ago

chx 4y ago
I can. Even if the answers accidentally happen to be correct, that's just the broken clock happens to be correct twice a day. The information value of answers by ChatGPT is zero.
- simonw 4y ago
  It sounds to me like you're letting your firmly held existing biases get in the way of you learning a powerful new tool. As I said in https://simonwillison.net/2023/Apr/8/llms-break-the-internet/ https://simonwillison.net/2023/Apr/8/llms-break-the-internet... > This is the thing I worry that people are sleeping on. People who think “these language models lie to you all the time” (which they do) and “they will produce buggy code with security holes”—every single complaint about these things is true, and yet, despite all of that, the productivity benefits you get if you lean into them and say OK, how do I work with something that’s completely unreliable, that invents things, that comes up with APIs that don’t exist… how do I use that to enhance my workflow anyway?
  - chx 4y ago
    That large language models are stochastic parrots is not a bias, it's a fact. You feed it a very large amount of structured data and then put up a question which it'll answer with a series of those which are most likely to follow. There is nothing else.
    - tempestn 4y ago
      Yeah. And in doing so, it will usually give you the correct answers.
    - simonw 4y ago
      I know how they work. What's so interesting to me about them is how useful they turn out to be despite the seemingly dumb and simple way they are built. I encourage you to experiment more! Here are my notes so far: https://simonwillison.net/tags/llms/ https://simonwillison.net/tags/llms/
    - p-e-w 4y ago
      Do you have any evidence that human minds work substantially different from that?
      - chx 4y ago
        Yes. Human minds often craft sentences which contain words together which never occurred in the same sentence before, not even close: > There was a boy called Eustace Clarence Scrubb, and he almost deserved it > Everything starts somewhere, although many physicists disagree. > It was a nice day. All the days had been nice. There had been rather more than seven of them so far, and rain hadn't been invented yet > My father had a face that could stop a clock. > It is important, when killing a nun, to ensure that you bring an army of sufficient size. > In the myriadic year of our Lord—the ten thousandth year of the King Undying, the kindly Prince of Death!— Gideon Nav packed her sword, her shoes, and her dirty magazines, and she escaped from the House of the Ninth. There's just no end to these. And, of course, science is full of these too, one that jumps to mind is Shinichi Mochizuki's claimed proof of the ABC conjecture which has been proven flawed as it often happens but it was certainly a credible proof despite written in a language no mathematician have ever seen.
        simonw 4y ago
        "Human minds often craft sentences which contain words together which never occurred in the same sentence before, not even close" Weirdly, LLMs do that too. Have you heard of the "Let’s be bear or bunny" pattern? My iPhone hallucinated that the other day, running a LLM directly on the phone! https://simonwillison.net/2023/May/1/lets-be-bear-or-bunny/ https://simonwillison.net/2023/May/1/lets-be-bear-or-bunny/
        chx 4y ago
        > Weirdly, LLMs do that too. Yes but those are as every other thing they emit, indeed , "Let’s be bear or bunny", I heard it called AI hallucination I personally like to call it word salad.
        pulvinar 4y ago
  - nklqwe 4y ago
    I guess these development practices are widespread in the Python community. Other languages have higher standards.
- tempestn 4y ago
  It's not correct by accident. It's not generating responses randomly, but based on extensive training data. The accuracy of answers is very high, especially on uncontroversial topics like most of the examples in the OP. And the author even described cross-checking the answers for validity and confirming that it was generally correct. (I was also an electrical engineer once upon a time and nothing jumped out at me as wrong, but admittedly it's been quite a while!) You can't rely in it being 100% correct, but that's very different from it having no informational value at all. When it comes down to it, you can't rely on anything being 100% correct. I recall finding multiple errors in textbooks in the past, and certainly Wikipedia is wrong about all kinds of things; that doesn't make them useless. It just means in situations where it's critical to be correct, you need to double-check. But often that's not necessary, and when it is, it's a lot easier to start with something and then verify it than to not have the tool in the first place.
- p-e-w 4y ago
  > The information value of answers by ChatGPT is zero. Nonsense. The only way this would be true is if ChatGPT's answers were as likely to be correct as random answers. They are much, much more likely to be correct, however, so their information value is greater than zero by definition. What you're claiming is equivalent to "search engines can find incorrect information, so search engines are worthless for information retrieval". Which is bollocks.
  - chx 4y ago
    > The only way this would be true is if ChatGPT's answers were as likely to be correct as random answers. Which is the case > They are much, much more likely to be correct, They are not. It's just apophenia to find meaning in the word salad.
    - olddustytrail 4y ago
      Of course it's not the case. You do realise that people have measured this? The irony of people complaining that LLMs confidently assert things that are completely wrong...

> They are not the answers. You can't possibly know that.

by p-e-w 4y ago

> They are not the answers. You can't possibly know that.

chx 4y ago

I can. Even if the answers accidentally happen to be correct, that's just the broken clock happens to be correct twice a day. The information value of answers by ChatGPT is zero.

simonw 4y ago
It sounds to me like you're letting your firmly held existing biases get in the way of you learning a powerful new tool. As I said in https://simonwillison.net/2023/Apr/8/llms-break-the-internet/ https://simonwillison.net/2023/Apr/8/llms-break-the-internet... > This is the thing I worry that people are sleeping on. People who think “these language models lie to you all the time” (which they do) and “they will produce buggy code with security holes”—every single complaint about these things is true, and yet, despite all of that, the productivity benefits you get if you lean into them and say OK, how do I work with something that’s completely unreliable, that invents things, that comes up with APIs that don’t exist… how do I use that to enhance my workflow anyway?
- chx 4y ago
  That large language models are stochastic parrots is not a bias, it's a fact. You feed it a very large amount of structured data and then put up a question which it'll answer with a series of those which are most likely to follow. There is nothing else.
  - tempestn 4y ago
    Yeah. And in doing so, it will usually give you the correct answers.
  - simonw 4y ago
    I know how they work. What's so interesting to me about them is how useful they turn out to be despite the seemingly dumb and simple way they are built. I encourage you to experiment more! Here are my notes so far: https://simonwillison.net/tags/llms/ https://simonwillison.net/tags/llms/
  - p-e-w 4y ago
    Do you have any evidence that human minds work substantially different from that?
    - chx 4y ago
      Yes. Human minds often craft sentences which contain words together which never occurred in the same sentence before, not even close: > There was a boy called Eustace Clarence Scrubb, and he almost deserved it > Everything starts somewhere, although many physicists disagree. > It was a nice day. All the days had been nice. There had been rather more than seven of them so far, and rain hadn't been invented yet > My father had a face that could stop a clock. > It is important, when killing a nun, to ensure that you bring an army of sufficient size. > In the myriadic year of our Lord—the ten thousandth year of the King Undying, the kindly Prince of Death!— Gideon Nav packed her sword, her shoes, and her dirty magazines, and she escaped from the House of the Ninth. There's just no end to these. And, of course, science is full of these too, one that jumps to mind is Shinichi Mochizuki's claimed proof of the ABC conjecture which has been proven flawed as it often happens but it was certainly a credible proof despite written in a language no mathematician have ever seen.
      - simonw 4y ago
        "Human minds often craft sentences which contain words together which never occurred in the same sentence before, not even close" Weirdly, LLMs do that too. Have you heard of the "Let’s be bear or bunny" pattern? My iPhone hallucinated that the other day, running a LLM directly on the phone! https://simonwillison.net/2023/May/1/lets-be-bear-or-bunny/ https://simonwillison.net/2023/May/1/lets-be-bear-or-bunny/
        chx 4y ago
        > Weirdly, LLMs do that too. Yes but those are as every other thing they emit, indeed , "Let’s be bear or bunny", I heard it called AI hallucination I personally like to call it word salad.
      - pulvinar 4y ago
- nklqwe 4y ago
  I guess these development practices are widespread in the Python community. Other languages have higher standards.
tempestn 4y ago
It's not correct by accident. It's not generating responses randomly, but based on extensive training data. The accuracy of answers is very high, especially on uncontroversial topics like most of the examples in the OP. And the author even described cross-checking the answers for validity and confirming that it was generally correct. (I was also an electrical engineer once upon a time and nothing jumped out at me as wrong, but admittedly it's been quite a while!) You can't rely in it being 100% correct, but that's very different from it having no informational value at all. When it comes down to it, you can't rely on anything being 100% correct. I recall finding multiple errors in textbooks in the past, and certainly Wikipedia is wrong about all kinds of things; that doesn't make them useless. It just means in situations where it's critical to be correct, you need to double-check. But often that's not necessary, and when it is, it's a lot easier to start with something and then verify it than to not have the tool in the first place.
p-e-w 4y ago
> The information value of answers by ChatGPT is zero. Nonsense. The only way this would be true is if ChatGPT's answers were as likely to be correct as random answers. They are much, much more likely to be correct, however, so their information value is greater than zero by definition. What you're claiming is equivalent to "search engines can find incorrect information, so search engines are worthless for information retrieval". Which is bollocks.
- chx 4y ago
  > The only way this would be true is if ChatGPT's answers were as likely to be correct as random answers. Which is the case > They are much, much more likely to be correct, They are not. It's just apophenia to find meaning in the word salad.
  - olddustytrail 4y ago
    Of course it's not the case. You do realise that people have measured this? The irony of people complaining that LLMs confidently assert things that are completely wrong...