4 ms·

I work in the field. The issue is that you will find 2+2=5 literally a hundred times more in all big datasets than 6+3=9 A lot of information is wrong, a lot i

by 317070 2y ago

I work in the field. The issue is that you will find 2+2=5 literally a hundred times more in all big datasets than 6+3=9 A lot of information is wrong, a lot is only true in context (e.g. 2+2=5 features prominent in the book 1984), even more is spam or machine generated. If you see the garbage that goes in, I am always amazed at how these models do what they do.

soloist11 2y ago
Yes, the issue is that statistical models can not reason and determine what is logically valid vs what is most probable (which I guess is also its own kind of logic).
- 317070 2y ago
  And context, so much context. The sqrt(-1) sometimes doesn't exist, sometimes it's 1i. 2+2=4, except in literature where it can be 5. 1+1=2, but sometimes 3 in advertisements or in ironical text. We often have some ideas about e.g how it works in a quiz, where you know there is only one factually correct answer. And we are disappointed if the model is wrong. But even in a quiz setting the jury gets that balance wrong every so often, where there are other answers than the official one which are also correct. Even "logically valid" is context dependend. This is not to say that models don't hallucinate, just that even within the logically valid answers, there is hidden context surrounding the data which is not expressed in the data itself. Fermats last problem is a solved problem in mathematics, but not in documents from before 1994.
  - soloist11 2y ago
    The models operate by the logic of boolean arithmetic so in that sense they can not be inconsistent. But in any case, it's pretty obvious no one in this thread understands what I'm getting at but maybe eventually there will be an AGI smart enough to get the point.
mdp2021 2y ago
And this is why input assessment¹ is crucial... (Idea assessment in general. Handled ideas in thought processes are still input.)
jfengel 2y ago
To be fair, I'm kind of amazed that humans can do basic arithmetic at all. Our attention spans suck. We're barely trustworthy on single digit addition. For anything more complicated I use a calculator.
fragmede 2y ago
Annotated data sets help, and ChatGPT-4o says 2+2=4 just fine. It also gives the right answer for 920,384 + 293,848. It can also read X - 12 = 14 out of an image and get 2. 1984 having 2 + 2 = 5 makes sense in context as a human reading the book, and ChatGPT dot-producting the book can also compute the context and not say that 2+2=5. ChatGPT's not Mathematica, and we already have calculators. My hammer is terrible for driving in nails, so I don't use it for that.

by 317070 2y ago

soloist11 2y ago

Yes, the issue is that statistical models can not reason and determine what is logically valid vs what is most probable (which I guess is also its own kind of logic).

317070 2y ago
And context, so much context. The sqrt(-1) sometimes doesn't exist, sometimes it's 1i. 2+2=4, except in literature where it can be 5. 1+1=2, but sometimes 3 in advertisements or in ironical text. We often have some ideas about e.g how it works in a quiz, where you know there is only one factually correct answer. And we are disappointed if the model is wrong. But even in a quiz setting the jury gets that balance wrong every so often, where there are other answers than the official one which are also correct. Even "logically valid" is context dependend. This is not to say that models don't hallucinate, just that even within the logically valid answers, there is hidden context surrounding the data which is not expressed in the data itself. Fermats last problem is a solved problem in mathematics, but not in documents from before 1994.
- soloist11 2y ago
  The models operate by the logic of boolean arithmetic so in that sense they can not be inconsistent. But in any case, it's pretty obvious no one in this thread understands what I'm getting at but maybe eventually there will be an AGI smart enough to get the point.

mdp2021 2y ago

And this is why input assessment¹ is crucial... (Idea assessment in general. Handled ideas in thought processes are still input.)

jfengel 2y ago

To be fair, I'm kind of amazed that humans can do basic arithmetic at all. Our attention spans suck. We're barely trustworthy on single digit addition. For anything more complicated I use a calculator.

fragmede 2y ago

Annotated data sets help, and ChatGPT-4o says 2+2=4 just fine. It also gives the right answer for 920,384 + 293,848. It can also read X - 12 = 14 out of an image and get 2. 1984 having 2 + 2 = 5 makes sense in context as a human reading the book, and ChatGPT dot-producting the book can also compute the context and not say that 2+2=5. ChatGPT's not Mathematica, and we already have calculators. My hammer is terrible for driving in nails, so I don't use it for that.