6 ms·

Tulips were not one of the greatest scientific advances in human history. AI that can write and converse intelligently on virtually any subject is. When the chi

Tulips were not one of the greatest scientific advances in human history. AI that can write and converse intelligently on virtually any subject is. When the chief complaint about AI is that it lies too much, that's a massive breakthrough.

thundergolfer 4y ago
It's a great engineering advancement, but scientific advancement implies some kind of gain in human knowledge, and that hasn't happened to a great extent. We still don't understand how these models really work, at least not in a way that teaches us anything about language or about humanity.
- pixl97 4y ago
  >implies some kind of gain in human knowledge I find that a strange take. Almost all scientific advancement ends in one thing. Another set of unanswered questions. Quite often we gain more knowledge, but know less because we realize there are thousands more questions we should have been asking. For example, theory of mind being observed out of just processing language is a big arrow that we may have been thinking about many things wrong in the past and open new avenues for 'scientific' tests that give us answers.
  - ianmcgowan 4y ago
    It's got to have shaken up the field of cognitive research or people studying mind/consciousness. Is there a "hacker news" of theory of mind researchers? Probably some interesting conversations happening over there.
- dd36 4y ago
  We’ve absolutely gained knowledge. We’ve learned how good these models can be. We’ve effectively made like a general teenager that can be up-trained for specific tasks. It’s super useful if you know how to use it.
- boringg 4y ago
  I'd 100% agree with this take. It's a great engineering achievement but it isn't a scientific breakthrough, much less one of the greatest scientific achievements of humankind.
- moffkalast 4y ago
  > but scientific advancement implies some kind of gain in productivity FTFY. At least any kind of actually notable scientific advancement like using metals, farming, steam power, electricity, cars, planes, computers, etc. If a person can use a chatbot to speed up their work then that's a major advancement in productivity right there.
  - deleted 4y ago
    [deleted]
  - ianmcgowan 4y ago
    Almost a textbook answer to "what is the difference between science and technology".
    - moffkalast 4y ago
      Two sides of the same coin.
- rcme 4y ago
  > We still don't understand how these models really work, at least not in a way that teaches us anything about language or about humanity. We understand how LLMs work conceptually. Human language, when parameterized as a sequence of tokens, has geometry and structure in a billion dimensional space. A priori, we didn't know that. So in a very real sense, a discovery was made.
- est 4y ago
  > but scientific advancement implies some kind of gain in human knowledge The unexplained Chain-of-Thought emerges from LLM surely would an huge gain in "knowledge". In fact I'd say it potentially the most important meta-knowledge human race can gain, you know, to unlock the mystery of "conciousness" itself.
Retric 4y ago
These chat bots can’t converse intelligently, they can fake well enough it can take a little while to realize. https://xkcd.com/451/ https://xkcd.com/451/ There’s some great chess videos for example where the bots start by making decent moves until you quickly find out they not only don’t know how to play well they don’t even understand the rules or the board. https://youtu.be/rSCNW1OCk_M https://youtu.be/rSCNW1OCk_M What’s fascinating is how easy it is to give something the benefit of the doubt because it’s using language, we just aren’t very suspicious of things that can talk.
- luckylion 4y ago
  Now instruct people who don't know how to play chess to pretend to know and play against someone. And when they fail because they don't know the rules and initially just mirrored their opponent, that's proof they're not a general intelligence?
  - Retric 4y ago
    ChatGPT can say the rules of chess it just doesn’t understand them, thus stupidly rather than ignorance. Further if you watch the video it wasn’t moving the pieces incorrectly it was just creating them from thin air. This wasn’t like an 8 year old learning the game this was like your cat randomly hitting the keyboard.
    - luckylion 4y ago
      > ChatGPT can say the rules of chess it just doesn’t understand them, thus stupidly rather than ignorance. It acts similar to a human that has a button for "recite chess rules" and presses it when you ask them to recite the chess rules. You'd then get a reply with the chess rules without the button-presser getting the same info, so they wouldn't gain any knowledge from it. I'm sure you can find a lot of people who could, if asked, read you the chess rules but even after reading wouldn't have understood them themselves. But are they not generally intelligent? I get a feeling that people compare ML-models to a smart person that concentrates on what you ask them, and LLMs fail in comparison but do so confidently. If you instead compare them to an average person at their daily average focus level, the ML probably looks pretty smart. I have a neighbor who is not very bright. He'll see some news on TV and will tell you about it, only he doesn't understand it and will just repeat words and half-sentences as best he can remember. He doesn't know he doesn't understand (or he does but doesn't care), so he'll confidently state utter nonsense. He's not smart and I wouldn't go to him for advice, but I think he qualifies as "intelligent" in the binary sense.
throw_away1525 4y ago
Well, they can certainly converse intelligibly. Intelligently is up for debate.
reeboo 4y ago
Projecting our humanity onto these models by assuming they are "lying" is a mistake. These models are not thinking, thus, they cannot lie.
inciampati 4y ago
Indeed. And furthermore, I notice that most criticism of these systems focuses on their lack of factual coherence and accuracy, ignoring their real incredible power of translation. Give concepts in any structure or language, at any level of coherency, and you can get out clear code, prose, or poetry that represents exactly the input. Ask for a biography of a random public figure without giving notes and there is 0 chance they will provide an accurate result. This may mean we need systems that are literally memories without the noise driven generalization, and that these need to be exposed to the said LLMs as a resource. It's not insurmountable and it certainly isn't tulips.
- pwinnski 4y ago
  > ignoring their real incredible power of translation. I feel like everybody commenting on this thread ought to have to write "it's text completion, not intelligence" on a chalkboard 500 times or so first.

thundergolfer 4y ago

It's a great engineering advancement, but scientific advancement implies some kind of gain in human knowledge, and that hasn't happened to a great extent. We still don't understand how these models really work, at least not in a way that teaches us anything about language or about humanity.

pixl97 4y ago
>implies some kind of gain in human knowledge I find that a strange take. Almost all scientific advancement ends in one thing. Another set of unanswered questions. Quite often we gain more knowledge, but know less because we realize there are thousands more questions we should have been asking. For example, theory of mind being observed out of just processing language is a big arrow that we may have been thinking about many things wrong in the past and open new avenues for 'scientific' tests that give us answers.
- ianmcgowan 4y ago
  It's got to have shaken up the field of cognitive research or people studying mind/consciousness. Is there a "hacker news" of theory of mind researchers? Probably some interesting conversations happening over there.
dd36 4y ago
We’ve absolutely gained knowledge. We’ve learned how good these models can be. We’ve effectively made like a general teenager that can be up-trained for specific tasks. It’s super useful if you know how to use it.
boringg 4y ago
I'd 100% agree with this take. It's a great engineering achievement but it isn't a scientific breakthrough, much less one of the greatest scientific achievements of humankind.
moffkalast 4y ago
> but scientific advancement implies some kind of gain in productivity FTFY. At least any kind of actually notable scientific advancement like using metals, farming, steam power, electricity, cars, planes, computers, etc. If a person can use a chatbot to speed up their work then that's a major advancement in productivity right there.
- deleted 4y ago
  [deleted]
- ianmcgowan 4y ago
  Almost a textbook answer to "what is the difference between science and technology".
  - moffkalast 4y ago
    Two sides of the same coin.
rcme 4y ago
> We still don't understand how these models really work, at least not in a way that teaches us anything about language or about humanity. We understand how LLMs work conceptually. Human language, when parameterized as a sequence of tokens, has geometry and structure in a billion dimensional space. A priori, we didn't know that. So in a very real sense, a discovery was made.
est 4y ago
> but scientific advancement implies some kind of gain in human knowledge The unexplained Chain-of-Thought emerges from LLM surely would an huge gain in "knowledge". In fact I'd say it potentially the most important meta-knowledge human race can gain, you know, to unlock the mystery of "conciousness" itself.

Retric 4y ago

These chat bots can’t converse intelligently, they can fake well enough it can take a little while to realize. https://xkcd.com/451/ https://xkcd.com/451/ There’s some great chess videos for example where the bots start by making decent moves until you quickly find out they not only don’t know how to play well they don’t even understand the rules or the board. https://youtu.be/rSCNW1OCk_M https://youtu.be/rSCNW1OCk_M What’s fascinating is how easy it is to give something the benefit of the doubt because it’s using language, we just aren’t very suspicious of things that can talk.

luckylion 4y ago
Now instruct people who don't know how to play chess to pretend to know and play against someone. And when they fail because they don't know the rules and initially just mirrored their opponent, that's proof they're not a general intelligence?
- Retric 4y ago
  ChatGPT can say the rules of chess it just doesn’t understand them, thus stupidly rather than ignorance. Further if you watch the video it wasn’t moving the pieces incorrectly it was just creating them from thin air. This wasn’t like an 8 year old learning the game this was like your cat randomly hitting the keyboard.
  - luckylion 4y ago
    > ChatGPT can say the rules of chess it just doesn’t understand them, thus stupidly rather than ignorance. It acts similar to a human that has a button for "recite chess rules" and presses it when you ask them to recite the chess rules. You'd then get a reply with the chess rules without the button-presser getting the same info, so they wouldn't gain any knowledge from it. I'm sure you can find a lot of people who could, if asked, read you the chess rules but even after reading wouldn't have understood them themselves. But are they not generally intelligent? I get a feeling that people compare ML-models to a smart person that concentrates on what you ask them, and LLMs fail in comparison but do so confidently. If you instead compare them to an average person at their daily average focus level, the ML probably looks pretty smart. I have a neighbor who is not very bright. He'll see some news on TV and will tell you about it, only he doesn't understand it and will just repeat words and half-sentences as best he can remember. He doesn't know he doesn't understand (or he does but doesn't care), so he'll confidently state utter nonsense. He's not smart and I wouldn't go to him for advice, but I think he qualifies as "intelligent" in the binary sense.

throw_away1525 4y ago

Well, they can certainly converse intelligibly. Intelligently is up for debate.

reeboo 4y ago

Projecting our humanity onto these models by assuming they are "lying" is a mistake. These models are not thinking, thus, they cannot lie.

inciampati 4y ago

Indeed. And furthermore, I notice that most criticism of these systems focuses on their lack of factual coherence and accuracy, ignoring their real incredible power of translation. Give concepts in any structure or language, at any level of coherency, and you can get out clear code, prose, or poetry that represents exactly the input. Ask for a biography of a random public figure without giving notes and there is 0 chance they will provide an accurate result. This may mean we need systems that are literally memories without the noise driven generalization, and that these need to be exposed to the said LLMs as a resource. It's not insurmountable and it certainly isn't tulips.

pwinnski 4y ago
> ignoring their real incredible power of translation. I feel like everybody commenting on this thread ought to have to write "it's text completion, not intelligence" on a chalkboard 500 times or so first.