5 ms·

The amount of non-technical people who have very little intuition around the limitations of LLMs is actually fairly startling - and I like to think of myself of

by extr 3y ago

The amount of non-technical people who have very little intuition around the limitations of LLMs is actually fairly startling - and I like to think of myself of someone who is usually good at that kind of thing. The realistic/coherence of the output is just so convincing, people cannot seem to stop themselves from attributing a theory of mind to it. Just earlier this week I saw someone saying that they were creating a ton of OpenAI accounts to try to advocate to ChatGPT for their activist cause.

AJ007 3y ago
How many people believe horoscopes and astrology carry any meaning? Most will never comprehend LLMs or even the simplest concepts of machine learning.
- erehweb 3y ago
  Autocorrect is fairly well understood and an LLM is an extension of that. (Yes, I know it's more complicated - still, it's a reasonable first approximation)
  - MandieD 3y ago
    I wouldn't say that autocorrect and autocomplete are well understood by most users, but are well-accepted and not feared and/or fetishized. Telling my non-tech relatives and friends that "ChatGPT is extremely fancy autocomplete" has settled most of them down a bit.
- hnfong 3y ago
  While we're on the topic of stroking our epeens, do you know how horoscopes and astrology work? Most people here with that condescending attitude will never comprehend it either. FWIW, it's one of the topics I've been trying to understand and come up with a consistent theory for a while. (Not to say those who believe in horoscopes and astrology know how it works though...)
  - deleted 3y ago
    [deleted]
  - knaik94 3y ago
    There is no complete theory, because having a complete theory would require a complete theory about cognition. But the general understanding is that horoscopes and astrology play into the fallacy of personal validation. People are bad at evaluating how generic certain descriptions are. It is a well researched and replicated result. The arguments supporting horoscopes and astrology are also written in a way to make them impossible to disprove, making them unscientific. https://en.wikipedia.org/wiki/Barnum_effect https://en.wikipedia.org/wiki/Barnum_effect
  - wizofaus 3y ago
    By "work" I assume you mean "convince people of their value"?
Xcelerate 3y ago
> The amount of non-technical people who have very little intuition around the limitations of LLMs is actually fairly startling Is this surprising though? To be honest, I don’t think we have any such thing as “AI experts” currently. LeCun, Hinton, etc. were early pioneers in neural networks and Hutter did a lot of the first work with reinforcement learning, but none of these people seem to agree on much now with regard to AGI, and I wouldn’t even say they have any particularly deep insight into LLMs beyond that of a research scientist at OpenAI (perhaps less even, given the time spent on social priorities now). We’re reaching a point where it’s likely that further development just isn’t going to be well-understood by either the general public or the people creating the tech, and this will lead to a lot of “voodoo” explanations of how things work. Progress is increasingly driven by trying things and seeing what works rather than a theory-first approach, and whenever this occurs, a lot of mysticism tends to accompany the process.
- hnfong 3y ago
  It isn't surprising to the least, and given how little humanity knows about quasi-human level intelligence, I would suggest taking any hard claims about limitations of LLMs with a grain of salt. > Progress is increasingly driven by trying things and seeing what works rather than a theory-first approach, and whenever this occurs, a lot of mysticism tends to accompany the process. This sounds wrong. It sounds as if you're arguing that empiricism and the usual scientific method ends up in mysticism. If it does, maybe the things we're creating should command that respect from us, as much as all other natural phenomena that we don't understand. The traditional idea that many CS experts popularized that computers are just applied maths and you could infer anything on a computer with pure thought is wrong IMHO.
  - Xcelerate 3y ago
    > This sounds wrong. It sounds as if you're arguing that empiricism and the usual scientific method ends up in mysticism I think we’re on the same page... I’m arguing that the current approach to creating better LLMs is more akin to an art involving educated guesses and tinkering than it is a hypothesis-driven process with randomized controls (e.g., traditional engineering disciplines or applied physics / chemistry).
bongobingo1 3y ago
"Any sufficiently advanced technology is indistinguishable from magic." While I don't think non-technical people think its "magic", you can hardly fault them considering 1) the "main stream media" hype around it and how it's going to be so transformative on jobs if nothing else, and 2) some people have been promised this kind of stuff since star trek or earlier, or their previous experience was having Siri say "sorry I don't understand what you said" back to them before setting the timer manually. Just having a coherent responses from a parser that's pretty hard to "trip up" is, well, unsurprisingly magical. I mean, I just suggested a pretty complicated model relationship to chat3.5 and asked for a DB schema and it gave be back quite a coherent answer, even I thought it was close enough to magic. Siri would have thrown a fit.
- unsupp0rted 3y ago
  Any sufficiently advanced technology has emergent properties that might make it turn into a person
Capricorn2481 3y ago
Really, the amount of technical people that don't understand it is also startling

by extr 3y ago

AJ007 3y ago

How many people believe horoscopes and astrology carry any meaning? Most will never comprehend LLMs or even the simplest concepts of machine learning.

erehweb 3y ago
Autocorrect is fairly well understood and an LLM is an extension of that. (Yes, I know it's more complicated - still, it's a reasonable first approximation)
- MandieD 3y ago
  I wouldn't say that autocorrect and autocomplete are well understood by most users, but are well-accepted and not feared and/or fetishized. Telling my non-tech relatives and friends that "ChatGPT is extremely fancy autocomplete" has settled most of them down a bit.
hnfong 3y ago
While we're on the topic of stroking our epeens, do you know how horoscopes and astrology work? Most people here with that condescending attitude will never comprehend it either. FWIW, it's one of the topics I've been trying to understand and come up with a consistent theory for a while. (Not to say those who believe in horoscopes and astrology know how it works though...)
- deleted 3y ago
  [deleted]
- knaik94 3y ago
  There is no complete theory, because having a complete theory would require a complete theory about cognition. But the general understanding is that horoscopes and astrology play into the fallacy of personal validation. People are bad at evaluating how generic certain descriptions are. It is a well researched and replicated result. The arguments supporting horoscopes and astrology are also written in a way to make them impossible to disprove, making them unscientific. https://en.wikipedia.org/wiki/Barnum_effect https://en.wikipedia.org/wiki/Barnum_effect
- wizofaus 3y ago
  By "work" I assume you mean "convince people of their value"?

Xcelerate 3y ago

> The amount of non-technical people who have very little intuition around the limitations of LLMs is actually fairly startling Is this surprising though? To be honest, I don’t think we have any such thing as “AI experts” currently. LeCun, Hinton, etc. were early pioneers in neural networks and Hutter did a lot of the first work with reinforcement learning, but none of these people seem to agree on much now with regard to AGI, and I wouldn’t even say they have any particularly deep insight into LLMs beyond that of a research scientist at OpenAI (perhaps less even, given the time spent on social priorities now). We’re reaching a point where it’s likely that further development just isn’t going to be well-understood by either the general public or the people creating the tech, and this will lead to a lot of “voodoo” explanations of how things work. Progress is increasingly driven by trying things and seeing what works rather than a theory-first approach, and whenever this occurs, a lot of mysticism tends to accompany the process.

hnfong 3y ago
It isn't surprising to the least, and given how little humanity knows about quasi-human level intelligence, I would suggest taking any hard claims about limitations of LLMs with a grain of salt. > Progress is increasingly driven by trying things and seeing what works rather than a theory-first approach, and whenever this occurs, a lot of mysticism tends to accompany the process. This sounds wrong. It sounds as if you're arguing that empiricism and the usual scientific method ends up in mysticism. If it does, maybe the things we're creating should command that respect from us, as much as all other natural phenomena that we don't understand. The traditional idea that many CS experts popularized that computers are just applied maths and you could infer anything on a computer with pure thought is wrong IMHO.
- Xcelerate 3y ago
  > This sounds wrong. It sounds as if you're arguing that empiricism and the usual scientific method ends up in mysticism I think we’re on the same page... I’m arguing that the current approach to creating better LLMs is more akin to an art involving educated guesses and tinkering than it is a hypothesis-driven process with randomized controls (e.g., traditional engineering disciplines or applied physics / chemistry).

bongobingo1 3y ago

"Any sufficiently advanced technology is indistinguishable from magic." While I don't think non-technical people think its "magic", you can hardly fault them considering 1) the "main stream media" hype around it and how it's going to be so transformative on jobs if nothing else, and 2) some people have been promised this kind of stuff since star trek or earlier, or their previous experience was having Siri say "sorry I don't understand what you said" back to them before setting the timer manually. Just having a coherent responses from a parser that's pretty hard to "trip up" is, well, unsurprisingly magical. I mean, I just suggested a pretty complicated model relationship to chat3.5 and asked for a DB schema and it gave be back quite a coherent answer, even I thought it was close enough to magic. Siri would have thrown a fit.

unsupp0rted 3y ago
Any sufficiently advanced technology has emergent properties that might make it turn into a person

Capricorn2481 3y ago

Really, the amount of technical people that don't understand it is also startling