5 ms·

I think what is interesting is even in the 1st sentence of Wolfram's essay, he acknowledges what ChatGPT is doing could merely be superficial. So we can't even

by calf 4y ago

I think what is interesting is even in the 1st sentence of Wolfram's essay, he acknowledges what ChatGPT is doing could merely be superficial. So we can't even tell if ChatGPT actually learned "language" in a superficial versus nontrivial/substantial sense. Lots of people say, I asked ChatGPT to write me a poem/essay, and it did! But was it really a poem/essay, or did it just look like one and on closer examination it is more like a fake out of a poem/essay? A piece of writing is not merely its form, but also its content.

svachalek 4y ago
When you ask a child to write a poem or essay, is it real or fake? I think if we accept that as real, it's hard not to accept ChatGPT as real. Sometimes its writing is not very good, sometimes its not even all that consistent, but from what I've seen the quality level is better than the average non-professional human writer.
- Barrin92 4y ago
  Realness has little to do with the quality of the output. Or else my TI-83 would be the realest mathematician on the planet, it can multiply numbers that even Terence Tao can't in his head. Judging these systems solely by their output is to repeat the msitakes of behaviorism, even a dumb markov chain or a parrot converses better than an infant, but unlike the infant does not acquire an understanding or representation of language.
  - jonahx 4y ago
    > has little to do with the quality of the output. Non-snarky question: What else can you judge by? Isn't any alternative just putting more precise conditions on the output? With ChatGPT, it's still easy enough to see it's mistakes, and its attempts at fiction and poetry, impressive as they are, are still clumsy to a trained eye, relative to expert human work. But what if they weren't? What happens when they're indistinguishable?
    - Barrin92 4y ago
      >Non-snarky question: What else can you judge by? its architecure. A child is a living and autonomous agent. It has (or develops) meta cognition, an awareness of its own mental state (and by extension use of language). These models don't have the capacity to do this even in theory given that they're static and pretrained. When you ask ChatGPT what it feels like to speak, there isn't some neural activity within the model, it has no model of itself that it actively inspects, it doesn't learn while it converses with you, it just tells you what someone wrote on Quora two years ago. >What happens when they're indistinguishable? Then the system is likely going to look a lot different than it does now because these aspects of cognition seem pretty important when you want something that is genuinely human-like rather than just mimicry or memorization.
      - jonahx 4y ago
        I take your point. And agree there is still a huge gulf. > Then the system is likely going to look a lot different than it does now I think I agree with this too, but to challenge the idea: I would never have thought "a fancy autocomplete architecture" could give rise to something as sophisticated as ChatGPT, its flaws notwithstanding. So I don't feel so confident that further iterations of the idea, or iterations that involve other architectures that are "still obviously fake" won't give rise to results that far more terrifyingly convincing than ChatGPT. Since we don't really understand these architectures, human or machine, I don't see how that can be used as the criteria. Ever more find-grained versions of "output" seem like the only ground truth. The goal posts can keep moving... they can do language but can't implement robots with proper voice or facial expressions, etc. But in theory if there were no more goal posts left, I feel like the architecture argument would ring hollow.
      - thfuran 4y ago
        The essay is the words that comprise it, not the means by which they were produced. I don't need to write it out by hand for it to be real. My printer can produce a real essay. A photocopier can produce a real essay. ChatGPT can produce a real essay, but it's just probably going to be a bit shit.
    - 4y ago
  - jck 4y ago
    What you're describing is arithmetic. I think most mathematicians will easily be beaten by a calculator - and they don't care. Mathematics isn't just mental arithmetic. Can your TI-83 do any proofs?
deleted 4y ago
[deleted]
kerpotgh 4y ago
[dead]
thfuran 4y ago
>But was it really a poem/essay, or did it just look like one and on closer examination it is more like a fake out of a poem/essay? What does that even mean?
- wankle 4y ago
  Perhaps they were referring to the nature of machine learning to be akin to the Ronco slicer/dicer from the 1970's but with data.

by calf 4y ago

svachalek 4y ago

When you ask a child to write a poem or essay, is it real or fake? I think if we accept that as real, it's hard not to accept ChatGPT as real. Sometimes its writing is not very good, sometimes its not even all that consistent, but from what I've seen the quality level is better than the average non-professional human writer.

Barrin92 4y ago
Realness has little to do with the quality of the output. Or else my TI-83 would be the realest mathematician on the planet, it can multiply numbers that even Terence Tao can't in his head. Judging these systems solely by their output is to repeat the msitakes of behaviorism, even a dumb markov chain or a parrot converses better than an infant, but unlike the infant does not acquire an understanding or representation of language.
- jonahx 4y ago
  > has little to do with the quality of the output. Non-snarky question: What else can you judge by? Isn't any alternative just putting more precise conditions on the output? With ChatGPT, it's still easy enough to see it's mistakes, and its attempts at fiction and poetry, impressive as they are, are still clumsy to a trained eye, relative to expert human work. But what if they weren't? What happens when they're indistinguishable?
  - Barrin92 4y ago
    >Non-snarky question: What else can you judge by? its architecure. A child is a living and autonomous agent. It has (or develops) meta cognition, an awareness of its own mental state (and by extension use of language). These models don't have the capacity to do this even in theory given that they're static and pretrained. When you ask ChatGPT what it feels like to speak, there isn't some neural activity within the model, it has no model of itself that it actively inspects, it doesn't learn while it converses with you, it just tells you what someone wrote on Quora two years ago. >What happens when they're indistinguishable? Then the system is likely going to look a lot different than it does now because these aspects of cognition seem pretty important when you want something that is genuinely human-like rather than just mimicry or memorization.
    - jonahx 4y ago
      I take your point. And agree there is still a huge gulf. > Then the system is likely going to look a lot different than it does now I think I agree with this too, but to challenge the idea: I would never have thought "a fancy autocomplete architecture" could give rise to something as sophisticated as ChatGPT, its flaws notwithstanding. So I don't feel so confident that further iterations of the idea, or iterations that involve other architectures that are "still obviously fake" won't give rise to results that far more terrifyingly convincing than ChatGPT. Since we don't really understand these architectures, human or machine, I don't see how that can be used as the criteria. Ever more find-grained versions of "output" seem like the only ground truth. The goal posts can keep moving... they can do language but can't implement robots with proper voice or facial expressions, etc. But in theory if there were no more goal posts left, I feel like the architecture argument would ring hollow.
    - thfuran 4y ago
      The essay is the words that comprise it, not the means by which they were produced. I don't need to write it out by hand for it to be real. My printer can produce a real essay. A photocopier can produce a real essay. ChatGPT can produce a real essay, but it's just probably going to be a bit shit.
  - 4y ago
- jck 4y ago
  What you're describing is arithmetic. I think most mathematicians will easily be beaten by a calculator - and they don't care. Mathematics isn't just mental arithmetic. Can your TI-83 do any proofs?

deleted 4y ago

[deleted]

kerpotgh 4y ago

[dead]

thfuran 4y ago

>But was it really a poem/essay, or did it just look like one and on closer examination it is more like a fake out of a poem/essay? What does that even mean?

wankle 4y ago
Perhaps they were referring to the nature of machine learning to be akin to the Ronco slicer/dicer from the 1970's but with data.