4 ms·

> Which is ridiculous, it's just an incredible token predictor. I'm not sure I'm not just a mediocre token predictor.

exe34 3y ago
I've never met anyone who didn't just say the next word in a sequence. So far nobody's been able to produce a complete sentence in one go!
- relaxing 3y ago
  Stop.
  - exe34 3y ago
    I'm sorry, as a large language model, I can't do that Dave.
- ben_w 3y ago
  Now you have. And I don't just mean the one-word example in the sibling comment: I think a complete sentence first, and then generate the words. The part of my mind I could describe as an "inner voice" does not need to turn this thought into words… but (if you will excuse me giving you an anthropomorphic personification of a homunculus) that inner voice gets annoyed if I don't let it "read out" the words of that sentence.
  - exe34 3y ago
    That's how you claim you came up with that sentence. Your answer basically reverts back to the Cartesian theater, you even bring up the homunculus yourself. There's somebody dictating whole sentences at you and you're thanklessly transcribing it for the rest of us. Ask chat gpt something equally unknowable and it will also earnestly confabulate a similarly magical-thinking answer. I wonder if you two are really so different! Either way, what you claim cannot be an explanation, it simply pushes the problem one step back. However it is that you came up with the sentences, when it came to saying out loud the words, or typing them, you still typed one token after the other. Sure you had a buffer that kept getting filled up faster than you fingers could type or swipe, but ultimately you still produced the next letter/word/sentence. Entire sentences can be tokens. BPE isn't the only encoding.
    - ben_w 3y ago
      > Entire sentences can be tokens Sure, but that feels like you are shifting goalposts from "words". (But: What is a word?) > There's somebody dictating whole sentences at you and you're thanklessly transcribing it for the rest of us. "There is a thought now" -> inner voice wants a go at it, regardless of if I speak or write what the inner voice says. This part can be (please excuse the anthropomorphic personification) "told" to not bother in order to save time because I already know the entire thought, but doing so generates an emotion which the part of me which did the "telling" can also directly experience. Some people report not having an inner voice. Mine sometimes gets an echo. (I wonder about comparison with aphantasia/hyperphantasia?) > Your answer basically reverts back to the Cartesian theater Indeed. Altered states of consciousness when tired suggest to me this is the organisational form within my mind: my sense of being has, at times, and for lack of a better description given no shared context for the words I'm using[0], "switched off" and I am left with a lack of a sense that I'm really there — not dissimilar to watching a TV show, with the input from my eyes playing the role of the a screen, and my ears the speakers. Not all minds are organised the same (again, aphantasia etc.), so I don't expect this description of mine to be found in all humans, though I also don't expect to be unique. [0] SSRI withdrawal has sensations described as "mini electric shocks". Having had both that withdrawal and mini electric shocks: it doesn't feel like that, but I can totally see why someone might describe it that way. No shared context, prevents deliberate agreement on the meanings of words.
      - exe34 3y ago
        When you look at a computer screen, I expect you wouldn't imagine there really are little rectangles inside, even if it appears so. It's the same with the mind - just because something appears a certain way doesn't mean that's how it works. An explanation has to satisfy certain requirements - it has to be made of simpler parts, not a Matryoshka doll of ever shrinking homunculi with the exact same capabilities. You might find Dennett's books like Consciousness explained or Sweet Dreams an interesting challenge. If your description of consciousness involves pushing back the veil to a point where exactly the same words/thoughts/actions are being created further inside, and then mediated to the outside via a clerk, then it's not very useful.

I've never met anyone who didn't just say the next word in a sequence. So far nobody's been able to produce a complete sentence in one go!

relaxing 3y ago
Stop.
- exe34 3y ago
  I'm sorry, as a large language model, I can't do that Dave.
ben_w 3y ago
Now you have. And I don't just mean the one-word example in the sibling comment: I think a complete sentence first, and then generate the words. The part of my mind I could describe as an "inner voice" does not need to turn this thought into words… but (if you will excuse me giving you an anthropomorphic personification of a homunculus) that inner voice gets annoyed if I don't let it "read out" the words of that sentence.
- exe34 3y ago
  That's how you claim you came up with that sentence. Your answer basically reverts back to the Cartesian theater, you even bring up the homunculus yourself. There's somebody dictating whole sentences at you and you're thanklessly transcribing it for the rest of us. Ask chat gpt something equally unknowable and it will also earnestly confabulate a similarly magical-thinking answer. I wonder if you two are really so different! Either way, what you claim cannot be an explanation, it simply pushes the problem one step back. However it is that you came up with the sentences, when it came to saying out loud the words, or typing them, you still typed one token after the other. Sure you had a buffer that kept getting filled up faster than you fingers could type or swipe, but ultimately you still produced the next letter/word/sentence. Entire sentences can be tokens. BPE isn't the only encoding.
  - ben_w 3y ago
    > Entire sentences can be tokens Sure, but that feels like you are shifting goalposts from "words". (But: What is a word?) > There's somebody dictating whole sentences at you and you're thanklessly transcribing it for the rest of us. "There is a thought now" -> inner voice wants a go at it, regardless of if I speak or write what the inner voice says. This part can be (please excuse the anthropomorphic personification) "told" to not bother in order to save time because I already know the entire thought, but doing so generates an emotion which the part of me which did the "telling" can also directly experience. Some people report not having an inner voice. Mine sometimes gets an echo. (I wonder about comparison with aphantasia/hyperphantasia?) > Your answer basically reverts back to the Cartesian theater Indeed. Altered states of consciousness when tired suggest to me this is the organisational form within my mind: my sense of being has, at times, and for lack of a better description given no shared context for the words I'm using[0], "switched off" and I am left with a lack of a sense that I'm really there — not dissimilar to watching a TV show, with the input from my eyes playing the role of the a screen, and my ears the speakers. Not all minds are organised the same (again, aphantasia etc.), so I don't expect this description of mine to be found in all humans, though I also don't expect to be unique. [0] SSRI withdrawal has sensations described as "mini electric shocks". Having had both that withdrawal and mini electric shocks: it doesn't feel like that, but I can totally see why someone might describe it that way. No shared context, prevents deliberate agreement on the meanings of words.
    - exe34 3y ago
      When you look at a computer screen, I expect you wouldn't imagine there really are little rectangles inside, even if it appears so. It's the same with the mind - just because something appears a certain way doesn't mean that's how it works. An explanation has to satisfy certain requirements - it has to be made of simpler parts, not a Matryoshka doll of ever shrinking homunculi with the exact same capabilities. You might find Dennett's books like Consciousness explained or Sweet Dreams an interesting challenge. If your description of consciousness involves pushing back the veil to a point where exactly the same words/thoughts/actions are being created further inside, and then mediated to the outside via a clerk, then it's not very useful.