5 ms·

When you ask yourself a question, that's recursion. The conversation with yourself can't go on until the question at the top of the stack is answered. The voice

When you ask yourself a question, that's recursion. The conversation with yourself can't go on until the question at the top of the stack is answered. The voice in your head, that is to say the closed brain loop of talking to yourself which we call "thinking" is recursive. Its a strange form of programming where everything is hacked out of api calls to localhost. There are no implementations, its api all the way down. These LLM's don't have that brain loop due to how they were constructed. They cannot do voice-in-your-head reasoning. Whatever is done in the loop structure has to be completely unrolled to be done in a single pass by an LLM. Needless to say, a lot comes for free in the recursive structure that has to be trained with great effort on the naive, unrolled, flat structure. This guy hacks a feedback loop into the LLM by manually feeding the output back to the input.

YeGoblynQueenne 4y ago
>> When you ask yourself a question, that's recursion (...) I don't know if any of that makes sense. I think you're misapplying some handy metaphors: you're anthropomorphising a computer and mechanomorphising a human. I have "localchost"? What, do I also have Perl scripts? Now I'm starting to feel like a character in a P. K. Dick story. But all this doesn't matter because we're not talking about a person, asking questions of themself. We're talking about someone doing things with a computer. And we know exactly what "recursion" means in the context of a computer. On a computer, then, if anyone wants to show "recursion" in LLMs, they better be able to show how to implement a push-down stack with the bot's conversation window. Then they can show how to calculate a factorial recursively, and how their calculation behaves differently when it's computed in a tail-recursive manner, and when it is not. Yeah, I can see what the author doing, and it's not recursion. But I'm trying to be kind so I won't say what it is.
- IIAOPSW 4y ago
  Don't you see it. Its giving chatGPT the ability to use chatGPT as a subroutine in formulating its response to a prompt. Conversation is a stack! When you pose a question (to yourself or anyone else), whatever line of thought you were working on stays paused until an answer is popped back off the stack. >But all this doesn't matter because we're not talking about a person, Lets not forget the "language" in LLM. Natural language is recursive in its construction. English is a subject-verb-object grammar, wherein the objects are themselves are sentence constructs. Take any sentence, preface it with "he said", and you have a new sentence. The words simultaneously describe the domain of discussion and are objects within that same domain. The brain does recursive tasks like language processing easily because it can feed information back to itself in a loop. For some dumb reason, current artificial networks don't let the neurons loop.
  - YeGoblynQueenne 4y ago
    >> Conversation is a stack! Eh, no. Say you start writing some text in a text editor. Then I write a bit. Then you write a bit. Then I write a bit. That's not recursion. It's not even iteration. It's just... adding stuff on top of other stuff. Exquisite Corpse, really. It's not recursive. Recursion is a wild horse. It runs away from you and it explodes in complexity: those are its characteristics in action, much more salient than its theoretical ones. Recursion is dangerous. If you have to compute it by hand, or help your little silly bot "run" it, by running it for it by hand... well then the bot is not running recursion. Nobody is, or your head would explode. But your head doesn't explode even if you start computing recursion. With your human mind you can see the undecidability of "this sentence is false" from a mile away. Really. I learned that as a joke when I was a schoolkid, we laughed at it and never thought twice about it. But a computer will get stuck in an infinite loop, because a computer is not human, and it can't see the things that humans can see, it can only compute, and sometimes it's just stuck computing until it runs out of resources because that's all it can do. As before, I think you're mechanomorphising humans and anthropomorphising machines. The "brain" may do whatever it likes, but computers aren't brains, and they don't do the things that brains do. And I have a sneaking suspicion that the "recursion" in human language that you allude to is something very different than what we are talking about here, also. But I'm no expert in human languages, so I can't really say.
    - IIAOPSW 4y ago
      Of all the angles you could take, you go with the neo-cartesian dualist "a computer can't replicate the brain". At some level you have to reconcile that a computer can do a physics simulation to whatever precision, and therefore either it can replicate a brain if by nothing else than sheer simulation, or you have to invoke some magical thinking about brains not being in the domain of physical law. How can you engage with AI, engage in conversation about AI, and also be rejecting the very premise that AI is possible? Now there's the one thing that really does explode my mind. >Eh, no. Say you start writing some text in a text editor. Then I write a bit. Then you write a bit. Then I write a bit. That's not recursion. It's not even iteration. It's just... adding stuff on top of other stuff. Exquisite Corpse, really. It's not recursive. Hey buddy, quick favor. Just zoom out on the page for a moment. You see all the conversational threads here. See how they are organized, right in front of your eyes, in a tree like structure? I don't think I need to explain how trees are inherently recursive. The recursive nature of conversation is staring you right in the face! You have managed to completely miss what is actually happening. He's not "holding the bots hand" or "running it for it by hand". He's not doing any computation on behalf of the model. He is merely jerry-rigging a means for the bot prompt itself. >And I have a sneaking suspicion that the "recursion" in human language that you allude to is something very different than what we are talking about here, also. But I'm no expert in human languages, so I can't really say. Strange, you knew exquisite corpse yet not colorless dreams? Its all trees maaaan. From the level of conversation on down to the level of the sentence. We organize information in hierarchies, but sometimes, I am a strange loop and the tree has no root. https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_furiously https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...

YeGoblynQueenne 4y ago

>> When you ask yourself a question, that's recursion (...) I don't know if any of that makes sense. I think you're misapplying some handy metaphors: you're anthropomorphising a computer and mechanomorphising a human. I have "localchost"? What, do I also have Perl scripts? Now I'm starting to feel like a character in a P. K. Dick story. But all this doesn't matter because we're not talking about a person, asking questions of themself. We're talking about someone doing things with a computer. And we know exactly what "recursion" means in the context of a computer. On a computer, then, if anyone wants to show "recursion" in LLMs, they better be able to show how to implement a push-down stack with the bot's conversation window. Then they can show how to calculate a factorial recursively, and how their calculation behaves differently when it's computed in a tail-recursive manner, and when it is not. Yeah, I can see what the author doing, and it's not recursion. But I'm trying to be kind so I won't say what it is.

IIAOPSW 4y ago
Don't you see it. Its giving chatGPT the ability to use chatGPT as a subroutine in formulating its response to a prompt. Conversation is a stack! When you pose a question (to yourself or anyone else), whatever line of thought you were working on stays paused until an answer is popped back off the stack. >But all this doesn't matter because we're not talking about a person, Lets not forget the "language" in LLM. Natural language is recursive in its construction. English is a subject-verb-object grammar, wherein the objects are themselves are sentence constructs. Take any sentence, preface it with "he said", and you have a new sentence. The words simultaneously describe the domain of discussion and are objects within that same domain. The brain does recursive tasks like language processing easily because it can feed information back to itself in a loop. For some dumb reason, current artificial networks don't let the neurons loop.
- YeGoblynQueenne 4y ago
  >> Conversation is a stack! Eh, no. Say you start writing some text in a text editor. Then I write a bit. Then you write a bit. Then I write a bit. That's not recursion. It's not even iteration. It's just... adding stuff on top of other stuff. Exquisite Corpse, really. It's not recursive. Recursion is a wild horse. It runs away from you and it explodes in complexity: those are its characteristics in action, much more salient than its theoretical ones. Recursion is dangerous. If you have to compute it by hand, or help your little silly bot "run" it, by running it for it by hand... well then the bot is not running recursion. Nobody is, or your head would explode. But your head doesn't explode even if you start computing recursion. With your human mind you can see the undecidability of "this sentence is false" from a mile away. Really. I learned that as a joke when I was a schoolkid, we laughed at it and never thought twice about it. But a computer will get stuck in an infinite loop, because a computer is not human, and it can't see the things that humans can see, it can only compute, and sometimes it's just stuck computing until it runs out of resources because that's all it can do. As before, I think you're mechanomorphising humans and anthropomorphising machines. The "brain" may do whatever it likes, but computers aren't brains, and they don't do the things that brains do. And I have a sneaking suspicion that the "recursion" in human language that you allude to is something very different than what we are talking about here, also. But I'm no expert in human languages, so I can't really say.
  - IIAOPSW 4y ago
    Of all the angles you could take, you go with the neo-cartesian dualist "a computer can't replicate the brain". At some level you have to reconcile that a computer can do a physics simulation to whatever precision, and therefore either it can replicate a brain if by nothing else than sheer simulation, or you have to invoke some magical thinking about brains not being in the domain of physical law. How can you engage with AI, engage in conversation about AI, and also be rejecting the very premise that AI is possible? Now there's the one thing that really does explode my mind. >Eh, no. Say you start writing some text in a text editor. Then I write a bit. Then you write a bit. Then I write a bit. That's not recursion. It's not even iteration. It's just... adding stuff on top of other stuff. Exquisite Corpse, really. It's not recursive. Hey buddy, quick favor. Just zoom out on the page for a moment. You see all the conversational threads here. See how they are organized, right in front of your eyes, in a tree like structure? I don't think I need to explain how trees are inherently recursive. The recursive nature of conversation is staring you right in the face! You have managed to completely miss what is actually happening. He's not "holding the bots hand" or "running it for it by hand". He's not doing any computation on behalf of the model. He is merely jerry-rigging a means for the bot prompt itself. >And I have a sneaking suspicion that the "recursion" in human language that you allude to is something very different than what we are talking about here, also. But I'm no expert in human languages, so I can't really say. Strange, you knew exquisite corpse yet not colorless dreams? Its all trees maaaan. From the level of conversation on down to the level of the sentence. We organize information in hierarchies, but sometimes, I am a strange loop and the tree has no root. https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_furiously https://en.wikipedia.org/wiki/Colorless_green_ideas_sleep_fu...