3 ms·

> You’re trying to avoid addressing my point. What can GPT do that’s interesting without a human in the loop doing the prompting? Oh, not much. Training a netw

> You’re trying to avoid addressing my point. What can GPT do that’s interesting without a human in the loop doing the prompting? Oh, not much. Training a network to do a prompted task is a lot easier than training a network to do unguided action selection because we don't really have a good textual dataset for that. > the chess engine can play checkers, poorly. Disagree: you're not gonna get any valid checkers moves out of the engine. It's a 0-to-1 thing. GPT-3 gets you to 1. There's a lot of Youtube videos of GPT-3 doing chess games that basically get through the opening and then fall over with invalid moves. This is the level of performance I refer to. > There are lots of games GPT can’t play, like hide-and-seek, tag, and tennis. Disagree: you can probably get GPT to play these. Show it a picture of the area and have it select a hiding spot or give you a search order. For tennis... something like have it write Python code for a tennis bot using image recognition? It'll probably fail, but it'll fail in the course of a recognizable attempt. > It doesn’t (currently) learn the rules from your prompts; you can’t teach it to play a new game by talking to it There's papers saying you're wrong, have you actually tried this? The whole novelty of 3+ was in-context learning.

dahart 2y ago
You moved your own goal posts. You said “very poor” quality counted, and referred to “any game imaginable”, not to only valid moves. We absolutely can ask GPT to play a game it doesn’t know, and fail to get any valid moves. I was trying to give your argument the benefit of the doubt, but I can’t do that when your argument is moving and inconsistent. You also failed to address the RNG, which always has a non-zero probability of returning a valid move. The papers you refer to do not say I’m wrong. In-context learning doesn’t stick, the neurons don’t change or adapt, the model doesn’t grow, and GPT forgets everything you told it… while you play the game I imagined. There are also papers demonstrating how bad GPT can be and reveal that it doesn’t understand, it’s just good at mimicking humans that understand. “ChatGPT is bullshit” https://link.springer.com/article/10.1007/s10676-024-09775-5 https://link.springer.com/article/10.1007/s10676-024-09775-5 (Note the important argument there that the more you insist that GPT has agency, the stronger the evidence becomes that its hallucinations are intentional lies and not just innocent accidents. Be careful what you wish for.) “GPT-4 can’t reason”. https://medium.com/@konstantine_45825/gpt-4-cant-reason-2eab795e2523 https://medium.com/@konstantine_45825/gpt-4-cant-reason-2eab... A fun example of LLMs being unable to understand their prompts is when you ask it not to do something. It picks up on the tokens you use and happily does the thing you ask it not to.
- FeepingCreature 2y ago
  > The papers you refer to do not say I’m wrong. In-context learning doesn’t stick, the neurons don’t change or adapt, the model doesn’t grow, and GPT forgets everything you told it… while you play the game I imagined. By this argument a human with anterograde amnesia is not a general reasoner. To be clear, I don't think GPT has significant or humanlike amounts of agency and understanding. I do think it has noticeable amounts of agency and understanding, whereas a RNG has no agency or understanding. In other words, GPT will try to play games and make valid moves at a level clearly above chance. > A fun example of LLMs being unable to understand their prompts is when you ask it not to do something. It picks up on the tokens you use and happily does the thing you ask it not to. Any human affected by reverse psychology: clearly not a general reasoner... This is strongly dependent on how you formulate the prompt. Generally, if you give GPT space to consider the thing but then change its mind, it will work. This is not dissimilar to how human consciousness exerts "veto power" over deliberate action, and how "do not think of a pink elephant" triggers our imagination to generate a pink elephant regardless of our wishes. In my experience, when writing prompts, if you treat the LLM's context window as its internal conscious narrative rather than its speech you usually get better results.
  - dahart 2y ago
    > a human with anterograde amnesia is not a general reasoner. Not what I claimed, straw man, and not generally true. Amnesia prevents some memory but not all learning. What is true, but irrelevant here, is that amnesia is a cognitive impairment. > Any human affected by reverse psychology: clearly not a general reasoner. This is another clearly false statement, and straw man with respect to GPT. Reverse psychology is not characterized by outright failure to understand a basic straightforward question. Nothing you’ve said so far demonstrates GPT has agency or initiative, that it will act without a human. Btw it doesn’t go unnoticed you didn’t respond to either link. Those papers make a stronger argument than me. Check em out.
    - FeepingCreature 2y ago
      > Not what I claimed, straw man, and not generally true. Amnesia prevents some memory but not all learning. What is true, but irrelevant here, is that amnesia is a cognitive impairment. The point is, if you blocked a human's ability to form permanent memory, this would not diminish its ability to function in the short term. GPT is incapable of forming long-term memories, but I don't think this says anything about the class of things it's capable of except in that it limits their extent in "time"/context space. > Reverse psychology is not characterized by outright failure to understand a basic straightforward question. You're treating the situation as if you are talking to a person; I think this is fundamentally a misunderstanding of what the context window is (admittedly, a very common one). You aren't talking to somebody, you're injecting sentences directly into the thing's awareness and reading back its reactions. In such a setup, even humans would often not respect negative conditionals- they literally wouldn't be capable of it. Thus again, the LLM's failure doesn't disprove anything. > Nothing you’ve said so far demonstrates GPT has agency or initiative, that it will act without a human. If you put a human in a harness where time only moves when another human tells it to do something, that human would also be unable to act without instruction. The fundamental setup of GPT makes it impossible for it to "act without a human" - it's not an issue of its cognitive architecture but of its runtime. Tell a GPT to "do something you enjoy" and I bet you get an answer back.

dahart 2y ago

You moved your own goal posts. You said “very poor” quality counted, and referred to “any game imaginable”, not to only valid moves. We absolutely can ask GPT to play a game it doesn’t know, and fail to get any valid moves. I was trying to give your argument the benefit of the doubt, but I can’t do that when your argument is moving and inconsistent. You also failed to address the RNG, which always has a non-zero probability of returning a valid move. The papers you refer to do not say I’m wrong. In-context learning doesn’t stick, the neurons don’t change or adapt, the model doesn’t grow, and GPT forgets everything you told it… while you play the game I imagined. There are also papers demonstrating how bad GPT can be and reveal that it doesn’t understand, it’s just good at mimicking humans that understand. “ChatGPT is bullshit” https://link.springer.com/article/10.1007/s10676-024-09775-5 https://link.springer.com/article/10.1007/s10676-024-09775-5 (Note the important argument there that the more you insist that GPT has agency, the stronger the evidence becomes that its hallucinations are intentional lies and not just innocent accidents. Be careful what you wish for.) “GPT-4 can’t reason”. https://medium.com/@konstantine_45825/gpt-4-cant-reason-2eab795e2523 https://medium.com/@konstantine_45825/gpt-4-cant-reason-2eab... A fun example of LLMs being unable to understand their prompts is when you ask it not to do something. It picks up on the tokens you use and happily does the thing you ask it not to.

FeepingCreature 2y ago
> The papers you refer to do not say I’m wrong. In-context learning doesn’t stick, the neurons don’t change or adapt, the model doesn’t grow, and GPT forgets everything you told it… while you play the game I imagined. By this argument a human with anterograde amnesia is not a general reasoner. To be clear, I don't think GPT has significant or humanlike amounts of agency and understanding. I do think it has noticeable amounts of agency and understanding, whereas a RNG has no agency or understanding. In other words, GPT will try to play games and make valid moves at a level clearly above chance. > A fun example of LLMs being unable to understand their prompts is when you ask it not to do something. It picks up on the tokens you use and happily does the thing you ask it not to. Any human affected by reverse psychology: clearly not a general reasoner... This is strongly dependent on how you formulate the prompt. Generally, if you give GPT space to consider the thing but then change its mind, it will work. This is not dissimilar to how human consciousness exerts "veto power" over deliberate action, and how "do not think of a pink elephant" triggers our imagination to generate a pink elephant regardless of our wishes. In my experience, when writing prompts, if you treat the LLM's context window as its internal conscious narrative rather than its speech you usually get better results.
- dahart 2y ago
  > a human with anterograde amnesia is not a general reasoner. Not what I claimed, straw man, and not generally true. Amnesia prevents some memory but not all learning. What is true, but irrelevant here, is that amnesia is a cognitive impairment. > Any human affected by reverse psychology: clearly not a general reasoner. This is another clearly false statement, and straw man with respect to GPT. Reverse psychology is not characterized by outright failure to understand a basic straightforward question. Nothing you’ve said so far demonstrates GPT has agency or initiative, that it will act without a human. Btw it doesn’t go unnoticed you didn’t respond to either link. Those papers make a stronger argument than me. Check em out.
  - FeepingCreature 2y ago
    > Not what I claimed, straw man, and not generally true. Amnesia prevents some memory but not all learning. What is true, but irrelevant here, is that amnesia is a cognitive impairment. The point is, if you blocked a human's ability to form permanent memory, this would not diminish its ability to function in the short term. GPT is incapable of forming long-term memories, but I don't think this says anything about the class of things it's capable of except in that it limits their extent in "time"/context space. > Reverse psychology is not characterized by outright failure to understand a basic straightforward question. You're treating the situation as if you are talking to a person; I think this is fundamentally a misunderstanding of what the context window is (admittedly, a very common one). You aren't talking to somebody, you're injecting sentences directly into the thing's awareness and reading back its reactions. In such a setup, even humans would often not respect negative conditionals- they literally wouldn't be capable of it. Thus again, the LLM's failure doesn't disprove anything. > Nothing you’ve said so far demonstrates GPT has agency or initiative, that it will act without a human. If you put a human in a harness where time only moves when another human tells it to do something, that human would also be unable to act without instruction. The fundamental setup of GPT makes it impossible for it to "act without a human" - it's not an issue of its cognitive architecture but of its runtime. Tell a GPT to "do something you enjoy" and I bet you get an answer back.