3 ms·

>An LLM will learn anything that helps it predict I'm not sure you quite understand the full meaning of this statement. If you did, your following paragraphs w

>An LLM will learn anything that helps it predict I'm not sure you quite understand the full meaning of this statement. If you did, your following paragraphs wouldn't follow.

HarHarVeryFunny 16d ago
Are you imagining that an LLM tasked with predicting a game continuation is going to play to win instead?
- famouswaffles 15d ago
  I imagine it will learn to win under some circumstances, perhaps in a case with some context expressing a desire to win. Drawing out an LLMs upper ability in the game should be fairly straightforward.
  - HarHarVeryFunny 15d ago
    If you asked it to try to win, to "plan lines step by step", etc, then it would do it's best to follow that instruction, but unless RLVR trained to reason about chess (easy to do, but not sure which models may have done it) then it'd have to instead rely on the chess reasoning it had seen during pre-training (post-game interviews etc), which I doubt is enough to do very well. However, if you just ask it to continue a game, halfway in progress, then by default it will try to predict the most likely continuation, which is that both players will continue to play at the level they have done so far. This isn't a theory - it's been documented, as well as what you'd expect.
    - famouswaffles 15d ago
      I mean sure, but I'm not sure what that has to do with the broader point. It will learn to play, and it will have a model of what it means to win.
      - HarHarVeryFunny 15d ago
        > I'm not sure you quite understand the full meaning of this statement. If you did, your following paragraphs wouldn't follow I was just explaining how this comment you made is wrong.
        famouswaffles 15d ago
        It's not wrong. You admit that LLMs will 'learn anything that helps them predict' and fail to realize the breadth of that statement. Your chess statements don't really help your case. It doesn't matter that it usually doesn't primarily care about winning. It still learnt how to play the game, and it still knows how to win. Similar outcomes for predicting emotions would mean it still developed an affective state, and that its ability to 'feel angry' is no less real.
        HarHarVeryFunny 15d ago
        I said an LLM will learn anything that helps it to predict, then gave examples of playing chess by prediction and predictive emotions, both of which you seem to now accept, so you are now accepting that my "following paragraphs" did in fact follow. Go figure! You want to argue that predictive emotions are just as real as animal emotions, but that doesn't stop them from being predictive (and that AI that smiles as it kills you still seems concerning). ¯\_(ツ)_/¯
        famouswaffles 15d ago
        We are talking past each other now I think. Correct me if I'm wrong but it doesn't look like the possibility of LLMs having qualia even registers to you because it's 'predictive emotions'. There's no better way to predict an angry response than to be angry, qualia and all. If transformers could 'learn whatever it needs to predict text', then that potentially includes the feeling of anger. You are making some kind of distinction between 'predictive emotions' and the kind that happens when get a promotion (or get passed on a promotion) and I'm telling you that if you really understood what you said, you'd realize it is possible the machine is experiencing it the same.

>An LLM will learn anything that helps it predict I'm not sure you quite understand the full meaning of this statement. If you did, your following paragraphs wouldn't follow.

HarHarVeryFunny 16d ago

Are you imagining that an LLM tasked with predicting a game continuation is going to play to win instead?

famouswaffles 15d ago
I imagine it will learn to win under some circumstances, perhaps in a case with some context expressing a desire to win. Drawing out an LLMs upper ability in the game should be fairly straightforward.
- HarHarVeryFunny 15d ago
  If you asked it to try to win, to "plan lines step by step", etc, then it would do it's best to follow that instruction, but unless RLVR trained to reason about chess (easy to do, but not sure which models may have done it) then it'd have to instead rely on the chess reasoning it had seen during pre-training (post-game interviews etc), which I doubt is enough to do very well. However, if you just ask it to continue a game, halfway in progress, then by default it will try to predict the most likely continuation, which is that both players will continue to play at the level they have done so far. This isn't a theory - it's been documented, as well as what you'd expect.
  - famouswaffles 15d ago
    I mean sure, but I'm not sure what that has to do with the broader point. It will learn to play, and it will have a model of what it means to win.
    - HarHarVeryFunny 15d ago
      > I'm not sure you quite understand the full meaning of this statement. If you did, your following paragraphs wouldn't follow I was just explaining how this comment you made is wrong.
      - famouswaffles 15d ago
        It's not wrong. You admit that LLMs will 'learn anything that helps them predict' and fail to realize the breadth of that statement. Your chess statements don't really help your case. It doesn't matter that it usually doesn't primarily care about winning. It still learnt how to play the game, and it still knows how to win. Similar outcomes for predicting emotions would mean it still developed an affective state, and that its ability to 'feel angry' is no less real.
        HarHarVeryFunny 15d ago
        I said an LLM will learn anything that helps it to predict, then gave examples of playing chess by prediction and predictive emotions, both of which you seem to now accept, so you are now accepting that my "following paragraphs" did in fact follow. Go figure! You want to argue that predictive emotions are just as real as animal emotions, but that doesn't stop them from being predictive (and that AI that smiles as it kills you still seems concerning). ¯\_(ツ)_/¯
        famouswaffles 15d ago
        We are talking past each other now I think. Correct me if I'm wrong but it doesn't look like the possibility of LLMs having qualia even registers to you because it's 'predictive emotions'. There's no better way to predict an angry response than to be angry, qualia and all. If transformers could 'learn whatever it needs to predict text', then that potentially includes the feeling of anger. You are making some kind of distinction between 'predictive emotions' and the kind that happens when get a promotion (or get passed on a promotion) and I'm telling you that if you really understood what you said, you'd realize it is possible the machine is experiencing it the same.