3 ms·

HN is no different than Reddit, or any social media for that matter, in that commenters pretend to read articles.

by uncivilized 10d ago

xdavidliu 10d ago
that is if it even a human commenter at all
- linkjuice4all 10d ago
  State-sponsored psyop meta comments aside, the models obviously continue to get better, but there is still a lot of 'guard railing' required to keep even the latest models completely on-task. The chess example is interesting because it's clearly a well-studied and established domain so the rules, strategies, and whatever else is in the training data should make yield excellent results; but clearly there is some behavior in these systems that's difficult to engineer out.
  - YeGoblynQueenne 10d ago
    For me the useful intuition is that LLMs haven't somehow magickally learned to implement any of the algorithms we know that we have used to make strong chess engines: alpha-beta minimax and Monte-Carlo Tree Search on the one hand, and obviously the ability to learn accurate evaluation functions by self-play. I mean we've done all this before in a task-specific fashion. It's useful to know that LLMs haven't managed to do that in the process of learning to represent the entire text on the web. On the other hand they have gotten say very good at machine translation without being trained exclusively (and I select the preceding word carefully) on machine translation. Edit: I'm saying this because there is this idea expressed by e.g. Ilya Sutskever, that in order to predict the next token accurately an LLM has to learn something about all of underlying reality. See for example this interview with Dwarkesh: https://x.com/biobootloader/status/1640512444958396416 https://x.com/biobootloader/status/1640512444958396416 Where Sutskever claims that "Predicting the next token well means you understand the underlying reality that led to the creation of that token". If that were true, we should have seen LLMs play good chess by now. There is a huge amount of data on playing chess floating around on the web in the form of algebraic chess notation and if LLMs were capable of learning the "underlying reality" of chess, they would already have. They haven't. Because they can't. What Sutskever is saying flies in the face of literally hundreds of years of statistical modelling, which is to say, building predictive models that, very explicitly, do not have to understand any "underlying reality" and only have to be good at modelling a dataset.
    - 27183 10d ago
      But it speaks in words, therefore it must be super duper extra smart!!11 /s Sarcasm aside, I think this is an easy cognitive trap to fall into. It does sometimes feel like the LLM must have some world model because it converses somewhat coherently. Examples like this failure to understand chess, or to count the number of Rs in "strawberry", seem difficult to explain if the models are intelligent. But that doesn't stop people believing they are anyway. I think there must be something about the conversational interface that fools us easily. I wonder if people trained in interrogation techniques are also fooled?
      - YeGoblynQueenne 9d ago
        I think your sarcasm is justified. I, too, am tired by the big claims that are only based on hype.
    - hackinthebochs 10d ago
      >If that were true, we should have seen LLMs play good chess by now. Not at all. LLMs learn by imbibing a mass of relationships as isolated fragments of information. There is a certain amount of sorting and indexing that happens during the training phase. There is also a certain amount of compute executed on these relationships during inference. LLMs can model processes that fit within the compute budget. Language translation works well because language is lookup-heavy while being light on compute. Chess is a compute heavy game of finding the best move out of many possibilities with wide variation in the quality of each move. Humans cut through the compute requirements by reinforcement and learning intuition. LLMs don't get reinforcement on chess so they must compute during inference a unified model of chess. Developing a strong model of chess from raw fragments of information is simply not in their compute budget.
      - YeGoblynQueenne 9d ago
        >> Not at all. LLMs learn by imbibing a mass of relationships as isolated fragments of information. You gotta be careful how you use the word "relation" here because there's an informal meaning (I'm related to my cousin) and a more strict, formal meaning, that is used in computer science e.g. in the "Relational Calculus" etc. In the formal sense, the one relation that LLMs learn during training is the co-occurrence of tokens in a corpus of text, what's called more technically a "collocation" relation. Nothing says that this is enough to play chess, so I'm indeed doubtful that they can.
  - TheOtherHobbes 10d ago
    I'm not sure why anyone is expecting stochastic systems to be deterministic. Chess is a deterministic game won by a combination of known movesets and constrained multi-level forward search. LLMs do neither of these things. They don't reproduce training data exactly, their next response is more 'inspired by' prompts and its own memory than produced deterministically, and they don't have the capability to do general forward search on their own. So when you ask an LLM to play chess you're getting the equivalent of a very compressed and lossy JPEG of chess rules and strategies with added per-turn random noise. They also don't have the ability to design their own chess engine, although it would be interesting to see what happens if you ask for one.
    - YeGoblynQueenne 10d ago
      >> I'm not sure why anyone is expecting stochastic systems to be deterministic. Monte Carlo Tree Search is stochastic.
    - dezsiszabi 10d ago
      I'm expecting that they at least don't forget about pieces between turns, we're in AGI era after all, according to the tech overlords. I, as a human AGI, would jever just forget and remove a piece from the board from one turn to the next.
nalekberov 10d ago
[flagged]
avadodin 10d ago
Back in 2001, our social medium was Slashdot and no one ever pretended to read the article. No one read the article either. It was slashdotted most of the time anyways.
- _superposition_ 10d ago
  Oh shit he said slash dotted. Havent heard that in a long time!

HN is no different than Reddit, or any social media for that matter, in that commenters pretend to read articles.

xdavidliu 10d ago

that is if it even a human commenter at all

linkjuice4all 10d ago
State-sponsored psyop meta comments aside, the models obviously continue to get better, but there is still a lot of 'guard railing' required to keep even the latest models completely on-task. The chess example is interesting because it's clearly a well-studied and established domain so the rules, strategies, and whatever else is in the training data should make yield excellent results; but clearly there is some behavior in these systems that's difficult to engineer out.
- YeGoblynQueenne 10d ago
  For me the useful intuition is that LLMs haven't somehow magickally learned to implement any of the algorithms we know that we have used to make strong chess engines: alpha-beta minimax and Monte-Carlo Tree Search on the one hand, and obviously the ability to learn accurate evaluation functions by self-play. I mean we've done all this before in a task-specific fashion. It's useful to know that LLMs haven't managed to do that in the process of learning to represent the entire text on the web. On the other hand they have gotten say very good at machine translation without being trained exclusively (and I select the preceding word carefully) on machine translation. Edit: I'm saying this because there is this idea expressed by e.g. Ilya Sutskever, that in order to predict the next token accurately an LLM has to learn something about all of underlying reality. See for example this interview with Dwarkesh: https://x.com/biobootloader/status/1640512444958396416 https://x.com/biobootloader/status/1640512444958396416 Where Sutskever claims that "Predicting the next token well means you understand the underlying reality that led to the creation of that token". If that were true, we should have seen LLMs play good chess by now. There is a huge amount of data on playing chess floating around on the web in the form of algebraic chess notation and if LLMs were capable of learning the "underlying reality" of chess, they would already have. They haven't. Because they can't. What Sutskever is saying flies in the face of literally hundreds of years of statistical modelling, which is to say, building predictive models that, very explicitly, do not have to understand any "underlying reality" and only have to be good at modelling a dataset.
  - 27183 10d ago
    But it speaks in words, therefore it must be super duper extra smart!!11 /s Sarcasm aside, I think this is an easy cognitive trap to fall into. It does sometimes feel like the LLM must have some world model because it converses somewhat coherently. Examples like this failure to understand chess, or to count the number of Rs in "strawberry", seem difficult to explain if the models are intelligent. But that doesn't stop people believing they are anyway. I think there must be something about the conversational interface that fools us easily. I wonder if people trained in interrogation techniques are also fooled?
    - YeGoblynQueenne 9d ago
      I think your sarcasm is justified. I, too, am tired by the big claims that are only based on hype.
  - hackinthebochs 10d ago
    >If that were true, we should have seen LLMs play good chess by now. Not at all. LLMs learn by imbibing a mass of relationships as isolated fragments of information. There is a certain amount of sorting and indexing that happens during the training phase. There is also a certain amount of compute executed on these relationships during inference. LLMs can model processes that fit within the compute budget. Language translation works well because language is lookup-heavy while being light on compute. Chess is a compute heavy game of finding the best move out of many possibilities with wide variation in the quality of each move. Humans cut through the compute requirements by reinforcement and learning intuition. LLMs don't get reinforcement on chess so they must compute during inference a unified model of chess. Developing a strong model of chess from raw fragments of information is simply not in their compute budget.
    - YeGoblynQueenne 9d ago
      >> Not at all. LLMs learn by imbibing a mass of relationships as isolated fragments of information. You gotta be careful how you use the word "relation" here because there's an informal meaning (I'm related to my cousin) and a more strict, formal meaning, that is used in computer science e.g. in the "Relational Calculus" etc. In the formal sense, the one relation that LLMs learn during training is the co-occurrence of tokens in a corpus of text, what's called more technically a "collocation" relation. Nothing says that this is enough to play chess, so I'm indeed doubtful that they can.
- TheOtherHobbes 10d ago
  I'm not sure why anyone is expecting stochastic systems to be deterministic. Chess is a deterministic game won by a combination of known movesets and constrained multi-level forward search. LLMs do neither of these things. They don't reproduce training data exactly, their next response is more 'inspired by' prompts and its own memory than produced deterministically, and they don't have the capability to do general forward search on their own. So when you ask an LLM to play chess you're getting the equivalent of a very compressed and lossy JPEG of chess rules and strategies with added per-turn random noise. They also don't have the ability to design their own chess engine, although it would be interesting to see what happens if you ask for one.
  - YeGoblynQueenne 10d ago
    >> I'm not sure why anyone is expecting stochastic systems to be deterministic. Monte Carlo Tree Search is stochastic.
  - dezsiszabi 10d ago
    I'm expecting that they at least don't forget about pieces between turns, we're in AGI era after all, according to the tech overlords. I, as a human AGI, would jever just forget and remove a piece from the board from one turn to the next.

nalekberov 10d ago

[flagged]

avadodin 10d ago

Back in 2001, our social medium was Slashdot and no one ever pretended to read the article. No one read the article either. It was slashdotted most of the time anyways.

_superposition_ 10d ago
Oh shit he said slash dotted. Havent heard that in a long time!