4 ms·

I know the rules of chess, but my rate of attempting illegal moves is above 0%.

• nickelpro 4y ago
  A human mind is not a rules model for Chess, and comparisons to a human mind are worthless. We're not asking if the LLM is human, only if it is modeling the game rules or modeling statistics My point is this: * If the results of the model were perfect, not a single counter-example, that would be a strong argument that the LLM has built an internal model of Othello * If counter-examples exist, the result of the experiment is indistinguishable from a sufficiently clever statistical inference and proves nothing
  • jackmott 4y ago
    [dead]
  • hcks 4y ago
    So you assert that parent doesn’t have an internal model of chess? Or that only proved expert systems built specifically to play chess have an internal model of it?
    • nickelpro 4y ago
      > So you assert that parent doesn’t have an internal model of chess? I think it's an interesting question, I find it more likely we have an imperfect ability to perceive the whole chess board, whatever the hell that might mean. In any case it's irrelevant to an LLM which has perfect information from the input set. So perhaps we rephrase the question, I think the parent, if given a list of individual chess moves (in whatever notation pleases them), should be able to tell if a given move is valid or not for the given piece. If the parent failed to do that, I would say they do not have a valid model of chess, ya.
      • shawnz 4y ago
        > If the parent failed to do that, I would say they do not have a valid model of chess Surely even the world's foremost chess experts will occasionally make errors in such a situation, so what's the point of holding language models to an intelligence standard that even humans can't meet? I see two ways of interpreting this: either language models do approximately learn world models, or else world models aren't necessary for human intelligence EDIT: I see from your other posts in this thread you're probably more inclined to believe the latter statement, which wasn't obvious to me from your first post and I think that explains why there is so much contention in this thread.
  • noiv 4y ago
    Well, if the model has a 100% grasp of the rules, why call it a model?
    • nickelpro 4y ago
      Because it's not a physical othello board? It's just an internal model of an othello board encoded in the weights of a neural net? Insert whatever semantics float your particular goat
      • noiv 4y ago
        I thought the whole point was about modelling rules not atoms.
• bjornsing 4y ago
  Especially if you’re playing blind. I’d say an error rate of less than %1 is pretty good in that case.