3 ms·

I know a fair deal about the subject of chess AI, but when I was reading this and I didn't understand. I was polarized, was I reading a mastermind that was way

by TZubiri 9mo ago

I know a fair deal about the subject of chess AI, but when I was reading this and I didn't understand. I was polarized, was I reading a mastermind that was way above my level? Or someone way too confident that learned enough buzzwords through an LLM to briefly delude someone else other than themselves? A quick visit at the homepage suggests that it's probably the latter. I don't want to be rude, not posting out of malice, but if someone else was reading this and was trying to parse it, I think it might be helpful to compare notes and evaluate whether it's better to discard the article altogether.

Paracompact 9mo ago
Curious, what has you believe that? As someone who doesn't know much about chess AI, I was mostly able to follow along, and figured there were simply some prereqs the author wasn't explaining (e.g. distillation, test-time search, RLVR). If the article is deeply confused in some way I would indeed like to calibrate my BS detectors.
- TZubiri 9mo ago
  Just to confirm, did you read Cosmo's article (cosmo.tardis.uk black background), or the girl.surgery (white background) article? ML isn't my strong suit so I wouldn't be able to explain how, but Cosmo's article is almost entirely a refutation of the points made by the root article. No doubt he is very friendly, as someone would be to anyone interested in their field. What I can speak about is the general construction of sentences, they read (in the most charitable of interpretations) like text messages: "Good model vs bad model is ~200 elo, but search is ~1200 elo, so even a bad model + search is essentially an oracle to a good model without, and you can distill from bad model + search → good model." I take it that by "is ~X elo" they mean that implementing that strategy results in a gain of 200 ELO? Which would still be undefined, as 1000 to 1200 is not the same as 2800 to 3000, and improvements are of course not cumulative. I get that this reads more like internal notes, but it was published, so there was some expectation that it would be understood by someone else. For a lot more reasons, the writing reminds me of notes written by me or by loved ones under influence of drugs. My estimation is that the article was written by a mind that used to be brilliant but is now just echoing that brilliance while, trying to keep their higher order cognitive functions while struggling to maintain the baseline of basic language use. I hope it is reversible and if per is reading this and my estimation is correct, that they perturb the weights in favour of quitting drugs and see if they win more or not.
  - randomNumber7 9mo ago
    I personally don't believe the argument that search with a bad model helps so much. In e.g. an open position with lots of possibilities you would need an insane amount of calculations to beat a positional/strategic player with a bad engine.
    - TZubiri 9mo ago
      It all depends on the nuances of the search and the board evaluation. I've done a traditional search based engine, you can try playing against it. It plays out as you mention, you beat it positionally. But tbh if you play against a search only engine without heuristics but with a more optimized algorithm, you may still fail to penetrate it unless you know anti engine strategy. lichess.com/@/TotomiBot It's currently uses a 3ply exhaustive search, with the exception that takes don't count toward the ply limit, so it actually evaluates all branches up until the third non taking move. On the evaluation, it uses two separate scoring values, one for material, using Lasker style piece values, and another for tiebreaking, which would be the positional score. Positional score is mostly determined by a bitboard with for each piece type, with positive and negative biases towards specific squares, for exmaple the king bishop pawn is heavily incentivized to stay put. The boards are perspective based, so it works the same if you are black or white without needing symmetry (which would make promotion strategies hard). There's also a couple of heuristics for king safety. The complexity has come to a point where it's hard to predict what will improve or make it worse by just fiddling with the heuristics. But there's probably a lot of room for improvement in terms of reducing and optimizing compute time. AI is python based but calculation (search) and evals are offloaded to a c lib for efficiency. ELO is around 1400, and you can pretty much only beat it positionally (or with a very aggressive early sacrifice. As there's almost no hanging of pieces. https://github.com/TZubiri/TotomiChessAI https://github.com/TZubiri/TotomiChessAI
  - yorwba 9mo ago
    > 1000 to 1200 is not the same as 2800 to 3000 Elo is defined such that the expected win-rate of a player should only depend on the difference in Elo rating to their opponent. https://en.wikipedia.org/wiki/Elo_rating_system#Mathematical_details https://en.wikipedia.org/wiki/Elo_rating_system#Mathematical...
potsandpans 9mo ago
This comment is another example of an "llm psychosis" that is currently occuring in common discourse. The mass delusion of, "I don't understand what I'm reading, therefore it must be produced by an llm." I think it's a pretty serious problem. Not that llm text exists on the internet, but that reasonable people are reflexively closed off to creativity because the mere existence of the possibility that something is created by an llm is in their minds grounds for disqualification.
- TZubiri 9mo ago
  Nono, the claim is not that it is produced by an llm, rather that author researches the subject with llms and generally is a high frequency user. A common property of llm psychosis is the development of an internal vocabulary that the llm learns, often reusing words but adopting specific meanings, for some reason quantum and quantic are very popular for this.
  - wizzwizz4 9mo ago
    I didn't see any of that in either article.
why_only_15 9mo ago
The chess people seemed to think my article was reasonably accurate. But I'm not really sure.

by TZubiri 9mo ago

Paracompact 9mo ago

Curious, what has you believe that? As someone who doesn't know much about chess AI, I was mostly able to follow along, and figured there were simply some prereqs the author wasn't explaining (e.g. distillation, test-time search, RLVR). If the article is deeply confused in some way I would indeed like to calibrate my BS detectors.

TZubiri 9mo ago
Just to confirm, did you read Cosmo's article (cosmo.tardis.uk black background), or the girl.surgery (white background) article? ML isn't my strong suit so I wouldn't be able to explain how, but Cosmo's article is almost entirely a refutation of the points made by the root article. No doubt he is very friendly, as someone would be to anyone interested in their field. What I can speak about is the general construction of sentences, they read (in the most charitable of interpretations) like text messages: "Good model vs bad model is ~200 elo, but search is ~1200 elo, so even a bad model + search is essentially an oracle to a good model without, and you can distill from bad model + search → good model." I take it that by "is ~X elo" they mean that implementing that strategy results in a gain of 200 ELO? Which would still be undefined, as 1000 to 1200 is not the same as 2800 to 3000, and improvements are of course not cumulative. I get that this reads more like internal notes, but it was published, so there was some expectation that it would be understood by someone else. For a lot more reasons, the writing reminds me of notes written by me or by loved ones under influence of drugs. My estimation is that the article was written by a mind that used to be brilliant but is now just echoing that brilliance while, trying to keep their higher order cognitive functions while struggling to maintain the baseline of basic language use. I hope it is reversible and if per is reading this and my estimation is correct, that they perturb the weights in favour of quitting drugs and see if they win more or not.
- randomNumber7 9mo ago
  I personally don't believe the argument that search with a bad model helps so much. In e.g. an open position with lots of possibilities you would need an insane amount of calculations to beat a positional/strategic player with a bad engine.
  - TZubiri 9mo ago
    It all depends on the nuances of the search and the board evaluation. I've done a traditional search based engine, you can try playing against it. It plays out as you mention, you beat it positionally. But tbh if you play against a search only engine without heuristics but with a more optimized algorithm, you may still fail to penetrate it unless you know anti engine strategy. lichess.com/@/TotomiBot It's currently uses a 3ply exhaustive search, with the exception that takes don't count toward the ply limit, so it actually evaluates all branches up until the third non taking move. On the evaluation, it uses two separate scoring values, one for material, using Lasker style piece values, and another for tiebreaking, which would be the positional score. Positional score is mostly determined by a bitboard with for each piece type, with positive and negative biases towards specific squares, for exmaple the king bishop pawn is heavily incentivized to stay put. The boards are perspective based, so it works the same if you are black or white without needing symmetry (which would make promotion strategies hard). There's also a couple of heuristics for king safety. The complexity has come to a point where it's hard to predict what will improve or make it worse by just fiddling with the heuristics. But there's probably a lot of room for improvement in terms of reducing and optimizing compute time. AI is python based but calculation (search) and evals are offloaded to a c lib for efficiency. ELO is around 1400, and you can pretty much only beat it positionally (or with a very aggressive early sacrifice. As there's almost no hanging of pieces. https://github.com/TZubiri/TotomiChessAI https://github.com/TZubiri/TotomiChessAI
- yorwba 9mo ago
  > 1000 to 1200 is not the same as 2800 to 3000 Elo is defined such that the expected win-rate of a player should only depend on the difference in Elo rating to their opponent. https://en.wikipedia.org/wiki/Elo_rating_system#Mathematical_details https://en.wikipedia.org/wiki/Elo_rating_system#Mathematical...

potsandpans 9mo ago

This comment is another example of an "llm psychosis" that is currently occuring in common discourse. The mass delusion of, "I don't understand what I'm reading, therefore it must be produced by an llm." I think it's a pretty serious problem. Not that llm text exists on the internet, but that reasonable people are reflexively closed off to creativity because the mere existence of the possibility that something is created by an llm is in their minds grounds for disqualification.

TZubiri 9mo ago
Nono, the claim is not that it is produced by an llm, rather that author researches the subject with llms and generally is a high frequency user. A common property of llm psychosis is the development of an internal vocabulary that the llm learns, often reusing words but adopting specific meanings, for some reason quantum and quantic are very popular for this.
- wizzwizz4 9mo ago
  I didn't see any of that in either article.

why_only_15 9mo ago

The chess people seemed to think my article was reasonably accurate. But I'm not really sure.