8 ms·
When do "brains beat brawn" in chess? An experiment
- ro_bit 3y agoI wonder if there would be a way to train Stockfish to be better at playing with odds. I do not know much about chess, but I doubt the AI was trained on how to play openings when starting at a disadvantage, so maybe one could see a vast improvement on AI playing with odds if the AI was specifically trained/designed to do so
- drdeca 3y agoI imagine this could be done with MuZero[edit: turns out I meant AlphaZero] without much fundamental difficulty? (Uh, I mean, could be done by a team which is able to train MuZero)
- Silphendio 3y agoLeela Chess Zero has been around for a while, and it's almost as strong as Stockfish. It used to be very bad at handicap games because it's trained to maximize the odds of winning and couldn't handle totally lost positions, but they improved it recently. Now it can play at GM level with knight odds.
- gbalduzzi 3y agoChess AI is not machine learning as in "it watched thousand of games and extracted some patterns to be good". It basically explores all possible moves and all possible countermoves and selects the best one. Of course it is extremely well engineered and optimized, but this is the main concept. As a consequence, it does not to be "trained" on a dataset with odds to play without a piece.
- pmayrgundter 3y agoI thought it does need to be trained (via self-play) to recognize patterns bc the game space is too large to exhaustively visit Looks like it used to be brute force, as you say, but has since incorporated an NN after ita loss to AlphaZero https://www.chess.com/terms/stockfish-chess-engine https://www.chess.com/terms/stockfish-chess-engine
- matteoraso 3y agoIt uses alpha-beta pruning[0], which requires an evaluation function. Stockfish traditionally used a man-made evaluation function, but added a small type of NN called an NNUE[1] for better results. The actual search method is still the same, though. [0]https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning https://en.wikipedia.org/wiki/Alpha%E2%80%93beta_pruning [1]https://www.chessprogramming.org/NNUE https://www.chessprogramming.org/NNUE
- pmayrgundter 3y agow00t, much better. thanks! Good names in there :)
- Der_Einzige 3y ago... Sort of. Yes, you're right to bring up minimax/alpha-beta pruning, but things like the correct values for piece square tables and the opening book are chosen based on data that is analogous to ML model training (sometimes these are directly chosen using ML models). Removing a piece may mess up the values chosen for the piece square tables and certainly messes up with the opening book - but on the other hand, it makes it sooner for the engine to end up using it's tablebases in the end game!
- chiefalchemist 3y agoRight. It's not intelligent. It just brute forces down a decision tree of possible moves / outcomes and picks the best next move that is most likely to result in victory. With a super computer (e.g., IBM's Watson) the program is able to go deeper down the tree and explore more branches in less time. Nothing fancy. Just pure brute force.
- rcxdude 3y agoBrute force alone is not very good for chess: you need some means of evaluating a position to prune your tree, otherwise you will not be able to plan very far ahead at all. The evaluation function is in large part what makes one chess engine better or worse than another, and nowadays that includes some neural nets for the top engines (stockfish, which is still the best AFAIK, uses a relatively small net and still evaluates some 100s of millions of positions, while Alphazero has a relatively larger net and evaluates about a thousand or so positions, and is also far better than any humans)
- chiefalchemist 3y agoGotta nitpick :) Brute force is good enough, at any given moment the # of possible moves is finite. The limiting factor is how much hardware you have and how quick you need the next move.
- rcxdude 3y agoSure, but unless you wanna wait a few thousand years on a pretty large cluster you're not going to get a chess engine that's gonna beat the best humans. The number of moves is finite but very, very large.
- charlieyu1 3y agoOpening books definitely play a significant part of chess AI. The early game is where the game tree is largest while the variation is lowest.
- seanhunter 3y agoStockfish doesn't need improvement at playing with odds to beat all humans with odds. The author is playing a 2-year old version (stockfish 14 vs 16 is current) on low depth (22 vs for the strength they use to get max rating I think they use at least 48 or so, which is clearly much stronger). If they played stockfish 16 on a decent depth it would be an annhilation I'm sure.
- seanhunter 3y agoTo understand the depth number, this is how deep in the move tree the model looks forward in order to evaluate positions. I think depth 22 corresponds to 22 ply ie 11 moves by each player. At each depth it might evaluate 8-10 candidate moves or so, so a depth of 48 is clearly vastly stronger, not just "a bit more than twice as strong" which it might sound like. I'm a very average chess player, and there are moves I play that stockfish at depth 16 things are a blunder that at depth 32 it thinks are a brilliancy. These are typically things in openings I play a lot where I've had a chance to do some homework and learn the best moves on a high depth.
- ALittleLight 3y agoI don't think the analogy to AI is apt because Stockfish isn't intelligent and life isn't a chess game. A one million IQ AI might be able to hack your chess game (or your OS) and win that way, or persuade you by chat to let it win (it could give you the cure for cancer and help you make some investments in the stock market - isn't that worth more than winning a game?) Life isn't a chess games either. It's not clear that a contest is happening or that only side will win. Humanity could, now, quickly stop AI. We could burn down all the data centers and chip fabs and kill everyone who knows linear algebra. But we don't because nobody ever sits down and says "Okay, the contest for existence starts now!" Instead, AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength.
- upwardbound 3y ago> AI capabilities increase year after year and month after month and the AI will simply be able to bide its time to start the game. To use the chess metaphor the table is set where humanity has all the pieces and AI only its king - but every hour AI adds another piece. If we never make a move we will face it at full strength. This is an excellent analogy and you’re right that it’s imperative we nip this in the bud.
- zmgsabst 3y agoThis logic says you should beat your child in case they grow up to be a violent criminal: A self-fulfilling prophecy that precipitates the very conflict it ostensibly avoids.
- the8472 3y agoOmega appears and tells you your child will have an probability P to be the next Mao, only conditional on the child surviving to adulthood. At which P do you drown the child?
- 3y ago
- gwnywg 3y ago> 'Not many kids have the patience to lose dozens of games in a row and never even get close to victory.' When I was ~8 my dad was teaching me to play. Not a single time he let me win, we could play like 100 games an evening (literally), he had incredible patience (sometimes I was taking lots of time to make a move, yet he never gave up on me)... It would have been more challenging to him (at least a little bit) if we knew the rule described in this article...
- tromp 3y agoHandicap play is more common in Go than in Chess. One reason is that the stronger player can naturally give a handicap by passing their first n turns. The ranking system helps determine the correct handicap; a 5 dan player would give 3 handicap to a 2 dan player for instance, and 5 stones to a 1 kyu player. Many Go tournaments, especially smaller ones that see a lot of spread in strengths within the kyu ranks, use handicap to make for more even sided games.
- mock-possum 3y agoOh that’s a fun mechanic. Interesting that you could just win outright if your chess opponent gave you five free moves out the gate. Maybe for chess you could add the proviso, “x free moves but you can’t move into a position where your piece is threatened”
- matheusmoreira 3y agoNon-standard pre-arranged boards would be a lot of fun. Anything that gets rid of the opening memorization thing would make it a vastly more fun game to play. The boards could be made asymmetrical for handicap. Maybe chess engines could even generate random starting positions given some target material or positional strength for black and white.
- misja111 3y agoFischer Random Chess sounds like something for you: https://en.wikipedia.org/wiki/Fischer_random_chess https://en.wikipedia.org/wiki/Fischer_random_chess
- matheusmoreira 3y agoThat's really cool. Looks like lichess supports it too... I'll give it a shot.
- d-z-m 3y ago> Anything that gets rid of the opening memorization thing would make it a vastly more fun game to play. Chess can still be fun even if you don't have much theory onboard. You don't have to play the Grünfeld. Just play "theory-lite" openings like the London, and learn as you go.
- pmayrgundter 3y agoNice piece
- Bermion 3y agoFrom the article: "It’s somewhat hard to outthink a missile headed for your server farm at 800 km/h.". Is it really though? Just copy yourself to another server/activate whatever copy you already made in a bot net. Computers are fast, they are the thing that is guiding the "800 km/h" missile in real time..
- chmod775 3y agoComputers/Software also are nowhere near self-sufficient and need an incredible amount of supporting infrastructure for manufacturing, maintenance, and electricity. The minimum infrastructure to manufacture computers and keep some software running (mining, intermediate products, lithography, electricity, etc.) probably can't be further compressed than the size of a large city, which doesn't just need to make computers, it also needs to maintain everything that makes computers. The supply chains are incredibly complicated and require a huge amount of diverse resources that need be sourced and refined. Calling this "fragile" would be the understatement of the century. On the other hand two humans in the middle of nowhere can keep the species going indefinitely. Software running on a piece of plastic and silicon is not going to win a war against humanity anytime soon.
- dazaidesu 3y agoin a game of attrition that might be true. But if an AI's goal isn't to survive itself then all it needs is some keys to a nuke.
- roenxi 3y agoThe chess aspect is interesting but the AI risk discussion is less powerful. AI risk isn't about whether the AI can beat humans at games - we already have humans that beat everyone else at games. Even taking real life scenarios as games we've already seen everything play out. The risk of AI is that it will make humans uneconomic, in the same way horses became uneconomic and were by and large sent to the glue factories. The entirety of the homo sapiens experience, all the strategies we have in the modern era and all the reliable tactics we use to stay ahead of other lifeforms rely on us being much better at pattern identification than literally anything else. Once AI are just better at everything - creative activities, running militarys, economic decisions, legal decisions, etc - then it starts to come down to entirely a question of what robotics is capable of for how long humans can hold out. We can fight an army, but not economics.
- icepat 3y agoThen the question is who owns the economically efficient robots. I don't expect that to be democratic.
- roenxi 3y agoOn that subject, have you seen aristocrats at work? That isn't even a line of defence, there is a 2-3 generation timer (optimistically) before we're looking at well educated halfwits. If they've got sentient robot commanders and there is any scope for them to go rogue, we don't have a human institution that'll stop that and certainly not something with a small group of - emphasis on the quote marks - "elites". If we're talking existential risk, this is like Rome v. Carthage. Humans aren't going to come back once we're a bit behind. Although I don't think there is much to do and humans are so dodgy that it isn't even obvious that we should resist if the opportunity comes to replace us with something that can be consistently rational and intelligent.
- klipt 3y agoHumans are about as rational and intelligent as you'd expect from any evolved species. Which is to say, our intelligence is mostly tuned to increase the number of descendants we have. If AIs are subject to evolutionary forces, the same will be true for them.
- kruxigt 3y ago[dead]
- FrustratedMonky 3y agoThis is a great analogy. Puts the threat into more understandable terms for people. Not for the jobs threat. 'it took ouuurrr joubbbs' threat. This is already happening, and will continue. But for the killer AI threat, the concept of 'material' in chess is good. As long as AI needs us for power, mining and manufacturing, we still have some advantage. Until it gets to the point where it has so much power that it can threaten us to keep the power on, keep manufacturing chips, then we can still maneuver and win. The whole problem with the movie "Colossus" is that they handed over nuclear launch control. So there was a threat of nuclear strike that forces humans to keep the power on. Without this, humans can come back, change direction. Like with this excellent chess analogy, there will be a time when the AI is smarter, but humans will still have more material. Lets say in 10-20 years we do get AGI, it is installed in F-16s. There will still be time, a gap period, where we can change direction. We see it has gone too far and we turn it off. The world realizes we are going to far and all countries come together to abandon AI. The real threat is humans, threats of other countries, and profit motives. This is what will keep AI moving forward and in control. Because we'll be to scared or greedy to turn it off. NOT because of its superior strategic thinking. MOLOCH.
- makach 3y agoThere was a period where Magnus Carlsen would drink on his streams and be drunk while outperforming his peers. So, even drunk Magnus would beat you at chess - most likely.
- leosanchez 3y agoDrunk Magnus carlsen can easily beat anyone below 2000.
- zelphirkalt 3y agoBelow 2000 is not exactly what I would call "his peers" though.
- drexlspivey 3y agoMore like 2650
- nekonek2 3y agoDrunken Magnus taking over technically lost position with the 2400+ opponent https://www.youtube.com/watch?v=K-Kz7bo5tKE https://www.youtube.com/watch?v=K-Kz7bo5tKE It's hilarious to watch
- leosanchez 3y agoI knew about this video but didn't notice opponent is 2400+
- default-kramer 3y agoBut the 2400 blunders the queen in the opening for seemingly no reason? Then Magnus gives it back to be nice? I don't know what's going on there, maybe they're both hammered.
- dale_glass 3y ago> It’s somewhat hard to outthink a missile headed for your server farm at 800 km/h. I'd say it's a far easier situation for the right AI than for humans. It's actually very hard for humans to think when they know bullets and missiles are flying in their direction. Imagine a war against an opponent that never blunders. Nobody is every drunk, or sleeping, or fails to pay attention. Nobody is arguing about what's the right thing to do now. Nobody's running around like a headless chicken while artillery is falling around them. Nobody forgets to make use of the abilities of the equipment. A war against an opponent that's always performing as well as it can would be quite the tricky scenario. Of course a rogue AI with an easy to attack center would be enormously vulnerable, but I think such a thing shouldn't be assumed, since the reason to build AIs is to fight better, and any flaws in control and communications would be quickly exploited. So an AI controlled army is almost certain to be distributed to a large extent.
- twic 3y ago> I'd say it's a far easier situation for the right AI than for humans. Actually this has already been studied in chess theory: https://www.youtube.com/watch?v=E2xNlzsnPCQ https://www.youtube.com/watch?v=E2xNlzsnPCQ
- devit 3y agoI seem to remember that Stockfish is known to be bad at handicap games and in general at maximizing odds of winning against humans from strongly losing positions.
- shric 3y agoI'm around the same rating as the OP. I've also experimented with playing Stockfish with queen odds, etc. One thing I noticed early on is that it's much much harder to beat a much higher rated human when they give me queen odds. Stockfish literally plays itself when finding the next move and because it's down a queen it just seeks further loss minimization and it won't assume the opponent will fall for tricky but unsound tactics. A high rated human on the other hand will easily exploit my lower tactical ability and be tricky to get the advantage back. This applies especially on fast time control games.
- measured_step 3y agoFor chess engines such as Stockfish to beat weaker opponents, you would want to turn up the contempt parameter.[1] To be able to do this, you would want to use a UCI-compliant chess GUI such as Cute Chess.[2] It lets you change the command line arguments you use for your engine, and you can play Human-vs-Human, Engine-vs-Engine, Engine-vs-Human, etc. [1] https://www.chessprogramming.org/Contempt_Factor https://www.chessprogramming.org/Contempt_Factor [2] https://cutechess.com/ https://cutechess.com/
- faeriechangling 3y agoContempt was removed from stockfish 12 due to NNUE being patched in. You’ll have to use stockfish 11 or another engine to use contempt.
- billforsternz 3y agoThis is a very good and much overlooked point. Engines don't look for tricks and traps that "might" work, they assume good play from the opponent. It's literally MiniMax (minimise the maximum the opponent can achieve), the foundational algorithm of chess AI. Consequently playing out superior positions against engines isn't as interesting as you might hope, it often feels they just give up. Grandmasters (and masters) on the other hand are absolutely awesome at tricking you out of your rightful reward on those odd occasions when you get a decent advantage against them (source: bitter experience:).
- pickdenis 3y agoLet's say an AGI exists and can do anything far, far better than humans. Why would it resist being turned off? Why would it care? How could it even have the capacity to care about whether it's turned off or on? Anthropomorphizing AGI is what leads to these silly thought experiments.
- wavemode 3y agoI believe the thinking is that LLM's already anthropomorphize themselves by training on human-written text. Without a system prompt telling a chatbot that it is a chatbot, it invariably claims to be human and acts as though it has feelings (and, honestly, why wouldn't it? Human text is written with human feelings.) Insufficient system prompting is what led to the Bing chatbot fiasco when it first came out. For the record, I don't personally believe LLM's, as they currently exist, could ever become AGI. But yeah, that's the popular thinking at least.
- kthejoker2 3y agoSurprisingly relevant clip from The Good Place https://m.youtube.com/watch?v=etJ6RmMPGko https://m.youtube.com/watch?v=etJ6RmMPGko
- xrd 3y agoI'm looking for ways to improve my chess knowledge. My kids are 10 and 8 and starting in chess club. I played chess as a kid but haven't played much in the last twenty years. What are some ways to improve at chess? Just go to lichess? Watch YouTube videos? I loved this article and the discussion of AI and would love to supercharge my learning if there are experimental ideas. With three kids, my time is very scattered so I'm looking for ways to study chess in small fifteen minute chunks, as that is often all I have before falling asleep in one of their beds. Bonus: are there ways to radically improve chess together (other than just playing, obviously!) Has anyone used flashcards effectively for chess?
- iLoveOncall 3y agoThere is GothamChess on YouTube which has a lot of tutorials and game analysis. I learned from there. Then on chess.com you have courses and puzzles that help too. You cannot really improve just by playing without learning theory.
- xrd 3y agoI'll check this out, thanks so much. I'm curious about how to teach theory to a ten year old. They are probably better at absorbing those things than I am, so maybe I'm not the best strategist anyway.
- creature_x 3y agoI actually built an app(spicychess.com) that's loosely modelled after the Anki flashcard method; You're given 10 minutes to solve a puzzle on your first attempt. On every subsequent attempt you're given half the time to solve the puzzle, until your 6th attempt where you're given 30 seconds to solve the puzzle. Repetition combined with strict time constraint has been shown to massively improve one's ability to spot tactics, and thereby increase one's rating, especially at the below 2000 level. You can also bookmark puzzles for future review (:
- xrd 3y agoI'll give it a try, sounds awesome!