9 ms·
Stockfish Wins Chess.com Computer Championship
- tromp 9y agoUnfortunately, Stockfish is not doing too well in the more recognized TCEC unofficial world computer chess championship at http://tcec.chessdom.com/live.php http://tcec.chessdom.com/live.php, due to lackluster performance against the weaker engines. With just over 3 rounds to go, it looks like it's out of the final: N Engine Rtng Pts Gm SB Ho Ko St Fi Ch Gi Bo An 1 Houdini 6.02 3184 17.0 25 187.25 ···· =01 === ==1= 111= 1=1= 1=1 =1== 2 Komodo 1959.00 3232 17.0 25 183.25 =10 ···· ==0 ==1= =1=1 11= =1=1 1=11 3 Stockfish 051117 3228 15.0 24 173.25 === ==1 ···· =1= ==== =11= =1= 1=== 4 Fire 6.2 3112 13.5 25 152.25 ==0= ==0= =0= ···· ==1 ===1 =1= 1==1 5 Chiron 251017 3013 9.5 25 114.00 000= =0=0 ==== ==0 ···· === ==== === 6 Ginkgo 2.01 3052 9.5 25 110.00 0=0= 00= =00= ===0 === ···· ==== 1== 7 Booot 6.2 3091 9.0 24 104.75 0=0 =0=0 =0= =0= ==== ==== ···· === 8 Andscacs 0.921 3100 8.5 25 107.25 =0== 0=00 0=== 0==0 === 0== === ···· It had better win its current game against Booot to keep some very slim chances. Which it now seems to be doing...
- _0ffh 9y agoI suppose I depends on the format. It seems like in a knockout style tournament Stockfish would surely win, as it's the only engine with zero losses. Looks like it might benefit from a more aggressive play style, in order to get more wins against weaker opponents.
- brownbat 9y agoThere's interesting metagame theory here... It should get more aggressive whenever it's behind its goal rank in the tournament. You can often trade variance for performance in competing strategies. The amount you're trading depends on your opponent's skill and strategy. If an engine has a way to learn the skill of its opponent, that might be another way to determine the ideal level of aggressiveness too. Also if you're behind in a game, cranking variance is often your best shot. Do any engines metagame yet? That could make things crazy.
- Scarblac 9y agoThe rules of the tournament forbid it, it's the same program in all rounds and current standings aren't an input. So there is also no luck for any engine getting a favourable order of opponents or so.
- brownbat 9y agoEven just with the input from the current game, the engine could get more aggressive by estimating the skill of its opponent.
- tephra 9y agoWorth noting that Stockfish is actually the only engine that hasn't lost a game. The lackluster performance is it just not beating the weaker engines, something that Komodo and Houdini have done (but they have also lost against stronger engines).
- umanwizard 9y agoSo can we conclude that Stockfish just plays more "drawishly" than the other top-tier engines? Or is this effect just random chance so far?
- Recursing 9y agoIt might have something to do with the "contempt" setting used by the developers for this tornaument
- pizza234 9y agoI'm not an expert, but it may be the equivalent of a strong human defensive player, e.g. Petrosian (https://en.wikipedia.org/wiki/Tigran_Petrosian https://en.wikipedia.org/wiki/Tigran_Petrosian).
- kregasaurusrex 9y agoScoring conditions for chess tournaments cause forcing a draw to be more favorable than a loss, likely causing the programmers to heavily weigh draws over losses. Contrast this with Go engines where there are no draws, the incentives for finishing chess games in a draw that would otherwise end in a loss is much higher.
- thom 9y agoThis is true, Stockfish expects its opponent to play as well as itself (outside of the 'contempt' setting, not sure if that was used or tuned per game). So it won't play edgy lines that invite blunders if they are even a centipawn worse than what it thinks is the 'correct' line. Funnily enough, I feel there's a lesson for startup people in this. I have seen many people (myself included) talk themselves out of big, bold bets just because they are very good at thinking up reasons something should never work.
- schoen 9y agoAlthough Stockfish beat Booot, it then tied against Komodo, giving Komodo 17.5 Houdini 17 Stockfish 16.5 It's quite bad for Stockfish now because each of these three top engines has 1 remaining remaining match it's pretty likely to win (against Ginkgo, Booot, and Fire, respectively), which if it happens wouldn't change the relative ranking of the top three at all (they would be at 18.5, 18, and 17.5). Also, Stockfish has a match against Houdini, while Houdini and Komodo have a match against each other. Stockfish can still in principle reach the final this way if Houdini and Komodo have a draw, because that would give Komodo 19.5 Stockfish 18.5 Houdini 18.0 assuming all three engines also win their respective remaining matches against non-top-3 engines, and Stockfish defeats Houdini. Very tough for Stockfish at the moment!
- schoen 9y agoBooot achieved a draw against Houdini and Ginkgo achieved a draw against Komodo! Therefore Stockfish's potential paths to the final have increased; for example, if Stockfish can beat Houdini and draw against Fire, while Komodo beats Houdini, then Stockfish would be in the final against Komodo. Or if Houdini defeats Komodo and Stockfish wins against both Houdini and Fire, Stockfish would be in the final against Houdini. Edit: But now I'm not sure of the original comment that only the two top engines from this round will go to the final; in previous years there were more rounds before the final, but I don't understand how many of the rounds have been eliminated this year.
- schoen 9y agoPeople in the TCEC chat definitely seem to agree with the interpretation that only the top two will go to the final. It's kind of down to the wire now: currently Komodo and Houdini lead Stockfish by exactly one point, while Komodo is playing Houdini in the final game for both of them. Stockfish has exactly one game remaining (against Fire). In order to reach the final, Stockfish needs to win its game against Fire, and also Komodo and Houdini need not to draw. Then Stockfish would be tied with whichever of Komodo and Houdini loses the current game, and I guess there would be a tiebreaker prior to the final. If Komodo and Houdini draw now, they're both guaranteed to face each other in the final. People in the TCEC chat are pointing out that if they were human players, they would face a strong incentive to deliberately play for a draw or to accept one if one were offered. But presumably the software has no idea of this!
- tromp 9y agoWith only one round left, Stockfish is hanging by a thread: N Engine Rtng Pts Gm SB Ho Ko St Fi Ch Gi An Bo 1 Houdini 6.02 3184 18.0 27 219.00 ···· =01 ==== ==1= 111= 1=1= =1== 1=1= 2 Komodo 1959.00 3232 18.0 27 214.75 =10 ···· ==0= ==1= =1=1 11== 1=11 =1=1 3 Stockfish 051117 3228 17.0 27 215.75 ==== ==1= ···· =1= ==== =11= 1=== =1=1 4 Fire 6.2 3112 15.0 27 180.50 ==0= ==0= =0= ···· ==1= ===1 1==1 =1=1 It needs to beat Fire, while Komodo vs Houdini needs to be decisive. Fingers crossed...
- chesscom 9y agoKeep in mind, this was a “Rapid” event with quicker than normal time controls. It was meant to be an exciting event for fans and to test the engines at quicker speeds that humans can both follow and enjoy. It certainly produced exciting games (in my opinion)!
- tim333 9y agoI wonder how a machine learning approach like AlphaGo zero would do.
- tromp 9y agoI think that on typical desktop hardware, there's little doubt that its cycles are much better spent on alpha beta search (with all its modern enhancements like late move reduction) and hand built evaluation function than on evaluating neural nets. If, however, your hardware were to include a few tensor processing units, then the answer is less clear (but I'd still bet on alpha beta).
- Recursing 9y agoI think it's really hard to parallelize alpha beta search, there might be a point where other approaches become more useful
- ZirconiumX 9y agoParallelising A-B has been done, but even the state of the art approaches like Lazy SMP (where there is no synchronisation between threads, and only at the root is effort made to disperse threads), YBWC (where only the first node is synchronised, and if it doesn't produce a cut is then parallelised) and DTS (where the threads choose their own work points) produce sub-linear scaling. I think research needs to be done on algorithms that are designed from the start to be parallel, but I'm not smart enough to do that.
- bluetwo 9y agoAlphaGo Zero is really about the MCTS not neural nets.
- Jach 9y agoMore specifically it's about driving MCTS with neural nets rather than pure random sampling as prior strong (but quite far from pro level) go bots were. Of course before AlphaGo was released there were papers on a pure deep net version of a go bot that beat Pachi and Fuego (both MCTS), not sure if it beat CrazyStone of the time.
- TwoBit 9y ago$1000 prize money? Can't they crowd source something more than that?
- mattnewton 9y agoThese people are all skilled programmers doing high speed analytics. They aren’t doing it for the money.
- gregorymichael 9y agoIf you're into chess, I'd encourage you to check out the last of three annotated games in that article (Stockfish v Houdini). Stockfish mates on move 174. Game would have been declared a draw by human players by move 40.
- sleepyandlazy 9y agoWhen I read your comment, I thought the reason Stockfish won was because the computer found an innovative tactic that a human would not see. Watching the game, it looks like if there was a human player instead of the Houdini computer, the human would be easily able to force the game into a draw instead of a loss. Nonetheless, as an intermediate chess player, the games were fun to watch because of how different the play style computers can sometimes have versus humans.
- EGreg 9y agoThat's because it WAS a draw if Black played correctly - by blocking all the pawns from advancing and only moving the rook back occasionally if attacked by the bishop (although probably it would have remained a draw even if Black had given up the exchange. There was no way for white to get in there.
- glinscott 9y agoCongratulations to Stockfish! The community is amazing, and the patches keep on flowing. The sheer number of ideas is pretty incredible. If you are interested in contributing, head over to http://tests.stockfishchess.org/tests http://tests.stockfishchess.org/tests. You can submit a test, and it will be run by the virtual cluster of user donated machines. It's been over four years since I put fishtest up, and in that time, there have been over 20,000 tests submitted. The really cool thing is that this distributed testing framework is only possible with an open source engine. So instead of being a disadvantage (everyone can read your ideas), it turns into an advantage!
- octalmage 9y agoThis is super cool! Are there any posts about how the fishtests work? I love reading about how people solve interesting testing problems.
- glinscott 9y agoThere are a ton scattered around :). Here is the announcement of fishtest on the talkchess forum: http://talkchess.com/forum/viewtopic.php?t=47885&highlight=sprt http://talkchess.com/forum/viewtopic.php?t=47885&highlight=s... Initial discussion of the introduction of SPRT into fishtest, which led to a dramatic increase in our ability to measure improvements in self-play, in a statistically sound manner: https://groups.google.com/forum/?fromgroups=#!searchin/fishcooking/sprt|sort:date/fishcooking/cgMSMBUC7JQ/fkdjLTiDuwMJ https://groups.google.com/forum/?fromgroups=#!searchin/fishc... SPRT background here: https://en.wikipedia.org/wiki/Sequential_probability_ratio_test https://en.wikipedia.org/wiki/Sequential_probability_ratio_t... Basically, we use a two-phase test to maximize testing resources. First a short time control test (15s/game), using more lenient SPRT termination criteria, then, a long time control (60s/game) test using more stringent criteria. That combined with setting the SPRT bounds to allow us to measure 2-3 ELO improvements has allowed the progress of Stockfish to be almost only improvements. Previously when developing an engine, you'd make 10 changes, and if you were lucky, 2 or 3 would be good enough to make up for the other bad or neutral ones. If you look at the graphs on http://www.sp-cc.de/ http://www.sp-cc.de/, you can see that it just keeps getting better, one small improvement at a time.
- sethbannon 9y agoIncredible that now the best engine (Stockfish) is open source engine and the best server / app (Lichess) is also open source. Well done chess community!
- sova 9y agoLichess is blazing fast and has so many fun variants. I enjoy playing Chess960 on there and I am always impressed with the "training mode" for Standard chess, because the puzzles it throws at you are historical blunders from people actually playing games online!
- wallace_f 9y agoLichess is all of the above, with flying colors: very well-designed user experience, pleasing aesthetics, responsive, coded well, FOSS.
- a13n 9y agoWhy is Lichess better than chess.com? I've used both and can't really say one is the "best".
- colmvp 9y agoI really wish I could customize Lichess to have an aesthetic similar to Chess.com chessboard/pieces. I prefer Lichess, but what I have found with Chess.com's aesthetic it sounds and looks similar to an old classic board but not in a super cheesy way whereas the iconography used on Lichess feel very stockish.
- thom 9y agoThe Lichess mobile app, at least, has themes that can make it as ugly as Chess.com, if that's what you're after. :)
- noir_lord 9y agoOn the desktop you can, I used Greasemonkey to completely customize the appearance and use the pieces from Fritz when I used it a while back. Don't know if it'll still work but it did a year ago and it's a starting point. https://kopy.io/gZy5n https://kopy.io/gZy5n
- znpy 9y agoWhere did the good old Fritz go ?
- noir_lord 9y agoFritz the engine hasn't been very strong for a few years, Fritz the UI (which also allows you to use other engines) is still the best thing out there for learning/play. They seem to have focussed more on making the Fritz engine a good tool for human analysis rather than the strongest possible engine. Not sure what the new stuff is like the last one I bought was Fritz 12 some time ago.
- srk72 9y agoI would also try that
- Someone 9y ago”One major facet of the tournament format was that there were no opening books” Code and data are equivalent (for example, instead of an opening book that says “if you play white, open with pawn E4” one could have an evaluation function that says “after the first move of white, having a pawn on E4 is worth a million points”), so how did they enforce that rule?
- chki 9y agoHardcoding an opening would be considered playing with an opening book. Regarding the enforcement of this rule: As all chess engines that competed are pretty well known (and sometimes even open source), I'm pretty sure that they (rightfully) trust the programmers not to do anything against the rules.
- gamegoblin 9y agoI wonder if someone were to come up with an neural network based, AlphaGo-style engine that was competitive with these other top engines, how they'd deal with these sort of rules. An "opening book" of sorts would likely be baked into the neural nets, unable to be turned on or off. AlphaGo, for example, has definitely played well-known human openings (and innovated on them) despite having no opening book.
- Recursing 9y agoAll these top chess engines definitely play well-known human openings and innovate on them, they just happen to be the best sequences of moves in certain situations (as far as we know) The rule is against "explicit" hardcoded opening books, to force the engines to calculate the best move each time, to encourage variation and (I think) to allow the developers to focus on building stronger engines instead of managing huge opening books. The encourage variation part didn't really succeed in this format, as they all tend to converge to the same openings
- thom 9y agoIt's sort of funny to consider that our opening books are the product of several hundred years of incredibly slow, inefficient Monte Carlo tree search done by humans playing over wooden boards. Seems odd to deny that to engines that can knock it all out in a weekend anyway.
- nindalf 9y ago> That said, there was some serious computer science behind the event, as each engine played from a powerful Amazon Web Services computer. Love the subtle product placement :D
- Jabbles 9y agoWithout wishing to denigrate the achievements of all the people involved in this tournament, nor the entertainment aspect I'd like to ask: What is the statistical power of a 90-game round robin? Would this be a publishable result with p < 0.05 (or the new 0.005) against the null hypothesis that Stockfish and Houdini (2nd place) were of equal skill?
- ykler 9y agoI don't know if you can really put a p-value on this result without a more specific null hypothesis, but anyway it looks like this tournament result provides extremely weak evidence that Stockfish is better than Houdini. In the round robin component, Stockfish and Houdini played two games against each other, each winning one and losing one. In the "superfinal", they had 15 draws, Stockfish won 3 games, and Houdini won 2 games.
- tranv94 9y agoGood to know I’ve been using the best to destroy my opponents on chess.com
- ZenoArrow 9y agoIsn't that cheating?
- slphil 9y agohttps://www.chess.com/member/tranv94 https://www.chess.com/member/tranv94 I can see why you would think you would need an engine to actually win games. Have you tried learning how to play chess? To those downvoting me: you should know how much chess players detest engine cheaters playing online.
- sctb 9y agoPersonal attacks are not OK on Hacker News. Could you please take a look at the guidelines? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- rurban 9y agoHoudini is still my favorite engine though. It plays much more interesting than Stockfish, which plays like a, well, a stockfish.
- openfuture 9y agoAccording to the commentators Houdini had more impressive wins and the game that decided the tournament (the third game shown in this article) was supposed to be a draw but somehow got blundered away due to the contempt rule. Basically it was very close and Houdini may actually be slightly better, especially at the abstract heuristic stuff. Although Houdini creates good positions for itself, Stockfish excels at brute force and somehow always manages to turn things around as it transitions into the endgame.
- josephernest 9y agoIs it only this code and nothing else? https://github.com/mcostalba/Stockfish/tree/ONE_PLY/src https://github.com/mcostalba/Stockfish/tree/ONE_PLY/src Is there no DB or machine learning inside?
- ZirconiumX 9y agoIt's actually https://github.com/official-stockfish/Stockfish/tree/master https://github.com/official-stockfish/Stockfish/tree/master No, SF doesn't use machine learning, it's entirely heuristic.
- josephernest 9y agoDoesn't it use a database? From README: > If the engine is searching a position that is not in the tablebases (e.g. a position with 7 pieces) and it speaks a lot about tablebases. What is it?
- ZirconiumX 9y agoOkay, I'll concede that one, my apologies. A tablebase is essentially a precomputed endgame solution for all positions with that many pieces. Likewise, it has an opening book, which is a database that contains moves for the first few turns of the game. It's still a heuristic searcher when it's not using those though.