7 ms·
Evolution of chess: Popularity of openings over time
- cven714 12y agoChess opening trends are like fashion--some high profile player(s), always on the search for new ideas, finds a resource in an unpopular line and suddenly it's all the rage. Everyone is playing it, working out the complications, finding ways to defend or neutralize the lines, then interest wanes until someone uncovers a fresh new plan somewhere else and the cycle repeats. Other times though, new resources aren't found and a line mostly dies out, like the Kings Gambit. So what I would be interested to see from your data set is a relation between opening performance and its popularity. Did people stop playing the Pirc due to sub-par results compared to other openings at the time (like I imagine happened with the Vienna) or did it simply fall out of fashion? It would be interesting to know which lines always had good results, but just stopped being popular for whatever reason. They could be due for a revival.
- dfan 12y agoUnfortunately, it isn't always obvious from the data which of your interpretations is correct. Take the following two fictional cases: Case 1: The Limburger Gambit was popular for a few decades, but then grandmasters started playing the Limburger Gambit Deferred instead, so it fell out of fashion, although it is still a perfectly good opening. Case 2: The Limburger Gambit was popular for a few decades, but then a response was found that refuted the whole line. It was played in one famous game, after which everybody abandoned it. As far as the data set is concerned, the difference between these two cases is a single game, so it's not going to show up in the stats.
- cven714 12y agoThis is true, but I think there are many more case 2s where rather than an entire line being refuted in a single blow, gradually one side was able to find antidotes and the line stopped being a good practical choice. The Vienna is an example. So while it isn't always obvious, I'm hoping enough fall between your two Limburgers to be interesting.
- dfan 12y agoThe two biggest examples I can think of of "opening fell out of fashion but then was brought back at the highest levels" are Kasparov reviving the Scotch in the 1990s and Kramnik reviving the Berlin variation of the Ruy Lopez in his world title match with Kasparov in 2000.
- rmrfrmrf 12y agoWith the advent of computer analysis, the serious chess player needs to have a basic understanding of almost every opening, with that understanding going deeper and deeper as you scale the ranking list. For example, the King's Gambit hasn't really fallen out of favor despite the lack of long grandmaster games. The Falkbeer Countergambit (1.e4 e5 2.f4 d5) in particular has seen a revival as of late with the Nimzowitsch variation (3.exd5 c6!?). You might not see Carlsen and Anand playing it in tournament (mostly due to the risk-reward factor of having to solve complex positions over the board under time pressure), but they've almost certainly studied it with their team of coaches and researchers. To comment on the Pirc: statistically, Black fares much worse with the Pirc in relationship to the most successful counter to 1.e4, the Sicilian defense (1...c5). From a gameplay perspective, Black has to commit much earlier with the Pirc than the Sicilian, since the Sicilian has about a billion variations starting as early as the second move.
- kristopolous 12y agoI'm under the impression that introducing the Queen too early (say < 6 moves) usually leads to it being taken and that player losing. I don't know how much this is actually true though. That would also be a good thing to look over
- dfan 12y agoIndeed, one possible consequence of developing the queen early is that it can be harried with tempo by the opponent; that is, he gets to make you respond while making developing moves he wanted to make anyway. However, except in cases of gross incompetence it doesn't lead to your queen being taken, it's just suboptimal. There are also plenty of exceptions to this rule, of course. The most obvious one is the main line of the Scandinavian Defense, 1.e4 d5 2.exd5 Qxd5. This isn't a grade-A opening but it's a solid B+ and occasionally gets used at the highest levels.
- dfan 12y agoThe Pirc spike in the 1850s looks interesting visually, but keep in mind that your data set back then is incredibly small. Looking at my own database, the spike seems to be entirely due to some guy name Mahescandra playing it 57 times against Cochrane. It certainly doesn't have anything to do with the increasing popularity of 1.d4 40 years later. I've never heard of Mahescandra, but Cochrane is the guy the famous Cochrane gambit in the Petroff is named after, where White sacrifices a piece on move 4 (1.e4 e5 2.Nf3 Nf6 3.Nxe5 d6 4.Nxf7).
- rhiever 12y agoI double-checked my data set to look into this, and you're right. Nearly all of the Pircs in that time period were done by a guy named Moheschunder Bannerjee: https://en.wikipedia.org/wiki/Moheschunder_Bannerjee https://en.wikipedia.org/wiki/Moheschunder_Bannerjee He contributed to the development of the Indian Defence.
- Blahah 12y agoWhy do you all have your own datasets of historical chess moves? Is this a popular playground for data analysis, or is it something chess learners study?
- Matetricks 12y agoGreat blog post. It's interesting to compare my knowledge of opening evolution with the historic data. During the Romantic Era, King Pawn games were clearly the norm. When Reti and Nimzowitch introduced hypermodernism in the 1920s Indian openings became much more popular.
- bane 12y agoOut of curiosity, is chess solvable yet by computers? Meaning, is it possible to simply brute force every possible legal game up to n moves and determine all the winning and losing move sets? What's this number look like theoretically? (I'm sure in the general sense games with a very large n aren't as I suppose a game could be played in perpetuity)
- patrickmay 12y agoNo, it is not yet possible to brute force all chess games. The largest extant endgame database has full solutions for seven pieces on the board. Once you get to that point, chess is solved. ;-)
- rmrfrmrf 12y agoJust to clarify what you alluded to, it's theoretically possible for chess to be solved, and progress is being made slowly, but surely (it took around 7 years for chess to be solved up to 7 pieces). Of course, with each additional piece comes an exponentially larger set of positions, so progress from 7 to 8 pieces should take much longer assuming there aren't any massive breakthroughs in computing speed.
- ThrustVectoring 12y ago>every possible legal game up to n moves That's easily above 10^30 positions. 10^30 is a very low lower bound, too. So, uh, no.
- bane 12y agoWell, that's why I specified n moves. Entire games can take 2 moves. So for n=2, it's probably pretty easily solvable. Probably for n=3 and maybe 4 as well. It's also not strictly exponential as n increases, pieces are removed from the board for example restricting the number of legal moves as the game progresses. There's probably interesting solvable subsets up to large numbers of n, like maximum captures as the game progresses, or all games where there are zero captures or some such. It's not a simple matter of calculating every possible theoretical legal or non-legal state of the board with all the pieces.
- adamconroy 12y agoThe analysis is interesting. However I'm not sure it has much practical value due to transpositions. For example, as white I play 1.Nf3 and if black plays d5 I play d4 and we have a d4 opening. If black plays c5 I play c4 and depending on what black does it will transpose into either an English opening (1. c4) or a maroczy sicilian (1. e4) or an indian defence (1. d4). So basically, my opening move would be classed as 'other' but really it is one of 1.d4, 1.e4, 1.c4 in terms of the classifications of this post.
- rhiever 12y agoIf any of the alternative paths were common enough, they would show up in the charts as well. I didn't limit the analysis to a particular set of moves; I simply counted all of the paths present in the data set and showed the most common ones. This is why two variations of the Indian Defence show up in the "White's second move" chart. I think it'd be interesting to try to combine all possible paths for an opening into a single count, but that would probably be complicated if multiple openings can be reached through the same path. (e.g., which opening would the shared path be assigned to?)
- adamconroy 12y agoIt is quite complicated. The examples I gave are quite simple, but in a lot of cases the transpositions can happen many moves into the game. The maroczy sicilian example I referred to is one of these cases, where I might not actually play e4 until say move 8-10, but in a general sense it probably should be classified as an e4 opening. As an overall guide I think what you have done is fine, but I suspect something in the order of 10-20% of games might be subject to transpositions where the opening move isn't an accurate categorization.
- CurtMonash 12y agoAnd by the way, two of the most important kinds of transposition are: 1. Players who want to play Queen's Pawn opening but avoid certain lines in them start in an English instead. 2. Players who want to play Queen's Pawn or English openings but avoid certain lines in them start with 1. Nf3 instead.
- Cogito 12y agoThe visualisation I would most like to see with respect to openings is a 'heat map' like diagram, showing how likely a given piece is to occupy each square after n moves. Are there any squares never used after the first move? Just what weight is given to the centre of the board after the first few moves? This sort of information could be communicated really well with a view of the board, and the pieces expected to be in each square.
- Anon84 12y agoWhere does one download the data? The first post in the series links to the website but there doesn't seem to be any download links anywhere
- themodelplumber 12y agoAs a novice player, it's frustrating to read stories about players beating other players "by the book," i.e. they know more openings than the other player does, so they play some unexpected opening and just destroy the opponent out of the gate. I guess this is possible in just about any game, not just chess, but it seems to reduce the optimal learning process for a novice to "start memorizing things". Bah. Edit: Not that memorization is bad or boring by necessity, but finding a way to make the memorization fun and interesting is in itself frustrating, knowing that you could learn openings really fast just by brute-forcing the various lines into your brain with things like memory palace techniques.
- danielbarla 12y agoThis is an argument that is raised a lot in chess circles. While there's some truth behind it, it tends to be applicable much more in an environment where both players are trying really hard to play "in book". Before you get too deep into openings, you should have a solid understanding of the strengths of individual pieces. This leads to a better understanding of why you'd want to put your pieces in certain places. It kind of explains the broad strokes of openings, without the little traps here and there. This is known as the Russian school of teaching. Basically, learning from this direction, you can spend much less time studying openings. You might want to know about few common traps in common situations that you tend to use, but that's not the focus. And you'd be surprised at how well this works - barring a tiny number of games where you're caught out, your better understanding of the overall game will win out. Essentially, overspecialising in opening repertoire at the novice level is a kind of arms race which only really works against other similar players. It's not a good long-run strategy.
- username42 12y agoYes, that is the reason many good chess players switch to the game of go. For example Jan Bogaerts in belgium was about 2350 ELO before switching to go.
- tygorius 12y agoThe "purpose" of the opening is to get yourself into the sort of middlegame position you like to play. (Some players love having white against a French Defense, for example, while other are quite comfortable with black's side of it.) Understanding which types of positions you prefer takes experience, and while you're gaining that experience at the lower levels tactical skill can dominate. So it can seem like knowing the "book" moves for a bunch of openings is important when in fact the problem is that your opponent has better tactical skills than you or perhaps you have trouble judging a position and developing a plan. That's why specializing in a few openings can be helpful -- you get lots of practice with similar middlegame positions at the same time you limit the number of lines you have to deal with. Now if you like open games such as the King's or Evans Gambits (i.e., fashions of 100+ years ago) things will get tactically bloody pretty quickly and so gaining a knowledge of where the booby traps are will be prudent. But if your taste runs to semi-open games, say, you can limit your opening study to a relatively small number. I happen to like the Dutch Stonewall formation as black for example, and the precise move order for getting into a position I'm comfortable with doesn't require a lot of memorization. Happily we live in a time when computers can not only entertain us in chess but also help us with learning chess. You can improve your tactical skills by studying a tactical positions card deck in Anki for example. Or you can work on a particular opening by setting the computer to always start games from a particular position. You can use that same starting position trick to train yourself in endgame play. The important thing is to find ways to keep the game fun while you improve your play.
- alien3d 12y agoi don't know much about opening but i play chess frequent in my younger age and sometimes computer. It's just a pattern.But to win with computer,must follow non pattern/opening to win.. Very long time not playing chessmaster software.
- geebee 12y agoI noticed that the queen's pawn opening has grown in popularity considerably. I don't really play chess anymore, but two decades ago, I played quite a bit. Much of the time, it was against a computer. I couldn't win. I finally found a book that really worked for me, and it taught me the "stonewall" approach, where you (playing white) advance the queen's pawn, lock up the game, and start maneuvering very tactically to restrict the movement of black's king and get a mate. I found that this was the only way I could beat the computer - largely because it allowed me to think along a very few narrow lines (often involving a sacrifice), while the computer would get bogged down in useless searches. This was all in the late 90s, my guess is that it wouldn't work anymore. So, I'm wondering if the surge in queen's pawn opening is the result of people being conditioned to play against computers? And it brings up another point - I've heard (man, I wish I could find a link to this theory on the web) that chess masters who lament the rise of computers actually aren't particularly bummed that computers can beat the strongest human chess players. They're bummed that the best human chess players are now playing a style that is heavily influenced by years of practice against a very specific style of opponent - a powerful computer that can't really make a "mistake". Young people have opportunities to train that never existed before, but because they don't train against stronger human players as often, some of the style of the game has been lost. I'm not (and never was) good enough at chess to know if this is true, but if anyone has an angle on it, please chime in! Oh - by the way, a very satisfying ending here. I knew a very brilliant guy who I couldn't beat at chess in high school. After I read this "stonewall" book, I was all ready for him. I advanced queen's pawn, all read to lock it down and take away his advantage. He simply advanced his king's pawn, basically telling me, go ahead and take it. The wide open game against an inferior opponent was worth far more to him than a pawn. Good for him, though not so good for me ;) That's the kind of thing you do when you're experienced playing against people rather than computers.
- dfan 12y agoWas that book with the Stonewall Attack "How to Think Ahead in Chess"? http://www.amazon.com/How-Think-Ahead-Chess-Techniques/dp/0671211382 http://www.amazon.com/How-Think-Ahead-Chess-Techniques/dp/06... I don't think that the current popularity of 1.d4 is due to playing against computers. The main reason it is currently more popular than 1.e4 at the highest levels is that Black has found a very effective way to counter 1.e4 (the Berlin variation of the Ruy Lopez) that makes it difficult for White to achieve more than a draw. This high-level popularity then trickles down to lower levels. It is definitely true that the style of play has changed somewhat due to computer engines, but it's not so much the result of playing against computers (top players generally don't find that very rewarding), but of analyzing with computers. You can play through a game or an opening and explore variations, with the computer constantly telling you what it thinks the best moves are and who is winning by how much. As a result, players become trained to evaluate positions closer to the way a computer would. One example is that computers don't mind grabbing material and defending an unpleasant position for a long time if they don't think that the opponent can break through. This has given players confidence to be more materialistic than they were in the past.