7 ms·
It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his v
by rao-v 1mo ago
It’s worth understanding that Shin Jinse has been significantly stronger than his nearest human opponents for a while now, more so than Magnus was even at his very peak.
In go ELO like scoring he’s something like 120 points over the next strongest player. No other player has ever broken a 3800 rating let alone 3850. Ke Jie (the previous long time champion) peaked at 3755. Shin Jinseo’s strength graph is the most absurd straight line.
https://www.goratings.org/en/ https://www.goratings.org/en/
2 stones is historically the gap between a 9P ranked and a 1P ranked professional player (very roughly the gap between super grandmasters and an almost grandmaster)
That is to say it’s shocking that Katago (almost certainly significantly stronger than AlphaGo) is a mere 2 stones stronger than Shin Jinseo. I suspect it would be 3-4 stones vs any other human pro.
- ninju 1mo agoDeep link to Shin Jinseo's strength graph https://www.goratings.org/en/players/1313.html https://www.goratings.org/en/players/1313.html
- avadodin 1mo agoIt would be interesting to find out what insight he discovered about the game to consistently rise like that. It can't be just play like AI. Any other Korean on the Korean Go program could have done the same. In fact, many did when AlphaGo was the pinnacle of AI.
- saucymew 1mo agoFor a non-Go player, do you think this trend will persist, or is it more of a dead-cat/human bounce?
- rudi-c 1mo agoOn one hand, Shin Jinseo is an outlier player of this generation. On the other hand, the newest generation of new pros will have exclusively learned by using the AI to tell them what the best move is, so there's reason to believe that peak human level has yet to be reached.
- loglog 1mo agoDistillation of our blessed models is no fair!
- rao-v 1mo agoCould someone sufficiently motivated invest in training Katago to be able to beat Shin Jinseo with 3 stones of handicap? Unfortunately - probably yes. This in no way detracts from how absurd and remarkable it is that Shin Jinseo can beat KataGo (it gets a LOT of training and architecture refinements https://katagotraining.org/#eloGraphButtons https://katagotraining.org/#eloGraphButtons) with 2 stones of handicap.
- mafuy 1mo agoDepends on where you put the perfect play ceiling. I think each side has good points in their respective favor. There have been strong pushes by both humans and computers in improving their best play in the past few years. A 2 stone handicap, however, does not scale linearly with strength. It becomes relatively more impactful at higher levels. For pro players, 2 stones are gigantic, and for beginner players, they make zero difference. Same for a point advantage (komi adjustment). So there might come a point where it is physically impossible for a computer to beat a top human under some handicap.
- afthonos 1mo agoImportant to note that KataGo was double-handicapped. 20 seconds per move maximum; it couldn’t read deep. Against an amateur, it doesn’t matter, but against a historically strong pro it matters a lot.
- harshreality 1mo ago...on a 4x 3090 rig. The game ran 299 moves, giving katago 100 minutes if it exhausted time on each move (which must be the optimal strategy under that time control). Shin used about 205 minutes, over twice as much time and of course had leeway to spend more time on difficult moves. Based on the youtube video, it looks like katago was only using 16 seconds per move, is that right? https://www.youtube.com/watch?v=-86zF4mTWOY https://www.youtube.com/watch?v=-86zF4mTWOY Is 20 seconds on that hardware really overkill and well into the diminishing-returns curve, as a top-level comment suggested, or is it plausible katago could have played better if given 40 seconds per move? match details: https://gostonebase.com/blog/shin-jinseo-vs-katago-kishin-match/ https://gostonebase.com/blog/shin-jinseo-vs-katago-kishin-ma...
- somenameforme 1mo agoAs another question, does it not operate similarly to the top chess engines? The way the neural network systems work is by using the probabilistic matching paired with a Monte Carlo simulation. So you can get to extreme depth very rapidly. Obviously the breadth is going to be limited, but if the neural network side is well tuned (so high probability hits are indeed generally the most challenging moves), then that's not such a problem. And you can run a huuuuuuuge number of sims in 16 seconds.
- nurbl 1mo agoI believe the basic idea is similar, but there's an enormous difference in the space of possible moves between chess and Go. Go has a larger board and moves are less restricted. There was a major breakthrough in Go playing programs a decade or two ago when good Monte Carlo methods were developed for it. But still I think the ability to simulate many moves is less powerful in Go.
- joe_the_user 1mo agoI don't think it's solely a matter of raw strength, but as Shin said, a willingness not to play to the program's strengths. I mean, one thing that rankled me about original Lee Sedol match was that Lee had no access to the program's "record" while the machine by the nature of the AI training process had effectively studied Lee's games in great detail. I recall a while back someone came up with a set of "anti-computer" strategies that allowed even an amateur to defeat a strong go program. These moves weren't anything like ordinary go moves (and perhaps the "loophole" has been closed now) but imo, their existence suggests that a study of programs may reveal other unexpected weakness.
- Ntrails 1mo agoI saw the same things when the OpenAI Dota bots could eviscerate humans 1v1 - even pros lost! Until a more average player confuses the AI with an unseen behaviour (pulling creeps between the towers etc) to get an advantage.
- arcticfox 1mo agoWe saw this with AlphaStar too, but ultimately it feels like simply an exploit. I expect even a relatively simple modern LLM/model working with the custom transformer would have been able to address these exploits after a game.
- Ntrails 1mo agoI don't think exploit is the right term? Anyway. Yes if you throw examples into training it will be able to handle the situation - but handling unseen things for me is a key goal.
- rao-v 1mo agoIn a march 2026 interview David Wu (lightvector, Katago’s creator at Jane Street) noted that he doesn’t have a systematic solution for the cyclic group problem, but adding examples to the training set mostly ensures Katago during MCT rollout figures it out. I don’t think there has been a post mid 2024 verified exploit. https://gomagic.org/david-wu-on-building-katago/ https://gomagic.org/david-wu-on-building-katago/
- mtlmtlmtlmtl 1mo agoProbably a better comparison from the chess world(in reasonably modern times, though perhaps players like Capablanca and Lasker could be mentioned as well. Alas, I don't think FIDE rating existed back then) is Bobby Fischer. In the july 1972 FIDE rating list he held a rating of 2785, the highest in history at the time, with Spassky in second sitting at a "measly" 2660, and only 13 players being above 2600 even.
- CurtMonash 1mo agoIf I counted corrected, Fischer went 24-3 in the world championship series around then. That excludes many draws and one forfeit vs. Spassky, a few draws vs. Petrosian, and nothing at all in his sweeps of Larsen and Taimonov.
- devy 1mo ago> Katago (almost certainly significantly stronger than AlphaGo) Interesting KataGo is an open sourced Go program written primarily by David Wu in C++ and recently heavily vibe coded by Claude. It's running on four Nvidia RTX-3090 GPUs with 96GB VRAM. [1] [1] https://github.com/lightvector/KataGo https://github.com/lightvector/KataGo
- zelphirkalt 1mo agoI read this comment before looking at the article and thought that the grandmaster beat the AI even giving the AI 2 stones. Too bad. But this way around is of course more realistic. And of course you would need to take into consideration the scale of go ratings and chess ratings when making that comparison. With top chess ratings being around 2800, being 1000 less than the top go ratings, one would have to apply a factor of roughly 3/4.
- hyperpape 1mo agoIf you scaled Shin Jinseo to 2800, you would have players with extremely negative ratings. This page shows ratings of European players on a roughly aligned scale: https://europeangodatabase.eu/EGD/createalleuro3.php?country=**&dgob=false https://europeangodatabase.eu/EGD/createalleuro3.php?country.... It still has negative numbers on it, and this only contains players who have attended a tournament (though it's more common for beginners to play tournaments in the west, since it's hard to find times to play). It's not a comparison of the worth of the games (I play both, though I'm better at Go, and prefer it), but the dynamic range of Go is larger. That said, any cross-game/sport comparisons of this kind are pretty tough to do properly.
- somenameforme 1mo agoI'm unfamiliar with Go ratings, but chess ratings are based on the Elo system which is a simple mathematical prediction system. Borrowing some figures from Wiki [1] we get: 1.00 +800 0.99 +677 0.9 +366 0.8 +240 0.7 +149 0.6 +72 0.5 0 0.4 −72 0.3 −149 0.2 −240 0.1 −366 0.01 −677 0.00 −800 The second column is your rating minus your opponent's, and the left is your predicted result. So if you are rating 1849 and your opponent is rated 1700 then you'd be expected to score about 70%. To have a 1% expected score against Magnus, you'd need a rating of about 2150. [1] - https://en.wikipedia.org/wiki/Elo_rating_system https://en.wikipedia.org/wiki/Elo_rating_system
- shwaj 1mo agoI was under the impression that Go also uses Elo, then I did a bit of cursory research and discovers that it varies. Two major federations are American Go Association (AGA) and European Go Federation (EGF). EGF uses an Elo-inspired update rule since 2021. AGA uses a quite-different Bayesian system without pairwise update; they provide a paper and a C++ reference impl. Asian countries don't bother with such numeric ratings. Instead, rankings are titles which are won through tournament promotion structures (sounds similar to Sumo to me). Interesting, because I always thought that it was more "apples to apples", and that the higher upper limits of Go rankings was somehow indicative of the higher "dynamic range" of the game compared to chess. For example, if Elo were applied to basketball, what would the Elo of Lebron James be compared to a playground hooper (leaving aside that 1-on-1 isn't the best part of Lebron's game)... would it be higher or lower than Magnus Carlsen in chess? I don't have an intuition.
- hyperpape 1mo agoYou're right that Shin Jinseo is a generational talent, and more dominant than anyone since Lee Changho (peaked in the 90s and was strong into the early-mid 2000s). However, you can't compare goratings over time, the top ranks are not nearly stable enough. https://www.goratings.org/en/history/ https://www.goratings.org/en/history/ (I think it's believable Shin Jinseo is better than Lee Changho, but not that there has been steady progress since the days of Lee Changho, so that there are now 20 players stronger than him).
- gamegoblin 1mo agoCould you just have superhuman Go AI just how good humans are somewhat more objectively? Not without flaws of course, but probably interesting
- rao-v 1mo agoThe problem is ambient go knowledge. A top 100 player would easily beat time traveling Lee Changho in his first few matchups. Of course give peak Lee Changho a fortnight to prep with Katago and … well that would be something!
- hyperpape 1mo agoIt's not obvious to me, but I lean towards saying this is false. The 2026 player would play a better AI inspired opening, but I'm not sure that would be enough to overcome the skill difference. (This is especially true if they don't get briefed "this is time-traveling Lee Changho, he doesn't know contemporary joseki, play a trap variation").
- rao-v 28d agoYour point is pretty fair. The reason I think the top 100 pro would win is that when I watch pros review they are very quick to catch when someone is playing a non-current or up to date style. I’ve also seen a few middle game situations where the classic famous go move is now “obviously” not good after we’ve seen how AI handles it. This all adds up to many ways good sound moves to Lee Changho have now obvious responses. Do I believe he would adapt inhumanly fast? Absolutely! Legendary fighting spirit
- zamadatix 1mo agoFor another comparison, top world class chess players will have solid odds to beat Leela Chess Zero when given a knight odds handicap (Leela Chess Zero starts with 1 fewer knight). For human vs human, I think this would be somewhere in the ballpark of the ~10,000th best chess player having fair odds against Magnus. I wonder if this means the best Go play is closer to theoretically perfect play or if it just happened the current computer methods didn't manage to get much farther than humans. Go has vastly more valid games but also a simpler ruleset, so I'm not sure if there is really a good way to tell beyond "keep trying and find out"?
- mafuy 1mo agoI'm not sure if I'm allowed to name the source on this here, but someone very familiar with the top Go program scene explained that there is a fair chance that a perfect game has been played already. This is based on the probability of a mistake per move in relation to the number of games there were played by top programs. Hard to believe at first, but perchance it is true. The problem is, of course, that we don't know which games are perfect.
- monster_truck 1mo agoThis was only running on 4x3090s man. The gap is far, far wider than you think. 8 5090s or 6000s and at least as many extra dedicated to doing nothing but running disgusting amounts of monte carlo in parallel from anything resembling a good choice (fuck it check some bad ones too) would lay him to waste. I am going to go out on a limb and say part of it is a matter of respect, and part "lets not discourage and scare the shit out of anyone who knows even a little about Go" A long time ago I ported a Go game to the iPhone for a job, the only way to control difficulty was to limit the amount of time it could spend monte carloing. On the iPhone 4, "Hard" would take 20-30 seconds a turn and drain your battery. I learned the game as I was making the app, got into single digits vs humans years after that. When the iPhone 5S came out it was capable of doing so many more iterations a second that even the easiest difficulty destroyed me until I spent enough time figuring out which moves created the worst spaces for it to search, which isn't really playing go anymore it's more like mining bitcoin by hand. E: 4x3090 tops out at ~1.4k nnEvals/s. 4x5090 is 3.2x. 8xRTX 6000 something like 7x.
- mafuy 1mo agoIf I remember the numbers correctly, your setup seems to be wrong if you only get 1.4k e/s with 4x 3090. What network did you use?
- monster_truck 29d agoI don't buy novideo trash, go ask the people running the benchmarks
- crsv 1mo agoGiving my guy the nickname Satoro Go-jo.
- simiones 1mo agoA small note - it's Elo scoring, not ELO. It's named after Arpad Elo, it's not an acronym of any kind.
- ecshafer 1mo agoThe consequences of League of Legends. The brain trust over there use ELO and its caused everyone to think Elo is an acronym with the wrong pronunciation.
- WhitneyLand 1mo agoI don’t know that it’s that shocking, remember Go it’s not solved game, so the the limits of what’s really possible is not known in all cases. For example, we don’t even know whether perfect White play can possibly overcome two correctly placed Black stones against perfect play.