4 ms·

I'm curious they seem to have so many problems, given that at it's heart the game is rather .. decentralized. They have access to this vast swath of cloud tech

I'm curious they seem to have so many problems, given that at it's heart the game is rather .. decentralized. They have access to this vast swath of cloud technology and a few Moores law iterations to boot yet they seem to perform worse than even WoW did on launch, with a game that is extremely forgiving to latency.

dom96 10y ago
Indeed. In addition to this their outages have, over the past few days, been very predictable. There is one happening right now [1], and one happened at the same time yesterday and the day before. As far as I can tell that should make it easy to decide when they should deploy new servers to handle load better. 1 - https://pokemongostatus.org https://pokemongostatus.org
- TeMPOraL 10y ago
  Yup. The load usually starts to happen when kids leave school, and then the servers blow up when adults leave work. Until this week, the game literally stopped working for me every day at 17:00.
simcop2387 10y ago
Everything I've seen based on previous outages, and rumours, seems to indicate it's probably a player data/authentication problem rather than the parts that are easily distributed (pokemon generation, image serving for pokestops, etc.). During a number of outages you'd be able to keep playing for a time iff you were already online and playing, at least according to some reports. I'd love to see actual postmortems from them but I doubt we will since they've said nothing at all so far. But the authentication could easily be a problem that they're using both their own service and google for providing authentication. Since they go down each time a new region is released it would make sense that they're getting hit with lots of account creation and google API hits, the things that are likely to be the worst for scaling, since google will rate limit most clients (though I'd think they could get something worked out since it's all real traffic). Along with that, their own authentication server has gone down near release (pokemon trainer accounts) while the google one was working, so they probably have had to throw servers at that and weren't expecting to. I'd also expect a lot of the load to be related to doing lots of geospatial queries, but I doubt they planned that badly since they had experience with ingress. I think it'd be related to the nearby pokemon feature that they've had broken since shortly after launch. There's definitely questions (e.g. why did nearby pokemon break the way it has?) about the particular bugs that people see that make me wonder how they've handled things but i suspect we'll never know since the answers are going to whatever secret sauce they've done to handle this kind of game.
regularfry 10y ago
1. They reused architecture (and possibly infrastructure) from an existing game, which has a different mechanic and different usage patterns. 2. It's massively more popular than they originally anticipated.

dom96 10y ago

Indeed. In addition to this their outages have, over the past few days, been very predictable. There is one happening right now [1], and one happened at the same time yesterday and the day before. As far as I can tell that should make it easy to decide when they should deploy new servers to handle load better. 1 - https://pokemongostatus.org https://pokemongostatus.org

TeMPOraL 10y ago
Yup. The load usually starts to happen when kids leave school, and then the servers blow up when adults leave work. Until this week, the game literally stopped working for me every day at 17:00.

simcop2387 10y ago

Everything I've seen based on previous outages, and rumours, seems to indicate it's probably a player data/authentication problem rather than the parts that are easily distributed (pokemon generation, image serving for pokestops, etc.). During a number of outages you'd be able to keep playing for a time iff you were already online and playing, at least according to some reports. I'd love to see actual postmortems from them but I doubt we will since they've said nothing at all so far. But the authentication could easily be a problem that they're using both their own service and google for providing authentication. Since they go down each time a new region is released it would make sense that they're getting hit with lots of account creation and google API hits, the things that are likely to be the worst for scaling, since google will rate limit most clients (though I'd think they could get something worked out since it's all real traffic). Along with that, their own authentication server has gone down near release (pokemon trainer accounts) while the google one was working, so they probably have had to throw servers at that and weren't expecting to. I'd also expect a lot of the load to be related to doing lots of geospatial queries, but I doubt they planned that badly since they had experience with ingress. I think it'd be related to the nearby pokemon feature that they've had broken since shortly after launch. There's definitely questions (e.g. why did nearby pokemon break the way it has?) about the particular bugs that people see that make me wonder how they've handled things but i suspect we'll never know since the answers are going to whatever secret sauce they've done to handle this kind of game.

regularfry 10y ago

1. They reused architecture (and possibly infrastructure) from an existing game, which has a different mechanic and different usage patterns. 2. It's massively more popular than they originally anticipated.