4 ms·
At 50x designed max load bad queries would be one of the easier problems to solve. We design systems all the time that have hard scaling limits, and it's only a
by kevan 10y ago
At 50x designed max load bad queries would be one of the easier problems to solve. We design systems all the time that have hard scaling limits, and it's only a problem when you operate the system past those limits. e.g. I wrote a service with ACID assumptions from the underlying database but now the largest ACID db box I can buy isn't big enough. Oops. There's a bunch of possible ways around that but they usually involve a nontrivial amount of engineering effort.
I wouldn't expect any service to survive that much unplanned load. Maybe they could've estimated better, but how likely was it for the game to go viral? Worth sinking lots of dev time into and delaying launch? That's a hard question to answer without the benefit of hindsight.
- sp527 10y agoHonestly I was surprised they were able to scale as elastically as they did and we now know the reason why: they used Datastore instead of something like Postgres with sharding. That decision alone is most of the reason they made it. By contrast, my team doesn't use managed NoSQL services for our projects because they're simply too expensive to deploy before seeing considerable growth.
- jshen 10y agoDatastore seems really cheap to me, am I missing something?
- viraptor 10y agoThey also have a system that's almost trivial to shard in any way they want, which would help even with postgres all around. Since there's no real interaction between users (only user-environment really), they can split databases by location, user groups, or some hashes as they want. Also the data they deal with is extremely cachable - as long as you play on one device.
- itcmcgrath 10y agoThere is some, when users battle gyms they are battling with other users Pokemon which would be from anywhere in the world. When the Pokemon on the Gym loses it needs to be removed in a transaction with an update on the gym. This takes the easiest form of sharding off the table, so it's not as simple as you make it seem.
- ec109685 10y agoIt doesn't really need to be a transaction since it is okay if it is eventually consistent (e.g. they have a cleanup job if a transaction is lost), that would be fine.
- late2part 10y agoThis: "but now the largest ACID db box I can buy isn't big enough." The hard part about scaling is knowing when to invest in scale. The binomial choice is to build and throw away, or build so as to be scalable. So hard to predict.