3 ms·

I kind of take issue with the "No amount of load testing could adequately prepare the server team behind Diablo 3 for firepower of this magnitude". If your loa

I kind of take issue with the "No amount of load testing could adequately prepare the server team behind Diablo 3 for firepower of this magnitude". If your load-testing does not prepare you for the worst then your load-testing plan is garbage. They already knew how many copies had been pre-ordered and could make a pretty good guess how many copies would be sold and activated on the first night. Take that worst case estimate, now double it and test for that. "But the cost of supporting worst-case scenarios!!" some may cry. This is where rented servers / cloud setups are useful for elastic scale without breaking the bank. There are companies that can simulate load from users across the globe, I have no doubt that blizzard would have the connections / influence /cash to set-up a kick-arse load-test system.

sounds 14y ago
Can I ask why this got downvoted? I'd think there are some discussion-worthy points here. For instance, I'm not proud of it but at a former employer I talked them out of load testing an app, which then had serious scaling issues. I learned my lesson the hard way.
stevestreza 14y ago
Behaviorally there will be differences between how your users act and how your load test plan is executed. A small difference in the two can cause problems you didn't expect. That's the point I was trying to make.
- rurounijones 14y ago
  For some things I can agree with you, usually web-apps with many possible branching execution paths where it is hard to know exactly what will be the most common use case to test for. However in this case what can the users do that can be different to their plan? The most basic example would be logging in, there are not that many execution paths I can think of for that one. As for the integration / reporting during gameplay, this is all done via a strictly defined API which is called by the game client in relatively predictable ways (game started, player does X, authentication ping every x seconds, achievement unlocked), unlike a web-app where users can do whatever they want the API usage flow is basically controlled by Blizzard. That is why I am not impressed with blizzard on this one; they control basically the entire use of this API apart from one thing, the number of users trying to use it at any one time, which is there the load testing plan should have worked.

sounds 14y ago

Can I ask why this got downvoted? I'd think there are some discussion-worthy points here. For instance, I'm not proud of it but at a former employer I talked them out of load testing an app, which then had serious scaling issues. I learned my lesson the hard way.

stevestreza 14y ago

Behaviorally there will be differences between how your users act and how your load test plan is executed. A small difference in the two can cause problems you didn't expect. That's the point I was trying to make.

rurounijones 14y ago
For some things I can agree with you, usually web-apps with many possible branching execution paths where it is hard to know exactly what will be the most common use case to test for. However in this case what can the users do that can be different to their plan? The most basic example would be logging in, there are not that many execution paths I can think of for that one. As for the integration / reporting during gameplay, this is all done via a strictly defined API which is called by the game client in relatively predictable ways (game started, player does X, authentication ping every x seconds, achievement unlocked), unlike a web-app where users can do whatever they want the API usage flow is basically controlled by Blizzard. That is why I am not impressed with blizzard on this one; they control basically the entire use of this API apart from one thing, the number of users trying to use it at any one time, which is there the load testing plan should have worked.