3 ms·

Big productions: Why 'Fail Early, Fail Often' Is The Wrong Approach

22 points by pushingbits 17y ago

diN0bot 17y ago
i think there are salient advantages to preferring to fail over documentation---which is about getting the implementation right before time-consuming bugs or poor code are introduced--and preferring to "fail fast"---which is about not investing in a product the market doesn't support.
Quarrelsome 17y ago
Yea, waterfall works if you do it properly. Is that so surprising?
- steveklabnik 17y ago
  Yep. Console games have different requirements than other projects, so waterfall actually kind of works. Your hardware isn't changing, you end up with a single, static product, your requirements can be known ahead of time...
kristiandupont 17y ago
It's "release early, release often" but I guess it doesn't matter much. Speaking as a former founder of a video games company, I think that this kind of thinking is inherent to the entire industry and it is killing it. The inspiration clearly comes from the movie industry where admittedly, you don't make a movie in small iterations. As it stands now, triple-A titles have to make it to the top ten to be profitable due to extreme development costs which drives up marketing prices which in turn drives up the price of the games to the consumer. This spiral is really bad for the industry. I think that indie and small developers will take over most of the market because they do much smaller and more frequent releases.
- jim-greer 17y ago
  How exactly are you supposed to "release early, release often" when you're shipping DVDs to tens of thousands of retail locations? After going through months and months of Microsoft/Sony/Nintendo certification on a 3-ring binder full of technical requirements? And having the ESRB play every level of your game to make sure it doesn't have boobs or entrails? Not to mention that for every programmer on these games there are 5 to 10 artists and level designers cranking out content. There's no automated regression suite to make sure that your last minute physics tweak didn't break one puzzle in one level somewhere. There's a reason that console developers have miserable lives - they have huge teams and launch dates that are very expensive to move. I'm not saying that things couldn't be done better, but it's a fundamentally hard problem.
  - teamonkey 17y ago
    Hence the qualifier about indie and small devs, I imagine.
- allenp 17y ago
  No he really is talking about failure not releasing. He is specifically talking about the core design and iterating over the design to optimize it before releasing. His take is that on mega-budget projects there isn't a luxury to do this and so it is better to just pick and freeze the design at the start and stay the course regardless of what happens. I totally agree with you that this thinking is self-destructive. What isn't mentioned here are development teams that aren't doing this but will (probably) be fairly successful, like the new Starcraft 2 game. Alternative methods like agile dev or stage gate would take longer and cost more but are more likely to produce more fun. I think this blog post is a good explanation: http://lostgarden.com/2007/02/rockets-cars-and-gardens-visualizing.html http://lostgarden.com/2007/02/rockets-cars-and-gardens-visua...
eswat 17y ago
Didn't Plourde just dive into 'Fail Early, Fail Often' with the way they did documentation?
- deleted 17y ago
  [deleted]
forensic 17y ago
Are there AAA game devs with 300 person teams who really think they can win without thorough documentation? WTF? "Release early, release often" is for webapps and small teams. When you're dealing with a AAA game there is such a huge network effect it's actually crucial to make the first release a good release. You then get all the word-of-mouth which drives the game industry. Most AAA titles that ship with bugs pay for it dearly in sales because gamers talk to each other and word gets out in a few days about just how finished the game is. "Release early, release often" can't possibly apply to AAA games with 300 person teams and fixed launch dates.
- steveklabnik 17y ago
  But it can apply to their internal dev process, where they 'release' to their testers and other stakeholders all the time.
dhyasama 17y ago
He advocates moving failures upstream to the design process because it is cheaper than failing during the development process. While that is true, it's not an either or proposition. There is no reason you can't fail early and often in each stage of the process.

22 points by pushingbits 17y ago

i think there are salient advantages to preferring to fail over documentation---which is about getting the implementation right before time-consuming bugs or poor code are introduced--and preferring to "fail fast"---which is about not investing in a product the market doesn't support.

Quarrelsome 17y ago

Yea, waterfall works if you do it properly. Is that so surprising?

steveklabnik 17y ago
Yep. Console games have different requirements than other projects, so waterfall actually kind of works. Your hardware isn't changing, you end up with a single, static product, your requirements can be known ahead of time...

kristiandupont 17y ago

It's "release early, release often" but I guess it doesn't matter much. Speaking as a former founder of a video games company, I think that this kind of thinking is inherent to the entire industry and it is killing it. The inspiration clearly comes from the movie industry where admittedly, you don't make a movie in small iterations. As it stands now, triple-A titles have to make it to the top ten to be profitable due to extreme development costs which drives up marketing prices which in turn drives up the price of the games to the consumer. This spiral is really bad for the industry. I think that indie and small developers will take over most of the market because they do much smaller and more frequent releases.

jim-greer 17y ago
How exactly are you supposed to "release early, release often" when you're shipping DVDs to tens of thousands of retail locations? After going through months and months of Microsoft/Sony/Nintendo certification on a 3-ring binder full of technical requirements? And having the ESRB play every level of your game to make sure it doesn't have boobs or entrails? Not to mention that for every programmer on these games there are 5 to 10 artists and level designers cranking out content. There's no automated regression suite to make sure that your last minute physics tweak didn't break one puzzle in one level somewhere. There's a reason that console developers have miserable lives - they have huge teams and launch dates that are very expensive to move. I'm not saying that things couldn't be done better, but it's a fundamentally hard problem.
- teamonkey 17y ago
  Hence the qualifier about indie and small devs, I imagine.
allenp 17y ago
No he really is talking about failure not releasing. He is specifically talking about the core design and iterating over the design to optimize it before releasing. His take is that on mega-budget projects there isn't a luxury to do this and so it is better to just pick and freeze the design at the start and stay the course regardless of what happens. I totally agree with you that this thinking is self-destructive. What isn't mentioned here are development teams that aren't doing this but will (probably) be fairly successful, like the new Starcraft 2 game. Alternative methods like agile dev or stage gate would take longer and cost more but are more likely to produce more fun. I think this blog post is a good explanation: http://lostgarden.com/2007/02/rockets-cars-and-gardens-visualizing.html http://lostgarden.com/2007/02/rockets-cars-and-gardens-visua...

eswat 17y ago

Didn't Plourde just dive into 'Fail Early, Fail Often' with the way they did documentation?

deleted 17y ago
[deleted]

forensic 17y ago

Are there AAA game devs with 300 person teams who really think they can win without thorough documentation? WTF? "Release early, release often" is for webapps and small teams. When you're dealing with a AAA game there is such a huge network effect it's actually crucial to make the first release a good release. You then get all the word-of-mouth which drives the game industry. Most AAA titles that ship with bugs pay for it dearly in sales because gamers talk to each other and word gets out in a few days about just how finished the game is. "Release early, release often" can't possibly apply to AAA games with 300 person teams and fixed launch dates.

steveklabnik 17y ago
But it can apply to their internal dev process, where they 'release' to their testers and other stakeholders all the time.

dhyasama 17y ago

He advocates moving failures upstream to the design process because it is cheaper than failing during the development process. While that is true, it's not an either or proposition. There is no reason you can't fail early and often in each stage of the process.