8 ms·

An Analysis of 155 Postmortems from Game Development [pdf]

91 points by akkartik 10y ago

fathom 10y ago
>utilize prototypes as a method of proving features and concepts before committing them to your design. Third, don’t be overly ambitious in your design. Be reasonable, and take into account your schedule and budget before adding something to your design. Building off of that, don’t be overly optimistic with your scheduling. If you make an estimate that initially feels optimistic to you, don’t give that estimate to your stakeholders. Revisit and reassess your design to form a better estimation.
sillysaurus3 10y ago
This is excellent for the history and for the examples of what worked and what didn't. But the interpretations are a little dubious: Takeaway: Game developers should create a well-defined concept before beginning development as opposed to an ad hoc method of game design. This would exclude Dota, for example. Gamedev is a world of exceptions. In general, it's best to playtest every idea you come up with and cut whatever isn't fun.
- stepvhen 10y ago
  Dota was a user made mod originally, so I don't think that counts.
  - sillysaurus3 10y ago
    Mods have been some of the most successful types of games, which seems like a pretty persuasive reason they should count. Some of the biggest names in the history of gaming started out as mods: Counterstrike, Team Fortress, etc. Valve's success is arguably because they find user-made mods and make them triple-A. Also, the advent of engines like Unity and UE blur what "modding" means. The WC3 engine was flexible enough to be made into Dota, which was then implemented on the Source Engine as Dota 2. But why was the original Dota a mod, whereas Dota 2 isn't? It's hard to come up with a definitive answer to that.
    - stepvhen 10y ago
      I think there is a misunderstanding: Dota doesn't apply to the quote "Game developers should create a well-defined concept ..." since it was neither conceptualized nor made by the Warcraft programmers. Additionally, the originally Dota started as a mod, was then published as a game, and Dota 2 would then not count as a mod. Its not hard to come up with a definitive answer to that. A game is a mod if it doesn't stand alone, if it requires _modifying_ the structure/code of another game.
      - sillysaurus3 10y ago
        The Warcraft 3 engine was flexible enough to implement Dota, just like the Unity engine was flexible enough to implement Rust. By your criteria, Rust is a mod, even though it's clearly not. It's better to say that mods are games and to treat them the same than to try to differentiate them. The same design process that yields a successful mod will yield a successful game.
        yelnatz 10y ago
        Rust is not a mod, but is a clone of a mod of Arma2 called DayZ.
        sillysaurus3 10y ago
        This is a strange comment, because the first half is what I said. Rust isn't a mod. Yet their criteria would classify it as one, since the Unity programmers did not conceptualize Rust when they created Unity. But the second half is completely false. Rust has nothing whatsoever to do with DayZ, except that it's vaguely in the same genre.
    - chipsy 10y ago
      Modding basically recoups the investment into making a game's community - an individual can present their result to an existing player community directly, with a brief forum post, and a small website with some images or video to advertise. That is enough to build a small empire on, going by the most popular examples. That's different from the typical lifecycle of a commercial game, which has a deeper level of marketing work to do before it can even consolidate a playerbase. There isn't a "Unity modding community", per se. There is a community of Unity developers, and players of individual games made in Unity, who have no broader associations. A game transitioning from the mod sphere of an existing game to an independently produced commercial title still has to overcome this gap, and most of them don't make it over the line. The advice is sensible, nevertheless. If you can extend the schedule to focus on the core elements of the design as a "R&D" process where the majority must be thrown away, and only scale it up towards a shipping product as the concept proves itself, you minimize the risk of the budget being wasted on a flawed concept. Modding scenes, game jams, and micro-budget productions all have the benefit of weeding out most of the really early, risky design experiments, without wasting enough of people's time or money to care. Even when conceiving the marketing for a larger production, the same advice works. The "trial balloon" or "landing page" method, etc. You still do design thinking when you market, but it's design on the topics of "how do we build a funnel" or "how do we make this a franchise." Still very easy to spend a lot of money on making a splash without getting the blueprint right.
- ekianjo 10y ago
  This advice is super generic.
  - sillysaurus3 10y ago
    No more generic than "Make something people want" or "Listen to your users." And yet most companies get these wrong, just like most game companies fail to playtest their ideas.
    - LionessLover 10y ago
      Why do you say "and yet" - since such generic "advice" is useless so is the conclusion "and yet they get it wrong!" (I told you something useless - why doesn't it help?)
      - sillysaurus3 10y ago
        Because if you follow this advice, you're more likely to yield a successful product. Playtesting isn't useless. It's how fun games are made. Perhaps people are disagreeing with this because they haven't tried to make a game. A curious phenomenon.
        LionessLover 10y ago
        You simultaneously complain the advice is generic and claim it's useful. You always bet on both colors in roulette? No, generic advice is not useful. It's... too generic.
        deleted 10y ago
        [deleted]
        squeaky-clean 10y ago
        But that's not advice. It's the same as telling a video game player that to win you need to "get a higher score than the enemy" and "play really well". And when someone loses, "I gave them such great advice, and yet they messed it up!" "Listen to your users" is not advice, it's the end goal. What's the most effective way to listen to your users? Professional playtesting? Open alpha/beta? Forums, IRC channel, a subreddit? Frequently scheduled AMA with the devs? Infrequently scheduled AMA with the devs?
- flashman 10y ago
  Having "a well-defined concept before beginning development" would seem to include Dota and other successful mods, because Valve is starting with a successful game concept (compared to the vast majority of mods that never find an audience). For Counter-Strike/TF2/Portal/Dota, Valve took a strong existing concept and developed it. It's exactly what's recommended in the takeaway.
  - wingerlang 10y ago
    You are probably thinking of Dota 2. Both Dota, CS and TF where most likely developed ad-hoc as they were mods that grew organically.
    - ben_jones 10y ago
      Their were Age of Empire mods that were essentially MOBA scenarios. I don't doubt many such ideas are ad hoc genius, but a lot of game dev is re-hashing with a twist.
shmerl 10y ago
> When looking at games developed for one platform versus games developed for multiple platforms, we found a few advantages and disadvantages to choosing one over the other. First, 46% of developers that used multiple platforms listed art as something that went right, while only 33% of developers that used a single platform listed this. Additionally, 15% of the developers that used a single platform listed this as something that went wrong, while only 17% of developers that used multiple platforms listed art as having gone wrong. Therefore, the evidence shows us that games developed for multiple platforms may be more likely to have better game art. I didn't quite understand the correlation between development for multiple platforms and quality of art. May be those who want to release for multiple platforms in general care about quality more?
- egypturnash 10y ago
  Maybe "having your art under control" is a prerequisite for "being big and seasoned enough to work on multiple platforms at once". If all your games look like trash, nobody's going to give you the funds to scale up to the point where you can easily develop on multiple platforms.
  - shmerl 10y ago
    I understand art beyond mere looks. It can include narrative, music, immersion and many other artistic aspects. May be the simple correlation is just being thorough and not superficial (which translates into reaching more users, rather than excluding everyone except one platform).
- stepvhen 10y ago
  I don't think it is a stretch to say that art directors don't have to worry about what platform the game is on, generally. Programmers, on the other hand, definitely have to worry. I imagine the art is seen in a better light on multi-platform games because a) what I first stated, and b) if something goes wrong in programming then "at least the art is good". I don't know if this is a specific kind of bias, but thats what it is. However I think the authors took a bit too much liberty with the conclusion (i.e. "may have better game art"). It sounds more like the designers of multiplatform games may just have a higher opinion on the art in the game, due to bias.
eudox 10y ago
Funny sidenote: the paper mentions a game titled Age of Ornithology under 5.1.g (Scope). Naturally I got excited at the idea of a game where you play some kind of bird collector, or an intrepid late 19th-century magnifying-glass-and-butterfly-net explorer and cataloguer of birds. But unfortunately this title is some kind of cruel joke: http://www.phobe.com/sfi/ornithology.html http://www.phobe.com/sfi/ornithology.html http://www.gamasutra.com/view/feature/130925/schandenfreudian_slips_age_of_.php http://www.gamasutra.com/view/feature/130925/schandenfreudia...
- stepvhen 10y ago
  I have a feeling this, and the rest of the games made by "Schadenfreude Interactive" aren't real, as much as I would love to play Grand Theft Ottoman or Accordion Hero, or Nazgul Thunder. Hannibal Crossing looks particularly enjoyable.
  - eudox 10y ago
    Personally I'm an IF fan, and the "Swiss text adventure games, Zurich (1980), Beyond Zurich (1987), and Return to Zurich (1993)" look like an interesting saga.
  - Exras 10y ago
    I saw the word 'Schadenfreude' and was reminded of a satire talkshow on a radio station in the PC game Grand Theft Auto IV Found it: Pacemaker - putting the heart back into the healthcare industry. https://youtu.be/jG9BgjnpGGY?t=940 https://youtu.be/jG9BgjnpGGY?t=940
- clentaminator 10y ago
  There's always Pokémon, as long as you're okay with trapping innocent creatures in tiny spheres and using them to battle as opposed to studying them (though you do get to catalogue them) ;)
Animats 10y ago
Reading this, it's clear that game development is now a mature technology. I was working on physics engines in the late 1990s, when nobody really knew how to do that very well. There were Gamasutra postmortems where the game physics went badly wrong and things flew apart. In the early years of the PS3, game projects were failing because the Cell architecture (a small PowerPC CPU plus six auxiliary Cell processors with tiny memory) required completely new game engine architectures. That was finally figured out, but development on the PS3 seemed to run about two years late, which set Sony back. Now there are mature game engines, mature physics engines, the graphics pipeline is in good shape, and the hardware is powerful enough. Problems in game development now are more about the game than the underlying technology. (Pre-1990s, game development was a cram job, trying to get stuff done on limited hardware.)
- greggman 10y ago
  apparently they aren't though. People porting from PC to other platforms are still finding huge perf issues and have to refactor tons of code. I thought this gen (PS4/XboxOne) were going to be easy ports but from every team I've talked to they aren't
  - mjevans 10y ago
    A lot of the criticisms I've seen for failed ports describe the exact type of behavior you would expect from an emulated system. The ports you reference behave like game systems that used to occupy 100% or almost 100% of the resource capacity of a dedicated system failing to execute correctly when they're running in a non-dedicated environment. They also had the advantage of targeting an exact set of hardware specifications in other ways and didn't have to scale down, up, or sideways (other ways of dividing labor) to the way that other platforms work. In short, they weren't a game designed to scale to a theoretical platform that then ran on the platforms, they were fit precisely to a single platform and then weren't designed to be portable at all.
    - Narishma 10y ago
      I think you got that backwards. Parent was talking about porting from PC to consoles.
ComodoHacker 10y ago
I wonder how long before this kind of analysis can be performed by AI just as efficiently and will be offered as a web service.
13hours 10y ago
"To avoid schedule slippage, game developers need to spend more time to plan out all the work that needs to be done so that no tasks are overlooked when giving estimates. By planning out tasks, you can also estimate each task individually, and give a more accurate prediction, instead of an optimistic one." When things don't work out according to the schedule, there are generally two reactions : 1. We didn't estimate well enough, we need to estimate better next time. 2. Maybe this estimation thing is impossible to get right, let's not put too much trust in being able to estimate accurately, but rather find ways of working that don't require estimating all work up front. I've seen that 1 gradually becomes 2 with experience (the experience of getting the estimates wrong every time).
- erickj 10y ago
  so true. the estimation game can work well on time scales of up to a few months (like 2 or 3)... maybe anything beyond that scale becomes the educated guessing game. no amount of analysis can flesh out every blocker that will be encountered. the deeper you go analyzing each task, the more the work just becomes doing that task.
- optforfon 10y ago
  I think you're talking about "first year of work" levels of experience. Estimation is hard - but a major goal of Agile development is to make it possible. I thought the same thing as you, but a month ago someone explained it to me in a way that makes a lot of sense https://news.ycombinator.com/item?id=11641316 https://news.ycombinator.com/item?id=11641316 Actually a lot of the comments about this PDF can be summed up with "practice Agile development"
  - LionessLover 10y ago
    How does that possibly solve the problem being discussed here? Are you the god who can define ALL tasks (in the agile meaning, a sprint consisting of a number of tasks) that a large project consists of up front, all the way to the end of the project 3 years later (for example)? Agile delivers because it doesn't attempt the impossible.
DrNuke 10y ago
Uh? It's not software eng or art but too many lions competing for too little a pie. CRUDs' marginal return is much better again (or as always).
greggman 10y ago
> Example of poor project management: Leonard Paul of Moderngroove stated that Modern Groove – The Ministry of Sound Edition suffered from poor project management because they had no dedicated project manager. Most of those responsibilities were given to the lead programmer, which resulted in an over burdened team. Paul said “I believe hiring a good manager early in the project and having clearer deadlines would have definitely helped the project.” > Takeaway: In order to avoid conflicts during the development process, teams need to have proper management. This seems like the exact opposite of Naughty Dog's advice http://www.latimes.com/business/technology/la-fi-tn-naughty-dog-uncharted-20160523-snap-htmlstory.html http://www.latimes.com/business/technology/la-fi-tn-naughty-...
norswap 10y ago
The conclusion: > Finally, based on our analysis of the data we collected, we make a few recommendations to game developers. First, be sure to practice good risk management techniques. This will help avoid some of the adverse e ects of obstacles that you may encounter during development. Second, prescribe to an iterative development process, and utilize prototypes as a method of proving features and concepts before commit- ting them to your design. Third, don't be overly ambitious in your design. Be reasonable, and take into account your schedule and budget before adding something to your de- sign. Building o of that, don't be overly optimistic with your scheduling. If you make an estimate that initially feels optimistic to you, don't give that estimate to your stake- holders. Revisit and reassess your design to form a better estimation. It's amazing how common-sensical this all his. But then again, we could all use a little more common sense sometimes. As Charlie Munger would put it: "It is remarkable how much long-term advantage people like [Warren Buffett and myself] have gotten by trying to be consistently not stupid, instead of trying to be very intelligent."
- LionessLover 10y ago
  When the results look bad it doesn't mean people don't follow common sense. For example, when the project would never get approval and a budget with realism but will get both when risks are left out when asking for money it's common sense to ignore risks. The team has a budget for another year, and what happens then is a different problem. The main objective of employees is not to get projects done, it is to get paid. Unfortunately our system has created perverse incentives: What manager or entrepreneur can afford to take a step back and say "what I'm proposing may not work, let's stop the project"? We have chosen to work under a pressure as if our lives are at stake (and they are - not our bodies, but our livelihood). The guy selling worthless shit, getting people to buy stuff they don't need, that may even be bad for them (or for those producing them) does not have problems. The same guy voluntarily giving up his job where he would have to do things he thinks are actually bad and/or useless is seen as a "parasite" on society - he's not working and we still have to feed him??? You bet all my estimates show the project is worthwhile if I don't have an (equally good) alternative.
xemdetia 10y ago
This is a strange paper to read because I think I have read most of the source material, something I certainly haven't felt before in a formal research paper! I was hoping that it would be more from the context of the publisher working with Microsoft PC/XBOX developers for new information though. Oh well, maybe next time.

91 points by akkartik 10y ago

>utilize prototypes as a method of proving features and concepts before committing them to your design. Third, don’t be overly ambitious in your design. Be reasonable, and take into account your schedule and budget before adding something to your design. Building off of that, don’t be overly optimistic with your scheduling. If you make an estimate that initially feels optimistic to you, don’t give that estimate to your stakeholders. Revisit and reassess your design to form a better estimation.

sillysaurus3 10y ago

This is excellent for the history and for the examples of what worked and what didn't. But the interpretations are a little dubious: Takeaway: Game developers should create a well-defined concept before beginning development as opposed to an ad hoc method of game design. This would exclude Dota, for example. Gamedev is a world of exceptions. In general, it's best to playtest every idea you come up with and cut whatever isn't fun.

stepvhen 10y ago
Dota was a user made mod originally, so I don't think that counts.
- sillysaurus3 10y ago
  Mods have been some of the most successful types of games, which seems like a pretty persuasive reason they should count. Some of the biggest names in the history of gaming started out as mods: Counterstrike, Team Fortress, etc. Valve's success is arguably because they find user-made mods and make them triple-A. Also, the advent of engines like Unity and UE blur what "modding" means. The WC3 engine was flexible enough to be made into Dota, which was then implemented on the Source Engine as Dota 2. But why was the original Dota a mod, whereas Dota 2 isn't? It's hard to come up with a definitive answer to that.
  - stepvhen 10y ago
    I think there is a misunderstanding: Dota doesn't apply to the quote "Game developers should create a well-defined concept ..." since it was neither conceptualized nor made by the Warcraft programmers. Additionally, the originally Dota started as a mod, was then published as a game, and Dota 2 would then not count as a mod. Its not hard to come up with a definitive answer to that. A game is a mod if it doesn't stand alone, if it requires _modifying_ the structure/code of another game.
    - sillysaurus3 10y ago
      The Warcraft 3 engine was flexible enough to implement Dota, just like the Unity engine was flexible enough to implement Rust. By your criteria, Rust is a mod, even though it's clearly not. It's better to say that mods are games and to treat them the same than to try to differentiate them. The same design process that yields a successful mod will yield a successful game.
      - yelnatz 10y ago
        Rust is not a mod, but is a clone of a mod of Arma2 called DayZ.
        sillysaurus3 10y ago
        This is a strange comment, because the first half is what I said. Rust isn't a mod. Yet their criteria would classify it as one, since the Unity programmers did not conceptualize Rust when they created Unity. But the second half is completely false. Rust has nothing whatsoever to do with DayZ, except that it's vaguely in the same genre.
  - chipsy 10y ago
    Modding basically recoups the investment into making a game's community - an individual can present their result to an existing player community directly, with a brief forum post, and a small website with some images or video to advertise. That is enough to build a small empire on, going by the most popular examples. That's different from the typical lifecycle of a commercial game, which has a deeper level of marketing work to do before it can even consolidate a playerbase. There isn't a "Unity modding community", per se. There is a community of Unity developers, and players of individual games made in Unity, who have no broader associations. A game transitioning from the mod sphere of an existing game to an independently produced commercial title still has to overcome this gap, and most of them don't make it over the line. The advice is sensible, nevertheless. If you can extend the schedule to focus on the core elements of the design as a "R&D" process where the majority must be thrown away, and only scale it up towards a shipping product as the concept proves itself, you minimize the risk of the budget being wasted on a flawed concept. Modding scenes, game jams, and micro-budget productions all have the benefit of weeding out most of the really early, risky design experiments, without wasting enough of people's time or money to care. Even when conceiving the marketing for a larger production, the same advice works. The "trial balloon" or "landing page" method, etc. You still do design thinking when you market, but it's design on the topics of "how do we build a funnel" or "how do we make this a franchise." Still very easy to spend a lot of money on making a splash without getting the blueprint right.
ekianjo 10y ago
This advice is super generic.
- sillysaurus3 10y ago
  No more generic than "Make something people want" or "Listen to your users." And yet most companies get these wrong, just like most game companies fail to playtest their ideas.
  - LionessLover 10y ago
    Why do you say "and yet" - since such generic "advice" is useless so is the conclusion "and yet they get it wrong!" (I told you something useless - why doesn't it help?)
    - sillysaurus3 10y ago
      Because if you follow this advice, you're more likely to yield a successful product. Playtesting isn't useless. It's how fun games are made. Perhaps people are disagreeing with this because they haven't tried to make a game. A curious phenomenon.
      - LionessLover 10y ago
        You simultaneously complain the advice is generic and claim it's useful. You always bet on both colors in roulette? No, generic advice is not useful. It's... too generic.
        deleted 10y ago
        [deleted]
      - squeaky-clean 10y ago
        But that's not advice. It's the same as telling a video game player that to win you need to "get a higher score than the enemy" and "play really well". And when someone loses, "I gave them such great advice, and yet they messed it up!" "Listen to your users" is not advice, it's the end goal. What's the most effective way to listen to your users? Professional playtesting? Open alpha/beta? Forums, IRC channel, a subreddit? Frequently scheduled AMA with the devs? Infrequently scheduled AMA with the devs?
flashman 10y ago
Having "a well-defined concept before beginning development" would seem to include Dota and other successful mods, because Valve is starting with a successful game concept (compared to the vast majority of mods that never find an audience). For Counter-Strike/TF2/Portal/Dota, Valve took a strong existing concept and developed it. It's exactly what's recommended in the takeaway.
- wingerlang 10y ago
  You are probably thinking of Dota 2. Both Dota, CS and TF where most likely developed ad-hoc as they were mods that grew organically.
  - ben_jones 10y ago
    Their were Age of Empire mods that were essentially MOBA scenarios. I don't doubt many such ideas are ad hoc genius, but a lot of game dev is re-hashing with a twist.

shmerl 10y ago

> When looking at games developed for one platform versus games developed for multiple platforms, we found a few advantages and disadvantages to choosing one over the other. First, 46% of developers that used multiple platforms listed art as something that went right, while only 33% of developers that used a single platform listed this. Additionally, 15% of the developers that used a single platform listed this as something that went wrong, while only 17% of developers that used multiple platforms listed art as having gone wrong. Therefore, the evidence shows us that games developed for multiple platforms may be more likely to have better game art. I didn't quite understand the correlation between development for multiple platforms and quality of art. May be those who want to release for multiple platforms in general care about quality more?

egypturnash 10y ago
Maybe "having your art under control" is a prerequisite for "being big and seasoned enough to work on multiple platforms at once". If all your games look like trash, nobody's going to give you the funds to scale up to the point where you can easily develop on multiple platforms.
- shmerl 10y ago
  I understand art beyond mere looks. It can include narrative, music, immersion and many other artistic aspects. May be the simple correlation is just being thorough and not superficial (which translates into reaching more users, rather than excluding everyone except one platform).
stepvhen 10y ago
I don't think it is a stretch to say that art directors don't have to worry about what platform the game is on, generally. Programmers, on the other hand, definitely have to worry. I imagine the art is seen in a better light on multi-platform games because a) what I first stated, and b) if something goes wrong in programming then "at least the art is good". I don't know if this is a specific kind of bias, but thats what it is. However I think the authors took a bit too much liberty with the conclusion (i.e. "may have better game art"). It sounds more like the designers of multiplatform games may just have a higher opinion on the art in the game, due to bias.

eudox 10y ago

Funny sidenote: the paper mentions a game titled Age of Ornithology under 5.1.g (Scope). Naturally I got excited at the idea of a game where you play some kind of bird collector, or an intrepid late 19th-century magnifying-glass-and-butterfly-net explorer and cataloguer of birds. But unfortunately this title is some kind of cruel joke: http://www.phobe.com/sfi/ornithology.html http://www.phobe.com/sfi/ornithology.html http://www.gamasutra.com/view/feature/130925/schandenfreudian_slips_age_of_.php http://www.gamasutra.com/view/feature/130925/schandenfreudia...

stepvhen 10y ago
I have a feeling this, and the rest of the games made by "Schadenfreude Interactive" aren't real, as much as I would love to play Grand Theft Ottoman or Accordion Hero, or Nazgul Thunder. Hannibal Crossing looks particularly enjoyable.
- eudox 10y ago
  Personally I'm an IF fan, and the "Swiss text adventure games, Zurich (1980), Beyond Zurich (1987), and Return to Zurich (1993)" look like an interesting saga.
- Exras 10y ago
  I saw the word 'Schadenfreude' and was reminded of a satire talkshow on a radio station in the PC game Grand Theft Auto IV Found it: Pacemaker - putting the heart back into the healthcare industry. https://youtu.be/jG9BgjnpGGY?t=940 https://youtu.be/jG9BgjnpGGY?t=940
clentaminator 10y ago
There's always Pokémon, as long as you're okay with trapping innocent creatures in tiny spheres and using them to battle as opposed to studying them (though you do get to catalogue them) ;)

Animats 10y ago

Reading this, it's clear that game development is now a mature technology. I was working on physics engines in the late 1990s, when nobody really knew how to do that very well. There were Gamasutra postmortems where the game physics went badly wrong and things flew apart. In the early years of the PS3, game projects were failing because the Cell architecture (a small PowerPC CPU plus six auxiliary Cell processors with tiny memory) required completely new game engine architectures. That was finally figured out, but development on the PS3 seemed to run about two years late, which set Sony back. Now there are mature game engines, mature physics engines, the graphics pipeline is in good shape, and the hardware is powerful enough. Problems in game development now are more about the game than the underlying technology. (Pre-1990s, game development was a cram job, trying to get stuff done on limited hardware.)

greggman 10y ago
apparently they aren't though. People porting from PC to other platforms are still finding huge perf issues and have to refactor tons of code. I thought this gen (PS4/XboxOne) were going to be easy ports but from every team I've talked to they aren't
- mjevans 10y ago
  A lot of the criticisms I've seen for failed ports describe the exact type of behavior you would expect from an emulated system. The ports you reference behave like game systems that used to occupy 100% or almost 100% of the resource capacity of a dedicated system failing to execute correctly when they're running in a non-dedicated environment. They also had the advantage of targeting an exact set of hardware specifications in other ways and didn't have to scale down, up, or sideways (other ways of dividing labor) to the way that other platforms work. In short, they weren't a game designed to scale to a theoretical platform that then ran on the platforms, they were fit precisely to a single platform and then weren't designed to be portable at all.
  - Narishma 10y ago
    I think you got that backwards. Parent was talking about porting from PC to consoles.

ComodoHacker 10y ago

I wonder how long before this kind of analysis can be performed by AI just as efficiently and will be offered as a web service.

13hours 10y ago

"To avoid schedule slippage, game developers need to spend more time to plan out all the work that needs to be done so that no tasks are overlooked when giving estimates. By planning out tasks, you can also estimate each task individually, and give a more accurate prediction, instead of an optimistic one." When things don't work out according to the schedule, there are generally two reactions : 1. We didn't estimate well enough, we need to estimate better next time. 2. Maybe this estimation thing is impossible to get right, let's not put too much trust in being able to estimate accurately, but rather find ways of working that don't require estimating all work up front. I've seen that 1 gradually becomes 2 with experience (the experience of getting the estimates wrong every time).

erickj 10y ago
so true. the estimation game can work well on time scales of up to a few months (like 2 or 3)... maybe anything beyond that scale becomes the educated guessing game. no amount of analysis can flesh out every blocker that will be encountered. the deeper you go analyzing each task, the more the work just becomes doing that task.
optforfon 10y ago
I think you're talking about "first year of work" levels of experience. Estimation is hard - but a major goal of Agile development is to make it possible. I thought the same thing as you, but a month ago someone explained it to me in a way that makes a lot of sense https://news.ycombinator.com/item?id=11641316 https://news.ycombinator.com/item?id=11641316 Actually a lot of the comments about this PDF can be summed up with "practice Agile development"
- LionessLover 10y ago
  How does that possibly solve the problem being discussed here? Are you the god who can define ALL tasks (in the agile meaning, a sprint consisting of a number of tasks) that a large project consists of up front, all the way to the end of the project 3 years later (for example)? Agile delivers because it doesn't attempt the impossible.

DrNuke 10y ago

Uh? It's not software eng or art but too many lions competing for too little a pie. CRUDs' marginal return is much better again (or as always).

greggman 10y ago

> Example of poor project management: Leonard Paul of Moderngroove stated that Modern Groove – The Ministry of Sound Edition suffered from poor project management because they had no dedicated project manager. Most of those responsibilities were given to the lead programmer, which resulted in an over burdened team. Paul said “I believe hiring a good manager early in the project and having clearer deadlines would have definitely helped the project.” > Takeaway: In order to avoid conflicts during the development process, teams need to have proper management. This seems like the exact opposite of Naughty Dog's advice http://www.latimes.com/business/technology/la-fi-tn-naughty-dog-uncharted-20160523-snap-htmlstory.html http://www.latimes.com/business/technology/la-fi-tn-naughty-...

norswap 10y ago

The conclusion: > Finally, based on our analysis of the data we collected, we make a few recommendations to game developers. First, be sure to practice good risk management techniques. This will help avoid some of the adverse e ects of obstacles that you may encounter during development. Second, prescribe to an iterative development process, and utilize prototypes as a method of proving features and concepts before commit- ting them to your design. Third, don't be overly ambitious in your design. Be reasonable, and take into account your schedule and budget before adding something to your de- sign. Building o of that, don't be overly optimistic with your scheduling. If you make an estimate that initially feels optimistic to you, don't give that estimate to your stake- holders. Revisit and reassess your design to form a better estimation. It's amazing how common-sensical this all his. But then again, we could all use a little more common sense sometimes. As Charlie Munger would put it: "It is remarkable how much long-term advantage people like [Warren Buffett and myself] have gotten by trying to be consistently not stupid, instead of trying to be very intelligent."

LionessLover 10y ago
When the results look bad it doesn't mean people don't follow common sense. For example, when the project would never get approval and a budget with realism but will get both when risks are left out when asking for money it's common sense to ignore risks. The team has a budget for another year, and what happens then is a different problem. The main objective of employees is not to get projects done, it is to get paid. Unfortunately our system has created perverse incentives: What manager or entrepreneur can afford to take a step back and say "what I'm proposing may not work, let's stop the project"? We have chosen to work under a pressure as if our lives are at stake (and they are - not our bodies, but our livelihood). The guy selling worthless shit, getting people to buy stuff they don't need, that may even be bad for them (or for those producing them) does not have problems. The same guy voluntarily giving up his job where he would have to do things he thinks are actually bad and/or useless is seen as a "parasite" on society - he's not working and we still have to feed him??? You bet all my estimates show the project is worthwhile if I don't have an (equally good) alternative.

xemdetia 10y ago

This is a strange paper to read because I think I have read most of the source material, something I certainly haven't felt before in a formal research paper! I was hoping that it would be more from the context of the publisher working with Microsoft PC/XBOX developers for new information though. Oh well, maybe next time.