32 ms·

I wanted to believe that a team of 6 to 10 people could make a game that looked and felt AAA. Boy was I wrong! When we made Crash Bandicoot (with a team of 7),

I wanted to believe that a team of 6 to 10 people could make a game that looked and felt AAA. Boy was I wrong! When we made Crash Bandicoot (with a team of 7), it was already virtually impossible to make a AAA game with 6-10 people, and that was 20 years ago. I tell inexperienced entrepreneurs to take their honest best estimate and multiply by 10. Or, as Mark Cerny (our producer on Crash) used to tell us, "add one and increase the unit: 1 week = 2 months; 2 months = 3 years; 3 years = you're doomed". For a less anecdotal version, read The Mythical Man Month. (The factor he arrives at is 9.)

jnardiello 11y ago
Sir, you made my days when I was a teenager. Thank you.
ivanca 11y ago
Offtpic but you worked on Crash Bandicoot!? That's so awesome. I loved each version of that game and it made our childhoods much more fun (at countries all over the world, just as an example I'm from Colombia)
- pixelbath 11y ago
  You might enjoy this then: http://all-things-andy-gavin.com/2011/02/02/making-crash-bandicoot-part-1/ http://all-things-andy-gavin.com/2011/02/02/making-crash-ban...
- emilsedgh 11y ago
  Confirming from Iran :)
- dmbaggett 11y ago
  I never get tired of hearing that -- thank you! :)
  - zarkone 11y ago
    I've tried and beat Crash Bandicoot and Crash Cart for the first time this year. (I'm 25, and neither me nor my friends had PS1 in my childhood). While playing this game I was like always been on the bleeding edge between difficulty, wishing to give up and encouragement... I mean, I just couldn't resist, -- I couldn't turn off the console, in spite of the fact that I had to re-run some levels about ~50-80 times to beat.. It was amazing, very challenging journey, i had always been feeling that arcade spirit while playing, which is such rarity nowdays.
    - freditup 11y ago
      I think that's part of what made the adventure Crash games such a success. They were challenging, but not impossible; death was penalized fairly; there were always more things to discover and collect. In addition, the games were easy enough to learn but not overly simplistic. The games weren't ground-breaking, but the balance and polish were fantastic and made the games great. I feel like CTR was slightly easier, but still a great arcade racing game.
- freditup 11y ago
  I as well loved Crash Bandicoot. Playing the first three games plus CTR with my siblings and cousins are part of my favorite childhood memories. We had so much good fun with those games! (From the US)
- aidenn0 11y ago
  Agreed. I've been gaming for about 30 years, and Crash Bandicoot is one of the games that stands-out as best-all-around for its time. It's really hard to convey how good the game looked in '96, and the gameplay was just fun. The combination of ahead-of-its-time graphics with really fun gameplay is rare, particularly after the 80s.
Balgair 11y ago
Great Book tip, thank you. For others, here is a summary version of the book: http://javatroopers.com/Mythical_Man_Month.html http://javatroopers.com/Mythical_Man_Month.html
kin 11y ago
Believe it or not I'm star struck. I remember reading an interview about how you guys dealt with fitting such a large game into such a small space with a paging system. Amazing and inspiring! Thanks for the childhood memories :)
Mahn 11y ago
Properly estimating is probably one of the hardest things to do in our field, specially so if you are starting a product or company where you have to go with your gut for a lot of the decisions. I think the key to survive this is not so much learning to get estimates right, but being prepared for them to be wrong, that's probably why the "multiplicative factor" gets thrown around so often. The good thing is that the more experience you have at estimating things wrong, the better you'll become at guessing what the "multiplicative factor" should be. I started with 10 a few years ago, nowadays it's down to 4 :)
- bhartzer 11y ago
  Properly estimating is not just difficult in your field, it's difficult anywhere in 'software' or tech. You always run into issues along the way, and you have to deal with them.
  - ergothus 11y ago
    I usually attribute this to the idea of reuse: by definition, every software problem we are solving is "new", because if it wasn't new we'd just use existing solutions and there'd be no team. (Now there are people that insist on resolving problems, but they put themselves into the "new" category by choice). New problems mean unknown ways to fail === hard to estimate
michaelt 11y ago
Also, the original kickstarter [1] was for $50,000 with $72,000 raised. I'm not an expert on the economics of the video game industry, but I can't believe that would pay a team of 6-10 people for very long. [1] https://www.kickstarter.com/projects/45588301/woolfe-the-red-hood-diaries/description https://www.kickstarter.com/projects/45588301/woolfe-the-red...
- jeremiep 11y ago
  Barely one programmer for less than a year.
- dragontamer 11y ago
  It was probably enough for a loan however. Which is probably why the company folded at this time, because they ran out of funding and couldn't make it. With that said, I've been following Patreon and the financial models at Patreon just make more sense than Kickstarter. Various game development groups have constantly slipped their schedule, and backers respond typically by keeping the donations steady. As long as the game developer releases a demo demonstrating progress, backers are more than eager to continue their $5/month or so funding. The main issue is the slow pace of early Patreon funding. A lot of games have to bootstrap with $500 / month or $1000/month for two or three months. Even "successful" Patreons only get $3000/month to $5000/month, and those are few and far between. (Patreon doesn't seem to have the reach of Kickstarter or Indiegogo quite yet. Perhaps if Patreon expanded and grew it'd be better?). The Patreon model is also better for "subscription" stuffs. Podcasts, youtube videos and the like are more popular there. I can't find too many games on Patreon by searching, but I've seen them advertized on various forums. In any case, even with $5000/month, it'd be hard to run a team of 6 on such meager money. I've been following a team of 3 and they're barely making it on that. But Patreon does strike me as more plausible than Kickstarter. ----------- On the other hand, Kickstarter is extremely good at measuring "hype". So perhaps the Kickstarter model is better if you're trying to prove to outside funding groups that your team is worth investing into.
  - chx 11y ago
    Patreon seems awesome for one person teams: musicians, cartoonists etc. One of my favorites, Peter Hollens is above $7k/month now.
jeremiep 11y ago
We just ported an AAA game with a team of 6 programmers and barely made it through with massive delays. The game was already feature complete and bug free when we got the source, all we needed to do was port it to mobile devices. It's so easy to underestimate the work to be done, graphics easily break when you change the underlying hardware and renderers, gameplay had to be adapted to touch controls, etc. It didn't help that our estimate multiplier was only 3. All of this gave me a much greater appreciation of the work you guys did at Naughty Dog.
rkangel 11y ago
Issues with estimating, particularly something like this, mostly comes down to meta-ignorance: you don't know what you don't know. Given a particular problem of a similar sort to something he's worked on, an experienced engineer will be able to list 10 different issues that might crop up. An inexperienced engineer won't know of those potential problems and will just see the most optimistic path possible. When you're moving massively outside your familiar zone, many unexpected problems will crop up. Best solution is a phased approach. Pick the bits you judge as most risky (you should hopefully have some idea) and implement prototypes in those areas, in the hope that most of the scaries will crop up. You've then got a better idea as you move on. I've written this in the context of doing engineering, but it's equally true in all fields - like trying to sell a product into a new an unfamiliar market.
- ChuckMcM 11y ago
  I like that term 'meta ignorance'. It is important to realize that ignorance isn't "bad" it can be fixed by learning, but its good to know that you have it so you can start fixing it. The other aspect of this that I see, even in seasoned engineers, is the 'presumption of simplicity'. Some things look really simple, but inside they are actually quite complicated. A smart phone is just a screen, a computer, a battery and a radio right? Too many times people see something, think "that doesn't look too hard I bet I could do that in a weekend"[1] but don't actually follow through on that thought and spend a weekend building what ever it was that was so easy. Sometimes the more experience you have the more dangerous this is. I can recall times in my own life we're I've grossly underestimated the difficulty of stuff, its lead me to be a lot more conscious about things I know about and things I think I know something about, especially when predicting schedules and work effort. [1] A very common theme on acquihire stories here.
  - forgetsusername 11y ago
    >It is important to realize that ignorance isn't "bad" it can be fixed by learning In the parlance of Mr. Rumsfeld, you can learn from the known unknowns, but often you don't even discover the unknown unknowns (the "meta-ignorance") until it's too late.
malkia 11y ago
For NHL2k2/NHL2k3 we had dozen of people just doing the UI - Granted good UI is crucial in a sports game. That also was back in 2001
- bluedino 11y ago
  For something like that, do those people work full time on just that one game? Or are they also working on NFL/NBA 2k2 at the same time, creating UI code that works across all the games in the series?
  - malkia 11y ago
    We were not a typical sports game studio (Treyarch), and the other sport project that was done at the same time was for EA (some baseball game), while the one I was and talking about was for SEGA. It was a bit of a problem really, so people were not allowed to get mixed into teams. After that game Treyarch moved to do mainly Spiderman, and then Call of Duty. The latest COD also has quite a spectacular UI system, but much less coders are involved into bringing it up. But today's standards are even much higher - cool animations, links, socially enabled (twitch, facebook, etc.). Some studio have made success with flash-compatible renderers, and we even thought for NHL2K2 to use one, but first it was only compatible with the Windows version of the Dreamcast (okay, I barely remember now, as we didn't used it, but there was a choice whether you can ship your game with some barebones NT system on the Dreamcast - but you had to lose 2mb to the oS). So that middleware (I think) required it, and it wanted somewhere from 2-4mb more on top. One big problem, back in the days, with the UI is that you can have tons of memory in your main menu, but you barely have anything while the game is playing. So your "PAUSE" menu might use completely different code path, and look quite differently. The alternative is to reload things after PAUSE, but this decreases the quality of the product, also much harder to test, and you would hear the DISC "screaming" soo much :) Nonetheless Dreamcast was fun to work on, spare the compiler, and the debugger :)
gambiting 11y ago
Completely agreed. I work in a large games studio and on our project the team that does nothing else but makes sure the game is compliant(so that it is accepted by Sony and Microsoft for release) consists of 7 people - 5 programmers, a designer and a QC tester. I imagine doing the work we are doing for an indie game would be extremely time consuming.
TomGullen 11y ago
What does "AAA Game" mean anyway? I'm a bit lost on the definition. Wikipedia says: > In the video game industry, AAA (pronounced "triple A") is a classification term used for games with the highest development budgets and levels of promotion. A title considered to be AAA is therefore expected to be a high quality game and to be among the year's bestsellers. By definition, a budget of $75k + 6-10 people wont make an AAA game. > When we made Crash Bandicoot (with a team of 7), it was already virtually impossible to make a AAA game with 6-10 people, and that was 20 years ago. I'm biased/invested in this area, but I think nowadays tools are really helping strip away the technical challenges of making a game. Increasingly game development is less about technical challenges but more about the creative challenge. Individuals are willing to pour thousands of hours of work into artistic and creative projects, and I am seeing far less resistance and challenges going forwards for these sorts of people to produce AAA quality games, when previously game creation was extremely inaccessible to them. Also, thank you for Crash Bandicoot. Amazing game :) One I loved playing growing up!
- vimota 11y ago
  It seems like they meant AAA-quality game, with a budget of $75k + 6-10 people
  - robotkilla 11y ago
    I think you're correct... saying that you created a AAA title means that you can compete with major game studios... I think. However, in my opinion AAA doesn't apply to scope or budget but quality of experience. Super Meat Boy is one of the best games I've played in years and it doesn't come anywhere near the scope of say GTA 5 (which is also awesome). I don't think anyone calls SMB a triple-a title though... so I'd say wikipedia's definition is correct. So then, I think the real issue here is aiming to make a game that compete with major studios who have budgets of $50m and employ hundreds of specialized team members. These devs claim it was a passion project, but then they throw in the towel on the entire concept of running an indie studio after the negative reviews come in. I understand the bankruptcy move, but if I were the founder's shoes, I would move somewhere cheap, take a side job and keep making games on the side. If I still loved the Wolfie project, I would fix it for free and I would find a way to send my KS backers whatever I owe them. I would read every negative review of my game, try to bucket the feed back (something like: legit, semi-legit, ignore) and then fix the game. I would also print my favorite positive reviews and hang them up around the house for motivation. If I no longer believed in the game, I would build something new but smaller. 13 years is a long time to wash down the drain due to one failure. Currently I'm trying to make indie games along with my gilfriend. She has no formal 3d art training but has been teaching herself via the internet. I learned Unity and have been teaching myself c# / cg and all the other shit that goes with 3d game development (I come from a python / js background). We will most likely fail hard but I don't understand the concept of quitting when you got into something for fun in the first place. edit: it also sounds like these guys tried to build their own engine... or something. If they did, that was a bad idea. If they didn't, I don't understand why they had so many issues with collision detection. Haven't played the game though... its just odd to hear that they had major collision detection issues.
catshirt 11y ago
hey i'm sure people following you around telling you this probably gets old but i started playing Crash when it came out, i was 8. i'm 27 now and working in gaming. i just wanted to say Crash was really a huge part of my life and i would be doing us both a disservice not to take this opportunity to say thanks for such a great work of art!!!
danabramov 11y ago
Off topic, but thank you for CB, it was my favorite game for years. (Fun anecdote: I'm from Russia and I didn't know English when I was a child. I had no idea what "save game" means so I started Crash Bandicoot 3 from scratch every time. Imagine my surprise when I realized how saving works.)
w0utert 11y ago
>> .. or, as Mark Cerny (our producer on Crash) used to tell us, "add one and increase the unit" Reading this made my day, it's the exact same rule I try to apply to my own effort estimate, to the letter. I was 100% convinced I came up with it myself, but now I'm starting to think I may have (unconsciously) picked it up from someone else at some point ;-)
- EdiX 11y ago
  It's from Murphy's Law.
atmosx 11y ago
You made 'Crash B' ? Boy you're a myth! It was one of the few games I wasn't bored to death while playing :-) congrats!
chazu 11y ago
Was that title developed using lisp? If so (or if you worked on other Naughty Dog games which did use lisp) do you feel that it was a force multiplier in any way?
- archimedespi 11y ago
  Yes! GOOL/GOAL (see [1]) was a lisp-family language. See [2] for an explanation by the guy who developed it. [1] - https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp [2] - http://all-things-andy-gavin.com/2011/03/12/making-crash-bandicoot-gool-part-9/ http://all-things-andy-gavin.com/2011/03/12/making-crash-ban...
lewisjoe 11y ago
You are the guys who made Crash Bandicoot? Thanks for making my childhood awesome.
kraig911 11y ago
Ok that's great that means you can make another one!
arethuza 11y ago
"estimate and multiply by 10" Reminds me of these rules: - Multiply by π for small, known teams, doing a new job, - by π^2 for unknown teams, - and √π for known teams who have already started, know their work and have a good track record. From http://alistair.cockburn.us/The+magic+of+pi+for+project+managers http://alistair.cockburn.us/The+magic+of+pi+for+project+mana...
VonGuard 11y ago
Steve Mcconnell has a great book on software estimation, but it's about enterprise software, where there are requirements and goals. [ http://www.stevemcconnell.com/est.htm http://www.stevemcconnell.com/est.htm ] The biggest problem with making games is the fact that there's no end goal of "Done." It's just a vague point in time, really. Is it done when you've made 10 levels or 11? Is it done when you've added 10 power-ups or 11, or 15? In a movie, you can't make the thing 50 hours long, no one would watch it all. But for a game, 50 hours is now considered medium sized, compared to MMO's and MOBAs that offer hundreds of hours of time to play. Scope creep is the biggest danger in the games industry. Reading this update, where they talk about going from 2D to 3D, my heart broke. That's such a major change, and as he writes, such a major uptick in difficulty, it was the beginning of the end. Terry Cavanaugh [http://distractionware.com/blog/ http://distractionware.com/blog/ ] gets scope right. Valve gets scope right. You can tell when it's done right because the pacing of the game feels damn near perfect, like Half-Life 2: there's little down time, and you're always moving forward, even if yer not always shooting. I feel bad for the Woolfe team, but they clearly had rose colored glasses on. There's a very good reason modern console games cost at least 8 figures. Hell, even with 8 figures of budget, many big name titles aren't even that good. The games industry is a harsh mistress. Like judging what's funny for a comedian, judging what's fun for a game developer is the toughest task they have.
eecks 11y ago
I was just playing Crash Team Racing a few days ago and it gave me a lot of good memories of my childhood. Thanks!
- spike021 11y ago
  I love that game! For my brother and I it was the alternative to Mario Kart since we didn't have a Nintendo console at the time. So much fun though.
  - eecks 11y ago
    I'd buy an updated version (online) in a heartbeat
    - spike021 11y ago
      Same.
dopamean 11y ago
I wasn't really into video games as a kid but my father bought me a Playstation (the first one) used from some place and it came with a copy of Crash Bandicoot. It is the only video game I can say I've played. It was my favorite thing growing up. Such good times. Thank you!
nadams 11y ago
> I wanted to believe that a team of 6 to 10 people could make a game that looked and felt AAA. What is your definition of AAA game? I believe Minecraft was created by 1 person (or at least started). I consider Terraria to be a AAA game and that was made by 3-4 people.
- oaktowner 11y ago
  In the parlance of the industry, AAA is indicative only of budget: "[the term AAA is] used for games with the highest development budgets and levels of promotion."[1] [1] https://en.wikipedia.org/wiki/AAA_(video_game_industry) https://en.wikipedia.org/wiki/AAA_(video_game_industry)
pitchups 11y ago
There is a nice long list of over 100 answers to the question of why software estimates are off at Quora: http://www.quora.com/Engineering-Management/Why-are-software-development-task-estimations-regularly-off-by-a-factor-of-2-3?share=1 http://www.quora.com/Engineering-Management/Why-are-software...
balls187 11y ago
Tangentially related, this is a fascinating read: http://www.quora.com/How-did-game-developers-pack-entire-games-into-so-little-memory-twenty-five-years-ago/answer/Dave-Baggett http://www.quora.com/How-did-game-developers-pack-entire-gam...
- flipmonk 11y ago
  Wow. "Ultimately Crash fit into the PS1's memory with 4 bytes to spare. Yes, 4 bytes out of 2097152. Good times." - the things we take for granted today!
blakeyrat 11y ago
From reading the reviews of the game on Steam, the main complaints are that it's buggy (especially with regards to enemy AI), and it's incredibly short (not even 2 hours' worth of content.) There are a lot of $10 games that will entertain for 4-5 or more hours (even excluding the outliers like Terraria). I'm surprised that the developer was surprised at the perceived value of their product. That said, he's right that the market for indie games is ridiculously saturated right now.
- robryan 11y ago
  Yes, unless you are have a really novel concept that will make people see past early bugs (such as ARK: Survival Evolved) you are competing with some games that set the bar very high. For example something like Ori and the Blind Forest, not exactly indie but a $20 title with amazing graphics and sound and around 6-8 hours of bug free gameplay.
SilasX 11y ago
Stupid, deceptively-simple question: Why can't you do something like "Game on Rails", where you set up a basic framework that has all the stuff a game normally needs, and then you only have to layer your assets and logic on top of it? Do game devs already use something like that and still have these delays?
- shortstuffsushi 11y ago
  There are lots of frameworks for game development, but the problem is exactly in what you've described: rails. You can only get so far within the constraints of those frameworks before you need something it just can't do, and have to develop yourself. Depending on the framework, this may not even be possible (which is what leads to creation of new frameworks). With that said, a lot of great (albeit typically somewhat simple) games are made with frameworks. When I when to a few game jams back in college, we typically started with something so that we could have a "complete" game by the end.
  - pimlottc 11y ago
    Also, using an existing engine often means giving up the possibility of having unique features or capabilities. Having the only game with multidimensional quad-resolution bumpmap tensors (in realtime!) can be a major selling point. Of course, in most cases, you'll get a better overall game by focusing on the art, story, gameplay, etc., but that doesn't always translate to sales.
- Impossible 11y ago
  Yes http://www.unity3d.com/ http://www.unity3d.com/ is essentially "Game on Rails". But, just as you can quickly and easily build a failed startup with Rails, Unity doesn't really save you from building a game that no one really wants.
- chipsy 11y ago
  It's a fidelity problem. If you restrict yourself to very simple interactions and behaviors, you can make a Twine game and get the effect you're looking for without touching code. But then your scope gets more ambitious and you want characters to be fully realized, a mobile camera, voice acting, combat simulation, and so on... and then the asset pipeline blows up into something way bigger, every interaction requires additional scripting steps, you start needing to customize the rendering code... No one of those features stops you, but you get a "death by paper cuts" effect, because you eventually hit a mine that requires original technology to be written - most commonly, something to do with collision code. A good team led by a competent producer is able to figure out how to get the right effect within the budget, but it's never an easy process, and game productions have a habit of "pitching the moon and shipping with swiss cheese".
joonap 11y ago
Thanks for Crash Bandicoot! Still one of my favourite games!
ransom1538 11y ago
I was in gaming for 4 years or so. It was horrifying to see how much code that is not reused (excluding math/physics libraries). IMHO - The art pipline is the real time suck and can make or break a game deadline. Just think of the staff alone to pull off a art pipeline. Producer -> Game Play expert -> Artist -> tools engineer -> graphics/engine guys -> game designers/ai guys. Just that chain alone from a tech perspective: LAU, databases, openGL, Maya3d, C++ tools, art vector tools, phyics engines, ai engines which are mostly proprietary. Budgets were like movies. Millions and millions. Oh yeah. And most are canceled after years of hard work. I think if you are making a 3d game you haven't thought things through.
fapjacks 11y ago
Crash Bandicoot is an awesome game! :) Also there is a parallel between your "add one and increase the unit" and the estimation of what kind of manpower you need to take an defended objective. The wisdom here is basically the same: If you are facing an objective defended by a squad, you will need a platoon to take it. If you are looking at a platoon dug in to the objective, you will need a company to take it. An entrenched company needs to be uprooted by a battalion, et cetera. It's from this (The Mythical Man Month as you mention, and the estimations the infantry taught me) that I have built a kind of mental model for estimating things, that ends up being sort of universal like the 80-20 rule.
Bartweiss 11y ago
This, I think, is fundamentally the equation driving modern game design and the "indie revolution". For AAA games, modern, high-end systems increase the difficulty of providing a product. Modern development tools help, but graphics demands and expectations are inexorably growing. In a world where animating reflections might be a full-time task, 6-10 people can't even count on developing a trailer for a AAA product. In response, then, indie games. Why the rebirth of pixel art and simple, geometric visuals? Not because it's 'retro' or 'clean', but because it's easy. Procedural generation, low-poly graphics, and user generated content aren't popular because they produce the best products, they're popular because they're all decisions that slash the number of employees you need. Woolfe's mistake was setting out to make their game as good as it could possibly be. For an indie team today that's simply not possible. To paraphrase your producer, "3 years = Duke Nukem Forever".
cylinder 11y ago
Thanks for creating a fun memory of my childhood.
ild 11y ago
Yet STALKER was not vaporware after all.
rurban 11y ago
I lived with this Mark Cerny rule for a few decades, and it was mostly true. But finally after I'm over 50 I came to proper and good estimates now. Mostly because I'm a much more experienced now. 2d == 1.5d, 1w == 5d and so on. But game programming is like architecture. There's so much more to do.
- themodelplumber 11y ago
  Can you explain a bit more? I'm intrigued. What does 2d == 1.5d mean? Was that how close you were nailing your estimates? If so, how did you get to that point?
automathematics 11y ago
Yeah, we made Fallout New Vegas was a team of ~50. Now I feel bad. I didn't know CB had such a small team! Props!
gobengo 11y ago
Reading this right now and I think it's 6, not 9. But yes.
hvidgaard 11y ago
After reading the Mythical Man Month, I derived an estimation scheme that was something like this: 1. I know this, I have done it before in this codebase. Estimate is accurate, multiply by 2 to account for watercooler talk, documentation, and other non coding work. 2. I know this codebase, I have a pretty good idea of what to do, how, and where. I have walked trough the code and can't identify any major roadblocks. Multiply by 5 3. I don't have an honest idea. Multiply by 20, and make it clear that it's a guesstimate at best. Usually a better estimate can be given after working on it some time. It's still off more than I like.
erikb 11y ago
I'm so surprised that there are people who actually have a sense of things. I try to tell others all the time that in software having an inexperienced developer you need to multiply with 3x for best hopes and with 10x for common case, sometimes even experienced devs ending up between 2x and 5x. People often don't believe me, some reason being that some developers really deliver on time and sell it as finished but then having the rest of the team fixing bugs and working around strange APIs for months. Often it's just that they also haven't experienced the pain of developing something of reasonable scale and stability. No man, these are not tears, I just got something in the eye.

jnardiello 11y ago

Sir, you made my days when I was a teenager. Thank you.

ivanca 11y ago

Offtpic but you worked on Crash Bandicoot!? That's so awesome. I loved each version of that game and it made our childhoods much more fun (at countries all over the world, just as an example I'm from Colombia)

pixelbath 11y ago
You might enjoy this then: http://all-things-andy-gavin.com/2011/02/02/making-crash-bandicoot-part-1/ http://all-things-andy-gavin.com/2011/02/02/making-crash-ban...
emilsedgh 11y ago
Confirming from Iran :)
dmbaggett 11y ago
I never get tired of hearing that -- thank you! :)
- zarkone 11y ago
  I've tried and beat Crash Bandicoot and Crash Cart for the first time this year. (I'm 25, and neither me nor my friends had PS1 in my childhood). While playing this game I was like always been on the bleeding edge between difficulty, wishing to give up and encouragement... I mean, I just couldn't resist, -- I couldn't turn off the console, in spite of the fact that I had to re-run some levels about ~50-80 times to beat.. It was amazing, very challenging journey, i had always been feeling that arcade spirit while playing, which is such rarity nowdays.
  - freditup 11y ago
    I think that's part of what made the adventure Crash games such a success. They were challenging, but not impossible; death was penalized fairly; there were always more things to discover and collect. In addition, the games were easy enough to learn but not overly simplistic. The games weren't ground-breaking, but the balance and polish were fantastic and made the games great. I feel like CTR was slightly easier, but still a great arcade racing game.
freditup 11y ago
I as well loved Crash Bandicoot. Playing the first three games plus CTR with my siblings and cousins are part of my favorite childhood memories. We had so much good fun with those games! (From the US)
aidenn0 11y ago
Agreed. I've been gaming for about 30 years, and Crash Bandicoot is one of the games that stands-out as best-all-around for its time. It's really hard to convey how good the game looked in '96, and the gameplay was just fun. The combination of ahead-of-its-time graphics with really fun gameplay is rare, particularly after the 80s.

Balgair 11y ago

Great Book tip, thank you. For others, here is a summary version of the book: http://javatroopers.com/Mythical_Man_Month.html http://javatroopers.com/Mythical_Man_Month.html

kin 11y ago

Believe it or not I'm star struck. I remember reading an interview about how you guys dealt with fitting such a large game into such a small space with a paging system. Amazing and inspiring! Thanks for the childhood memories :)

Mahn 11y ago

Properly estimating is probably one of the hardest things to do in our field, specially so if you are starting a product or company where you have to go with your gut for a lot of the decisions. I think the key to survive this is not so much learning to get estimates right, but being prepared for them to be wrong, that's probably why the "multiplicative factor" gets thrown around so often. The good thing is that the more experience you have at estimating things wrong, the better you'll become at guessing what the "multiplicative factor" should be. I started with 10 a few years ago, nowadays it's down to 4 :)

bhartzer 11y ago
Properly estimating is not just difficult in your field, it's difficult anywhere in 'software' or tech. You always run into issues along the way, and you have to deal with them.
- ergothus 11y ago
  I usually attribute this to the idea of reuse: by definition, every software problem we are solving is "new", because if it wasn't new we'd just use existing solutions and there'd be no team. (Now there are people that insist on resolving problems, but they put themselves into the "new" category by choice). New problems mean unknown ways to fail === hard to estimate

michaelt 11y ago

Also, the original kickstarter [1] was for $50,000 with $72,000 raised. I'm not an expert on the economics of the video game industry, but I can't believe that would pay a team of 6-10 people for very long. [1] https://www.kickstarter.com/projects/45588301/woolfe-the-red-hood-diaries/description https://www.kickstarter.com/projects/45588301/woolfe-the-red...

jeremiep 11y ago
Barely one programmer for less than a year.
dragontamer 11y ago
It was probably enough for a loan however. Which is probably why the company folded at this time, because they ran out of funding and couldn't make it. With that said, I've been following Patreon and the financial models at Patreon just make more sense than Kickstarter. Various game development groups have constantly slipped their schedule, and backers respond typically by keeping the donations steady. As long as the game developer releases a demo demonstrating progress, backers are more than eager to continue their $5/month or so funding. The main issue is the slow pace of early Patreon funding. A lot of games have to bootstrap with $500 / month or $1000/month for two or three months. Even "successful" Patreons only get $3000/month to $5000/month, and those are few and far between. (Patreon doesn't seem to have the reach of Kickstarter or Indiegogo quite yet. Perhaps if Patreon expanded and grew it'd be better?). The Patreon model is also better for "subscription" stuffs. Podcasts, youtube videos and the like are more popular there. I can't find too many games on Patreon by searching, but I've seen them advertized on various forums. In any case, even with $5000/month, it'd be hard to run a team of 6 on such meager money. I've been following a team of 3 and they're barely making it on that. But Patreon does strike me as more plausible than Kickstarter. ----------- On the other hand, Kickstarter is extremely good at measuring "hype". So perhaps the Kickstarter model is better if you're trying to prove to outside funding groups that your team is worth investing into.
- chx 11y ago
  Patreon seems awesome for one person teams: musicians, cartoonists etc. One of my favorites, Peter Hollens is above $7k/month now.

jeremiep 11y ago

We just ported an AAA game with a team of 6 programmers and barely made it through with massive delays. The game was already feature complete and bug free when we got the source, all we needed to do was port it to mobile devices. It's so easy to underestimate the work to be done, graphics easily break when you change the underlying hardware and renderers, gameplay had to be adapted to touch controls, etc. It didn't help that our estimate multiplier was only 3. All of this gave me a much greater appreciation of the work you guys did at Naughty Dog.

rkangel 11y ago

Issues with estimating, particularly something like this, mostly comes down to meta-ignorance: you don't know what you don't know. Given a particular problem of a similar sort to something he's worked on, an experienced engineer will be able to list 10 different issues that might crop up. An inexperienced engineer won't know of those potential problems and will just see the most optimistic path possible. When you're moving massively outside your familiar zone, many unexpected problems will crop up. Best solution is a phased approach. Pick the bits you judge as most risky (you should hopefully have some idea) and implement prototypes in those areas, in the hope that most of the scaries will crop up. You've then got a better idea as you move on. I've written this in the context of doing engineering, but it's equally true in all fields - like trying to sell a product into a new an unfamiliar market.

ChuckMcM 11y ago
I like that term 'meta ignorance'. It is important to realize that ignorance isn't "bad" it can be fixed by learning, but its good to know that you have it so you can start fixing it. The other aspect of this that I see, even in seasoned engineers, is the 'presumption of simplicity'. Some things look really simple, but inside they are actually quite complicated. A smart phone is just a screen, a computer, a battery and a radio right? Too many times people see something, think "that doesn't look too hard I bet I could do that in a weekend"[1] but don't actually follow through on that thought and spend a weekend building what ever it was that was so easy. Sometimes the more experience you have the more dangerous this is. I can recall times in my own life we're I've grossly underestimated the difficulty of stuff, its lead me to be a lot more conscious about things I know about and things I think I know something about, especially when predicting schedules and work effort. [1] A very common theme on acquihire stories here.
- forgetsusername 11y ago
  >It is important to realize that ignorance isn't "bad" it can be fixed by learning In the parlance of Mr. Rumsfeld, you can learn from the known unknowns, but often you don't even discover the unknown unknowns (the "meta-ignorance") until it's too late.

malkia 11y ago

For NHL2k2/NHL2k3 we had dozen of people just doing the UI - Granted good UI is crucial in a sports game. That also was back in 2001

bluedino 11y ago
For something like that, do those people work full time on just that one game? Or are they also working on NFL/NBA 2k2 at the same time, creating UI code that works across all the games in the series?
- malkia 11y ago
  We were not a typical sports game studio (Treyarch), and the other sport project that was done at the same time was for EA (some baseball game), while the one I was and talking about was for SEGA. It was a bit of a problem really, so people were not allowed to get mixed into teams. After that game Treyarch moved to do mainly Spiderman, and then Call of Duty. The latest COD also has quite a spectacular UI system, but much less coders are involved into bringing it up. But today's standards are even much higher - cool animations, links, socially enabled (twitch, facebook, etc.). Some studio have made success with flash-compatible renderers, and we even thought for NHL2K2 to use one, but first it was only compatible with the Windows version of the Dreamcast (okay, I barely remember now, as we didn't used it, but there was a choice whether you can ship your game with some barebones NT system on the Dreamcast - but you had to lose 2mb to the oS). So that middleware (I think) required it, and it wanted somewhere from 2-4mb more on top. One big problem, back in the days, with the UI is that you can have tons of memory in your main menu, but you barely have anything while the game is playing. So your "PAUSE" menu might use completely different code path, and look quite differently. The alternative is to reload things after PAUSE, but this decreases the quality of the product, also much harder to test, and you would hear the DISC "screaming" soo much :) Nonetheless Dreamcast was fun to work on, spare the compiler, and the debugger :)

gambiting 11y ago

Completely agreed. I work in a large games studio and on our project the team that does nothing else but makes sure the game is compliant(so that it is accepted by Sony and Microsoft for release) consists of 7 people - 5 programmers, a designer and a QC tester. I imagine doing the work we are doing for an indie game would be extremely time consuming.

TomGullen 11y ago

What does "AAA Game" mean anyway? I'm a bit lost on the definition. Wikipedia says: > In the video game industry, AAA (pronounced "triple A") is a classification term used for games with the highest development budgets and levels of promotion. A title considered to be AAA is therefore expected to be a high quality game and to be among the year's bestsellers. By definition, a budget of $75k + 6-10 people wont make an AAA game. > When we made Crash Bandicoot (with a team of 7), it was already virtually impossible to make a AAA game with 6-10 people, and that was 20 years ago. I'm biased/invested in this area, but I think nowadays tools are really helping strip away the technical challenges of making a game. Increasingly game development is less about technical challenges but more about the creative challenge. Individuals are willing to pour thousands of hours of work into artistic and creative projects, and I am seeing far less resistance and challenges going forwards for these sorts of people to produce AAA quality games, when previously game creation was extremely inaccessible to them. Also, thank you for Crash Bandicoot. Amazing game :) One I loved playing growing up!

vimota 11y ago
It seems like they meant AAA-quality game, with a budget of $75k + 6-10 people
- robotkilla 11y ago
  I think you're correct... saying that you created a AAA title means that you can compete with major game studios... I think. However, in my opinion AAA doesn't apply to scope or budget but quality of experience. Super Meat Boy is one of the best games I've played in years and it doesn't come anywhere near the scope of say GTA 5 (which is also awesome). I don't think anyone calls SMB a triple-a title though... so I'd say wikipedia's definition is correct. So then, I think the real issue here is aiming to make a game that compete with major studios who have budgets of $50m and employ hundreds of specialized team members. These devs claim it was a passion project, but then they throw in the towel on the entire concept of running an indie studio after the negative reviews come in. I understand the bankruptcy move, but if I were the founder's shoes, I would move somewhere cheap, take a side job and keep making games on the side. If I still loved the Wolfie project, I would fix it for free and I would find a way to send my KS backers whatever I owe them. I would read every negative review of my game, try to bucket the feed back (something like: legit, semi-legit, ignore) and then fix the game. I would also print my favorite positive reviews and hang them up around the house for motivation. If I no longer believed in the game, I would build something new but smaller. 13 years is a long time to wash down the drain due to one failure. Currently I'm trying to make indie games along with my gilfriend. She has no formal 3d art training but has been teaching herself via the internet. I learned Unity and have been teaching myself c# / cg and all the other shit that goes with 3d game development (I come from a python / js background). We will most likely fail hard but I don't understand the concept of quitting when you got into something for fun in the first place. edit: it also sounds like these guys tried to build their own engine... or something. If they did, that was a bad idea. If they didn't, I don't understand why they had so many issues with collision detection. Haven't played the game though... its just odd to hear that they had major collision detection issues.

catshirt 11y ago

hey i'm sure people following you around telling you this probably gets old but i started playing Crash when it came out, i was 8. i'm 27 now and working in gaming. i just wanted to say Crash was really a huge part of my life and i would be doing us both a disservice not to take this opportunity to say thanks for such a great work of art!!!

danabramov 11y ago

Off topic, but thank you for CB, it was my favorite game for years. (Fun anecdote: I'm from Russia and I didn't know English when I was a child. I had no idea what "save game" means so I started Crash Bandicoot 3 from scratch every time. Imagine my surprise when I realized how saving works.)

w0utert 11y ago

>> .. or, as Mark Cerny (our producer on Crash) used to tell us, "add one and increase the unit" Reading this made my day, it's the exact same rule I try to apply to my own effort estimate, to the letter. I was 100% convinced I came up with it myself, but now I'm starting to think I may have (unconsciously) picked it up from someone else at some point ;-)

EdiX 11y ago
It's from Murphy's Law.

atmosx 11y ago

You made 'Crash B' ? Boy you're a myth! It was one of the few games I wasn't bored to death while playing :-) congrats!

chazu 11y ago

Was that title developed using lisp? If so (or if you worked on other Naughty Dog games which did use lisp) do you feel that it was a force multiplier in any way?

archimedespi 11y ago
Yes! GOOL/GOAL (see [1]) was a lisp-family language. See [2] for an explanation by the guy who developed it. [1] - https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp [2] - http://all-things-andy-gavin.com/2011/03/12/making-crash-bandicoot-gool-part-9/ http://all-things-andy-gavin.com/2011/03/12/making-crash-ban...

lewisjoe 11y ago

You are the guys who made Crash Bandicoot? Thanks for making my childhood awesome.

kraig911 11y ago

Ok that's great that means you can make another one!

arethuza 11y ago

"estimate and multiply by 10" Reminds me of these rules: - Multiply by π for small, known teams, doing a new job, - by π^2 for unknown teams, - and √π for known teams who have already started, know their work and have a good track record. From http://alistair.cockburn.us/The+magic+of+pi+for+project+managers http://alistair.cockburn.us/The+magic+of+pi+for+project+mana...

VonGuard 11y ago

Steve Mcconnell has a great book on software estimation, but it's about enterprise software, where there are requirements and goals. [ http://www.stevemcconnell.com/est.htm http://www.stevemcconnell.com/est.htm ] The biggest problem with making games is the fact that there's no end goal of "Done." It's just a vague point in time, really. Is it done when you've made 10 levels or 11? Is it done when you've added 10 power-ups or 11, or 15? In a movie, you can't make the thing 50 hours long, no one would watch it all. But for a game, 50 hours is now considered medium sized, compared to MMO's and MOBAs that offer hundreds of hours of time to play. Scope creep is the biggest danger in the games industry. Reading this update, where they talk about going from 2D to 3D, my heart broke. That's such a major change, and as he writes, such a major uptick in difficulty, it was the beginning of the end. Terry Cavanaugh [http://distractionware.com/blog/ http://distractionware.com/blog/ ] gets scope right. Valve gets scope right. You can tell when it's done right because the pacing of the game feels damn near perfect, like Half-Life 2: there's little down time, and you're always moving forward, even if yer not always shooting. I feel bad for the Woolfe team, but they clearly had rose colored glasses on. There's a very good reason modern console games cost at least 8 figures. Hell, even with 8 figures of budget, many big name titles aren't even that good. The games industry is a harsh mistress. Like judging what's funny for a comedian, judging what's fun for a game developer is the toughest task they have.

eecks 11y ago

I was just playing Crash Team Racing a few days ago and it gave me a lot of good memories of my childhood. Thanks!

spike021 11y ago
I love that game! For my brother and I it was the alternative to Mario Kart since we didn't have a Nintendo console at the time. So much fun though.
- eecks 11y ago
  I'd buy an updated version (online) in a heartbeat
  - spike021 11y ago
    Same.

dopamean 11y ago

I wasn't really into video games as a kid but my father bought me a Playstation (the first one) used from some place and it came with a copy of Crash Bandicoot. It is the only video game I can say I've played. It was my favorite thing growing up. Such good times. Thank you!

nadams 11y ago

> I wanted to believe that a team of 6 to 10 people could make a game that looked and felt AAA. What is your definition of AAA game? I believe Minecraft was created by 1 person (or at least started). I consider Terraria to be a AAA game and that was made by 3-4 people.

oaktowner 11y ago
In the parlance of the industry, AAA is indicative only of budget: "[the term AAA is] used for games with the highest development budgets and levels of promotion."[1] [1] https://en.wikipedia.org/wiki/AAA_(video_game_industry) https://en.wikipedia.org/wiki/AAA_(video_game_industry)

pitchups 11y ago

There is a nice long list of over 100 answers to the question of why software estimates are off at Quora: http://www.quora.com/Engineering-Management/Why-are-software-development-task-estimations-regularly-off-by-a-factor-of-2-3?share=1 http://www.quora.com/Engineering-Management/Why-are-software...

balls187 11y ago

Tangentially related, this is a fascinating read: http://www.quora.com/How-did-game-developers-pack-entire-games-into-so-little-memory-twenty-five-years-ago/answer/Dave-Baggett http://www.quora.com/How-did-game-developers-pack-entire-gam...

flipmonk 11y ago
Wow. "Ultimately Crash fit into the PS1's memory with 4 bytes to spare. Yes, 4 bytes out of 2097152. Good times." - the things we take for granted today!

blakeyrat 11y ago

From reading the reviews of the game on Steam, the main complaints are that it's buggy (especially with regards to enemy AI), and it's incredibly short (not even 2 hours' worth of content.) There are a lot of $10 games that will entertain for 4-5 or more hours (even excluding the outliers like Terraria). I'm surprised that the developer was surprised at the perceived value of their product. That said, he's right that the market for indie games is ridiculously saturated right now.

robryan 11y ago
Yes, unless you are have a really novel concept that will make people see past early bugs (such as ARK: Survival Evolved) you are competing with some games that set the bar very high. For example something like Ori and the Blind Forest, not exactly indie but a $20 title with amazing graphics and sound and around 6-8 hours of bug free gameplay.

SilasX 11y ago

Stupid, deceptively-simple question: Why can't you do something like "Game on Rails", where you set up a basic framework that has all the stuff a game normally needs, and then you only have to layer your assets and logic on top of it? Do game devs already use something like that and still have these delays?

shortstuffsushi 11y ago
There are lots of frameworks for game development, but the problem is exactly in what you've described: rails. You can only get so far within the constraints of those frameworks before you need something it just can't do, and have to develop yourself. Depending on the framework, this may not even be possible (which is what leads to creation of new frameworks). With that said, a lot of great (albeit typically somewhat simple) games are made with frameworks. When I when to a few game jams back in college, we typically started with something so that we could have a "complete" game by the end.
- pimlottc 11y ago
  Also, using an existing engine often means giving up the possibility of having unique features or capabilities. Having the only game with multidimensional quad-resolution bumpmap tensors (in realtime!) can be a major selling point. Of course, in most cases, you'll get a better overall game by focusing on the art, story, gameplay, etc., but that doesn't always translate to sales.
Impossible 11y ago
Yes http://www.unity3d.com/ http://www.unity3d.com/ is essentially "Game on Rails". But, just as you can quickly and easily build a failed startup with Rails, Unity doesn't really save you from building a game that no one really wants.
chipsy 11y ago
It's a fidelity problem. If you restrict yourself to very simple interactions and behaviors, you can make a Twine game and get the effect you're looking for without touching code. But then your scope gets more ambitious and you want characters to be fully realized, a mobile camera, voice acting, combat simulation, and so on... and then the asset pipeline blows up into something way bigger, every interaction requires additional scripting steps, you start needing to customize the rendering code... No one of those features stops you, but you get a "death by paper cuts" effect, because you eventually hit a mine that requires original technology to be written - most commonly, something to do with collision code. A good team led by a competent producer is able to figure out how to get the right effect within the budget, but it's never an easy process, and game productions have a habit of "pitching the moon and shipping with swiss cheese".

joonap 11y ago

Thanks for Crash Bandicoot! Still one of my favourite games!

ransom1538 11y ago

I was in gaming for 4 years or so. It was horrifying to see how much code that is not reused (excluding math/physics libraries). IMHO - The art pipline is the real time suck and can make or break a game deadline. Just think of the staff alone to pull off a art pipeline. Producer -> Game Play expert -> Artist -> tools engineer -> graphics/engine guys -> game designers/ai guys. Just that chain alone from a tech perspective: LAU, databases, openGL, Maya3d, C++ tools, art vector tools, phyics engines, ai engines which are mostly proprietary. Budgets were like movies. Millions and millions. Oh yeah. And most are canceled after years of hard work. I think if you are making a 3d game you haven't thought things through.

fapjacks 11y ago

Crash Bandicoot is an awesome game! :) Also there is a parallel between your "add one and increase the unit" and the estimation of what kind of manpower you need to take an defended objective. The wisdom here is basically the same: If you are facing an objective defended by a squad, you will need a platoon to take it. If you are looking at a platoon dug in to the objective, you will need a company to take it. An entrenched company needs to be uprooted by a battalion, et cetera. It's from this (The Mythical Man Month as you mention, and the estimations the infantry taught me) that I have built a kind of mental model for estimating things, that ends up being sort of universal like the 80-20 rule.

Bartweiss 11y ago

This, I think, is fundamentally the equation driving modern game design and the "indie revolution". For AAA games, modern, high-end systems increase the difficulty of providing a product. Modern development tools help, but graphics demands and expectations are inexorably growing. In a world where animating reflections might be a full-time task, 6-10 people can't even count on developing a trailer for a AAA product. In response, then, indie games. Why the rebirth of pixel art and simple, geometric visuals? Not because it's 'retro' or 'clean', but because it's easy. Procedural generation, low-poly graphics, and user generated content aren't popular because they produce the best products, they're popular because they're all decisions that slash the number of employees you need. Woolfe's mistake was setting out to make their game as good as it could possibly be. For an indie team today that's simply not possible. To paraphrase your producer, "3 years = Duke Nukem Forever".

cylinder 11y ago

Thanks for creating a fun memory of my childhood.

ild 11y ago

Yet STALKER was not vaporware after all.

rurban 11y ago

I lived with this Mark Cerny rule for a few decades, and it was mostly true. But finally after I'm over 50 I came to proper and good estimates now. Mostly because I'm a much more experienced now. 2d == 1.5d, 1w == 5d and so on. But game programming is like architecture. There's so much more to do.

themodelplumber 11y ago
Can you explain a bit more? I'm intrigued. What does 2d == 1.5d mean? Was that how close you were nailing your estimates? If so, how did you get to that point?

automathematics 11y ago

Yeah, we made Fallout New Vegas was a team of ~50. Now I feel bad. I didn't know CB had such a small team! Props!

gobengo 11y ago

Reading this right now and I think it's 6, not 9. But yes.

hvidgaard 11y ago

After reading the Mythical Man Month, I derived an estimation scheme that was something like this: 1. I know this, I have done it before in this codebase. Estimate is accurate, multiply by 2 to account for watercooler talk, documentation, and other non coding work. 2. I know this codebase, I have a pretty good idea of what to do, how, and where. I have walked trough the code and can't identify any major roadblocks. Multiply by 5 3. I don't have an honest idea. Multiply by 20, and make it clear that it's a guesstimate at best. Usually a better estimate can be given after working on it some time. It's still off more than I like.

erikb 11y ago

I'm so surprised that there are people who actually have a sense of things. I try to tell others all the time that in software having an inexperienced developer you need to multiply with 3x for best hopes and with 10x for common case, sometimes even experienced devs ending up between 2x and 5x. People often don't believe me, some reason being that some developers really deliver on time and sell it as finished but then having the rest of the team fixing bugs and working around strange APIs for months. Often it's just that they also haven't experienced the pain of developing something of reasonable scale and stability. No man, these are not tears, I just got something in the eye.