9 ms·

John Carmack: Script Interpreters Considered Harmful

131 points by LiveTheDream 15y ago

ja2ke 15y ago
Gaming "toy user interfaces" like the one pictured lower in the article generally take so much code because, very often, art directed animation which drives those UIs is often actually assembled entirely in script. Comparing it to an early Unix kernel I guess paints a nice visual image, but is probably not a fair comparison. In game, in-world "toy UI" like that is intended to look like the sort of sci-fi UI you see in movies, but also be interactive. The stuff in movies is made by a motion graphics designer with a copy of Adobe After Effects. Replicating that movie like look, with all the animations and visual effects you see on the big screen, but ALSO making it work as a fully user-aware, state-aware computer interface, isn't cheap, either on an artist man-hours standpoint, or from a lines of code standpoint. I imagine its expense could be reduced if a lot of the stuff up on screen was boiled down to some kind of binary data - a lot of work is put into optimizing performance of engine-native animation files and effects which are played on the characters and environments of a video game world - but usually UI (including fictional in world "toy UI") isn't considered part of the main tool chain or process, so it ends up either being highly special cased by hand, or very highly interpreted (sometimes running interpreted code through middleware which reinterprets the code again) requiring quite a lot of bloat to look and behave in a way that is presentable and on par with the rest of the game. (Usually at this point in the games industry, people build their UIs in flash, and then run them through middleware like Scaleform, which reinterprets their flash based bitmap and vector artwork as polygonal art, and reinterprets their action script into an engine-savvy language... not efficient for the game, but efficient for a former marketing guy who now wants to do game graphic and UI design, I guess.) (For what its worth, I agree with Carmack's assessment. Making highly authorable and also highly efficient at runtime, "AAA visual quality" game UI is just a hard problem to solve, and it's a problem which few studios deem worth solving because after so many years of the current system, the benefits of a change have almost become an intangible.)
- pagekalisedown 15y ago
  UI is a solved-problem from my point of view with middleware like Scaleform.
  - rdw 15y ago
    Carmack is specifically talking about Scaleform, though! And he's not alone -- I know a few game developers who hate that thing with a passion because it's so slow that it's introduced noticeable performance problems in AAA games. Because he's clearly talking about ScaleForm/Flash specifically, though, I don't think that his comments can be correctly generalized to all "script interpreters". It's kind of the equivalent of buying a Maylong Android Tablet and going around saying that Android sucks or the tablet market is dead. You just happen to have picked the worst possible representative.
    - barrettcolin 15y ago
      He may be referring to Scaleform (or another Flash GUI library; I don't think he specifies), but he's also talking about his experience with scripting in Doom 3- which I'm fairly sure didn't use Flash.
  - malkia 15y ago
    You would also have to find flash-experienced people willing to work the way console game developers work. They actually might feel a little bit under appreciated for the stuff they do, and not feel very rewarding at work, especially when they have to constraint themselves with memory, cpu, gpu, i/o issues that might not be visible on the desktop. Also at some point certain integration between the engine and scaleform is needed, and you would have to lose a programmer there supporting it. We tried it, and the consensus was it's not of use to us. I wasn't the one evaluating it, so I can't tell for sure. But resource budgeting (memory, cpu, video memory (ps3)) is very debated thing in the console game development, and everyone wants more budget for his need (audio, animation, builders, ui, etc.) We also gave a try of an early flash middleware for Dreamcast for NHL2K2 (2000) - there were only 16mb of RAM, and flash was taking significant amount (no scaleform relation). We actually got some pretty good menus, and for a video sports game you need very detailed UI for rosters, trades, etc. We also got people to do it back then. But then the problem was during gameplay. While in the main menu we had the memory to pull it off with flash, during gameplay all memory was for vertex buffers, textures, game data, sound, animations, etc. We were thinking of swapping out game data (and reloading later) when the PAUSE button is hit (and the menu appears), so flash would have enough memory - but it turned out to have some bad latency - few to couple of seconds. Ideally PAUSE ON/OFF should take no time (otherwise it's annoying). Another thing was that we still had to render the rest of the HUD (scores, replays, etc.) not using flash - so we had to support TWO technologies for UI. Hence we turned our backs on it. It was nice, but not for our game.
rwmj 15y ago
The real question is: why do people keep writing interpreters, when writing compilers is not hard. It's not "interpreters vs C++" as Carmack supposedly said, but crappy slow badly implemented user languages vs efficiently implemented user languages vs a guru writing C++. The middle option is the one to go for. Especially with LLVM which makes compiling your code very easy, and it's got a BSD-ish license, so no problem linking it with your awesome proprietary game.
- delinka 15y ago
  With an interpreter you sort of get an automatic sandbox because your badly implemented user language is handled by a buch of switch-case or if-else statements. Not literally, but the 'user language' is entirely limited within the interpreter and doesn't mingle with the real game code unless you code up that specific interface. Enter compiled code. Now we have the opportunity to run unknown binary code right within the game's address space. Bad News in many ways. To get the Happy Medium (compile code within the game engine, load and run the resulting executable safely) requires implementing operating system concepts. It's a Big Job. This leads my thoughts on a slight tangent: why isn't there a Super Awesome Open Source Game Engine that's the heart of all 3D games? SAOSGE could even provide the extensibility framework (compiling, loading), ready for you to expose parts of your games to player programmers.
  - nostrademons 15y ago
    If you control the compiler and the language is memory-safe, you get the same guarantees with JIT-compilation that you do with interpretation. All execution runs within the memory boundaries specifically allocated by the runtime system, and any other interaction with the machine must go through the runtime libraries, where you can perform any checks you want.
    - prodigal_erik 15y ago
      I don't see this kind of sandboxing as likely to help. Whether your scripting language is compiled or interpreted, if any accessible portion of your system is written in a memory-unsafe language like C, that is where crackers will find the most damaging exploits. E.g., http://stackoverflow.com/questions/381171/help-me-understand-this-javascript-exploit http://stackoverflow.com/questions/381171/help-me-understand... which attacks a native XML toolset (some "data binding" IE feature I don't understand) without breaking any javascript rules.
robterrell 15y ago
Despite the author's characterization, Carmack's quote doesn't say that script interpreters are harmful or evil, but "bad" in the sense that they're not performant enough for Rage. Lua was certainly fine for, say, Angry Birds, and has clearly been a huge win for Rovio in rapidly porting the game.
- petsos 15y ago
  That's interesting. Do you have any links on how Angry Birds uses Lua?
  - malkia 15y ago
    You can actually see the lua usage by peeking into the .ipa iPhone packages (they are just .zip files). Maybe android and other systems are not much different. I've seen at least the lua levels described there.
  - StavrosK 15y ago
    As an aside, are you from Edessa?
  - deleted 15y ago
    [deleted]
  - DanielRibeiro 15y ago
    You can see it on the indie level editor: http://www.badboll.nu/acme/able/ http://www.badboll.nu/acme/able/ Also on the dissassembled apk (it is compiled lua by the way, so you might need this[1] to see it more clearly). [1] http://chunkspy.luaforge.net/ http://chunkspy.luaforge.net/
- 6ren 15y ago
  His bolded quote is script interpreters are bad, from a performance, debugging, development standpoint.
shabble 15y ago
As far back as Quake 1 (in fact, before Quake needed version numbers), there was https://secure.wikimedia.org/wikipedia/en/wiki/QuakeC https://secure.wikimedia.org/wikipedia/en/wiki/QuakeC which was an interpreted language, although later on it got a compiler to turn it into a native dll.
- aw3c2 15y ago
  I am clueless about this stuff but: It was interpreted but it was compiled. QuakeC was not a scripting language as one would say. You had/have to compile it. The engine has a VM to handle it.
  - shabble 15y ago
    yeah, you're right. I grabbed the first line of the WP article and missed the part about bytecode and progs.dat. Quite embarrassing, since quakeC was one of the first languages I played around with, and I don't remember the compile step. Then again, I was working from tutorials so it's possible there was just a 'compile & run' batch file or something.
buster 15y ago
Makes me wondering with all those WoW-Addons which are Lua, it certainly can't be _that_ bad..
- forrestthewoods 15y ago
  Sure it can. When I play WoW years ago there were addons that could take up over a gig of ram. The mods that were item database related could often obliterate performance. I had a beefy PC at the time and there were still a few popular mods I couldn't run. The perceived advantage of something like Lua is that it's easy to write, allows for fast iteration, and designers/artists can use it. The issue is that designers/artists really shouldn't be using it. To write fast code it requires a programmer and if you have a programmer why not just do it in C++ where it will be several times faster? A common game development pattern is to write the bulk of gameplay code in a scripting language and at the 11th hour in a desperate panic start moving as much of it as possible to C++ until you fit in 360/PS3 memory and are fast enough.
  - rikthevik 15y ago
    I think there's something to be said for the ease of patching with a scripting language, though. If you need to update a quest, just ship a few new lua files and it's done - seems much easier than messing with a binary.
  - slowpoke 15y ago
    To write fast code it requires a programmer and if you have a programmer why not just do it in C++ where it will be several times faster? Why should I as a programmer not profit from extremely fast design/implement/test/fix/repeat iterations? I'm not saying scripting/high level languages like Lua, Python or Ruby should always be preferred. But it's equally wrong to assert that C-oid languages are always the superior choice. It's much more a question of knowing when to use what and why, which in my opinion makes an important part of being a good programmer.
    - to3m 15y ago
      I've never seen the effort involved in making a nice quick-feedback REPL-style affair pay off. Perhaps for certain types of game it would be worth it, and perhaps amortized over a long enough period the result would become value for money. My experience has been that making a script system work that way, and work reliably, and provide a nice interface, just takes up more time than you end up saving. Anyway, all the programmers I've known would want good debugging facilities as well, like the ones you get in Visual Studio/ProDG/Code Warrior/Xcode/gdb/etc. - oops, only kidding about gdb! This thing really needs a GUI. This all just expands the time-suck further. That's not to say it can't be made to work, of course! People can do anything, if they set their mind to it hard enough, and if they wait for long enough, maybe it will even become value for money.
dasil003 15y ago
Why should it be surprising that performance still is of primary concern to Carmack? If you're pushing the state of the art forward then I think that will always be the case.
WiseWeasel 15y ago
Google's cached version (site was down for me): http://webcache.googleusercontent.com/search?q=cache:veFy_mxlZeIJ:www.codingthewheel.com/game-dev/john-carmack-script-interpreters-considered-harmful+http://www.codingthewheel.com/game-dev/john-carmack-script-interpreters-considered-harmful&cd=1&hl=en&ct=clnk&gl=us&client=firefox-a&source=www.google.com http://webcache.googleusercontent.com/search?q=cache:veFy_mx...
floppydisk 15y ago
I'm surprised no one picked up on this gem, yet. >". . . but you know one of the big lessons of a big project is you don't want people that aren't really programmers programming, you'll suffer for it!" Edit: Fixed formatting of quote.
- derefr 15y ago
  A lot of other companies' games integrate scripting engines to ease level/mod design, because they don't want to ship a compiler toolchain along with the game. Carmack's games, on the other hand, are just eventually open-sourced, and modded "natively."
richcollins 15y ago
scripting languages weren't really designed for large-scale development efforts involving millions of lines of code. They typically lack the code-reuse abstractions and development toolsets I find that dynamic languages with first class functions provide much better abstractions and opportunities for code-reuse than languages like C and C++.
- enneff 15y ago
  That paragraph jumped out at me, too. I agree with the point, but not its reasoning. The reason why scripting (read: dynamic) languages fall apart at scale is largely because they're too flexible. Abstractions that are convenient in the small can create unmanageable complexity in the large (unless you are extremely disciplined).
jrockway 15y ago
scripting languages weren't really designed for large-scale development efforts involving millions of lines of code Neither was C. There are languages developed afterwards with large-scale development in mind, but you have to ask, what did they add? True type safety? Nope. True encapsulation? Nope. Automatic resource management? Nope. C++ is C with several times more ways to write horrifyingly unmaintainable code. All big C++ shops have a very long lists of constructs that must never be used and very rigorous code reviews to ensure that you don't use any of those features by accident. At this point, we see that C++ has one advantage over C, and that's namespaces. Helpful, but not helpful enough for "large-scale software development". The major innovation that C++ brought was waking people up to the reality that a maintainable codebase must be curated with extensive automatic testing and manual code reviews. Anything else leads to epic failure. But wait, that's easy to fix! Let's invent a new programming language! This time we'll call it Java. It will be like C++ but with all the ways to write bad code removed. No operator overloading! No multiple inheritance. And it's true that Java helped in a number of ways. But it didn't solve the real problems. There is still no type safety; null is an instance of every class in Java, but you can't call any methods on it, for example. So instead of a segfault, you get a NullPointerException, but all that means is that the source of the error is easier to determine. But you still have to write a lot of tests to make sure that your code handles nulls properly. (This is compounded by laziness in design like writing "loggedInUser = null" instead of writing a subclass of User that indicates it's not logged in.) Multiple inheritance in C++ was a mess, but Java's solution isn't much better. What's the conceptual difference between abstract classes and interfaces? Abstract classes are non-composable class-parts that contain API, implementation, and state. Interfaces are composable class-parts that contain API. Why are these two separate concepts? Why not have a generic "traits" feature? (The answer is because Java is mostly a copy of C++, and C++ focuses on irrelevant OO features like 4 different levels of member visibility rather than semantic annotations like "if you compose this method into another class, it should run after the method from the class it's being composed into". Of course, when you have these annotations, multiple-inheritance or multiple-trait-application works perfectly -- see CLOS or Moose. But all C++ had was public/protected/private/friend, and so that's all Java has.) This is getting ramble-y so I'll get to the point. No modern programming language helps you write huge codebases. If you want to be able to maintain millions of lines of code, you are going to need very rigorous standards and millions of lines of automatic tests. It's the only way we know of. All more modern languages did was push us from "we're leaking memory because we forgot to call free()" to "we're leaking memory because it's pretty convenient to keep all these huge objects around". They certainly haven't helped us write more maintainable large projects. If anything, a dynamic scripting language embedded in your game means that you'll have fewer lines of code and clearer separation between components. But it's not such a win that you won't have to review code or write tests anymore.
watmough 15y ago
It's refreshing to see such a great communicator. He spoke, off-the-cuff, for over an hour, at a level that seemed pitched to appeal widely. The best thing, though, was the obvious love and enthusiasm he has for his team and their software.
peteretep 15y ago
> but you know one of the big lessons of a big project is you don't want people that aren't really programmers programming, you'll suffer for it! Take that and shove it up your cross-functional agile team :-D

131 points by LiveTheDream 15y ago

ja2ke 15y ago

Gaming "toy user interfaces" like the one pictured lower in the article generally take so much code because, very often, art directed animation which drives those UIs is often actually assembled entirely in script. Comparing it to an early Unix kernel I guess paints a nice visual image, but is probably not a fair comparison. In game, in-world "toy UI" like that is intended to look like the sort of sci-fi UI you see in movies, but also be interactive. The stuff in movies is made by a motion graphics designer with a copy of Adobe After Effects. Replicating that movie like look, with all the animations and visual effects you see on the big screen, but ALSO making it work as a fully user-aware, state-aware computer interface, isn't cheap, either on an artist man-hours standpoint, or from a lines of code standpoint. I imagine its expense could be reduced if a lot of the stuff up on screen was boiled down to some kind of binary data - a lot of work is put into optimizing performance of engine-native animation files and effects which are played on the characters and environments of a video game world - but usually UI (including fictional in world "toy UI") isn't considered part of the main tool chain or process, so it ends up either being highly special cased by hand, or very highly interpreted (sometimes running interpreted code through middleware which reinterprets the code again) requiring quite a lot of bloat to look and behave in a way that is presentable and on par with the rest of the game. (Usually at this point in the games industry, people build their UIs in flash, and then run them through middleware like Scaleform, which reinterprets their flash based bitmap and vector artwork as polygonal art, and reinterprets their action script into an engine-savvy language... not efficient for the game, but efficient for a former marketing guy who now wants to do game graphic and UI design, I guess.) (For what its worth, I agree with Carmack's assessment. Making highly authorable and also highly efficient at runtime, "AAA visual quality" game UI is just a hard problem to solve, and it's a problem which few studios deem worth solving because after so many years of the current system, the benefits of a change have almost become an intangible.)

pagekalisedown 15y ago
UI is a solved-problem from my point of view with middleware like Scaleform.
- rdw 15y ago
  Carmack is specifically talking about Scaleform, though! And he's not alone -- I know a few game developers who hate that thing with a passion because it's so slow that it's introduced noticeable performance problems in AAA games. Because he's clearly talking about ScaleForm/Flash specifically, though, I don't think that his comments can be correctly generalized to all "script interpreters". It's kind of the equivalent of buying a Maylong Android Tablet and going around saying that Android sucks or the tablet market is dead. You just happen to have picked the worst possible representative.
  - barrettcolin 15y ago
    He may be referring to Scaleform (or another Flash GUI library; I don't think he specifies), but he's also talking about his experience with scripting in Doom 3- which I'm fairly sure didn't use Flash.
- malkia 15y ago
  You would also have to find flash-experienced people willing to work the way console game developers work. They actually might feel a little bit under appreciated for the stuff they do, and not feel very rewarding at work, especially when they have to constraint themselves with memory, cpu, gpu, i/o issues that might not be visible on the desktop. Also at some point certain integration between the engine and scaleform is needed, and you would have to lose a programmer there supporting it. We tried it, and the consensus was it's not of use to us. I wasn't the one evaluating it, so I can't tell for sure. But resource budgeting (memory, cpu, video memory (ps3)) is very debated thing in the console game development, and everyone wants more budget for his need (audio, animation, builders, ui, etc.) We also gave a try of an early flash middleware for Dreamcast for NHL2K2 (2000) - there were only 16mb of RAM, and flash was taking significant amount (no scaleform relation). We actually got some pretty good menus, and for a video sports game you need very detailed UI for rosters, trades, etc. We also got people to do it back then. But then the problem was during gameplay. While in the main menu we had the memory to pull it off with flash, during gameplay all memory was for vertex buffers, textures, game data, sound, animations, etc. We were thinking of swapping out game data (and reloading later) when the PAUSE button is hit (and the menu appears), so flash would have enough memory - but it turned out to have some bad latency - few to couple of seconds. Ideally PAUSE ON/OFF should take no time (otherwise it's annoying). Another thing was that we still had to render the rest of the HUD (scores, replays, etc.) not using flash - so we had to support TWO technologies for UI. Hence we turned our backs on it. It was nice, but not for our game.

rwmj 15y ago

The real question is: why do people keep writing interpreters, when writing compilers is not hard. It's not "interpreters vs C++" as Carmack supposedly said, but crappy slow badly implemented user languages vs efficiently implemented user languages vs a guru writing C++. The middle option is the one to go for. Especially with LLVM which makes compiling your code very easy, and it's got a BSD-ish license, so no problem linking it with your awesome proprietary game.

delinka 15y ago
With an interpreter you sort of get an automatic sandbox because your badly implemented user language is handled by a buch of switch-case or if-else statements. Not literally, but the 'user language' is entirely limited within the interpreter and doesn't mingle with the real game code unless you code up that specific interface. Enter compiled code. Now we have the opportunity to run unknown binary code right within the game's address space. Bad News in many ways. To get the Happy Medium (compile code within the game engine, load and run the resulting executable safely) requires implementing operating system concepts. It's a Big Job. This leads my thoughts on a slight tangent: why isn't there a Super Awesome Open Source Game Engine that's the heart of all 3D games? SAOSGE could even provide the extensibility framework (compiling, loading), ready for you to expose parts of your games to player programmers.
- nostrademons 15y ago
  If you control the compiler and the language is memory-safe, you get the same guarantees with JIT-compilation that you do with interpretation. All execution runs within the memory boundaries specifically allocated by the runtime system, and any other interaction with the machine must go through the runtime libraries, where you can perform any checks you want.
  - prodigal_erik 15y ago
    I don't see this kind of sandboxing as likely to help. Whether your scripting language is compiled or interpreted, if any accessible portion of your system is written in a memory-unsafe language like C, that is where crackers will find the most damaging exploits. E.g., http://stackoverflow.com/questions/381171/help-me-understand-this-javascript-exploit http://stackoverflow.com/questions/381171/help-me-understand... which attacks a native XML toolset (some "data binding" IE feature I don't understand) without breaking any javascript rules.

robterrell 15y ago

Despite the author's characterization, Carmack's quote doesn't say that script interpreters are harmful or evil, but "bad" in the sense that they're not performant enough for Rage. Lua was certainly fine for, say, Angry Birds, and has clearly been a huge win for Rovio in rapidly porting the game.

petsos 15y ago
That's interesting. Do you have any links on how Angry Birds uses Lua?
- malkia 15y ago
  You can actually see the lua usage by peeking into the .ipa iPhone packages (they are just .zip files). Maybe android and other systems are not much different. I've seen at least the lua levels described there.
- StavrosK 15y ago
  As an aside, are you from Edessa?
- deleted 15y ago
  [deleted]
- DanielRibeiro 15y ago
  You can see it on the indie level editor: http://www.badboll.nu/acme/able/ http://www.badboll.nu/acme/able/ Also on the dissassembled apk (it is compiled lua by the way, so you might need this[1] to see it more clearly). [1] http://chunkspy.luaforge.net/ http://chunkspy.luaforge.net/
6ren 15y ago
His bolded quote is script interpreters are bad, from a performance, debugging, development standpoint.

shabble 15y ago

As far back as Quake 1 (in fact, before Quake needed version numbers), there was https://secure.wikimedia.org/wikipedia/en/wiki/QuakeC https://secure.wikimedia.org/wikipedia/en/wiki/QuakeC which was an interpreted language, although later on it got a compiler to turn it into a native dll.

aw3c2 15y ago
I am clueless about this stuff but: It was interpreted but it was compiled. QuakeC was not a scripting language as one would say. You had/have to compile it. The engine has a VM to handle it.
- shabble 15y ago
  yeah, you're right. I grabbed the first line of the WP article and missed the part about bytecode and progs.dat. Quite embarrassing, since quakeC was one of the first languages I played around with, and I don't remember the compile step. Then again, I was working from tutorials so it's possible there was just a 'compile & run' batch file or something.

buster 15y ago

Makes me wondering with all those WoW-Addons which are Lua, it certainly can't be _that_ bad..

forrestthewoods 15y ago
Sure it can. When I play WoW years ago there were addons that could take up over a gig of ram. The mods that were item database related could often obliterate performance. I had a beefy PC at the time and there were still a few popular mods I couldn't run. The perceived advantage of something like Lua is that it's easy to write, allows for fast iteration, and designers/artists can use it. The issue is that designers/artists really shouldn't be using it. To write fast code it requires a programmer and if you have a programmer why not just do it in C++ where it will be several times faster? A common game development pattern is to write the bulk of gameplay code in a scripting language and at the 11th hour in a desperate panic start moving as much of it as possible to C++ until you fit in 360/PS3 memory and are fast enough.
- rikthevik 15y ago
  I think there's something to be said for the ease of patching with a scripting language, though. If you need to update a quest, just ship a few new lua files and it's done - seems much easier than messing with a binary.
- slowpoke 15y ago
  To write fast code it requires a programmer and if you have a programmer why not just do it in C++ where it will be several times faster? Why should I as a programmer not profit from extremely fast design/implement/test/fix/repeat iterations? I'm not saying scripting/high level languages like Lua, Python or Ruby should always be preferred. But it's equally wrong to assert that C-oid languages are always the superior choice. It's much more a question of knowing when to use what and why, which in my opinion makes an important part of being a good programmer.
  - to3m 15y ago
    I've never seen the effort involved in making a nice quick-feedback REPL-style affair pay off. Perhaps for certain types of game it would be worth it, and perhaps amortized over a long enough period the result would become value for money. My experience has been that making a script system work that way, and work reliably, and provide a nice interface, just takes up more time than you end up saving. Anyway, all the programmers I've known would want good debugging facilities as well, like the ones you get in Visual Studio/ProDG/Code Warrior/Xcode/gdb/etc. - oops, only kidding about gdb! This thing really needs a GUI. This all just expands the time-suck further. That's not to say it can't be made to work, of course! People can do anything, if they set their mind to it hard enough, and if they wait for long enough, maybe it will even become value for money.

dasil003 15y ago

Why should it be surprising that performance still is of primary concern to Carmack? If you're pushing the state of the art forward then I think that will always be the case.

WiseWeasel 15y ago

Google's cached version (site was down for me): http://webcache.googleusercontent.com/search?q=cache:veFy_mxlZeIJ:www.codingthewheel.com/game-dev/john-carmack-script-interpreters-considered-harmful+http://www.codingthewheel.com/game-dev/john-carmack-script-interpreters-considered-harmful&cd=1&hl=en&ct=clnk&gl=us&client=firefox-a&source=www.google.com http://webcache.googleusercontent.com/search?q=cache:veFy_mx...

floppydisk 15y ago

I'm surprised no one picked up on this gem, yet. >". . . but you know one of the big lessons of a big project is you don't want people that aren't really programmers programming, you'll suffer for it!" Edit: Fixed formatting of quote.

derefr 15y ago
A lot of other companies' games integrate scripting engines to ease level/mod design, because they don't want to ship a compiler toolchain along with the game. Carmack's games, on the other hand, are just eventually open-sourced, and modded "natively."

richcollins 15y ago

scripting languages weren't really designed for large-scale development efforts involving millions of lines of code. They typically lack the code-reuse abstractions and development toolsets I find that dynamic languages with first class functions provide much better abstractions and opportunities for code-reuse than languages like C and C++.

enneff 15y ago
That paragraph jumped out at me, too. I agree with the point, but not its reasoning. The reason why scripting (read: dynamic) languages fall apart at scale is largely because they're too flexible. Abstractions that are convenient in the small can create unmanageable complexity in the large (unless you are extremely disciplined).

jrockway 15y ago

scripting languages weren't really designed for large-scale development efforts involving millions of lines of code Neither was C. There are languages developed afterwards with large-scale development in mind, but you have to ask, what did they add? True type safety? Nope. True encapsulation? Nope. Automatic resource management? Nope. C++ is C with several times more ways to write horrifyingly unmaintainable code. All big C++ shops have a very long lists of constructs that must never be used and very rigorous code reviews to ensure that you don't use any of those features by accident. At this point, we see that C++ has one advantage over C, and that's namespaces. Helpful, but not helpful enough for "large-scale software development". The major innovation that C++ brought was waking people up to the reality that a maintainable codebase must be curated with extensive automatic testing and manual code reviews. Anything else leads to epic failure. But wait, that's easy to fix! Let's invent a new programming language! This time we'll call it Java. It will be like C++ but with all the ways to write bad code removed. No operator overloading! No multiple inheritance. And it's true that Java helped in a number of ways. But it didn't solve the real problems. There is still no type safety; null is an instance of every class in Java, but you can't call any methods on it, for example. So instead of a segfault, you get a NullPointerException, but all that means is that the source of the error is easier to determine. But you still have to write a lot of tests to make sure that your code handles nulls properly. (This is compounded by laziness in design like writing "loggedInUser = null" instead of writing a subclass of User that indicates it's not logged in.) Multiple inheritance in C++ was a mess, but Java's solution isn't much better. What's the conceptual difference between abstract classes and interfaces? Abstract classes are non-composable class-parts that contain API, implementation, and state. Interfaces are composable class-parts that contain API. Why are these two separate concepts? Why not have a generic "traits" feature? (The answer is because Java is mostly a copy of C++, and C++ focuses on irrelevant OO features like 4 different levels of member visibility rather than semantic annotations like "if you compose this method into another class, it should run after the method from the class it's being composed into". Of course, when you have these annotations, multiple-inheritance or multiple-trait-application works perfectly -- see CLOS or Moose. But all C++ had was public/protected/private/friend, and so that's all Java has.) This is getting ramble-y so I'll get to the point. No modern programming language helps you write huge codebases. If you want to be able to maintain millions of lines of code, you are going to need very rigorous standards and millions of lines of automatic tests. It's the only way we know of. All more modern languages did was push us from "we're leaking memory because we forgot to call free()" to "we're leaking memory because it's pretty convenient to keep all these huge objects around". They certainly haven't helped us write more maintainable large projects. If anything, a dynamic scripting language embedded in your game means that you'll have fewer lines of code and clearer separation between components. But it's not such a win that you won't have to review code or write tests anymore.

watmough 15y ago

It's refreshing to see such a great communicator. He spoke, off-the-cuff, for over an hour, at a level that seemed pitched to appeal widely. The best thing, though, was the obvious love and enthusiasm he has for his team and their software.

peteretep 15y ago

> but you know one of the big lessons of a big project is you don't want people that aren't really programmers programming, you'll suffer for it! Take that and shove it up your cross-functional agile team :-D