9 ms·
Doom 3 source released
- gcv 15y agoAny advice on how to dive into reading this codebase?
- deleted 15y ago[deleted]
- JonnieCache 15y agoIt's more a legal thing which you will see with any commercially released codebase. The iD hackers have to jump through innumerable hoops with their publishers' legal department to get this code out to us at all, so it kinda sounds ungrateful to moan about the licenses. Also, different parts of the code are covered by many different licenses and the copyright on them belongs to many different people, so it is important to be very explicit about everything. When it's an OSS project owned and operated by a small group of people, there's little motivation to worry about this stuff. Also when you choose a license like the MIT one which basically says "I really don't give a fuck what you do unless you sue me," you have a lot less incentive to plaster it over everything, because you're one step away from being in the public domain anyway. When you're releasing the doom3 code, things are very different.
- frou_dh 15y agoEDIT: Deleted my GP post to stem conflict. What I meant (probably poorly phrased) is that for readability for personal study (like gcv was asking about), I think it's worth stripping huge license preambles from source files. There's obviously good legal reason for them being present to start with.
- jerf 15y agoYou're a quick couple of shell commands away from stripping them off yourself, if it's that big a deal.
- cobrausn 15y agoI've heard the best way to tackle the Quake codebase was to pick a 'component', such as the renderer, and start to understand how that works. Work your way out, one component at a time. Eventually the bigger picture comes together. The same would probably work for this, though unless you're well versed in game and graphics programming nomenclature some of it will go over your head, which is expected - when you see something like that, think of it as an opportunity to learn a new concept and look it up.
- corysama 15y agoMy favorite technique is to run the code, place breakpoints in interesting locations and manually type out psuedocode descriptions of each function up the callstacks. That's the fastest way I know to get an understanding of the structure of a new code base. Barring running the code... I prefer to start with a quick scan through the foundation code. What does the math library look like? How about the file and network I/O? Next, locate the top-level main loop, then the top-level Update Everything/Render Everything functions. At that point, you'll understand the general theme of the code and you'll have spotted lots of interesting avenues to explore.
- JabavuAdams 15y agodoxygen EXTRACT_ALL
- mqqq 15y agoDon't waste your time. It doen't have anything ground breaking.
- ortatherox 15y agogit clone https://github.com/TTimo/doom3.gpl.git https://github.com/TTimo/doom3.gpl.git and just browse around locally in your favourite editor.
- metashurick 15y agoI wonder why the depth fail shadow rendering algorithm had been explicitly excluded from the open source release. It's been around for years now, after all.
- TeHCrAzY 15y agoThere was a patent issue; this delayed the release slightly whilst it was removed.
- daeken 15y agoI give it a week before someone forks and adds it back in, just to spite Creative.
- enqk 15y agoPossibly because it's patented. There was a dispute with Creative Labs about the method.
- davyjones 15y agoWould you happen to know if this is Carmack's reverse or something totally unrelated? Edit: OK. I just googled and it indeed is Carmack's reverse.
- Arjuna 15y agoIt is a patent issue [1]. Check out draw_common.cpp [2] - the patent work-around is contained in the RB_T_Shadow method: // patent-free work around if ( !external ) { // "preload" the stencil buffer with the number of volumes // that get clipped by the near or far clip plane qglStencilOp( GL_KEEP, tr.stencilDecr, tr.stencilDecr ); GL_Cull( CT_FRONT_SIDED ); RB_DrawShadowElementsWithCounters( tri, numIndexes ); qglStencilOp( GL_KEEP, tr.stencilIncr, tr.stencilIncr ); GL_Cull( CT_BACK_SIDED ); RB_DrawShadowElementsWithCounters( tri, numIndexes ); } [1] See my previous comment for more info: http://news.ycombinator.com/item?id=2945889 http://news.ycombinator.com/item?id=2945889 [2] https://github.com/TTimo/doom3.gpl/blob/master/neo/renderer/draw_common.cpp https://github.com/TTimo/doom3.gpl/blob/master/neo/renderer/...
- metachris 15y agoThe source for the physics engine (https://github.com/TTimo/doom3.gpl/tree/master/neo/game/physics https://github.com/TTimo/doom3.gpl/tree/master/neo/game/phys...) looks rather interesting. Might contain valuable additions for other physics engines to pick up.
- deleted 15y ago[deleted]
- Impossible 15y agoIIRC the source for most of the physics engine has been available for years with the SDK. I think it only including rigid body dynamics and the constraint solver though, no collision code, so it's nice to have that part of it.
- barrettcolin 15y agoI'm not sure where I picked up the info, and I haven't looked at the code to confirm it, but I believe the physics engine implementation is tied tightly to both the collision detection system and the entity update, both of which are quite specific to Doom (in particular, each entity steps physics independently, rather than a whole simulation advancing at once as is common in 'pluggable' physics engines like ODE, Havok etc). Also, and this is subjective I suppose, I don't remember Doom 3 doing anything that 'felt' or looked better than even what the contemporaneous engines did (such as the version of Havok used in Half-Life 2). So I guess, there probably aren't valuable additions, but I imagine there is much to tinker with.
- barrettcolin 15y agoFor the sake of completeness, here is where I read that: http://www.ode.org/old_list_archives/2004-December/014735.html http://www.ode.org/old_list_archives/2004-December/014735.ht... From looking at the game code, I see odd things like rolling barrels animating their rotation as though the physics engine itself doesn't handle rolling contact.
- afhof 15y agoI was under the impression that most large projects end up having to write a custom memory allocator, but after a few random probes into the source I didn't see anything that stood out. Did I just miss it, or did they use the built in one?
- ars 15y agoWhy do you have that impression? That makes very little sense to do - every single project? Don't you think someone would have released a library to implement the "new and improved allocator" if everyone needs to use it?
- vilya 15y agoThat has been borne out by my experience so far too. There have been several successful efforts at writing better performing general-purpose allocators: * TCMalloc (http://goog-perftools.sourceforge.net/doc/tcmalloc.html) * jemalloc (http://www.canonware.com/jemalloc/) There's a great Facebook Engineering blog post about jemalloc which describes why a fast allocator is needed & why the standard malloc is not always sufficient: https://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919 https://www.facebook.com/notes/facebook-engineering/scalable...
- forrestthewoods 15y agoI've worked with several projects where a TCMalloc based allocator (threaded) was used to great success. A simple switch to DLMalloc (non-threaded) can also provide a significant performance boost. Moderate usage of STL can do nasty, nasty things to memory performance. In one case a game I worked on ran at 5-10fps and a simple, fast change to DLMalloc instantly took perf to 60fps.
- caf 15y agoIt's worth noting that at least on some platforms the system malloc is dlmalloc (presuming that DL is Doug Lea).
- zeratul 15y ago196 clones, 171 watchers, 23 forks; that's just in the first hour from release. Not bad, not bad at all. EDIT: Doom 1 source code seems dirty when compared to Doom 3.
- waterlesscloud 15y agoSkimming over the D3 source, it all looks so very readable. Which is impressive in itself.
- vilya 15y agoA bit off-topic, but I'm curious: why do you think that's impressive? I thought it was pretty commonly accepted these days that readable code saves more time than it costs over the life of a project...?
- waterlesscloud 15y agoBecause it may be commonly accepted, but it's rarely actually done so well.
- wladimir 15y agoGame code specifically is usually the exception to this rule, because it tends to be written to very strict deadlines, ad-hoc requirements, short project lifetime (no need to maintain or support it for years after), and not much expectation of re-use. This does not all hold for Doom3 as it was also an engine sold to other companies for other games.
- InclinedPlane 15y agoA lot of things are generally accepted as "good ideas", but how often are they practiced? Most software management would throw their own mother under a bus if it got the product out the door on anything close to schedule or budget. Which is to say, code cleanliness is very much not the norm anywhere. And if anything the game industry seems to be a good 10-20 years behind the rest of the software industry in terms of hewing to good practices (clean code, automated tests, etc.)
- baby 15y agoDoes that mean we don't have to pay to get the game now? Compile the source and playable?
- JonnieCache 15y agoNope, it's only the engine. If it's anything like the other iD software code releases, you will be able to use the content from the game disc with the engine built from source, or you will be able to extract the content from the demo if you dont have the game.
- Splenivore 15y agoNo. “This source release does not contain any game data, the game data is still covered by the original EULA and must be obeyed as usual.” You should be able to play a total conversion that does not rely on the original game’s assets, however.
- deleted 15y ago
- dgallagher 15y ago$ sudo perl cloc-1.55.pl --unicode doom3.gpl/ 2014 text files. 1907 unique files. 476 files ignored. http://cloc.sourceforge.net v 1.55 T=40.0 s (36.6 files/s, 22123.3 lines/s) ------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- C++ 517 87051 113094 366423 C/C++ Header 615 29640 26891 110991 C 170 11407 15566 53520 Bourne Shell 36 4529 5476 33717 m4 10 1079 232 9025 HTML 55 391 76 4142 Objective C++ 6 709 654 2605 Perl 10 523 411 2380 yacc 1 95 97 912 Python 10 108 182 895 Objective C 1 145 20 768 make 22 160 253 579 DOS Batch 5 0 0 61 Teamcenter def 4 3 0 51 Lisp 1 5 20 25 awk 1 2 1 17 ------------------------------------------------------------------------------- SUM: 1464 135847 162973 586111 -------------------------------------------------------------------------------
- JonnieCache 15y agoQuestion 1: Why did you run cloc as root? Question 2: Why does Doom 3 involve 33,717 lines of bash? That's a lot of bash. I realise it's used heavily for build scripts, but thirty three thousand lines seems excessive.
- wollw 15y agoNo clue about question 1 but 32,607 of those lines are all part of curl which seems to be distributed with the source code.
- JonnieCache 15y agoAny gambling types fancy a little wager? Which will come first, a stable JVM port of doom 3, or a stable build of Textmate 2? The only problem is, I really can't think which side I'd come down on. NOTE: For those primed to accuse me of hating: this is as much a compliment to certain hackers and their fixated attitude towards porting iD code dumps, as it is a snide reference to textmate2's status as vaporware.
- shocks 15y agoOnly took 8 minutes and 22 seconds to compile this, I was quite surprised. Going to have some fun over the next few days that's for sure! Thanks Carmack, heh.
- akacd 15y agoCan you give me some advice how to add the gamedata to the code?I got compile errors for I had no doom3 on my computer(windows 7),I'm downloading it now.
- shocks 15y agoSure. To compile you only need to get the DirectX SDK (from here http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=6812 http://www.microsoft.com/download/en/details.aspx?displaylan...). Then open Visual Studio (I'm using 2010 Ultimate, but Express should be fine). (these are the steps that worked for me) Open doom.sln Go to View -> Property Manager Expand Debug | Win32 Right click "Microsoft.Cpp.Win32.user" and choose Properties Select VC++ Directories You need to add the following directories C:\Program Files (x86)\Microsoft DirectX SDK June 2010\Include C:\Program Files (x86)\Microsoft DirectX SDK June 2010\Lib\x86 Do the same for Release | Win32. Doom3 should now compile. Note that 2 projects might fail and 1 could be skipped. You should find Doom3.exe under doom3.gpl\build\Win32\Release From your Doom3 installation copy the "base" folder, if you have the expansion copy that too. Doom3 should now launch. If you have any more problems, feel free to ask!
- kunqiana 15y agohi I got his error, on ubuntu 11.04. I moved the base folder to the neo folder but it still didn't work. Is it because it's missing the game data? DOOM 1.3.1.1304-debug linux-x86 Nov 24 2011 00:27:04 found interface lo - loopback found interface eth0 - 10.0.0.19/255.255.255.0 ------ Initializing File System ------ Current search path: /home/xx/.doom3/base /home/xx/doom3.gpl/neo/base game DLL: 0x0 in pak: 0x0 Addon pk4s: file system initialized. -------------------------------------- ----- Initializing Decls ----- ------------------------------ ------- Initializing renderSystem -------- idRenderSystem::Shutdown() Sys_Error: _default material not found
- dbbo 15y agoI hope this means that there will be an actual Doom package for Debian in the near future.
- goodweeds 15y agoI can't seem to find references to this anymore, but I seem to remember that the doom3 source code was stolen and released in like '03 or '04. So if that's true then my response to this article is, "Again?".
- wmil 15y agoIt was in '02. Id gave alpha code to ATI so they could tune their drivers. One of their employees leaked it. http://forums.gamershell.com/showthread.php?407-About-Doom-3-Leaked-Beta http://forums.gamershell.com/showthread.php?407-About-Doom-3...
- rplnt 15y agoThis time it wasn't leaked.
- Too 15y agoOh yea. Anyone still has the old code? Would be interesting to make a diff between the alpha and this one. Not to see specific line-by-line fixes but more just to get a general impression of how feature complete it was back then.
- rmc 15y agoThe difference is that this is an offical release and is under GPL. So you can fork this code and sell the results.
- jigs_up 15y agoAre there any parts of the quake source code in here?
- bengl3rt 15y agoHmm... Looks like this is an Xcode 3 project file. I can still get Xcode 3 from Apple's developer downloads, but it looks like it only includes the older iOS SDKs, not the OS X 10.5 SDK required to build this.
- forbes 15y agoTry: http://stackoverflow.com/questions/5378518/how-to-add-base-sdk-for-10-5-in-xcode-4 http://stackoverflow.com/questions/5378518/how-to-add-base-s...
- maranas 15y agothanks for this. i haven't even fully gone through quake 3's source yet.
- atomicdog 15y agoI'm still on Doom :/
- exDM69 15y agoThis seems to be pretty hard wired to build 32 bit binaries only, I'm having a little trouble build on Ubuntu w/ amd64. First you need to install multilibs and 32 bit gcc/g++. But I got stuck when I'd need some other libs in 32 bit versions (zlib, curl and probably openal and others). Ubuntu doesn't seem to ship with 32 bit versions of some libraries as some other distros do (like Arch, for example). Or am I wrong, what packages and what repositories I need to get e.g. zlib 32 bit binaries? Has anyone had success building this on a 64 bit machine? Is my only option to set up a virtual machine with a 32 bit distro?
- mackal 15y agomake sure you install g++-multilib so it pulls in the correct version. lib32z1-dev will work for zlib (need to create a symlink since it doesn't look in /usr/lib32). I don't see anything for 32-bit curl though :(
- exDM69 15y agoI did install g++-multilib and a few other packages in order to get the build forward. My build finally failed when trying to link the "doom3" executable due to the lack of 32 bit libs for zlib and curl (maybe others too). I tried to use a very unofficial Ubuntu utility called getlibs (http://frozenfox.freehostia.com/cappy http://frozenfox.freehostia.com/cappy) but that didn't really help, the library search path was wrong and the correct libs were not found.
- sparky 15y agoFix SConscript.curl to look in /usr/lib32 for zlib (package name: lib32z1-dev ), and to pass -m32 to your compiler. /media/d2/sw/doom3.gpl/neo$ git diff diff --git a/neo/sys/scons/SConscript.curl b/neo/sys/scons/SConscript.curl index 4d889b9..7271dfc 100644 --- a/neo/sys/scons/SConscript.curl +++ b/neo/sys/scons/SConscript.curl @@ -14,7 +14,7 @@ class idBuildCurl( scons_utils.idSetupBase ): def Compile( self, target = None, source = None, env = None ): self.TrySimpleCommand( 'cd curl ; make clean' ) - cmd = 'cd curl ; CC=\'' + env['CC'] + '\' ./configure --enable-shared=no --enable-static=yes --enable-http --enable-ftp --disable-gopher --enable-file --disable-lda + cmd = 'cd curl ; CC=\'' + env['CC'] + '\' CFLAGS=\'-m32\' ./configure --enable-shared=no --enable-static=yes --enable-http --enable-ftp --disable-gopher -- enable-fi if ( self.debug ): cmd += '--enable-debug' else: @@ -37,5 +37,5 @@ else: g_env.Command( target_name, None, Action( build.Compile ) ) -curl_libs = [ target_name, '/usr/lib/libz.a' ] +curl_libs = [ target_name, '/usr/lib32/libz.a' ] Return( 'curl_libs' ) Linked for me on Ubuntu 11.04 64-bit.
- kitsune_ 15y agoI'm impressed by the cleanliness of the code base, very readable. From my experience, this is not the case with many C++ open source projects.
- beaumartinez 15y ago...That's because Doom 3's engine is not an "open source project" as we know them ;)
- burgerbrain 15y agoNor was it closed source as we knew it. I wish I got paid to work on code that nice...
- jc-denton 15y agoInstalled Doom3 using the CDs and updated to the latest release. Then opened the sln file in VS 2010 and changed the command line args for debugging to point to the data of the game installed from the cd (instead of steam). Now what is the matter with default.cfg and what about "Unknown command 'vid_restart'"? I am not familiar with the structure of idgames. ------ Initializing File System ------ Current search path: C:\Program Files (x86)\DOOM 3\base/base game DLL: 0x0 in pak: 0x0 Addon pk4s: file system initialized. -------------------------------------- Unknown command 'vid_restart' idRenderSystem::Shutdown() Shutting down OpenGL subsystem ...shutting down QGL Couldn't load default.cfg
- hiena03 15y agoHow do I compile the map editor in linux? I guess it's in neo/tools/radiant but I haven't found a way to build it.
- barrettcolin 15y agoYou don't. It's only implemented for Win32 (see neo/tools/edit_stub.cpp).
- huskyr 15y agoIt will only be a matter of time before somebody converts this to Javascript :)
- admg 15y agoAnyone else having trouble trying to run it? got the assets in place, get to the menu fine, hit new game then get a black screen with an empty box in the middle. Suspect its the copy protection?
- smackfu 15y agoIt's kind of sad to see the expressions of surprise that it actually builds and runs. Implying that most source code releases fail test case #1.
- mdaniel 15y agoIn my experience, it is usually an environmental assumption that is mismanaged rather than the "source code" itself being broken. Whether that is failure to check for a dependency, failure to document a dependency or failure to clearly specify which specific version of a dependency is required. And the pain of autotools is a completely separate chapter in describing how "my" environment can differ from "theirs." All of that just goes to show that state management is _the_ hardest part of software.
- smackfu 15y agoTrue. Generally the failure of the packager is that they never do a clean install and build. And yes that is tedious but it finds so many issues, and isn't so bad with VMs.
- MtrL 15y agoHeard that the experimental render path with more advanced features is included in the source. Can anyone here confirm/deny that?
- kunqiana 15y agoI am getting the error, "scons: * [build/debug/core/sys/scons/doom] Source `/usr/lib/libz.a' not found, needed by target `build/debug/core/sys/scons/doom'. scons: building terminated because of errors." Anyone knows what I should install?
- enduser 15y agozlib or zlib-dev if your distribution separates out development packages
- kunqiana 15y agoAnyone got this error? DOOM 1.3.1.1304-debug linux-x86 Nov 24 2011 00:27:04 found interface lo - loopback found interface eth0 - 10.0.0.19/255.255.255.0 ------ Initializing File System ------ Current search path: /home/xx/.doom3/base /home/xx/doom3.gpl/neo/base game DLL: 0x0 in pak: 0x0 Addon pk4s: file system initialized. -------------------------------------- ----- Initializing Decls ----- ------------------------------ ------- Initializing renderSystem -------- idRenderSystem::Shutdown() Sys_Error: _default material not found