5 ms·

Most in-house game engines built after a certain point use a non-trivial amount of third party code, including console stuff which is under strict NDA, so it's

Most in-house game engines built after a certain point use a non-trivial amount of third party code, including console stuff which is under strict NDA, so it's a huge hassle to open source them. Most iterations of the Source engine use Havok physics for example. IdTech probably was only open sourced because Carmack pushed for it, but it helps that IdTech of that vintage was all in-house code exclusively targeting the PC. I think the only thing they had to cut out for legal reasons was the patented shadowing algorithm in Doom 3.

ThatPlayer 2y ago
Even the original Doom source code release was the Linux source code because the Dos version of the game uses a 3rd party audio library, DMX.
- dcrazy 2y ago
  I believe DMX was also used for Descent, and the company successfully shut down open source projects to reverse engineer the file format. Interplay was also extremely protective of their full-motion video codec, which only got well-documented after the release of Baldur’s Gate II.
phire 2y ago
Id were using a 3rd party sound library for Doom to handle the nitty-gritty aspects of DOS sound hardware, so the open source release was based on the linux port of Doom. People had to port it back to DOS. They couldn't release the windows port of Doom either, as that had been done by Microsoft, and would therefore include Microsoft copyrighted code. With Quake, Id did their own windows port, so it was possible to release the source code for winquake.
- stevage 2y ago
  > as that had been done by Microsoft, and would therefore include Microsoft copyrighted code. How does that follow? Normally copyright would transfer to the company paying for it.
  - nocman 2y ago
    It all depends on how the contract between Microsoft and Id was worded.
    - phire 2y ago
      If Id had paid Microsoft to do the port, it would automatically be work-for-hire and the copyrights would belong to Id, unless the contract said otherwise. But Microsoft essentially licensed the Doom source code and ported it themselves, it's not work-for-hire, so the copyright on any changes would default to Microsoft. Though, I'm really not sure if Microsoft paid anything for this licence, or if there even was a contract.
      - kragen 2y ago
        This is a comprehensively incorrect explanation of copyright and contract law.
      - lmm 2y ago
        > If Id had paid Microsoft to do the port, it would automatically be work-for-hire and the copyrights would belong to Id, unless the contract said otherwise. Nope, not true. Work by non employees is only work-for-hire if it falls into one of the qualifying categories and the contract explicitly says that it is work-for-hire.
      - zeroq 2y ago
        It's definitely a topic for a CSI: HN, but I think it's a was most likely a part of a larger agreement between a relatively small shop and a behemoth.
      - bitwize 2y ago
        It's not automatically work-for-hire; that has to be stipulated in the contract. Recently there was a lawsuit about this involving, of all things, Sonic the Hedgehog. An artist and writer named Ken Penders, working for Archie Comics on Sonic comic books, sued his former employer claiming ownership over certain characters created for the comic, most notably those connected to Knuckles the Echidna. Because Archie couldn't prove that Penders's comic work was work for hire (they lost the original contract), they settled out of court in favor of Penders. This ended up losing the Sonic license and had various effects on the Sonic universe. For example, he threatened to sue Paramount over the idea of Knuckles having a father, because Knuckles's Archie Comic universe father was a Penders creation. So rather than depict his father directly for the Knuckles series, they represented him with a puppet in an abstract sequence -- a puppet bearing a suspiciously Penders-like moustache.
- ndiddy 2y ago
  The wording used in the release notes was: > The bad news: this code only compiles and runs on linux. We couldn't release the dos code because of a copyrighted sound library we used (wow, was that a mistake -- I write my own sound code now), and I honestly don't even know what happened to the port that microsoft did to windows. > Still, the code is quite portable, and it should be straightforward to bring it up on just about any platform. It seems like they just didn't have immediate access to the code for the Windows version. The DOS source code eventually leaked a couple years ago along with the code to the Mac port of Doom. https://archive.org/details/doom-mac-source https://archive.org/details/doom-mac-source
- tadfisher 2y ago
  Fun fact: Gabe Newell built the original WinDoom port, and led the team that built Doom95!
  - kridsdale3 2y ago
    That IS a fun fact.
    - Me001 2y ago
      [dead]
  - sho_hn 2y ago
    To be clear, Robert Hess was the primary engineer on WinDoom, Newell was the team lead.
  - Scuds 2y ago
    I thought Monolith worked on doom95.
gmueckl 2y ago
If a team had access to some licensed source code related the third party component (e.g. restrictively licensed/NDA'd sample code), clearing the engine source for release may involve a thorough code audit. Who is going to put in that work to prepare a freebie?
yieldcrv 2y ago
that and the hodgepodge structure of the codebase is probably embarrassing
lawlessone 2y ago
ah, that NDA stuff is interesting, that probably explains it.
- chefandy 2y ago
  I wonder how UE manages it. It compiles for consoles but you can download and modify the engine source. The license isn’t open but you can still get the source.
  - arduinomancer 2y ago
    Not sure about UE but I’ve worked on other engines and usually you just have an interface that abstracts platform APIs and each platform implements the same interface The implementations get compiled in based on whatever platform you’re compiling for using a bunch of #ifdef statements In that case it’s just a matter of not including certain .h/.cpp files
  - jsheard 2y ago
    The source you can easily get access to doesn't include any of the console stuff, you have to prove you're licensed to develop on those consoles to get access to that code.
    - chefandy 2y ago
      Makes sense. Definitely something valve would have to deliberately set up and it probably would be more complicated than it seems at the outset.
znpy 2y ago
> Most in-house game engines built after a certain point use a non-trivial amount of third party code, including console stuff which is under strict NDA, so it's a huge hassle to open source them. reminds of the whole "opensourcing solaris" drama that Bryan Cantrill talked about in his speech at usenix some ten years ago. i wonder if the culprit is i18n again, "lol"
jamesfinlayson 2y ago
Source does, although within the last year they replaced Miles for MP3 and Rad Games Tools for Bink with tinymp3 and free WebM library. Probably other stuff there but not sure off the top of my head.
- account42 2y ago
  You see WebM in quite a few projects that have had Linux ports. Apparently Bink requires separate licenses per platform. OGG is also popular instead of MP3 but with the MP3 patens expired that isn't a legal concern anymore, just a technical superiority one.
  - jamesfinlayson 2y ago
    Oh, I didn't realise separate platforms required separate licenses so that makes sense. And yeah too much effort to re-encode all the game audio most likely.
ryandrake 2y ago
> non-trivial amount of third party code, including console stuff which is under strict NDA This just seems like a lazy excuse. Ok so some of the code can’t be released. Fine, what stops you from open sourcing the rest of it that isn’t licensed? The OS community will surely fill in The blanks.
- Imustaskforhelp 2y ago
  exactly. I do think that the source code of source was leaked. Its just that valve has to themselves give the code of source. FLOSS community would really do it. Trust me.
  - account42 2y ago
    Well for the FLOSS community to do anything Valve would not just need to release the coede but also put it under a free software license, which isn't the case even for the game code released here. Modders in general won't care though.
- L3viathan 2y ago
  Well in a sense you are right that it is lazy, in that it would be quite a lot of work to disentangle those parts. So they could probably do it, but it would be costly, while just releasing the TF2 code is basically free.
  - lupusreal 2y ago
    Don't bother with subtle disentanglement. Just take an axe to it and release a broken mess. It will be fixed and running within a month.
- trinix912 2y ago
  It depends on how the code is structured. It might be that the licensed parts are sprinkled throughout the functions, or that they would have to reorganize things to not reveal too much about a specific NDA'd technology (like a console SDK)... The resulting code might end up being just as useless as no code at all, and I assume it would be very hard to get the management to allocate resources for that.
zbendefy 2y ago
Not just that, they had specific renderer backends, one for GeForce, one for GeForce3, one for Radeon 8500 that they had to cut out as they used proprietary information or code perhaps.
- zbendefy 2y ago
  Correction, I didnt remember correctly as those are actually there in the code release, here is the r200 (radeon 8500) renderer specific code: https://github.com/id-Software/DOOM-3/blob/master/neo/renderer/draw_r200.cpp https://github.com/id-Software/DOOM-3/blob/master/neo/render...
skhr0680 2y ago
> I think the only thing they had to cut out for legal reasons was the patented shadowing algorithm in Doom 3. I remember that being somewhere in the neighborhood of 1 LOC to fix, and exactly where to change it was “common knowledge” the moment the source came out
- account42 2y ago
  It's quite silly that depth fail shadow volumes were ever granted a patent.
vanderZwan 2y ago
I recall Ton Roosendaal saying something similar about the origins of Blender: convincing his then-employers to let him open source the original abandoned engine was a huge hassle, but the lack of third party code blocking it helped a lot.
- zeroq 2y ago
  I had similar experience working for BigCo. On one hand, at one point we had CEO and CTO publicly stating in townhalls that they want us to be a technological forefront and looking for ways to showcase our capabilities. At the same time it was virtually impossible to open source even something as trivial as a string formatting library, because once you started talking to stakeholders about approval their POV was that it's part of a project that took X amount of employees Y amount of time and that equation is in millions, so that has to be really valuable.
  - jandrese 2y ago
    Middle managers look at that XKCD comic[1] and say "we need to buy out that guy in Nebraska and monetize it". [1] https://xkcd.com/2347/ https://xkcd.com/2347/
    - zeroq 2y ago
      strong leftPad vibes here hell, when I joined the frontend circus in 2013 I was flabbergasted that most people needed to install 3rd party library to perform "isEven(num)" operation ... not to mention that aforementioned library had at least 3 dependencies itself.
account42 2y ago
Even for the source release we have gotten it's not too uncommon that some proprietary middleware is missing and needs to be replaced before being able to build the code.

ThatPlayer 2y ago

Even the original Doom source code release was the Linux source code because the Dos version of the game uses a 3rd party audio library, DMX.

dcrazy 2y ago
I believe DMX was also used for Descent, and the company successfully shut down open source projects to reverse engineer the file format. Interplay was also extremely protective of their full-motion video codec, which only got well-documented after the release of Baldur’s Gate II.

phire 2y ago

Id were using a 3rd party sound library for Doom to handle the nitty-gritty aspects of DOS sound hardware, so the open source release was based on the linux port of Doom. People had to port it back to DOS. They couldn't release the windows port of Doom either, as that had been done by Microsoft, and would therefore include Microsoft copyrighted code. With Quake, Id did their own windows port, so it was possible to release the source code for winquake.

stevage 2y ago
> as that had been done by Microsoft, and would therefore include Microsoft copyrighted code. How does that follow? Normally copyright would transfer to the company paying for it.
- nocman 2y ago
  It all depends on how the contract between Microsoft and Id was worded.
  - phire 2y ago
    If Id had paid Microsoft to do the port, it would automatically be work-for-hire and the copyrights would belong to Id, unless the contract said otherwise. But Microsoft essentially licensed the Doom source code and ported it themselves, it's not work-for-hire, so the copyright on any changes would default to Microsoft. Though, I'm really not sure if Microsoft paid anything for this licence, or if there even was a contract.
    - kragen 2y ago
      This is a comprehensively incorrect explanation of copyright and contract law.
    - lmm 2y ago
      > If Id had paid Microsoft to do the port, it would automatically be work-for-hire and the copyrights would belong to Id, unless the contract said otherwise. Nope, not true. Work by non employees is only work-for-hire if it falls into one of the qualifying categories and the contract explicitly says that it is work-for-hire.
    - zeroq 2y ago
      It's definitely a topic for a CSI: HN, but I think it's a was most likely a part of a larger agreement between a relatively small shop and a behemoth.
    - bitwize 2y ago
      It's not automatically work-for-hire; that has to be stipulated in the contract. Recently there was a lawsuit about this involving, of all things, Sonic the Hedgehog. An artist and writer named Ken Penders, working for Archie Comics on Sonic comic books, sued his former employer claiming ownership over certain characters created for the comic, most notably those connected to Knuckles the Echidna. Because Archie couldn't prove that Penders's comic work was work for hire (they lost the original contract), they settled out of court in favor of Penders. This ended up losing the Sonic license and had various effects on the Sonic universe. For example, he threatened to sue Paramount over the idea of Knuckles having a father, because Knuckles's Archie Comic universe father was a Penders creation. So rather than depict his father directly for the Knuckles series, they represented him with a puppet in an abstract sequence -- a puppet bearing a suspiciously Penders-like moustache.
ndiddy 2y ago
The wording used in the release notes was: > The bad news: this code only compiles and runs on linux. We couldn't release the dos code because of a copyrighted sound library we used (wow, was that a mistake -- I write my own sound code now), and I honestly don't even know what happened to the port that microsoft did to windows. > Still, the code is quite portable, and it should be straightforward to bring it up on just about any platform. It seems like they just didn't have immediate access to the code for the Windows version. The DOS source code eventually leaked a couple years ago along with the code to the Mac port of Doom. https://archive.org/details/doom-mac-source https://archive.org/details/doom-mac-source
tadfisher 2y ago
Fun fact: Gabe Newell built the original WinDoom port, and led the team that built Doom95!
- kridsdale3 2y ago
  That IS a fun fact.
  - Me001 2y ago
    [dead]
- sho_hn 2y ago
  To be clear, Robert Hess was the primary engineer on WinDoom, Newell was the team lead.
- Scuds 2y ago
  I thought Monolith worked on doom95.

gmueckl 2y ago

If a team had access to some licensed source code related the third party component (e.g. restrictively licensed/NDA'd sample code), clearing the engine source for release may involve a thorough code audit. Who is going to put in that work to prepare a freebie?

yieldcrv 2y ago

that and the hodgepodge structure of the codebase is probably embarrassing

lawlessone 2y ago

ah, that NDA stuff is interesting, that probably explains it.

chefandy 2y ago
I wonder how UE manages it. It compiles for consoles but you can download and modify the engine source. The license isn’t open but you can still get the source.
- arduinomancer 2y ago
  Not sure about UE but I’ve worked on other engines and usually you just have an interface that abstracts platform APIs and each platform implements the same interface The implementations get compiled in based on whatever platform you’re compiling for using a bunch of #ifdef statements In that case it’s just a matter of not including certain .h/.cpp files
- jsheard 2y ago
  The source you can easily get access to doesn't include any of the console stuff, you have to prove you're licensed to develop on those consoles to get access to that code.
  - chefandy 2y ago
    Makes sense. Definitely something valve would have to deliberately set up and it probably would be more complicated than it seems at the outset.

znpy 2y ago

> Most in-house game engines built after a certain point use a non-trivial amount of third party code, including console stuff which is under strict NDA, so it's a huge hassle to open source them. reminds of the whole "opensourcing solaris" drama that Bryan Cantrill talked about in his speech at usenix some ten years ago. i wonder if the culprit is i18n again, "lol"

jamesfinlayson 2y ago

Source does, although within the last year they replaced Miles for MP3 and Rad Games Tools for Bink with tinymp3 and free WebM library. Probably other stuff there but not sure off the top of my head.

account42 2y ago
You see WebM in quite a few projects that have had Linux ports. Apparently Bink requires separate licenses per platform. OGG is also popular instead of MP3 but with the MP3 patens expired that isn't a legal concern anymore, just a technical superiority one.
- jamesfinlayson 2y ago
  Oh, I didn't realise separate platforms required separate licenses so that makes sense. And yeah too much effort to re-encode all the game audio most likely.

ryandrake 2y ago

> non-trivial amount of third party code, including console stuff which is under strict NDA This just seems like a lazy excuse. Ok so some of the code can’t be released. Fine, what stops you from open sourcing the rest of it that isn’t licensed? The OS community will surely fill in The blanks.

Imustaskforhelp 2y ago
exactly. I do think that the source code of source was leaked. Its just that valve has to themselves give the code of source. FLOSS community would really do it. Trust me.
- account42 2y ago
  Well for the FLOSS community to do anything Valve would not just need to release the coede but also put it under a free software license, which isn't the case even for the game code released here. Modders in general won't care though.
L3viathan 2y ago
Well in a sense you are right that it is lazy, in that it would be quite a lot of work to disentangle those parts. So they could probably do it, but it would be costly, while just releasing the TF2 code is basically free.
- lupusreal 2y ago
  Don't bother with subtle disentanglement. Just take an axe to it and release a broken mess. It will be fixed and running within a month.
trinix912 2y ago
It depends on how the code is structured. It might be that the licensed parts are sprinkled throughout the functions, or that they would have to reorganize things to not reveal too much about a specific NDA'd technology (like a console SDK)... The resulting code might end up being just as useless as no code at all, and I assume it would be very hard to get the management to allocate resources for that.

zbendefy 2y ago

Not just that, they had specific renderer backends, one for GeForce, one for GeForce3, one for Radeon 8500 that they had to cut out as they used proprietary information or code perhaps.

zbendefy 2y ago
Correction, I didnt remember correctly as those are actually there in the code release, here is the r200 (radeon 8500) renderer specific code: https://github.com/id-Software/DOOM-3/blob/master/neo/renderer/draw_r200.cpp https://github.com/id-Software/DOOM-3/blob/master/neo/render...

skhr0680 2y ago

> I think the only thing they had to cut out for legal reasons was the patented shadowing algorithm in Doom 3. I remember that being somewhere in the neighborhood of 1 LOC to fix, and exactly where to change it was “common knowledge” the moment the source came out

account42 2y ago
It's quite silly that depth fail shadow volumes were ever granted a patent.

vanderZwan 2y ago

I recall Ton Roosendaal saying something similar about the origins of Blender: convincing his then-employers to let him open source the original abandoned engine was a huge hassle, but the lack of third party code blocking it helped a lot.

zeroq 2y ago
I had similar experience working for BigCo. On one hand, at one point we had CEO and CTO publicly stating in townhalls that they want us to be a technological forefront and looking for ways to showcase our capabilities. At the same time it was virtually impossible to open source even something as trivial as a string formatting library, because once you started talking to stakeholders about approval their POV was that it's part of a project that took X amount of employees Y amount of time and that equation is in millions, so that has to be really valuable.
- jandrese 2y ago
  Middle managers look at that XKCD comic[1] and say "we need to buy out that guy in Nebraska and monetize it". [1] https://xkcd.com/2347/ https://xkcd.com/2347/
  - zeroq 2y ago
    strong leftPad vibes here hell, when I joined the frontend circus in 2013 I was flabbergasted that most people needed to install 3rd party library to perform "isEven(num)" operation ... not to mention that aforementioned library had at least 3 dependencies itself.

account42 2y ago

Even for the source release we have gotten it's not too uncommon that some proprietary middleware is missing and needs to be replaced before being able to build the code.