10 ms·

New DirectX Shader Compiler Based on Clang/LLVM Open-Sourced

125 points by xamlhacker 10y ago

pitaj 10y ago
I gotta say, this "new, open Microsoft" is really surprising me. I think what this signals is the end of selling software. Microsoft has seen the writing on the wall and realize that the money is in subscriptions and services. Due to piracy, a company can no longer just release an incremental update of their software package every couple years. They have to have rolling updates and constant improvements to keep the money stream rolling in from their customers. This is why MS is focusing on Azure, Office 360, Windows 10, etc.
- wolfgke 10y ago
  > Due to piracy, a company can no longer just release an incremental update of their software package every couple years. In earlier days (before Microsoft introduced much stronger copy protection schemes, such as the necessity to activate the products or WGA) a lot more installed copies of Microsoft products were illegal copies (in particular on computers owned by private users). And Microsoft survived quite well. So if you really believe in the "piracy explanation", you are deeply brainwashed by rightholder's propaganda. The IMHO most plausible reason why the model worked in former days, but worse today (though "worse" is probably a really bad word: revenue today is surely still much larger than in the former days), is that as long as Moore's law correlated to increased speed for existing applications (without need for new programming tricks such as having to use multithreading, new SIMD instructions etc. in the programs to make use of the additional power of new processors), there was a good reason to buy a new PC every few years. With each new PC there came a new OEM version of Windows and sometimes some Microsoft Office applications. So the formerly existing correlation between Moore's law and speed for existing applications drove Microsoft's sales.
shmerl 10y ago
Hopefully it will be of some help for this effort: https://github.com/KhronosGroup/glslang/issues/362 https://github.com/KhronosGroup/glslang/issues/362 I'm kind of surprised that this happened, though I've heard it could happen from glslang developers. It helps reducing DX/HLSL lock-in, and it's as if usual MS management responsible for such lock-in fell asleep, and developers managed to release something good for the world for once.
- dtech 10y ago
  Microsoft has been going away from that path for quite a while now, ever since Balmer left. For example the .NET runtime now runs on BSD/Unix/Mac OS X and sql server was released for Linux. Ultimately it is still in the interest of Microsoft if their toolchain is interesting because it is compatible among all target devices (PC, Xbox, PS, Mobile) so I'm unsurprised they are moving in that direction.
  - wolfgke 10y ago
    > For example the .NET runtime now runs on BSD/Unix/Mac OS X Under Balmer's time, there already publicly existed Microsoft Rotor - a .net runtime that ran under FreeBSD and OS X: > https://en.wikipedia.org/w/index.php?title=Shared_Source_Common_Language_Infrastructure&oldid=722315020 https://en.wikipedia.org/w/index.php?title=Shared_Source_Com... So don't do a corruption of historical facts.
    - josephcooney 10y ago
      While you're technically correct, ROTOR hadn't received any updates for a long time, and the source was licensed in such a was as to make sure anyone who could benefit from it running on *NIX platforms was heavily incentivized not to do so.
      - krylon 10y ago
        IIRC, Rotor's license explicitly forbade porting it to Linux. The impression I got back then was not so much of Microsoft showing how "cross-platform" .Net was, more like a big middle finger towards Linux.
        my123 10y ago
        .NET Core on Linux runs on guess what... Rotor. Rotor has been relicensed to MIT
        deleted 10y ago
frik 10y ago
Oh they are still working on DirectX 12. Microsoft has the tendency to downside and almost stop development after reaching near monopoly of a niche. Everyone remembers the Internet Explorer 6 years, it took years and Firefox reaching 25% market share to continue development of IE7. The same with DirectX: DirectX 9 was too successful, and OpenGL supported was limited to OpenGL 1 in WinXP and onwards (only tricks like bootstrapping allows OpenGL2+ on Win). DirectX 9 was around for many years. When OpenGL 3 and 4 came around, Microsoft restarted development of DirectX 10. DirectX 11 was merely a maintenance release. When the new AMD API (now Vulcan) came along, Microsoft restarted development with DirectX 12. Nowadays 99.9% of all new games are DirectX10/11 or Vulcan and support Win7+ and are usually available over Steam and/or GoG. And PlayStation 4 is very successful worldwide. XBoxOne is mainly successful in US and has little presents worldwide, Microsoft even stopped announcing sales two years ago - it's that bad. Nor are there any new exclusive games to speak of, PS4 has dozends of exclusive games, Win7+ has millions of exclusive games. Win10 Store is a complete desaster, worst software ever and hardly anyone would use it to buy games, if there are far better alternatives like Steam and GoG. Is DirectX 12 still a thing?
- dogma1138 10y ago
  Because DX12 gives you support for 2 out of the 3 major AAA platforms. Sony seems to have no inclination of supporting Vulcan as a native API for the PS4/PS4pro. Vulkan is also not Mantle, it's similar to the original Spec which was written by Johan Andersson from DICE. AMD did give Khronos the spec but there isn't some shared code implementation for Vulkan that is based on Mantle, there isn't a common shared library in general as the implementation for each IHV/System is proprietary. Vulkan is still a camel, not as much as OpenGL but a camel nonetheless, it's currently quite behind DX as far as featureset goes and Vulkan Next which would add needed features such as mGPU support seems lag into late 2017. Vulkan is a good alternative but it already shows some signs of getting it self into the same position that OpenGL found it self. It's also worth noting that DX12 development has nothing to do with Mantle, it's a continuation of the Xbox 360 API and effectively the mature version of the API developed for the Xbox One.
  - Tuna-Fish 10y ago
faragon 10y ago
So now all the "NDA" excuses for not releasing graphic drivers source code are gone?
- leonatan 10y ago
  One really has nothing to do with the other in this case.
- valarauca1 10y ago
  1. It never did exist 2. AMD has 3. Nvidia won't because their GPU does a lot of magic to fix DX9/DX10/DX11 screw ups some people shops who've partnered with Nvidia leave in their code.
  - john_reel 10y ago
    I’ve heard a bit about the last point but I’d like to know more. Can you elaborate?
    - blencdr 10y ago
      I think the third point is about that testimony: https://www.gamedev.net/topic/666419-what-are-your-opinions-on-dx12vulkanmantle/#entry5215019 https://www.gamedev.net/topic/666419-what-are-your-opinions-...
      - exDM69 10y ago
        That gets linked every now and then and it's pretty much BS, coming from someone who was an intern for a few months. Yes, there are workarounds and tweaks in the driver but it there was no other way back when games were shipped on a CD and updates were necessarily not applied. But even today it's not an option for AMD/NVidia/Intel to ship a driver update that will knowingly break Doom/Quake or any other shipping title. If a released product relies on a driver bug, the driver bug must be there indefinitely unless the game/app developer ships an update (which they might not do, the game might be years old and no longer supported). If you got a driver update that broke your game or hindered performance, you would be upset. Case in point: GLSL compilers which accept invalid input. That just can't be fixed without breaking some existing use. Then there are different strategies how the driver might implement Direct3D/OpenGL feature X. If 98% of the cases Strategy A is better but for a popular game Startegy B is better, the driver will selectively enable strategy that for a specific game. This is typically done in collaboration with the game developer and the GPU driver developers. Some years ago, cheating in benchmarks was rather common (and unsurprising). Benchmark results have disproportionate impact on sales and cheating, however dishonest, was a way to make a lot of sales with relatively little effort. I am under the impression that these days there are legally binding contracts between the manufacturers and benchmark companies that disallow this dubious practice. But in reality, most of the driver workarounds/hacks are artifacts of closed source development and binary releases and almost everyone benefits from them. The customer gets better performance, the game developer doesn't have to do as many workarounds (although it's beneficial for them to have a working relationship with the GPU vendors) and the GPU company gets sales because your favorite game runs better. I understand that this is giving some grief to indie developers and open source hackers. Not everyone can or wants to sign an NDA required for collaborating with GPU vendors. Please stop re-posting that forum post. It does not accurately reflect the reality. disclaimer: I'm a GPU driver programmer.
DannyBee 10y ago
While it's really nice they built this on open source compiler infrastructure, it's also too bad they took all the LLVM code and changed the license headers on it (additionally pretending to relicense it and changing the copyright string) when that's 100% not okay: https://github.com/Microsoft/DirectXShaderCompiler/blob/master/include/llvm/IR/Instruction.h#L5-L7 https://github.com/Microsoft/DirectXShaderCompiler/blob/mast... I'm sure it was a script, but this needs to be fixed, stat. Sadly, not even the first time: https://github.com/Microsoft/WinObjC/issues/35 https://github.com/Microsoft/WinObjC/issues/35 https://news.ycombinator.com/item?id=10024377 https://news.ycombinator.com/item?id=10024377 https://news.ycombinator.com/item?id=10018208 https://news.ycombinator.com/item?id=10018208 (which devolved into a licensing discussion) It'd be nice if folks paid a little more attention, because for releases from large companies, stuff spreads quickly, which means a year from now, there are still likely to be "mit licensed" copies of this that are still wrong :(
- pjc50 10y ago
  If there's anyone reading this who's a LLVM contributor (and therefore copyright holder), can you send a DMCA notice to github over this please? https://help.github.com/articles/dmca-takedown-policy/ https://help.github.com/articles/dmca-takedown-policy/
  - karim 10y ago
    I don't get it – why use the nuclear option when you could open an issue and ask them to look into it? This may be just an oversight.
    - jpgvm 10y ago
      It's almost certainly an oversight, an issue is the correct course of action here. You can always resort to DCMA if that isn't effective.
      - 234dd57d2c8db 10y ago
        If you don't defend your copyright you lose it. Taking legal action is appropriate. How many times does Microsoft have to make this "mistake" before it's seen as just ripping off people's code, hoping they don't notice, and then getting to make money off of said code? Microsoft is one of the largest software companies in the world, and they don't have the tooling in place to be able to use other people's code without keeping the license in tact and legal? I don't buy it.

125 points by xamlhacker 10y ago

pitaj 10y ago

I gotta say, this "new, open Microsoft" is really surprising me. I think what this signals is the end of selling software. Microsoft has seen the writing on the wall and realize that the money is in subscriptions and services. Due to piracy, a company can no longer just release an incremental update of their software package every couple years. They have to have rolling updates and constant improvements to keep the money stream rolling in from their customers. This is why MS is focusing on Azure, Office 360, Windows 10, etc.

wolfgke 10y ago
> Due to piracy, a company can no longer just release an incremental update of their software package every couple years. In earlier days (before Microsoft introduced much stronger copy protection schemes, such as the necessity to activate the products or WGA) a lot more installed copies of Microsoft products were illegal copies (in particular on computers owned by private users). And Microsoft survived quite well. So if you really believe in the "piracy explanation", you are deeply brainwashed by rightholder's propaganda. The IMHO most plausible reason why the model worked in former days, but worse today (though "worse" is probably a really bad word: revenue today is surely still much larger than in the former days), is that as long as Moore's law correlated to increased speed for existing applications (without need for new programming tricks such as having to use multithreading, new SIMD instructions etc. in the programs to make use of the additional power of new processors), there was a good reason to buy a new PC every few years. With each new PC there came a new OEM version of Windows and sometimes some Microsoft Office applications. So the formerly existing correlation between Moore's law and speed for existing applications drove Microsoft's sales.

shmerl 10y ago

Hopefully it will be of some help for this effort: https://github.com/KhronosGroup/glslang/issues/362 https://github.com/KhronosGroup/glslang/issues/362 I'm kind of surprised that this happened, though I've heard it could happen from glslang developers. It helps reducing DX/HLSL lock-in, and it's as if usual MS management responsible for such lock-in fell asleep, and developers managed to release something good for the world for once.

dtech 10y ago
Microsoft has been going away from that path for quite a while now, ever since Balmer left. For example the .NET runtime now runs on BSD/Unix/Mac OS X and sql server was released for Linux. Ultimately it is still in the interest of Microsoft if their toolchain is interesting because it is compatible among all target devices (PC, Xbox, PS, Mobile) so I'm unsurprised they are moving in that direction.
- wolfgke 10y ago
  > For example the .NET runtime now runs on BSD/Unix/Mac OS X Under Balmer's time, there already publicly existed Microsoft Rotor - a .net runtime that ran under FreeBSD and OS X: > https://en.wikipedia.org/w/index.php?title=Shared_Source_Common_Language_Infrastructure&oldid=722315020 https://en.wikipedia.org/w/index.php?title=Shared_Source_Com... So don't do a corruption of historical facts.
  - josephcooney 10y ago
    While you're technically correct, ROTOR hadn't received any updates for a long time, and the source was licensed in such a was as to make sure anyone who could benefit from it running on *NIX platforms was heavily incentivized not to do so.
    - krylon 10y ago
      IIRC, Rotor's license explicitly forbade porting it to Linux. The impression I got back then was not so much of Microsoft showing how "cross-platform" .Net was, more like a big middle finger towards Linux.
      - my123 10y ago
        .NET Core on Linux runs on guess what... Rotor. Rotor has been relicensed to MIT
        deleted 10y ago

frik 10y ago

Oh they are still working on DirectX 12. Microsoft has the tendency to downside and almost stop development after reaching near monopoly of a niche. Everyone remembers the Internet Explorer 6 years, it took years and Firefox reaching 25% market share to continue development of IE7. The same with DirectX: DirectX 9 was too successful, and OpenGL supported was limited to OpenGL 1 in WinXP and onwards (only tricks like bootstrapping allows OpenGL2+ on Win). DirectX 9 was around for many years. When OpenGL 3 and 4 came around, Microsoft restarted development of DirectX 10. DirectX 11 was merely a maintenance release. When the new AMD API (now Vulcan) came along, Microsoft restarted development with DirectX 12. Nowadays 99.9% of all new games are DirectX10/11 or Vulcan and support Win7+ and are usually available over Steam and/or GoG. And PlayStation 4 is very successful worldwide. XBoxOne is mainly successful in US and has little presents worldwide, Microsoft even stopped announcing sales two years ago - it's that bad. Nor are there any new exclusive games to speak of, PS4 has dozends of exclusive games, Win7+ has millions of exclusive games. Win10 Store is a complete desaster, worst software ever and hardly anyone would use it to buy games, if there are far better alternatives like Steam and GoG. Is DirectX 12 still a thing?

dogma1138 10y ago
Because DX12 gives you support for 2 out of the 3 major AAA platforms. Sony seems to have no inclination of supporting Vulcan as a native API for the PS4/PS4pro. Vulkan is also not Mantle, it's similar to the original Spec which was written by Johan Andersson from DICE. AMD did give Khronos the spec but there isn't some shared code implementation for Vulkan that is based on Mantle, there isn't a common shared library in general as the implementation for each IHV/System is proprietary. Vulkan is still a camel, not as much as OpenGL but a camel nonetheless, it's currently quite behind DX as far as featureset goes and Vulkan Next which would add needed features such as mGPU support seems lag into late 2017. Vulkan is a good alternative but it already shows some signs of getting it self into the same position that OpenGL found it self. It's also worth noting that DX12 development has nothing to do with Mantle, it's a continuation of the Xbox 360 API and effectively the mature version of the API developed for the Xbox One.
- Tuna-Fish 10y ago

faragon 10y ago

So now all the "NDA" excuses for not releasing graphic drivers source code are gone?

leonatan 10y ago
One really has nothing to do with the other in this case.
valarauca1 10y ago
1. It never did exist 2. AMD has 3. Nvidia won't because their GPU does a lot of magic to fix DX9/DX10/DX11 screw ups some people shops who've partnered with Nvidia leave in their code.
- john_reel 10y ago
  I’ve heard a bit about the last point but I’d like to know more. Can you elaborate?
  - blencdr 10y ago
    I think the third point is about that testimony: https://www.gamedev.net/topic/666419-what-are-your-opinions-on-dx12vulkanmantle/#entry5215019 https://www.gamedev.net/topic/666419-what-are-your-opinions-...
    - exDM69 10y ago
      That gets linked every now and then and it's pretty much BS, coming from someone who was an intern for a few months. Yes, there are workarounds and tweaks in the driver but it there was no other way back when games were shipped on a CD and updates were necessarily not applied. But even today it's not an option for AMD/NVidia/Intel to ship a driver update that will knowingly break Doom/Quake or any other shipping title. If a released product relies on a driver bug, the driver bug must be there indefinitely unless the game/app developer ships an update (which they might not do, the game might be years old and no longer supported). If you got a driver update that broke your game or hindered performance, you would be upset. Case in point: GLSL compilers which accept invalid input. That just can't be fixed without breaking some existing use. Then there are different strategies how the driver might implement Direct3D/OpenGL feature X. If 98% of the cases Strategy A is better but for a popular game Startegy B is better, the driver will selectively enable strategy that for a specific game. This is typically done in collaboration with the game developer and the GPU driver developers. Some years ago, cheating in benchmarks was rather common (and unsurprising). Benchmark results have disproportionate impact on sales and cheating, however dishonest, was a way to make a lot of sales with relatively little effort. I am under the impression that these days there are legally binding contracts between the manufacturers and benchmark companies that disallow this dubious practice. But in reality, most of the driver workarounds/hacks are artifacts of closed source development and binary releases and almost everyone benefits from them. The customer gets better performance, the game developer doesn't have to do as many workarounds (although it's beneficial for them to have a working relationship with the GPU vendors) and the GPU company gets sales because your favorite game runs better. I understand that this is giving some grief to indie developers and open source hackers. Not everyone can or wants to sign an NDA required for collaborating with GPU vendors. Please stop re-posting that forum post. It does not accurately reflect the reality. disclaimer: I'm a GPU driver programmer.

DannyBee 10y ago

While it's really nice they built this on open source compiler infrastructure, it's also too bad they took all the LLVM code and changed the license headers on it (additionally pretending to relicense it and changing the copyright string) when that's 100% not okay: https://github.com/Microsoft/DirectXShaderCompiler/blob/master/include/llvm/IR/Instruction.h#L5-L7 https://github.com/Microsoft/DirectXShaderCompiler/blob/mast... I'm sure it was a script, but this needs to be fixed, stat. Sadly, not even the first time: https://github.com/Microsoft/WinObjC/issues/35 https://github.com/Microsoft/WinObjC/issues/35 https://news.ycombinator.com/item?id=10024377 https://news.ycombinator.com/item?id=10024377 https://news.ycombinator.com/item?id=10018208 https://news.ycombinator.com/item?id=10018208 (which devolved into a licensing discussion) It'd be nice if folks paid a little more attention, because for releases from large companies, stuff spreads quickly, which means a year from now, there are still likely to be "mit licensed" copies of this that are still wrong :(

pjc50 10y ago
If there's anyone reading this who's a LLVM contributor (and therefore copyright holder), can you send a DMCA notice to github over this please? https://help.github.com/articles/dmca-takedown-policy/ https://help.github.com/articles/dmca-takedown-policy/
- karim 10y ago
  I don't get it – why use the nuclear option when you could open an issue and ask them to look into it? This may be just an oversight.
  - jpgvm 10y ago
    It's almost certainly an oversight, an issue is the correct course of action here. You can always resort to DCMA if that isn't effective.
    - 234dd57d2c8db 10y ago
      If you don't defend your copyright you lose it. Taking legal action is appropriate. How many times does Microsoft have to make this "mistake" before it's seen as just ripping off people's code, hoping they don't notice, and then getting to make money off of said code? Microsoft is one of the largest software companies in the world, and they don't have the tooling in place to be able to use other people's code without keeping the license in tact and legal? I don't buy it.