4 ms·

Can someone knowledgeable about the space please help me understand why this is important? The article discusses this with regards to "Nouveau", which seems to

Can someone knowledgeable about the space please help me understand why this is important? The article discusses this with regards to "Nouveau", which seems to be an open source Nvidia specific driver - https://en.wikipedia.org/wiki/Nouveau_(software) https://en.wikipedia.org/wiki/Nouveau_(software) What do these 3d header files do, and why/how will this make life easier for "Nouveau"? Finally, why (aside from just open source being the best source) will this benefit people like myself who don't understand what just happened?

monocasa 4y ago
Inside just about every Nvidia GPU there's a bunch of little engines they internally call classes. This documents the 3D graphics rendering engines' method calls (the commands in the physical GPU command list) and memory layout for instances of these classes.
stevehawk 4y ago
basically they just published a large amount of API for developers to use
jpollock 4y ago
When looking at a binary blob for the driver, the person writing the docs of what the driver does sees "jump $addr1" but they don't know what's at the other end. To write the specification for the clean room reimplementation, they need to read the destination assembly and document that. This provides documentation on what is at the other end of the address and what it does, and what the layout of each parameter is. That means they will hopefully avoid having to recurse and understand all of the binary code to write the specification.
- averne_ 4y ago
  That's not exactly correct. This is register maps for the 3d engine (also called class), what you describe would be closer to the shader ISA. In driver code you'll see them building command buffers that set registers in those classes to certain values. It could be the RGBA values of the clear color, or a virtual address in the GPU space. This documents the names of these registers. This makes reverse engineering somewhat easier as you don't really have to guess anymore. But in most cases it was pretty clear from the start, and for some generations those were already well documented by open source efforts. One of best known gens is probably Maxwell since it was used in the Nintendo Switch, see for instance [1] (or code in yuzu/Ryujinx) which is the equivalent of those headers NV published. However this isn't a very big step in documenting their GPUs. The exact functions of those registers aren't explained, but most importantly the shader ISA isn't documented at all, which is essential to build a good open-source driver. Source: I have reverse engineered some driver code for Maxwell (and used similar headers to write drivers for nvdec and nvjpg). [1] https://github.com/devkitPro/deko3d/blob/master/source/maxwell/engine_3d.def https://github.com/devkitPro/deko3d/blob/master/source/maxwe...
  - digdugdirk 4y ago
    You say this isn't a very big step - out of curiosity, how much work would have been involved in getting this release open sourced? And what might be preventing them from releasing the shader ISA? Specifically curious if this inherently would "let the cat out of the bag" about the specifics of their chip design, for example.
    - averne_ 4y ago
      > how much work would have been involved in getting this release open sourced Close to no actual effort (the headers are autogenerated). However there was probably a lot of work behind the scenes with their legal team/whatever to clear the release. About the shader ISA, I wish I knew. It's certain that the documentation exists, because they provide it to some developers (I've been told the Maxwell ISA docs are part of the Nintendo Switch SDK), so it's not like they have to write it from scratch. And AMD provides full docs about it [1] (not sure about Intel), so I don't see how it could provide a significant edge over their competition. Maybe raytracing instructions? But for a motivated reverse engineer this stuff isn't impossible to figure out. I think it's down to company culture and inertia. [1] https://developer.amd.com/wp-content/resources/RDNA2_Shader_ISA_November2020.pdf https://developer.amd.com/wp-content/resources/RDNA2_Shader_...
      - monocasa 4y ago
        > not sure about Intel I don't know about Arc, but Intel has historically probably been the most open with their GPU docs. AMD's is (while laudable, particularly compared to Nvidia) unfortunately a bit of a autogenned dump without context. Where as Intel's is closer to what I expect out of decent SoC documentation; it'll be about half prose explaining the why(!!), and half the autogenned tables of registers and opcode formats.
        ahartmetz 4y ago
        AMD's approach seems to be documentation in the style you describe plus employed FOSS developers who get access to internal people, simulators, possibly even source code. Makes it annoying for externals to try to contribute code, but the drivers are good.
deleted 4y ago
[deleted]

monocasa 4y ago

Inside just about every Nvidia GPU there's a bunch of little engines they internally call classes. This documents the 3D graphics rendering engines' method calls (the commands in the physical GPU command list) and memory layout for instances of these classes.

stevehawk 4y ago

basically they just published a large amount of API for developers to use

jpollock 4y ago

When looking at a binary blob for the driver, the person writing the docs of what the driver does sees "jump $addr1" but they don't know what's at the other end. To write the specification for the clean room reimplementation, they need to read the destination assembly and document that. This provides documentation on what is at the other end of the address and what it does, and what the layout of each parameter is. That means they will hopefully avoid having to recurse and understand all of the binary code to write the specification.

averne_ 4y ago
That's not exactly correct. This is register maps for the 3d engine (also called class), what you describe would be closer to the shader ISA. In driver code you'll see them building command buffers that set registers in those classes to certain values. It could be the RGBA values of the clear color, or a virtual address in the GPU space. This documents the names of these registers. This makes reverse engineering somewhat easier as you don't really have to guess anymore. But in most cases it was pretty clear from the start, and for some generations those were already well documented by open source efforts. One of best known gens is probably Maxwell since it was used in the Nintendo Switch, see for instance [1] (or code in yuzu/Ryujinx) which is the equivalent of those headers NV published. However this isn't a very big step in documenting their GPUs. The exact functions of those registers aren't explained, but most importantly the shader ISA isn't documented at all, which is essential to build a good open-source driver. Source: I have reverse engineered some driver code for Maxwell (and used similar headers to write drivers for nvdec and nvjpg). [1] https://github.com/devkitPro/deko3d/blob/master/source/maxwell/engine_3d.def https://github.com/devkitPro/deko3d/blob/master/source/maxwe...
- digdugdirk 4y ago
  You say this isn't a very big step - out of curiosity, how much work would have been involved in getting this release open sourced? And what might be preventing them from releasing the shader ISA? Specifically curious if this inherently would "let the cat out of the bag" about the specifics of their chip design, for example.
  - averne_ 4y ago
    > how much work would have been involved in getting this release open sourced Close to no actual effort (the headers are autogenerated). However there was probably a lot of work behind the scenes with their legal team/whatever to clear the release. About the shader ISA, I wish I knew. It's certain that the documentation exists, because they provide it to some developers (I've been told the Maxwell ISA docs are part of the Nintendo Switch SDK), so it's not like they have to write it from scratch. And AMD provides full docs about it [1] (not sure about Intel), so I don't see how it could provide a significant edge over their competition. Maybe raytracing instructions? But for a motivated reverse engineer this stuff isn't impossible to figure out. I think it's down to company culture and inertia. [1] https://developer.amd.com/wp-content/resources/RDNA2_Shader_ISA_November2020.pdf https://developer.amd.com/wp-content/resources/RDNA2_Shader_...
    - monocasa 4y ago
      > not sure about Intel I don't know about Arc, but Intel has historically probably been the most open with their GPU docs. AMD's is (while laudable, particularly compared to Nvidia) unfortunately a bit of a autogenned dump without context. Where as Intel's is closer to what I expect out of decent SoC documentation; it'll be about half prose explaining the why(!!), and half the autogenned tables of registers and opcode formats.
      - ahartmetz 4y ago
        AMD's approach seems to be documentation in the style you describe plus employed FOSS developers who get access to internal people, simulators, possibly even source code. Makes it annoying for externals to try to contribute code, but the drivers are good.

deleted 4y ago

[deleted]