4 ms·

When looking at a binary blob for the driver, the person writing the docs of what the driver does sees "jump $addr1" but they don't know what's at the other end

When looking at a binary blob for the driver, the person writing the docs of what the driver does sees "jump $addr1" but they don't know what's at the other end. To write the specification for the clean room reimplementation, they need to read the destination assembly and document that. This provides documentation on what is at the other end of the address and what it does, and what the layout of each parameter is. That means they will hopefully avoid having to recurse and understand all of the binary code to write the specification.

averne_ 4y ago
That's not exactly correct. This is register maps for the 3d engine (also called class), what you describe would be closer to the shader ISA. In driver code you'll see them building command buffers that set registers in those classes to certain values. It could be the RGBA values of the clear color, or a virtual address in the GPU space. This documents the names of these registers. This makes reverse engineering somewhat easier as you don't really have to guess anymore. But in most cases it was pretty clear from the start, and for some generations those were already well documented by open source efforts. One of best known gens is probably Maxwell since it was used in the Nintendo Switch, see for instance [1] (or code in yuzu/Ryujinx) which is the equivalent of those headers NV published. However this isn't a very big step in documenting their GPUs. The exact functions of those registers aren't explained, but most importantly the shader ISA isn't documented at all, which is essential to build a good open-source driver. Source: I have reverse engineered some driver code for Maxwell (and used similar headers to write drivers for nvdec and nvjpg). [1] https://github.com/devkitPro/deko3d/blob/master/source/maxwell/engine_3d.def https://github.com/devkitPro/deko3d/blob/master/source/maxwe...
- digdugdirk 4y ago
  You say this isn't a very big step - out of curiosity, how much work would have been involved in getting this release open sourced? And what might be preventing them from releasing the shader ISA? Specifically curious if this inherently would "let the cat out of the bag" about the specifics of their chip design, for example.
  - averne_ 4y ago
    > how much work would have been involved in getting this release open sourced Close to no actual effort (the headers are autogenerated). However there was probably a lot of work behind the scenes with their legal team/whatever to clear the release. About the shader ISA, I wish I knew. It's certain that the documentation exists, because they provide it to some developers (I've been told the Maxwell ISA docs are part of the Nintendo Switch SDK), so it's not like they have to write it from scratch. And AMD provides full docs about it [1] (not sure about Intel), so I don't see how it could provide a significant edge over their competition. Maybe raytracing instructions? But for a motivated reverse engineer this stuff isn't impossible to figure out. I think it's down to company culture and inertia. [1] https://developer.amd.com/wp-content/resources/RDNA2_Shader_ISA_November2020.pdf https://developer.amd.com/wp-content/resources/RDNA2_Shader_...
    - monocasa 4y ago
      > not sure about Intel I don't know about Arc, but Intel has historically probably been the most open with their GPU docs. AMD's is (while laudable, particularly compared to Nvidia) unfortunately a bit of a autogenned dump without context. Where as Intel's is closer to what I expect out of decent SoC documentation; it'll be about half prose explaining the why(!!), and half the autogenned tables of registers and opcode formats.
      - ahartmetz 4y ago
        AMD's approach seems to be documentation in the style you describe plus employed FOSS developers who get access to internal people, simulators, possibly even source code. Makes it annoying for externals to try to contribute code, but the drivers are good.
  - salawat 4y ago
    It isn't really, and doesn't really do squat for nouveau. Nouveau has been thwarted by the secretful FALCON units, and Nvidia's decision to lock power management and clock management behind high secure mode. See, in order to make those calls, the command buffer is submitted along with a hash value, plus the firmware itself iirc. Been a while. That firmware is signed with a private key known only to Nvidia, and they don't sign third party firmware. So nouveau can write all the firmware they want, but they can't actually get access to any functionality gated behind high-secure mode. There are ways*, but unfortunately all of them run afoul of the spirit of DMCA. Firmware is the enemy.

averne_ 4y ago

That's not exactly correct. This is register maps for the 3d engine (also called class), what you describe would be closer to the shader ISA. In driver code you'll see them building command buffers that set registers in those classes to certain values. It could be the RGBA values of the clear color, or a virtual address in the GPU space. This documents the names of these registers. This makes reverse engineering somewhat easier as you don't really have to guess anymore. But in most cases it was pretty clear from the start, and for some generations those were already well documented by open source efforts. One of best known gens is probably Maxwell since it was used in the Nintendo Switch, see for instance [1] (or code in yuzu/Ryujinx) which is the equivalent of those headers NV published. However this isn't a very big step in documenting their GPUs. The exact functions of those registers aren't explained, but most importantly the shader ISA isn't documented at all, which is essential to build a good open-source driver. Source: I have reverse engineered some driver code for Maxwell (and used similar headers to write drivers for nvdec and nvjpg). [1] https://github.com/devkitPro/deko3d/blob/master/source/maxwell/engine_3d.def https://github.com/devkitPro/deko3d/blob/master/source/maxwe...

digdugdirk 4y ago
You say this isn't a very big step - out of curiosity, how much work would have been involved in getting this release open sourced? And what might be preventing them from releasing the shader ISA? Specifically curious if this inherently would "let the cat out of the bag" about the specifics of their chip design, for example.
- averne_ 4y ago
  > how much work would have been involved in getting this release open sourced Close to no actual effort (the headers are autogenerated). However there was probably a lot of work behind the scenes with their legal team/whatever to clear the release. About the shader ISA, I wish I knew. It's certain that the documentation exists, because they provide it to some developers (I've been told the Maxwell ISA docs are part of the Nintendo Switch SDK), so it's not like they have to write it from scratch. And AMD provides full docs about it [1] (not sure about Intel), so I don't see how it could provide a significant edge over their competition. Maybe raytracing instructions? But for a motivated reverse engineer this stuff isn't impossible to figure out. I think it's down to company culture and inertia. [1] https://developer.amd.com/wp-content/resources/RDNA2_Shader_ISA_November2020.pdf https://developer.amd.com/wp-content/resources/RDNA2_Shader_...
  - monocasa 4y ago
    > not sure about Intel I don't know about Arc, but Intel has historically probably been the most open with their GPU docs. AMD's is (while laudable, particularly compared to Nvidia) unfortunately a bit of a autogenned dump without context. Where as Intel's is closer to what I expect out of decent SoC documentation; it'll be about half prose explaining the why(!!), and half the autogenned tables of registers and opcode formats.
    - ahartmetz 4y ago
      AMD's approach seems to be documentation in the style you describe plus employed FOSS developers who get access to internal people, simulators, possibly even source code. Makes it annoying for externals to try to contribute code, but the drivers are good.
- salawat 4y ago
  It isn't really, and doesn't really do squat for nouveau. Nouveau has been thwarted by the secretful FALCON units, and Nvidia's decision to lock power management and clock management behind high secure mode. See, in order to make those calls, the command buffer is submitted along with a hash value, plus the firmware itself iirc. Been a while. That firmware is signed with a private key known only to Nvidia, and they don't sign third party firmware. So nouveau can write all the firmware they want, but they can't actually get access to any functionality gated behind high-secure mode. There are ways*, but unfortunately all of them run afoul of the spirit of DMCA. Firmware is the enemy.