6 ms·

Mesh Shaders and Meshlet Culling in Metal 3

48 points by ibobev 4y ago

watersb 4y ago
I am not a graphics programmer, but this procedural geometry reminds me of "tessellation" from ten years ago.
- SG- 4y ago
  Tessellation has been in metal since it’s release or close to I believe.
  - revolvingcur 4y ago
    Tessellation debuted in Metal in 2016, so yeah, relatively shortly after the first release on iOS (2014) and macOS (2015).
- atq2119 4y ago
  There are definitely similarities if you zoom out far enough. The object shader (task shader in Vulkan) basically maps to the tess control shader / hull shader, and the mesh shader maps to the tess eval / domain shader. One big difference is that tessellation is fixed function and doesn't allow you to control the topology / primitive connectivity within each patch. Another big difference is that the mesh shader runs as a workgroup, which allows things like parallel geometry decompression where all threads of a mesh shader workgroup collaborate in the decompression.
apatheticonion 4y ago
Imagine if Apple would just add Vulkan support to MacOS - I might actually be able to use my powerful graphics hardware for something
- soup10 4y ago
  the modern graphics API landscape is a disaster. they killed opengl es, they basically killed opengl. Apple is trying to lock everyone in with metal, khronos group is trying to get everyone to move to vulkan, microsoft is doing the same with d3d12. The only people winning here are the major engines and AAA studios who have the scale to deal with all the platform complexity and benefit from it because it creates a moat from competitors. There's an argument for regulation here because the market is dysfunctional but I don't think we'll ever see the day where politicians understand complex technology issues enough to fix it.
  - pjmlp 4y ago
    The graphic API landscape is how it always has been, specially on game consoles, really OpenGL only had a shot thanks to id Software, and even Carmack changed his mind later. > Speaking to bit-tech for a forthcoming Custom PC feature about the future of OpenGL in PC gaming, Carmack said 'I actually think that Direct3D is a rather better API today.' He also added that 'Microsoft had the courage to continue making significant incompatible changes to improve the API, while OpenGL has been held back by compatibility concerns. Direct3D handles multi-threading better, and newer versions manage state better.' https://www.bit-tech.net/news/gaming/pc/carmack-directx-better-opengl/1/ https://www.bit-tech.net/news/gaming/pc/carmack-directx-bett... On the subject of this thread, it was also DirectX 12 Ultimate that introduced Mesh Shaders. Meanwhile had it not been for AMD Mantle, Khronos would still be wondering how OpenGL vNext would be like, Vulkan is already reaching the same level of OpenGL extension spaghetti, and the latest extension acknowledges pipelines were a design error that failed to gain adoption. "On the application side, many developers considering or implementing Vulkan and similar APIs found them unable to efficiently support important use cases which were easily supportable in earlier APIs. This has not been simply a matter of developers being stuck in an old way of thinking or unwilling to "rein in" an unnecessarily large number of state combinations, but a reflection of the reality that the natural design patterns of the most demanding class of applications which use graphics APIs — video games — are inherently and deeply dependent on the very "dynamism" that pipelines set out to constrain." https://github.com/KhronosGroup/Vulkan-Docs/blob/main/proposals/VK_EXT_shader_object.adoc https://github.com/KhronosGroup/Vulkan-Docs/blob/main/propos... Design by committee at its best.
- pjmlp 4y ago
  Imagine if you would use one of the middleware engines that support Metal, instead of complaining.
  - Animats 4y ago
    I use WGPU, which tries to abstract over Metal, Vulkan, and Direct-X, with reasonable success. But it adds additional concurrency restrictions, which the developers are struggling to remove. It's a huge headache building and maintaining compatibility layers.
    - pjmlp 4y ago
      Not all middleware is born equal. Just like the extension spaghetti from OpenGL and Vulkan is an "headache building and maintaining compatibility layers" of its own doing. Khronos losing steam on improving GLSL, rather backing on Microsoft and HLSL for future shading language improvements. And then there are the Web versions that are also their own set of issues, and what better way to cater for adoption than introduce yet another shading language.
- flohofwoe 4y ago
  You can use Vulkan via MoltenVk, personally I take Metal over Vulkan any day though (the Metal API is a lot more ergonomic - dare I say 'enjoyable' - than Vulkan).
  - pjmlp 4y ago
    With a modern shading language. At Vulkanised they stated that eventually would rather adopt whatever Microsoft is doing with HLSL than trying to improve GLSL (no resources for that currently).
    - flohofwoe 4y ago
      The one good thing coming out of Vulkan is the whole software ecosystem which evolved around SPIRV (so that details like shading languages don't matter all that much anymore).
  - apatheticonion 4y ago
    You're probably right about the semantics of the API from the perspective of a developer - however from the perspective of a consumer, the best API in the world means nothing if I can't run any games on my machine. That said, it's not a binary choice - MacOS could support both, they just choose not to.
    - flohofwoe 4y ago
      A custom 3D API wrapper around Metal and Vulkan is quite trivial compared to the other challenges of porting a game to another platform though (and the main reason why "macOS got no games" isn't even technical, but it's usually just not worthwhile from a business perspective). And MoltenVk definitely does a 'good enough job' if all you have is a Vulkan-based renderer (but most cross-platform games already have a wrapper over platform-specific APIs, including the 3D API). ...also, if Apple would provide a builtin Vulkan API, it would also (most likely) just be a wrapper over Metal, just as they did with OpenGL, so there wouldn't be much of an advantage over a 3rd party solution like MoltenVk.
- MrBuddyCasino 4y ago
  I don't think a missing Vulkan driver is the big obstacle here, Unreal Engine supports Metal since some time - the Intel laptop graphics hardware was just too slow, thus games only got released on iPhones and iPads. Now that Apple Silicon macs have a reasonably fast GPU, maybe we'll finally see some releases? Also, a future viable route could be via Asahi Linux and Wine/Proton, but thats a different can of worms.
deleted 4y ago
[deleted]
Animats 4y ago
Whether this is a win is an difficult question. Rend3 recently put in meshlets, and performance got worse for my use case. It's a way to reduce draw work at some cost in GPU memory. But in a metaverse application, where you have user-created content, you have less instancing, and tend to run out of GPU memory before running out of rendering power. Like mipmapping, trying to do level of detail at the GPU level only results in less rendering load, not less GPU memory consumption. There's the question of whether you want to do occlusion culling down at that level. Yes, it cuts GPU load way down when you're in a windowless room. Then go outside and the load goes way up. If you have windows in your buildings, the occlusion culling doesn't help as much. This trick improves average GPU load for a scene, but not worst-case GPU load, which is what matters for playability. When I tried this on an older GPU with slow compute shaders, the frame rate got worse by a factor of 3, and the GPU memory consumption increased. If you want to do occlusion culling, there's an argument of doing it by pre-computing occluded zones for a level. Unreal Engine does that sort of thing. Analyze once at build time, not once per frame. That's a problem with pushing work down to the GPU level. You have to do it on every frame, which may be a lose.
- Pulcinella 4y ago
  Like mipmapping, trying to do level of detail at the GPU level only results in less rendering load, not less GPU memory consumption. Though on Apple’s ARM devices (and consoles) memory is shared between the CPU and GPU. I am unsure if making some asset available to the GPU to render (e.g. a texture) requires making an extra copy in ram just for the GPU, but if it is not required and you can just have one copy, then shouldn’t doing LODs GPU vs CPU require roughly the same amount of memory?
  - Animats 4y ago
    If you don't need a higher LOD for a texture yet, it doesn't need to be in memory at all. It can be out in a file or cache. Big-world networked games have something that looks like a cache hierarchy. There's content in the GPU, content in main memory, decompressed content in a cache on SSD, content that hasn't been decompressed yet, and content that's still on a server somewhere. Much of this data wrangling is done at a slower time scale than the frame rate. So you don't want to push all that down to the GPU level. The heavy machinery behind this is much of building a metaverse is really about. If you don't do all this, the result is low-rez, like Decentraland and Horizon. If you do it right, things look pretty good.

48 points by ibobev 4y ago

watersb 4y ago

I am not a graphics programmer, but this procedural geometry reminds me of "tessellation" from ten years ago.

SG- 4y ago
Tessellation has been in metal since it’s release or close to I believe.
- revolvingcur 4y ago
  Tessellation debuted in Metal in 2016, so yeah, relatively shortly after the first release on iOS (2014) and macOS (2015).
atq2119 4y ago
There are definitely similarities if you zoom out far enough. The object shader (task shader in Vulkan) basically maps to the tess control shader / hull shader, and the mesh shader maps to the tess eval / domain shader. One big difference is that tessellation is fixed function and doesn't allow you to control the topology / primitive connectivity within each patch. Another big difference is that the mesh shader runs as a workgroup, which allows things like parallel geometry decompression where all threads of a mesh shader workgroup collaborate in the decompression.

apatheticonion 4y ago

Imagine if Apple would just add Vulkan support to MacOS - I might actually be able to use my powerful graphics hardware for something

soup10 4y ago
the modern graphics API landscape is a disaster. they killed opengl es, they basically killed opengl. Apple is trying to lock everyone in with metal, khronos group is trying to get everyone to move to vulkan, microsoft is doing the same with d3d12. The only people winning here are the major engines and AAA studios who have the scale to deal with all the platform complexity and benefit from it because it creates a moat from competitors. There's an argument for regulation here because the market is dysfunctional but I don't think we'll ever see the day where politicians understand complex technology issues enough to fix it.
- pjmlp 4y ago
  The graphic API landscape is how it always has been, specially on game consoles, really OpenGL only had a shot thanks to id Software, and even Carmack changed his mind later. > Speaking to bit-tech for a forthcoming Custom PC feature about the future of OpenGL in PC gaming, Carmack said 'I actually think that Direct3D is a rather better API today.' He also added that 'Microsoft had the courage to continue making significant incompatible changes to improve the API, while OpenGL has been held back by compatibility concerns. Direct3D handles multi-threading better, and newer versions manage state better.' https://www.bit-tech.net/news/gaming/pc/carmack-directx-better-opengl/1/ https://www.bit-tech.net/news/gaming/pc/carmack-directx-bett... On the subject of this thread, it was also DirectX 12 Ultimate that introduced Mesh Shaders. Meanwhile had it not been for AMD Mantle, Khronos would still be wondering how OpenGL vNext would be like, Vulkan is already reaching the same level of OpenGL extension spaghetti, and the latest extension acknowledges pipelines were a design error that failed to gain adoption. "On the application side, many developers considering or implementing Vulkan and similar APIs found them unable to efficiently support important use cases which were easily supportable in earlier APIs. This has not been simply a matter of developers being stuck in an old way of thinking or unwilling to "rein in" an unnecessarily large number of state combinations, but a reflection of the reality that the natural design patterns of the most demanding class of applications which use graphics APIs — video games — are inherently and deeply dependent on the very "dynamism" that pipelines set out to constrain." https://github.com/KhronosGroup/Vulkan-Docs/blob/main/proposals/VK_EXT_shader_object.adoc https://github.com/KhronosGroup/Vulkan-Docs/blob/main/propos... Design by committee at its best.
pjmlp 4y ago
Imagine if you would use one of the middleware engines that support Metal, instead of complaining.
- Animats 4y ago
  I use WGPU, which tries to abstract over Metal, Vulkan, and Direct-X, with reasonable success. But it adds additional concurrency restrictions, which the developers are struggling to remove. It's a huge headache building and maintaining compatibility layers.
  - pjmlp 4y ago
    Not all middleware is born equal. Just like the extension spaghetti from OpenGL and Vulkan is an "headache building and maintaining compatibility layers" of its own doing. Khronos losing steam on improving GLSL, rather backing on Microsoft and HLSL for future shading language improvements. And then there are the Web versions that are also their own set of issues, and what better way to cater for adoption than introduce yet another shading language.
flohofwoe 4y ago
You can use Vulkan via MoltenVk, personally I take Metal over Vulkan any day though (the Metal API is a lot more ergonomic - dare I say 'enjoyable' - than Vulkan).
- pjmlp 4y ago
  With a modern shading language. At Vulkanised they stated that eventually would rather adopt whatever Microsoft is doing with HLSL than trying to improve GLSL (no resources for that currently).
  - flohofwoe 4y ago
    The one good thing coming out of Vulkan is the whole software ecosystem which evolved around SPIRV (so that details like shading languages don't matter all that much anymore).
- apatheticonion 4y ago
  You're probably right about the semantics of the API from the perspective of a developer - however from the perspective of a consumer, the best API in the world means nothing if I can't run any games on my machine. That said, it's not a binary choice - MacOS could support both, they just choose not to.
  - flohofwoe 4y ago
    A custom 3D API wrapper around Metal and Vulkan is quite trivial compared to the other challenges of porting a game to another platform though (and the main reason why "macOS got no games" isn't even technical, but it's usually just not worthwhile from a business perspective). And MoltenVk definitely does a 'good enough job' if all you have is a Vulkan-based renderer (but most cross-platform games already have a wrapper over platform-specific APIs, including the 3D API). ...also, if Apple would provide a builtin Vulkan API, it would also (most likely) just be a wrapper over Metal, just as they did with OpenGL, so there wouldn't be much of an advantage over a 3rd party solution like MoltenVk.
MrBuddyCasino 4y ago
I don't think a missing Vulkan driver is the big obstacle here, Unreal Engine supports Metal since some time - the Intel laptop graphics hardware was just too slow, thus games only got released on iPhones and iPads. Now that Apple Silicon macs have a reasonably fast GPU, maybe we'll finally see some releases? Also, a future viable route could be via Asahi Linux and Wine/Proton, but thats a different can of worms.

deleted 4y ago

[deleted]

Animats 4y ago

Whether this is a win is an difficult question. Rend3 recently put in meshlets, and performance got worse for my use case. It's a way to reduce draw work at some cost in GPU memory. But in a metaverse application, where you have user-created content, you have less instancing, and tend to run out of GPU memory before running out of rendering power. Like mipmapping, trying to do level of detail at the GPU level only results in less rendering load, not less GPU memory consumption. There's the question of whether you want to do occlusion culling down at that level. Yes, it cuts GPU load way down when you're in a windowless room. Then go outside and the load goes way up. If you have windows in your buildings, the occlusion culling doesn't help as much. This trick improves average GPU load for a scene, but not worst-case GPU load, which is what matters for playability. When I tried this on an older GPU with slow compute shaders, the frame rate got worse by a factor of 3, and the GPU memory consumption increased. If you want to do occlusion culling, there's an argument of doing it by pre-computing occluded zones for a level. Unreal Engine does that sort of thing. Analyze once at build time, not once per frame. That's a problem with pushing work down to the GPU level. You have to do it on every frame, which may be a lose.

Pulcinella 4y ago
Like mipmapping, trying to do level of detail at the GPU level only results in less rendering load, not less GPU memory consumption. Though on Apple’s ARM devices (and consoles) memory is shared between the CPU and GPU. I am unsure if making some asset available to the GPU to render (e.g. a texture) requires making an extra copy in ram just for the GPU, but if it is not required and you can just have one copy, then shouldn’t doing LODs GPU vs CPU require roughly the same amount of memory?
- Animats 4y ago
  If you don't need a higher LOD for a texture yet, it doesn't need to be in memory at all. It can be out in a file or cache. Big-world networked games have something that looks like a cache hierarchy. There's content in the GPU, content in main memory, decompressed content in a cache on SSD, content that hasn't been decompressed yet, and content that's still on a server somewhere. Much of this data wrangling is done at a slower time scale than the frame rate. So you don't want to push all that down to the GPU level. The heavy machinery behind this is much of building a metaverse is really about. If you don't do all this, the result is low-rez, like Decentraland and Horizon. If you do it right, things look pretty good.