8 ms·

UE5 Nanite in WebGPU

394 points by vouwfietsman 2y ago

blurbleblurble 2y ago
The camera controls on my phone are very hard to get down
- aaroninsf 2y ago
  Browser/touchpad also :)
- bob1029 2y ago
  Couldn't we use something like this to provide a more intuitive experience for the mobile web targets? https://developer.mozilla.org/en-US/docs/Web/API/Device_orientation_events/Detecting_device_orientation https://developer.mozilla.org/en-US/docs/Web/API/Device_orie...
jsheard 2y ago
It's cool that it kind of works, but they had to make some nasty compromises to get around WebGPUs lack of 64 bit atomics. Hopefully that will be added as an optional extension at some point, hardware support is almost ubiquitous on desktop-class hardware at least (AMD and Nvidia have had it forever but Apple has only had it since the M3).
- throwaway17_17 2y ago
  What is the use case for atomics in the rasterizer? I can’t figure out what the atomic operations do inside the rendering pipeline. I looked at the GitHub, but couldn’t find the place the atomic were hoped for.
  - hrydgard 2y ago
    Pack Z and 32-bit color together into a 64-bit integer, then do an atomic min (or max with reversed Z) to effectively do a Z-query and a write really, really fast.
    - jsheard 2y ago
      Nanite writes out the ID of the primitive at that pixel rather than the color, but otherwise yeah that's the idea. After rasterization is done a separate pass uses that ID to fetch the vertex data again and reconstruct all of the material parameters, which can be freely written out without atomics since there's exactly one thread per pixel at that point.
  - neomantra 2y ago
    Visibility buffer needing atomics is noted briefly in the long README. Link to discussion detailing it: https://github.com/Scthe/nanite-webgpu/issues/1 https://github.com/Scthe/nanite-webgpu/issues/1
  - jsheard 2y ago
    With traditional hardware rasterization there are specialized hardware blocks which handle atomically updating the framebuffer to whichever sample is currently the closest to the camera, and discarding anything behind that. Nanite does software rasterization instead, and one of their insights was figuring out a practical way to cram all of the data needed for each pixel into just 64 bits (depth in the high bits and everything else in the low bits) which allows them to do efficient depth sorting using min/max atomics from a compute shader instead. The 64 bits are crucial though, that's the absolute bare minimum useful amount of data per pixel so you really need 64 bit atomics. Nanite doesn't even try to work without them. To kind of get it working with 32 bit atomics this demo is reducing depth to just 16 bits (not enough to avoid artifacts) and only encoding a normal vector into the other 16 bits, which is why the compute rasterized pixels are untextured. There just aren't enough bits to store any more material parameters or a primitive ID, the latter being how Nanite does it.
    - 01HNNWZ0MV43FF 2y ago
      Software rasterization, in a GPGPU compute shader, in hardware.
- my123 2y ago
  Since the M2
  - jsheard 2y ago
    Right you are, 64 bit atomics were added with the Apple8 GPU but only in M-series chips (M2 and up) and then the Apple9 GPU made it universal (A17 Pro and up). https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf https://developer.apple.com/metal/Metal-Feature-Set-Tables.p...
moralestapia 2y ago
Outstanding work. Also, thanks for proving actual demos of the tech. I get 60-120fps on my MBP which is phenomenal given the amount of triangles in the scene.
TaylorAlexander 2y ago
It says my iPhone 12 Pro Max doesn’t have WebGPU, but I enabled it in experimental features and another website[1] with WebGPU demos now works. Has anyone gotten this working on iPhone? Would be nice if the web app gave more info on what failed. [1] https://webgpu.github.io/webgpu-samples/?sample=texturedCube https://webgpu.github.io/webgpu-samples/?sample=texturedCube
- KMnO4 2y ago
  I enabled WebGPU in Safari but I'm seeing a bunch of shader errors. WebGPU error [init][validation]: 6 errors generated while compiling the shader: 50:22: unresolved call target 'pack4x8snorm' 50:9: cannot bitcast from 'â¥' to 'f32' 54:10: unresolved call target 'unpack4x8snorm' 59:22: unresolved call target 'pack4x8unorm' 59:9: cannot bitcast from 'â¥' to 'f32' 63:9: unresolved call target 'unpack4x8unorm'
  - deleted 2y ago
    [deleted]
- nox101 2y ago
  WebGPU support is not finished in Safari which is why it's still experimental.
soulofmischief 2y ago
Is the demo using user agent strings to determine compatibility? That's not good, and feature compatibility should be determined on a case-by-case basis by simply attempting to detect/use the specific feature. I am on Chromium, not Chrome, and use WebGPU all the time, but the demos tell me to use Chrome, which I cannot do ethically. Would love to try the demos out, this looks like a lot of hard work!
- robin_reala 2y ago
  Don’t think so. I’m on a Firefox that has experimental WebGPU support enabled, and it fails with shader compilation errors rather than any message.
- drusepth 2y ago
  If this is the case, I imagine it'd be pretty easy to spoof your UA and see the demo, even from Chromium.
- gpm 2y ago
  > and use WebGPU all the time I'm curious, what for?
  - soulofmischief 2y ago
    I've used it to build and/or run various machine learning models for text generation, speech recognition, image generation, depth estimation, etc. in the browser, in support of an agentic system I've been building out. Lots of future possibilities as well once support is more ubiquitous!
    - password4321 2y ago
      Your ideas are intriguing to me and I wish to subscribe to your newsletter.
      - soulofmischief 2y ago
        I appreciate that, anything in particular catch your interest?
SaintSeiya 2y ago
Honest question: It is calim that software rasterizer is faster than hardware one. Can someone explain me why? isn't the purpose of the GPU to accelerate rasterization itself? Unless is a recent algorithm or the "software rasterizer" is actually running on the GPU and not the CPU I don't see how
- janito 2y ago
  I'm also curious. From what I could read in the repository's references, I think that the problem is that the GPU is bad at rasterizing small triangles. Apparently each triangle in the fixed function pipeline generates a batch of pixels to render (16 in one of the slides I saw), so if the triangle covers only one or two pixels, all others in the batch are wasted. I speculate that the idea is to then detect these small triangles and draw them quickly using less pixel shaders (still on the GPU, but without using the graphics specific fixed functions), but I'm honestly not sure I understand what's happening.
- NotGMan 2y ago
  I'm a bit out of the GPU game but so this might be slightly wrong in some places: the issue is in small triangles because you end up paying a huge cost. GPUs ALWAYS shade in 2x2 blocks of pixels, not 1x1 pixels. So if you have a very small triangle (small as in how many pixels on the screen it covers) that covers 1 pixel you will still pay the price of a 2x2 block (4 pixels instead of 1), so you just wasted 300% of your performance. Nanite auto-picks the best triangle to minimize this and probably many more perf metrics that I have no idea about. So even if you do it in software the point is that if you can get rid of that 2x2 block penalty as much as possible you could be faster than GPU doing 2x2 blocks in hardware since pixel shaders can be very expensive. This issue gets worse the larger the rendering resolution is. Nanite then picks larger triangles instead of those tiny 1-pixel ones since those are too small to give any visual fidelity anyway. Nanite is also not used for large triangles since those are more efficient to do in hardware.
  - kllrnohj 2y ago
    > So even if you do it in software the point is that if you can get rid of that 2x2 block penalty as much as possible you could be faster than GPU doing 2x2 blocks in hardware since pixel shaders can be very expensive. Of course the obvious problem with that is if you don't have most of the screen covered in such small triangles then you're paying a large cost for nanite vs traditional means.
moffkalast 2y ago
> No WebGPU available. Please use Chrome. Getting that on Chromium, lol.
- gpm 2y ago
  I'm getting that in google chrome proper, but what completes the joke is that in firefox I just get a blank page without the message to use chrome. Edit: WebGPU in chrome is behind a flag on linux: https://github.com/gpuweb/gpuweb/wiki/Implementation-Status#implementation-status https://github.com/gpuweb/gpuweb/wiki/Implementation-Status#...
Animats 2y ago
Oh, nice. Third party implementations of Nanite playback. Nanite is a very clever representation of graphics meshes. They're directed acyclic graphs rather than trees. Repetition is a link, not a copy. It's recursive; meshes can share submeshes, which in turn can share submeshes, all the way down. It's also set up for within-mesh level of detail support, so the submeshes drop out when they're small enough. So you can have repetitive content of very large size with a finite amount of data and fast rendering times. The insight is that there are only so many pixels on screen, so there's an upper bound on rendering work really needed. There's a really good SIGGRAPH video on this from someone at Epic. Current GPU designs are a mismatch for Nanite, Some new hardware operations are needed to do more of this in the GPU, where it belongs. Whether that will happen, with NVidia distracted by the AI market, is a good question. The scene needs a lot of instancing for this to pay off. Unreal Engine demos show such things as a hall of identical statues. If each statue was different, Nanite would help far less. So it works best for projects where a limited number of objects are reused to create large areas of content. That's the case for most AAA titles. Watch a video of Cyberpunk 2077, and look for railings and trash heaps. You'll see the same ones over and over in totally different contexts. Making a nanite mesh is complicated, with a lot of internal offsets for linking, and so far only Unreal Engine's editor does it. With playback now open source, someone will probably do that. Those internal offsets in the format present an attack surface which probably can be exploited with carefully crafted bad content, like hostile Microsoft Word .doc files.
- turtledragonfly 2y ago
  I think the SIGGRAPH talk you referred to is: "A Deep Dive into Nanite Virtualized Geometry" (https://www.youtube.com/watch?v=eviSykqSUUw https://www.youtube.com/watch?v=eviSykqSUUw) There's also this short high-level intro (2.5 min) that I thought was decent: "What is virtualized micropolygon geometry? An explainer on Nanite" (https://www.youtube.com/watch?v=-50MJf7hyOw https://www.youtube.com/watch?v=-50MJf7hyOw)
- Jasper_ 2y ago
  > Repetition is a link, not a copy. It's recursive; meshes can share submeshes, which in turn can share submeshes, all the way down. While it does construct a DAG to perform the graph cut, the final data set on disk is just a flat list of clusters for consideration, along with their cutoffs for inclusion/rejection. There seems to be a considerable misunderstanding of what the DAG is used for, and how it's constructed. It's constructed dynamically based on the vertex data, and doesn't have anything to do with how the artist constructed submeshes and things, nor does "repetition become a link". > The scene needs a lot of instancing for this to pay off. Unreal Engine demos show such things as a hall of identical statues. If each statue was different, Nanite would help far less. What makes you say this? The graph cut is different for each instance of the object, so they can't use traditional instancing, and I don't even see how it could help.
spintin 2y ago
[dead]
devit 2y ago
Name and description are very confusing and a trademark violation since despite the claims it seems to be completely unrelated to actual Nanite in UE5, just an implementation of something similar by a person unaffiliated with UE5. There is also Bevy's Virtual Geometry that provides similar functionality and is probably much more useful since it's written in Rust and integrated with a game engine: https://jms55.github.io/posts/2024-06-09-virtual-geometry-bevy-0-14/ https://jms55.github.io/posts/2024-06-09-virtual-geometry-be...
- KMnO4 2y ago
  I don’t think it’s really an issue. It’s clear from the readme that it’s an implementation. If I made an “implementation of OpenAI’s GPT-3 in JS” you would understand that to mean I took the architecture from the whitepaper and reimplemented it.
  - cubefox 2y ago
    The technique is commonly called virtual geometry or virtualized geometry, or dynamic LOD in research papers. Really no need to reuse the name of a specific implementation.
smartmic 2y ago
Wow, I can't remember the last time I read a project summary with so much jargon - I literally didn't understand anything: > UE5's Nanite implementation using WebGPU. Includes the meshlet LOD hierarchy, software rasterizer and billboard impostors. Culling on both per-instance and per-meshlet basis.
- goodcjw2 2y ago
  Guess this really shows how much domain specific knowledge in the Computer Graphics... Yet still, this post is now ranked top 1 on HN.
- deleted 2y ago
  [deleted]
- bogwog 2y ago
  UE5 Nanite -> https://dev.epicgames.com/documentation/en-us/unreal-engine/nanite-virtualized-geometry-in-unreal-engine https://dev.epicgames.com/documentation/en-us/unreal-engine/... WebGPU -> https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API Meshlet -> https://developer.nvidia.com/blog/introduction-turing-mesh-shaders/ https://developer.nvidia.com/blog/introduction-turing-mesh-s... LOD -> https://en.wikipedia.org/wiki/Level_of_detail_(computer_graphics) https://en.wikipedia.org/wiki/Level_of_detail_(computer_grap... Software rasterizer -> https://en.wikipedia.org/wiki/Rasterisation https://en.wikipedia.org/wiki/Rasterisation ("software" means it runs on the CPU instead of GPU) Billboard imposters -> https://www.alanzucconi.com/2018/08/25/shader-showcase-saturday-7/ https://www.alanzucconi.com/2018/08/25/shader-showcase-satur... Culling -> https://en.wikipedia.org/wiki/Hidden-surface_determination https://en.wikipedia.org/wiki/Hidden-surface_determination
  - nicebyte 2y ago
    > ("software" means it runs on the CPU instead of GPU) no, in this context it means that the rasterisation algorithm is implemented in a compute kernel, rather than using the fixed hw built into the gpu. so rasterization still happens on the gpu, just using programmable blocks.
theogravity 2y ago
Using latest chrome on M2 Max for the jinx demo: WebGPU error [frame][validation]: Fill size (7398781) is not a multiple of 4 bytes. - While encoding [CommandEncoder "main-frame-cmd-buffer"].ClearBuffer([Buffer "rasterize-sw"], 0, 7398781).
- stephc_int13 2y ago
  I have the same error on Windows 11, GPU is a RTX4090. Browser is Edge.
- daemonologist 2y ago
  Same (different number) on Chrome on Android (Pixel 7).
- flockonus 2y ago
  If helpful to author, on M1 no errors, can see 15+ fps at all times.
replete 2y ago
Intel Mac, Chrome and ungoogled chromium: index.web.ts:159 Uncaught (in promise) OperationError: Instance dropped in popErrorScope
eigenvalue 2y ago
Whenever I see rendered scenes like this (I.e., lots of repetitive static geometry) I imagine that annoying guy’s voice going on about “unlimited detail” from that old vaporware video. I guess nanite really did solve that problem for real, as opposed to whatever that old thing was using (I remember something about oct-trees or something).
- HappMacDonald 2y ago
  I recall those claims being made by a company called "Euclidean", from Australia I think. Online rumors suggested they might have been using octtrees, but later Euclidean videos flatly denied that.
  - raphlinus 2y ago
    It's Euclideon. And it is octtrees. My interpretation after reading a fascinating Reddit thread [1] is that these denials were misdirection. There's definitely new interest in splatting techniques (Gaussian in particular), though they've long been an alternative to triangles in the 3D world. I think it'd be fun to experiment with implementing some of that using modern compute shaders. [1]: https://www.reddit.com/r/VoxelGameDev/comments/1bz5vvy/a_small_update_on_cpu_octree_splatting_feat/ https://www.reddit.com/r/VoxelGameDev/comments/1bz5vvy/a_sma...
readyplayernull 2y ago
Will virtual geometry be integrated into GPUs some day?
- cubefox 2y ago
  Some aspects of it will likely receive hardware acceleration in the future. For example support for a standard virtual geometry mesh format, like this one proposed by AMD: https://gpuopen.com/download/publications/DGF.pdf https://gpuopen.com/download/publications/DGF.pdf
forrestthewoods 2y ago
Note: this isn't actually UE5 Nanite in WebGPU. It's a totally independent implementation of the same idea as Nanite. This technique is starting to appear in a variety of places. Nanite definitely made the idea famous, but Nanite is the name a specific implementation, not the name of the technique.
tech-no-logical 2y ago
getting the message No WebGPU available. Please use Chrome. on chrome (Version 129.0.6668.29 (Official Build) beta (64-bit)) , under windows
- Joel_Mckay 2y ago
  Look at the #enable-unsafe-webgpu flag in chrome. Turn it back off when done, as tools like noscript only block webgl tags. Cheers =3
jms55 2y ago
It's been mentioned a couple of times in this thread, but Bevy also has an implementation of Nanite's ideas (sometimes called Virtual Geometry). I'm the author of that, happy to answer questions :) As for this project, Scthe did a great job! I've been talking with them about several parts of the process, culminating in some improvements to Bevy's code based on their experience (https://github.com/bevyengine/bevy/pull/15023 https://github.com/bevyengine/bevy/pull/15023). Always happy to see more people working on this, Nanite has a ton of cool ideas.
KronisLV 2y ago
I wonder how other engines compare when it comes to LODs and similar systems. Godot has automatic LOD which seems pretty cool for what it is: https://docs.godotengine.org/en/stable/tutorials/3d/mesh_lod.html https://docs.godotengine.org/en/stable/tutorials/3d/mesh_lod... Unity also has an LOD system, though despite how popular the engine is, you have to create LOD models manually: https://docs.unity3d.com/Manual/LevelOfDetail.html https://docs.unity3d.com/Manual/LevelOfDetail.html (unless you dig through the asset store and find a plugin) I did see an interesting approach in a lesser known engine called NeoAxis: https://www.neoaxis.com/docs/html/NeoAxis_Levels.htm https://www.neoaxis.com/docs/html/NeoAxis_Levels.htm however that engine ran very poorly for me on my old RX580, although I haven't tried on my current A580. As far as I can tell, Unreal is really quite far ahead of the competition when it comes to putting lots of things on the screen, except the downside of this is that artists will be tempted to include higher quality assets in their games, bloating the install sizes quite far.
- kllrnohj 2y ago
  In theory Nanite is superior to precomputed LODs. In practice it's less clear cut as they aren't going to be as good as artist-created LODs and it's not entirely reasonable to expect them to do so. Also the performance cost is huge as Nanite/virtual geometry is a poor fit for modern GPUs. iirc peak fill rate is 1/4th or something like that as GPU rasterization works on 2x2 quads not per-pixel like shaders do.
  - jsheard 2y ago
    Rasterizing very small triangles in hardware is indeed inefficient due to the 2x2 quad tax, but one of Nanites tent-pole features is a software rasterizer which sidesteps that problem entirely. IIRC they said that for a screen entirely filled with triangles roughly the size of a pixel, their software raster ends up being about 3x faster than using the dedicated raster hardware.
    - kllrnohj 2y ago
      Yes but I'm talking the other way around. Nanite is 1/4th the performance for triangles that aren't 1-3 pixels in size, which is the majority of the time. The main selling point of Nanite is really just to reduce artist costs by avoiding manual LODs. But a high quality automatic LOD at build time may (read: almost certainly does) strike a much better balance for both current and near future hardware
hising 2y ago
I would love to see this but it wont work on Linux + Chrome even if WebGPU is enabled.
jesse__ 2y ago
> If you want to add this tech to the existing engine, I'm not a person you should be asking (I don't work in the industry). Fucking .. bravo man.
lerpgame 2y ago
[dead]
astlouis44 2y ago
Here's an actual implementation of UE5 in WebGPU, for anyone interested. Just a disclaimer that it will only work on WebGPU-enabled browser on Windows (Chrome, Edge, etc) unfortunately Mac has issues for now. Also, there is no Nanite in this demo, but it will be possible in the future. https://play.spacelancers.com/ https://play.spacelancers.com/
- mdaniel 2y ago
  I was curious what "issues" Mac has, and at least for me it didn't explode for any good reason, it puked trying to JSON.stringify() some capabilities object into localStorage which is a pretty piss-poor reason to bomb loading a webpage, IMHO
mbforbes 2y ago
Funny coincidence, was just reading through an amazing thread on the three.js form a couple days ago about a web graphics implementation of virtual geometry (nanite). webgl, 2021: https://discourse.threejs.org/t/virtually-geometric/28420 https://discourse.threejs.org/t/virtually-geometric/28420 it's closed source, but I found the discussion and description of the tradeoffs interesting
cubefox 2y ago
Here is the somewhat neglected original 2009 dissertation by Federico Ponchio, the guy who invented the dynamic mesh simplification algorithm on which Nanite is based, with lots of illustrations: https://vcg.isti.cnr.it/~ponchio/download/ponchio_phd.pdf https://vcg.isti.cnr.it/~ponchio/download/ponchio_phd.pdf (107 pages!)
nox101 2y ago
This is amazing! I will be so great when Safari and Firefox but finish their WebGPU implementations so it runs everywhere.
vladde 2y ago
Got to love in the stated goals: > I could have built this with Vulkan and Rust. None would touch it.
desdenova 2y ago
The examples don't work, though.
- swiftcoder 2y ago
  They run fine for me on Chrome/Mac
pjmlp 2y ago
On Windows it can't handle device loss. ID3D12Device::GetDeviceRemovedReason failed with DXGI_ERROR_DEVICE_HUNG (0x887A0006) - While handling unexpected error type Internal when allowed errors are (Validation|DeviceLost). at CheckHRESULTImpl (..\..\third_party\dawn\src\dawn\native\d3d\D3DError.cpp:119) at CheckAndUpdateCompletedSerials (..\..\third_party\dawn\src\dawn\native\d3d12\QueueD3D12.cpp:179) at CheckPassedSerials (..\..\third_party\dawn\src\dawn\native\ExecutionQueue.cpp:48) at Tick (..\..\third_party\dawn\src\dawn\native\Device.cpp:1730) Backend messages: \* Device removed reason: DXGI_ERROR_DEVICE_HUNG (0x887A0006)
meindnoch 2y ago
Can someone explain what Nanite is? The other day someone was saying it uses software rendering because the triangles are so small. Wtf?

394 points by vouwfietsman 2y ago

blurbleblurble 2y ago

The camera controls on my phone are very hard to get down

aaroninsf 2y ago
Browser/touchpad also :)
bob1029 2y ago
Couldn't we use something like this to provide a more intuitive experience for the mobile web targets? https://developer.mozilla.org/en-US/docs/Web/API/Device_orientation_events/Detecting_device_orientation https://developer.mozilla.org/en-US/docs/Web/API/Device_orie...

jsheard 2y ago

It's cool that it kind of works, but they had to make some nasty compromises to get around WebGPUs lack of 64 bit atomics. Hopefully that will be added as an optional extension at some point, hardware support is almost ubiquitous on desktop-class hardware at least (AMD and Nvidia have had it forever but Apple has only had it since the M3).

throwaway17_17 2y ago
What is the use case for atomics in the rasterizer? I can’t figure out what the atomic operations do inside the rendering pipeline. I looked at the GitHub, but couldn’t find the place the atomic were hoped for.
- hrydgard 2y ago
  Pack Z and 32-bit color together into a 64-bit integer, then do an atomic min (or max with reversed Z) to effectively do a Z-query and a write really, really fast.
  - jsheard 2y ago
    Nanite writes out the ID of the primitive at that pixel rather than the color, but otherwise yeah that's the idea. After rasterization is done a separate pass uses that ID to fetch the vertex data again and reconstruct all of the material parameters, which can be freely written out without atomics since there's exactly one thread per pixel at that point.
- neomantra 2y ago
  Visibility buffer needing atomics is noted briefly in the long README. Link to discussion detailing it: https://github.com/Scthe/nanite-webgpu/issues/1 https://github.com/Scthe/nanite-webgpu/issues/1
- jsheard 2y ago
  With traditional hardware rasterization there are specialized hardware blocks which handle atomically updating the framebuffer to whichever sample is currently the closest to the camera, and discarding anything behind that. Nanite does software rasterization instead, and one of their insights was figuring out a practical way to cram all of the data needed for each pixel into just 64 bits (depth in the high bits and everything else in the low bits) which allows them to do efficient depth sorting using min/max atomics from a compute shader instead. The 64 bits are crucial though, that's the absolute bare minimum useful amount of data per pixel so you really need 64 bit atomics. Nanite doesn't even try to work without them. To kind of get it working with 32 bit atomics this demo is reducing depth to just 16 bits (not enough to avoid artifacts) and only encoding a normal vector into the other 16 bits, which is why the compute rasterized pixels are untextured. There just aren't enough bits to store any more material parameters or a primitive ID, the latter being how Nanite does it.
  - 01HNNWZ0MV43FF 2y ago
    Software rasterization, in a GPGPU compute shader, in hardware.
my123 2y ago
Since the M2
- jsheard 2y ago
  Right you are, 64 bit atomics were added with the Apple8 GPU but only in M-series chips (M2 and up) and then the Apple9 GPU made it universal (A17 Pro and up). https://developer.apple.com/metal/Metal-Feature-Set-Tables.pdf https://developer.apple.com/metal/Metal-Feature-Set-Tables.p...

moralestapia 2y ago

Outstanding work. Also, thanks for proving actual demos of the tech. I get 60-120fps on my MBP which is phenomenal given the amount of triangles in the scene.

TaylorAlexander 2y ago

It says my iPhone 12 Pro Max doesn’t have WebGPU, but I enabled it in experimental features and another website[1] with WebGPU demos now works. Has anyone gotten this working on iPhone? Would be nice if the web app gave more info on what failed. [1] https://webgpu.github.io/webgpu-samples/?sample=texturedCube https://webgpu.github.io/webgpu-samples/?sample=texturedCube

KMnO4 2y ago
I enabled WebGPU in Safari but I'm seeing a bunch of shader errors. WebGPU error [init][validation]: 6 errors generated while compiling the shader: 50:22: unresolved call target 'pack4x8snorm' 50:9: cannot bitcast from 'â¥' to 'f32' 54:10: unresolved call target 'unpack4x8snorm' 59:22: unresolved call target 'pack4x8unorm' 59:9: cannot bitcast from 'â¥' to 'f32' 63:9: unresolved call target 'unpack4x8unorm'
- deleted 2y ago
  [deleted]
nox101 2y ago
WebGPU support is not finished in Safari which is why it's still experimental.

soulofmischief 2y ago

Is the demo using user agent strings to determine compatibility? That's not good, and feature compatibility should be determined on a case-by-case basis by simply attempting to detect/use the specific feature. I am on Chromium, not Chrome, and use WebGPU all the time, but the demos tell me to use Chrome, which I cannot do ethically. Would love to try the demos out, this looks like a lot of hard work!

robin_reala 2y ago
Don’t think so. I’m on a Firefox that has experimental WebGPU support enabled, and it fails with shader compilation errors rather than any message.
drusepth 2y ago
If this is the case, I imagine it'd be pretty easy to spoof your UA and see the demo, even from Chromium.
gpm 2y ago
> and use WebGPU all the time I'm curious, what for?
- soulofmischief 2y ago
  I've used it to build and/or run various machine learning models for text generation, speech recognition, image generation, depth estimation, etc. in the browser, in support of an agentic system I've been building out. Lots of future possibilities as well once support is more ubiquitous!
  - password4321 2y ago
    Your ideas are intriguing to me and I wish to subscribe to your newsletter.
    - soulofmischief 2y ago
      I appreciate that, anything in particular catch your interest?

SaintSeiya 2y ago

Honest question: It is calim that software rasterizer is faster than hardware one. Can someone explain me why? isn't the purpose of the GPU to accelerate rasterization itself? Unless is a recent algorithm or the "software rasterizer" is actually running on the GPU and not the CPU I don't see how

janito 2y ago
I'm also curious. From what I could read in the repository's references, I think that the problem is that the GPU is bad at rasterizing small triangles. Apparently each triangle in the fixed function pipeline generates a batch of pixels to render (16 in one of the slides I saw), so if the triangle covers only one or two pixels, all others in the batch are wasted. I speculate that the idea is to then detect these small triangles and draw them quickly using less pixel shaders (still on the GPU, but without using the graphics specific fixed functions), but I'm honestly not sure I understand what's happening.
NotGMan 2y ago
I'm a bit out of the GPU game but so this might be slightly wrong in some places: the issue is in small triangles because you end up paying a huge cost. GPUs ALWAYS shade in 2x2 blocks of pixels, not 1x1 pixels. So if you have a very small triangle (small as in how many pixels on the screen it covers) that covers 1 pixel you will still pay the price of a 2x2 block (4 pixels instead of 1), so you just wasted 300% of your performance. Nanite auto-picks the best triangle to minimize this and probably many more perf metrics that I have no idea about. So even if you do it in software the point is that if you can get rid of that 2x2 block penalty as much as possible you could be faster than GPU doing 2x2 blocks in hardware since pixel shaders can be very expensive. This issue gets worse the larger the rendering resolution is. Nanite then picks larger triangles instead of those tiny 1-pixel ones since those are too small to give any visual fidelity anyway. Nanite is also not used for large triangles since those are more efficient to do in hardware.
- kllrnohj 2y ago
  > So even if you do it in software the point is that if you can get rid of that 2x2 block penalty as much as possible you could be faster than GPU doing 2x2 blocks in hardware since pixel shaders can be very expensive. Of course the obvious problem with that is if you don't have most of the screen covered in such small triangles then you're paying a large cost for nanite vs traditional means.

moffkalast 2y ago

> No WebGPU available. Please use Chrome. Getting that on Chromium, lol.

gpm 2y ago
I'm getting that in google chrome proper, but what completes the joke is that in firefox I just get a blank page without the message to use chrome. Edit: WebGPU in chrome is behind a flag on linux: https://github.com/gpuweb/gpuweb/wiki/Implementation-Status#implementation-status https://github.com/gpuweb/gpuweb/wiki/Implementation-Status#...

Animats 2y ago

Oh, nice. Third party implementations of Nanite playback. Nanite is a very clever representation of graphics meshes. They're directed acyclic graphs rather than trees. Repetition is a link, not a copy. It's recursive; meshes can share submeshes, which in turn can share submeshes, all the way down. It's also set up for within-mesh level of detail support, so the submeshes drop out when they're small enough. So you can have repetitive content of very large size with a finite amount of data and fast rendering times. The insight is that there are only so many pixels on screen, so there's an upper bound on rendering work really needed. There's a really good SIGGRAPH video on this from someone at Epic. Current GPU designs are a mismatch for Nanite, Some new hardware operations are needed to do more of this in the GPU, where it belongs. Whether that will happen, with NVidia distracted by the AI market, is a good question. The scene needs a lot of instancing for this to pay off. Unreal Engine demos show such things as a hall of identical statues. If each statue was different, Nanite would help far less. So it works best for projects where a limited number of objects are reused to create large areas of content. That's the case for most AAA titles. Watch a video of Cyberpunk 2077, and look for railings and trash heaps. You'll see the same ones over and over in totally different contexts. Making a nanite mesh is complicated, with a lot of internal offsets for linking, and so far only Unreal Engine's editor does it. With playback now open source, someone will probably do that. Those internal offsets in the format present an attack surface which probably can be exploited with carefully crafted bad content, like hostile Microsoft Word .doc files.

turtledragonfly 2y ago
I think the SIGGRAPH talk you referred to is: "A Deep Dive into Nanite Virtualized Geometry" (https://www.youtube.com/watch?v=eviSykqSUUw https://www.youtube.com/watch?v=eviSykqSUUw) There's also this short high-level intro (2.5 min) that I thought was decent: "What is virtualized micropolygon geometry? An explainer on Nanite" (https://www.youtube.com/watch?v=-50MJf7hyOw https://www.youtube.com/watch?v=-50MJf7hyOw)
Jasper_ 2y ago
> Repetition is a link, not a copy. It's recursive; meshes can share submeshes, which in turn can share submeshes, all the way down. While it does construct a DAG to perform the graph cut, the final data set on disk is just a flat list of clusters for consideration, along with their cutoffs for inclusion/rejection. There seems to be a considerable misunderstanding of what the DAG is used for, and how it's constructed. It's constructed dynamically based on the vertex data, and doesn't have anything to do with how the artist constructed submeshes and things, nor does "repetition become a link". > The scene needs a lot of instancing for this to pay off. Unreal Engine demos show such things as a hall of identical statues. If each statue was different, Nanite would help far less. What makes you say this? The graph cut is different for each instance of the object, so they can't use traditional instancing, and I don't even see how it could help.

spintin 2y ago

[dead]

devit 2y ago

Name and description are very confusing and a trademark violation since despite the claims it seems to be completely unrelated to actual Nanite in UE5, just an implementation of something similar by a person unaffiliated with UE5. There is also Bevy's Virtual Geometry that provides similar functionality and is probably much more useful since it's written in Rust and integrated with a game engine: https://jms55.github.io/posts/2024-06-09-virtual-geometry-bevy-0-14/ https://jms55.github.io/posts/2024-06-09-virtual-geometry-be...

KMnO4 2y ago
I don’t think it’s really an issue. It’s clear from the readme that it’s an implementation. If I made an “implementation of OpenAI’s GPT-3 in JS” you would understand that to mean I took the architecture from the whitepaper and reimplemented it.
- cubefox 2y ago
  The technique is commonly called virtual geometry or virtualized geometry, or dynamic LOD in research papers. Really no need to reuse the name of a specific implementation.

smartmic 2y ago

Wow, I can't remember the last time I read a project summary with so much jargon - I literally didn't understand anything: > UE5's Nanite implementation using WebGPU. Includes the meshlet LOD hierarchy, software rasterizer and billboard impostors. Culling on both per-instance and per-meshlet basis.

goodcjw2 2y ago
Guess this really shows how much domain specific knowledge in the Computer Graphics... Yet still, this post is now ranked top 1 on HN.
deleted 2y ago
[deleted]
bogwog 2y ago
UE5 Nanite -> https://dev.epicgames.com/documentation/en-us/unreal-engine/nanite-virtualized-geometry-in-unreal-engine https://dev.epicgames.com/documentation/en-us/unreal-engine/... WebGPU -> https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API Meshlet -> https://developer.nvidia.com/blog/introduction-turing-mesh-shaders/ https://developer.nvidia.com/blog/introduction-turing-mesh-s... LOD -> https://en.wikipedia.org/wiki/Level_of_detail_(computer_graphics) https://en.wikipedia.org/wiki/Level_of_detail_(computer_grap... Software rasterizer -> https://en.wikipedia.org/wiki/Rasterisation https://en.wikipedia.org/wiki/Rasterisation ("software" means it runs on the CPU instead of GPU) Billboard imposters -> https://www.alanzucconi.com/2018/08/25/shader-showcase-saturday-7/ https://www.alanzucconi.com/2018/08/25/shader-showcase-satur... Culling -> https://en.wikipedia.org/wiki/Hidden-surface_determination https://en.wikipedia.org/wiki/Hidden-surface_determination
- nicebyte 2y ago
  > ("software" means it runs on the CPU instead of GPU) no, in this context it means that the rasterisation algorithm is implemented in a compute kernel, rather than using the fixed hw built into the gpu. so rasterization still happens on the gpu, just using programmable blocks.

theogravity 2y ago

Using latest chrome on M2 Max for the jinx demo: WebGPU error [frame][validation]: Fill size (7398781) is not a multiple of 4 bytes. - While encoding [CommandEncoder "main-frame-cmd-buffer"].ClearBuffer([Buffer "rasterize-sw"], 0, 7398781).

stephc_int13 2y ago
I have the same error on Windows 11, GPU is a RTX4090. Browser is Edge.
daemonologist 2y ago
Same (different number) on Chrome on Android (Pixel 7).
flockonus 2y ago
If helpful to author, on M1 no errors, can see 15+ fps at all times.

replete 2y ago

Intel Mac, Chrome and ungoogled chromium: index.web.ts:159 Uncaught (in promise) OperationError: Instance dropped in popErrorScope

eigenvalue 2y ago

Whenever I see rendered scenes like this (I.e., lots of repetitive static geometry) I imagine that annoying guy’s voice going on about “unlimited detail” from that old vaporware video. I guess nanite really did solve that problem for real, as opposed to whatever that old thing was using (I remember something about oct-trees or something).

HappMacDonald 2y ago
I recall those claims being made by a company called "Euclidean", from Australia I think. Online rumors suggested they might have been using octtrees, but later Euclidean videos flatly denied that.
- raphlinus 2y ago
  It's Euclideon. And it is octtrees. My interpretation after reading a fascinating Reddit thread [1] is that these denials were misdirection. There's definitely new interest in splatting techniques (Gaussian in particular), though they've long been an alternative to triangles in the 3D world. I think it'd be fun to experiment with implementing some of that using modern compute shaders. [1]: https://www.reddit.com/r/VoxelGameDev/comments/1bz5vvy/a_small_update_on_cpu_octree_splatting_feat/ https://www.reddit.com/r/VoxelGameDev/comments/1bz5vvy/a_sma...

readyplayernull 2y ago

Will virtual geometry be integrated into GPUs some day?

cubefox 2y ago
Some aspects of it will likely receive hardware acceleration in the future. For example support for a standard virtual geometry mesh format, like this one proposed by AMD: https://gpuopen.com/download/publications/DGF.pdf https://gpuopen.com/download/publications/DGF.pdf

forrestthewoods 2y ago

Note: this isn't actually UE5 Nanite in WebGPU. It's a totally independent implementation of the same idea as Nanite. This technique is starting to appear in a variety of places. Nanite definitely made the idea famous, but Nanite is the name a specific implementation, not the name of the technique.

tech-no-logical 2y ago

getting the message No WebGPU available. Please use Chrome. on chrome (Version 129.0.6668.29 (Official Build) beta (64-bit)) , under windows

Joel_Mckay 2y ago
Look at the #enable-unsafe-webgpu flag in chrome. Turn it back off when done, as tools like noscript only block webgl tags. Cheers =3

jms55 2y ago

It's been mentioned a couple of times in this thread, but Bevy also has an implementation of Nanite's ideas (sometimes called Virtual Geometry). I'm the author of that, happy to answer questions :) As for this project, Scthe did a great job! I've been talking with them about several parts of the process, culminating in some improvements to Bevy's code based on their experience (https://github.com/bevyengine/bevy/pull/15023 https://github.com/bevyengine/bevy/pull/15023). Always happy to see more people working on this, Nanite has a ton of cool ideas.

KronisLV 2y ago

I wonder how other engines compare when it comes to LODs and similar systems. Godot has automatic LOD which seems pretty cool for what it is: https://docs.godotengine.org/en/stable/tutorials/3d/mesh_lod.html https://docs.godotengine.org/en/stable/tutorials/3d/mesh_lod... Unity also has an LOD system, though despite how popular the engine is, you have to create LOD models manually: https://docs.unity3d.com/Manual/LevelOfDetail.html https://docs.unity3d.com/Manual/LevelOfDetail.html (unless you dig through the asset store and find a plugin) I did see an interesting approach in a lesser known engine called NeoAxis: https://www.neoaxis.com/docs/html/NeoAxis_Levels.htm https://www.neoaxis.com/docs/html/NeoAxis_Levels.htm however that engine ran very poorly for me on my old RX580, although I haven't tried on my current A580. As far as I can tell, Unreal is really quite far ahead of the competition when it comes to putting lots of things on the screen, except the downside of this is that artists will be tempted to include higher quality assets in their games, bloating the install sizes quite far.

kllrnohj 2y ago
In theory Nanite is superior to precomputed LODs. In practice it's less clear cut as they aren't going to be as good as artist-created LODs and it's not entirely reasonable to expect them to do so. Also the performance cost is huge as Nanite/virtual geometry is a poor fit for modern GPUs. iirc peak fill rate is 1/4th or something like that as GPU rasterization works on 2x2 quads not per-pixel like shaders do.
- jsheard 2y ago
  Rasterizing very small triangles in hardware is indeed inefficient due to the 2x2 quad tax, but one of Nanites tent-pole features is a software rasterizer which sidesteps that problem entirely. IIRC they said that for a screen entirely filled with triangles roughly the size of a pixel, their software raster ends up being about 3x faster than using the dedicated raster hardware.
  - kllrnohj 2y ago
    Yes but I'm talking the other way around. Nanite is 1/4th the performance for triangles that aren't 1-3 pixels in size, which is the majority of the time. The main selling point of Nanite is really just to reduce artist costs by avoiding manual LODs. But a high quality automatic LOD at build time may (read: almost certainly does) strike a much better balance for both current and near future hardware

hising 2y ago

I would love to see this but it wont work on Linux + Chrome even if WebGPU is enabled.

jesse__ 2y ago

> If you want to add this tech to the existing engine, I'm not a person you should be asking (I don't work in the industry). Fucking .. bravo man.

lerpgame 2y ago

[dead]

astlouis44 2y ago

Here's an actual implementation of UE5 in WebGPU, for anyone interested. Just a disclaimer that it will only work on WebGPU-enabled browser on Windows (Chrome, Edge, etc) unfortunately Mac has issues for now. Also, there is no Nanite in this demo, but it will be possible in the future. https://play.spacelancers.com/ https://play.spacelancers.com/

mdaniel 2y ago
I was curious what "issues" Mac has, and at least for me it didn't explode for any good reason, it puked trying to JSON.stringify() some capabilities object into localStorage which is a pretty piss-poor reason to bomb loading a webpage, IMHO

mbforbes 2y ago

Funny coincidence, was just reading through an amazing thread on the three.js form a couple days ago about a web graphics implementation of virtual geometry (nanite). webgl, 2021: https://discourse.threejs.org/t/virtually-geometric/28420 https://discourse.threejs.org/t/virtually-geometric/28420 it's closed source, but I found the discussion and description of the tradeoffs interesting

cubefox 2y ago

Here is the somewhat neglected original 2009 dissertation by Federico Ponchio, the guy who invented the dynamic mesh simplification algorithm on which Nanite is based, with lots of illustrations: https://vcg.isti.cnr.it/~ponchio/download/ponchio_phd.pdf https://vcg.isti.cnr.it/~ponchio/download/ponchio_phd.pdf (107 pages!)

nox101 2y ago

This is amazing! I will be so great when Safari and Firefox but finish their WebGPU implementations so it runs everywhere.

vladde 2y ago

Got to love in the stated goals: > I could have built this with Vulkan and Rust. None would touch it.

desdenova 2y ago

The examples don't work, though.

swiftcoder 2y ago
They run fine for me on Chrome/Mac

pjmlp 2y ago

On Windows it can't handle device loss. ID3D12Device::GetDeviceRemovedReason failed with DXGI_ERROR_DEVICE_HUNG (0x887A0006) - While handling unexpected error type Internal when allowed errors are (Validation|DeviceLost). at CheckHRESULTImpl (..\..\third_party\dawn\src\dawn\native\d3d\D3DError.cpp:119) at CheckAndUpdateCompletedSerials (..\..\third_party\dawn\src\dawn\native\d3d12\QueueD3D12.cpp:179) at CheckPassedSerials (..\..\third_party\dawn\src\dawn\native\ExecutionQueue.cpp:48) at Tick (..\..\third_party\dawn\src\dawn\native\Device.cpp:1730) Backend messages: \* Device removed reason: DXGI_ERROR_DEVICE_HUNG (0x887A0006)

meindnoch 2y ago

Can someone explain what Nanite is? The other day someone was saying it uses software rendering because the triangles are so small. Wtf?