11 ms·
WASM GC isn't ready for realtime graphics
- kevingadd 2y agoIt's sort of baffled me that people appear to be shipping real code using WasmGC since the limitations described in this post are so severe. Maybe it's fine because they're just manipulating DOM nodes? Every time I've looked at WasmGC I've gone "there's no way I could use this yet" and decided to check back a year later and see if it's There Yet. Hopefully it gets there. The uint8array example from this post was actually a surprise to me, I'd just assumed it would be efficient to access a typed array via WasmGC! Beyond the limitations in this post there are other things needed to be able to target WasmGC with existing stuff written in other languages, like interior references or dependent handles. But that's okay, I think, it can be worthwhile for it to exist as-is even if it can't support i.e. existing large-scale apps in memory safe languages. It's a little frustrating though.
- refulgentis 2y agoI've been shipping a Flutter app that uses it for months. Pretty heavy stuff, its doing everything from LLM inference to model inference to maintaining a vector store and indexeddb in your browser. Frame latency feels like it's gone, there's 100% a significant decrease in perceived latency. I did have a frustrating performance issues with 3rd party code doing "source code parsing" via RegEx, thought it was either the library or Flutters fault, but from the article content, sounds like it was WASM GC. (saw a ton of time spent converting objects from JS<->WASM on a 50 KLOC file) From that perspective, the article sounds a bit maximalist in its claims, but only from my perspective. I think if you read "real time graphics" as "3d game" it gives a better understanding of where it's at, my anecdata aside.
- lukasb 2y agoWhen you said "jump in perceived latency", did you mean perceived latency went up or down?
- refulgentis 2y agoDown, significantly
- ripped_britches 2y agoWhich libraries caused these problems for you?
- refulgentis 2y agoDon't wanna name names, because it's on me, it's a miracle it exists, and works. I don't think there's a significant # of alternatives, so hopefully Flutter syntax highlighting library, as used in a package for making markdown columns, is enough to be helpful. Problem was some weird combo of lots of regex and an absolutely huge amount of code. It's one of those problems it's hard for me to draw many conclusions from: - Flutter may be using browser APIs for regex, so there's some sort of JS/WASM barrier copying cost - The markdown column renderer is doing nothing at all to handle this situation lazily, i.e. if any portion of the column is displayed, syntax highlighting must be done on the complete markdown input - Each different color text, so pretty much every word, gets its own object in the view hierarchy, tens if not hundreds of thousands this case. Can't remember if this is due to the syntax highlighting library or the markdown package - Regex is used to parse to code and for all I know one of them has pathological performance like backtracking unintentionally.
- stiles11 2y agoWhat's the name of the app I want to try it out
- azakai 2y agoDefinitely a lot is missing, yeah, and adding more will take time. But it works well already for pure computational code. For example, Google Sheets uses WasmGC for Java logic: https://web.dev/case-studies/google-sheets-wasmgc#the_final_result https://web.dev/case-studies/google-sheets-wasmgc#the_final_...
- wffurr 2y ago>> The uint8array example from this post was actually a surprise to me, I'd just assumed it would be efficient to access a typed array via WasmGC! The problem is that the Scheme i8 array is not actually a UInt8Array with WasmGC. It’s a separate heap allocated object that is opaque to the JS runtime. In the linear memory Wasm model, the Scheme i8 array is allocated in the wasm memory array, and so one can create an UInt8Array view that exactly maps to the same bytes in the linear memory buffer. This isn’t possible (yet?) with the opaque WasmGC object type.
- davexunit 2y agoYes, that's right. I'm hoping there will be a way to do this in a future revision of Wasm GC.
- mickael-kerjean 2y agoI'm a happy wasm user, last week I was looking at adding support for psd file on my app [1], the choices was in between using a full js lib that is almost 1MB large, doesn't work very well and is slow OR leverage some C stuff. Without any kind of optimisation, the wasm version is 10 times faster and 5 times smaller and that's before digging through SIMD. There will always people complaining about a lot of things but wasm is already fine enough for a lot of use cases [1] https://github.com/mickael-kerjean/filestash https://github.com/mickael-kerjean/filestash
- davexunit 2y agoHope my post didn't come across as complaining because I agree! Wasm is great right now for lots of things, just wanted to highlight a use case that isn't great yet.
- sestep 2y agoI was excited to read this post because I haven't yet tried WasmGC for anything beyond tiny toy examples, but was disappointed to find no actual numbers for performance. I don't know the author well enough to be able to assess their assertions that various things are "slow" without data.
- davexunit 2y agoThis was a quickie post for me. Just a tale of what my experience has been. No in-depth, apples to apples comparison so I don't know the magnitude of the performance differential.
- out_of_protocol 2y agoReally liked NaCl (and PNaCl) idea, which allows running arbitrary code, sanitized, with ~90% speed of native execution. Playing Bastion game in browser was refreshing. Unfortunately communication with js code and bootstrap issues (can't run code without plugin, no one except chrome supported this) ruined that tech
- thegeomaster 2y agoWASM nowadays has become quite the monstrosity compared to NaCl/PNaCl. Just look at this WASM GC spaghetti, trying to compile a GC'd language but hooking it up V8/JavaScriptCore's GC, while upholding a strict security model... That sounds like it won't cause any problems whatsoever! Sometimes I wonder if the industry would have been better off with NaCl as a standard. Old, mature tooling would by and large still be applicable (it's still your ordinary x86/ARM machine code) instead of the nascent and buggy ecosystem we have now. I don't know why, but the JS folks just keep reinventing everything all the time.
- zdragnar 2y agoI'm reminded of writing JavaScript way back in the old Internet Explorer days (6 and to a lesser extent 7), when you had to manually null out any references to DOM elements if you were done with them, or else the JS and the DOM nodes wouldn't get garbage collected because IE had two different garbage collectors and cycles between them didn't get collected immediately.
- duskwuff 2y ago> Old, mature tooling would by and large still be applicable (it's still your ordinary x86/ARM machine code) It wasn't, though. Since NaCl ran code in the same process as the renderer, it depended upon a verifier for security, and required the generated code to follow some unusual constraints to support that verification. For example, on x86, all branch targets were required to be 32-byte aligned, and all indirect branches were required to use a specific instruction sequence to enforce that alignment. Generating code to meet these constraints required a modified compiler, and reduced code density and speed. In any case, NaCl would have run into the exact same GC issues if it had been used more extensively. The only reason it didn't was that most of the applications it saw were games which barely interacted with the JS/DOM "world".
- simon_void 2y agoso what about realtime graphics with wasm without GC? (compiled from languages not needing a GC like Rust, C/C++, Odin, ...)
- davexunit 2y agoAs mentioned, that works quite well already but it's not the topic of this post.
- simon_void 2y agotrue, it's just that the topic of this post seemed strange to me, since you wouldn't use a programming language with GC for high intensity graphics app in native either, hmmm.
- davexunit 2y agoGC'd languages are used all the time for this. C# is a huge language in game development, for example.
- pjmlp 2y agoBetter, but WebGPU and WebGL aren't going to win any performance prizes either, and tooling is pretty much non existent. Nothing like Pix, Instruments or Renderdoc, SpectorJS is the only thing you get after almost 15 years since WebGL 1.0. And from the hardware level they support, it about PlayStation 3 kind of graphics, if the browser doesn't block the GPU, nor selects the integrated one instead of dedicated one. Your are left with shaders as the only way to actually push the hardware.
- amelius 2y agoShouldn't it be possible to implement your own GC in WASM? Why does WASM try to be everything?
- davexunit 2y agoSlower, single threaded, greatly increases binary size, separate heap from JS so bad interop with extern refs. Wasm GC is a great thing.
- dzaima 2y agoYou can't GC together with the host environment if you do a custom GC (i.e. a wasm object and a JS object in a cycle wouldn't have any way to ever be GC'd).
- bhelx 2y agoyes, it's regularly done. But I think you are misunderstanding. WASM GC isn't a GC implementation.
- fulafel 2y agoYes, this is how it's done eg with Python and Go. An advantage of a common GC could be interop between languages.
- James_K 2y agoNot really. As I understand it, WASM offers no facility to unwind the stack which is needed for tracing garbage collectors. The only solution here is to manually write your own implementation of the stack on the heap and force all data to live there instead of in registers. This is a huge performance penalty.
- weinzierl 2y agoWasn't WASM GC a prerequisite for getting direct DOM access from WASM? Does progress for WASM GC mean progress for DOM access as well? Every time I check back on that the initiative seems to run under a different name. What is the best way to track progress on that front?
- josephg 2y agoIt’s not a prerequisite for using the DOM from wasm. See, for example, the rust web frameworks of leptos and dioxus. They’re honestly great, and usable today as replacements for react and friends. (With the single caveat that wasm bundle size is a bit bigger than .js size). They work by exposing a number of browser methods through to wasm, and then they call them through a custom wasm/JS API bridge. All rust objects and DOM objects are completely isolated. Rust objects are allocated via an embedded malloc implementation and JS objects are managed by V8 (or whatever). but the DOM can still be manipulated via (essentially) message passing over an RPC like interface. But the rust code needs to compile malloc specially for wasm. This is ok in rust - malloc is 75kb or something. But in languages like C#, Go or Python, the runtime GC is much bigger and harder to fit in a little wasm bundle. The upside of wasm-gc is that this divide goes away. Objects are just objects, shared between both languages. So wasm bundles can use & reference JS/DOM objects directly. And wasm programs can piggyback on V8’s GC without needing to ship their own. This is good in rust, and great in GC languages. I saw an example with blazor where a simple C# wasm todo app went from 2mb or something to 10kb when wasmgc was used. TLDR: wasm-gc isn’t strictly needed. You can use DOM from wasm today. It just makes wasm bundles smaller and wasm-dom interaction easier (and theoretically faster).
- quotemstr 2y agoWasm GC also solved the problem of reference cycles between objects in disparate heap managers leading to memory leaks. It's not just a performance or size win: it's a correctness win.
- geokon 2y agoOut of curiosity, why is malloc 75kb ? That seems like an crazy amount of code (if this after linking and dead code removal for platform specific magic?)
- Dwedit 2y agoI just wish WASM could use more than one ArrayBuffer at a time. Would eliminate unnecessary copying for interop with JS code.
- Dwedit 2y agoWell I just thought of something obvious... Have a function that lets you pass in an ArrayBuffer, then it brings it into the virtual address space of the WASM program. Function would return the virtual address that was assigned to that array buffer. From there, you call into WASM again with that pointer, and the program can take action. Then there would be another function to relinquish ownership of the ArrayBuffer.
- throwaway290 2y agoThere's no SharedArrayBuffer support? Or I misunderstand the idea
- flohofwoe 2y agoThere is, but then you'd need to declare the entire WASM heap as a single SharedArrayBuffer. It only makes sense for shared-memory multithreading (but not that support for SharedArrayBuffer only works in 'cross-origin isolated' contexts). There is a related 'multiple memories' proposal btw: https://github.com/WebAssembly/multi-memory https://github.com/WebAssembly/multi-memory
- o11c 2y ago> Unsatisfying workarounds [...] Use linear memory for bytevectors It never makes sense to use GC for leaf memory if you're in a language that offers both, since mere refcounting (or a GC'ed object containing a unique pointer) is trivial to implement. There are a lot of languages where it's expensive to make the mistake this post is making. (I don't know much about WASM in particular; it may still have other errors).
- davexunit 2y agoSorry but it's just a different choice not a mistake. I do realtime graphics just fine in non-web managed memory languages.
- pjmlp 2y agoTo be fair neither are WebGL and WebGPU, versus the native API counterparts, the best you can get are shadertoy demos, and product visualisation on ecommerce sites. Due to tooling, sandboxing and not having any control about what GPU gets selected, or why the browser blakckboxes it and switches into software rendering.
- chilmers 2y agoFigma uses WebGL for rendering and they seem to be doing ok.
- pjmlp 2y agoYeah, at a level like GDI+, CoreGraphics, XWindows hardware surfaces,.... This isn't really what real-time graphics is all about in modern times. This is, https://youtu.be/AV279wThmVU?si=Ou04h5z0Mju7kiJ0 https://youtu.be/AV279wThmVU?si=Ou04h5z0Mju7kiJ0 The demo is from 2018, 7 years ago!
- chilmers 2y agoThe claim was that with WebGL was "the best you can get are shadertoy demos, and product visualisation on ecommerce sites". Figma is neither, regardless of how it's making use of WebGL under the hood. Not sure what relevance an Unreal engine demo is, as you seem to think I was making a claim about real-time graphics that I wasn't.
- pjmlp 2y agoFigma falls under the graphics requirements of ecommerce sites, I have my doubts that they even make use of WebGL 2.0 features. It only a way to hopefully get hardware accelerated canvas, if the browser doesn't consider GPU blacklisted for 3D acceleration. That isn't real time graphics in the sense of games programming.
- fidotron 2y agoI had this argument with pjmlp not too long ago, and it goes in circles. Basically they define anything less than pushing the extreme limits of rendering technology to be worthless, while simultaneously not actually understanding what that is beyond the marketing hype. The fact most users would not be able to run that seven year old demo on their systems today, even natively, would be beside the point of course. WebGL particularly absolutely has problems, but the revealing thing is how few people state what they really are, such as the API being synchronous or the inability to use inverted z-buffers. Instead it's a lot of noise about ray tracing etc. WASM per call overhead is a whole other problem too, GC or not.
- flohofwoe 2y agoNot surprising tbh, no automatic memory management solution is ready for realtime graphics - it always requires a lot of manual care to reduce memory management overhead (for instance working around the GC instead of using it), which then kinda defeats the purpose of the 'automatic' in automatic memory management. There simply is no memory management silver bullet when performance matters.
- davexunit 2y agoPerhaps I should've said this in the post to ward off comments from the anti-GC crowd but I do realtime graphics in managed memory languages just fine outside of the Wasm context. This is a Wasm problem.
- wavemode 2y ago> no automatic memory management solution is ready for realtime graphics Unreal Engine and Unity don't exist then?
- flohofwoe 2y agoUnity games have plenty of GC related stutter issues and working around the Unity GC to prevent those issues is basically its own field of engineering. As for UE, I wonder if they would go down the same route today, or whether they just can't rip out their GC because it would mean a complete rewrite of the engine. Games don't need lifetime management at object granularity.
- torginus 2y agoSorry to be ignorant, but I have a couple questions about Wasm: - Will the Wasm GC pave the way to having native object references to JS objects/classes (seeing how its the same GC as used by JS)? - Is it possible to do lightweigt interop between a high level language using Wasm GC (such as Java), with a low-level one (such as C++) that doesn't use it? If, so, how? Imo the biggest shortcoming of Wasm is the incredibly awkward way in which it integrates into the browser
- whizzter 2y agoLike the article says, you can have _opaque_ references. It's really a small step forward but does address a major pain point. So for your first question. Yes you can hold a reference, so the major benefit is that dead objects via things like memory cycles aren't an issue any more (previously connecting objects between WASM and JS worlds needed manual references that outside of something like C++ RAII would be brittle). For the second part, it would depend on the compiler but since Java/C++ would both live in a WASM world they should be possible to combine fairly painlessly (given a sane way to declare linked functions that are named properly), I'd put it's interoperability issues at the same level as PInvoke from C#.
- torginus 2y agoPInvoke is awesome. If you use MSVC for C++ it can do stuff like SEH/stack debugging that goes from .NET frames to C++ and back.
- pjmlp 2y agoEven better, if Windows is the only deployment target, throw away P/Invoke, use C++/CLI and there is no need to guess how to get P/Invoke attributes exactly correct for each API call.
- flohofwoe 2y ago> Will the Wasm GC pave the way to having native object references to JS objects/classes AFAIK that was already possible via 'WASM reference types': https://caniuse.com/wasm-reference-types https://caniuse.com/wasm-reference-types Seeing that this is not yet used by Emscripten JS shims I wonder if there are downsides which prevent its use for something like WebGL objects, or whether just nobody got around yet rewriting the Emscripten shims to externrefs. PS: probably not worth the effort (see last comment: https://github.com/emscripten-core/emscripten/issues/20021 https://github.com/emscripten-core/emscripten/issues/20021)
- zamalek 2y ago> This would require a little malloc/free implementation and a way to reclaim memory for GC'd bytevectors. Allocation is always expensive in the hot path. Arenas, pools, etc. are a thing even in C engines for a reason. If it's possible to mix GC and linear, that's ridiculously powerful.
- davexunit 2y agoIn Scheme what you want to do is allocate a big bytevector and use it over and over. This is what I already do outside of Wasm. I don't want or need linear memory involved, I just want access to my (array i8) from JS.
- hoppp 2y agoThe code snippets were very cool. I'll check out hoot! Overall I don't think Wasm GC is for realtime graphics. Wouldn't the garbage collector cause a framerate drop when rendering? Usually garbage collectors are not used for graphics heavy apps. My impression was that WasmGC was perfect for writing business logic in Go or C# or other garbage collected languages that were not running perfectly before in wasm.
- davexunit 2y agoYou can absolutely do graphics rendering with GC'd languages. The trick is the same as with manual memory management: Minimize allocation. You mention C# which is a language that has been used very successfully in the games industry.
- hoppp 2y agoUnity uses C# but the engine is written in C++, but you are probably right and there are other engines out there written in C#