4 ms·

Hot take alert > When is WASM finally going to be able to touch the DOM? Coming from a web background, and having transitioned to games / realtime 3D applicat

Hot take alert > When is WASM finally going to be able to touch the DOM? Coming from a web background, and having transitioned to games / realtime 3D applications... Fuck the DOM dude. The idea that programming your UI via not one but TWO DSLs, and a scripting language, is utter madness. In principal, it might sound good (something something separation of concerns, or whatever-the-fuck), but in reality you always end up with this tightly coupled garbage fire split across a pile of different files and languages. This is not the way. We need to build better native UI libraries that just open up a WebGL context and draw shit to that. DearIMGUI can probably already do like 85% of what modern webapps do. Anyways .. /rant

tormeh 1y ago
Preach. HTML and CSS are markup languages for creating documents. Using them for GUI applications is wild and obviously a bad idea.
- silon42 1y ago
  HTML is pretty bad (XUL was better), but (subset) of CSS is probably OK.
- chrismorgan 1y ago
  Most major GUI frameworks operate on something very similar to HTML + CSS.
  - jesse__ 1y ago
    Can you elaborate? I don't see how this is true.
    - pjmlp 1y ago
      QML and XAML for example, or the layout authoring tools from Motif back in the day.
- notnullorvoid 1y ago
  HTML isn't even a particularly good markup language for documents, web or otherwise. > Using them for GUI applications is wild and obviously a bad idea. I agree it's wild, but bad idea implies there are better options for cross platform applications with the same ease of distribution, and accessibility as the web.
halfcat 1y ago
> DearIMGUI can probably already do like 85% of what modern webapps do I’m with you. Main blocker I’ve seen to “just use ImGui for everything” (which I’d love to adopt), is if I run ImGui in WASM the keyboard doesn’t open on mobile. This seems possible in theory because egui does it. Even though running ImGui on mobile via WASM isn’t the primary use case, inevitably the boss or someone is going to need to “just do a quick thing” and won’t be able to on mobile, and this seems to be a hard ceiling with no real escape hatch or workaround. One of those scenarios where, if we have to use a totally different implementation (React or whatever) to handle this 1% edge case, we might as well just use that for the other 99%.
- jesse__ 1y ago
  I'd say two things about this. 1. Opening the native keyboard and plumbing those events through to the WASM runtime sounds pretty easy. It's probably not cause modern software, but conceptually it should be trivial.. right?? 2. In terms of 'the boss' wanting to do 'that one weird thing' that there isn't a library/plugin/whatever for in DearImgui land. If dev time for everything else gets faster, than the 10x cost of that small corner case can be absorbed by net win. Now, I'm pretty sus on the claim everything else gets better today, but we can certainly imagine a world where they do, and it's probably not far away
  - halfcat 1y ago
    > Opening the native keyboard… I think this is the roadblock, that there isn’t always a way to pop open the keyboard programmatically. Rather, the mobile keyboard only pops up when there’s a DOM input element detected. So it would need a hidden input layered on top of the ImGui WASM app and mapping coordinates, or would need an HTML input element overlayed on top of every text input.
    - jesse__ 1y ago
      I mean, yikes. But having a hidden html input you just piggy back off of also doesn't sound that bad
jenadine 1y ago
That's true. But without the DOM we also loose the browser native accessibility support and text interaction and so on.
- norman784 1y ago
  I know nothing about how accessibility supports works in the browser, but could they offer a way to use the accessibility API with your custom UI rendered in WebGL/WebGPU?
  - flykespice 1y ago
    There doesn't exist a way to infer the individual elements of a rendered frame in canvas because it just encodes the pixels. You can't know which pixels make up a text or button, that information is lost, unlike a DOM tree.
preisschild 1y ago
Couldn't you implement something like HTMX in wasm then and still have locality-of-behavior by specifying the behavior as html attributes?
nmfisher 1y ago
> We need to build better native UI libraries that just open up a WebGL context and draw shit to that. This is what Flutter does. It works well, but you do lose some nice things that the browser/DOM provides (accessibility, zooming, text sizing/selection, etc). There’s also a noticeable overhead when the app + renderer is initially downloaded).
dualogy 1y ago
> We need to build better native UI libraries that just open up a WebGL context and draw shit to that. DearIMGUI can probably already do like 85% of what modern webapps do. Would be totally into that! But then as soon as you're developing for a worldwide paying audience (or your employer is), one manager realizes the need for ARIA stuff, another for SEO, another for fully-native input controls (textboxes, checkboxes, radios, buttons etc) for most-fluid&familiar UX on every OS, another for embedding externally loaded markup+scripts (whether that's social media sharing widgets, auth providers, Google Maps / OpenStreetMaps etc), for supporting user-generated `<img>`s and `<video>` in all the various formats the browser natively furnishes, and on and on and on =)

tormeh 1y ago

Preach. HTML and CSS are markup languages for creating documents. Using them for GUI applications is wild and obviously a bad idea.

silon42 1y ago
HTML is pretty bad (XUL was better), but (subset) of CSS is probably OK.
chrismorgan 1y ago
Most major GUI frameworks operate on something very similar to HTML + CSS.
- jesse__ 1y ago
  Can you elaborate? I don't see how this is true.
  - pjmlp 1y ago
    QML and XAML for example, or the layout authoring tools from Motif back in the day.
notnullorvoid 1y ago
HTML isn't even a particularly good markup language for documents, web or otherwise. > Using them for GUI applications is wild and obviously a bad idea. I agree it's wild, but bad idea implies there are better options for cross platform applications with the same ease of distribution, and accessibility as the web.

halfcat 1y ago

> DearIMGUI can probably already do like 85% of what modern webapps do I’m with you. Main blocker I’ve seen to “just use ImGui for everything” (which I’d love to adopt), is if I run ImGui in WASM the keyboard doesn’t open on mobile. This seems possible in theory because egui does it. Even though running ImGui on mobile via WASM isn’t the primary use case, inevitably the boss or someone is going to need to “just do a quick thing” and won’t be able to on mobile, and this seems to be a hard ceiling with no real escape hatch or workaround. One of those scenarios where, if we have to use a totally different implementation (React or whatever) to handle this 1% edge case, we might as well just use that for the other 99%.

jesse__ 1y ago
I'd say two things about this. 1. Opening the native keyboard and plumbing those events through to the WASM runtime sounds pretty easy. It's probably not cause modern software, but conceptually it should be trivial.. right?? 2. In terms of 'the boss' wanting to do 'that one weird thing' that there isn't a library/plugin/whatever for in DearImgui land. If dev time for everything else gets faster, than the 10x cost of that small corner case can be absorbed by net win. Now, I'm pretty sus on the claim everything else gets better today, but we can certainly imagine a world where they do, and it's probably not far away
- halfcat 1y ago
  > Opening the native keyboard… I think this is the roadblock, that there isn’t always a way to pop open the keyboard programmatically. Rather, the mobile keyboard only pops up when there’s a DOM input element detected. So it would need a hidden input layered on top of the ImGui WASM app and mapping coordinates, or would need an HTML input element overlayed on top of every text input.
  - jesse__ 1y ago
    I mean, yikes. But having a hidden html input you just piggy back off of also doesn't sound that bad

jenadine 1y ago

That's true. But without the DOM we also loose the browser native accessibility support and text interaction and so on.

norman784 1y ago
I know nothing about how accessibility supports works in the browser, but could they offer a way to use the accessibility API with your custom UI rendered in WebGL/WebGPU?
- flykespice 1y ago
  There doesn't exist a way to infer the individual elements of a rendered frame in canvas because it just encodes the pixels. You can't know which pixels make up a text or button, that information is lost, unlike a DOM tree.

preisschild 1y ago

Couldn't you implement something like HTMX in wasm then and still have locality-of-behavior by specifying the behavior as html attributes?

nmfisher 1y ago

> We need to build better native UI libraries that just open up a WebGL context and draw shit to that. This is what Flutter does. It works well, but you do lose some nice things that the browser/DOM provides (accessibility, zooming, text sizing/selection, etc). There’s also a noticeable overhead when the app + renderer is initially downloaded).

dualogy 1y ago

> We need to build better native UI libraries that just open up a WebGL context and draw shit to that. DearIMGUI can probably already do like 85% of what modern webapps do. Would be totally into that! But then as soon as you're developing for a worldwide paying audience (or your employer is), one manager realizes the need for ARIA stuff, another for SEO, another for fully-native input controls (textboxes, checkboxes, radios, buttons etc) for most-fluid&familiar UX on every OS, another for embedding externally loaded markup+scripts (whether that's social media sharing widgets, auth providers, Google Maps / OpenStreetMaps etc), for supporting user-generated `<img>`s and `<video>` in all the various formats the browser natively furnishes, and on and on and on =)