5 ms·

Thanks for writing this article! I am super excited about WebGPU. One not-so-fancy prospect worth commenting about on HN, though, is replacing Electron. With W

Thanks for writing this article! I am super excited about WebGPU. One not-so-fancy prospect worth commenting about on HN, though, is replacing Electron. With WASM-focused initiatives to create hardware accelerated UI in the browser, we may soon see a toolchain that deploys to a WebGPU canvas and WASM in the browser, deploys native code linked to WGPU outside the browser, and gives the industry a roadmap to Electron-style app development without the Chromium overhead.

danShumway 3y ago
This is also my fear, but... I don't know, I think the potential outweighs the risks. In some ways the problem with everyone trying to render to canvases and skip the DOM starts from education and a lack of native equivalents to the DOM that really genuinely showcase the strengths beyond having a similar API. I think developers come into the web and they have a particular mindset about graphics that pushes them away from "there should be a universal semantic layer that I talk to and also other things might talk to it", and instead the mindset is "I just want to put very specific pixels on the screen, and people shouldn't be using my application on weird screen configurations or with random extensions/customizations anyway." And I vaguely think that's something that needs to be solved more by just educating developers. It'll be a problem until something happens and native platforms either get support for a universal semantic application layer that's accessible to the user and developers start seeing the benefits of that, or... I don't know. That's maybe a long conversation. But there has to be a shift, I don't think it's feasible to just hold off on GPU features. At some point native developers need to figure out why the DOM matters or we'll just keep on having this debate. People wanting to write code that runs on both native and the web is good, it's a reasonable instinct. Electron-style app development isn't a bad goal. It's just how those apps get developed and what parts of the web get thrown out because developers aren't considering them to be important.
- syntheweave 3y ago
  When it comes down to it, I think most of our issue is that the technical details still matter enough that we can't afford to provide a universal human interface layer...and when that falls on the app developer, either they compromise the app or they compromise the user, because nobody is capable of filling in every rough edge and also doing the thing they set out to do in that moment. If they actually set out to create a fix, that quickly turns into their career. A lot of the "web frontend churn" meme is derived from the quest for a slightly better compromise - which has produced some results if what you are building resembles a web page. But over and over, native devs will say, "no, I want the hardware access. This layer still doesn't address my use case." That's been true of anything deeply I/O related on the Web - graphics, audio, input. Things where you can't make a simplifying assumption if you actually start covering all use cases.
  - danShumway 3y ago
    > I think most of our issue is that the technical details still matter enough that we can't afford to provide a universal human interface layer I don't think this is true for the majority of native apps I run. I think a lot of developers think they need low-level control over exactly what pixels on the screen turn what colors, but I don't think most apps are in that category. Devs that use the GPU a lot are just used to working that way. There are cases where that kind of access is needed. I use WebGL right now, I will use WebGPU when it comes out. But how much of your desktop software genuinely, legitimately could not be described using a pure-text tree? Maybe Blender? Even an app like Krita really only needs precise control over pixels for the actual drawing part. It's mostly games and very experimental apps and visual content creation tools. Everything else is at most "we need a buffer rendered via the GPU embedded in this interface for one thing". My take on this is: if your app works with a screenreader, that app is probably representable via some kind of universal interface. And the advantages to having a layer like that on native platforms would be profound. People talk about how great the terminal is, imagine if you could do everything you can do on the terminal -- piping output between programs, scraping content, building wrappers around interfaces, easy remote access, etc -- imagine if you could do all of that for all of the GUI apps on your computer too. What we would lose in developer flexibility to define an initial interface we would make up for ten-fold with the additional flexibility developers and users would have to build around existing interfaces and to extend them. > A lot of the "web frontend churn" meme is derived from the quest for a slightly better compromise I'm also not sure this is entirely true. Yes, performance is a thing people talk about but systems like React came out of developers wanting a declarative model for building interfaces. In many ways it was going the opposite direction of canvas/webGL code. A lot of other libraries are similar -- you look at something like d3, that's not really about how things get drawn on the screen, what makes it special is data bindings. Debates about the virtual DOM are kind of a small subset of the churn that we see online, and my impression is that the churn is pushed more by people debating how to build interfaces in the first place and how to further abstract that process, not how this stuff should get rendered to the screen. We can have a high diversity of tools for building interfaces, what makes the DOM special is that it's universal render target. Whatever tool you use, the user gets the same semantics. And the DOM isn't necessarily a good universal render target. But it's kind of telling that even though the DOM has a ton of issues, the underlying philosophy of "you should have to describe what you want to put on the screen using text first and then we'll let you style it" is itself really empowering for end-users.
crazygringo 3y ago
The only possible application I can imagine for that would be videogames, though. Because HTML+CSS+JS provides a fantastic cross-platform UI toolkit that everybody knows how to use. Videogames create their own UI in order to have lots of shiny effects and a crazy immersive audio-filled controller-driven experience... but non-videogames don't need or want that. Heck, I'm actually expecting the opposite -- for the entire OS interface to become based on Chromium HTML+CSS+JS, and eventually Electron apps don't bundle a runtime, because they're just apps. My Synology DSM is an entire operating system whose user interface runs in the browser and it just... makes sense.
- uoaei 3y ago
  What is holding back a UI framework for WebGPU? It won't have HTML but couldn't you just simulate a scaffold inside the WebGPU canvas?
  - iainmerrick 3y ago
    For most applications you'll want to do things like render text nicely, select text, copy and paste text, right-click to inspect, boost font sizes and change colors for accessibility, auto-translate to other languages. A lot of it is about text, I guess! Once you've re-implemented all that in a canvas, you've more or less built another web browser, so why not just use HTML?
    - uoaei 3y ago
      Presumably the argument in this thread is that lugging around an Electron app just for HTML is overkill if you could put the same functionality in a much smaller package.
      - rezmason 3y ago
        Also, we may be able to create those packages from parts of browsers, which wouldn't be a re-implementation, and would benefit from the rigor of the standards. I agree that it would be kind of silly to then put this in the browser. Instead, I think it could be used to build the native target. And just to clarify my earlier statement, I'm not a fan of phasing out the use of the DOM in the browser— it has graceful degradation and accessibility features that are really important. It's more that there are people with existing Qt or JavaFX or Cocoa dependent projects, or whatever, who are now able to deploy to the web by implementing the UI layers in WASM/WebGPU.
croes 3y ago
You will always have the overhead otherwise you need to check with every browser update if something got broken. Same problem with WebView
udbhavs 3y ago
I wonder if application frameworks like Flutter will move to WebGPU? I imagine it shouldn't be that hard to get Skia running on a wgpu backend. The current web target generates a lot of markup that isn't really semantic or representative of a web app's structure with lots of canvases anyway, so I imagine moving to a uniform render target will make things smoother. They're already experimenting with Dart in WASM instead of transpiling to JS as well.
flohofwoe 3y ago
I'd like to see this too, but it's quite unlikely (it could have already happened with WebGL), the developer experience won't be much different than writing a native app on top of one of the native cross-platform libraries (like SDL2, GLFW or - shameless plug - the sokol headers).

danShumway 3y ago

This is also my fear, but... I don't know, I think the potential outweighs the risks. In some ways the problem with everyone trying to render to canvases and skip the DOM starts from education and a lack of native equivalents to the DOM that really genuinely showcase the strengths beyond having a similar API. I think developers come into the web and they have a particular mindset about graphics that pushes them away from "there should be a universal semantic layer that I talk to and also other things might talk to it", and instead the mindset is "I just want to put very specific pixels on the screen, and people shouldn't be using my application on weird screen configurations or with random extensions/customizations anyway." And I vaguely think that's something that needs to be solved more by just educating developers. It'll be a problem until something happens and native platforms either get support for a universal semantic application layer that's accessible to the user and developers start seeing the benefits of that, or... I don't know. That's maybe a long conversation. But there has to be a shift, I don't think it's feasible to just hold off on GPU features. At some point native developers need to figure out why the DOM matters or we'll just keep on having this debate. People wanting to write code that runs on both native and the web is good, it's a reasonable instinct. Electron-style app development isn't a bad goal. It's just how those apps get developed and what parts of the web get thrown out because developers aren't considering them to be important.

syntheweave 3y ago
When it comes down to it, I think most of our issue is that the technical details still matter enough that we can't afford to provide a universal human interface layer...and when that falls on the app developer, either they compromise the app or they compromise the user, because nobody is capable of filling in every rough edge and also doing the thing they set out to do in that moment. If they actually set out to create a fix, that quickly turns into their career. A lot of the "web frontend churn" meme is derived from the quest for a slightly better compromise - which has produced some results if what you are building resembles a web page. But over and over, native devs will say, "no, I want the hardware access. This layer still doesn't address my use case." That's been true of anything deeply I/O related on the Web - graphics, audio, input. Things where you can't make a simplifying assumption if you actually start covering all use cases.
- danShumway 3y ago
  > I think most of our issue is that the technical details still matter enough that we can't afford to provide a universal human interface layer I don't think this is true for the majority of native apps I run. I think a lot of developers think they need low-level control over exactly what pixels on the screen turn what colors, but I don't think most apps are in that category. Devs that use the GPU a lot are just used to working that way. There are cases where that kind of access is needed. I use WebGL right now, I will use WebGPU when it comes out. But how much of your desktop software genuinely, legitimately could not be described using a pure-text tree? Maybe Blender? Even an app like Krita really only needs precise control over pixels for the actual drawing part. It's mostly games and very experimental apps and visual content creation tools. Everything else is at most "we need a buffer rendered via the GPU embedded in this interface for one thing". My take on this is: if your app works with a screenreader, that app is probably representable via some kind of universal interface. And the advantages to having a layer like that on native platforms would be profound. People talk about how great the terminal is, imagine if you could do everything you can do on the terminal -- piping output between programs, scraping content, building wrappers around interfaces, easy remote access, etc -- imagine if you could do all of that for all of the GUI apps on your computer too. What we would lose in developer flexibility to define an initial interface we would make up for ten-fold with the additional flexibility developers and users would have to build around existing interfaces and to extend them. > A lot of the "web frontend churn" meme is derived from the quest for a slightly better compromise I'm also not sure this is entirely true. Yes, performance is a thing people talk about but systems like React came out of developers wanting a declarative model for building interfaces. In many ways it was going the opposite direction of canvas/webGL code. A lot of other libraries are similar -- you look at something like d3, that's not really about how things get drawn on the screen, what makes it special is data bindings. Debates about the virtual DOM are kind of a small subset of the churn that we see online, and my impression is that the churn is pushed more by people debating how to build interfaces in the first place and how to further abstract that process, not how this stuff should get rendered to the screen. We can have a high diversity of tools for building interfaces, what makes the DOM special is that it's universal render target. Whatever tool you use, the user gets the same semantics. And the DOM isn't necessarily a good universal render target. But it's kind of telling that even though the DOM has a ton of issues, the underlying philosophy of "you should have to describe what you want to put on the screen using text first and then we'll let you style it" is itself really empowering for end-users.

crazygringo 3y ago

The only possible application I can imagine for that would be videogames, though. Because HTML+CSS+JS provides a fantastic cross-platform UI toolkit that everybody knows how to use. Videogames create their own UI in order to have lots of shiny effects and a crazy immersive audio-filled controller-driven experience... but non-videogames don't need or want that. Heck, I'm actually expecting the opposite -- for the entire OS interface to become based on Chromium HTML+CSS+JS, and eventually Electron apps don't bundle a runtime, because they're just apps. My Synology DSM is an entire operating system whose user interface runs in the browser and it just... makes sense.

uoaei 3y ago
What is holding back a UI framework for WebGPU? It won't have HTML but couldn't you just simulate a scaffold inside the WebGPU canvas?
- iainmerrick 3y ago
  For most applications you'll want to do things like render text nicely, select text, copy and paste text, right-click to inspect, boost font sizes and change colors for accessibility, auto-translate to other languages. A lot of it is about text, I guess! Once you've re-implemented all that in a canvas, you've more or less built another web browser, so why not just use HTML?
  - uoaei 3y ago
    Presumably the argument in this thread is that lugging around an Electron app just for HTML is overkill if you could put the same functionality in a much smaller package.
    - rezmason 3y ago
      Also, we may be able to create those packages from parts of browsers, which wouldn't be a re-implementation, and would benefit from the rigor of the standards. I agree that it would be kind of silly to then put this in the browser. Instead, I think it could be used to build the native target. And just to clarify my earlier statement, I'm not a fan of phasing out the use of the DOM in the browser— it has graceful degradation and accessibility features that are really important. It's more that there are people with existing Qt or JavaFX or Cocoa dependent projects, or whatever, who are now able to deploy to the web by implementing the UI layers in WASM/WebGPU.

croes 3y ago

You will always have the overhead otherwise you need to check with every browser update if something got broken. Same problem with WebView

udbhavs 3y ago

I wonder if application frameworks like Flutter will move to WebGPU? I imagine it shouldn't be that hard to get Skia running on a wgpu backend. The current web target generates a lot of markup that isn't really semantic or representative of a web app's structure with lots of canvases anyway, so I imagine moving to a uniform render target will make things smoother. They're already experimenting with Dart in WASM instead of transpiling to JS as well.

flohofwoe 3y ago

I'd like to see this too, but it's quite unlikely (it could have already happened with WebGL), the developer experience won't be much different than writing a native app on top of one of the native cross-platform libraries (like SDL2, GLFW or - shameless plug - the sokol headers).