6 ms·

this sounds cool, but why do we need it? why can't the web directly expose opengl?

fulafel 7y ago
Untrusted JS can't be given direct access to native graphics APIs because they are not hardened against malicious code. (Or even benign code, it sometimes feels like...) Hence WebGL/WebGPU.
- sesuximo 7y ago
  this was my first thought, but I don't know what kind of malicious code a GPU can run
  - klhiouyter 7y ago
    First of all, your code passes through the driver, so you can exploit vulnerabilities there. Even after that, the GPU can do DMA requests, so you can abuse that to read/write arbitrary memory locations.
    - sesuximo 7y ago
      ok makes sense. didn't know it could access arbitrary memory
  - fulafel 7y ago
    Graphics APIs are defined and implemented in memory-unsafe languages on the CPU side, C/C++, and will read from and write to memory based on data and pointers you pass to them without any provisions for untrusted data. -> instant pwnage (Not to say GPU code execution is safe, but you wouldn't need it to ruin the users day)
  - kevingadd 7y ago
    GPUs effectively have general-purpose compute now and they can DMA bytes into and out of system memory. At a basic level, sneaky on-GPU code run by an attacker could grab pixels off your screen (text from your emails, etc), but it's also possible for the GPU-based attack code to harvest data from system memory or even mess with data in order to attack a process running on the CPU.
saurik 7y ago
Assuming the "directly" wasn't the most important part of your comment, that is WebGL: WebGPU is designed to be more like Vulkan/Metal (with an initial prototype having been done by Apple, that has since been renamed WebMetal).
- kvark 7y ago
  WebGPU was prototyped by all parties, not just Apple. See https://en.m.wikipedia.org/wiki/WebGPU https://en.m.wikipedia.org/wiki/WebGPU
  - saurik 7y ago
    No. It doesn't matter for the context, but if you are going to try to correct this, from the very link you pasted: > On February 7, 2017, Apple's WebKit team proposed the creation of the W3C community group to design the API. At the same time they announced a technical proof of concept and proposal under the name "WebGPU", based on concepts in Apple's Metal.[5][6][7] The WebGPU name was later adopted by the community group as a working name for the future standard rather than just Apple's initial proposal.[2] The initial proposal has been renamed to "WebMetal" to avoid further confusion.[8] > The W3C "GPU for the Web" Community Group was launched on February 16, 2017. At this time, all of Apple, Google, and Mozilla had experiments in the area, but only Apple's proposal was officially submitted to the "gpuweb-proposals" repository.[9][10][11] Shortly after, on March 21, 2017, Mozilla submitted a proposal for WebGL Next within Khronos repository, based on the Vulkan design.[12][13]
    - kvark 7y ago
      You must be reading this wrong. The quoted text says: 1. WebGPU name was taken from Apple 2. Apple initiated the working group in W3C (by refusing to work with Khronos). What this does NOT say is: - Apple's prototype was the basis for the API we develop
mschuetz 7y ago
OpenGL kind of sucks, tbh. I know because I work full time with it. There are so many things in OpenGL that are either inneficient, outdated, or plain stupid. You also can't directly expose it anyway because of poor support on Windows and MacOS. The former requires you to install proper drivers that users may not have, the latter deprecated it and to my knowledge already didn't properly support the latest versions. So whatever you do in the browser, it would need some kind of translation layer to another API anyway. On windows that's usually DirectX via ANGLE. Even today, whatever you do with WebGL will actually be translated to DirectX calls. On MacOS, WebGL is probably be translated to Metal, right now. Same with Vulkan.
- sesuximo 7y ago
  it makes sense to me that people would want a cross platform way to write GPU code (I thought that was the point of opengl). I don't see why the web is a special case of this. probably you can tell I don't work with gpus I'm just curious about them
- etaioinshrdlu 7y ago
  I'll be the first to agree that OpenGL is kind of garbage, but I will say that there's a lot of value in having a stable API over decades. I like platforms that stay backward compatible long-term, like web tech. So given that, I don't really see anything wrong per se with webGL, apart from it being ugly, and I think we should just put up with that or abstract it. It would also be great if webGPU becomes a big thing and we can just use old webGL code through a compatibility layer, either in JS or built into browsers.
- jdashg 7y ago
  On macOS WebGL in all browsers goes through OpenGL, same as on Linux and Android. On Windows, WebGL generally calls into ANGLE-on-D3D, though both Firefox and Chrome support using "native GL" experimentally as well.
rezmason 7y ago
Good question! Maybe with my side project, I can shed some light on the state of OpenGL exposed through the web: I maintain a Matrix code rain project: https://github.com/Rezmason/matrix https://github.com/Rezmason/matrix . The effect is basically a compute shader with post-processing steps applied on top of it. It's built on top of WebGL via ThreeJS, and I can attest that its cross-browser, cross-device support is currently terrible. That's because ThreeJS's (otherwise nice!) compute shader implementation is written on top of WebGL 1's fragment shaders, rendering to floating point buffers enabled through the "oes_texture_float" extension. This extension's implementation has been plagued by spec mistakes, made possible (I believe) by ambiguities and tacit assumptions about differences between the speccing process of OGLES 2 and WebGL 1 that proved incorrect. Fixing it has frayed implementors' constitutions and made the API more complicated; WebGL 2 incorporates this extension's behavior into its core, but isn't widely adopted, because (in my opinion) asking a browser maker to double down on their investment in WebGL again after the aforementioned debacle is a tall order. Or maybe there are security implications. Or maybe the only Mobile Safari programmers available decided OpenGL is a weird-shaped API to expose through JavaScript, and have lost interest. One can only speculate. Anyway! In the meantime, to do any sort of GPU compute across all major browsers (including mobile), you're way better off enabling the "oes_texture_half_float" extension instead. Half floats, or Float16s, are common in the mobile graphics sphere, but not on the desktop, where most web development takes place; it's entirely understandable why a web developer might try and use "oes_texture_float", get it working on their desktop machines, fail to see it working in Mobile Safari, assume that the extension isn't supported, and call it a day. And even if they DO find the Stack Overflow answers that'd prompt them to switch to "oes_texture_half_float", JavaScript has no native Float16Array data view, so they have to make/download a slow JavaScript implementation or spin their own float to half float converter. This is the state of things with WebGL.
- fulafel 7y ago
  WebGL 2 is pretty widely implemented. I think Safari is the only remaining laggard now that MS switched to Chrome based Edge? It's true that WebGL support has been slow to mature in browsers, as will WebGPU be I fear...

Untrusted JS can't be given direct access to native graphics APIs because they are not hardened against malicious code. (Or even benign code, it sometimes feels like...) Hence WebGL/WebGPU.

sesuximo 7y ago
this was my first thought, but I don't know what kind of malicious code a GPU can run
- klhiouyter 7y ago
  First of all, your code passes through the driver, so you can exploit vulnerabilities there. Even after that, the GPU can do DMA requests, so you can abuse that to read/write arbitrary memory locations.
  - sesuximo 7y ago
    ok makes sense. didn't know it could access arbitrary memory
- fulafel 7y ago
  Graphics APIs are defined and implemented in memory-unsafe languages on the CPU side, C/C++, and will read from and write to memory based on data and pointers you pass to them without any provisions for untrusted data. -> instant pwnage (Not to say GPU code execution is safe, but you wouldn't need it to ruin the users day)
- kevingadd 7y ago
  GPUs effectively have general-purpose compute now and they can DMA bytes into and out of system memory. At a basic level, sneaky on-GPU code run by an attacker could grab pixels off your screen (text from your emails, etc), but it's also possible for the GPU-based attack code to harvest data from system memory or even mess with data in order to attack a process running on the CPU.

saurik 7y ago

Assuming the "directly" wasn't the most important part of your comment, that is WebGL: WebGPU is designed to be more like Vulkan/Metal (with an initial prototype having been done by Apple, that has since been renamed WebMetal).

kvark 7y ago
WebGPU was prototyped by all parties, not just Apple. See https://en.m.wikipedia.org/wiki/WebGPU https://en.m.wikipedia.org/wiki/WebGPU
- saurik 7y ago
  No. It doesn't matter for the context, but if you are going to try to correct this, from the very link you pasted: > On February 7, 2017, Apple's WebKit team proposed the creation of the W3C community group to design the API. At the same time they announced a technical proof of concept and proposal under the name "WebGPU", based on concepts in Apple's Metal.[5][6][7] The WebGPU name was later adopted by the community group as a working name for the future standard rather than just Apple's initial proposal.[2] The initial proposal has been renamed to "WebMetal" to avoid further confusion.[8] > The W3C "GPU for the Web" Community Group was launched on February 16, 2017. At this time, all of Apple, Google, and Mozilla had experiments in the area, but only Apple's proposal was officially submitted to the "gpuweb-proposals" repository.[9][10][11] Shortly after, on March 21, 2017, Mozilla submitted a proposal for WebGL Next within Khronos repository, based on the Vulkan design.[12][13]
  - kvark 7y ago
    You must be reading this wrong. The quoted text says: 1. WebGPU name was taken from Apple 2. Apple initiated the working group in W3C (by refusing to work with Khronos). What this does NOT say is: - Apple's prototype was the basis for the API we develop

mschuetz 7y ago

OpenGL kind of sucks, tbh. I know because I work full time with it. There are so many things in OpenGL that are either inneficient, outdated, or plain stupid. You also can't directly expose it anyway because of poor support on Windows and MacOS. The former requires you to install proper drivers that users may not have, the latter deprecated it and to my knowledge already didn't properly support the latest versions. So whatever you do in the browser, it would need some kind of translation layer to another API anyway. On windows that's usually DirectX via ANGLE. Even today, whatever you do with WebGL will actually be translated to DirectX calls. On MacOS, WebGL is probably be translated to Metal, right now. Same with Vulkan.

sesuximo 7y ago
it makes sense to me that people would want a cross platform way to write GPU code (I thought that was the point of opengl). I don't see why the web is a special case of this. probably you can tell I don't work with gpus I'm just curious about them
etaioinshrdlu 7y ago
I'll be the first to agree that OpenGL is kind of garbage, but I will say that there's a lot of value in having a stable API over decades. I like platforms that stay backward compatible long-term, like web tech. So given that, I don't really see anything wrong per se with webGL, apart from it being ugly, and I think we should just put up with that or abstract it. It would also be great if webGPU becomes a big thing and we can just use old webGL code through a compatibility layer, either in JS or built into browsers.
jdashg 7y ago
On macOS WebGL in all browsers goes through OpenGL, same as on Linux and Android. On Windows, WebGL generally calls into ANGLE-on-D3D, though both Firefox and Chrome support using "native GL" experimentally as well.

rezmason 7y ago

Good question! Maybe with my side project, I can shed some light on the state of OpenGL exposed through the web: I maintain a Matrix code rain project: https://github.com/Rezmason/matrix https://github.com/Rezmason/matrix . The effect is basically a compute shader with post-processing steps applied on top of it. It's built on top of WebGL via ThreeJS, and I can attest that its cross-browser, cross-device support is currently terrible. That's because ThreeJS's (otherwise nice!) compute shader implementation is written on top of WebGL 1's fragment shaders, rendering to floating point buffers enabled through the "oes_texture_float" extension. This extension's implementation has been plagued by spec mistakes, made possible (I believe) by ambiguities and tacit assumptions about differences between the speccing process of OGLES 2 and WebGL 1 that proved incorrect. Fixing it has frayed implementors' constitutions and made the API more complicated; WebGL 2 incorporates this extension's behavior into its core, but isn't widely adopted, because (in my opinion) asking a browser maker to double down on their investment in WebGL again after the aforementioned debacle is a tall order. Or maybe there are security implications. Or maybe the only Mobile Safari programmers available decided OpenGL is a weird-shaped API to expose through JavaScript, and have lost interest. One can only speculate. Anyway! In the meantime, to do any sort of GPU compute across all major browsers (including mobile), you're way better off enabling the "oes_texture_half_float" extension instead. Half floats, or Float16s, are common in the mobile graphics sphere, but not on the desktop, where most web development takes place; it's entirely understandable why a web developer might try and use "oes_texture_float", get it working on their desktop machines, fail to see it working in Mobile Safari, assume that the extension isn't supported, and call it a day. And even if they DO find the Stack Overflow answers that'd prompt them to switch to "oes_texture_half_float", JavaScript has no native Float16Array data view, so they have to make/download a slow JavaScript implementation or spin their own float to half float converter. This is the state of things with WebGL.

fulafel 7y ago
WebGL 2 is pretty widely implemented. I think Safari is the only remaining laggard now that MS switched to Chrome based Edge? It's true that WebGL support has been slow to mature in browsers, as will WebGPU be I fear...