12 ms·

Feasibility of low-level GPU access on the Web

128 points by kvark 9y ago

Animats 9y ago
What problem does this solve for the end user? Few sites even use WebGL for anything interesting.
- na85 9y ago
  Most of these fads seem to be driven not by what's good for the user but what's good for the advertiser and the developer. Low level GPU access will just make it easier for them to run montero miners on my browser.
  - Animats 9y ago
    There's an ICO for that.[1] Really. They want to offer browser based virtual reality which mines tokens in the background. The initial token sale is in progress now. The virtual reality system appears to be total vaporware. [1] https://55cancri.io/ https://55cancri.io/
  - pitaj 9y ago
    Monero isn't efficiently mined on GPUs (it works better on CPUs which is why it's so popular for browser mining already), but other cryptos with equihash algorithms, sure.
- ghostwreck 9y ago
  The future of AR/VR exists in 3D worlds. If the web wants to be part of that future, developers will need access powerful 3D APIs. Otherwise, the web will miss another tech wave like it missed mobile (as native mobile was the clear winner over web mobile).
  - digi_owl 9y ago
    So we are back at VRML?
    - paulie_a 9y ago
      Which crashed and burned for a reason, it was incredibly pointless, current vr is just a novelty at a 2.0 stage
      - digi_owl 9y ago
        Yeah i am more excited about the AR glasses Intel recently unveiled than anything VR. But in the end the problem with be battery life...
- randomerr 9y ago
  I see it more as way to use the GPU to do calculations in JavaScript similar to the speed C/C++ does for crypto-currencies. Yes you can create a Native Application (NaCl) but that is only going to work in Chrome/Chromium browsers and it's compatible variants. Electron and NodeWebkit/NWJS based projects are the most common. But NaCl still doesn't give you direct GPU access because of the sandboxing.
- fabian2k 9y ago
  I've used WebGL to visualize scientific data. It's nothing special, but the size of the data was large enough that anything like SVG or even Canvas were just slow. There is a lot of interesting stuff you can do even just in 2D in WebGL, because you're closer to the hardware. What's annoying is that you don't get double precision or any of the compute stuff on WebGL currently. The interesting stuff isn't just graphics, but also doing computations directly in the browser.
  - KitDuncan 9y ago
    Do you have any resources on how you did that? I am developing a web app and I have to visualize a ton of datapoints in a bunch of charts and all the charting libraries I've tried are way too slow. I've been looking for something that could utilize WebGL to speed things up, but haven't found anything so far.
jcoffland 9y ago
For the Folding@home project we would love to have GPU compute access widely available from the browser.
- kvark 9y ago
  If you just care about GPU compute and not much about the shape of the API or it's overhead, then something like OpenGL ES 3.1 Compute Shaders or WebCL would be much easier to reach than WebGPU, technically.
  - johndough 9y ago
    Mozilla won't support WebCL in favor of compute shaders, but that announcement has been over 4 years ago and there still is no general compute in browsers: https://bugzilla.mozilla.org/show_bug.cgi?id=664147 https://bugzilla.mozilla.org/show_bug.cgi?id=664147
- gambiting 9y ago
  How many people do you think are actually interested in running the client in browser? I have Folding@Home on about 5-6 machines and I can't imagine a situation where I could run a web client but not a full client.
  - sitkack 9y ago
    It isn't for you. It is for the person who can open a tab, but doesn't want to run a full client. Installation, sandboxing, laziness, etc. A university or corp could push a tab open script to thousands of desk tops, while they wouldn't push Folding@Home client.
  - RandomInteger4 9y ago
    Chromebooks?
    - ComputerGuru 9y ago
      Those are pretty woefully underpowered for folding..
bobajeff 9y ago
I have this idea that if they split off WebAssembly, WebGL and WebAudio into a external format for applications it would be easier to implement than a full browser.
- rhelmer 9y ago
  You could probably do this now as a standalone Node app, if you want more explicit control over which APIs are available and where code can be run from. The browser is indeed quite complex (and not just because of the massive historical baggage), but its job is to give the user control while safely downloading untrusted code and running it locally. So, if your goal is to just have a simple standalone app I think you could stay largely compatible with APIs available in the browser environment.
  - bobajeff 9y ago
    Easier implementation is the goal. There are currently only four companies working on a web implementation. Also, I think users would have more control if documents didn't automatically gain the same privileges as applications.
    - rhelmer 9y ago
      > Easier implementation is the goal. There are currently only four companies working on a web implementation. Easier implementation of a browser? You might find it interesting to see what Servo has chosen to implement and what they have not. Some things you'd think would be easily removable (such as document.write) turn out to not be so simple to skip. One of the most valuable things about the Web is the care taken around backwards compatibility. I do think it'd be quite interesting if you had a user agent that did the DOM differently (not sure what you have in mind specifically re: "documents didn't automatically gain the same privileges as applications") and focused just on providing a GL canvas and audio APIs. I think you might find that these APIs aren't quite as nice when it comes to re-implementing things that CSS and DOM make easy, and it'd be hard for such a browser to really compete with existing browsers given the backwards-compat situation on the web (mandating GL would leave some devices behind, and web authors as a whole don't really adapt all that quickly). In any case I think it might still be useful as a reference implementation / proof-of-concept on how minimal a web user agent can be, if it was just focused on hosting applications.
sitkack 9y ago
Couldn't many of the GPU operation be accessible via functions over typed arrays? How does this relate to using SIMD from JS?
baybal2 9y ago
It has same downside as WASM. Running precompiled bytecode is a bad for security as it always been. It was no more than a year when a remotely exploitable WASM hole was exposed (derivatives of Spectre and co.) Knowledgeable people told that ISA level hole that can be exploited remotely over the web will be "a one minute global IT disaster" if somebody would resort to propagating it through a big adnet or paid traffic scheme. As for WebGL as it is now, there were numerous sites on my memory that froze/crashed/rebooted both Linux and Windows systems, which means that the prime suspect there was a buggy shader as it is the only thing resembling raw instructions that can be passed to gpu through webgl.
- Felz 9y ago
  Weren't Meltdown/Spectre exploitable via Javascript?
  - dralley 9y ago
    Yes
  - baybal2 9y ago
    Actually, the POC required both sharedbuffer object and ASM js (actually an even worse thing than a bytecode)
- jandrese 9y ago
  You are being downvoted, but it is important to remember that graphics APIs were not developed with security as a first class requirement. They tend to be large, arcane, and often interfacing with large binary blob drivers on the system. Their threat surface is enormous. IMHO it is just a matter of time until exploits for WebGL and the like start showing up regularly.
deleted 9y ago
[deleted]
happyguy43 9y ago
How does this interact with WASM?
Retr0spectrum 9y ago
Maybe GPUs need to be fundamentally redesigned to allow multi-user access with enforceable security boundaries. I'm sure this would be highly non-trivial to implement, but it would be great for other scenarios like shared access to GPU compute resources in "the cloud".
- kvark 9y ago
  This is called GPU virtualization, and there have been experiments ("Sugar" [1]) of using it for WebGL. We looked into it, consider it promising but not urgent. It addresses some of the security concerns at the cost of performance (and implementation complexity), but the portability issues are unchanged. [1] http://newport.eecs.uci.edu/~amowli/hpcfactory/publication/asplos18/ http://newport.eecs.uci.edu/~amowli/hpcfactory/publication/a...
- monocasa 9y ago
  That has existed for over a decade. In fact that's sort of the point of Vulkan, et al.; with all user code being behind an MMU context, it's safe to provide a more console like 'bare metal' API. There are bugs, but the infrastructure is all there. The end goal is that you can only crash your own process. The issue with this in the browser is that the API isn't part of the JavaScript/WebASM, and just exposing it in the same way will allow you to subvert the sandboxing of the VM>
c-smile 9y ago
This situation reminds me days when we, at W3C HTML5 WG, were trying to sneak SQL specification into HTML5 "umbrella". And one particular flavor of it - the SQLite's SQL as it was defined at the moment of writing. Haven't got through for many good reasons as we know. As of GPU exposure to the Web ... It makes sense only when we will have stable and unified GPU abstraction. As for now DirectX.12, Vulcan and Metal are close but different. Like the WebGL that is (more or less unified) OpenGL. But even that looks too foreign to HTML/CSS/script - immediate mode rendering in architecturally retained display model of web documents. And conceptually: HTML5 umbrella is large but not infinite. 3D rendering is too far from HTML "endless flat text tape" model. I remember those <applet> days when browser was used as a component delivery platform for stuff that does not fit into DOM and structured yet styled text. That was conceptually the right way "to grasp the immensity". These days, with WebAssembly, we have another incarnation of the <applet> idea and I think GPU functionality belongs to it rather than to HTML. GLSL to be expressed in WebAssembly bytecode terms but not in JS. Web standards have to be backward compatible and I doubt that current still ugly GPU paradigms will survive on the long run. Like tomorrow someone will come with practical voxel based system instead of current abstract vector ones, what will we do?
- revelation 9y ago
  This is a bit of a rambling mess, but to just make one point: the winner has been chosen. Android is implementing Vulkan, and so Vulkan it will be.
  - dleslie 9y ago
    It will be Vulcan and Dx12. Microsoft is nothing if not a stubborn 800 lbs gorilla.
    - revelation 9y ago
      Eh, not for WebGL it won't. Nobody even implements DX other than high-powered desktop GPUs. Will it live on? Sure, we will still have games and Xboxes. But if you're going to pick a standard that can work on mobile and desktop, there is no contender other than Vulkan.
fulafel 9y ago
WebGL 2 already gives you pretty good GPU gfx & compute. The use is limited by implementation quality in browsers. Let's give it time.
- Marat_Dukhan 9y ago
  WebGL 2 is based on OpenGL ES 3.0, it doesn't give you compute. Compute shaders were added in OpenGL ES 3.1
  - blarg1 9y ago
    khronos said they were working on compute for webgl2, who knows when it will be ready though.
  - fulafel 9y ago
    The OpenGL compute shader feature is not needed for doing GPU compute. It's just another type of shader that is not connected to other GL rendering that may be happening at the same time. People have been going GPGPU with the traditional types shaders for a long time. And WebGL 2.0 is a huge upgrade from 1.0 from a GPGPU point of view. Also, a compute shader extension may be coming[1] for WebGL 2.0, it's mentioned in some meeting slides at least: https://www.khronos.org/assets/uploads/developers/library/2017-gdc-webgl-webvr-gltf-meetup/2%20-%20WebGL%202.0%20@%20Silicon%20Valley%20WebGL%20Meetup%20-%20Mar17.pdf https://www.khronos.org/assets/uploads/developers/library/20...
    - johndough 9y ago
      > The OpenGL compute shader feature is not needed for doing GPU compute but it is needed for doing GPU compute efficiently for many workloads.
    - Marat_Dukhan 9y ago
      It is possible to perform some computations using OpenGL ES 3.0 / WebGL 2.0, but many types of operations (e.g. anything that involves random-access writes) are impossible, and many others (anything that normally requires shared memory) are very inefficient. Programming GPU through WebGL 2.0 is akin to programming desktop GPUs pre-CUDA: it is too intricate to take off. Compute shader extension for WebGL 2.0 would be cool, but it would require to port a large part of OpenGL ES 3.1: OpenGL ES 3.0 / WebGL 2.0 doesn't include even random access buffers (SSBOs)
swix 9y ago
Canvas and webgl is lovely in the way it quickly lets you get something on the screen... Pixel pushing on canvas is, easy and accessible. It makes me wonder if anyone has created some sort of port to a standalone app with no browser involved where you could use javascript/canvas-api/webgl to draw pixels on a canvas-like surface... without the fatness of the browser. Just spawning some window, that would be a lovely scripting/game-dev environment, maybe with some sdl-bindings or whatever. Anyway just rambling, does such a project exist? Anyone knows? :)
- tambourine_man 9y ago
  Canvas was introduced and inspired by Apple's drawing tech, so you might want to check that.
- icebraining 9y ago
  I suggest https://love2d.org/ https://love2d.org/
  - sitkack 9y ago
    I think you meant this, love2d running in the browser. http://tannerrogalsky.com/mari0/ http://tannerrogalsky.com/mari0/
- haxiomic 9y ago
  I built exactly that a few years ago: V8 with WebGL bindings that passed through to OpenGL ES. It was to enable WebGL experiences on the GearVR, where a full browser wouldn't cut it. I've got sample using an OS X window but it should be easy enough to do the same for Windows It's been sitting in a private repo since then but I can give you access if you'd like That being said, I think a better alternative for a thin graphics scripting environment would be haxe with Lime or snowkit (which provides OpenGL, SDL and windowing). I've used these in the past and loved working then
visarga 9y ago
Hmm... I wouldn't want a random webpage to start mining bitcoin on my computer. How would we protect against that?
- ramses0 9y ago
  Net Neutrality, right? It doesn't matter what the page does, it only matters that you are allowed to visit any page on "The Inter-Web" that you like, right? Safari has "tab paused/reloaded due to high power consumption" (good), Chrome has "auto-mute tabs" extension (which I have turned on), I have "tab suspender" extension installed. Basically, I'm trying to be a responsible consumer: - this tab would like access to your hard drive files - this tab would like access to your video camera - this tab would like access to your microphone - this tab would like to play sounds - this tab would like to download more than 5mb of data - this tab would like animation/movement - this tab would like to use your CPU a lot - this tab appears to be using a lot of your battery - this tab would like to use your GPU (at all) - this tab would like to maintain state > 24hr the_internet.js is actually potentially really hostile (suck 9999mb at full speed, ddos@1.2.3.4, while(1){alert(1)}, mine_bitcoin( $hacker_wallet )), and I am much in favor of treating it as untrusted by default (low access, limited # of cpu cycles) until "trusted" (ie: android permissions swap: i give you the executable, you give me the permissions). YouTube? Yes to whatever they ask. ShadyWebsite.1234.some-random-domain.ru? You can d/l 300kb and can't do anything else (ie: web 1.0/no-script). While each "tab" in a web-browser attempts to provide "safe" access to the computer resources, it is still not "permitted" access to computer resources. Whatever the browser defines as "safe" is 100% ok, which has to work equally well for WASM-unreal-tech-demo-castle as well as cnn.com. I'd prefer cnn.com only had 300kb download, no external domains, no sound, no battery, no cpu, etc. As I _trust_ cnn.com more (to the same level as youtube), I would then permit sound by default, permit large downloads, permit gpu, permit animation/video/etc.
z3t4 9y ago
i think the web browser is the modern terminal and it should make use of all capabilities of the device, including hardware accelerated rendering. i think the current webgl is too low level though and only needed if you want to make your own 3d engine. if it was a higher level 3d renderer it would be easier for browser wendors to make secure and optimize, and easier for developers to use.
- rimliu 9y ago
  And I start to think that web tech is becoming a fast food of software development.
matt_m 9y ago
Well, as a small developer I'd love to see a higher-level, easier-to-use API (than Vulkan) that can be used as a modern, cross platform replacement for OpenGL. If it could be used both in a browser and standalone even better. If it was available outside of a browser (even as a Vulkan/Metal wrapper), I think it could become a no-brainer replacement for where OpenGL ES is used today. I understand that engine developers can get some (small?) percent improvement from an ultra-low-level API that exposes more platform specific details. But they almost always support multiple APIs natively, and already use the low level ones where available. There may be a small performance benefit (over a higher level API) in the browser, but I don't think the browser is a likely target for ultra heavyweight apps / games, that usually are multi-GB downloads anyway, in the near future. And keep in mind that if smaller software sticks with WebGL because of API complexity, that might be a big performance loss for the user. But a higher level API would have great benefits everywhere! Right now the only real cross-platform graphics API is OpenGL ES 3.0, which is becoming more and more obsolete, with little sign of the situation changing. No compute shaders, no AZDO, etc. Any move beyond that feature set now requires multiple APIs, shaders etc. And the easiest way for a small developer to get those features is still to skip Vulkan and to stick with GL ES 3.1+ on Win/Linux/Android and choose Metal on Mac/iOS. Of course on the web for those features there are no options at all. I think if a new API was available that was easy to use and truly cross platform (including web), it would be the obvious first API to implement for all new graphics software. And this would be a much larger benefit than an unknown performance improvement that is accessible mostly to engine developers.
- Vekz 9y ago
  Not cross platform (WebGL only) but Regl: http://regl.party/ http://regl.party/ Is, In my opinion, the way GLSL GPU bindings should be done.
- zanny 9y ago
  If you are making a commercial product Vulkan + MoltenVK should be perfectly serviceable as a target. There are also free software attempts to make a wrapper. At this point, so long as Apple is in your target market, there will never be an everywhere API because Apple does not want there to be one. The whole point of Metal is to make your life harder so developers currently writing for Apple first are less likely to port their software elsewhere.
guidedlight 9y ago
There is an upcoming Xbox game (Crackdown 3) that uses web-based GPU processing to share compute amongst players in a multiplayer game.
- andrewmcwatters 9y ago
  What does that even mean? It's an Xbox, they all literally have the same variations of hardware. There aren't 8000 different build specs, there's at most, what 8? How could compute sharing provide any possible enhancement in experience at all? If anything, it's eating frame budget and it shouldn't.
  - stordoff 9y ago
    When it was first announced, there was talk of off-loading certain tasks to (presumably) Azure. That may be what OP is thinking of. I'd also wonder if you could share the power of any Xbox One Xs connected to a multiplayer session, given the gap between an original Xbox One and the X is rather large (seems like it'd be far more trouble than it's worth though).
deleted 9y ago
[deleted]
slavik81 9y ago
I don't understand why this is a priority when WebGL is still so rough. Maybe we wouldn't need a new API for performance if WebGL worked better. There seems to be lots of room for improvement. My WebGL programs were much slower and were harder to write than the native versions of the same programs. We should also probably sort out the native low-level APIs before setting the standard for the web, because otherwise we're building on top of a big mess. Though, my impression is that the WebGPU initiative is basically just another battleground in that struggle. I don't have any faith that this is being done for the good of users. It's just strategic ground to capture.
- stillkicking 9y ago
  Because WebGL is an evolutionary dead end, for a variety of reasons. The initial idea was to track OpenGL ES, but that isn't really true anymore. Because of Windows, WebGL has to stick to a subset that can be easily translated to Direct3D. Because of GPU process sandboxing, anything that flows back from GPU to CPU is a huge synchronization problem. On top of that, mobile GPU drivers continue to suck badly, which means most of the extensions that were supposed to expand WebGL's scope into this decade are still out of reach, with 50% or less support in practice. On the flipside, the native low-level APIs have all diverged. Vulkan, D3D12 and Metal each made different decisions, so it's pretty much inevitable that a 4th standard will have to be created to unify them. It will probably be higher level than any of the 3, and it will still be subject to strong sandboxing limitations. Personally I think the big issue is that people stare themselves blind on the traditional graphics pipeline. Modern renderers have evolved past this, with various compute-driven and/or tiled approaches common place now. They're a nightmare to implement against current APIs, because a small change in strategy requires a rewrite of much of the orchestration code. The job of figuring out how to map your desired pipeline onto the hardware's capabilities should be the job of a compiler, but instead people still do it by hand. Plus, for GPU compute-driven graphics to be useful for interactive purposes beyond just looking pretty (i.e. actual direct manipulation), you absolutely need to be able to read back structured data efficiently. It's not just about RGB pixels. There's an immense amount of potential locked inside, but the programming model is a decade or two out of date. Only AAA game companies and the vendors themselves have the resources to do novel work under these constraints. Everyone else has to throw together the scraps. Even the various attempts at LISPy GPU composition fall short, because they don't attempt to transcend the existing pipeline.

128 points by kvark 9y ago

Animats 9y ago

What problem does this solve for the end user? Few sites even use WebGL for anything interesting.

na85 9y ago
Most of these fads seem to be driven not by what's good for the user but what's good for the advertiser and the developer. Low level GPU access will just make it easier for them to run montero miners on my browser.
- Animats 9y ago
  There's an ICO for that.[1] Really. They want to offer browser based virtual reality which mines tokens in the background. The initial token sale is in progress now. The virtual reality system appears to be total vaporware. [1] https://55cancri.io/ https://55cancri.io/
- pitaj 9y ago
  Monero isn't efficiently mined on GPUs (it works better on CPUs which is why it's so popular for browser mining already), but other cryptos with equihash algorithms, sure.
ghostwreck 9y ago
The future of AR/VR exists in 3D worlds. If the web wants to be part of that future, developers will need access powerful 3D APIs. Otherwise, the web will miss another tech wave like it missed mobile (as native mobile was the clear winner over web mobile).
- digi_owl 9y ago
  So we are back at VRML?
  - paulie_a 9y ago
    Which crashed and burned for a reason, it was incredibly pointless, current vr is just a novelty at a 2.0 stage
    - digi_owl 9y ago
      Yeah i am more excited about the AR glasses Intel recently unveiled than anything VR. But in the end the problem with be battery life...
randomerr 9y ago
I see it more as way to use the GPU to do calculations in JavaScript similar to the speed C/C++ does for crypto-currencies. Yes you can create a Native Application (NaCl) but that is only going to work in Chrome/Chromium browsers and it's compatible variants. Electron and NodeWebkit/NWJS based projects are the most common. But NaCl still doesn't give you direct GPU access because of the sandboxing.
fabian2k 9y ago
I've used WebGL to visualize scientific data. It's nothing special, but the size of the data was large enough that anything like SVG or even Canvas were just slow. There is a lot of interesting stuff you can do even just in 2D in WebGL, because you're closer to the hardware. What's annoying is that you don't get double precision or any of the compute stuff on WebGL currently. The interesting stuff isn't just graphics, but also doing computations directly in the browser.
- KitDuncan 9y ago
  Do you have any resources on how you did that? I am developing a web app and I have to visualize a ton of datapoints in a bunch of charts and all the charting libraries I've tried are way too slow. I've been looking for something that could utilize WebGL to speed things up, but haven't found anything so far.

jcoffland 9y ago

For the Folding@home project we would love to have GPU compute access widely available from the browser.

kvark 9y ago
If you just care about GPU compute and not much about the shape of the API or it's overhead, then something like OpenGL ES 3.1 Compute Shaders or WebCL would be much easier to reach than WebGPU, technically.
- johndough 9y ago
  Mozilla won't support WebCL in favor of compute shaders, but that announcement has been over 4 years ago and there still is no general compute in browsers: https://bugzilla.mozilla.org/show_bug.cgi?id=664147 https://bugzilla.mozilla.org/show_bug.cgi?id=664147
gambiting 9y ago
How many people do you think are actually interested in running the client in browser? I have Folding@Home on about 5-6 machines and I can't imagine a situation where I could run a web client but not a full client.
- sitkack 9y ago
  It isn't for you. It is for the person who can open a tab, but doesn't want to run a full client. Installation, sandboxing, laziness, etc. A university or corp could push a tab open script to thousands of desk tops, while they wouldn't push Folding@Home client.
- RandomInteger4 9y ago
  Chromebooks?
  - ComputerGuru 9y ago
    Those are pretty woefully underpowered for folding..

bobajeff 9y ago

I have this idea that if they split off WebAssembly, WebGL and WebAudio into a external format for applications it would be easier to implement than a full browser.

rhelmer 9y ago
You could probably do this now as a standalone Node app, if you want more explicit control over which APIs are available and where code can be run from. The browser is indeed quite complex (and not just because of the massive historical baggage), but its job is to give the user control while safely downloading untrusted code and running it locally. So, if your goal is to just have a simple standalone app I think you could stay largely compatible with APIs available in the browser environment.
- bobajeff 9y ago
  Easier implementation is the goal. There are currently only four companies working on a web implementation. Also, I think users would have more control if documents didn't automatically gain the same privileges as applications.
  - rhelmer 9y ago
    > Easier implementation is the goal. There are currently only four companies working on a web implementation. Easier implementation of a browser? You might find it interesting to see what Servo has chosen to implement and what they have not. Some things you'd think would be easily removable (such as document.write) turn out to not be so simple to skip. One of the most valuable things about the Web is the care taken around backwards compatibility. I do think it'd be quite interesting if you had a user agent that did the DOM differently (not sure what you have in mind specifically re: "documents didn't automatically gain the same privileges as applications") and focused just on providing a GL canvas and audio APIs. I think you might find that these APIs aren't quite as nice when it comes to re-implementing things that CSS and DOM make easy, and it'd be hard for such a browser to really compete with existing browsers given the backwards-compat situation on the web (mandating GL would leave some devices behind, and web authors as a whole don't really adapt all that quickly). In any case I think it might still be useful as a reference implementation / proof-of-concept on how minimal a web user agent can be, if it was just focused on hosting applications.

sitkack 9y ago

Couldn't many of the GPU operation be accessible via functions over typed arrays? How does this relate to using SIMD from JS?

baybal2 9y ago

It has same downside as WASM. Running precompiled bytecode is a bad for security as it always been. It was no more than a year when a remotely exploitable WASM hole was exposed (derivatives of Spectre and co.) Knowledgeable people told that ISA level hole that can be exploited remotely over the web will be "a one minute global IT disaster" if somebody would resort to propagating it through a big adnet or paid traffic scheme. As for WebGL as it is now, there were numerous sites on my memory that froze/crashed/rebooted both Linux and Windows systems, which means that the prime suspect there was a buggy shader as it is the only thing resembling raw instructions that can be passed to gpu through webgl.

Felz 9y ago
Weren't Meltdown/Spectre exploitable via Javascript?
- dralley 9y ago
  Yes
- baybal2 9y ago
  Actually, the POC required both sharedbuffer object and ASM js (actually an even worse thing than a bytecode)
jandrese 9y ago
You are being downvoted, but it is important to remember that graphics APIs were not developed with security as a first class requirement. They tend to be large, arcane, and often interfacing with large binary blob drivers on the system. Their threat surface is enormous. IMHO it is just a matter of time until exploits for WebGL and the like start showing up regularly.

deleted 9y ago

[deleted]

happyguy43 9y ago

How does this interact with WASM?

Retr0spectrum 9y ago

Maybe GPUs need to be fundamentally redesigned to allow multi-user access with enforceable security boundaries. I'm sure this would be highly non-trivial to implement, but it would be great for other scenarios like shared access to GPU compute resources in "the cloud".

kvark 9y ago
This is called GPU virtualization, and there have been experiments ("Sugar" [1]) of using it for WebGL. We looked into it, consider it promising but not urgent. It addresses some of the security concerns at the cost of performance (and implementation complexity), but the portability issues are unchanged. [1] http://newport.eecs.uci.edu/~amowli/hpcfactory/publication/asplos18/ http://newport.eecs.uci.edu/~amowli/hpcfactory/publication/a...
monocasa 9y ago
That has existed for over a decade. In fact that's sort of the point of Vulkan, et al.; with all user code being behind an MMU context, it's safe to provide a more console like 'bare metal' API. There are bugs, but the infrastructure is all there. The end goal is that you can only crash your own process. The issue with this in the browser is that the API isn't part of the JavaScript/WebASM, and just exposing it in the same way will allow you to subvert the sandboxing of the VM>

c-smile 9y ago

This situation reminds me days when we, at W3C HTML5 WG, were trying to sneak SQL specification into HTML5 "umbrella". And one particular flavor of it - the SQLite's SQL as it was defined at the moment of writing. Haven't got through for many good reasons as we know. As of GPU exposure to the Web ... It makes sense only when we will have stable and unified GPU abstraction. As for now DirectX.12, Vulcan and Metal are close but different. Like the WebGL that is (more or less unified) OpenGL. But even that looks too foreign to HTML/CSS/script - immediate mode rendering in architecturally retained display model of web documents. And conceptually: HTML5 umbrella is large but not infinite. 3D rendering is too far from HTML "endless flat text tape" model. I remember those <applet> days when browser was used as a component delivery platform for stuff that does not fit into DOM and structured yet styled text. That was conceptually the right way "to grasp the immensity". These days, with WebAssembly, we have another incarnation of the <applet> idea and I think GPU functionality belongs to it rather than to HTML. GLSL to be expressed in WebAssembly bytecode terms but not in JS. Web standards have to be backward compatible and I doubt that current still ugly GPU paradigms will survive on the long run. Like tomorrow someone will come with practical voxel based system instead of current abstract vector ones, what will we do?

revelation 9y ago
This is a bit of a rambling mess, but to just make one point: the winner has been chosen. Android is implementing Vulkan, and so Vulkan it will be.
- dleslie 9y ago
  It will be Vulcan and Dx12. Microsoft is nothing if not a stubborn 800 lbs gorilla.
  - revelation 9y ago
    Eh, not for WebGL it won't. Nobody even implements DX other than high-powered desktop GPUs. Will it live on? Sure, we will still have games and Xboxes. But if you're going to pick a standard that can work on mobile and desktop, there is no contender other than Vulkan.

fulafel 9y ago

WebGL 2 already gives you pretty good GPU gfx & compute. The use is limited by implementation quality in browsers. Let's give it time.

Marat_Dukhan 9y ago
WebGL 2 is based on OpenGL ES 3.0, it doesn't give you compute. Compute shaders were added in OpenGL ES 3.1
- blarg1 9y ago
  khronos said they were working on compute for webgl2, who knows when it will be ready though.
- fulafel 9y ago
  The OpenGL compute shader feature is not needed for doing GPU compute. It's just another type of shader that is not connected to other GL rendering that may be happening at the same time. People have been going GPGPU with the traditional types shaders for a long time. And WebGL 2.0 is a huge upgrade from 1.0 from a GPGPU point of view. Also, a compute shader extension may be coming[1] for WebGL 2.0, it's mentioned in some meeting slides at least: https://www.khronos.org/assets/uploads/developers/library/2017-gdc-webgl-webvr-gltf-meetup/2%20-%20WebGL%202.0%20@%20Silicon%20Valley%20WebGL%20Meetup%20-%20Mar17.pdf https://www.khronos.org/assets/uploads/developers/library/20...
  - johndough 9y ago
    > The OpenGL compute shader feature is not needed for doing GPU compute but it is needed for doing GPU compute efficiently for many workloads.
  - Marat_Dukhan 9y ago
    It is possible to perform some computations using OpenGL ES 3.0 / WebGL 2.0, but many types of operations (e.g. anything that involves random-access writes) are impossible, and many others (anything that normally requires shared memory) are very inefficient. Programming GPU through WebGL 2.0 is akin to programming desktop GPUs pre-CUDA: it is too intricate to take off. Compute shader extension for WebGL 2.0 would be cool, but it would require to port a large part of OpenGL ES 3.1: OpenGL ES 3.0 / WebGL 2.0 doesn't include even random access buffers (SSBOs)

swix 9y ago

Canvas and webgl is lovely in the way it quickly lets you get something on the screen... Pixel pushing on canvas is, easy and accessible. It makes me wonder if anyone has created some sort of port to a standalone app with no browser involved where you could use javascript/canvas-api/webgl to draw pixels on a canvas-like surface... without the fatness of the browser. Just spawning some window, that would be a lovely scripting/game-dev environment, maybe with some sdl-bindings or whatever. Anyway just rambling, does such a project exist? Anyone knows? :)

tambourine_man 9y ago
Canvas was introduced and inspired by Apple's drawing tech, so you might want to check that.
icebraining 9y ago
I suggest https://love2d.org/ https://love2d.org/
- sitkack 9y ago
  I think you meant this, love2d running in the browser. http://tannerrogalsky.com/mari0/ http://tannerrogalsky.com/mari0/
haxiomic 9y ago
I built exactly that a few years ago: V8 with WebGL bindings that passed through to OpenGL ES. It was to enable WebGL experiences on the GearVR, where a full browser wouldn't cut it. I've got sample using an OS X window but it should be easy enough to do the same for Windows It's been sitting in a private repo since then but I can give you access if you'd like That being said, I think a better alternative for a thin graphics scripting environment would be haxe with Lime or snowkit (which provides OpenGL, SDL and windowing). I've used these in the past and loved working then

visarga 9y ago

Hmm... I wouldn't want a random webpage to start mining bitcoin on my computer. How would we protect against that?

ramses0 9y ago
Net Neutrality, right? It doesn't matter what the page does, it only matters that you are allowed to visit any page on "The Inter-Web" that you like, right? Safari has "tab paused/reloaded due to high power consumption" (good), Chrome has "auto-mute tabs" extension (which I have turned on), I have "tab suspender" extension installed. Basically, I'm trying to be a responsible consumer: - this tab would like access to your hard drive files - this tab would like access to your video camera - this tab would like access to your microphone - this tab would like to play sounds - this tab would like to download more than 5mb of data - this tab would like animation/movement - this tab would like to use your CPU a lot - this tab appears to be using a lot of your battery - this tab would like to use your GPU (at all) - this tab would like to maintain state > 24hr the_internet.js is actually potentially really hostile (suck 9999mb at full speed, ddos@1.2.3.4, while(1){alert(1)}, mine_bitcoin( $hacker_wallet )), and I am much in favor of treating it as untrusted by default (low access, limited # of cpu cycles) until "trusted" (ie: android permissions swap: i give you the executable, you give me the permissions). YouTube? Yes to whatever they ask. ShadyWebsite.1234.some-random-domain.ru? You can d/l 300kb and can't do anything else (ie: web 1.0/no-script). While each "tab" in a web-browser attempts to provide "safe" access to the computer resources, it is still not "permitted" access to computer resources. Whatever the browser defines as "safe" is 100% ok, which has to work equally well for WASM-unreal-tech-demo-castle as well as cnn.com. I'd prefer cnn.com only had 300kb download, no external domains, no sound, no battery, no cpu, etc. As I _trust_ cnn.com more (to the same level as youtube), I would then permit sound by default, permit large downloads, permit gpu, permit animation/video/etc.

z3t4 9y ago

i think the web browser is the modern terminal and it should make use of all capabilities of the device, including hardware accelerated rendering. i think the current webgl is too low level though and only needed if you want to make your own 3d engine. if it was a higher level 3d renderer it would be easier for browser wendors to make secure and optimize, and easier for developers to use.

rimliu 9y ago
And I start to think that web tech is becoming a fast food of software development.

matt_m 9y ago

Well, as a small developer I'd love to see a higher-level, easier-to-use API (than Vulkan) that can be used as a modern, cross platform replacement for OpenGL. If it could be used both in a browser and standalone even better. If it was available outside of a browser (even as a Vulkan/Metal wrapper), I think it could become a no-brainer replacement for where OpenGL ES is used today. I understand that engine developers can get some (small?) percent improvement from an ultra-low-level API that exposes more platform specific details. But they almost always support multiple APIs natively, and already use the low level ones where available. There may be a small performance benefit (over a higher level API) in the browser, but I don't think the browser is a likely target for ultra heavyweight apps / games, that usually are multi-GB downloads anyway, in the near future. And keep in mind that if smaller software sticks with WebGL because of API complexity, that might be a big performance loss for the user. But a higher level API would have great benefits everywhere! Right now the only real cross-platform graphics API is OpenGL ES 3.0, which is becoming more and more obsolete, with little sign of the situation changing. No compute shaders, no AZDO, etc. Any move beyond that feature set now requires multiple APIs, shaders etc. And the easiest way for a small developer to get those features is still to skip Vulkan and to stick with GL ES 3.1+ on Win/Linux/Android and choose Metal on Mac/iOS. Of course on the web for those features there are no options at all. I think if a new API was available that was easy to use and truly cross platform (including web), it would be the obvious first API to implement for all new graphics software. And this would be a much larger benefit than an unknown performance improvement that is accessible mostly to engine developers.

Vekz 9y ago
Not cross platform (WebGL only) but Regl: http://regl.party/ http://regl.party/ Is, In my opinion, the way GLSL GPU bindings should be done.
zanny 9y ago
If you are making a commercial product Vulkan + MoltenVK should be perfectly serviceable as a target. There are also free software attempts to make a wrapper. At this point, so long as Apple is in your target market, there will never be an everywhere API because Apple does not want there to be one. The whole point of Metal is to make your life harder so developers currently writing for Apple first are less likely to port their software elsewhere.

guidedlight 9y ago

There is an upcoming Xbox game (Crackdown 3) that uses web-based GPU processing to share compute amongst players in a multiplayer game.

andrewmcwatters 9y ago
What does that even mean? It's an Xbox, they all literally have the same variations of hardware. There aren't 8000 different build specs, there's at most, what 8? How could compute sharing provide any possible enhancement in experience at all? If anything, it's eating frame budget and it shouldn't.
- stordoff 9y ago
  When it was first announced, there was talk of off-loading certain tasks to (presumably) Azure. That may be what OP is thinking of. I'd also wonder if you could share the power of any Xbox One Xs connected to a multiplayer session, given the gap between an original Xbox One and the X is rather large (seems like it'd be far more trouble than it's worth though).

deleted 9y ago

[deleted]

slavik81 9y ago

I don't understand why this is a priority when WebGL is still so rough. Maybe we wouldn't need a new API for performance if WebGL worked better. There seems to be lots of room for improvement. My WebGL programs were much slower and were harder to write than the native versions of the same programs. We should also probably sort out the native low-level APIs before setting the standard for the web, because otherwise we're building on top of a big mess. Though, my impression is that the WebGPU initiative is basically just another battleground in that struggle. I don't have any faith that this is being done for the good of users. It's just strategic ground to capture.

stillkicking 9y ago
Because WebGL is an evolutionary dead end, for a variety of reasons. The initial idea was to track OpenGL ES, but that isn't really true anymore. Because of Windows, WebGL has to stick to a subset that can be easily translated to Direct3D. Because of GPU process sandboxing, anything that flows back from GPU to CPU is a huge synchronization problem. On top of that, mobile GPU drivers continue to suck badly, which means most of the extensions that were supposed to expand WebGL's scope into this decade are still out of reach, with 50% or less support in practice. On the flipside, the native low-level APIs have all diverged. Vulkan, D3D12 and Metal each made different decisions, so it's pretty much inevitable that a 4th standard will have to be created to unify them. It will probably be higher level than any of the 3, and it will still be subject to strong sandboxing limitations. Personally I think the big issue is that people stare themselves blind on the traditional graphics pipeline. Modern renderers have evolved past this, with various compute-driven and/or tiled approaches common place now. They're a nightmare to implement against current APIs, because a small change in strategy requires a rewrite of much of the orchestration code. The job of figuring out how to map your desired pipeline onto the hardware's capabilities should be the job of a compiler, but instead people still do it by hand. Plus, for GPU compute-driven graphics to be useful for interactive purposes beyond just looking pretty (i.e. actual direct manipulation), you absolutely need to be able to read back structured data efficiently. It's not just about RGB pixels. There's an immense amount of potential locked inside, but the programming model is a decade or two out of date. Only AAA game companies and the vendors themselves have the resources to do novel work under these constraints. Everyone else has to throw together the scraps. Even the various attempts at LISPy GPU composition fall short, because they don't attempt to transcend the existing pipeline.