3 ms·

Cool to see this :) We wrote the JS version of Arrow with some of these insights in mind. We even had a wasm + webcl backends even before building arrow and the

Cool to see this :) We wrote the JS version of Arrow with some of these insights in mind. We even had a wasm + webcl backends even before building arrow and then using it to standardize some of our protocols. Going in the wild these last few years, we've had a few fundamental lessons-learned here. A lot of the above is still true, and in fact even more so. At the same time, for the data world, there are tough points like client heterogeneity and data scale. The way I like to describe it now is we've moved from a "thin <> thick" client/server model to "thick <> supercomputer" client/server. With modern latencies and bandwidth, your interactive compute session is distributed GPUs: it's a weird way to build apps and think about what a computer is, but totally commoditized and doable now. So Arrow is great for streaming MB/s to the browser. Potentially in a couple years, GB/s to v8/browser, mostly waiting for some basic issues Chrome devs don't prioritize, not fundamental ones. This gets hard when we want to interactively visually analyze say 10 GB of click trails: that can quickly get into 100 GB of in-memory representation needing streaming / out-of-core tricks, and download speed issues. Likewise, WebGPU is cool, but it's still not even what CUDA / OpenCL were exposing 10 years ago, again, mostly b/c Google WebGL people prioritize other stuff. Given that, at least for the next few years, we've been continuing to build for "1-10 GB client GPU <> 10GB-1TB server GPUs". On the plus side, "on the internet, nobody knows you're a dog", so we've been free to do GPU nodejs extenstions that WebGL standards people don't get to dictate.

rektide 5y ago
> Potentially in a couple years, GB/s to v8/browser, mostly waiting for some basic issues Chrome devs don't prioritize, not fundamental ones. This gets hard when we want to interactively visually analyze say 10 GB of click trails: that can quickly get into 100 GB of in-memory representation needing streaming / out-of-core tricks, and download speed issues. what issues aren't being prioritized? > Likewise, WebGPU is cool, but it's still not even what CUDA / OpenCL were exposing 10 years ago, again, mostly b/c Google WebGL people prioritize other stuff. what's missing vs say opencl? a lot of shade cast with very few contestable claims. I want to listen to your points but right now the article seems far more right to me than your ill described counterclaims. I don't see the limits you claim with webgpu sand I don't see what the browser folk need to be attending to.
- lmeyerov 5y ago
  ex: As GPUs can now have 40-80GB RAM, it'd be great to create & move around big buffers, esp. using optimized i/o interfaces like cuFile. But way before then, we should be able to allocate a big buffer: https://stackoverflow.com/questions/8974375/whats-the-maximum-size-of-a-node-js-buffer#:~:text=3%20Answers&text=Maximum%20length%20of%20a%20typed,2Gb%20%2D%201byte%20on%2064%2Dbit https://stackoverflow.com/questions/8974375/whats-the-maximu... ex: gpu programming quickly gets into things like controlling reads/writes of data across warps as soon as you leave stuff like doing recursive fractals and wanting to do data structures designed for data parallelism, with some of the most elementary benefiting from this being things like on-gpu creation and manipulation of trees and graphs. I don't expect people learning/using webgl2 to want to do that, but I 100% expect the frameworks they're relying on to be doing it for how they implement things like reductions. I was checking the spec every year or so to see if it supports basic memory model pragma equivs but still no, so basically still a basic opengl mindset, which is fair, just not letting webdevs do late 2000's style gpgpu. I tried working w/ khronos etc. on standards here but Google & HW manufacturers killed it for effectively competitive + resourcing reasons. That was ~10 years ago, and main issues are still true. Nowadays, I'm more focused on areas we can control, and aren't subject to these standards bodies. NodeJS ecosystems are not restricted, and the ecosystem is incentivized to compete on advancing HW/SW in the data center. So instead, I get to work on questions like "so each PCI card gives us 16-64 GB/s bandwidth to a GPU, and the new SSD arrays give us 12 GB/s per controller, so if we do a few cheap GPUs on a box, can we do interactive visual analytics on that?" And "Is on-GPU decompression faster than sending uncompressed when all same-node?" I wish standards were building for this kind of stuff, but maybe you have a better idea on how to make it happen...

rektide 5y ago

> Potentially in a couple years, GB/s to v8/browser, mostly waiting for some basic issues Chrome devs don't prioritize, not fundamental ones. This gets hard when we want to interactively visually analyze say 10 GB of click trails: that can quickly get into 100 GB of in-memory representation needing streaming / out-of-core tricks, and download speed issues. what issues aren't being prioritized? > Likewise, WebGPU is cool, but it's still not even what CUDA / OpenCL were exposing 10 years ago, again, mostly b/c Google WebGL people prioritize other stuff. what's missing vs say opencl? a lot of shade cast with very few contestable claims. I want to listen to your points but right now the article seems far more right to me than your ill described counterclaims. I don't see the limits you claim with webgpu sand I don't see what the browser folk need to be attending to.

lmeyerov 5y ago
ex: As GPUs can now have 40-80GB RAM, it'd be great to create & move around big buffers, esp. using optimized i/o interfaces like cuFile. But way before then, we should be able to allocate a big buffer: https://stackoverflow.com/questions/8974375/whats-the-maximum-size-of-a-node-js-buffer#:~:text=3%20Answers&text=Maximum%20length%20of%20a%20typed,2Gb%20%2D%201byte%20on%2064%2Dbit https://stackoverflow.com/questions/8974375/whats-the-maximu... ex: gpu programming quickly gets into things like controlling reads/writes of data across warps as soon as you leave stuff like doing recursive fractals and wanting to do data structures designed for data parallelism, with some of the most elementary benefiting from this being things like on-gpu creation and manipulation of trees and graphs. I don't expect people learning/using webgl2 to want to do that, but I 100% expect the frameworks they're relying on to be doing it for how they implement things like reductions. I was checking the spec every year or so to see if it supports basic memory model pragma equivs but still no, so basically still a basic opengl mindset, which is fair, just not letting webdevs do late 2000's style gpgpu. I tried working w/ khronos etc. on standards here but Google & HW manufacturers killed it for effectively competitive + resourcing reasons. That was ~10 years ago, and main issues are still true. Nowadays, I'm more focused on areas we can control, and aren't subject to these standards bodies. NodeJS ecosystems are not restricted, and the ecosystem is incentivized to compete on advancing HW/SW in the data center. So instead, I get to work on questions like "so each PCI card gives us 16-64 GB/s bandwidth to a GPU, and the new SSD arrays give us 12 GB/s per controller, so if we do a few cheap GPUs on a box, can we do interactive visual analytics on that?" And "Is on-GPU decompression faster than sending uncompressed when all same-node?" I wish standards were building for this kind of stuff, but maybe you have a better idea on how to make it happen...