5 ms·

No, it can't. WebGL is at a completely different level of the stack than Datashader. WebGL is even lower level than what most people use for 3D graphics (hence

by mhalle 8y ago

No, it can't. WebGL is at a completely different level of the stack than Datashader. WebGL is even lower level than what most people use for 3D graphics (hence threejs and babylonjs). Data visualization is at a higher semantic level than rendering; ideally you don't want to deal with pixels and polygons. D3, for instance, binds graphical primitives (usually in SVG) with data representations but requires more programming to do actual data visualization (and that's why a bunch of software layers on top of D3). Bokeh deals with still higher level primitives closer to the data set level (plotting and charts). And Datashader carves out a niche where there's too much data to have a 1:1 ratio of data element to graphical primitive on the screen. It does that by rasterizing, but then also handling the hard part of mapping backwards from image to data for selection and interactivity (I hope that's right; I got it from watching the 2016 video). Anyone who has had to do this stuff for a living knows it is hard to do right, and that good modular tools are always welcome. I don't see how this repeated "it's just like X" line of responses is benefiting the discussion. Datashader is not just like WebGL or basic low-level rendering, any more than D3 is just SVG or web apps are just TCP connections. Completely different levels of abstraction with lots of value add in between (and hard work, I'm sure).

BubRoss 8y ago
I didn't say that webgl would do everything this can do, I said it would do the things I copied from the post I replied to. Again, my issue is that they seem to put a lot of focus on this having some sort of sophisticated new rendering when it seems to be marketing of trivial techniques. People seem to like what this library does, but they didn't invent new rendering algorithms and their buzzwords and clever names just show a lack of awareness of what they are doing in the rendering department.
- jbednar 8y ago
  I'm not sure if you're objecting to the name "Datashader", but surely every library needs a name, and this one is accurate in that it allows the sort of shading that one does for 3D rendering to be applied to 2D data plotting. Or are there other buzzwords used in the docs you find objectionable?
  - BubRoss 8y ago
    If I said I was an expert in 'big data visualization with billions of points' and had written my own 'out of core' rendering library that I dubbed 'data shader', complete with a paper where I coined the term 'Abstract Rendering' or 'AR' for short, then you found out that I was just reading points from disk and drawing them with opengl's draw points function, what would you think? The term 'out of core' rendering comes from raytracing, where you really do need all the geometry available. They are applying it to trivial accumulation where it was never a problem in the first place. That's like me writing a paper on how to make a balloon air tight. That's how it has always worked, why would I take credit for something that was never a problem?
    - pwang 8y ago
      I think you are deliberately trying to misunderstand what is being done in this project. It's not about what APIs are being used to render whatever. At that level of analysis, all that anybody is every doing, is just doing memcpy and bitblt. Rather, datashader provides a framework for applying semantically meaningful, mathematical transformations on datasets as they're being accumulated, as those accumulations are converted into aesthetic/geom primitives, and as those primitives are rendered into colors. It really is "renderman for data", along with arbitrary vertex/texture shaders, driven by a dynamic rasterizer that can use whatever bins in data-space (not merely physical pixels). BTW "Out of core" does NOT come from raytracing; in fact its history in computing is a term for anything that exceeds physical memory. We use it all the time in scientific/HPC and data science because datasets are frequently much larger than available memory. https://en.wikipedia.org/wiki/External_memory_algorithm https://en.wikipedia.org/wiki/External_memory_algorithm
      - BubRoss 8y ago
        I think you are misrepresenting what is being done in this project. People seem to like it. They say it has workflow refinements. That's great, but there isn't anything new being done in rendering here unless doing something trivial in a pointlessly complex way and renaming fundamental techniques counts as a breakthrough. Focus on the workflow refinements, saying g there are rendering breakthroughs here is snake oil.

by mhalle 8y ago

BubRoss 8y ago

I didn't say that webgl would do everything this can do, I said it would do the things I copied from the post I replied to. Again, my issue is that they seem to put a lot of focus on this having some sort of sophisticated new rendering when it seems to be marketing of trivial techniques. People seem to like what this library does, but they didn't invent new rendering algorithms and their buzzwords and clever names just show a lack of awareness of what they are doing in the rendering department.

jbednar 8y ago
I'm not sure if you're objecting to the name "Datashader", but surely every library needs a name, and this one is accurate in that it allows the sort of shading that one does for 3D rendering to be applied to 2D data plotting. Or are there other buzzwords used in the docs you find objectionable?
- BubRoss 8y ago
  If I said I was an expert in 'big data visualization with billions of points' and had written my own 'out of core' rendering library that I dubbed 'data shader', complete with a paper where I coined the term 'Abstract Rendering' or 'AR' for short, then you found out that I was just reading points from disk and drawing them with opengl's draw points function, what would you think? The term 'out of core' rendering comes from raytracing, where you really do need all the geometry available. They are applying it to trivial accumulation where it was never a problem in the first place. That's like me writing a paper on how to make a balloon air tight. That's how it has always worked, why would I take credit for something that was never a problem?
  - pwang 8y ago
    I think you are deliberately trying to misunderstand what is being done in this project. It's not about what APIs are being used to render whatever. At that level of analysis, all that anybody is every doing, is just doing memcpy and bitblt. Rather, datashader provides a framework for applying semantically meaningful, mathematical transformations on datasets as they're being accumulated, as those accumulations are converted into aesthetic/geom primitives, and as those primitives are rendered into colors. It really is "renderman for data", along with arbitrary vertex/texture shaders, driven by a dynamic rasterizer that can use whatever bins in data-space (not merely physical pixels). BTW "Out of core" does NOT come from raytracing; in fact its history in computing is a term for anything that exceeds physical memory. We use it all the time in scientific/HPC and data science because datasets are frequently much larger than available memory. https://en.wikipedia.org/wiki/External_memory_algorithm https://en.wikipedia.org/wiki/External_memory_algorithm
    - BubRoss 8y ago
      I think you are misrepresenting what is being done in this project. People seem to like it. They say it has workflow refinements. That's great, but there isn't anything new being done in rendering here unless doing something trivial in a pointlessly complex way and renaming fundamental techniques counts as a breakthrough. Focus on the workflow refinements, saying g there are rendering breakthroughs here is snake oil.