4 ms·

>either CSS filtering or WebGL CSS only has basic filters, and WebGL is only possible for per-pixel manipulation, such as brightness or contrast. This is becau

>either CSS filtering or WebGL CSS only has basic filters, and WebGL is only possible for per-pixel manipulation, such as brightness or contrast. This is because WebGL is just shaders, and half of shaders are per-pixel, along with some other fun things you can use with multiple buffers, but you can't actually get /back/ the processed data even if you got all the pixels into a single loop. With WebGL 2, this may be different due to some new buffer types, but I don't think its made for it. Any time you do an algorithm that is dependent upon surrounding pixels, such as dithering, you're stuck with JS. https://tools.czaux.com/space-pixels/ https://tools.czaux.com/space-pixels/ Here's an example of in-browser dithering to a fixed palette set that works pretty well, using RGBQuant modified with alpha support. The overall canvas uses FabricJS for manipulation.

dahart 10y ago
> but you can't actually get /back/ the processed data even if you got all the pixels into a single loop Oh if you need the pixels back, you can certainly get them using glReadPixels(). It's a bit expensive in WebGL land, though it is much less expensive than looping over pixels using JS. And often there are ways to get around needing the pixels back, depending on what you're doing. You can also do loads of non per-pixel tricks, including filters that need the surround, using multiple passes and clever shaders, you are not as limited with WebGL 1 as you think. https://5013.es/toys/dithering/ https://5013.es/toys/dithering/ https://www.shadertoy.com/view/MslGR8 https://www.shadertoy.com/view/MslGR8 This is what I'm talking about, you have to slice your program differently if you optimize it. Your high level structure and outermost loops will look different if you use WebGL instead of JS. And yes WebGL 2 is even better. But, this is all much trickier than writing your straightforward loops in JS. That's the price you pay for performance, your architecture will be kinda crazy if you do it in WebGL, and it will not be easy to go back if you discover you optimized for the wrong thing.
- devwastaken 10y ago
  Random-order dithering can be done in WebGL, but they are of poor quality. I am talking about error-diffusion dithering, where you are reading and writing pixels, and then re-reading those modified pixels that change the rest of the pixels. In WebGL, you can read the pixels of the image, but you cannot modify and read. You are returning the actual pixel you have modified, which is what glReadPixels() returns. You could use webgl, but you'd have to use glReadPixels() for every pixel modified so that the next pixel knows the new data.
  - dahart 10y ago
    Yes, ordered dither is easier in a shader than error diffusion, that's true, but error diffusion is definitely possible on a GPU. If you need it. Do you really need it? Why not still use the GPU to accelerate whatever parts you can? Even if you dither on the CPU, doing your color filters on the GPU instead of in JS could make the difference between interactive and not. I don't know what you're doing exactly, but when you do need dithering, it's usually the last thing you need to do before output, and it's usually less important that the dither is fast than most other operations you need in an image editor. BTW, random order is lower quality, and error diffusion is better, only for for very low res color palettes. If your result is more than 256 colors, it's irrelevant, and for high quality dithering, e.g., 16 bits per channel colors to 8 bits per channel for print, random ordered dithering is superior. https://community.arm.com/graphics/b/blog/posts/when-parallelism-gets-tricky-accelerating-floyd-steinberg-on-the-mali-gpu https://community.arm.com/graphics/b/blog/posts/when-paralle... http://www.cs.cmu.edu/~imisra/projects/dithering/HybridDithering.pdf http://www.cs.cmu.edu/~imisra/projects/dithering/HybridDithe... https://www.shadertoy.com/view/4dt3W7 https://www.shadertoy.com/view/4dt3W7 That last one is ugly, but it's a great proof of concept here. You could get error diffusion in a shader using 2 passes. The first pass you render to texture, and the second pass, you can sample the texture however you want. Note how in this example, the error diffusion propagates backwards from what you would normally do in JS or C++, because it's a shader looping over destination pixels, not code that loops over source pixels. > In WebGL, you can read the pixels of the image, but you cannot modify and read. I don't know exactly what you mean here. You can modify & read pixels using render to texture, or using multiple passes. Render to texture will be faster, if you can do it. Multiple round-trips from CPU to GPU and back will be slower, so you want to limit the number of trips, but it's easy to do.
    - devwastaken 10y ago
      >Even if you dither on the CPU, doing your color filters on the GPU instead of in JS could make the difference between interactive and not. Dithering requires quantization. If you mean things like brightness and contrast, yes, webgl is better for that. But quantization with error diffusion dithering is still based upon previous modified pixels, so you cannot just send a bunch of pixel info to webgl, you have to do each pixel seperately. Meaning, pushing through 2.5Million seperate inputs sequentially, and having to use glReadPixels() 2.5Million times. >BTW, random order is lower quality, and error diffusion is better, only for for very low res color palettes. Not in any of the images I have used. Ordered dithering looks fake, because it appears like a texture to the image. Dithering with error diffusion kernels, like Floyd-Steinberg create a less obvious texturization that still preserves the underlying image. >https://www.shadertoy.com/view/4dt3W7 https://www.shadertoy.com/view/4dt3W7 From what I'm reading, for every pixel, it is processing a 250 iteration loop that gets the pixels in the row and builds errors based upon those. However, it is not modifying any pixels as it goes, this is a parallel operation independent of each other. Good error diffusion with a kernel requires previous pixels to be modified, and errors to be built upon them, which is why it is a sequential operation. I don't see any destination-dependant pixel manipulation. >Multiple round-trips from CPU to GPU and back will be slower, so you want to limit the number of trips, but it's easy to do. If its possible to get data back from Uniform Buffer Objects in WebGL2, it may be possible to send up to 1000 or so uniform pixel values, but that is the general cap. Some GPU's have less. Unless I am missing some magic buffer you can use to write out many pixels too, and in that case I would be very interested in testing that, but shaders are designed to output a single pixel per instance. >https://community.arm.com/graphics/b/blog/posts/when-parallelism-gets-tricky-accelerating-floyd-steinberg-on-the-mali-gpu https://community.arm.com/graphics/b/blog/posts/when-paralle... From what I'm reading, this is using OpenCL, and parrallelizing in rows, but these rows have height. A single thread processes one row and sequentially goes through the pixels like normal error diffusion. Just that this breaks different parts of the image up. I may have to try this out in JS and see if perhaps it can thread better that way. ty.

dahart 10y ago

> but you can't actually get /back/ the processed data even if you got all the pixels into a single loop Oh if you need the pixels back, you can certainly get them using glReadPixels(). It's a bit expensive in WebGL land, though it is much less expensive than looping over pixels using JS. And often there are ways to get around needing the pixels back, depending on what you're doing. You can also do loads of non per-pixel tricks, including filters that need the surround, using multiple passes and clever shaders, you are not as limited with WebGL 1 as you think. https://5013.es/toys/dithering/ https://5013.es/toys/dithering/ https://www.shadertoy.com/view/MslGR8 https://www.shadertoy.com/view/MslGR8 This is what I'm talking about, you have to slice your program differently if you optimize it. Your high level structure and outermost loops will look different if you use WebGL instead of JS. And yes WebGL 2 is even better. But, this is all much trickier than writing your straightforward loops in JS. That's the price you pay for performance, your architecture will be kinda crazy if you do it in WebGL, and it will not be easy to go back if you discover you optimized for the wrong thing.

devwastaken 10y ago
Random-order dithering can be done in WebGL, but they are of poor quality. I am talking about error-diffusion dithering, where you are reading and writing pixels, and then re-reading those modified pixels that change the rest of the pixels. In WebGL, you can read the pixels of the image, but you cannot modify and read. You are returning the actual pixel you have modified, which is what glReadPixels() returns. You could use webgl, but you'd have to use glReadPixels() for every pixel modified so that the next pixel knows the new data.
- dahart 10y ago
  Yes, ordered dither is easier in a shader than error diffusion, that's true, but error diffusion is definitely possible on a GPU. If you need it. Do you really need it? Why not still use the GPU to accelerate whatever parts you can? Even if you dither on the CPU, doing your color filters on the GPU instead of in JS could make the difference between interactive and not. I don't know what you're doing exactly, but when you do need dithering, it's usually the last thing you need to do before output, and it's usually less important that the dither is fast than most other operations you need in an image editor. BTW, random order is lower quality, and error diffusion is better, only for for very low res color palettes. If your result is more than 256 colors, it's irrelevant, and for high quality dithering, e.g., 16 bits per channel colors to 8 bits per channel for print, random ordered dithering is superior. https://community.arm.com/graphics/b/blog/posts/when-parallelism-gets-tricky-accelerating-floyd-steinberg-on-the-mali-gpu https://community.arm.com/graphics/b/blog/posts/when-paralle... http://www.cs.cmu.edu/~imisra/projects/dithering/HybridDithering.pdf http://www.cs.cmu.edu/~imisra/projects/dithering/HybridDithe... https://www.shadertoy.com/view/4dt3W7 https://www.shadertoy.com/view/4dt3W7 That last one is ugly, but it's a great proof of concept here. You could get error diffusion in a shader using 2 passes. The first pass you render to texture, and the second pass, you can sample the texture however you want. Note how in this example, the error diffusion propagates backwards from what you would normally do in JS or C++, because it's a shader looping over destination pixels, not code that loops over source pixels. > In WebGL, you can read the pixels of the image, but you cannot modify and read. I don't know exactly what you mean here. You can modify & read pixels using render to texture, or using multiple passes. Render to texture will be faster, if you can do it. Multiple round-trips from CPU to GPU and back will be slower, so you want to limit the number of trips, but it's easy to do.
  - devwastaken 10y ago
    >Even if you dither on the CPU, doing your color filters on the GPU instead of in JS could make the difference between interactive and not. Dithering requires quantization. If you mean things like brightness and contrast, yes, webgl is better for that. But quantization with error diffusion dithering is still based upon previous modified pixels, so you cannot just send a bunch of pixel info to webgl, you have to do each pixel seperately. Meaning, pushing through 2.5Million seperate inputs sequentially, and having to use glReadPixels() 2.5Million times. >BTW, random order is lower quality, and error diffusion is better, only for for very low res color palettes. Not in any of the images I have used. Ordered dithering looks fake, because it appears like a texture to the image. Dithering with error diffusion kernels, like Floyd-Steinberg create a less obvious texturization that still preserves the underlying image. >https://www.shadertoy.com/view/4dt3W7 https://www.shadertoy.com/view/4dt3W7 From what I'm reading, for every pixel, it is processing a 250 iteration loop that gets the pixels in the row and builds errors based upon those. However, it is not modifying any pixels as it goes, this is a parallel operation independent of each other. Good error diffusion with a kernel requires previous pixels to be modified, and errors to be built upon them, which is why it is a sequential operation. I don't see any destination-dependant pixel manipulation. >Multiple round-trips from CPU to GPU and back will be slower, so you want to limit the number of trips, but it's easy to do. If its possible to get data back from Uniform Buffer Objects in WebGL2, it may be possible to send up to 1000 or so uniform pixel values, but that is the general cap. Some GPU's have less. Unless I am missing some magic buffer you can use to write out many pixels too, and in that case I would be very interested in testing that, but shaders are designed to output a single pixel per instance. >https://community.arm.com/graphics/b/blog/posts/when-parallelism-gets-tricky-accelerating-floyd-steinberg-on-the-mali-gpu https://community.arm.com/graphics/b/blog/posts/when-paralle... From what I'm reading, this is using OpenCL, and parrallelizing in rows, but these rows have height. A single thread processes one row and sequentially goes through the pixels like normal error diffusion. Just that this breaks different parts of the image up. I may have to try this out in JS and see if perhaps it can thread better that way. ty.