6 ms·

What do people actually use Web Workers for? The examples I've seen, including this one, seem contrived. I keep hoping they'll change it to allow background ima

What do people actually use Web Workers for? The examples I've seen, including this one, seem contrived. I keep hoping they'll change it to allow background image manipulation, but I haven't seen much real progress in that front.

deleted 10y ago
[deleted]
Klathmon 10y ago
I've used it for background image manipulation before. Paint the image to a canvas, then grab the imagedata off of it, split it into as many parts as you have threads, then use the "transferrableObjects" property of postMessage to zero-copy transfer the data to each worker to be processed, transferred back, and re-stitched together. It's pretty powerful and suprisingly easy to work with once you understand it. [0] is a snippet from the code, but be gentile... It was a personal project where I was trying out polymer 0.5 and made a lot of questionable design choices... Also, I've heard of the idea of using webworkers as a "first class" platform. That is do all of the core parts of your application in them and only use the "main" thread as a "ui" thread. I haven't gotten a chance to try it out, but it seems like a great idea that could really work well in some SPAs. [0] https://github.com/Klathmon/stitchpics/blob/master/app/elements/cross-stitch/element.js#L95-L118 https://github.com/Klathmon/stitchpics/blob/master/app/eleme...
- bartread 10y ago
  That's interesting but, AFAIK (and, believe me, I'd be happy to be corrected), what you can't do is create a canvas element (even one not attached to the DOM), and paint directly to it using the standard 2D context and drawing primitives. Like I say, more than happy to be corrected, because that sort of thing would be incredibly helpful. Really, anything that lets you mess with a disconnected DOM in the background and then attach it in the foreground could be useful but (and, again, I'm very happy to be corrected), I don't think you can do this.
  - vanderZwan 10y ago
    You're talking about OffscreenCanvas. It's available in Firefox and nowhere else: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCa... Having said that, if you want fast image manipulation, you're probably better off doing direct manipulation of Uint8Clamped arrays anyway, since that can be much faster.
    - Klathmon 10y ago
      And if you can get away with it and don't mind a lot of bit shifting, it's even better working with the Uint32array which packs all 3 colors and alpha into one element which reduces your loops by 4X
      - vanderZwan 10y ago
        Because matching the native 32 word size is better for the prefetcher, right? Wouldn't most CPU's these days be smart enough to detect advancing the index by 4, and then using offsets? So: for(let i = 0; i < someUint8Array.length; i += 4){ let R = someUint8Array[i], G = someUint8Array[i+1], B = someUint8Array[i+2], A = someUint8Array[i+3]; // ... manipulations here }
        Klathmon 10y ago
        I'm honestly not sure why it is, but across all browsers on both ARM and x86_64 arches it was almost 3x faster than doing what you wrote. I have a feeling it's more of a JS JIT thing than a CPU prefetcher thing, but honestly I'm not really sure. In my program I linked above, it was actually faster to use Uint32array everywhere and then use functions to pull the 4 color values from it and another function to push the 4 values back to a uint32. Granted, it's been over a year since I last benchmarked that code, but I did reuse some of the image code recently and found iterating over a Uint32array to be significantly faster. (And funnily enough, manually unrolling the loop of Uint32array to something similar to what you wrote gave an additional small performance boost, but it was small enough to be not worth the extra weirdness in the code to me)
phpnode 10y ago
Anything that does significant work that would otherwise result in an unresponsive UI. For example code on the following page is compiled as you type, without a webworker this would give a very horrible user experience because compilation can take a long time for very large input - https://codemix.github.io/flow-runtime/#/try https://codemix.github.io/flow-runtime/#/try
- deleted 10y ago
  [deleted]
SirensOfTitan 10y ago
Maybe heavy JSON parsing? Better to parse off UI-thread than drop a frame. * Did not read article yet and am not sure about SharedArray structure.
deleted 10y ago
[deleted]
vanderZwan 10y ago
Well, I'm not using them yet, but I working on a SPA where I have multiple react-virtualized-select[0] components, each with thousands of options. Creating the fastFilter options[1] for those takes multiple seconds. Although I only do it once and then re-use it, that still freezes up the page at the start, so if I could do that in parallel without freezing the UI that would improve the app for sure. I guess for off-screen image manipulation (if for whatever reason you can't use WebGL and shaders) you'd have to resort to manual pixel manipulation by sending Uint8ClampedArrays back and forth. At least they're transferable by default. [0] https://github.com/bvaughn/react-virtualized-select https://github.com/bvaughn/react-virtualized-select [1] https://github.com/bvaughn/react-select-fast-filter-options https://github.com/bvaughn/react-select-fast-filter-options
ridiculous_fish 10y ago
I used WebWorkers for some math - basically lots of Fourier transforms, which took a few seconds to complete. I expected to instantiate a WebWorker with a function, and was surprised to discover you have to give it a separate JavaScript file. It is not easy to adopt.
- gp7 10y ago
  You can just call toString on a function and Blob the string. You have to keep your head straight about the fact that scopes won't transfer (no closures), which is probably why the API doesn't do it for you.
  - ridiculous_fish 10y ago
    This didn't work in my case because I had references to other functions, for complex number handling and the like. I ended up just loading the whole minified .js file again in the Worker, which worked though it felt ridiculous.
    - btbuildem 10y ago
      One of the native functions available in a web worker is importScript - it allows to load include other scripts (eg lodash) into the worker scope.
  - bartread 10y ago
    Yeah, this ticked me off as well because the use of Web Workers feels extremely un-idiomatic, but what you've said makes sense. If you just passed a function it might dupe people into thinking closures would work or, and this is perhaps worse, force the standard down the route of copying all the values available in the current closure into the worker's scope, which just sounds like a terrible idea to me.
  - vanderZwan 10y ago
    I wonder if bundlers like WebPack could make life simpler here..
pesfandiar 10y ago
I used it to create a sandbox for an online JavaScript coding practice site (if you're interested in the write-up: http://www.pesfandiar.com/blog/2016/05/12/javascript-online-hosting-static-site-cheaply http://www.pesfandiar.com/blog/2016/05/12/javascript-online-...). I should say it's a very niche use case and the sandbox is not exactly secure, but it's nice to have a separate disposable environment to run your scripts.
joshontheweb 10y ago
I run a timeout function that checks for delays in execution and I cross reference this with what is running at the time and move that to a web worker. In my cause this ends up being when I do heavy lifting like merging lots of Float32Arrays (audio stream data) or doing audio encoding. Edit: Shameless Recruiting Plug. If any of you out there are js performance pros and well acquainted with this sort of thing, please reach out as I'm looking for help optimizing further and I'm a bit out of my depth on some of it. Currently this would be a contract position but could grow to something more if that is desired.
bhouston 10y ago
Clara.io users web workers to decompression mesh data in parallel.
rcfox 10y ago
I work on a browser-based electronics design tool. We use web workers when taking large, complex polygons and transforming them into a bunch of triangles.
ww520 10y ago
Build thumbnail of images, encode audio/video, upload a file in background. Anything that takes more than a few seconds.
colordrops 10y ago
In an embedded application with low resources you could push processing into a worker if your CPU is multi core.
deleted 10y ago
[deleted]
throwanem 10y ago
I used them last year for parsing and analyzing data from 250-300M XML files, to fulfill a business need. Took a couple of days to implement and worked quite nicely.
gliese1337 10y ago
I've used them to do real-time audio and video transcoding, and for doing (non-security sensitive) hashing and decryption operations. They are an excellent fit for anything that requires heavy number-crunching, and little or no DOM manipulation or network communication.
nol13 10y ago
Extracting large .zip's of images to process before uploading, reading and writing large excel files. Basically to move as much processing client side as possible/reasonable so the server doesn't have to do it.
avgp 10y ago
We are experimenting with using SharedArrayBuffers and fetch to stream data, process it and fill it into a SharedArrayBuffer to then render with WebGL in the main thread. This has been working pretty great so far, but we'll have to wait for SharedArrayBuffer to get wider adoption. Another thing was to offload tasks such as encoding GIF frames, which is also working pretty well.
driverdan 10y ago
A few years back I built an app that took in data from APIs and did a lot of calculations for visualization. The page could take 5+ sec to render for large data sets, freezing everything while loading. If I built it today I'd use Web Workers.
jhgg 10y ago
We are experimenting with webworkers to power a very complicated autocomplete and scoring system in our client. So far so good. We're able to keep the UI running at 60fps while we match, score and sort results in a web-worker.

deleted 10y ago

[deleted]

Klathmon 10y ago

I've used it for background image manipulation before. Paint the image to a canvas, then grab the imagedata off of it, split it into as many parts as you have threads, then use the "transferrableObjects" property of postMessage to zero-copy transfer the data to each worker to be processed, transferred back, and re-stitched together. It's pretty powerful and suprisingly easy to work with once you understand it. [0] is a snippet from the code, but be gentile... It was a personal project where I was trying out polymer 0.5 and made a lot of questionable design choices... Also, I've heard of the idea of using webworkers as a "first class" platform. That is do all of the core parts of your application in them and only use the "main" thread as a "ui" thread. I haven't gotten a chance to try it out, but it seems like a great idea that could really work well in some SPAs. [0] https://github.com/Klathmon/stitchpics/blob/master/app/elements/cross-stitch/element.js#L95-L118 https://github.com/Klathmon/stitchpics/blob/master/app/eleme...

bartread 10y ago
That's interesting but, AFAIK (and, believe me, I'd be happy to be corrected), what you can't do is create a canvas element (even one not attached to the DOM), and paint directly to it using the standard 2D context and drawing primitives. Like I say, more than happy to be corrected, because that sort of thing would be incredibly helpful. Really, anything that lets you mess with a disconnected DOM in the background and then attach it in the foreground could be useful but (and, again, I'm very happy to be corrected), I don't think you can do this.
- vanderZwan 10y ago
  You're talking about OffscreenCanvas. It's available in Firefox and nowhere else: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCa... Having said that, if you want fast image manipulation, you're probably better off doing direct manipulation of Uint8Clamped arrays anyway, since that can be much faster.
  - Klathmon 10y ago
    And if you can get away with it and don't mind a lot of bit shifting, it's even better working with the Uint32array which packs all 3 colors and alpha into one element which reduces your loops by 4X
    - vanderZwan 10y ago
      Because matching the native 32 word size is better for the prefetcher, right? Wouldn't most CPU's these days be smart enough to detect advancing the index by 4, and then using offsets? So: for(let i = 0; i < someUint8Array.length; i += 4){ let R = someUint8Array[i], G = someUint8Array[i+1], B = someUint8Array[i+2], A = someUint8Array[i+3]; // ... manipulations here }
      - Klathmon 10y ago
        I'm honestly not sure why it is, but across all browsers on both ARM and x86_64 arches it was almost 3x faster than doing what you wrote. I have a feeling it's more of a JS JIT thing than a CPU prefetcher thing, but honestly I'm not really sure. In my program I linked above, it was actually faster to use Uint32array everywhere and then use functions to pull the 4 color values from it and another function to push the 4 values back to a uint32. Granted, it's been over a year since I last benchmarked that code, but I did reuse some of the image code recently and found iterating over a Uint32array to be significantly faster. (And funnily enough, manually unrolling the loop of Uint32array to something similar to what you wrote gave an additional small performance boost, but it was small enough to be not worth the extra weirdness in the code to me)

phpnode 10y ago

Anything that does significant work that would otherwise result in an unresponsive UI. For example code on the following page is compiled as you type, without a webworker this would give a very horrible user experience because compilation can take a long time for very large input - https://codemix.github.io/flow-runtime/#/try https://codemix.github.io/flow-runtime/#/try

deleted 10y ago
[deleted]

SirensOfTitan 10y ago

Maybe heavy JSON parsing? Better to parse off UI-thread than drop a frame. * Did not read article yet and am not sure about SharedArray structure.

deleted 10y ago

[deleted]

vanderZwan 10y ago

Well, I'm not using them yet, but I working on a SPA where I have multiple react-virtualized-select[0] components, each with thousands of options. Creating the fastFilter options[1] for those takes multiple seconds. Although I only do it once and then re-use it, that still freezes up the page at the start, so if I could do that in parallel without freezing the UI that would improve the app for sure. I guess for off-screen image manipulation (if for whatever reason you can't use WebGL and shaders) you'd have to resort to manual pixel manipulation by sending Uint8ClampedArrays back and forth. At least they're transferable by default. [0] https://github.com/bvaughn/react-virtualized-select https://github.com/bvaughn/react-virtualized-select [1] https://github.com/bvaughn/react-select-fast-filter-options https://github.com/bvaughn/react-select-fast-filter-options

ridiculous_fish 10y ago

I used WebWorkers for some math - basically lots of Fourier transforms, which took a few seconds to complete. I expected to instantiate a WebWorker with a function, and was surprised to discover you have to give it a separate JavaScript file. It is not easy to adopt.

gp7 10y ago
You can just call toString on a function and Blob the string. You have to keep your head straight about the fact that scopes won't transfer (no closures), which is probably why the API doesn't do it for you.
- ridiculous_fish 10y ago
  This didn't work in my case because I had references to other functions, for complex number handling and the like. I ended up just loading the whole minified .js file again in the Worker, which worked though it felt ridiculous.
  - btbuildem 10y ago
    One of the native functions available in a web worker is importScript - it allows to load include other scripts (eg lodash) into the worker scope.
- bartread 10y ago
  Yeah, this ticked me off as well because the use of Web Workers feels extremely un-idiomatic, but what you've said makes sense. If you just passed a function it might dupe people into thinking closures would work or, and this is perhaps worse, force the standard down the route of copying all the values available in the current closure into the worker's scope, which just sounds like a terrible idea to me.
- vanderZwan 10y ago
  I wonder if bundlers like WebPack could make life simpler here..

pesfandiar 10y ago

I used it to create a sandbox for an online JavaScript coding practice site (if you're interested in the write-up: http://www.pesfandiar.com/blog/2016/05/12/javascript-online-hosting-static-site-cheaply http://www.pesfandiar.com/blog/2016/05/12/javascript-online-...). I should say it's a very niche use case and the sandbox is not exactly secure, but it's nice to have a separate disposable environment to run your scripts.

joshontheweb 10y ago

I run a timeout function that checks for delays in execution and I cross reference this with what is running at the time and move that to a web worker. In my cause this ends up being when I do heavy lifting like merging lots of Float32Arrays (audio stream data) or doing audio encoding. Edit: Shameless Recruiting Plug. If any of you out there are js performance pros and well acquainted with this sort of thing, please reach out as I'm looking for help optimizing further and I'm a bit out of my depth on some of it. Currently this would be a contract position but could grow to something more if that is desired.

bhouston 10y ago

Clara.io users web workers to decompression mesh data in parallel.

rcfox 10y ago

I work on a browser-based electronics design tool. We use web workers when taking large, complex polygons and transforming them into a bunch of triangles.

ww520 10y ago

Build thumbnail of images, encode audio/video, upload a file in background. Anything that takes more than a few seconds.

colordrops 10y ago

In an embedded application with low resources you could push processing into a worker if your CPU is multi core.

deleted 10y ago

[deleted]

throwanem 10y ago

I used them last year for parsing and analyzing data from 250-300M XML files, to fulfill a business need. Took a couple of days to implement and worked quite nicely.

gliese1337 10y ago

I've used them to do real-time audio and video transcoding, and for doing (non-security sensitive) hashing and decryption operations. They are an excellent fit for anything that requires heavy number-crunching, and little or no DOM manipulation or network communication.

nol13 10y ago

Extracting large .zip's of images to process before uploading, reading and writing large excel files. Basically to move as much processing client side as possible/reasonable so the server doesn't have to do it.

avgp 10y ago

We are experimenting with using SharedArrayBuffers and fetch to stream data, process it and fill it into a SharedArrayBuffer to then render with WebGL in the main thread. This has been working pretty great so far, but we'll have to wait for SharedArrayBuffer to get wider adoption. Another thing was to offload tasks such as encoding GIF frames, which is also working pretty well.

driverdan 10y ago

A few years back I built an app that took in data from APIs and did a lot of calculations for visualization. The page could take 5+ sec to render for large data sets, freezing everything while loading. If I built it today I'd use Web Workers.

jhgg 10y ago

We are experimenting with webworkers to power a very complicated autocomplete and scoring system in our client. So far so good. We're able to keep the UI running at 60fps while we match, score and sort results in a web-worker.