5 ms·

The fact that creating a worker requires you to pass a module (file) name makes them extremely unergonomic to use. This is clearly visible in the lack of any

The fact that creating a worker requires you to pass a module (file) name makes them extremely unergonomic to use. This is clearly visible in the lack of any library ecosystem around them. We don't have any highly used threadpool or executor libraries using workers. Everyone seems to be manually setting up a worker and setting up the job scheduling logic from scratch. All the boilerplate and restrictions really restrict its usage IMO. I know you can pass a Blob URL but you can't write libraries this way since you risk running foul of the downstream consumer's browser CSP.

MuffinFlavored 5y ago
My bigger complaint is... how you cleanly do synchronous RPC style calling? Even with clever async/await tricks, the serialization to and from input/output complicated structs seems so expensive.
The_rationalist 5y ago
There are solutions to those problems: https://github.com/GoogleChromeLabs/comlink https://github.com/GoogleChromeLabs/comlink https://github.com/Bnaya/objectbuffer https://github.com/Bnaya/objectbuffer
csmpltn 5y ago
> "The fact that creating a worker requires you to pass a module (file) name makes them extremely unergonomic to use." Can anybody explain how did this ever make it into the official spec and the default implementation? The browser will make a network call to fetch your Web Worker .js file on every instantiation of your Web Worker. Instantiations of the same Web Worker "module" aren't cached, so in a thread-pool scenario your browser would be fetching the same file over and over again. What were the people designing this even thinking? This is so wasteful and frustrating. The API simply sucks.
- dmitriid 5y ago
  > Can anybody explain how did this ever make it into the official spec and the default implementation? This and service workers. Both are... weird, to say the least
  - csmpltn 5y ago
    When I'm seeing half-arsed APIs like this being introduced into so-called "modern" incarnations of the web, I start doubting the technical chops of the people involved and the process as a whole. Is it truly possible that there wasn't anyone with sufficient experience in building concurrency/multi-threading/parallelism APIs involved in building this? Failing to account for the thread-pool scenario, as an example, is just mind boggling.
- TAForObvReasons 5y ago
  "best guess": API decisions are opinionated, reflecting the designers' views of how users should write code. For example, FileReader API is async in the main thread. There's an equivalent FileReaderSync for sync operations but that is only available in Web Workers. Why isn't FileReaderSync available on the main thread? Because the designers didn't want people to do things that would block the main thread. The Web Worker argument probably went along the lines of "If we allow users to pass arbitrary function objects, they may try to do something that accesses local variables from the site where the worker is created. That is obviously an error, so we should design the API so that it can't happen. Creating a separate script creates a clear mental separation and avoids that class of error"
fabiospampinato 5y ago
Workers are awesome but you are right, working with them can be painful without the right tooling. Personally I've written my own libraries for abstracting all this away and I'm having a blast working with workers now, maybe check them out: - WorkTank [1]: This abstracts away the difference between browser workers and Node worker threads, it makes it easy to make worker pools, and it can transfer simple functions to a worker at runtime too. - WorkTank loader: This abstracts away loading asynchronous function from a worker basically, you just add ".worker" to your file name and that file and all its dependencies are transparently moved to a worker (or worker pool), all the rest of the app (TS types for example) won't even notice anything happened, it just works, transparently. You might want to check out the more popular "comlink" library too, although it didn't work for me for whatever reason when I tried it, and it doesn't support worker pools I believe. [1]: https://github.com/fabiospampinato/worktank https://github.com/fabiospampinato/worktank [2]: https://github.com/fabiospampinato/worktank-loader https://github.com/fabiospampinato/worktank-loader
Jasper_ 5y ago
A big complaint I have is that the built-in scheduling for workers assumes a very specific scenario. You can send messages to a single worker, but the message loop has to be operated by the browser. You also don't know the length of the message queue, even though the browser has it available, so you can't easily send a message to the worker with the least work queued. If a worker starts working on low-priority items, and you want to interject with a high-priority message, you also can't interrupt the worker, nor can the worker loop the messages on its own accord. You also can't re-sort the message queue, it's FIFO. Basically, any sort of work scheduling that you would like to do to queue high-priority messages, or have workers share a pool of work, is impossible to build with the onmessage-style of WebWorkers. It feels like an API made by someone who had read about threads once, rather than someone who's built an actual many-workers processing system like this. The event loop being unpumpable from user code feels like a giant kick in the face. My workaround for this was to kill workers working on low-priority items, relaunch them, and resort the queue, but that all feels like a mess. Also, at some point a Chrome update made this strategy crash. I ended up just removing the WebWorker code; more trouble than it was worth for marginal code improvements. I know that SAB and atomics add new low-level primitives to support this, but SAB is still poorly supported on widely deployed platforms. And you still need to do the serialization yourself.
IshKebab 5y ago
I agree it is weird, but there is a Blob API so you don't strictly have to use a filename. https://stackoverflow.com/a/6454685/265521 https://stackoverflow.com/a/6454685/265521 And do you have a reference for the caching issue? I tried to Google it but all I found was people saying that web workers were cached too much, not that they aren't cached at all.

MuffinFlavored 5y ago

My bigger complaint is... how you cleanly do synchronous RPC style calling? Even with clever async/await tricks, the serialization to and from input/output complicated structs seems so expensive.

The_rationalist 5y ago

There are solutions to those problems: https://github.com/GoogleChromeLabs/comlink https://github.com/GoogleChromeLabs/comlink https://github.com/Bnaya/objectbuffer https://github.com/Bnaya/objectbuffer

csmpltn 5y ago

> "The fact that creating a worker requires you to pass a module (file) name makes them extremely unergonomic to use." Can anybody explain how did this ever make it into the official spec and the default implementation? The browser will make a network call to fetch your Web Worker .js file on every instantiation of your Web Worker. Instantiations of the same Web Worker "module" aren't cached, so in a thread-pool scenario your browser would be fetching the same file over and over again. What were the people designing this even thinking? This is so wasteful and frustrating. The API simply sucks.

dmitriid 5y ago
> Can anybody explain how did this ever make it into the official spec and the default implementation? This and service workers. Both are... weird, to say the least
- csmpltn 5y ago
  When I'm seeing half-arsed APIs like this being introduced into so-called "modern" incarnations of the web, I start doubting the technical chops of the people involved and the process as a whole. Is it truly possible that there wasn't anyone with sufficient experience in building concurrency/multi-threading/parallelism APIs involved in building this? Failing to account for the thread-pool scenario, as an example, is just mind boggling.
TAForObvReasons 5y ago
"best guess": API decisions are opinionated, reflecting the designers' views of how users should write code. For example, FileReader API is async in the main thread. There's an equivalent FileReaderSync for sync operations but that is only available in Web Workers. Why isn't FileReaderSync available on the main thread? Because the designers didn't want people to do things that would block the main thread. The Web Worker argument probably went along the lines of "If we allow users to pass arbitrary function objects, they may try to do something that accesses local variables from the site where the worker is created. That is obviously an error, so we should design the API so that it can't happen. Creating a separate script creates a clear mental separation and avoids that class of error"

fabiospampinato 5y ago

Workers are awesome but you are right, working with them can be painful without the right tooling. Personally I've written my own libraries for abstracting all this away and I'm having a blast working with workers now, maybe check them out: - WorkTank [1]: This abstracts away the difference between browser workers and Node worker threads, it makes it easy to make worker pools, and it can transfer simple functions to a worker at runtime too. - WorkTank loader: This abstracts away loading asynchronous function from a worker basically, you just add ".worker" to your file name and that file and all its dependencies are transparently moved to a worker (or worker pool), all the rest of the app (TS types for example) won't even notice anything happened, it just works, transparently. You might want to check out the more popular "comlink" library too, although it didn't work for me for whatever reason when I tried it, and it doesn't support worker pools I believe. [1]: https://github.com/fabiospampinato/worktank https://github.com/fabiospampinato/worktank [2]: https://github.com/fabiospampinato/worktank-loader https://github.com/fabiospampinato/worktank-loader

Jasper_ 5y ago

A big complaint I have is that the built-in scheduling for workers assumes a very specific scenario. You can send messages to a single worker, but the message loop has to be operated by the browser. You also don't know the length of the message queue, even though the browser has it available, so you can't easily send a message to the worker with the least work queued. If a worker starts working on low-priority items, and you want to interject with a high-priority message, you also can't interrupt the worker, nor can the worker loop the messages on its own accord. You also can't re-sort the message queue, it's FIFO. Basically, any sort of work scheduling that you would like to do to queue high-priority messages, or have workers share a pool of work, is impossible to build with the onmessage-style of WebWorkers. It feels like an API made by someone who had read about threads once, rather than someone who's built an actual many-workers processing system like this. The event loop being unpumpable from user code feels like a giant kick in the face. My workaround for this was to kill workers working on low-priority items, relaunch them, and resort the queue, but that all feels like a mess. Also, at some point a Chrome update made this strategy crash. I ended up just removing the WebWorker code; more trouble than it was worth for marginal code improvements. I know that SAB and atomics add new low-level primitives to support this, but SAB is still poorly supported on widely deployed platforms. And you still need to do the serialization yourself.

IshKebab 5y ago

I agree it is weird, but there is a Blob API so you don't strictly have to use a filename. https://stackoverflow.com/a/6454685/265521 https://stackoverflow.com/a/6454685/265521 And do you have a reference for the caching issue? I tried to Google it but all I found was people saying that web workers were cached too much, not that they aren't cached at all.