7 ms·

Show HN: Ts-Chan – Go-Like Concurrency Primitives for TypeScript/JavaScript

60 points by joeycumines 3y ago

Hey HN, I’m sharing ts-chan, an NPM package providing Go-like concurrency primitives, including channels and select statements, for TypeScript and JavaScript, supporting Node.js, Deno, Bun, and browsers. This is something I've built to make implementing Go-style "control loops" feasible in JavaScript, but there are many possible applications. Highlights: - Features a FIFO processing Chan class and versatile Select class for concurrency control. - Supports buffered channel and channel close semantics very close to Go's. - TypeScript-first implementation. - Defines a simple "channel protocol" inspired by JavaScript's iteration protocols, that's used by `Select`, and implemented by `Chan`. - Makes an effort to mitigate cycles caused by the behavior of JavaScript's microtask queue. - Ongoing project with active iteration for a production-ready module (pre-v1, so the API isn't guaranteed to be stable, but the implementation itself is). NPM: ts-chan GitHub: github.com/joeycumines/ts-chan Thanks!

eplawless 3y ago
It might be worthwhile to expand a bit in the docs on the problems you see with JS concurrency (why you think it sucks, maybe some examples) and then show how ts-chan fixes those problems.
- joeycumines 3y ago
  That's a good idea, thanks :) Honestly, I've been struggling to come up with examples that aren't extremely contrived, but are still self-contained enough to easily demonstrate. It might actually be easier to just document patterns, though.
- artisin 3y ago
  I somewhat disagree with this take, as I felt the intro and "The microtask queue: a footgun" [1] section in the README does an adequate job of laying out the 'why' and the problems with JS's concurrency model. However, it does presume some understanding of Go's channels, so a more explicit example contrasting ts-chan with native JS concurrency could better clarify its benefits for those less familiar. Granted, there is an /example directory, but the benchmarking complexity muddles the readability. Regardless, upon a quick run-through, it looks to be an A-grade library that seems promising for practical use, plus well-referenced, composed, and quite thorough. [1] https://github.com/joeycumines/ts-chan#the-microtask-queue-a-footgun https://github.com/joeycumines/ts-chan#the-microtask-queue-a...
  - rpbiwer2 3y ago
    Maybe I'm dumb but that section didn't explain the problem to me in the slightest.
    - artisin 3y ago
      Not necessarily, and after giving it more thought, I somewhat retract my previous comment. You do make a good point; it's explained well, but not in concrete terms without assumptions. So, I'll take a stab at it: The core idea is that async functions A and B can communicate through Chan instances, with the Select class overseeing multiple Chan operations, waiting for one to be ready before proceeding. While ts-chan might seem unnecessary for just two async functions, what if you had to manage 8, 16, 32, or more? At some point, Promise.all won't cut it, and that's where ts-chan comes to the rescue. It defines a protocol for channels to better manage communication between asynchronous functions in JS, offering a structure similar to how goroutines communicate in Go.
  - Aeolun 3y ago
    The only thing that told me was that the author is used to Go primitives and doesn’t like switching to Javascript. That might be completely wrong, but it’s the impression I get when someone says that something the rest of the world uses without issue sucks.
    - joeycumines 3y ago
      I might be completely wrong, but the impression I got from your comment is that you haven't been exposed to many implementations using non-trivial concurrency :) Fair call though, I guess. It doesn't really matter, but I'm certainly used to TypeScript and JavaScript.
      - Aeolun 3y ago
        > you haven't been exposed to many implementations using non-trivial concurrency That is entirely accurate. I struggle to imagine scenarios in which I’d need two parallel routines to communicate with each other.
- pyrolistical 3y ago
  Agreed. I don’t know go. So I still don’t understand what this is trying to solve
  - joeycumines 3y ago
    I think y'all have very fair points, for the record. Unfortunately it is a lot easier to write documentation for an audience that shares the same context / background / experience. The README was written with an audience in mind consisting primarily of those familiar with Go (or more convoluted "communicating sequential processes" implementations), who were frustrated that things that are very easy in Go, are so much harder in JS. It's not something I considered deeply, but I was imagining that it was unlikely that someone would be searching for "channels in JS" without a base level of understanding. I work/have worked with some pretty talented people, but (in the past) I've found it difficult to convey the value of the sorts of patterns that `ts-chan` is intended to enable, to those without first-hand experience with such patterns. Documentation is hard :P
    - yunohn 3y ago
      I’m pretty sure you could articulate how exactly this “better” pattern works versus the default “bad” one. Right now the readme is a pile of illegible jargon to me as a non-go person.
      - joeycumines 3y ago
        Sure, I intend to give it a shot. I will say though, I personally get the impression that attempts at "concurrency" in JavaScript (in production code) are quite rare, which I attribute to how difficult it is. That is to say, I don't know if there really _is_ a "default pattern".
        yunohn 3y ago
        I think it would be useful to generally explain what these primitives do and how they interact with each other. A lot of JS/TS users haven’t used golang, but would appreciate a better solution if they understand it (me included). Regarding the default vs better, a comparative example with a real concurrent task coded with/out your library would be my preferred way to understand it clearly.
- 29athrowaway 3y ago
  Real JS concurrency = Worker threads, which already make use of channels for communication. Promises are for asynchronous programming, which are not concurrent.
- joeycumines 3y ago
  First instalment of docs/examples complete, feedback appreciated: https://news.ycombinator.com/item?id=38183241 https://news.ycombinator.com/item?id=38183241
29athrowaway 3y ago
It's based on the JavaScript Promise type. Where is the concurrency? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... Asynchronous is not the same as concurrent.
- joeycumines 3y ago
  The concurrency comes from providing mechanisms to communicate between independent, asynchronous chains of operations. Here is an example, that I based on an example from within the Go spec: https://github.com/joeycumines/ts-chan/blob/main/examples/concurrent-prime-sieve/index.js https://github.com/joeycumines/ts-chan/blob/main/examples/co... Here is the same example in Go: https://github.com/joeycumines/ts-chan/blob/main/examples/concurrent-prime-sieve/main.go https://github.com/joeycumines/ts-chan/blob/main/examples/co... The JS example is, obviously, much slower, and is fairly contrived - I wouldn't expect anyone to actually want to do that, specifically. The JS one is more complicated primarily because I used it in a benchmark, so needed a mechanism to stop the concurrent, asynchronous chains of operations, once it reached the target number of primes. I hope that explains it for you.
  - yunohn 3y ago
    Excuse my multiple comments, just trying to understand this better since I’m very intrigued by your library. What does the JS “if x != x” check do? I don’t see the equivalent in the golang example?
    - joeycumines 3y ago
      From the concurrent prime sieve example? That's just something I copy and pasted from the documentation for the benchmark package I used. Allegedly, it is to prevent "certain compiler optimisations", which is something I'm familiar with when it comes to benchmarking Go code, for example. I lack specific knowledge as to whether it's actually necessary, sorry :)
      - 3y ago
graup 3y ago
I really like Go channels, and I was looking for something like this in JS. One of the earlier implementations is this one, albeit not maintained anymore: https://github.com/NodeGuy/channel https://github.com/NodeGuy/channel I think I prefer its more concise API, e.g. for select (https://github.com/NodeGuy/channel/blob/main/API.md#examples https://github.com/NodeGuy/channel/blob/main/API.md#examples). ts-chan's API looks a bit too verbose to my taste. Here's an example from ts-chan: import {recv, Chan, Select} from 'ts-chan'; const ch1 = new Chan<number>(); const ch2 = new Chan<string>(); void sendsToCh1ThenEventuallyClosesIt(); void sendsToCh2(); const select = new Select([recv(ch1), recv(ch2)]); for (let running = true; running;) { const i = await select.wait(); switch (i) { case 0: { const v = select.recv(select.cases[i]); if (v.done) { running = false; break; } console.log(`rounded value: ${Math.round(v.value)}`); break; } case 1: { const v = select.recv(select.cases[i]); if (v.done) { throw new Error('ch2 unexpectedly closed'); } console.log(`uppercase string value: ${v.value.toUpperCase()}`); break; } default: throw new Error('unreachable'); } } I would consider rewriting the API to something like this: import { receive, Channel, select } from 'ts-chan'; const ch1 = new Channel<number>(); const ch2 = new Channel<string>(); void sendsToCh1ThenEventuallyClosesIt(); void sendsToCh2(); for (let running = true; running;) { switch(await select([receive(ch1), receive(ch2)])) { case ch1: { if (ch1.done) { running = false; break; } console.log(`rounded value: ${Math.round(ch1.value)}`); } case ch2: { if (ch1.done) { throw new Error('ch2 unexpectedly closed'); } console.log(`uppercase string value: ${ch2.value.toUpperCase()}`); break; } } }
- joeycumines 3y ago
  I've been considering adding a `select` function, but I was concerned with the overhead of recreating the select case each time, and wanted to reserve it for a time once I've mulled over my options (I'm aiming to avoid making breaking changes, even though it's pre-v1). With receives, there's a value attached, and that presents some difficulties regarding typing (for TypeScript). You have given me an idea, however: I _might_ be able to wrangle together a mechanism that avoids making it a two-step deal (get the index, then resolve the type). It'd probably support only a subset of the functionality, though. It's on my list to add a simple example to the README using the `Chan` class. Communicating with a single channel (with or without abort support) is very simple, and I'm happy with the API for that.
crabmusket 3y ago
Any comparison with the web-standard channel messaging API? https://developer.mozilla.org/en-US/docs/Web/API/Channel_Messaging_API https://developer.mozilla.org/en-US/docs/Web/API/Channel_Mes...
- joeycumines 3y ago
  I actually considered including one in the README, but there were (are) lots of other topics to consider. The tl;dr is that there is some cross over, but the use cases and patterns they support are largely different. Message channels are two-way, channels are one way. (I believe) message channels are 1-1, while `ts-chan`'s channels are n-n, though each send corresponds to a single receive. The primary use case I aim to address is coordination between independent, asynchronous operations (concurrency in JavaScript). Having an analogue to Go's "select statement" is fairly key to that, and I am not quite sure how to articulate the value it provides, but it does not overlap with message channels. Example using a single channel: `ts-chan` may be used to implement a set of workers, which accept requests from n sources (e.g. incoming http requests in a web server), then perform some operation, as a mechanism to bound the concurrency of that operation. Personally, I find it much easier to model it as: 1. Start the desired number of workers 2. Wire up workers to await incoming requests then process them, in a loop 3. Send requests to said workers Rather than (for example) tracking the number of running operations, starting a new worker only if possible, after enqueuing the operation, then having the workers need to operate like "check for any work, increment number of operations, perform operation, check for any work (loop), otherwise decrement number of operations". The `ts-chan` pattern (which is, really, just a Go pattern) also makes it much easier when the requirements are more complex, e.g. if the requester needs to wait for a response, and might timeout before the operation is even started - in which case the operation should be cancelled. I should probably include demonstrative examples for all that, heh.
  - crabmusket 3y ago
    > I should probably include demonstrative examples for all that, heh. Documentation is a lot of work! Thanks for the detailed reply, that makes a lot of sense.
pineapple_guy 3y ago
Regarding the first example in the readme; where is the result variable declared?
- joeycumines 3y ago
  Ah, that example is demonstrating an internal mechanism used to ensure the high-level asynchronous API calls take at least one tick of the event loop to complete. There is no `result` variable - I was just attempting to convey that "some async stuff happens, then it waits if we can't verify that it's not the same tick as when the call started". The "async stuff", in the actual implementation, is sending or receiving.
eddtries 3y ago
The thing that made me love Golang was the Time channel, where you can have a channel execute a function every X milliseconds, without having to worry about blocking threads with time.sleep() etc. It made a lot of stuff I wanted to write so much easier I basically went all in on Go and haven't regretted it.
- joeycumines 3y ago
  That one is actually pretty easy in JavaScript - in the past I've implemented the same sort of behavior using async generators. (I also like Go's time.Ticker behavior quite a lot)
  - eddtries 3y ago
    Do you have an example? For my case it was an internal lambda service that could add functions with specific intervals to the time.Ticker match, basically you edit a config to add a path to an executable and an interval and then I added it to my script
    - joeycumines 3y ago
      Sure do, I haven't actually used this one (I implemented it just then) but it should do the trick: https://gist.github.com/joeycumines/6206f2a6cd79875c7c16473823ce69de#file-newticker-ts https://gist.github.com/joeycumines/6206f2a6cd79875c7c164738...
      - joeycumines 3y ago
        Actually, it was fun, so I've made it into an actual package. The one I slapped together works, but this one is better (fixes issues with actually stopping it properly): https://github.com/joeycumines/generator-ticker https://github.com/joeycumines/generator-ticker On NPM as generator-ticker.
        eddtries 3y ago
mattlondon 3y ago
It might be nice to implement go's defer here too (e.g. for closing the channel) to keep some of the DX. I guess a JavaScript defer could just be enqueueMicrotask and an anonymous arrow function?
- joeycumines 3y ago
  If you want it to run on exiting the function in reverse order, something like Python's context managers would probably be necessary. I've dabbled in that too, though I don't actually use it for anything, currently: https://github.com/Mcsavvy/contextlib/pull/1 https://github.com/Mcsavvy/contextlib/pull/1
thecopy 3y ago
Could someone explain how this adds concurrency to JS without changing the VM/runtime?
- Philpax 3y ago
  Concurrency is not parallelism - you can multitask without (adding) threads.
- joeycumines 3y ago
  Concurrency is just the ability to handle multiple tasks at the same time, to use my quickly googled definition. JavaScript already supports concurrency, e.g. handling more than one HTTP request concurrently, despite the runtime being single-threaded in nature. Also provided by JavaScript is the ability to fork and join with other asynchronous tasks, e.g. using `Promise.all` to wait for multiple HTTP requests to complete. The purpose of `ts-chan` is to make it easier to coordinate concurrent operations, via communication, which definitely isn't something JavaScript makes easy. So part of this is addressing usability issues, and providing necessary boilerplate. I've modelled most of that after Go's channels. Another is providing a mitigation for a common gotcha (well, it's gotten me), when attempting to implement behavior that involves multiple independent asynchronous operations, communicating with one another. Specifically, JS has a "microtask queue", an optimisation I believe was created to power async/await. Details / links to MDN here: https://github.com/joeycumines/ts-chan#the-microtask-queue-a-footgun https://github.com/joeycumines/ts-chan#the-microtask-queue-a...
  - deleted 3y ago
    [deleted]
  - thecopy 3y ago
    Thanks. I was confused by the terminology. I thought Concurrency = Parallelism, however i realise now that concurrency does not mean that the code is being executed at the same time (not considering waiting for I/O as "executing") while Parallelism means executing at the same time.
anon23432343 3y ago
I mean the foot gun is using TS/JS itself for such things instead of already using a lang that is good at this like Go.
- joeycumines 3y ago
  I'd agree with you for personal projects, but there are plenty of companies etc using TS/JS for backend systems, my current place of work included.
  - anon23432343 3y ago
    I'm a front end dev which was working mostly for the last 10 years with ts/js and I think we need to stop that trend to write everything in ts/js.
kreetx 3y ago
This doesn't make JavaScript actually run in parallel, does it? I.e if there is any IO to wait on then JS runs, but if there are multiple pure JS asyncs scheduled to run then these will still run one at a time? Secondly, there is async iteration which is already pretty good syntax wise.
- joeycumines 3y ago
  Yep, JS is still single threaded, and all that. I agree async iteration is pretty good - the `Chan` class implements both `Iterable` and `AsyncIterable`. For quite a few scenarios, async generators are a better choice. They do have limitations, however. For example, you _can_ implement fan-in, but there's no easy mechanism to fan-in in a blocking manner, from sources that aren't known ahead of time. It's also hard to implement a "fair" mechanism, that doesn't bias certain inputs (like you might desire if multiplexing logs, for example).
  - felipellrocha 3y ago
    Couldn’t you spin N workers and assign work to them?
    - joeycumines 3y ago
      Fan-in is the other way, but yeah you can do batches of concurrent operations in JS fairly easily. I still like "pipelining" over that, but that can also be implemented with just an iterator, pretty trivially.
  - kreetx 3y ago
    The value proposition of the library became much clearer, thanks for explaining!
lakomen 3y ago
But, correct me if I'm wrong, JS in the browser is single threaded and service workers kind of make it multi threaded. How do you extend JS to become multi threaded when it's not capable to do so?
gnarlouse 3y ago
I think my question is why do this if JavaScript is a single threaded language anyway? If you’ve got concurrent processes running in JavaScript, they’re not actually “concurrent”, right? If you’re stuck using JS aren’t you fundamentally better off separating them into two distinct process packages, and having them communicate via some IPC system? Furthermore; why not just *avoid doing concurrency intensive work in JS* in the first place? Isn’t that why people learned Go? Seems like a solution looking for a problem kind of, but that’s my low effort critique so maybe I should stuff it.
- joeycumines 3y ago
  I'm working on improving documentation and examples, still a WIP, but you may find this interesting: https://news.ycombinator.com/item?id=38183241 https://news.ycombinator.com/item?id=38183241
  - gnarlouse 3y ago
    I mean isn't this what we have RxJS for?

60 points by joeycumines 3y ago

eplawless 3y ago

It might be worthwhile to expand a bit in the docs on the problems you see with JS concurrency (why you think it sucks, maybe some examples) and then show how ts-chan fixes those problems.

joeycumines 3y ago
That's a good idea, thanks :) Honestly, I've been struggling to come up with examples that aren't extremely contrived, but are still self-contained enough to easily demonstrate. It might actually be easier to just document patterns, though.
artisin 3y ago
I somewhat disagree with this take, as I felt the intro and "The microtask queue: a footgun" [1] section in the README does an adequate job of laying out the 'why' and the problems with JS's concurrency model. However, it does presume some understanding of Go's channels, so a more explicit example contrasting ts-chan with native JS concurrency could better clarify its benefits for those less familiar. Granted, there is an /example directory, but the benchmarking complexity muddles the readability. Regardless, upon a quick run-through, it looks to be an A-grade library that seems promising for practical use, plus well-referenced, composed, and quite thorough. [1] https://github.com/joeycumines/ts-chan#the-microtask-queue-a-footgun https://github.com/joeycumines/ts-chan#the-microtask-queue-a...
- rpbiwer2 3y ago
  Maybe I'm dumb but that section didn't explain the problem to me in the slightest.
  - artisin 3y ago
    Not necessarily, and after giving it more thought, I somewhat retract my previous comment. You do make a good point; it's explained well, but not in concrete terms without assumptions. So, I'll take a stab at it: The core idea is that async functions A and B can communicate through Chan instances, with the Select class overseeing multiple Chan operations, waiting for one to be ready before proceeding. While ts-chan might seem unnecessary for just two async functions, what if you had to manage 8, 16, 32, or more? At some point, Promise.all won't cut it, and that's where ts-chan comes to the rescue. It defines a protocol for channels to better manage communication between asynchronous functions in JS, offering a structure similar to how goroutines communicate in Go.
- Aeolun 3y ago
  The only thing that told me was that the author is used to Go primitives and doesn’t like switching to Javascript. That might be completely wrong, but it’s the impression I get when someone says that something the rest of the world uses without issue sucks.
  - joeycumines 3y ago
    I might be completely wrong, but the impression I got from your comment is that you haven't been exposed to many implementations using non-trivial concurrency :) Fair call though, I guess. It doesn't really matter, but I'm certainly used to TypeScript and JavaScript.
    - Aeolun 3y ago
      > you haven't been exposed to many implementations using non-trivial concurrency That is entirely accurate. I struggle to imagine scenarios in which I’d need two parallel routines to communicate with each other.
pyrolistical 3y ago
Agreed. I don’t know go. So I still don’t understand what this is trying to solve
- joeycumines 3y ago
  I think y'all have very fair points, for the record. Unfortunately it is a lot easier to write documentation for an audience that shares the same context / background / experience. The README was written with an audience in mind consisting primarily of those familiar with Go (or more convoluted "communicating sequential processes" implementations), who were frustrated that things that are very easy in Go, are so much harder in JS. It's not something I considered deeply, but I was imagining that it was unlikely that someone would be searching for "channels in JS" without a base level of understanding. I work/have worked with some pretty talented people, but (in the past) I've found it difficult to convey the value of the sorts of patterns that `ts-chan` is intended to enable, to those without first-hand experience with such patterns. Documentation is hard :P
  - yunohn 3y ago
    I’m pretty sure you could articulate how exactly this “better” pattern works versus the default “bad” one. Right now the readme is a pile of illegible jargon to me as a non-go person.
    - joeycumines 3y ago
      Sure, I intend to give it a shot. I will say though, I personally get the impression that attempts at "concurrency" in JavaScript (in production code) are quite rare, which I attribute to how difficult it is. That is to say, I don't know if there really _is_ a "default pattern".
      - yunohn 3y ago
        I think it would be useful to generally explain what these primitives do and how they interact with each other. A lot of JS/TS users haven’t used golang, but would appreciate a better solution if they understand it (me included). Regarding the default vs better, a comparative example with a real concurrent task coded with/out your library would be my preferred way to understand it clearly.
29athrowaway 3y ago
Real JS concurrency = Worker threads, which already make use of channels for communication. Promises are for asynchronous programming, which are not concurrent.
joeycumines 3y ago
First instalment of docs/examples complete, feedback appreciated: https://news.ycombinator.com/item?id=38183241 https://news.ycombinator.com/item?id=38183241

29athrowaway 3y ago

It's based on the JavaScript Promise type. Where is the concurrency? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... Asynchronous is not the same as concurrent.

joeycumines 3y ago
The concurrency comes from providing mechanisms to communicate between independent, asynchronous chains of operations. Here is an example, that I based on an example from within the Go spec: https://github.com/joeycumines/ts-chan/blob/main/examples/concurrent-prime-sieve/index.js https://github.com/joeycumines/ts-chan/blob/main/examples/co... Here is the same example in Go: https://github.com/joeycumines/ts-chan/blob/main/examples/concurrent-prime-sieve/main.go https://github.com/joeycumines/ts-chan/blob/main/examples/co... The JS example is, obviously, much slower, and is fairly contrived - I wouldn't expect anyone to actually want to do that, specifically. The JS one is more complicated primarily because I used it in a benchmark, so needed a mechanism to stop the concurrent, asynchronous chains of operations, once it reached the target number of primes. I hope that explains it for you.
- yunohn 3y ago
  Excuse my multiple comments, just trying to understand this better since I’m very intrigued by your library. What does the JS “if x != x” check do? I don’t see the equivalent in the golang example?
  - joeycumines 3y ago
    From the concurrent prime sieve example? That's just something I copy and pasted from the documentation for the benchmark package I used. Allegedly, it is to prevent "certain compiler optimisations", which is something I'm familiar with when it comes to benchmarking Go code, for example. I lack specific knowledge as to whether it's actually necessary, sorry :)
    - 3y ago

graup 3y ago

I really like Go channels, and I was looking for something like this in JS. One of the earlier implementations is this one, albeit not maintained anymore: https://github.com/NodeGuy/channel https://github.com/NodeGuy/channel I think I prefer its more concise API, e.g. for select (https://github.com/NodeGuy/channel/blob/main/API.md#examples https://github.com/NodeGuy/channel/blob/main/API.md#examples). ts-chan's API looks a bit too verbose to my taste. Here's an example from ts-chan: import {recv, Chan, Select} from 'ts-chan'; const ch1 = new Chan<number>(); const ch2 = new Chan<string>(); void sendsToCh1ThenEventuallyClosesIt(); void sendsToCh2(); const select = new Select([recv(ch1), recv(ch2)]); for (let running = true; running;) { const i = await select.wait(); switch (i) { case 0: { const v = select.recv(select.cases[i]); if (v.done) { running = false; break; } console.log(`rounded value: ${Math.round(v.value)}`); break; } case 1: { const v = select.recv(select.cases[i]); if (v.done) { throw new Error('ch2 unexpectedly closed'); } console.log(`uppercase string value: ${v.value.toUpperCase()}`); break; } default: throw new Error('unreachable'); } } I would consider rewriting the API to something like this: import { receive, Channel, select } from 'ts-chan'; const ch1 = new Channel<number>(); const ch2 = new Channel<string>(); void sendsToCh1ThenEventuallyClosesIt(); void sendsToCh2(); for (let running = true; running;) { switch(await select([receive(ch1), receive(ch2)])) { case ch1: { if (ch1.done) { running = false; break; } console.log(`rounded value: ${Math.round(ch1.value)}`); } case ch2: { if (ch1.done) { throw new Error('ch2 unexpectedly closed'); } console.log(`uppercase string value: ${ch2.value.toUpperCase()}`); break; } } }

joeycumines 3y ago
I've been considering adding a `select` function, but I was concerned with the overhead of recreating the select case each time, and wanted to reserve it for a time once I've mulled over my options (I'm aiming to avoid making breaking changes, even though it's pre-v1). With receives, there's a value attached, and that presents some difficulties regarding typing (for TypeScript). You have given me an idea, however: I _might_ be able to wrangle together a mechanism that avoids making it a two-step deal (get the index, then resolve the type). It'd probably support only a subset of the functionality, though. It's on my list to add a simple example to the README using the `Chan` class. Communicating with a single channel (with or without abort support) is very simple, and I'm happy with the API for that.

crabmusket 3y ago

Any comparison with the web-standard channel messaging API? https://developer.mozilla.org/en-US/docs/Web/API/Channel_Messaging_API https://developer.mozilla.org/en-US/docs/Web/API/Channel_Mes...

joeycumines 3y ago
I actually considered including one in the README, but there were (are) lots of other topics to consider. The tl;dr is that there is some cross over, but the use cases and patterns they support are largely different. Message channels are two-way, channels are one way. (I believe) message channels are 1-1, while `ts-chan`'s channels are n-n, though each send corresponds to a single receive. The primary use case I aim to address is coordination between independent, asynchronous operations (concurrency in JavaScript). Having an analogue to Go's "select statement" is fairly key to that, and I am not quite sure how to articulate the value it provides, but it does not overlap with message channels. Example using a single channel: `ts-chan` may be used to implement a set of workers, which accept requests from n sources (e.g. incoming http requests in a web server), then perform some operation, as a mechanism to bound the concurrency of that operation. Personally, I find it much easier to model it as: 1. Start the desired number of workers 2. Wire up workers to await incoming requests then process them, in a loop 3. Send requests to said workers Rather than (for example) tracking the number of running operations, starting a new worker only if possible, after enqueuing the operation, then having the workers need to operate like "check for any work, increment number of operations, perform operation, check for any work (loop), otherwise decrement number of operations". The `ts-chan` pattern (which is, really, just a Go pattern) also makes it much easier when the requirements are more complex, e.g. if the requester needs to wait for a response, and might timeout before the operation is even started - in which case the operation should be cancelled. I should probably include demonstrative examples for all that, heh.
- crabmusket 3y ago
  > I should probably include demonstrative examples for all that, heh. Documentation is a lot of work! Thanks for the detailed reply, that makes a lot of sense.

pineapple_guy 3y ago

Regarding the first example in the readme; where is the result variable declared?

joeycumines 3y ago
Ah, that example is demonstrating an internal mechanism used to ensure the high-level asynchronous API calls take at least one tick of the event loop to complete. There is no `result` variable - I was just attempting to convey that "some async stuff happens, then it waits if we can't verify that it's not the same tick as when the call started". The "async stuff", in the actual implementation, is sending or receiving.

eddtries 3y ago

The thing that made me love Golang was the Time channel, where you can have a channel execute a function every X milliseconds, without having to worry about blocking threads with time.sleep() etc. It made a lot of stuff I wanted to write so much easier I basically went all in on Go and haven't regretted it.

joeycumines 3y ago
That one is actually pretty easy in JavaScript - in the past I've implemented the same sort of behavior using async generators. (I also like Go's time.Ticker behavior quite a lot)
- eddtries 3y ago
  Do you have an example? For my case it was an internal lambda service that could add functions with specific intervals to the time.Ticker match, basically you edit a config to add a path to an executable and an interval and then I added it to my script
  - joeycumines 3y ago
    Sure do, I haven't actually used this one (I implemented it just then) but it should do the trick: https://gist.github.com/joeycumines/6206f2a6cd79875c7c16473823ce69de#file-newticker-ts https://gist.github.com/joeycumines/6206f2a6cd79875c7c164738...
    - joeycumines 3y ago
      Actually, it was fun, so I've made it into an actual package. The one I slapped together works, but this one is better (fixes issues with actually stopping it properly): https://github.com/joeycumines/generator-ticker https://github.com/joeycumines/generator-ticker On NPM as generator-ticker.
      - eddtries 3y ago

mattlondon 3y ago

It might be nice to implement go's defer here too (e.g. for closing the channel) to keep some of the DX. I guess a JavaScript defer could just be enqueueMicrotask and an anonymous arrow function?

joeycumines 3y ago
If you want it to run on exiting the function in reverse order, something like Python's context managers would probably be necessary. I've dabbled in that too, though I don't actually use it for anything, currently: https://github.com/Mcsavvy/contextlib/pull/1 https://github.com/Mcsavvy/contextlib/pull/1

thecopy 3y ago

Could someone explain how this adds concurrency to JS without changing the VM/runtime?

Philpax 3y ago
Concurrency is not parallelism - you can multitask without (adding) threads.
joeycumines 3y ago
Concurrency is just the ability to handle multiple tasks at the same time, to use my quickly googled definition. JavaScript already supports concurrency, e.g. handling more than one HTTP request concurrently, despite the runtime being single-threaded in nature. Also provided by JavaScript is the ability to fork and join with other asynchronous tasks, e.g. using `Promise.all` to wait for multiple HTTP requests to complete. The purpose of `ts-chan` is to make it easier to coordinate concurrent operations, via communication, which definitely isn't something JavaScript makes easy. So part of this is addressing usability issues, and providing necessary boilerplate. I've modelled most of that after Go's channels. Another is providing a mitigation for a common gotcha (well, it's gotten me), when attempting to implement behavior that involves multiple independent asynchronous operations, communicating with one another. Specifically, JS has a "microtask queue", an optimisation I believe was created to power async/await. Details / links to MDN here: https://github.com/joeycumines/ts-chan#the-microtask-queue-a-footgun https://github.com/joeycumines/ts-chan#the-microtask-queue-a...
- deleted 3y ago
  [deleted]
- thecopy 3y ago
  Thanks. I was confused by the terminology. I thought Concurrency = Parallelism, however i realise now that concurrency does not mean that the code is being executed at the same time (not considering waiting for I/O as "executing") while Parallelism means executing at the same time.

anon23432343 3y ago

I mean the foot gun is using TS/JS itself for such things instead of already using a lang that is good at this like Go.

joeycumines 3y ago
I'd agree with you for personal projects, but there are plenty of companies etc using TS/JS for backend systems, my current place of work included.
- anon23432343 3y ago
  I'm a front end dev which was working mostly for the last 10 years with ts/js and I think we need to stop that trend to write everything in ts/js.

kreetx 3y ago

This doesn't make JavaScript actually run in parallel, does it? I.e if there is any IO to wait on then JS runs, but if there are multiple pure JS asyncs scheduled to run then these will still run one at a time? Secondly, there is async iteration which is already pretty good syntax wise.

joeycumines 3y ago
Yep, JS is still single threaded, and all that. I agree async iteration is pretty good - the `Chan` class implements both `Iterable` and `AsyncIterable`. For quite a few scenarios, async generators are a better choice. They do have limitations, however. For example, you _can_ implement fan-in, but there's no easy mechanism to fan-in in a blocking manner, from sources that aren't known ahead of time. It's also hard to implement a "fair" mechanism, that doesn't bias certain inputs (like you might desire if multiplexing logs, for example).
- felipellrocha 3y ago
  Couldn’t you spin N workers and assign work to them?
  - joeycumines 3y ago
    Fan-in is the other way, but yeah you can do batches of concurrent operations in JS fairly easily. I still like "pipelining" over that, but that can also be implemented with just an iterator, pretty trivially.
- kreetx 3y ago
  The value proposition of the library became much clearer, thanks for explaining!

lakomen 3y ago

But, correct me if I'm wrong, JS in the browser is single threaded and service workers kind of make it multi threaded. How do you extend JS to become multi threaded when it's not capable to do so?

gnarlouse 3y ago

I think my question is why do this if JavaScript is a single threaded language anyway? If you’ve got concurrent processes running in JavaScript, they’re not actually “concurrent”, right? If you’re stuck using JS aren’t you fundamentally better off separating them into two distinct process packages, and having them communicate via some IPC system? Furthermore; why not just *avoid doing concurrency intensive work in JS* in the first place? Isn’t that why people learned Go? Seems like a solution looking for a problem kind of, but that’s my low effort critique so maybe I should stuff it.

joeycumines 3y ago
I'm working on improving documentation and examples, still a WIP, but you may find this interesting: https://news.ycombinator.com/item?id=38183241 https://news.ycombinator.com/item?id=38183241
- gnarlouse 3y ago
  I mean isn't this what we have RxJS for?