5 ms·

What if JavaScript had a synchronous async flavor?

38 points by protoduction 5y ago

lioeters 5y ago
Brilliant and horrifying - I love it. It's a transpiler that puts "await" in front of every expression, resulting in code like: (async () => { async function fetchPageBody(url) { return await (await (await fetch)(await url)).text(); } async function sleep(ms) { return await new (await Promise)(await (async (resolve) => await (await setTimeout)(await (async () => await (await resolve)()), await ms))); } const body = await (await fetchPageBody)("https://enable-cors.org/"); const numDivs = (await ((await (await body).match(/<div/g)) || (await []))).length; await (await console).log("Number of divs:", await numDivs); await (await sleep)(2000); await (await console).log("This prints after two seconds."); })();
- protoduction 5y ago
  That's right. And it also makes every function async. Literal expressions are not awaited - a tiny optimization ;)
- brundolf 5y ago
  I actually looked into this just yesterday because I was interested in doing this in my JS-targeted language :) As far as I can tell, the only real problem is performance. Wrapping synchronous expressions in Promise.resolve() (which is what this effectively does) is apparently not optimized out by V8, and is egregiously inefficient: https://madelinemiller.dev/blog/javascript-promise-overhead/ https://madelinemiller.dev/blog/javascript-promise-overhead/ For my use-case, probably what I'll end up doing is having the compiler be smart and only invisibly insert those asyncs/awaits where it actually needs to
  - Jasper_ 5y ago
    Because of how the microtask queue works, it can't optimize that out without introducing ordering issues. If you have two Promises, A and B, and each one has a callback that will then do another await, the implementation needs to interleave them, since microtasks are guaranteed to run in FIFO order. So a single await, even to resolve a constant expression, can't be omitted.
    - brundolf 5y ago
      Fair enough. In practice I think it's wildly dangerous to make those kinds of assumptions about async task ordering, but I can understand that engine authors have to follow the spec and don't have the luxury of dodging the issue.
recursivedoubts 5y ago
Somewhat related, I am working on a front end scripting language that implements this idea, but at the runtime layer: https://hyperscript.org/docs/#async https://hyperscript.org/docs/#async Basically, the result of every expression is inspected and, if a Promise is returned, the promise is resolved before execution continues. It's all interpreted and really slow for CPU-intensive stuff, but if you are writing async code that isn't CPU intensive it lets you write stuff in the linear manner: fetch /message put it into my innerHTML One interesting thing that fell out of that work is that you can have event-driven control flow, like a loop that executes until an event is received by an element: https://hyperscript.org/docs/#event-control-flow https://hyperscript.org/docs/#event-control-flow <button id="pulsar" _="on click repeat until event stop add .pulse then settle remove .pulse then settle end"> Click me to Pulse... </button>
- inkeddeveloper 5y ago
  Or, people can just learn to deal with the nuances of Javascript.
- protoduction 5y ago
  I'm following hyperscript's development keenly :). I've considered putting together a small hyperscript plugin for Starboard Notebook. Often one simply wants to put together a quick interface with some interactivity, together with lit-html and bootstrap styles this could be really productive.
ttty2 5y ago
I was thinking about it s while ago. Why do I care if it's sync or async. Usually 99% I need to await, therefore is better to have a way to tell the compiler to don't wait. Adjust typescript kind of fixes the bugs the author mentions
ermir 5y ago
I tried this exact thing last year, and it does not work for a particular reason: the JavaScript event loop guarantees that a function will run to the end before any callbacks will run, but an await is an implicit callback (a Promise under the hood). This means that you break the guarantee that variables or other memory locations will have the same value between statements. This is exactly the same problem that languages such as C++ have when dealing with multithreading, and why constructs such as semaphores, mutexes, etc. exist. JS does not have these, and using the technique in the article will completely break the language. Compare these two fiddles I prepared and see for yourself: This one uses await to block for a second, which in the meantime changes the value of the variable between statements: https://jsfiddle.net/gq1btLdu/ https://jsfiddle.net/gq1btLdu/ This one uses a for loop to do synchronous work for about a second, then prints the initial value twice. You will notice that the setInterval never had the chance to run, since callbacks are scheduled to run only once the current function ends: https://jsfiddle.net/gq1btLdu/1/ https://jsfiddle.net/gq1btLdu/1/
- protoduction 5y ago
  You are right that it's a terrible idea (I hope it was clear from the article that it's an experiment and should stay that way), but could you help me understand why this is fundamentally broken? I would say that by using setInterval you are buying into this kind of behavior and would have to create your own synchronization. If you are saying that this transpilation step changes the semantics of the code, then I completely agree!
  - ermir 5y ago
    If you have code like: let x = 0; doSomethingAsync(); // does not touch x console.log(x); There's no guarantee that x will have the value 0 by the time you get to console.log(x). The value of x might have been changed by an outside function that just happened to run between your statements. In synchronous JS this would never happen as callbacks are scheduled to run only at the end of a function, and if they change x the change will only be applied after x is printed. The change in behavior will break a lot of existing code, since the synchronous behavior of a function is a core feature and assumption of the language.
franciscop 5y ago
I went experimenting in a slightly different direction, there's usually a lot of concatenation going on in Javascript with e.g. array manipulation, so I decided to try to delay the `.then()` as much as possible. So instead of these: const value = await Promise.all(data.map(op1)) .then(value => value.filter(op2)) .then(value => Promise.all(value.map(op3))); // OR (a more readable version) let value = await Promise.all(data.map(op1)); value = value.filter(op2); value = await Promise.all(value.map(op3)); Now you can do this: const value = await swear(data).map(op1).filter(op2).map(op3); I integrated it with my other library `files`, where it's very useful out of the box: const list = await read('data.csv').split('\n').map(...); const readmes = await walk('readme.md').map(read); https://www.npmjs.com/package/swear https://www.npmjs.com/package/swear https://www.npmjs.com/package/files https://www.npmjs.com/package/files
- mikeyjk 5y ago
  Ooo, I like that. So I'm newer to JS, but this suggests chaining from promises isn't possible, and that's effectively the change you've written?
  - franciscop 5y ago
    It is possible chaining promises (as in my first example), just just need to wrap it all in a `.then(value => value.[operation]())`, but this way you can just use the `.[operation]()` straight on the data like you would do if there was no promise in the way.
hetman 5y ago
I'm not sure what you mean about Python. In Python, if you run an async function without an await, you will simply get a coroutine object that is ready to be executed by... someone (it's essentially a generator under the hood). In fact, unlike JavaScript, you can't even call await from a synchronous module context. You have to either schedule that couroutine object onto an existing async event loop, or create a new event loop and run it your self.
- akdor1154 5y ago
  Yup, Python async functions need to be explicitly awaited / scheduled pretty much exactly the same way as JS... article might be confusing this with the fact that most of the stdlib is just plain synchronous, even for web requests etc.
  - jtwaleson 5y ago
    Maybe they used gevent which "magically" makes things async without the keywords.
hliyan 5y ago
What if this is baked into the runtime as a mode? 'use async';
- foreigner 5y ago
  Yes, or a keyword that you use to declare that a single function works this way "asyncAll".
  - hliyan 5y ago
    Or, insert 'use async' inside the target function body (just like with 'use strict' now).
TekMol 5y ago
In languages like Python and Go you write asynchronous code as if it’s synchronous. How do you do that in Python?
- brundolf 5y ago
  My interpretation was that the author meant "you (normally) write things like web requests in a synchronous (blocking) way". There are important differences though between that and what async/await (even this invisible version of it) does. It's also different from what Go does, in a similar way.
superasn 5y ago
There are many things to hate about JavaScript but the inbuilt forced async nature is one thing I absolutely love about it. Thank god that except a few reminants like `alert` and `prompt` almost everything is async by default. I remember that just until a few years ago the whole tabs and everything would freeze if a site showed a blocking alert popup. Can't imagine what it would do if other things like http req, dom operations, etc were also synchronous.
- awestroke 5y ago
  DOM operations are always synchronous. When modifying DOM, there is often a full layout+paint cycle before the next line is run.
- thomasz 5y ago
  There is no inbuild async nature. It just did not have a stdlib whatsoever, so node could come along and offer a completely async one. There were async server frameworks in python before, like twisted and tornado. For a long, long time, the language itself lacked almost all of the niceties that make async programming bearable. Node code quickly devolved into a giant mess of callbacks nested twenty levels deep.
madsbuch 5y ago
> With this flavor of Javascript you can no longer do two asynchronous tasks at the same time, everything will be executed synchronously. The main pattern I use to dispatch multiple promises is resilient to this: const [r1, r2, r3] = await Promise.all([ async () => { ... }, async () => { ... }, async () => { ... } ])
- nayuki 5y ago
  No. I think your code will be transformed into: const [r1, r2, r3] = await Promise.all(await [ await (async () => { ... }), await (async () => { ... }), await (async () => { ... }), ]);
tomaszs 5y ago
Great thought experiment. I also wished it would be possible to code in Javascript without thinking about asynchronous stuff. It should be possible theoretically.
- rasz 5y ago
  Id settle for implied async whenever function contains awaits.
hsn915 5y ago
I think this doesn't work for javascript because it actually matters when you await. It has implications for the UI. It also has implications about object life times. Of course, the same problem can happen in Go if you use it to make a GUI application, but Go is built entirely around the concept of goroutines so it has obvious ways to handle the problem. Javascript does not. `await` as an explicit suspension point is a pretty good way to handle the problem. I view being "annoyed" at having to be explicit about control flow to be not a real problem. By that I mean, whatever "solution" you come up with is likely to bring with it its own set of problems that no one wants to deal with. And for what price? Because you are "annoyed" at having to type "await"?
- BigJono 5y ago
  Exactly. What is with this fascination of removing information from code? My first thought when reading this was, ok, OP is "annoyed" at having to "run functions to see if they return a value or a Promise", then goes on to describe a world in which you have no idea whether anything returns a value or a Promise when reading the code. That's 100x worse. At least when I'm writing it I'm probably running the function anyway. And it's worth pointing out that you shouldn't need to run anything to know what it returns, that's what docs are for. Or you could just look at the return statement. But even if for whatever reason you don't know what data you're working with, I'd rather figure it out once and then make it explicit, than have to figure it out every single time the code is read (which could be hundreds of times, or more).
jtwaleson 5y ago
If we designed a new async language today, would it include async/await keywords? I think they don't add anything conceptually and it's just a patch for languages that didn't design for async from the start. Am I missing something?
- jayd16 5y ago
  Maybe not async as I think the await keyword is probably enough on its own. As for why it exists... Await/explicit yielding enables cooperative multitasking where the programmer has exclusive control of some execution context, be it a thread or fiber or whatever, until they decide to yield. Without that you only have implicit scheduling and execution could be yielded at unpredictable times. The programmer then needs to add other things like semaphores to specify the critical sections and you run the risk of deadlocks. There might be a new language that comes along but right now await exists for a reason.
- ledauphin 5y ago
  I think this is a compelling vision, but as a practitioner of functional core/imperative shell, I make use of the async/await keywords to understand where I/O is happening and to work towards moving it to the edges of my program. I can do this without those keywords, but there's actually something quite nice about being certain that a given function does no I/O simply based on its function signature.
smasher164 5y ago
If you're interested by this, take a look at Stopify[1], which further adds first-class continuations to javascript. [1] https://www.stopify.org/ https://www.stopify.org/
- lioeters 5y ago
  Ooh, thank you for a fascinating reference about Stopify. I'll be reading the paper [1] it's based on with great interest. Just recently I was reading "Exceptional Continuations in JavaScript" [2], which I see is cited in the Stopify paper - so it's like a continuing saga. [1] Putting in All the Stops: Execution Control for JavaScript - https://arxiv.org/pdf/1802.02974.pdf https://arxiv.org/pdf/1802.02974.pdf (PDF) [2] Florian Loitsch. 2007. Exceptional Continuations in JavaScript. In Workshop on Scheme and Functional Programming. http://www.schemeworkshop.org/2007/procPaper4.pdf http://www.schemeworkshop.org/2007/procPaper4.pdf (PDF)
kennu 5y ago
I remember debating this when async/await support was first coming out. My opinion was (and still is) that it would have been better to make "await" the default behavior and have a separate "noawait" keyword for those rare cases when you actually really want the promise instead of the value.
BrandoElFollito 5y ago
I am a casual user of JS and was always wondering how to write true asynchronous routines in JS. The kind you fire and forget and they end when they end. An example would be fetch, but all the docs/tutorials I saw are needed on async await where you have to explicitly yield in your code (as opposed, say, to threading in python or goroutines)

38 points by protoduction 5y ago

Brilliant and horrifying - I love it. It's a transpiler that puts "await" in front of every expression, resulting in code like: (async () => { async function fetchPageBody(url) { return await (await (await fetch)(await url)).text(); } async function sleep(ms) { return await new (await Promise)(await (async (resolve) => await (await setTimeout)(await (async () => await (await resolve)()), await ms))); } const body = await (await fetchPageBody)("https://enable-cors.org/"); const numDivs = (await ((await (await body).match(/<div/g)) || (await []))).length; await (await console).log("Number of divs:", await numDivs); await (await sleep)(2000); await (await console).log("This prints after two seconds."); })();

protoduction 5y ago
That's right. And it also makes every function async. Literal expressions are not awaited - a tiny optimization ;)
brundolf 5y ago
I actually looked into this just yesterday because I was interested in doing this in my JS-targeted language :) As far as I can tell, the only real problem is performance. Wrapping synchronous expressions in Promise.resolve() (which is what this effectively does) is apparently not optimized out by V8, and is egregiously inefficient: https://madelinemiller.dev/blog/javascript-promise-overhead/ https://madelinemiller.dev/blog/javascript-promise-overhead/ For my use-case, probably what I'll end up doing is having the compiler be smart and only invisibly insert those asyncs/awaits where it actually needs to
- Jasper_ 5y ago
  Because of how the microtask queue works, it can't optimize that out without introducing ordering issues. If you have two Promises, A and B, and each one has a callback that will then do another await, the implementation needs to interleave them, since microtasks are guaranteed to run in FIFO order. So a single await, even to resolve a constant expression, can't be omitted.
  - brundolf 5y ago
    Fair enough. In practice I think it's wildly dangerous to make those kinds of assumptions about async task ordering, but I can understand that engine authors have to follow the spec and don't have the luxury of dodging the issue.

recursivedoubts 5y ago

Somewhat related, I am working on a front end scripting language that implements this idea, but at the runtime layer: https://hyperscript.org/docs/#async https://hyperscript.org/docs/#async Basically, the result of every expression is inspected and, if a Promise is returned, the promise is resolved before execution continues. It's all interpreted and really slow for CPU-intensive stuff, but if you are writing async code that isn't CPU intensive it lets you write stuff in the linear manner: fetch /message put it into my innerHTML One interesting thing that fell out of that work is that you can have event-driven control flow, like a loop that executes until an event is received by an element: https://hyperscript.org/docs/#event-control-flow https://hyperscript.org/docs/#event-control-flow <button id="pulsar" _="on click repeat until event stop add .pulse then settle remove .pulse then settle end"> Click me to Pulse... </button>

inkeddeveloper 5y ago
Or, people can just learn to deal with the nuances of Javascript.
protoduction 5y ago
I'm following hyperscript's development keenly :). I've considered putting together a small hyperscript plugin for Starboard Notebook. Often one simply wants to put together a quick interface with some interactivity, together with lit-html and bootstrap styles this could be really productive.

ttty2 5y ago

I was thinking about it s while ago. Why do I care if it's sync or async. Usually 99% I need to await, therefore is better to have a way to tell the compiler to don't wait. Adjust typescript kind of fixes the bugs the author mentions

ermir 5y ago

I tried this exact thing last year, and it does not work for a particular reason: the JavaScript event loop guarantees that a function will run to the end before any callbacks will run, but an await is an implicit callback (a Promise under the hood). This means that you break the guarantee that variables or other memory locations will have the same value between statements. This is exactly the same problem that languages such as C++ have when dealing with multithreading, and why constructs such as semaphores, mutexes, etc. exist. JS does not have these, and using the technique in the article will completely break the language. Compare these two fiddles I prepared and see for yourself: This one uses await to block for a second, which in the meantime changes the value of the variable between statements: https://jsfiddle.net/gq1btLdu/ https://jsfiddle.net/gq1btLdu/ This one uses a for loop to do synchronous work for about a second, then prints the initial value twice. You will notice that the setInterval never had the chance to run, since callbacks are scheduled to run only once the current function ends: https://jsfiddle.net/gq1btLdu/1/ https://jsfiddle.net/gq1btLdu/1/

protoduction 5y ago
You are right that it's a terrible idea (I hope it was clear from the article that it's an experiment and should stay that way), but could you help me understand why this is fundamentally broken? I would say that by using setInterval you are buying into this kind of behavior and would have to create your own synchronization. If you are saying that this transpilation step changes the semantics of the code, then I completely agree!
- ermir 5y ago
  If you have code like: let x = 0; doSomethingAsync(); // does not touch x console.log(x); There's no guarantee that x will have the value 0 by the time you get to console.log(x). The value of x might have been changed by an outside function that just happened to run between your statements. In synchronous JS this would never happen as callbacks are scheduled to run only at the end of a function, and if they change x the change will only be applied after x is printed. The change in behavior will break a lot of existing code, since the synchronous behavior of a function is a core feature and assumption of the language.

franciscop 5y ago

I went experimenting in a slightly different direction, there's usually a lot of concatenation going on in Javascript with e.g. array manipulation, so I decided to try to delay the `.then()` as much as possible. So instead of these: const value = await Promise.all(data.map(op1)) .then(value => value.filter(op2)) .then(value => Promise.all(value.map(op3))); // OR (a more readable version) let value = await Promise.all(data.map(op1)); value = value.filter(op2); value = await Promise.all(value.map(op3)); Now you can do this: const value = await swear(data).map(op1).filter(op2).map(op3); I integrated it with my other library `files`, where it's very useful out of the box: const list = await read('data.csv').split('\n').map(...); const readmes = await walk('readme.md').map(read); https://www.npmjs.com/package/swear https://www.npmjs.com/package/swear https://www.npmjs.com/package/files https://www.npmjs.com/package/files

mikeyjk 5y ago
Ooo, I like that. So I'm newer to JS, but this suggests chaining from promises isn't possible, and that's effectively the change you've written?
- franciscop 5y ago
  It is possible chaining promises (as in my first example), just just need to wrap it all in a `.then(value => value.[operation]())`, but this way you can just use the `.[operation]()` straight on the data like you would do if there was no promise in the way.

hetman 5y ago

I'm not sure what you mean about Python. In Python, if you run an async function without an await, you will simply get a coroutine object that is ready to be executed by... someone (it's essentially a generator under the hood). In fact, unlike JavaScript, you can't even call await from a synchronous module context. You have to either schedule that couroutine object onto an existing async event loop, or create a new event loop and run it your self.

akdor1154 5y ago
Yup, Python async functions need to be explicitly awaited / scheduled pretty much exactly the same way as JS... article might be confusing this with the fact that most of the stdlib is just plain synchronous, even for web requests etc.
- jtwaleson 5y ago
  Maybe they used gevent which "magically" makes things async without the keywords.

hliyan 5y ago

What if this is baked into the runtime as a mode? 'use async';

foreigner 5y ago
Yes, or a keyword that you use to declare that a single function works this way "asyncAll".
- hliyan 5y ago
  Or, insert 'use async' inside the target function body (just like with 'use strict' now).

TekMol 5y ago

In languages like Python and Go you write asynchronous code as if it’s synchronous. How do you do that in Python?

brundolf 5y ago
My interpretation was that the author meant "you (normally) write things like web requests in a synchronous (blocking) way". There are important differences though between that and what async/await (even this invisible version of it) does. It's also different from what Go does, in a similar way.

superasn 5y ago

There are many things to hate about JavaScript but the inbuilt forced async nature is one thing I absolutely love about it. Thank god that except a few reminants like `alert` and `prompt` almost everything is async by default. I remember that just until a few years ago the whole tabs and everything would freeze if a site showed a blocking alert popup. Can't imagine what it would do if other things like http req, dom operations, etc were also synchronous.

awestroke 5y ago
DOM operations are always synchronous. When modifying DOM, there is often a full layout+paint cycle before the next line is run.
thomasz 5y ago
There is no inbuild async nature. It just did not have a stdlib whatsoever, so node could come along and offer a completely async one. There were async server frameworks in python before, like twisted and tornado. For a long, long time, the language itself lacked almost all of the niceties that make async programming bearable. Node code quickly devolved into a giant mess of callbacks nested twenty levels deep.

madsbuch 5y ago

> With this flavor of Javascript you can no longer do two asynchronous tasks at the same time, everything will be executed synchronously. The main pattern I use to dispatch multiple promises is resilient to this: const [r1, r2, r3] = await Promise.all([ async () => { ... }, async () => { ... }, async () => { ... } ])

nayuki 5y ago
No. I think your code will be transformed into: const [r1, r2, r3] = await Promise.all(await [ await (async () => { ... }), await (async () => { ... }), await (async () => { ... }), ]);

tomaszs 5y ago

Great thought experiment. I also wished it would be possible to code in Javascript without thinking about asynchronous stuff. It should be possible theoretically.

rasz 5y ago
Id settle for implied async whenever function contains awaits.

hsn915 5y ago

I think this doesn't work for javascript because it actually matters when you await. It has implications for the UI. It also has implications about object life times. Of course, the same problem can happen in Go if you use it to make a GUI application, but Go is built entirely around the concept of goroutines so it has obvious ways to handle the problem. Javascript does not. `await` as an explicit suspension point is a pretty good way to handle the problem. I view being "annoyed" at having to be explicit about control flow to be not a real problem. By that I mean, whatever "solution" you come up with is likely to bring with it its own set of problems that no one wants to deal with. And for what price? Because you are "annoyed" at having to type "await"?

BigJono 5y ago
Exactly. What is with this fascination of removing information from code? My first thought when reading this was, ok, OP is "annoyed" at having to "run functions to see if they return a value or a Promise", then goes on to describe a world in which you have no idea whether anything returns a value or a Promise when reading the code. That's 100x worse. At least when I'm writing it I'm probably running the function anyway. And it's worth pointing out that you shouldn't need to run anything to know what it returns, that's what docs are for. Or you could just look at the return statement. But even if for whatever reason you don't know what data you're working with, I'd rather figure it out once and then make it explicit, than have to figure it out every single time the code is read (which could be hundreds of times, or more).

jtwaleson 5y ago

If we designed a new async language today, would it include async/await keywords? I think they don't add anything conceptually and it's just a patch for languages that didn't design for async from the start. Am I missing something?

jayd16 5y ago
Maybe not async as I think the await keyword is probably enough on its own. As for why it exists... Await/explicit yielding enables cooperative multitasking where the programmer has exclusive control of some execution context, be it a thread or fiber or whatever, until they decide to yield. Without that you only have implicit scheduling and execution could be yielded at unpredictable times. The programmer then needs to add other things like semaphores to specify the critical sections and you run the risk of deadlocks. There might be a new language that comes along but right now await exists for a reason.
ledauphin 5y ago
I think this is a compelling vision, but as a practitioner of functional core/imperative shell, I make use of the async/await keywords to understand where I/O is happening and to work towards moving it to the edges of my program. I can do this without those keywords, but there's actually something quite nice about being certain that a given function does no I/O simply based on its function signature.

smasher164 5y ago

If you're interested by this, take a look at Stopify[1], which further adds first-class continuations to javascript. [1] https://www.stopify.org/ https://www.stopify.org/

lioeters 5y ago
Ooh, thank you for a fascinating reference about Stopify. I'll be reading the paper [1] it's based on with great interest. Just recently I was reading "Exceptional Continuations in JavaScript" [2], which I see is cited in the Stopify paper - so it's like a continuing saga. [1] Putting in All the Stops: Execution Control for JavaScript - https://arxiv.org/pdf/1802.02974.pdf https://arxiv.org/pdf/1802.02974.pdf (PDF) [2] Florian Loitsch. 2007. Exceptional Continuations in JavaScript. In Workshop on Scheme and Functional Programming. http://www.schemeworkshop.org/2007/procPaper4.pdf http://www.schemeworkshop.org/2007/procPaper4.pdf (PDF)

kennu 5y ago

I remember debating this when async/await support was first coming out. My opinion was (and still is) that it would have been better to make "await" the default behavior and have a separate "noawait" keyword for those rare cases when you actually really want the promise instead of the value.

BrandoElFollito 5y ago

I am a casual user of JS and was always wondering how to write true asynchronous routines in JS. The kind you fire and forget and they end when they end. An example would be fetch, but all the docs/tutorials I saw are needed on async await where you have to explicitly yield in your code (as opposed, say, to threading in python or goroutines)