8 ms·

The Problem of Async Programming, and a Crazy Idea for Solving It

14 points by lorean_victor 8y ago

imcotton 8y ago
Yes, because XML is too old to us.
- lorean_victor 8y ago
  and to be fair its a really redundant format. I mean I personally would prefer the current undesirable situation rather than doing XML-based coding all day.
flukus 8y ago
I don't think the leap to visual programming is necessary considering we have tools like make that work essentially identically and remain text based. All this is really doing is moving the dependency declaration to lines instead of words and this would get very messy very quickly. An example in make (if I'm reading in the right direction) format which I'd argue is more readable and could scale much better would be: doA: doC doE doB: doD doE doC: doE doD: doE
- lorean_victor 8y ago
  Shouldn't it be like doE: doC, doB, doA doD: doB etc? anyways the problem with this approach is that doD() is assumed to be a general function (or statement or async command) that is not by definition bound to doB(), but rather in this specific case we need it to be executed after doB() is done.
  - flukus 8y ago
    > but rather in this specific case we need it to be executed after doB() is done. Why is that? If it needs any result from doB() then it is a dependencies that must be specified. If it needs to be done only after doB is successful then it is a dependency, just an implicit one we rely on the procedural execution to handle. In either case some sort of surrogate dependency (or better syntax) would have to be introduced. But the point is that either would necessitate or be aided by graphical programming.
    - lorean_victor 8y ago
      ok imagine doB() is a function that loads a record from the database and doD() is for example a password hash matcher (really imperfect example, just trying to get the point across). You might want to use both doB() and doD() in other contexts as well, so basically it is not the case that you would want to execute doD() always after executing doB(), neither is it the case that executing doD() always requires executing doB() beforehand. It just happens that in a specific case you would want to execute doB() and then pass its result to doD().
jmull 8y ago
> To see that for yourself, try to re-write our async code snippet again using only async/await statements and see how does that pan out. With async/await, wouldn't the example just be: let aOut = doA(); let bOut = doB(); let cOut = doC(aOut); doD(bOut); doE(cOut, aOut, bOut); That is, it looks the same as the sync version. The difference is in the "do" functions themselves: they would be async (meaning they return a promise, and they accept promises as parameters).
- lorean_victor 8y ago
  so in your code sample, would 'aOut' be a promise or a value? in case it is a promise, then basically doC() would needed to be re-written to handle when it gets a promised value instead of an actual value, and if not, then doA() must be invoked synchronously, i.e. the `let bOut = doB()` statement must be awaiting `let aOut = doA()` statement which is not what the desired async execution flow is.
  - jmull 8y ago
    Yes, aOut would be a promise. It would look like this: async function doA() { ... return "the-result-of-a"; } (Which returns a promise. Assuming execution reaches that last line of my sample code, the promise will resolve to "the-result-of-a" when you "await" it.) > ...then basically doC() would needed to be re-written to handle when it gets a promised value instead of an actual value Well, wait a minute... it wasn't written in the first place! Let's write it though. It would look like this: async function doC( param ) { ... // we use "await param" where we want to // access the value of param. e.g.: const paramValue = await param; ... } So not bad.
    - lorean_victor 8y ago
      yeah really not bad at all, didn't think of that solution originally. edited the article accordingly.
Ooolin 8y ago
Is the author aware of Dataflow programming ? These do give the described benefits, however, the tradeoffs are massive.
- lorean_victor 8y ago
  to a pretty limited degree, could you elaborate more?
  - CyberDildonics 8y ago
    You basically reinvented it, along with directed acyclical graphs and called it a 'crazy idea for solving async programming'.
    - lorean_victor 8y ago
      The "crazy" label is merely a hint to the inherent resistance of most programmers towards "visual" tools, disregarding them for reasons not-withstanding in this case. yet here it seems like actually the "visual" way is the approach with more fine-control and detailed (and maybe sometimes more complex as you embrace the async nature further). It was by no mean an indicator of any of this being a "new" or "novel" idea. p.s. while it originally used to be acyclic graphs, I had to relax that constraint due to people's request.
  - Ooolin 8y ago
    - Syntax is slower to write. Copying and pasting is a non trivial operation. In order to benefit for the clean representation of the process, you need to spend a good amount of time ordering the boxes around. The representation can be misleading (wires crossing, overlapping boxes) - Very low density of information per pixel compared to any textual language (if no zoom feature). Any long name that has to be rendered on screen become fugly. Since we tend to order the boxes in a process from left to right, only the equivalent of a single line of code can be represented on the screen at once when displaying the names of variable and functions. - Functions / sub diagrams generally sit in their own separate window (or worse, file). This discourage making functions, to a degree depending on how good/bad the UI is. Getters and setters in OO visual code are a pain. - Execution order is not guaranteed to be sequential, so no try/catch structure. Error handling has to be explicit. Witch mean dragging along an additional error variable all over the place. - The graph representation doesn't translate well into source control and diff tools. - If you need to fire an arbitrary number of parallel stuff at runtime, you still need a "call this asynchronously" structure. Coming from a LabVIEW dev. The benefits described in the article are still absolutely valid.
karmakaze 8y ago
Doesn't seem new at all, just like functional programming with lazy values. The new part isn't the async it's the visual programming of which there are others. Does incorporation async make visual programming different?
- lorean_victor 8y ago
  Yeah I don't think so that on its own the visual programming makes any difference (or even adds any value). The main point here is that the "visual programming" is coincidentally a solution for async programming.
  - karmakaze 8y ago
    The actual problem with async functions is mixing them with non-async ones. Having all futures/promises makes it easy. Now if the visual style could represent current mixed js code, that could be awesome for display and editing I'd have to try before deciding.
    - lorean_victor 8y ago
      Yeah we reached that point pretty early on while developing the PaaS with the tool, so had to incorporate support for mixed synchronous JS blocks. you can check out how to do that here: https://medium.com/connect-platform/inline-coding-in-connect-platform-534fce3c8cdf https://medium.com/connect-platform/inline-coding-in-connect...
ricardobeat 8y ago
Callback hell was never about promises, but callbacks (!) which would require a third-party library or some gymnastics to implement control flow. Promises and async/await both solve it reasonably well. The problem presented can be rewritten as: Promise.all([ doA().then(a => doC(a).then(c => [a, c])), doB().then(b => doD(b), b_; return b; }) ]).then(([a, c], b) => doE(c, a, b)) No need for `new Promise(resolve => ...)`. Or with async/await: const [a, b] = Promise.all([doA(), doB()]) doD(b) const c = await doC(a) doE(c, a, b) Is it worth introducing the complexity of visual programming to solve this 'problem'? I have a much harder time understanding the graph than the code.
- lorean_victor 8y ago
  yeah I think the article actually starts with that Promise-based code snippet. Even the sequential graph is based on the promise-based code.
  - ricardobeat 8y ago
    Look closer - the original is way more convoluted - there is no reason to use `new Promise` constructors when you're dealing with other promise-returning functions.
    - lorean_victor 8y ago
      yes you are right the constructors aren't indeed needed. as for the complexity of the graph, basically the way I see it is that even this promised based method is a way to solve the problem of putting the graph into the proper promise-based format (for example, chain doA and doC and return an array of their result) vs simply drawing the intended graph. I mean it does take sometime to get used to it specially if you are really used to just working with code (which I myself originally was), but since the graph is the more intuitive way to represent it compared to other methods, after going through the learning curve it seems to be less complex and easier to manage. that said its not like I have conducted proper academic research on the matter and it might as well be a pretty subjective thing.

14 points by lorean_victor 8y ago

imcotton 8y ago

Yes, because XML is too old to us.

lorean_victor 8y ago
and to be fair its a really redundant format. I mean I personally would prefer the current undesirable situation rather than doing XML-based coding all day.

flukus 8y ago

I don't think the leap to visual programming is necessary considering we have tools like make that work essentially identically and remain text based. All this is really doing is moving the dependency declaration to lines instead of words and this would get very messy very quickly. An example in make (if I'm reading in the right direction) format which I'd argue is more readable and could scale much better would be: doA: doC doE doB: doD doE doC: doE doD: doE

lorean_victor 8y ago
Shouldn't it be like doE: doC, doB, doA doD: doB etc? anyways the problem with this approach is that doD() is assumed to be a general function (or statement or async command) that is not by definition bound to doB(), but rather in this specific case we need it to be executed after doB() is done.
- flukus 8y ago
  > but rather in this specific case we need it to be executed after doB() is done. Why is that? If it needs any result from doB() then it is a dependencies that must be specified. If it needs to be done only after doB is successful then it is a dependency, just an implicit one we rely on the procedural execution to handle. In either case some sort of surrogate dependency (or better syntax) would have to be introduced. But the point is that either would necessitate or be aided by graphical programming.
  - lorean_victor 8y ago
    ok imagine doB() is a function that loads a record from the database and doD() is for example a password hash matcher (really imperfect example, just trying to get the point across). You might want to use both doB() and doD() in other contexts as well, so basically it is not the case that you would want to execute doD() always after executing doB(), neither is it the case that executing doD() always requires executing doB() beforehand. It just happens that in a specific case you would want to execute doB() and then pass its result to doD().

jmull 8y ago

> To see that for yourself, try to re-write our async code snippet again using only async/await statements and see how does that pan out. With async/await, wouldn't the example just be: let aOut = doA(); let bOut = doB(); let cOut = doC(aOut); doD(bOut); doE(cOut, aOut, bOut); That is, it looks the same as the sync version. The difference is in the "do" functions themselves: they would be async (meaning they return a promise, and they accept promises as parameters).

lorean_victor 8y ago
so in your code sample, would 'aOut' be a promise or a value? in case it is a promise, then basically doC() would needed to be re-written to handle when it gets a promised value instead of an actual value, and if not, then doA() must be invoked synchronously, i.e. the `let bOut = doB()` statement must be awaiting `let aOut = doA()` statement which is not what the desired async execution flow is.
- jmull 8y ago
  Yes, aOut would be a promise. It would look like this: async function doA() { ... return "the-result-of-a"; } (Which returns a promise. Assuming execution reaches that last line of my sample code, the promise will resolve to "the-result-of-a" when you "await" it.) > ...then basically doC() would needed to be re-written to handle when it gets a promised value instead of an actual value Well, wait a minute... it wasn't written in the first place! Let's write it though. It would look like this: async function doC( param ) { ... // we use "await param" where we want to // access the value of param. e.g.: const paramValue = await param; ... } So not bad.
  - lorean_victor 8y ago
    yeah really not bad at all, didn't think of that solution originally. edited the article accordingly.

Ooolin 8y ago

Is the author aware of Dataflow programming ? These do give the described benefits, however, the tradeoffs are massive.

lorean_victor 8y ago
to a pretty limited degree, could you elaborate more?
- CyberDildonics 8y ago
  You basically reinvented it, along with directed acyclical graphs and called it a 'crazy idea for solving async programming'.
  - lorean_victor 8y ago
    The "crazy" label is merely a hint to the inherent resistance of most programmers towards "visual" tools, disregarding them for reasons not-withstanding in this case. yet here it seems like actually the "visual" way is the approach with more fine-control and detailed (and maybe sometimes more complex as you embrace the async nature further). It was by no mean an indicator of any of this being a "new" or "novel" idea. p.s. while it originally used to be acyclic graphs, I had to relax that constraint due to people's request.
- Ooolin 8y ago
  - Syntax is slower to write. Copying and pasting is a non trivial operation. In order to benefit for the clean representation of the process, you need to spend a good amount of time ordering the boxes around. The representation can be misleading (wires crossing, overlapping boxes) - Very low density of information per pixel compared to any textual language (if no zoom feature). Any long name that has to be rendered on screen become fugly. Since we tend to order the boxes in a process from left to right, only the equivalent of a single line of code can be represented on the screen at once when displaying the names of variable and functions. - Functions / sub diagrams generally sit in their own separate window (or worse, file). This discourage making functions, to a degree depending on how good/bad the UI is. Getters and setters in OO visual code are a pain. - Execution order is not guaranteed to be sequential, so no try/catch structure. Error handling has to be explicit. Witch mean dragging along an additional error variable all over the place. - The graph representation doesn't translate well into source control and diff tools. - If you need to fire an arbitrary number of parallel stuff at runtime, you still need a "call this asynchronously" structure. Coming from a LabVIEW dev. The benefits described in the article are still absolutely valid.

karmakaze 8y ago

Doesn't seem new at all, just like functional programming with lazy values. The new part isn't the async it's the visual programming of which there are others. Does incorporation async make visual programming different?

lorean_victor 8y ago
Yeah I don't think so that on its own the visual programming makes any difference (or even adds any value). The main point here is that the "visual programming" is coincidentally a solution for async programming.
- karmakaze 8y ago
  The actual problem with async functions is mixing them with non-async ones. Having all futures/promises makes it easy. Now if the visual style could represent current mixed js code, that could be awesome for display and editing I'd have to try before deciding.
  - lorean_victor 8y ago
    Yeah we reached that point pretty early on while developing the PaaS with the tool, so had to incorporate support for mixed synchronous JS blocks. you can check out how to do that here: https://medium.com/connect-platform/inline-coding-in-connect-platform-534fce3c8cdf https://medium.com/connect-platform/inline-coding-in-connect...

ricardobeat 8y ago

Callback hell was never about promises, but callbacks (!) which would require a third-party library or some gymnastics to implement control flow. Promises and async/await both solve it reasonably well. The problem presented can be rewritten as: Promise.all([ doA().then(a => doC(a).then(c => [a, c])), doB().then(b => doD(b), b_; return b; }) ]).then(([a, c], b) => doE(c, a, b)) No need for `new Promise(resolve => ...)`. Or with async/await: const [a, b] = Promise.all([doA(), doB()]) doD(b) const c = await doC(a) doE(c, a, b) Is it worth introducing the complexity of visual programming to solve this 'problem'? I have a much harder time understanding the graph than the code.

lorean_victor 8y ago
yeah I think the article actually starts with that Promise-based code snippet. Even the sequential graph is based on the promise-based code.
- ricardobeat 8y ago
  Look closer - the original is way more convoluted - there is no reason to use `new Promise` constructors when you're dealing with other promise-returning functions.
  - lorean_victor 8y ago
    yes you are right the constructors aren't indeed needed. as for the complexity of the graph, basically the way I see it is that even this promised based method is a way to solve the problem of putting the graph into the proper promise-based format (for example, chain doA and doC and return an array of their result) vs simply drawing the intended graph. I mean it does take sometime to get used to it specially if you are really used to just working with code (which I myself originally was), but since the graph is the more intuitive way to represent it compared to other methods, after going through the learning curve it seems to be less complex and easier to manage. that said its not like I have conducted proper academic research on the matter and it might as well be a pretty subjective thing.