16 ms·

What Color Is Your Function? (2015)

106 points by chrfrasco 6y ago

benhoyt 6y ago
I love this about Go. All functions are simple and synchronous, but if you want to call them concurrently, just "go SlowThing()" and coordinate with a channel. Compare that to the async stuff in C#, Python, etc -- it's bolted on later, and you see double of everything.
- bcrosby95 6y ago
  You're just using the channel as a future. None of this is really problematic at the top level view of things. But when you need to compose libraries or applications that make use of these things - yes, even channels in Go - you can start running into problems. Especially if you don't actually control the process you're running in. This is why promises and async/await really exist. E.g. if you have code you need to fork off the main UI thread so you don't block it, but need to re-capture the UI thread so you can call UI update code when you're done with your long running task. async/await is so much cleaner here. Async/await allow for much more complex control of flow than simple goroutines and channels. Sometimes it's necessary. Sometimes it isn't. It was necessary in nodejs. It's extremely useful in applications where you have a "main thread" that drives your application that you don't want to unnecessarily block. Go doesn't solve this problem out-the-box. There's a reason why c# adopted async/await despite having futures and all sorts of other tools.
  - philosopher1234 6y ago
    I don’t see how futures can be more flexible than go routines. Could you explain that more? And why couldn’t you just spawn a goroutine to avoid blocking your main thread?
    - deleted 6y ago
      [deleted]
    - jayd16 6y ago
      Sometimes you want to block the main thread, or at least finish what you were doing. With async/await and cooperative concurrency you can be explicit about what will run on a thread. You retain control of the thread until you yield or await. You can ask tasks to complete on other threads or post back to the main thread. You have a lot of control. Its easy to write code without locks that runs concurrently on the main thread but is still able to build a UI in a threaded way. I don't really know how go UI frameworks work. How do you have multiple, preemptively scheduled goroutines on the UI thread but without critical sections? You have to use channels and message passing back to a main thread manager to handle this, yes? I think Java's Loom would have the same issue but again, I don't really know. Perhaps worrying about a UI thread is 'fighting the last war' and we should work on new UI paradigms in these new language features.
      - anonymoushn 6y ago
        You can spawn a goroutine that immediately waits on a channel, so that it will not actually do anything until you want it to. This seems at least as expressive as a single-threaded switch() primitive. I think it can also express the threaded async pattern you want, but I'm not sure.
        jayd16 6y ago
        Are there any popular, idiomatic Go UI frameworks I could explore?
        oefrha 6y ago
        You can try your luck with https://github.com/avelino/awesome-go#gui https://github.com/avelino/awesome-go#gui I explored the landscape earlier this year when I was building a GUI version of a CLI tool written in Go. I was throughly disappointed by all the “native Go” and non-webview options — narrow selection of widgets with basic functionality missing, UIs ranging from lack of polish to horrendous looking (that makes Java GUIs look like godsend), lots of bugs, etc. I ended up using the Qt binding which, despite its own share of problems, at least worked fairly reliably and didn’t constantly get in my way in every way imaginable: https://github.com/therecipe/qt https://github.com/therecipe/qt
    - bcrosby95 6y ago
      The point of async/await is to make code that exits and returns to some kind of execution context - without blocking either context - read like plain, synchronous code. A simple example of where this might be useful is in UI code. Every UI framework I've worked in has a single UI thread, where all UI changes must go - e.g. adding a button, adding text to the UI, etc. At the same time, you don't want to block the UI thread, because that will cause the UI to become unresponsive. A common thing you might do in a UI is: on button click, make an API call, then update the UI with that data. The on button click will be called from the UI thread. You don't want to make the API call there though because that would block the UI thread. So you spin it off into a worker thread. But after that worker finishes, you need to regain control of the UI thread so you can update the UI. The way you traditionally do this is you have some kind of main loop which is running the UI thread, which has (among other things) some kind of queue of functions it should call (I assume it would be a channel in Go). So your worker thread will hold onto a reference to this queue. When it finishes, it pushes a function call onto the queue. The main UI loop occasionally checks this queue and calls all functions in it. So eventually it gets called, allowing you to update the UI in the UI thread. This can be fine for simple scenarios. But if you have deeply nested or complex interactions between multiple threads, the code can get confusing and turn into "callback hell". This is the problem that was ran into in early versions of nodejs. This is the problem async/await was intended to solve. It's a bigger problem in nodejs due to its design, but it can still be useful in other languages that don't share that design depending upon the application you're writing.
  - kjksf 6y ago
    A channel can be used as future / promise but that's not what it is and that's not how it's used by majority of Go programs. Channel is what it says it is: a way to send data between goroutines. Also, Go works just fine for UI code, out of the box. See https://github.com/lxn/walk https://github.com/lxn/walk for one of many examples. You just lock main goroutine with runtime.LockOSThread() to its thread and that's your UI thread and marshall code that touches UI to UI thread. Which is the same thing you must do in C# (or any other language). See https://github.com/golang/go/wiki/LockOSThread https://github.com/golang/go/wiki/LockOSThread
- BiteCode_dev 6y ago
  Async def are not colored functions in python. In fact there are not function at all, just like generators are not. Also coroutines can be called from functions and vice versa. You don't have to choose. The problem is not the color. The problem is the default paradigm is to be blocking I/O, and you add non blocking on top. It's way easier to make everything non blocking, then add blocking things on top. It's not a matter of syntax, but of core paradigm. Go and erlang were designed from the non blocking perspective from the get go. But even go add to use something to manage async control flow, so it uses channels, which you don't need if you have coroutines. Erland went deeper and made everything immutable, and baked in actors in the runtime. To me that's more impressive.
  - anonymoushn 6y ago
    Can you expand on your notion of what is and is not a colored function? I might use this test: When I wish to add a call to a function that yields control other than by returning to the middle of a regular function, so that the latter portion of the regular function can use the results of yielding, do I then have to change the function declaration and every single call site? You clearly disagree. What test should we use?
    - BiteCode_dev 6y ago
      It's like calling classes colored functions because you can both call them. Classes are not functions. They are classes. Coroutines are not functions, they are coroutines. Different primitives. A function has a single exit entry point and a single exit point and no state. If you call a function, you run its body. A coroutine has several entry points and exit points, and an internal state. If you call a coroutine, you make an instanciation (the body doesn't run). You can argue than using coroutine for async handling has drawbacks, but colored functions is not the proper analogy and make people very confused about the whole thing. The fact the syntax to define them is so similar doesn't help, and I see most people difficulties comming from their attempt to reconciliate models that have no reason to be. Just like a hashmap and an array are 2 different things despite you can index both, functions and coroutines must be understood as separated concepts.
- tsimionescu 6y ago
  You can actually do the same thing as `go SlowThing() ` in C#, though it has more boilerplate. For example, you can do `Task.Factory.StartNew(() => SlowThing())`. The thing is, Go doesn't have async functions, because it doesn't have await. That is why you don't have colored functions in Go: it doesn't support anything as advanced. Sure, the runtime does really cool things under the hood, but the Go programming model is more like Threads than async/await, because goroutines can't return data and they can't throw exceptions.
  - dullgiulio 6y ago
    Go does not have exception (exactly because of this problem.) For passing information, you use channels, which can pass more than one value back to the caller. The big thing is not running tasks in the background but the tight integration of channels and runtime scheduler that allows having an invisible event loop on top of what is synchronous programming.
    - tsimionescu 6y ago
      I think there are many reasons why Go doesn't use exceptions, though this is possibly one of them. Channels are nice, but they are not that hard to replicate in other languages (probably less efficiently). They have their uses, but they can't completely supplant shared memory or async/await - all 3 are nice to have in different situations.
  - jayd16 6y ago
    Agreed. Await seems much nicer for the case of some sleep/wait causing subroutine that returns something. var foo = await bar(); vs c1 := make(chan float64, 1) go bar(c1) foo := <-c1 Sure you have to be in an async method to use await but its really not that bad if you embrace it. Go seems better suited for high throughput message passing.
    - philwelch 6y ago
      Except in Go you could just as easily call: foo := bar() and if bar() is a function that does something annoying or time-consuming before finishing its work and returning the value, it just behaves like a normal, synchronous function call. “Await” is, as TFA explains, syntactic sugar for slapping an async function into behaving like a normal synchronous function call. In Go there is no reason you can’t just write your function calls synchronously in the first place.
- hombre_fatal 6y ago
  Go routines just act like threads. Something you can spawn in any language with the same guarantees. Even C/C++ have channel libraries. Channels just easily introduce complexity and channel-hell which is why they aren't all that popular as abstractions. Even in Go. Frankly, I have a hard time imagining too many seasoned developers clutching channels to their chest as something superior to async/await. Async code with channels is not simple, you have in-channels, out-channels, and channels-over-channels. It's a pretty niche instrument working best when you only have uni-directional communication in your application. For example, I've worked on channel code where I'd rather be debugging mutex deadlocks.
- ghostwriter 6y ago
  Async stuff in Python can be encoded in multiple ways, with gevent everything is a regular function. http://sdiehl.github.io/gevent-tutorial/ http://sdiehl.github.io/gevent-tutorial/
- aaai 6y ago
  EXACTLY! But... don't bother too much trying to explain this to ppl who don't intuitively grok it right away... some just seem to never get it no matter how hard you try and explain it to them, it's like their brains are "wired differently" when it comes to reading and understanding code, they don't get the advantage and meaning of unification / universality / "one solution for many problems" etc. Go is NOT my favorite programming language, but there's a stroke of simple genius in it that probably only got materialized bc its creators were left alone to work on it their way inside Google and just implemented their solution to things without being bothered by "language experts"... EDIT+: not saying "colorless + channels" is always better or anything like that, all's tradeoffs... probably async/await code is much more readable than channels hell for many/most cases (but because it's less powerful - no true parallelism possible).
knrz 6y ago
Also one of my favourite points about Elixir/Erlang — having no distinctions between async/await makes programming flow better.
- BiteCode_dev 6y ago
  That's the benefit of integrating async in the runtime itself. It abstracts it from the language, which doesn't have to know about it. Just like a GC abstract memory handling.
  - geocar 6y ago
    Right. Even POSIX.1 has the same "messaging" functionality, so: * spawn/1 becomes fork() * send/2 becomes kill() * receive F -> ... G -> ... end becomes sigaddset(&s,F);sigaddset(&s,G);sigwait(&s,&r) There are of course other issues, like there aren't that many signals, or that memory management is hard, or whatever, but these are solved by other languages as well, so you can imagine this is at least straightforward to implement. The downside is that it means you can have functions that you can't write. That can be a big bummer to a lisp programmer, but I don't think python, javascript, etc, programmers care about that, so I think history will judge the trend to colour functions as lazy and shortsighted.
dang 6y ago
See also... 2018: https://news.ycombinator.com/item?id=16732948 https://news.ycombinator.com/item?id=16732948 discussed at the time: https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648 a bit more from the same day: https://news.ycombinator.com/item?id=8982494 https://news.ycombinator.com/item?id=8982494
franciscop 6y ago
I do Javascript and wish throwing quick scripts was a bit easier, which are made a bit harder with promises/async since now you have to separate your scripts and functions depending on their color. So I made a library to help me[1]: const name = await swear(fetch('/some.json')).json().user.name; console.log(name); // Francisco const error = await swear(readFile('./error.log')).split('\n').pop(); console.log(error); // *latest error log message* It makes all functions to look like blue functions, but internally they are all red. I made this by using `Proxy()`[2], then queuing the operations and waiting for any unfinished one on the last operation, which is always a `.then()` (since there's an `await`). It is fully compatible with native promises. While I do not use it directly since adding a library to make syntax slightly shorter defeats the point, I've included it into some of my async libraries: • File handler `files`: https://www.npmjs.com/package/files https://www.npmjs.com/package/files • Simple command runner `atocha`: https://www.npmjs.com/package/atocha https://www.npmjs.com/package/atocha • Enhanced fetch() `fch`: https://www.npmjs.com/package/fch https://www.npmjs.com/package/fch [1] https://www.npmjs.com/package/swear https://www.npmjs.com/package/swear [2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
sergiotapia 6y ago
Thank god i code in elixir
- waynesonfire 6y ago
  I have similar sentiment when I'm reviewing modern java code that's riddled with Futures.
afc 6y ago
Interesting read. I'm not sure I agree with the conclusion: I find using futures with very carefully controlled threading a preferable paradigm (for how I structure my programs in C++, at least for my text editor: https://github.com/alefore/edge https://github.com/alefore/edge). In my experience, using sync code looks more readable on the surface but just kicks the can down the road: you'll still need to deal with the complexity of threading, and, in my experience, it's going to be waaay uglier. What you gain with your "superficial" simplicity, you pay a hundred times over with the algorithmic complexity of having to use threads with shared state. Every time you're troubleshooting some weird race condition that you can't easily reproduce you'll be wishing you had just used futures. What I do is that the bulk of my processing runs in the main thread and I occasionally dispatch work to other threads, making sure that no mutable state is shared. When the "async" work is finished, I just have the background threads communicate the results to the main thread by setting a future (i.e., scheduling in the main thread the execution of the future's consumer on the results). Async IO operations are modeled just the same. In the beginning I had used callbacks spaghetti (mostly writing things in continuation passing style), but I started trying futures and found them much nicer. I'll admit that on the surface this makes my code look slightly uglier than if it was directly sync code; however, it allows me to very safely use multiple threads. I think not having to make my classes thread safe (I typically stop at making them thread-compatible) and not having to troubleshoot difficult race conditions (and not having to block on IO or being able to easily run background operations) has been a huge win. If I had to rewrite this from scratch, I'd likely choose this model again. My editor only needs to use mutexes and such in very very few places; it suffices to make my classes thread-compatible and to ensure that all work in threads other than the main thread happens on const objects (and that such objects don't get deallocated before the background threads are done using them). I rolled out my own implementation of futures here: https://github.com/alefore/edge/blob/master/src/futures/futures.h https://github.com/alefore/edge/blob/master/src/futures/futu... (One notable characteristic is that it only allows a single listener, which I found worked well with "move" semantics for supporting non-copyable types that the listener gets ownership of.) Here is one example of where it is used: https://github.com/alefore/edge/blob/5cb6f67e1e0726f8fbe12db3d1e0cd12b07d5135/src/buffer.cc#L919 https://github.com/alefore/edge/blob/5cb6f67e1e0726f8fbe12db... In this example, it implements the operation of reloading a buffer, which is somewhat complex in that it requires several potentially long running or async operations (such as the evaluation of several "buffer-reload.cc" programs, opening the file, opening a log for the file, notifying listeners for reload operations...). It may seem uglier than if it was all in sync code, but then I would have to either block the main thread (unacceptable in this context) or make all my classes thread safe (which I think would be significantly more complexity). I think this is cleaner than callbacks spaghetti because the code still reflects somewhat closely its logical structure. For loops I do have to use futures::ForEach functions, as the example shows, which is unfortunate but acceptable. In my experience, with callbacks spaghetti, it is very difficult to make code reflect its logical structure.
smabie 6y ago
I thought he was talking about IO functions in Haskell. But I guess async works too.
- afc 6y ago
  Yeah, that was where I also had guessed he was going as I was reading. :-)
KingOfCoders 6y ago
Still we need a language where every function call is async and the runtime decides what to inline.
- hombre_fatal 6y ago
  Looking at my async code, I just don't see how that would make things more clear instead of less clear. For example, something as simple as: const result1 = promise() // start this promise first but don't await it yet. // ... const results = await Promise.map([promise(), promise(), result1])
  - KingOfCoders 6y ago
    It's more that you don't need to add 'async' to each function, because every function is async. And working with promises automatically maps over them. Then if you want to 'unbox' the Promise you'd have a keyword, like await. So instead of a.map(v => f(v)) ... or Promise.map(a, f) ... or a.map(f) ... it's just f(a). If you then want the value, in the end, if at all (e.g. the web framework understands Promise and you return a Promise in your controller and not a value, you don't need await at all). You'd need some syntax for Promise.all(...) though to "join" concurrent executions again. The language should probably have some syntax for multiple writers, e.g. and atomic compareAndSet. shared v = { .... } v.change(v, f) where f has no side effects and can be executed again. With IO (DB) one would need support for idempotency ala Stripe. But your functions would not have colors.
longtermd 6y ago
I love the idea of coloring functions based on how "heavy" they are on the CPU or how long they take to execute on an "average client desktop/mobile browser".
- saagarjha 6y ago
  I think you’re imagining a flamegraph.
z3t4 6y ago
This article made me angry then I first read it, and now it makes my blood boil. First off there where no colors in JavaScript pre ES6, there where only functions. You can call them first class functions, but those are still functions. A callback is just a function! Then we have side effects, but executing the side effect while returning a promise like in ES6 you are still executing the side effect, even if yo do not call .then() so nothing really changes. So first you think there is a problem, but there are not any, but by solving the imagined problem you do actually create it. You now have generator functions and "async" (with the async keyword) functions that always returns a promise, and normal functions.
- saagarjha 6y ago
  I think you didn’t really understand the argument. The author is not saying that the language lacks first-class functions, in fact he requires it for his real point, which is that async and sync functions are not really compatible.
  - z3t4 6y ago
    Prior to ES6 JavaScript did not have async functions, just normal functions. Async functions came from the runtime, not JavaScript the language. For example there is no setTimeout function in JavaScript, it has to be implemented by the framework that runs the JavaScript. Talking about an function being async makes it confusing though. It's better to think of them as events. You call a function that executes a job, and specify a function to be called (an event handler) when the job has been completed. Putting the callback function in the function arguments is just a convention, other conventions are foo.on("ready") = handlerFunction; foo.ready = handlerFunction; foo.error = errorHandler; bar(handlerFunction); biz(successHandler, errorHandler); The callback convention is less verbose and probably why it's the most popular. Using just functions makes you more powerful...
gorgoiler 6y ago
Unrelated but by complete coincidence I came across the author’s blog yesterday, while researching book creation in AsciiDoctor and Markdown. His write up on the process is also interesting: http://journal.stuffwithstuff.com/2014/11/03/bringing-my-web-book-to-print-and-ebook/ http://journal.stuffwithstuff.com/2014/11/03/bringing-my-web...
Garlef 6y ago
I think this is just a matter of perspective and actually having different colors is a good thing: Thinking in terms of functional programming / category theory, this coloring seems to boil down to working with different arrows. The coloring in this case would correspont to signifying the target category of the arrows `arr/lift` function (For example async functions in JS should be morphisms in smth like the Kleisli category of the `Promise` functor). The issue now seems to be that there are no nice native language constructs in most languages to work with arrows. What's missing are arrow comprehensions - the bridge between the compositional notation used in functional languages and the more traditional way of writing things like `const x = ...`. The vanilla functions without `async/await` are arrow comprehensions for the identity monad. The reason why the `async/await` syntax works so well as compared to using `.then` is that it is basically arrow comprehension for the promise monad.
- rocqua 6y ago
  I have some vague knowledge of category theory, but not enough to follow your argument here. Especially the concept of an 'Arrow' could you explain more clearly what an arrow, and an arrow comprehension is?
  - Garlef 6y ago
    https://en.wikipedia.org/wiki/Arrow_(computer_science) https://en.wikipedia.org/wiki/Arrow_(computer_science) https://en.wikipedia.org/wiki/List_comprehension https://en.wikipedia.org/wiki/List_comprehension https://www.haskell.org/arrows/syntax.html https://www.haskell.org/arrows/syntax.html https://www.sciencedirect.com/science/article/pii/S157106611100051X https://www.sciencedirect.com/science/article/pii/S157106611...
- sfvisser 6y ago
  If you really need to run your computations in a separate context/arrow, making them explicit in the types is nice. Having general syntactic rules to build, compose and run them: great. However, if you can avoid the additional arrow entirely: even better! I mean Haskell of all languages has green threads and doesn’t need the entire promise shenanigans. Even in Haskell I sometimes I build very specific DSLs for my domain to be interpreted in a separate step so I can freely fix my pure and impure code.
bvrmn 6y ago
There is an interesting approach in Zig to explicitly kick call frames to achieve colorless asyncness. https://www.youtube.com/watch?v=zeLToGnjIUM https://www.youtube.com/watch?v=zeLToGnjIUM
k__ 6y ago
I saw TypeScript has now an awaited keyword that will eat promise wrapped and plain values.
elwell 6y ago
Original discussion: https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648
joshribakoff 6y ago
Classic article. In rxjs mapping over a sync array or async stream is exactly the same, for me the problem in the article is not such a big problem in practice for day to day work, especially with tools like rxjs which makes composing mixed code easy
andrewzah 6y ago
This is something I ran into recently when learning about promises and async functions in javsscript. It took me a while to grok it, and I'm still not sure if I fully understand it, but we settled on using try/catch blocks with typescript's await keyword on calls we need to block on. I wanted to use @usefultools/monads to have Maybe types, etc, but it's easier to just stick with promises throughout the chain. Essentially working with functions that return promises (like database calls with Mongoose) makes you convert your functions to also use promises. As far as I can tell there's no way to unwrap a promise (similar to Rust) without the await keyword inside a try block. I don't know if this is exactly the best approach, but it's significantly less verbose than .then().catch() callbacks. I also learned about Promise.all() and Promise.allSettled() recently here [0]. I wish I could stick to futures with Rust or goroutines with golang. Bleh. [0]: https://news.ycombinator.com/item?id=23223881 https://news.ycombinator.com/item?id=23223881
devxpy 6y ago
I've been thinking why exactly async-await was chosen at the function level, and not the caller level. I mean for languages that have event loops at their core, why isn't every function `async` by default? Let the caller decide how it wants to use the function. `async` functions don't wait for a result of a `Future` unless `await` is used. Instead of putting all that effort into putting `await` everywhere, why not invert that logic and make them `await` by default, and instead have the async-await syntax used by callers? Like if one to were to write a transpiler to do this in dart, it might compile the following - void main() { Future<int> xFuture = async doSomething() int x = doSomething() assert(await xFuture == x); } int doSomething { ... } into - Future<void> main() async { Future<int> xFuture = doSomething() int x = await doSomething() assert(await xFuture == x); } Future<int> doSomething async { ... } This might be stupid, but my solution to the "what color is your function problem" is to make every function async. I looks an awful lot like what go does with its `go` syntax, but because you have an event loop with Future and Async Streams, you don't need to worry about dealing with CSP.
- chaorace 6y ago
  I suspect this is for two reasons: * Many popular languages today predate modern asynchronous computing. This makes async-as-default impossible because it's an afterthought * Async computing is harder to learn and confusing for those just picking up the language. There are some pretty major ergonomics issues that you have to solve if you want anything more than a DSL to achieve noteworthy adoption levels.
  - devxpy 6y ago
    > Many popular languages today predate modern asynchronous computing. This makes async-as-default impossible because it's an afterthought Here's a rough sketch of how this can be achieved for a language that already has event loops - Transpile the following - void main() { Future<int> xFuture = async doSomething() int x = doSomething() assert(await xFuture == x); } Into - Future<void> main() async { Future<int> xFuture = (() async => await doSomething())(); int x = await doSomething(); assert(await xFuture == x); } Now people can keep using a pre-existing `doSomething()` (that's either async or sync) and its compatible with our new caller based async statement. BTW that transpiler output is runnable dart code actually works. I know this has tons of edge cases that this simple example doesn't capture, but it's definitely "possible".

106 points by chrfrasco 6y ago

I love this about Go. All functions are simple and synchronous, but if you want to call them concurrently, just "go SlowThing()" and coordinate with a channel. Compare that to the async stuff in C#, Python, etc -- it's bolted on later, and you see double of everything.

bcrosby95 6y ago
You're just using the channel as a future. None of this is really problematic at the top level view of things. But when you need to compose libraries or applications that make use of these things - yes, even channels in Go - you can start running into problems. Especially if you don't actually control the process you're running in. This is why promises and async/await really exist. E.g. if you have code you need to fork off the main UI thread so you don't block it, but need to re-capture the UI thread so you can call UI update code when you're done with your long running task. async/await is so much cleaner here. Async/await allow for much more complex control of flow than simple goroutines and channels. Sometimes it's necessary. Sometimes it isn't. It was necessary in nodejs. It's extremely useful in applications where you have a "main thread" that drives your application that you don't want to unnecessarily block. Go doesn't solve this problem out-the-box. There's a reason why c# adopted async/await despite having futures and all sorts of other tools.
- philosopher1234 6y ago
  I don’t see how futures can be more flexible than go routines. Could you explain that more? And why couldn’t you just spawn a goroutine to avoid blocking your main thread?
  - deleted 6y ago
    [deleted]
  - jayd16 6y ago
    Sometimes you want to block the main thread, or at least finish what you were doing. With async/await and cooperative concurrency you can be explicit about what will run on a thread. You retain control of the thread until you yield or await. You can ask tasks to complete on other threads or post back to the main thread. You have a lot of control. Its easy to write code without locks that runs concurrently on the main thread but is still able to build a UI in a threaded way. I don't really know how go UI frameworks work. How do you have multiple, preemptively scheduled goroutines on the UI thread but without critical sections? You have to use channels and message passing back to a main thread manager to handle this, yes? I think Java's Loom would have the same issue but again, I don't really know. Perhaps worrying about a UI thread is 'fighting the last war' and we should work on new UI paradigms in these new language features.
    - anonymoushn 6y ago
      You can spawn a goroutine that immediately waits on a channel, so that it will not actually do anything until you want it to. This seems at least as expressive as a single-threaded switch() primitive. I think it can also express the threaded async pattern you want, but I'm not sure.
      - jayd16 6y ago
        Are there any popular, idiomatic Go UI frameworks I could explore?
        oefrha 6y ago
        You can try your luck with https://github.com/avelino/awesome-go#gui https://github.com/avelino/awesome-go#gui I explored the landscape earlier this year when I was building a GUI version of a CLI tool written in Go. I was throughly disappointed by all the “native Go” and non-webview options — narrow selection of widgets with basic functionality missing, UIs ranging from lack of polish to horrendous looking (that makes Java GUIs look like godsend), lots of bugs, etc. I ended up using the Qt binding which, despite its own share of problems, at least worked fairly reliably and didn’t constantly get in my way in every way imaginable: https://github.com/therecipe/qt https://github.com/therecipe/qt
  - bcrosby95 6y ago
    The point of async/await is to make code that exits and returns to some kind of execution context - without blocking either context - read like plain, synchronous code. A simple example of where this might be useful is in UI code. Every UI framework I've worked in has a single UI thread, where all UI changes must go - e.g. adding a button, adding text to the UI, etc. At the same time, you don't want to block the UI thread, because that will cause the UI to become unresponsive. A common thing you might do in a UI is: on button click, make an API call, then update the UI with that data. The on button click will be called from the UI thread. You don't want to make the API call there though because that would block the UI thread. So you spin it off into a worker thread. But after that worker finishes, you need to regain control of the UI thread so you can update the UI. The way you traditionally do this is you have some kind of main loop which is running the UI thread, which has (among other things) some kind of queue of functions it should call (I assume it would be a channel in Go). So your worker thread will hold onto a reference to this queue. When it finishes, it pushes a function call onto the queue. The main UI loop occasionally checks this queue and calls all functions in it. So eventually it gets called, allowing you to update the UI in the UI thread. This can be fine for simple scenarios. But if you have deeply nested or complex interactions between multiple threads, the code can get confusing and turn into "callback hell". This is the problem that was ran into in early versions of nodejs. This is the problem async/await was intended to solve. It's a bigger problem in nodejs due to its design, but it can still be useful in other languages that don't share that design depending upon the application you're writing.
- kjksf 6y ago
  A channel can be used as future / promise but that's not what it is and that's not how it's used by majority of Go programs. Channel is what it says it is: a way to send data between goroutines. Also, Go works just fine for UI code, out of the box. See https://github.com/lxn/walk https://github.com/lxn/walk for one of many examples. You just lock main goroutine with runtime.LockOSThread() to its thread and that's your UI thread and marshall code that touches UI to UI thread. Which is the same thing you must do in C# (or any other language). See https://github.com/golang/go/wiki/LockOSThread https://github.com/golang/go/wiki/LockOSThread
BiteCode_dev 6y ago
Async def are not colored functions in python. In fact there are not function at all, just like generators are not. Also coroutines can be called from functions and vice versa. You don't have to choose. The problem is not the color. The problem is the default paradigm is to be blocking I/O, and you add non blocking on top. It's way easier to make everything non blocking, then add blocking things on top. It's not a matter of syntax, but of core paradigm. Go and erlang were designed from the non blocking perspective from the get go. But even go add to use something to manage async control flow, so it uses channels, which you don't need if you have coroutines. Erland went deeper and made everything immutable, and baked in actors in the runtime. To me that's more impressive.
- anonymoushn 6y ago
  Can you expand on your notion of what is and is not a colored function? I might use this test: When I wish to add a call to a function that yields control other than by returning to the middle of a regular function, so that the latter portion of the regular function can use the results of yielding, do I then have to change the function declaration and every single call site? You clearly disagree. What test should we use?
  - BiteCode_dev 6y ago
    It's like calling classes colored functions because you can both call them. Classes are not functions. They are classes. Coroutines are not functions, they are coroutines. Different primitives. A function has a single exit entry point and a single exit point and no state. If you call a function, you run its body. A coroutine has several entry points and exit points, and an internal state. If you call a coroutine, you make an instanciation (the body doesn't run). You can argue than using coroutine for async handling has drawbacks, but colored functions is not the proper analogy and make people very confused about the whole thing. The fact the syntax to define them is so similar doesn't help, and I see most people difficulties comming from their attempt to reconciliate models that have no reason to be. Just like a hashmap and an array are 2 different things despite you can index both, functions and coroutines must be understood as separated concepts.
tsimionescu 6y ago
You can actually do the same thing as `go SlowThing() ` in C#, though it has more boilerplate. For example, you can do `Task.Factory.StartNew(() => SlowThing())`. The thing is, Go doesn't have async functions, because it doesn't have await. That is why you don't have colored functions in Go: it doesn't support anything as advanced. Sure, the runtime does really cool things under the hood, but the Go programming model is more like Threads than async/await, because goroutines can't return data and they can't throw exceptions.
- dullgiulio 6y ago
  Go does not have exception (exactly because of this problem.) For passing information, you use channels, which can pass more than one value back to the caller. The big thing is not running tasks in the background but the tight integration of channels and runtime scheduler that allows having an invisible event loop on top of what is synchronous programming.
  - tsimionescu 6y ago
    I think there are many reasons why Go doesn't use exceptions, though this is possibly one of them. Channels are nice, but they are not that hard to replicate in other languages (probably less efficiently). They have their uses, but they can't completely supplant shared memory or async/await - all 3 are nice to have in different situations.
- jayd16 6y ago
  Agreed. Await seems much nicer for the case of some sleep/wait causing subroutine that returns something. var foo = await bar(); vs c1 := make(chan float64, 1) go bar(c1) foo := <-c1 Sure you have to be in an async method to use await but its really not that bad if you embrace it. Go seems better suited for high throughput message passing.
  - philwelch 6y ago
    Except in Go you could just as easily call: foo := bar() and if bar() is a function that does something annoying or time-consuming before finishing its work and returning the value, it just behaves like a normal, synchronous function call. “Await” is, as TFA explains, syntactic sugar for slapping an async function into behaving like a normal synchronous function call. In Go there is no reason you can’t just write your function calls synchronously in the first place.
hombre_fatal 6y ago
Go routines just act like threads. Something you can spawn in any language with the same guarantees. Even C/C++ have channel libraries. Channels just easily introduce complexity and channel-hell which is why they aren't all that popular as abstractions. Even in Go. Frankly, I have a hard time imagining too many seasoned developers clutching channels to their chest as something superior to async/await. Async code with channels is not simple, you have in-channels, out-channels, and channels-over-channels. It's a pretty niche instrument working best when you only have uni-directional communication in your application. For example, I've worked on channel code where I'd rather be debugging mutex deadlocks.
ghostwriter 6y ago
Async stuff in Python can be encoded in multiple ways, with gevent everything is a regular function. http://sdiehl.github.io/gevent-tutorial/ http://sdiehl.github.io/gevent-tutorial/
aaai 6y ago
EXACTLY! But... don't bother too much trying to explain this to ppl who don't intuitively grok it right away... some just seem to never get it no matter how hard you try and explain it to them, it's like their brains are "wired differently" when it comes to reading and understanding code, they don't get the advantage and meaning of unification / universality / "one solution for many problems" etc. Go is NOT my favorite programming language, but there's a stroke of simple genius in it that probably only got materialized bc its creators were left alone to work on it their way inside Google and just implemented their solution to things without being bothered by "language experts"... EDIT+: not saying "colorless + channels" is always better or anything like that, all's tradeoffs... probably async/await code is much more readable than channels hell for many/most cases (but because it's less powerful - no true parallelism possible).

knrz 6y ago

Also one of my favourite points about Elixir/Erlang — having no distinctions between async/await makes programming flow better.

BiteCode_dev 6y ago
That's the benefit of integrating async in the runtime itself. It abstracts it from the language, which doesn't have to know about it. Just like a GC abstract memory handling.
- geocar 6y ago
  Right. Even POSIX.1 has the same "messaging" functionality, so: * spawn/1 becomes fork() * send/2 becomes kill() * receive F -> ... G -> ... end becomes sigaddset(&s,F);sigaddset(&s,G);sigwait(&s,&r) There are of course other issues, like there aren't that many signals, or that memory management is hard, or whatever, but these are solved by other languages as well, so you can imagine this is at least straightforward to implement. The downside is that it means you can have functions that you can't write. That can be a big bummer to a lisp programmer, but I don't think python, javascript, etc, programmers care about that, so I think history will judge the trend to colour functions as lazy and shortsighted.

dang 6y ago

See also... 2018: https://news.ycombinator.com/item?id=16732948 https://news.ycombinator.com/item?id=16732948 discussed at the time: https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648 a bit more from the same day: https://news.ycombinator.com/item?id=8982494 https://news.ycombinator.com/item?id=8982494

franciscop 6y ago

I do Javascript and wish throwing quick scripts was a bit easier, which are made a bit harder with promises/async since now you have to separate your scripts and functions depending on their color. So I made a library to help me[1]: const name = await swear(fetch('/some.json')).json().user.name; console.log(name); // Francisco const error = await swear(readFile('./error.log')).split('\n').pop(); console.log(error); // *latest error log message* It makes all functions to look like blue functions, but internally they are all red. I made this by using `Proxy()`[2], then queuing the operations and waiting for any unfinished one on the last operation, which is always a `.then()` (since there's an `await`). It is fully compatible with native promises. While I do not use it directly since adding a library to make syntax slightly shorter defeats the point, I've included it into some of my async libraries: • File handler `files`: https://www.npmjs.com/package/files https://www.npmjs.com/package/files • Simple command runner `atocha`: https://www.npmjs.com/package/atocha https://www.npmjs.com/package/atocha • Enhanced fetch() `fch`: https://www.npmjs.com/package/fch https://www.npmjs.com/package/fch [1] https://www.npmjs.com/package/swear https://www.npmjs.com/package/swear [2] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

sergiotapia 6y ago

Thank god i code in elixir

waynesonfire 6y ago
I have similar sentiment when I'm reviewing modern java code that's riddled with Futures.

afc 6y ago

Interesting read. I'm not sure I agree with the conclusion: I find using futures with very carefully controlled threading a preferable paradigm (for how I structure my programs in C++, at least for my text editor: https://github.com/alefore/edge https://github.com/alefore/edge). In my experience, using sync code looks more readable on the surface but just kicks the can down the road: you'll still need to deal with the complexity of threading, and, in my experience, it's going to be waaay uglier. What you gain with your "superficial" simplicity, you pay a hundred times over with the algorithmic complexity of having to use threads with shared state. Every time you're troubleshooting some weird race condition that you can't easily reproduce you'll be wishing you had just used futures. What I do is that the bulk of my processing runs in the main thread and I occasionally dispatch work to other threads, making sure that no mutable state is shared. When the "async" work is finished, I just have the background threads communicate the results to the main thread by setting a future (i.e., scheduling in the main thread the execution of the future's consumer on the results). Async IO operations are modeled just the same. In the beginning I had used callbacks spaghetti (mostly writing things in continuation passing style), but I started trying futures and found them much nicer. I'll admit that on the surface this makes my code look slightly uglier than if it was directly sync code; however, it allows me to very safely use multiple threads. I think not having to make my classes thread safe (I typically stop at making them thread-compatible) and not having to troubleshoot difficult race conditions (and not having to block on IO or being able to easily run background operations) has been a huge win. If I had to rewrite this from scratch, I'd likely choose this model again. My editor only needs to use mutexes and such in very very few places; it suffices to make my classes thread-compatible and to ensure that all work in threads other than the main thread happens on const objects (and that such objects don't get deallocated before the background threads are done using them). I rolled out my own implementation of futures here: https://github.com/alefore/edge/blob/master/src/futures/futures.h https://github.com/alefore/edge/blob/master/src/futures/futu... (One notable characteristic is that it only allows a single listener, which I found worked well with "move" semantics for supporting non-copyable types that the listener gets ownership of.) Here is one example of where it is used: https://github.com/alefore/edge/blob/5cb6f67e1e0726f8fbe12db3d1e0cd12b07d5135/src/buffer.cc#L919 https://github.com/alefore/edge/blob/5cb6f67e1e0726f8fbe12db... In this example, it implements the operation of reloading a buffer, which is somewhat complex in that it requires several potentially long running or async operations (such as the evaluation of several "buffer-reload.cc" programs, opening the file, opening a log for the file, notifying listeners for reload operations...). It may seem uglier than if it was all in sync code, but then I would have to either block the main thread (unacceptable in this context) or make all my classes thread safe (which I think would be significantly more complexity). I think this is cleaner than callbacks spaghetti because the code still reflects somewhat closely its logical structure. For loops I do have to use futures::ForEach functions, as the example shows, which is unfortunate but acceptable. In my experience, with callbacks spaghetti, it is very difficult to make code reflect its logical structure.

smabie 6y ago

I thought he was talking about IO functions in Haskell. But I guess async works too.

afc 6y ago
Yeah, that was where I also had guessed he was going as I was reading. :-)

KingOfCoders 6y ago

Still we need a language where every function call is async and the runtime decides what to inline.

hombre_fatal 6y ago
Looking at my async code, I just don't see how that would make things more clear instead of less clear. For example, something as simple as: const result1 = promise() // start this promise first but don't await it yet. // ... const results = await Promise.map([promise(), promise(), result1])
- KingOfCoders 6y ago
  It's more that you don't need to add 'async' to each function, because every function is async. And working with promises automatically maps over them. Then if you want to 'unbox' the Promise you'd have a keyword, like await. So instead of a.map(v => f(v)) ... or Promise.map(a, f) ... or a.map(f) ... it's just f(a). If you then want the value, in the end, if at all (e.g. the web framework understands Promise and you return a Promise in your controller and not a value, you don't need await at all). You'd need some syntax for Promise.all(...) though to "join" concurrent executions again. The language should probably have some syntax for multiple writers, e.g. and atomic compareAndSet. shared v = { .... } v.change(v, f) where f has no side effects and can be executed again. With IO (DB) one would need support for idempotency ala Stripe. But your functions would not have colors.

longtermd 6y ago

I love the idea of coloring functions based on how "heavy" they are on the CPU or how long they take to execute on an "average client desktop/mobile browser".

saagarjha 6y ago
I think you’re imagining a flamegraph.

z3t4 6y ago

This article made me angry then I first read it, and now it makes my blood boil. First off there where no colors in JavaScript pre ES6, there where only functions. You can call them first class functions, but those are still functions. A callback is just a function! Then we have side effects, but executing the side effect while returning a promise like in ES6 you are still executing the side effect, even if yo do not call .then() so nothing really changes. So first you think there is a problem, but there are not any, but by solving the imagined problem you do actually create it. You now have generator functions and "async" (with the async keyword) functions that always returns a promise, and normal functions.

saagarjha 6y ago
I think you didn’t really understand the argument. The author is not saying that the language lacks first-class functions, in fact he requires it for his real point, which is that async and sync functions are not really compatible.
- z3t4 6y ago
  Prior to ES6 JavaScript did not have async functions, just normal functions. Async functions came from the runtime, not JavaScript the language. For example there is no setTimeout function in JavaScript, it has to be implemented by the framework that runs the JavaScript. Talking about an function being async makes it confusing though. It's better to think of them as events. You call a function that executes a job, and specify a function to be called (an event handler) when the job has been completed. Putting the callback function in the function arguments is just a convention, other conventions are foo.on("ready") = handlerFunction; foo.ready = handlerFunction; foo.error = errorHandler; bar(handlerFunction); biz(successHandler, errorHandler); The callback convention is less verbose and probably why it's the most popular. Using just functions makes you more powerful...

gorgoiler 6y ago

Unrelated but by complete coincidence I came across the author’s blog yesterday, while researching book creation in AsciiDoctor and Markdown. His write up on the process is also interesting: http://journal.stuffwithstuff.com/2014/11/03/bringing-my-web-book-to-print-and-ebook/ http://journal.stuffwithstuff.com/2014/11/03/bringing-my-web...

Garlef 6y ago

I think this is just a matter of perspective and actually having different colors is a good thing: Thinking in terms of functional programming / category theory, this coloring seems to boil down to working with different arrows. The coloring in this case would correspont to signifying the target category of the arrows `arr/lift` function (For example async functions in JS should be morphisms in smth like the Kleisli category of the `Promise` functor). The issue now seems to be that there are no nice native language constructs in most languages to work with arrows. What's missing are arrow comprehensions - the bridge between the compositional notation used in functional languages and the more traditional way of writing things like `const x = ...`. The vanilla functions without `async/await` are arrow comprehensions for the identity monad. The reason why the `async/await` syntax works so well as compared to using `.then` is that it is basically arrow comprehension for the promise monad.

rocqua 6y ago
I have some vague knowledge of category theory, but not enough to follow your argument here. Especially the concept of an 'Arrow' could you explain more clearly what an arrow, and an arrow comprehension is?
- Garlef 6y ago
  https://en.wikipedia.org/wiki/Arrow_(computer_science) https://en.wikipedia.org/wiki/Arrow_(computer_science) https://en.wikipedia.org/wiki/List_comprehension https://en.wikipedia.org/wiki/List_comprehension https://www.haskell.org/arrows/syntax.html https://www.haskell.org/arrows/syntax.html https://www.sciencedirect.com/science/article/pii/S157106611100051X https://www.sciencedirect.com/science/article/pii/S157106611...
sfvisser 6y ago
If you really need to run your computations in a separate context/arrow, making them explicit in the types is nice. Having general syntactic rules to build, compose and run them: great. However, if you can avoid the additional arrow entirely: even better! I mean Haskell of all languages has green threads and doesn’t need the entire promise shenanigans. Even in Haskell I sometimes I build very specific DSLs for my domain to be interpreted in a separate step so I can freely fix my pure and impure code.

bvrmn 6y ago

There is an interesting approach in Zig to explicitly kick call frames to achieve colorless asyncness. https://www.youtube.com/watch?v=zeLToGnjIUM https://www.youtube.com/watch?v=zeLToGnjIUM

k__ 6y ago

I saw TypeScript has now an awaited keyword that will eat promise wrapped and plain values.

elwell 6y ago

Original discussion: https://news.ycombinator.com/item?id=8984648 https://news.ycombinator.com/item?id=8984648

joshribakoff 6y ago

Classic article. In rxjs mapping over a sync array or async stream is exactly the same, for me the problem in the article is not such a big problem in practice for day to day work, especially with tools like rxjs which makes composing mixed code easy

andrewzah 6y ago

This is something I ran into recently when learning about promises and async functions in javsscript. It took me a while to grok it, and I'm still not sure if I fully understand it, but we settled on using try/catch blocks with typescript's await keyword on calls we need to block on. I wanted to use @usefultools/monads to have Maybe types, etc, but it's easier to just stick with promises throughout the chain. Essentially working with functions that return promises (like database calls with Mongoose) makes you convert your functions to also use promises. As far as I can tell there's no way to unwrap a promise (similar to Rust) without the await keyword inside a try block. I don't know if this is exactly the best approach, but it's significantly less verbose than .then().catch() callbacks. I also learned about Promise.all() and Promise.allSettled() recently here [0]. I wish I could stick to futures with Rust or goroutines with golang. Bleh. [0]: https://news.ycombinator.com/item?id=23223881 https://news.ycombinator.com/item?id=23223881

devxpy 6y ago

I've been thinking why exactly async-await was chosen at the function level, and not the caller level. I mean for languages that have event loops at their core, why isn't every function `async` by default? Let the caller decide how it wants to use the function. `async` functions don't wait for a result of a `Future` unless `await` is used. Instead of putting all that effort into putting `await` everywhere, why not invert that logic and make them `await` by default, and instead have the async-await syntax used by callers? Like if one to were to write a transpiler to do this in dart, it might compile the following - void main() { Future<int> xFuture = async doSomething() int x = doSomething() assert(await xFuture == x); } int doSomething { ... } into - Future<void> main() async { Future<int> xFuture = doSomething() int x = await doSomething() assert(await xFuture == x); } Future<int> doSomething async { ... } This might be stupid, but my solution to the "what color is your function problem" is to make every function async. I looks an awful lot like what go does with its `go` syntax, but because you have an event loop with Future and Async Streams, you don't need to worry about dealing with CSP.

chaorace 6y ago
I suspect this is for two reasons: * Many popular languages today predate modern asynchronous computing. This makes async-as-default impossible because it's an afterthought * Async computing is harder to learn and confusing for those just picking up the language. There are some pretty major ergonomics issues that you have to solve if you want anything more than a DSL to achieve noteworthy adoption levels.
- devxpy 6y ago
  > Many popular languages today predate modern asynchronous computing. This makes async-as-default impossible because it's an afterthought Here's a rough sketch of how this can be achieved for a language that already has event loops - Transpile the following - void main() { Future<int> xFuture = async doSomething() int x = doSomething() assert(await xFuture == x); } Into - Future<void> main() async { Future<int> xFuture = (() async => await doSomething())(); int x = await doSomething(); assert(await xFuture == x); } Now people can keep using a pre-existing `doSomething()` (that's either async or sync) and its compatible with our new caller based async statement. BTW that transpiler output is runnable dart code actually works. I know this has tons of edge cases that this simple example doesn't capture, but it's definitely "possible".