7 ms·

I'll go against the grain here and say that async/await, wether implemented by one thread-per-core like here or by stackless coroutines is not the solution. As

• unrealhoang 6y ago
  Is there a mechanism to group some goroutines onto a single CPU thread to avoid locking? In the environments that have to rely on thread-per-core (sensitive to even 50us), blocking functions will become obvious in the profiling report.
  • BenoitP 6y ago
    > Is there a mechanism to group some goroutines onto a single CPU thread to avoid locking? Python's GIL comes to mind :p
• penberg 6y ago
  So thread-per-core is not just about eliminating context switches. It's also about partitioning application-level data to reduce inter-core synchronization to let speculative, out-of-order cores run independently as much as possible. Don't get me wrong, user-level threads are great, and arguably much simpler programming model than async/await. But they're not _the solution_ either.
  • BenoitP 6y ago
    They're not the solution to well-sliced data processing, but I can't see how they would hurt the performance.
    • penberg 6y ago
      User-level threads do not solve the problem of synchronization and data movement. That is, with a “thread-per-core” model, you eliminate most needs to synchronize between multiple CPU cores, and, therefore, eliminate the overhead of acquiring and releasing a lock and allow the out-of-order CPU cores run at full speed. Furthermore, “thread-per-core” model ensures that memory accesses are always CPU local, which eliminates the need to move data between CPU caches, which eliminates the expensive CPU cache consistency protocol (that makes scaling to multiple cores more difficult). That said, I am not claiming thread-per-core is _the solution_ either, just saying that if you can partition data at application-level, you can make things run plenty fast. Of course, you’re also exposing yourself to other issues, like “hot shards” where some CPUs get disproportionately more work than others. However, as we scale to more and more cores, it seems inevitable that we must partition our systems at some level.
      • imtringued 6y ago
        It's pretty much common knowledge that if you want linear performance scaling on a multiprocessor then each workload has to be effectively independent from the others. Synchronization and communication is expensive and only takes away from your per core performance but it never makes a given core faster. So yes, sharding is a critical part of this "thread per core" architecture.
        • gpderetta 6y ago
          To be fair, concurrent writes do not scale, but single writers multiple readers work just fine, so you only need to an hard partition of writers and can allow all threads (or the appropriate NUMA subset) to access any data.
      • gpderetta 6y ago
• CJefferson 6y ago
  I agree -- I was just around in the last days of Windows 3.1 and co-operative multitasking (which is close to async with a different coat), and it was a horrible experience, and fighting hangs was a never-ending task. I would go further, and say assuming your program is doing "interesting work" (not just calling a database / reading a file and returning the results), use a threadpool. In almost all applications threads are fine, certainly when you are comparing threaded Rust to people using Python/Ruby.
• est31 6y ago
  When I read "Colored function problem" and "make complexity explode" I thought it was some weird NP-complete scheduling issue having to do with graph coloring or something, but it turns out it's just a fancy term for not wanting to add async to everything that calls async. Basically just an ergonomics issue. First, in Rust this isn't really a problem. You can always turn async calls into blocking ones in Rust by calling block_on [0]. In some languages block_on doesn't exist, like in in-browser js, because here, code is supposed to be async. But in Rust there is no requirement, so there's no colored function problem here. Second, I don't think it's a big problem in the first place. In one of my projects, I'm using an async library and have isolated the async-ness by creating a dedicated thread that communicates with the library. The thread provides a queue of messages that the remaining code of my project can handle. [0]: https://docs.rs/tokio/0.3.3/tokio/runtime/struct.Runtime.html#method.block_on https://docs.rs/tokio/0.3.3/tokio/runtime/struct.Runtime.htm...
  • kitten_mittens_ 6y ago
    In browser JS can drop down to the old Promise callbacks (.then) to avoid coloring synchronous callers.
    • brundolf 6y ago
      If any part of your function depends on the result of a promise then it has to return a promise; this is true whether you use async/await syntax or then(). Unfortunately JS doesn't have something like block_on, which now that I think about it is probably because it's single-threaded, so any block would block the entire app.
  • BenoitP 6y ago
    > Basically just an ergonomics issue. A viral one, and number-of-possible-states-exploding one at that. > First, in Rust this isn't really a problem. You can always turn async calls into blocking ones in Rust by calling block_on That's not exactly what we're trying to do here.
  • glaubercosta 6y ago
    I must admit I had no idea what that was either and the first thing that came to my mind was some combinatorial explosion thing. I truly don't care about this issue...
• Joker_vD 6y ago
  There is an easy solution to the colour problem: make all functions async. All of them. Of course, having code like this async func foo(x int) task[int] { return await(2 * x); } // No "async" only because cpu_bound_future_factory() returns a task func bar(y int) task[int] { return cpu_bound_future_factory(async func() task[int] { return await(y - 5); }); } async func baz(a,b int) task[int] { return await(await foo(a) + await bar(b)); } is not fun, especially when you have to await the result of the arithmetic operators, and write all those now-redundant async and await, so let's also drop "async" and make "await" implicit--although we'll need something for non-awaiting. Let's call it "nowait". Now we can write code like this: func foo(x int) task[int] { return 2 * x; } func bar(y int) task[int] { return nowait cpu_bound_future_factory(func() task[int] { return y - 5; }); } func baz(a, b int) task[int] { return foo(a) + bar(b); } A-a-and we're back to multithreading, basically. So yeah, looks like cheap context switches is the solution.
  • glaubercosta 6y ago
    Maybe I am part of the problem by emphasizing context switching too much, but as penberg noticed, there is more to thread per core than context switching.
  • gpderetta 6y ago
    Agree :) . Semantically there is no difference, but in practice async functions are compiled differently than normal functions. what you are proposing in practice is to do a global CPS transform and possibly reconstruct back the original function when the continuation is not needed (i.e. it is always invoked in strict LIFO order). There are functional languages where this is the norm, but I don't think this is appropriate for a system language like rust (it would greatly complicate interop with C for example). My idea is to make the continuation explicit and CPS transform all and only the functions that have a continuation as parameter (and any generic function with a type that is a continuation or contains a continuation). Fall back to dynamic stacks if the continuation escapes. It is always continuations all the way down.
    • Joker_vD 6y ago
• glaubercosta 6y ago
  it would be good if Rust would have as many programming models as possible. Different applications are likely to benefit from different approaches so having a wide array of choice is good.
• bxk1 6y ago
  No, the point of async "coloring" is to indicate which function call does context switching, so that you would know in which piece of code you could do synchronization without something like a semaphore, as you'd need a semaphore if you synchronize shared memory access across async calls, but you don't if you do it in between async calls. And also to know which piece of code blocks event loop (i.e. everything in between async calls), so that you could optimize for performance: throughput, latency. Assuming, of course, that you run one event loop per thread, not doing dumb things like some Rust executors scheduling tasks to different cores. Goroutine (loom) style concurrency only makes things worse both ergonomically and performance-wise, pushing programmers towards slow buggy lock-riddled code as the only way to use such models.
• nemothekid 6y ago
  >I sincerely believe this is Rust's ballpark; hell they even started the project with that idea in mind. The problem with custom stacks is you essentially need to have a custom runtime and it makes calling into C functions a lot more difficult (cgo isn’t a cakewalk) This runs counter to Rust’s C++ inherited belief that you don’t pay for what you don’t use and would have have made Rust less feasible for all kinds of other projects. async/await might not have the best developer ergonomics, but it does have the best implementation ergonomics from a language point of view
• Diggsey 6y ago
  The "Colored function problem" is a complete fallacy: aside from the fact that async and non-async fns are interoperable in Rust (block_on) and other languages, the same argument could be made against functions with any preconditions. > The solution to expensive context switches is cheap context switches, plain and simple. Except the performance difference between a kernel-mode context switch and a user-mode one is only going to narrow in the future. The overhead that cannot be eliminated from context switches is their effect on the cache, since you start to run into the laws of physics at that point... The real solution to expensive context switches is to just do fewer of them... No context switch is always faster than a "fast" context switch. > I sincerely believe this is Rust's ballpark I think it's plausible that Rust could get a library-level solution for fibers that does not rely on unstable details of the compiler. Rust will never again have that baked in to the language, as it would make the language completely unsuitable for many low-level tasks. Fibers, especially the way they are implemented in Go, come with a lot of their own complexity. Just look at this issue: https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/ https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/ This is just one of the segfaults caused by Go's complex stack control. I don't want to rely on a runtime that contains these sorts of bugs, and the best way to avoid that is to avoid having a runtime in the first place.
  • Joker_vD 6y ago
    I don't know too much about Rust, but in .NET, blocking on Task.Result is considered an anti-pattern and a Bad Thing To Do. Not in the least because it very easily leads to deadlocks.
    • davidhyde 6y ago
      This is considered bad in .NET because callbacks can "Post" back to the original SynchronizationContext which depends on what async/await is being used for. For example, a call to await on a WPF UI thread will join to the calling thread so if you call Task.Result without configuring the task to not join back to the calling thread then you can deadlock the callback processing queue. To avoid this you would use ConfigureAwait(false) depending on your situation. It's the source of a lot of confusion in .NET. I don't believe that Rust has this "feature" and if you somehow wrote code to achieve the same thing as .NET does then it probably wouldn't compile in Rust due to the ownership rules. Source: https://devblogs.microsoft.com/dotnet/configureawait-faq/ https://devblogs.microsoft.com/dotnet/configureawait-faq/
• devit 6y ago
  That doesn't work because async functions need to be compiled for minimal stack usage (i.e. segmented stacks, packed variables), while non-async functions need to be compiled for minimal CPU usage (i.e. no stack checking or allocation, no variable packing, only one stack adjustment per function). If you compile both for the same goal the system will have suboptimal memory usage or performance, and in general you don't know whether a function will be interrupted in advance if you don't have colored functions (also you can't move the stack if you have references unless you have a GC, which is terrible).
• yxhuvud 6y ago
  Nothing stops a model with lightweight threads from limiting a thread from having a separate pool of fibers running on it. It would probably require a construct like a nursery configured to run the fibers in a dedicated thread.
• bfrydl 6y ago
  I'm quite tired of “What Color is Your Function”. In the five years since it was written I have yet to care about this supposed explosive problem in any language that uses async/await. It just seems like something that gets trotted out whenever anyone dares to suggest there are benefits to the approach or to talk about Rust. I'm not sure why I am supposed to accept it as meaningful truth.
• jzoch 6y ago
  You turn a real engineering issue into a programming language dogma war. Really? Dont use async-await then. Its just syntactic sugar (minus a few details that make it so ergonomic that you can do things that otherwise would be more work than they are worth). Just use plain futures. Be joyful as there will be no "colored" functions in your editor - only 10x more code (and less readable!). I think there is a place for arguing about async await syntax and whether fibers are the better solution (try implementing them without a large runtime! Rust got rid of theirs.). Clearly, the benefits provided by continuations are worthwhile in avoiding context switches - to argue otherwise is wild. Context switches are very expensive and, in performance critical code, a huge red flag.