6 ms·

Async seems like an underbaked idea across the board. Regular code was already async. When you need to wait for an async operation, the thread sleeps until read

• swiftcoder 6mo ago
  > So threads was the right programming model. For problems that aren't overly concerned with performance/memory, yes. You should probably reach for threads as a default, unless you know a priori that your problem is not in this common bucket. Unfortunately there is quite a lot of bookkeeping overhead in the kernel for threads, and context switches are fairly expensive, so in a number of high performance scenarios we may not be able to afford kernel threading
  • groundzeros2015 6mo ago
    In that sentence I’m referring to the abstract idea of a thread of execution as a model of programming, not OS threads. A green thread implementation could do it too. But what you said about kernel implementation is true. But are we really saying that the primary motivation for async/await is performance? How many programmers would give that answer? How many programs are actually hitting that bottleneck? Doesn’t that buck the trend of every other language development in the past 20 years, emphasizing correctness and expressively over raw performance?
    • swiftcoder 6mo ago
      > But are we really saying that the primary motivation for async/await is performance? The original motivation for not using OS threads was indeed performance. Async/await is mostly syntax sugar to fix some of the ergonomic problems of writing continuation-based code (Rust more or less skipped the intermediate "callback hell" with futures that Javascript/Python et al suffered through).
      • PunchyHamster 6mo ago
        In some languages, yes, in others (js/python) async is just workaround about not having proper threading.
        • swiftcoder 6mo ago
          Python used multiple threads to handle I/O long before async/await was a glimmer in anyone's mind (despite the GIL). nodejs is one of the very few languages I can think of that was born single-threaded and used an asynchronous runtime from the get-go
    • nchie 6mo ago
      > But are we really saying that the primary motivation for async/await is performance? Of course - what else would it be? The whole async trend started because moving away from each http request spawning (or being bound to) an OS thread gave quite extreme improvements in requests/second metrics, didn't it?
      • groundzeros2015 6mo ago
        I agree. Managing many http requests or responses was a motivating problem. What I question is whether 1. Most programs resemble that, so that they make it an invasive feature of every general purpose language. 2. Whether programmers are making a conscious choice because they ruled out the perf overhead of the simpler model we have by default.
        • swiftcoder 6mo ago
          That is why we have the function colouring problem and a split ecosystem in the first place - if it were obviously better in all cases, we'd make async the default, and get rid of the split altogether (and there are languages, like Erlang, that fall on this side of the fence)
      • lukaslalinsky 6mo ago
        It was not for performance reasons, but for scaling up.
        • pjc50 6mo ago
          That's the same thing?
    • sureglymop 6mo ago
      Importantly though, performance might be worse depending on use case and program. Specifically with scheduling in user space it can negatively impact branch prediction as your CPU is already hyper optimized for doing things differently. It's all nuanced and what to choose requires careful evaluation.
• pkolaczk 6mo ago
  Threads are neither better or worse than async+callbacks. They are different. There are problems which map nicely to threads and there are problems which are much nicer to express with async.
  • groundzeros2015 6mo ago
    Such as? The entire premise of async is that callbacks were a mistake because they broke sequential reasoning and control. Every explanation of the feature starts with managing callback hell.
    • codedokode 6mo ago
      The callbacks should be just hidden from programmer, that's what async/await are for.
    • repelsteeltje 6mo ago
      Beware, they are different concepts. Threads offer concurrent execution, async (futures) offer concurrent waiting. Loosely speaking, threads make sense for CPU bound problems, while async makes sense for IO bound problems.
      • groundzeros2015 6mo ago
        Why? You write the same code with async await but with a keyword at the beginning of every function.
        • f_devd 6mo ago
          Because if you go down the callstack eventually you won't get the await keyword anymore; you'll get the actual 'waiters' and 'wakers' which define your scheduling
          • groundzeros2015 6mo ago
            Yeah. The OS handles scheduling and preemption so it’s done for you rather than a call in the stack.
• nananana9 6mo ago
  > the thread sleeps until ready and the kernel abstracts it away. Sure, but once you involve the kernel and OS scheduler things get 3 to 4 orders of magnitude slower than what they should be. The last time I was working on our coroutine/scheduling code creating and joining a thread that exited instantly was ~200us, and creating one of our green threads, scheduling it and waiting for it was ~400ns. You don't need to wait 10 years for someone else to design yet another absurdly complex async framework, you can roll your own green threads/stackful coroutines in any systems language with 20 lines of ASM.
  • groundzeros2015 6mo ago
    1. Why can’t we have better green threads implementations with better scheduling models? 2. Unchecked array operations are a lot faster. Manual memory management is a lot faster. Shared memory is a lot faster. Usually when you see someone reach for sharp and less expressive tools it’s justified by a hot code path. But here we jump immediately to the perf hack? 3. How many simultaneous async operations does your program have?
    • vlovich123 6mo ago
      Well, if you offload heavy compute into an async task, then usually it depends strictly on how many concurrent inputs you are given. But even something as “simple” as a performance editor benefits from this if done well - that’s why JS text editors have reasonably acceptable performance whereas Java IDEs always struggled (historically anyway since even Java has adopted green threads).
      • groundzeros2015 6mo ago
        You think IDEs are written in JS because of the performance benefits of the threading model? I thought it was because they could copy chromium.
        • vlovich123 6mo ago
          Why do you think they don’t struggle with input latency? Because the non blocking nature built into the browser model is so powerful and you cannot get that with threads.
• hacker_homie 6mo ago
  I’m just waiting for them to try co-operative multithreading again.
  • jayd16 6mo ago
    That's what async/await is, no? Yielding by awaiting is co-operative.
• usrnm 6mo ago
  The problem comes from trying to sit on both chairs: we want async but want to be able to opt out. This is what causes most of the ugliness, including function colouring. Just look at golang, where everything is async with no way to change it, it's great. It's, probably, not well-suited for things like microcontrollers, where every byte matters, but if you can afford the overhead, it's so much better than Rust async. Before async Rust was an interesting and reasonable language, now it's just a hot mess that makes your eyes bleed for no reason.
  • vanderZwan 6mo ago
    > It's, probably, not well-suited for things like microcontrollers, where every byte matters, but if you can afford the overhead, it's so much better than Rust async. There is one hill I'll die on, as far as programming languages go, which is that more people should study Céu's structured synchronous concurrency model. It specifically was designed to run on microcontrollers: it compiles down to a finite state machine with very little memory overhead (a few bytes per event). It has some limitations in terms of how its "scheduler" scales when there are many trails activated by the same event, but breaking things up into multiple asynchronous modules would likely alleviate that problem. I'm certain a language that would suppprt the "Globally Asynchronous, Locally Synchronous" (GALS) paradigm could have their cake and eat it too. Meaning something that combines support for a green threading model of choice for async events, with structured local reactivity a la Céu. F'Santanna, the creator of Céu, actually has been chipping away at a new programming language called Atmos that does support the GALS paradigm. However, it's a research language that compiles to Lua 5.4. So it won't really compete with the low-level programming languages there. [0] https://ceu-lang.org/ https://ceu-lang.org/ [1] https://github.com/atmos-lang/atmos https://github.com/atmos-lang/atmos
  • baq 6mo ago
    > not well-suited for things like microcontrollers, where every byte matters except when a RAM fetch is so expensive a load is basically an async call - and it's a single machine code instruction at the same time
  • PunchyHamster 6mo ago
    Everything is not async in Go. If your threads are "free" you can just run 400 copies of a synchronous code and blocking in one just frees the thread to work on other. async within same goroutine is still very much opt in (you have to manually create goroutine that writes to channel that you then receive on), it just isn't needed where "spawn a thread for each connecton" costs you barely few kb per connection.
• vlovich123 6mo ago
  > Regular code was already async. When you need to wait for an async operation, the thread sleeps until ready and the kernel abstracts it away Not really. I’ve observed async code often is written in such a way that it doesn’t maximize how much concurrency can be expressed (eg instead of writing “here’s N I/O operations to do them all concurrently” it’s “for operation X, await process(x)”). However, in a threaded world this concurrency problem gets worse because you have no way to optimize towards such concurrency - threads are inherently and inescapably too heavy weight to express concurrency in an efficient way. This is is not a new lesson - work stealing executors have long been known to offer significantly lower latency with more consistent P99 than traditional threads. This has been known since forever - in the early 00s this is why Apple developed GCD. Threads simply don’t provide any richer information it needs in the scheduler to the kernel about the workload and kernel threads are an insanely heavy mechanism for achieving fine grained concurrency and even worse when this concurrency is I/O or a mixed workload instead of pure compute that’s embarrassingly easily to parallelize. Do all programs need this level of performance? No, probably not. But it is significantly more trivial to achieve a higher performance bar and in practice achieve a latency and throughput level that traditional approaches can’t match with the same level of effort. You can tell async is directionally kind of correct in that io_uring is the kernel’s approach to high performance I/O and it looks nothing like traditional threading and syscalls and completion looks a lot closer to async concurrency (although granted exploiting it fully is much harder in an async world because async/await is an insufficient number of colors to express how async tasks interrelate)
  • groundzeros2015 6mo ago
    I am not saying threads are the model for all programming problems. For example a dependency graph like an excel spreadsheet can be analyzed and parallelized. But as you observed, async/await fails to express concurrency any better. It’s also a thread, it’s just a worse implementation.
    • vlovich123 6mo ago
      That’s incorrect. Even when expressed suboptimally, it still tends to result in overall higher throughput and consistently lower latency (work stealing executors specifically). And when you’re in this world, you can always do an optimization pass to better express the concurrency. If you’ve not written it async to start with, then you’re boned and have no easy escape hatch to optimize with.
• BlackFly 6mo ago
  I think that callbacks are actually easier to reason about: When it comes time to test your concurrent processing, to ensure you handle race conditions properly, that is much easier with callbacks because you can control their scheduling. Since each callback represents a discrete unit, you see which events can be reordered. This enables you to more easily consider all the different orderings. Instead with threads it is easy to just ignore the orderings and not think about this complexity happening in a different thread and when it can influence the current thread. It isn't simpler, it is simplistic. Moreover, you cannot really change the scheduling and test the concurrent scenarios without introducing artificial barriers to stall the threads or stubbing the I/O so you can pass in a mock that you will then instrument with a callback to control the ordering... The problem with callbacks is that the call stack when captured isn't the logical callstack unless you are in one of the few libraries/runtimes that put in the work to make the call stacks make sense. Otherwise you need good error definitions. You can of course mix the paradigms and have the worst of both worlds.
  • groundzeros2015 6mo ago
    I agree. I don’t think callbacks are an underbaked language feature.
    • t-writescode 6mo ago
      In another part of the thread, you lament the use of callbacks. May I ask you what you think async/await is, except syntactic sugar that wraps around the callback pattern?
      • groundzeros2015 6mo ago
        Certain architectures focused around callbacks have problems. But their existence is not a burden on the language design. Node.js has a problem where every standard library function has a callback and blocking version. At least they just committed to doing both.
• codedokode 6mo ago
  As I understand, "green threads" are also expensive, for example you either need to allocate a large stack for each "thread", or hook stack allocation to grow the stack dynamically (like Go does), and if you grow the stack, you might have to move it and cannot have pointers to stack objects.
  • PunchyHamster 6mo ago
    works fine in Go. Yes, you're not getting Rust performance (tho good part of it is their own compiler vs using all LLVM goodness) but performance is good enough and benefits for developers are great, having goroutines be so cheap means you don't even need to do anything explicitly async to get what you want
    • aw1621107 6mo ago
      Rust chose a different design space for their async implementation though, so what works well for Go wouldn't work well for Rust. In particular, the Rust devs wanted zero-cost FFI that external code doesn't need to know about, which precludes Go-like green threads.
    • surajrmal 6mo ago
      Rust can be used in contexts like dynamic linkers, kernels, libc, microcontrollers, dynamic libraries, and all sorts of places go has no business running. And it can use async in many of them. Go works fine for many contexts but we already have languages like go that work for those contexts. Rust is for the contexts it doesn't work well for. It's painful that it keeps being pushed to support things that would make it more difficult to support the areas it is unique in supporting.
  • kgeist 6mo ago
    >and if you grow the stack, you might have to move it Most stacks are tiny and have bounded growth. Really large stacks usually happen with deep recursion, but it's not a very common pattern in non-functional languages (and functional languages have tail call optimization). OS threads allocate megabytes upfront to accommodate the worst case, which is not that common. And a tiny stack is very fast to copy. The larger the stack becomes, the less likely it is to grow further. >cannot have pointers to stack objects In Go, pointers that escape from a function force heap allocation, because it's unsafe to refer to the contents of a destroyed stack frame later on in principle. And if we only have pointers that never escape, it's relatively trivial to relocate such pointers during stack copying: just detect that a pointer is within the address range of the stack being relocated and recalculate it based on the new stack's base address.
• pjmlp 6mo ago
  Proper modern languages offer both, you can keep your threads and reach out to async only when it makes sense to do. Now the languages that don't offer choice is another matter.
• K0nserv 6mo ago
  I think you are correct, in so far that often N:M threading is overkill for the problem at hand. However, some IO bound problems truly do require it. I haven't kept up with the details, but AFAIK the fallout from Spectre and Meltdown also means context switches are more expensive than they were historically, which is another downside with regular threads. I also want to address something that I've seen in several sub-threads here: Rust's specific async implementation. The key limitation, compared to the likes of Go and JS, is that Rust attempts to implement async as a zero-cost abstraction, which is a much harder problem than what Go and JS does. Saying some variant of "Rust should just do the same thing as Go", is missing the point.
  • groundzeros2015 6mo ago
    I think rust didn’t need async at all.
    • aw1621107 6mo ago
      The question then becomes what, if anything, should take its place, and what are the corresponding tradeoffs?
• the__alchemist 6mo ago
  What is kernel in this context?
• dgellow 6mo ago
  You don’t have threads on embedded, but you want a way to express concurrent waiting. Different problems altogether
  • girvo 6mo ago
    You can, though. We used pthreads (well, pthresd compatible API) in production at massive scale on the ESP32-S3.
• otabdeveloper4 6mo ago
  > Now language runtimes prefer “green threads” for portability and performance "Green threads" only exist in crappy interpreted languages, and only because they have stop-the-world single-threaded garbage collection.
  • atombender 6mo ago
    Go and Java both have green threads, and are not interpreted nor limited to single threaded GC.
• Salgat 6mo ago
  Awaiting allows you to efficiently yield the thread to other tasks instead of blocking it. That's one of its biggest advantages.
  • groundzeros2015 6mo ago
    When you block the OS does the same thing - yields to other threads.
    • Salgat 6mo ago
      And how much slower is that? What happens when I run a thousand async tasks? I'll give you a hint, with async/await, it has barely any overhead.
      • bigstrat2003 6mo ago
        The vast, vast majority of programmers are going to be writing software where there are only a handful of threads (if that). The "I need thousands of concurrent executions" case is simply not relevant to most people.
        • Salgat 6mo ago
          You do realize what servers do in parallel right? Async/await allows ASP.NET to scale beyond 1 thread per request.
          • groundzeros2015 6mo ago
            Are you going to put multiple customer’s data in the same OS process? Did you know you can get even more performance if you manually manage memory and don’t use virtual functions?
    • jandrewrogers 6mo ago
      Yes, and it is extremely expensive. This is a well-known design problem in database engines. The computational cost of context-switching threads at yield points is often many times higher than the actual workload executed between yield points. To address this you either need fewer yield points, which reduces concurrency, or you need to greatly reduce the cost of yielding. An async architecture reduces the cost of yielding by multiple orders of magnitude relative to threads.
• jandrewrogers 6mo ago
  > So threads was the right programming model. It depends on what you are doing. Threads are the right model for compute-bound workloads. Async is the right model for bandwidth-bound workloads. Optimization of bandwidth-bound code is an exercise in schedule design. In a classic multithreading model you have limited control over scheduling. In an async model you can have almost perfect control over scheduling. A well-optimized async schedule is much faster than the equivalent multithreaded architecture for the same bandwidth-bound workload. It isn't even close. Most high-performance code today is bandwidth-bound. Async exists to make optimization of these workloads easier.
  • groundzeros2015 6mo ago
    If this is a classic exercise can you show me the material? Why can’t a scheduler be written which optimizes around IO? What additional information is present in code that has async/await annotations?
    • tombert 6mo ago
      I believe that's actually how the virtual threads in the newer Java works. It's smart enough to notice IO and properly park it and move to another thread. I think it's still basically doing epoll behind the scenes [1], but you have straightforward sequential code in the process and the actual implementation is invisible to the user, and you can use old boring blocking code with an object that is a drop-in replacement for Thread. I personally still kind of prefer the explicit async stuff with Futures and Vert.x since I kind of like the idea that async is encoded into the type itself so you're more directly aware of it, but I'm definitely an outlier for that. [1] Genuinely, please correct me if I'm wrong, it's very possible that I am.
      • p2detar 6mo ago
        > but I'm definitely an outlier for that You are not. I prefer the same and that's how my product works right now. My HTTP API is Vert.x-only with futures. My particular use case is thousands of devices sending small packages to the API in undefined periods of time or in bursts, so I find Vert.x event-loop performance quite a good match for my use case. In fact it has been very positive given customer feedback thusfar. Background tasks in my app are processed in a different module, which uses plain old ScheduledExecutorService-based thread pool to poll. The tasks are visible in the UI as well. I still haven't switched to VTs, because I don't know what load-implications that may have on the database pool. The JEP writes `Do not pool virtual threads` [0]. I assume if a db connection is not available in the pool, the VT will get parked, but I feel this isn't quite what a background scheduler should look like, e.g., hundreds of "in-process" tasks blocked while waiting for db connection to free up. Testing is on my todo list for some time now. The JEP doesn't mention epoll, but there is a write up about that on github: `On Linux the poller uses epoll, and on Windows wepoll (which provides an epoll-like API on the Ancillary Function Driver for Winsock)` [1] 0 - https://openjdk.org/jeps/444#Do-not-pool-virtual-threads https://openjdk.org/jeps/444#Do-not-pool-virtual-threads 1 - https://gist.github.com/ChrisHegarty/0689ae92a01b4311bc8939f33fde9fd9 https://gist.github.com/ChrisHegarty/0689ae92a01b4311bc8939f...
• jayd16 6mo ago
  That immediately falls apart if you want to attempt the extremely common pattern of wait free usage of a main/UI thread.