4 ms·
It seems to me that async io struggles whenever people try it. For instance it is where Rust goes to die because it subverts the stack-based paradigm behind ow
by PaulHoule 1y ago
It seems to me that async io struggles whenever people try it.
For instance it is where Rust goes to die because it subverts the stack-based paradigm behind ownership. I used to find it was fun to write little applications like web servers in aio Python, particularly if message queues and websockets were involved, but for ordinary work you're better off using gunicorn. The trouble is that conventional async i/o solutions are all single threaded and in an age where it's common to have a 16 core machine on your desktop it makes no sense. It would be like starting a chess game dumping out all your pieces except for your King.
Unfashionable languages like Java and .NET that have quality multithreaded runtimes are the way to go because they provide a single paradigm to manage both concurrency and parallelism.
- erichocean 1y agoProject Loom makes Java in particular really nice, virtual threads can "block" without blocking the underlying OS thread. No callbacks at all, and you can even use Structured Concurrency to implement all sorts of Go- and Erlang-like patterns. (I use it from Clojure, where it pairs great with the "thread" version of core.async (i.e. Go-style) channels.)
- vlovich123 1y ago> Unfashionable languages like Java and .NET that have quality multithreaded runtimes are the way to go because they provide a single paradigm to manage both concurrency and parallelism. At the cost of not being able to actually provide the same throughput, latency, or memory usage that lower level languages that don't enforce the same performance pessimizing abstractions on everything can. Engineering is about tradeoffs but pretending like Java or .NET have solved this is naiive.
- pron 1y ago> At the cost of not being able to actually provide the same throughput, latency, or memory usage Only memory usage is true with regards to Java in this context (.NET actually doesn't offer a shared thread abstraction; it's Java and Go that do), and even that is often misunderstood. Low-level languages are optimised for minimal memory usage, which is very important on RAM-constrained devices, but is could be wasting CPU on most machines: https://youtu.be/mLNFVNXbw7I https://youtu.be/mLNFVNXbw7I This optimisation for memory footprint also makes it harder for low-level languages to implement user-mode threading as efficiently as high-level languages. Another matter is that there are two different use cases for asynchronous constructs that may tempt implementors to address them with a single implementation. One is the generator use case. What makes it special is that there are exactly two communicating parties, and both of their state may fit in the CPU cache. The other use case is general concurrency, primarily for IO. In that situation, a scheduler juggles a large number of user-mode threads, and because of that, there is likely a cache miss on every context switch, no matter how efficient it is. However, in the second case, almost all of the performance is due to Little's law rather than context switch time (see my explanation here: https://inside.java/2020/08/07/loom-performance/ https://inside.java/2020/08/07/loom-performance/). That means that a "stackful" implementation of user-mode threads can have no significant performance penalty for the second use case (which, BTW, I think has much more value than the first), even though a more performant implementation is possible for the first use case. In Java we decided to tackle the second use case with virtual threads, and so far we've not offered something for the first (for which the demand is significantly lower). What happens in languages that choose to tackle both use cases with the same construct is that in the second and more important use case they gain no more than negligible performance (at best), but they're paying for that with a substantial degradation in user experience.
- vlovich123 1y agoIt sounds like you’re disagreeing yet no case is made that throughout and latency isn’t worse. For example, the best frameworks on TechEmpower are all Rust, C and C++ with the best Java coming in at 25% slower on that microbenchmark. My point stands - it is generally true that well written rust/c/c++ outperforms well written Java and .Net and not just with lower memory usage. The “engineering effort per performance” maybe skews to Java but that’s different than absolute performance. With rust to me it’s also less clear if that is actually even true. [1] https://www.techempower.com/benchmarks/#section=data-r23 https://www.techempower.com/benchmarks/#section=data-r23
- vips7L 1y agoI honestly doubt any of the frameworks in that benchmark are using virtual threads yet. The top one is still using vert.x which is an event loop on native platform threads.
- CJefferson 1y agoHonestly, if these languages are only winning by 25% in microbenchmarks, where I’d expect the difference to be biggest, that’s a strong boost for Java for me. I didn’t realise it was so close, and I hate async programming so I’m definitely not doing it for an, at most, 25% boost.
- vlovich123 1y agoI didn’t make the claim that it’s worth it. But when it is absolutely needed Java has no solution. And remember, we’re talking about a very niche and specific I/O microbenchmark. Start looking at things like SIMD (currently - I know Java is working on it) or in general more compute bound and the gap will widen. Java still doesn’t yet have the tools to write really high performance code.
- pjmlp 1y agoIt is called JNI, or Panama nowadays. Too many people go hard on must be 100% pure, meanwhile Python is taking over the AI world, via native library bindings.
- pjmlp 1y agoWhat matters is if it is good enough for project acceptance criteria.
- brandonbloom 1y agoIf you watched the video closely, you'll have noticed that this design parameterizes the code by an `io` interface, which enables pluggable implementations. Correctly written code in this style can work transparently with evented or threaded runtimes.
- PaulHoule 1y agoReally? Ordinary synchronous code calls an I/O routine which returns. Asynchronous code calls an I/O routine and then gets called back. That’s a fundamental difference and you can only square it by making the synchronous code look like asynchronous code (callback gets called right away) or asynchronous code look like synchronous code (something like async python which breaks up a subroutine into multiple subroutines and has an event loop manage who calls who.
- int_19h 1y agoOP didn't say that code looks like ordinary sync code, only that it's possible to write code that works equally well for both sync and async. If you RTFA, it looks like this: var a = io.async(doWork, .{ io, "hard" }); ... a.await(io); If your `io` is async, this behaves like an async call returning a promise and then awaiting said promise. If `io` is sync then `io.async` will make the call immediately and `await` is a no-op.
- andrewmcwatters 1y agoI haven’t actually seen it in the wild yet, just talked about in technical talks from engineers at different studios, but I’m interested in designs in which there isn’t a traditional main thread anymore. Instead, everything is a job, and even what is considered the main thread is no longer an orchestration thread, but just another worker after some nominal set up between scaffolding enough threads, usually minus one, to all serve as lockless, work-stealing worker threads. Conventional async programming relies too heavily on a critical main thread. I think it’s been so successful though, that unfortunately we’ll be stuck with it for much longer than some of us will want. It reminds me of how many years of inefficient programming we have been stuck with because cache-unfriendly traditional object-oriented programming was so successful.
- tele_ski 1y agoThis sounds like running an event loop per thread instead of 1 event loop with a backing thread pool. Or am I misunderstanding you? It works great for small tasks but larger tasks block local events and you can get weird latency issues, that was the major tradeoff I ran into when I used it. Works great if your tasks are tiny though, not having the event loop handoff to the worker thread is a good throughput boost. But then we started having latency issues and we introduced larger tasks which would hang the local event loop from getting those events. I think Scylladb works somewhat like this but does message passing to put certain data on certain threads so any thread can handle incoming events but it still moves the request to the pinned thread the data lives on. One thread can get overwhelmed if your data isn't well distributed.
- hinkley 1y agoI know that it depends on how much you disentangle your network code from your business logic. The question is the degree. Is it enough, or does it just dull the pain? If you give your business logic the complete message or send it a stream, then the flow of ownership stays much cleaner. And the unit tests stay substantially easier to write and more importantly, to maintain. I know too many devs who don't see when they bias their decisions to avoid making changes that will lead to conflict with bad unit tests and declare that our testing strategy is Just Fine. It's easier to show than to debate, but it still takes an open mind to accept the demonstration.
- pron 1y ago> Unfashionable languages like Java and .NET that have quality multithreaded runtimes are the way to go because they provide a single paradigm to manage both concurrency and parallelism. First, that would be Java and Go, not Java and .NET, as .NET offers a separate construct (async/await) for high-throughput concurrency. Second, while "unfashionable" in some sense, I guess, it's no wonder that Java is many times popular than any "fashionable" language. Also, if "fashionable" means "much discussed on HN", then that has historically been a terrible predictor of language success. There's almost an inverse correlation between how much a language is discussed on HN and its long-term success, and that's not surprising, as it's the less commonplace things that are more interesting to talk about. HN is more Vogue magazine than the New York Times.
- vanviegen 1y agoHacker News has mostly discussed JavaScript and TypeScript over the past 15 years. These languages do seem to have some long-term succes.
- funflame 1y agoJavaScript has success because it has a monopoly in the browser, anything you want to do there has to go through JavaScript, not because of any merit of the language.
- pron 1y agoI don't think so. Things built in those languages may have been discussed on HN, but the amount of discussion about those languages has not been proportional at all to their popularity.
- CapsAdmin 1y agoIn this context I interpret unfashionable as boring/normal/works/good enough/predictable etc.
- pjmlp 1y agoKind of, in .NET you also have structured concurrency and dataflow, which are another way without having to explicitly write async/await. Yes, sadly Java and .NET are unfashionable in circles like HN, and recent SaaS startups, I keep seeing products that only offer nodejs based SDKs, when they offer Java/.NET SDKs they are generaly always outdated verus the nodejs one.
- josephg 1y ago> It seems to me that async io struggles whenever people try it. Promises work great in javascript, either in the browser or in node/bun. They're easy to use, and easy to reason about (once you understand them). And the language has plenty of features for using them in lots of ways - for example, Promise.all(), "for await" loops, async generators and so on. I love this stuff. Its fast, simple to use and easy to reason about (once you understand it). Personally I've always thought the "function coloring problem" was overstated. I'm happy to have some codepaths which are async and some which aren't. Mixing sync and async code willy nilly is a code smell. Personally I'd be happy to see more explicit effects (function colors) in my languages. For example, I'd like to be able to mark which functions can't panic. Or effects for non-divergence, or capability safety, and so on.
- nine_k 1y agoPromises in JS are particularly easy because JS is single-threaded. You can be certain that your execution flow won't be preepted at an arbitrary point. This greatly reduces the need for locks, atomics, etc.
- o11c 1y agoAlso task-local variables, which almost all systems other than C-level threads basically give up on despite being widely demanded.
- int_19h 1y ago.NET has had task-local vars for about a decade now: https://learn.microsoft.com/en-us/dotnet/api/system.threading.asynclocal-1 https://learn.microsoft.com/en-us/dotnet/api/system.threadin... Python added them in 3.7: https://docs.python.org/3/library/contextvars.html https://docs.python.org/3/library/contextvars.html
- o11c 1y agoI'll admit to unfamiliarity with the .NET version, but for Python even `threading.local` is a useless implementation if you care at all about performance. Performant thread-local variables require ahead-of-time mapping to a 1-or-2-level integer sequence with a register to quickly the base array, and some kind of trap to handle the "not allocated" case. Task-local variables are worse than thread-locals since they are swapped out much more frequently. This requires special compiler support, not being a mere library.
- seabrookmx 1y ago> but for ordinary work you're better off using gunicorn I'd like to see some evidence for this. Other than simplicity, IMO there's very little reason to use synchronous Python for a web server these days. Streaming files, websockets, etc. are all areas where asyncio is almost a necessity (in the past you might have used twisted), to say nothing of the performance advantage for typical CRUD workloads. The developer ergonomics are also much better if you have to talk to multiple downstream services or perform actions outside of the request context. Needing to manage a thread pool for this or defer to a system like Celery is a ton more code (and infrastructure, typically). > async i/o solutions are all single threaded And your typical gunicorn web server is single threaded as well. Yes you can spin up more workers (processes), but you can also do that with an asgi server and get significantly higher performance per process / for the same memory footprint. You can even use uvicorn as a gunicorn worker type and continue to use it as your process supervisor, though if you're using something like Kubernetes that's not really necessary.
- no_flaks_given 1y agoNot many use cases actually need websockets. We're still building new shit in sync python and avoiding the complexity of all the other bullshit
- ddorian43 1y agoMaybe he meant gevent? Which is better than async io in python.
- seabrookmx 1y agoAgree to disagree. Monkey patching the stdlib is a terrible hack and having to debug non-trivial gevent apps is a nightmare (not that asyncio apps are perfect either).