12 ms·
I'm not feeling the async pressure
- davidjnelson 7y agoDoesn’t solve the issue with calling an api that doesn’t return a promise, but the tslint rule “no-floating-promises” flags missing awaits which is quite useful. Also useful is flagging awaits that don’t return a promise, which can be done with the tslint rule “await-promise”.
- fyp 7y agoI really wish we had more control over the scheduling of async tasks. For a javascript example I ran into recently, say I am firing off a fetch for each image that comes into view in a large gallery. If I suddenly scroll down to the 1000th image, a naive implementation might fire off 1000 fetches for all the images we scrolled past. Then you'll be waiting a long time before the images in your current viewport is loaded. Backpressure can save you a little bit here. Say you do the semaphore trick mentioned in the article and only allow a max of say 10 fetches in flight at once. Then if you quickly scroll through, all the subsequent fetches after the initial should fail, including the ones at the viewport you stop at. But since the queue is short, when the images in your current viewport retries it should now succeed. This works but it isn't ideal. Ideally I would be able to just reprioritize the newer fetches to be LIFO instead of FIFO. Or maybe inspect what's currently queued up (and how big the queue is) so I can cancel everything that I don't need. The backpressure solutions might just be a symptom of async tasks not being controllable in any way once started which is why you're forced to commit to it or not from the start even if that might not be the best point in time to make that decision.
- crooked-v 7y agoThere's a (overcomplicated) way to cancel pending fetches now: https://developer.mozilla.org/en-US/docs/Web/API/AbortController/abort https://developer.mozilla.org/en-US/docs/Web/API/AbortContro...
- girvo 7y agoSo I entirely agree with you. Having some sort of control over the scheduling and execution of async tasks is quite important. And every time I try to implement it in, say, Typescript, I realise that I'm reimplementing Erlang/BEAM. I truly believe BEAM's runtime has a million good ideas in this space that we should be stealing!
- dnautics 7y agoor seriously, just use the BEAM. BEAM is coming to the javascript runtime (via WASM)
- winrid 7y agoInstead of starting the fetch when the image comes into view - check every 250ms or so what images are in the viewport.
- zamadatix 7y agoIf your answer to a JavaScript control flow problem is "fix it with a timer" it's almost never what you want to do. Checking every 250ms just makes things feel delayed for users on fast systems (particularly after a large scroll) and slow for users where 10 images won't load in 250ms while also wasting resources when there is nothing to do. https://news.ycombinator.com/item?id=21932726 https://news.ycombinator.com/item?id=21932726 said exactly what I was typing out on how to handle this general situation without the upcoming API for the exact use case mentioned in this comment https://news.ycombinator.com/item?id=21932328 https://news.ycombinator.com/item?id=21932328
- winrid 7y agoYeah, no. I would use a queue. Capture the picture-enter-viewport event and then 50ms later (sorry I said 250... probably too long) verify it's still in the viewport like in the case of fast scrolling. Denounce so you don't fire timers for events close together. Works cross browsers, doesn't waste resources, will be smooth.
- Noumenon72 7y agoDid you mean "debounce" instead of "denounce"? Otherwise Google has failed me.
- winrid 7y agoYeah sorry :)
- zamadatix 7y agoThat's not a queue it's just a delayed async call (i.e. if 5,000 calls ask to be called they will all go without queuing in 50ms), what the other comment described is a queue. 50ms is way too short, a user on a 1080p screen would need to scroll at 360 pixels per second to avoid loading every image along the way. Even trying to do that on purpose is hard as hell https://clickspeedtest.com/scroll-test.html https://clickspeedtest.com/scroll-test.html. Guaranteed that every mobile user is going to be loading every image on scroll over 4g as well creating the original back pressure issue now with 3 more frames of lag. "smooth" does not mean "check every 3 frames" it means schedule things in a way they can happen immediately when the browser is ready, not your timer.
- DylanDmitri 7y agoI've found a hybrid model to be generically useful. Keep the first ~10 calls in a FIFO queue, and if that queue is full put additional calls into an LIFO stack that functions as overflow storage. You can even reuse the same data structure and just flip the behavior under load, what's important is having FIFO behavior normally and then LIFO behavior when congestion occurs.
- jorblumesea 7y agoDon't load balancers have some concept of adaptive load balancing? Maybe this is a good idea to port over to the async world.
- z3t4 7y agoJust load all images. No JavaScript! Let the browser handle it. I hate when web sites only display 1-3 pages worth of content and unload/load more when I scroll. The JS code for that uses more resources then pulling everything would. It's a user nightmare, where I cannot use "find" and I can't zoom out, or scroll fast. The browser can easily handle 10,000 DOM elements. There is already optimizations in place in browsers which solve render issues. Beating the browser at it's own job will require a tremendously amount of work. These problem was already solved when computers only had 256MB of memory. And your web site would be blazingly fast today if you did not complicate things.
- fyp 7y agoNative lazy loading is an extremely recent feature! https://web.dev/native-lazy-loading/ https://web.dev/native-lazy-loading/ https://caniuse.com/#feat=loading-lazy-attr https://caniuse.com/#feat=loading-lazy-attr So no, browsers don't have these problems solved out of the box. And it seems like what they are implementing will suffer from the exact problem I described here too. I think you might also be thinking of bad implementations of infinite scroll which isn't what I am talking about. Have you never use something like photos.google.com? Scrubbing/scrolling to an arbitrary point in time is honestly a great user experience. Would you rather wait hours/days for a webpage to load all the past photos images you've ever taken? Or have to deal with pagination and click through hundreds of pages to find what you want?
- dr-detroit 7y agoIsn't infinite scroll lazy loading ubiquitous in 2020? Without searching Im going to guess theres 500000 JS libraries to do this work and at least 14 good ones.
- grok2 7y agoHow do actor model languages (like ponylang) handle this? It seems like not having back-pressure support would be a fundamental issue with the language.
- ramchip 7y agoThere’s a detailed article on the topic at: https://ferd.ca/handling-overload.html https://ferd.ca/handling-overload.html
- grok2 7y agoIt seems like the gist is that the responsibility is on the application to be architect-ed (perhaps using one of the various libraries/strategies) to handle flow-control/back-pressure. The language itself doesn't directly help.
- ncmncm 7y agoI wonder if the advent of async/await in all the popular and upcoming programming languages will be seen, after some (buffering) interval, as a disaster of major proportions. And, I wonder how the programming world will respond to the disaster. It seems advisable to begin that response now. The linked article might be the beginning of such a response, but it seems too tentative. We may need an Iron Law of Flow Control, visibly acknowledged and observed in each system that uses an async/await facility, at the point of use, or a note explaining where it is handled farther back up the chain. TCP vs IP is an excellent example of such an alternative: IP does not bother with buffering, except as a completely local performance optimization, and happily drops packets at the first hint of trouble, assured that somebody closer to the source has buffered copies of whatever they actually care about.
- sk5t 7y ago> IP does not bother with buffering Did you mean UDP? But even if so, this description overall seems way off in the weeds.
- detaro 7y agoTCP is not based on UDP, so that would not make sense at all. The parent describes it quite accurately: IP makes no guarantee on delivery and allows dropping packets, TCP promises to fix that on a higher level for applications that need it.
- sk5t 7y agoNobody's claiming TCP/IP to be based on UDP/IP. I mean to suggest that buffering isn't a proper feature distinction between TCP and unreliable protocols. Separately, it seems a tad obscure to use IP as the exemplar here--doesn't most everyone want UDP features on top?
- ncmncm 7y agoHardly anybody relies on any of the extras UDP provides, so they are typically just wasted overhead. But user-level libraries offer UDP, so that is what people use.
- andrewstuart 7y agoThe thing I like about async Python is being able to run subprocesses and capture and processes their stdout/stderr
- viraptor 7y agoYou can do that without async in python. What did you find specific to it?
- ComputerGuru 7y agoI imagine he means without having to understand/use the platform or framework equivalent of select/poll or start dedicated threads or write a threadpool to handle reading from streams simultaneously without blocking. Now he has someone else wrote that code for him ;)
- andrewstuart 7y agoIt's live and immediate and simultaneous, which you can only get with async or multithreading. Async Python gives you the ability to set up functions that act upon multiple stdout/stderr streams and process them in real time.
- jph 7y agoIdeally async and backpressure will advance to make better use of scheduling, prioritizing, quality of service shaping, termination conditioning, and the like. As an example, there's a big difference between the async needs of one paying customer who's loading one medical chart web page vs. some free third-party web crawler trying a daily full text scan of an entire site. Async with cost functions feels like a promising area for real-world use cases.
- j88439h84 7y agoAs mentioned in the article, Python's Trio solves all of these issues much better than asyncio does. https://trio.readthedocs.io https://trio.readthedocs.io
- jsmeaton 7y agoDo you happen to know how it compares to curio?
- j88439h84 7y agoCurio is more like an experimental sandbox for its author. Trio was created as a production-oriented application of similar ideas. There are some places where Trio diverges from Curio, and I think Trio's choices are improvements there. For example, the handling of schedule points and cancel points (https://trio.readthedocs.io/en/stable/design.html#cancel-points-and-schedule-points https://trio.readthedocs.io/en/stable/design.html#cancel-poi...).
- mayoff 7y agoBackpressure is the big thing that Reactive Streams adds to Rx/ReactiveX. https://www.reactive-streams.org/ https://www.reactive-streams.org/
- deleted 7y ago[deleted]
- LennyWhiteJr 7y agoI wish he would have commented on .NET's async implementation. Microsoft really got it right here and it's arguably the best implementation I've seen. All .NET async APIs take an optional cancellation token parameter which solves his flow-control problem by allowing the async request to be canceled at any time. If the token is canceled, the async task will (or should) throw an OperationCanceledException which can then be cleanly handled in a standard try/catch block up the stack. The best part about this is that it pervades the entire .NET runtime, the APIs, code examples, and has excellent documentation on correct usage patterns. Sure, a 3rd party library could choose not to support cancellation tokens, but they would be going against the entire .NET ecosystem by doing so. Every other async implementation I've seen has really seemed like a haphazard bolt-on. I honestly don't know how I would write robust async code without cancellation tokens, so I guess he has a point when it comes to javascript and python ecosystems.
- mikewhy 7y agoThis is the direction JavaScript is going as well. Having used cancellation tokens in C#, I quite like the pattern. https://developer.mozilla.org/en-US/docs/Web/API/AbortController https://developer.mozilla.org/en-US/docs/Web/API/AbortContro...
- alpb 7y agoHow's it different than Go's context.Done() channel? Similarly it bubbles up everywhere the context is passed immediately, letting you terminate concurrent routines. I've used both .NET async/await (4 years ago) and Go ever since. The semantics and feel is nearly the same, the look is different.
- egeozcan 7y agoIMHO, being able to throw the error and it moving magically up the async stack (aka try/catch) makes it easier to get started while go makes it easier to maintain by forcing you to design a system around channels. This opinion is coming from a decade of .NET but just a few months of go experience so it's very skewed.
- deleted 7y ago[deleted]
- christiansakai 7y agoWhy is Go mentioned here? AFAIK, Go's goroutine makes async not async like in NodeJS async sense, but just lightweight user space threading, so just blocking like regular threading.
- kd5bjo 7y ago> So why is write not doing an implicit drain? Well it's a massive API oversight and I'm not exactly sure how it happened. Separating these two operations allows code to use multiple write() calls to build up a single record atomically before yielding control to the system, where some other task might also write to the same stream. This reasoning is only valid if the program is running on a single thread, but that’s a reasonable architecture decision for many programs.
- heftig 7y agoCouldn't you have an API with a write() return value that you can await to trigger a drain(), but which you can also leave unused, in which case it acts like the current write()?
- PudgePacket 7y agoNot a super great argument. It would be easy to just build up the data to be written then write it in 1 call, while at the same time preventing people from being able to use it incorrectly at all.
- Doxin 7y agoIt seems to me that all that this boils down to is the fact that await/async makes back pressure something you need to deal with explicitly. Having the default be buffering isn't ideal but since each application will have their own idea of what to do with backpressure it'd be hard to figure out a different default that works better. In any case all this can be solved without major rewrites by making sure every awaitable is awaited at some point in the future. Instead of doing this: while connection.accept(): handle_connection() you might do something like this: connection_pool = [] while connection.accept(): connection_pool.append(handle_connection()) if len(connection_pool)>=MAX_CONNECTIONS: await wait_any(connection_pool) And there you go. Anytime there's more than MAX_CONNECTIONS the program stops accepting new connections, providing back pressure. It's more code but it's also defining exactly HOW to provide back pressure. Your specific use case might warrant providing back pressure not based on connection count but cpu usage or average response times. You might want to have a single global maximum connection count instead of one per thread. All of these aren't much more complex than what I've shown above as long as you keep the cardinal rule in mind: any awaitable MUST be awaited. And in fact in python -- and other programming languages too I bet -- you get a warning on exit if there are any awaitables that never got awaited. In my opinion that should be an error. I can't think of a single scenario where it'd be proper form to never await an awaitable. You might await immediately, sometime in the future, or at the end of the program. But you never don't await at all. Threading isn't any easer, or harder. If you spawn a thread for each connection you run into the same issue as await unless you do something about it. If you use a pool of threads you get backpressure for free, but the same goes for a pool of awaitables! tl;dr: async/await has the exact same problems as threads, the tooling around async/await is just less mature. Rewriting async code to provide back pressure is near trivial.
- majke 7y ago> In most async systems … you end up in a world where you chain a bunch of async functions together with no regard of back pressure. yup. Back pressure doesn’t compose in the world of callbacks / async. It does compose if designed well in coroutine world (see: erlang). > async/await is great but it encourages writing stuff that will behave catastrophically when overloaded. yup. It’s very hard, in larger systems impossible, to do back pressure right with callbacks / async programming model. This is how I assess software projects I look at. How fast is database is one thing. What does it do when I send it 2GiB of requests not reading responses? What happens when I open a bazillion connections to it? Will a previously established connections have priority over handling new connections?
- pantulis 7y agoIMHO all the rage on async comes from an era when interpreted languages where the main trend as they demonstrably increased developer productivity --I'm looking at you, Rails, Django. Those platforms were not designed for runtime speed, so it made sense to push the bottleneck to the most inmediate backend system in the chain (i.e.: your trusty database, which is much faster than your agile framework of choice, remember the discussions of ORMs versus pure SQL?) Then there came async frameworks a la Node, Twisted et al. and changed everything. Again in my opinion, async code is harder to reason about versus synchronous code. Things to keep in mind: - Are you really working at scale? Does the arguably added complexity of async benefit your particular use case? Specially when SPA technologies allow to build simpler backend for frontend systems (pure API, no HTML rendering). And not only regarding pure operational performance, Rails is still impossibly hard to beat when it comes to productivity. - New players like Go, Rust have async capabilities but you dont necessarily need to use them to perform closer to native speed, hence becoming simpler solutions than Node, Ruby, or Python. Guess that also applies to old dogs with new tricks in the JVM (Micronaut, Quarkus...)
- qaq 7y agoAs always it all depends on the use case. If you have a lot of persistent connections from clients async might be a good option.
- skohan 7y ago> Then there came async frameworks a la Node, Twisted et al. and changed everything. Again in my opinion, async code is harder to reason about versus synchronous code. One change here which has also made this less important is the trend toward serverless/microservice architecture. In a world where you were running your MVP on a single VPS instance somewhere, and you were running a monolithic webserver which handled everything from request parsing to business logic, then moving from one-thread-per-request to a runloop implementation represented a potentially huge win in terms of performance. But nowadays we largely work with more atomized bits of logic: business logic is implemented in terms of small, self-contained operations, and the problems of scale are delegated to some other system or orchestration framework. In that world it matters much less if your database access is blocking a thread or not, because throughput issues can be solved at a different level.
- panitaxx 7y agoIn nodejs you can use streams or rxjs. Both handle backpressure quite nicely. They work on bytes and on objects as well.
- deleted 7y ago[deleted]
- iforgotpassword 7y agoMy main problem with async is that it's much harder to build a mental model of what exactly is going on. Admittedly, I only had to deal with existing code so far doing only minor work in it so you can definitely say it's due to lack of experience. However it seems that while problems like concurrent access to shared memory known from traditional threaded code don't exist, having to think about what can block etc. is of equal burden.
- carapace 7y agoFWIW, having climbed Twisted's learning curve long ago the current async-all-the-things fad looks so childish to me. I remember when Tornado came out and I was like, why would you use a go-kart when there's a free Maserati right there? https://twistedmatrix.com/trac/ https://twistedmatrix.com/trac/