13 ms·

I feel like I'm too dumb to understand any of this. And I've been writing python for 12 years. Just give me greenlets or whatever and let me run synchronous c

I feel like I'm too dumb to understand any of this. And I've been writing python for 12 years. Just give me greenlets or whatever and let me run synchronous code concurrently. async def proxy(dest_host, dest_port, main_task, source_sock, addr): await main_task.cancel() dest_sock = await curio.open_connection(dest_host, dest_port) async with dest_sock: await copy_all(source_sock, dest_sock) Are you kidding me? Simplified that is async def func(): await f() dest_sock = await f() async with dest_sock: await f() Every other token is async or await. No thank you.

dismantlethesun 10y ago
Welcome, the wonderful world of writing anything in Javascript. Imagine the same thing using Promises: def proxy(dest_host, dest_port, main_task, source_sock, addr): main_task.cancel()\ .then(lamdba _: curio.open_connection(dest_host, dest_port))\ .then(lambda dest_sock: copy_all(source_sock, dest_sock)
- noobiemcfoob 10y ago
  It's statements like this that have kept me from ever learning _anything_ in Javascript...
foota 10y ago
I don't think most code will be this dense with await.
- sidlls 10y ago
  And Rust's developers think that 'unsafe' in third-party crates will be well-vetted and therefore actually "safe", most C developers don't think somebody will incorrectly free or screw with memory they've allocated and passed back to the caller, most C++ developers don't think anybody will (ab)use 'const_cast', and so on. A lot of terrible bugs in code is caused by people making assumptions such as yours.
  - piotrjurkiewicz 10y ago
    He didn't make an 'assumption' like those ones you described. This is an artificial example of a function copying unmodified data from source to destination. There are async and await tokens in every line, because every line is doing an IO operation. I a real world app this data would be somehow processed in between, using synchronous function calls, therefore without async/await tokens.
  - imtringued 10y ago
    >most C++ developers don't think anybody will (ab)use 'const_cast', and so on. These constructs are opt-in. If you don't want them in your codebase you can find their location by a simple text based search and remove them. In C everything is "unsafe". You can't opt-out.
  - steveklabnik 10y ago
    I don't think anyone is saying that you'll never see crates with bad usage of unsafe. What you will hear them say is that by having the ability to share code, since more people are looking at and using the same codebase, it's more likely issues will be found, and that when they're fixed, they help everyone using the package, rather than just those who found it.
- int_19h 10y ago
  I've been writing async/await code for the past 2.5 years, and no, it actually is typically this dense, if you count tokens (real identifiers are obviously longer, so it's not as bad character-wise, and awaits are not quite as prominent).
  - foota 10y ago
    Interesting, thanks for sharing that insight. Do you feel that your work is representative, or is there some reason that the code you write would have a higher than usual density? It seems like a lot of code, which is just business logic, would not use these constructs other than on the io barriers.
    - int_19h 10y ago
      Don't forget that async is "viral": if you call an async function and need to do something with its result, the calling function must in turn be async for await to work inside it. So the moment you start doing some async I/O at the bottom of some call stack, the entire stack becomes "infected" by async, and needs have awaits inserted in every frame. And it so happens that I work on the kind of products where a lot of useful work revolves around I/O: IDEs.
      - Sacho 10y ago
        It doesn't have to be viral. C#'s tasks have a blocking Wait[0] method which allows you to use an asynchronous Task without changing the signature of your synchronous function. The tradeoff is more verbosity. [0] - https://msdn.microsoft.com/en-us/library/dd235635(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/dd235635(v=vs.110)....
        nickkell 10y ago
        That's not the only tradeoff. It completely negates the benefit of asynchrony and can be a source of deadlocks
        int_19h 10y ago
quotemstr 10y ago
People keep inventing funky new ways of representing threads. With or without async, we're writing threads. (Promise chains are _also_ threads, very awkwardly spelled.) Really, we're arguing over whether we want our preemption points to be explicit or implicit. I prefer implicit myself, because the implicit style leads to much clearer code. I understand how the JavaScript people might be excited that they can finally have threads, even if ugly ones, but there's no reason to get the rest of the world to switch to explicit-preemption-point threads.
- int_19h 10y ago
  > Really, we're arguing over whether we want our preemption points to be explicit or implicit. It's not even that! It's not like you actually get to decide where to await in async/await code - you have to await on any call that is async, if you expect to get the result. Now, if the underlying framework uses hot tasks - meaning the async operation starts executing as soon as it's invoked, and not when the returned task is awaited (as in e.g. .NET/C#) - you can choose to omit async to, effectively, fork your async "thread". So NOT doing await on something is just a fork operation. It's the reverse from regular sync code, where thread forks are explicit, and sequential flow on a single thread is implicit. One other case where you wouldn't await is when you need to await on a combination of any or all tasks at the same time (i.e., wait until all tasks complete, or wait until one of the tasks completes). But the first one is equivalent to a thread join in sync code, and the second to a condition variable. So, again, you get a case where something more explicit in sync code is more implicit in async code, and vice versa. Now note that all this is solely about syntax! You can take the C# compiler, and change it so that every awaitable statement is automatically awaited, except when the newly introduced operator "taskof" is applied, in which case you get the raw future instead. Voila! Cooperative future-based multitasking with implicit preemption points. Yet it works exactly the same, and will even be able to call into and be called from any existing C# code compiled by the original compiler. I suspect that this will be the next step after async/await, once enough people notice that the default (non-await) behavior is something that they need very rarely, and figure out that it's better to rather change the syntax so that the much more common thing (await) is implicit. Similar to how the use of =/== for assignment and comparison has won out over :=/= in imperative languages.
jeswin 10y ago
Are you saying using greenlets are any simpler than this? IMO that mechanism looks way more complex compared to this. And will probably be less efficient. The point is this: threads are still expensive in bulk (the CPU has to shuffle a lot of data every time you switch). So all kernels have mechanisms to support parallel IO operations. An async library will use the best available kernel mechanism for IO; epoll on Linux, kqueue on BSDs, maybe IO Completion Ports on Windows (not sure). Turns out, doing that requires some help from the language itself or the code turns into a pyramidal mess. Async keyword addresses the readability aspect of code. So: a) It's more complex than synchronous code b) But it solves the performance problem without too much cognitive overhead (once you get used to it).
- quotemstr 10y ago
  > threads are still expensive in bulk They don't have to be. First of all, even ordinary threads are more efficient than you might think. On a really awful low-end Android 4.1 device, I can pthread_create and pthread_join over 5,000 threads per second. On a real computer, my X1 Carbon Gen4, I can create and join over 110,000 threads per second. (And keep in mind that each create-join pair also forces two full context switches.) For most applications, performance of regular threads is perfectly adequate. In these environments, the maintainability and debuggability advantages of using plain old boring threads makes it really hard to justify using something exotic. But suppose you do have big performance requirements: you can still use normal-looking threaded code. There's a difference between how we represent threads in source code and how we implement them. It's possible to provide green, userspace-switched threads without requiring "await" and "async" keywords everywhere. GNU Pth did it a long time ago, and there are lots of other fibers implementations. > the CPU has to shuffle a lot of data every time you switch Any green-threaded system (with or without explicit preemption points) also does context switches! Such a system maintains in user space a queue of things to work on: as the system switches from one of these work items to another, it's switching contexts! You have the same kind of register reloading and cache coldness problems that switching thread contexts has. There's no particular reason that you can do it much better than the kernel can do it, especially since switching threads in the same address space is pretty efficient.
bufordsharkley 10y ago
After watching (Curio creator) David Beazley's presentation from earlier this year on async/await[0], I feel I finally get it. Recommended watching. [0] https://www.youtube.com/watch?v=E-1Y4kSsAFc https://www.youtube.com/watch?v=E-1Y4kSsAFc
- insertnickname 10y ago
  The amount of times Beazley says "insane", "nightmare", etc. in this talk makes me wary.
BonoboBoner 10y ago
Well it is only useful when you really rely on asynchronous programming. Nobody states that every piece of code is supposed to be written like this. You should only use async/await when a thorough performance analysis shows that it is your bottleneck. Think of handling a web request, where you have to do parallel I/O requests to subsystems like a database, a webservice, redis, and so on. I think async/await gives us a nice standard way of describing "hit me back once X is done".
imtringued 10y ago
It's reasonable compared to the old way of having three layers of callbacks in Node.js.
rudolf0 10y ago
I still use gevent any time I need async code. It's also easy to tack onto existing projects with its monkey patching. I've never seen a need to migrate away from gevent, even if it's inarguably a language hack.

dismantlethesun 10y ago

Welcome, the wonderful world of writing anything in Javascript. Imagine the same thing using Promises: def proxy(dest_host, dest_port, main_task, source_sock, addr): main_task.cancel()\ .then(lamdba _: curio.open_connection(dest_host, dest_port))\ .then(lambda dest_sock: copy_all(source_sock, dest_sock)

noobiemcfoob 10y ago
It's statements like this that have kept me from ever learning _anything_ in Javascript...

foota 10y ago

I don't think most code will be this dense with await.

sidlls 10y ago
And Rust's developers think that 'unsafe' in third-party crates will be well-vetted and therefore actually "safe", most C developers don't think somebody will incorrectly free or screw with memory they've allocated and passed back to the caller, most C++ developers don't think anybody will (ab)use 'const_cast', and so on. A lot of terrible bugs in code is caused by people making assumptions such as yours.
- piotrjurkiewicz 10y ago
  He didn't make an 'assumption' like those ones you described. This is an artificial example of a function copying unmodified data from source to destination. There are async and await tokens in every line, because every line is doing an IO operation. I a real world app this data would be somehow processed in between, using synchronous function calls, therefore without async/await tokens.
- imtringued 10y ago
  >most C++ developers don't think anybody will (ab)use 'const_cast', and so on. These constructs are opt-in. If you don't want them in your codebase you can find their location by a simple text based search and remove them. In C everything is "unsafe". You can't opt-out.
- steveklabnik 10y ago
  I don't think anyone is saying that you'll never see crates with bad usage of unsafe. What you will hear them say is that by having the ability to share code, since more people are looking at and using the same codebase, it's more likely issues will be found, and that when they're fixed, they help everyone using the package, rather than just those who found it.
int_19h 10y ago
I've been writing async/await code for the past 2.5 years, and no, it actually is typically this dense, if you count tokens (real identifiers are obviously longer, so it's not as bad character-wise, and awaits are not quite as prominent).
- foota 10y ago
  Interesting, thanks for sharing that insight. Do you feel that your work is representative, or is there some reason that the code you write would have a higher than usual density? It seems like a lot of code, which is just business logic, would not use these constructs other than on the io barriers.
  - int_19h 10y ago
    Don't forget that async is "viral": if you call an async function and need to do something with its result, the calling function must in turn be async for await to work inside it. So the moment you start doing some async I/O at the bottom of some call stack, the entire stack becomes "infected" by async, and needs have awaits inserted in every frame. And it so happens that I work on the kind of products where a lot of useful work revolves around I/O: IDEs.
    - Sacho 10y ago
      It doesn't have to be viral. C#'s tasks have a blocking Wait[0] method which allows you to use an asynchronous Task without changing the signature of your synchronous function. The tradeoff is more verbosity. [0] - https://msdn.microsoft.com/en-us/library/dd235635(v=vs.110).aspx https://msdn.microsoft.com/en-us/library/dd235635(v=vs.110)....
      - nickkell 10y ago
        That's not the only tradeoff. It completely negates the benefit of asynchrony and can be a source of deadlocks
      - int_19h 10y ago

quotemstr 10y ago

People keep inventing funky new ways of representing threads. With or without async, we're writing threads. (Promise chains are _also_ threads, very awkwardly spelled.) Really, we're arguing over whether we want our preemption points to be explicit or implicit. I prefer implicit myself, because the implicit style leads to much clearer code. I understand how the JavaScript people might be excited that they can finally have threads, even if ugly ones, but there's no reason to get the rest of the world to switch to explicit-preemption-point threads.

int_19h 10y ago
> Really, we're arguing over whether we want our preemption points to be explicit or implicit. It's not even that! It's not like you actually get to decide where to await in async/await code - you have to await on any call that is async, if you expect to get the result. Now, if the underlying framework uses hot tasks - meaning the async operation starts executing as soon as it's invoked, and not when the returned task is awaited (as in e.g. .NET/C#) - you can choose to omit async to, effectively, fork your async "thread". So NOT doing await on something is just a fork operation. It's the reverse from regular sync code, where thread forks are explicit, and sequential flow on a single thread is implicit. One other case where you wouldn't await is when you need to await on a combination of any or all tasks at the same time (i.e., wait until all tasks complete, or wait until one of the tasks completes). But the first one is equivalent to a thread join in sync code, and the second to a condition variable. So, again, you get a case where something more explicit in sync code is more implicit in async code, and vice versa. Now note that all this is solely about syntax! You can take the C# compiler, and change it so that every awaitable statement is automatically awaited, except when the newly introduced operator "taskof" is applied, in which case you get the raw future instead. Voila! Cooperative future-based multitasking with implicit preemption points. Yet it works exactly the same, and will even be able to call into and be called from any existing C# code compiled by the original compiler. I suspect that this will be the next step after async/await, once enough people notice that the default (non-await) behavior is something that they need very rarely, and figure out that it's better to rather change the syntax so that the much more common thing (await) is implicit. Similar to how the use of =/== for assignment and comparison has won out over :=/= in imperative languages.

jeswin 10y ago

Are you saying using greenlets are any simpler than this? IMO that mechanism looks way more complex compared to this. And will probably be less efficient. The point is this: threads are still expensive in bulk (the CPU has to shuffle a lot of data every time you switch). So all kernels have mechanisms to support parallel IO operations. An async library will use the best available kernel mechanism for IO; epoll on Linux, kqueue on BSDs, maybe IO Completion Ports on Windows (not sure). Turns out, doing that requires some help from the language itself or the code turns into a pyramidal mess. Async keyword addresses the readability aspect of code. So: a) It's more complex than synchronous code b) But it solves the performance problem without too much cognitive overhead (once you get used to it).

quotemstr 10y ago
> threads are still expensive in bulk They don't have to be. First of all, even ordinary threads are more efficient than you might think. On a really awful low-end Android 4.1 device, I can pthread_create and pthread_join over 5,000 threads per second. On a real computer, my X1 Carbon Gen4, I can create and join over 110,000 threads per second. (And keep in mind that each create-join pair also forces two full context switches.) For most applications, performance of regular threads is perfectly adequate. In these environments, the maintainability and debuggability advantages of using plain old boring threads makes it really hard to justify using something exotic. But suppose you do have big performance requirements: you can still use normal-looking threaded code. There's a difference between how we represent threads in source code and how we implement them. It's possible to provide green, userspace-switched threads without requiring "await" and "async" keywords everywhere. GNU Pth did it a long time ago, and there are lots of other fibers implementations. > the CPU has to shuffle a lot of data every time you switch Any green-threaded system (with or without explicit preemption points) also does context switches! Such a system maintains in user space a queue of things to work on: as the system switches from one of these work items to another, it's switching contexts! You have the same kind of register reloading and cache coldness problems that switching thread contexts has. There's no particular reason that you can do it much better than the kernel can do it, especially since switching threads in the same address space is pretty efficient.

bufordsharkley 10y ago

After watching (Curio creator) David Beazley's presentation from earlier this year on async/await[0], I feel I finally get it. Recommended watching. [0] https://www.youtube.com/watch?v=E-1Y4kSsAFc https://www.youtube.com/watch?v=E-1Y4kSsAFc

insertnickname 10y ago
The amount of times Beazley says "insane", "nightmare", etc. in this talk makes me wary.

BonoboBoner 10y ago

Well it is only useful when you really rely on asynchronous programming. Nobody states that every piece of code is supposed to be written like this. You should only use async/await when a thorough performance analysis shows that it is your bottleneck. Think of handling a web request, where you have to do parallel I/O requests to subsystems like a database, a webservice, redis, and so on. I think async/await gives us a nice standard way of describing "hit me back once X is done".

imtringued 10y ago

It's reasonable compared to the old way of having three layers of callbacks in Node.js.

rudolf0 10y ago

I still use gevent any time I need async code. It's also easy to tack onto existing projects with its monkey patching. I've never seen a need to migrate away from gevent, even if it's inarguably a language hack.