5 ms·

I had understood your concern as wanting to avoid the complexity of asynchronous code execution in your codebase, I did not realize your concern is about writin

I had understood your concern as wanting to avoid the complexity of asynchronous code execution in your codebase, I did not realize your concern is about writing very low level systems code. In that case, you are doing the right thing: libraries like ureq, minreq, Isahc, curl, and more all offer what you want. It is unclear to me what you mean by keeping the community "in check". There are a lot of people who rely on and enjoy the async story, and they will continue to produce code that improves that story. Simultaneously, there are people who do not need that, and they are not hindered by this. People will build what they want and need. You've just picked some libraries from some of the biggest async contributors in the community and requested that they be kept in check so that you don't have to switch to a synchronous alternative, of which there are plenty.

AgalmicVentures 6y ago
You're replying to John Nagle, as in https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm -- perhaps it would be worth considering his words further, before dismissing his concerns? Surely we can all agree that spinning up unnecessary threads is undesirable?
- bobthebuilders 6y ago
  Appeal to authority fallacy
- deleted 6y ago
  [deleted]
- woah 6y ago
  Doesn’t matter who he is, if he’s mad that libraries made for and by web and network developers are using a concurrency model that works well for their applications, he should use different libraries.
  - AgalmicVentures 6y ago
    It does matter -- he was one of the first "network developers". He published RFC 896 over 35 years ago; he has more experience on this topic than almost anyone. > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. This is not the strongest plausible interpretation of what he said -- He's not asking for people to not develop async code. He's asking for them to not hide it in synchronous code. If you're expecting a blocking system call, and actually get a brand new background thread that's polling, it's quite reasonable to be frustrated.
    - cycloptic 6y ago
      >If you're expecting a blocking system call, and actually get a brand new background thread that's polling, it's quite reasonable to be frustrated. It really isn't if the documentation doesn't outright say that it's single threaded and not thread safe. For a lot of simpler use cases where you just want to ship a thread-safe API (e.g. application does not have its own thread pool) then it just makes sense in a lot of cases to use some kind of automatic thread pooling. The caller does not have to know or care how the internal state machine is implemented. If you have implemented your own thread pool it seems you should know enough to dig down enough to the lower layers to where you can get to that blocking syscall, or least to the point where you can strip off the O_NONBLOCK flags yourself.
Animats 6y ago
It's not that "low level". It's that it doesn't fit the model of "mostly waiting for the network". Here are some of the things I have going on: - Incoming events UDP packets from multiple servers. These arrive at the client and go into a queue for processing. - Refreshing the 3D window. This ties up one thread almost full time. At the beginning of each frame, it reads queued events that tell it what needs to change in the GPU. The rest of the time it feeds the GPU. - Some incoming events require querying external HTTP servers to retrieve assets. When the results come back, they include compressed items which have to be decompressed, processed, and turned into GPU-ready textures or meshes. This is prioritized by how important it is to display that object right now, based on the viewpoint. So there are priority queues along with multithreading. There's more, but you get the idea. What's so great about Rust is that you can do stuff like this without crashing all the time, spending your life in the debugger, or recopying big objects to safely pass them around. The previous implementation, in C++, looked a lot more like an "async" model. It had lots of "coroutines", mostly running off a single thread. It also had a few things running as independent threads because they were so CPU-intensive. It was very prone to short stalls that annoyed players. This happened because something had to do more work than expected and briefly stalled out the coroutine system. The killer in async systems is the subroutine that is usually fast but sometimes slow. So I've seen this problem done in "async" style, and it didn't work well. I've previously done robotics work which had many asynchronous tasks. I've used QNX for that, and I've used ROS. There, you have a lot of intercommunicating processes, which works but has more overhead. None of this maps well to an "async" model. Part of the problem here may be that I've done a lot of multi-thread programming and am used to it. It's an alien approach to programmers who came up from Javascript. That's a big fraction of the web backend crowd. (Personally, when I have to do a web service, I write it in Go. That's the use case for which Go is designed. It has the libraries for that, and the goroutine concept is well matched to that task.)
- dunkelheit 6y ago
  > I've previously done robotics work which had many asynchronous tasks. I've used QNX for that, and I've used ROS. There, you have a lot of intercommunicating processes, which works but has more overhead. I wonder, why is this kind of system a bad fit for the async model? Is it because all threads must be reliably preemptable?

AgalmicVentures 6y ago

You're replying to John Nagle, as in https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm https://en.m.wikipedia.org/wiki/Nagle%27s_algorithm -- perhaps it would be worth considering his words further, before dismissing his concerns? Surely we can all agree that spinning up unnecessary threads is undesirable?

bobthebuilders 6y ago
Appeal to authority fallacy
deleted 6y ago
[deleted]
woah 6y ago
Doesn’t matter who he is, if he’s mad that libraries made for and by web and network developers are using a concurrency model that works well for their applications, he should use different libraries.
- AgalmicVentures 6y ago
  It does matter -- he was one of the first "network developers". He published RFC 896 over 35 years ago; he has more experience on this topic than almost anyone. > Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith. This is not the strongest plausible interpretation of what he said -- He's not asking for people to not develop async code. He's asking for them to not hide it in synchronous code. If you're expecting a blocking system call, and actually get a brand new background thread that's polling, it's quite reasonable to be frustrated.
  - cycloptic 6y ago
    >If you're expecting a blocking system call, and actually get a brand new background thread that's polling, it's quite reasonable to be frustrated. It really isn't if the documentation doesn't outright say that it's single threaded and not thread safe. For a lot of simpler use cases where you just want to ship a thread-safe API (e.g. application does not have its own thread pool) then it just makes sense in a lot of cases to use some kind of automatic thread pooling. The caller does not have to know or care how the internal state machine is implemented. If you have implemented your own thread pool it seems you should know enough to dig down enough to the lower layers to where you can get to that blocking syscall, or least to the point where you can strip off the O_NONBLOCK flags yourself.

Animats 6y ago

It's not that "low level". It's that it doesn't fit the model of "mostly waiting for the network". Here are some of the things I have going on: - Incoming events UDP packets from multiple servers. These arrive at the client and go into a queue for processing. - Refreshing the 3D window. This ties up one thread almost full time. At the beginning of each frame, it reads queued events that tell it what needs to change in the GPU. The rest of the time it feeds the GPU. - Some incoming events require querying external HTTP servers to retrieve assets. When the results come back, they include compressed items which have to be decompressed, processed, and turned into GPU-ready textures or meshes. This is prioritized by how important it is to display that object right now, based on the viewpoint. So there are priority queues along with multithreading. There's more, but you get the idea. What's so great about Rust is that you can do stuff like this without crashing all the time, spending your life in the debugger, or recopying big objects to safely pass them around. The previous implementation, in C++, looked a lot more like an "async" model. It had lots of "coroutines", mostly running off a single thread. It also had a few things running as independent threads because they were so CPU-intensive. It was very prone to short stalls that annoyed players. This happened because something had to do more work than expected and briefly stalled out the coroutine system. The killer in async systems is the subroutine that is usually fast but sometimes slow. So I've seen this problem done in "async" style, and it didn't work well. I've previously done robotics work which had many asynchronous tasks. I've used QNX for that, and I've used ROS. There, you have a lot of intercommunicating processes, which works but has more overhead. None of this maps well to an "async" model. Part of the problem here may be that I've done a lot of multi-thread programming and am used to it. It's an alien approach to programmers who came up from Javascript. That's a big fraction of the web backend crowd. (Personally, when I have to do a web service, I write it in Go. That's the use case for which Go is designed. It has the libraries for that, and the goroutine concept is well matched to that task.)

dunkelheit 6y ago
> I've previously done robotics work which had many asynchronous tasks. I've used QNX for that, and I've used ROS. There, you have a lot of intercommunicating processes, which works but has more overhead. I wonder, why is this kind of system a bad fit for the async model? Is it because all threads must be reliably preemptable?