4 ms·

This seems to be a response to Google's switchto work in the Linux kernel, which is motivated by task-switching costs, as per this 2013 LPC presentation: http:/

This seems to be a response to Google's switchto work in the Linux kernel, which is motivated by task-switching costs, as per this 2013 LPC presentation: http://pdxplumbers.osuosl.org/2013/ocw//system/presentations/1653/original/LPC%20-%20User%20Threading.pdf http://pdxplumbers.osuosl.org/2013/ocw//system/presentations... Note the parenthetical in the article: "And that is how user-mode threads help: they increase L by orders of magnitude with potentially millions of user-mode threads instead of the meager thousands the OS can support (but don’t expect a 1000x increase in capacity; we’ve neglected computation costs and are bound to hit bottlenecks in the auxiliary services.)" This is something I'd like to see more of a focus on. For the generator use case, I can easily see how the kernel thread spawn operation is the bottleneck. But for the thread-per-connection server use case, I'm not sure how expensive this cost is relative to all the other work that the thread does. My suspicion is that Amdahl's Law is going to quickly rear its head here. Take stack size for example: assuming the kernel stack is 10kB, if your thread itself uses 10kB of stack, you've cut the theoretical memory advantage of M:N down from the cited 1000x to a mere 2x…

pron 6y ago
> This seems to be a response to Google's switchto work in the Linux kernel Actually, it's a response to a discussion where somebody asked, "isn't it all about context-switch cost?" But user-scheduled kernel thread is something we had in mind when designing Loom, and we've made sure we can be compatible with them. We've introduced the concept of a pluggable custom scheduler, and that could be used for kernel threads just as it's used for virtual threads. The user can choose a thread implementation -- all kernel, all user-mode, or part kernel/part usermode -- without changing any code. It's all an implementation detail. > My suspicion is that Amdahl's Law is going to quickly rear its head here. That depends. Amdahl's law is about accelerating one job by parallelising it, while here we're more concerned with Little's law, which is about the rate of independent requests you can process. > Take stack size for example: assuming the kernel stack is 10kB, if your thread itself uses 10kB of stack, you've cut the theoretical memory advantage of M:N down from the cited 1000x to a mere 2x… Ah, except that's not so easy to do. It's very hard to have "tight" stacks that are managed by the kernel for the reasons I mentioned here: https://news.ycombinator.com/item?id=24082951 https://news.ycombinator.com/item?id=24082951
- scott_s 6y ago
  > Amdahl's law is about accelerating one job by parallelising it, Amdahl's law is about the limitations of improving the speed of something, but not necessarily through parallelization. Its main point is that if you have some overall task, and you then speed up some subtask within it, the improvement to the overall time is limited to the contribution from the subtask. In context of pcwalton's comment, I think they meant that thread creation time may be tiny compared to the amount of time a newly created thread will do work in a server context. If that is the case, improving the thread creation time will have limited benefit to serve time.
  - pron 6y ago
    > Amdahl's law is about the limitations of improving the speed of something, but not necessarily through parallelization. Right, but the goal here is not to improve the speed of something, but rather to handle as many different, mostly independent somethings as possible, without necessarily improving their speed (latency), at all. Amdahl's law comes into effect in the delta between mostly and completely. > they meant that thread creation time may be tiny compared to the amount of time a newly created thread will do work in a server context. If that is the case, improving the thread creation time will have limited benefit to serve time. Right, this is the same argument as for the context-switch overhead. Still, I have to say that both virtual thread creation and context-switching are much better than for OS thread, but the point of the post was to show that in many common use cases that is not where most of the benefit comes from; rather it comes from the number of threads you can have. For example, if thread creation time was high but you could create millions of them, you could create all of them up front and pool them and still get most/all of the benefits. But if thread creation time was low but you could only have a few thousand, then you'd still lose big because of Little's law.
    - scott_s 6y ago
      Agreed, strongly. I was just concerned the original comment may not have been fully understood.
Matthias247 6y ago
> But for the thread-per-connection server use case, I'm not sure how expensive this cost is relative to all the other work that the thread does The answer for the basic thread-per-connection servers you are referring to is pretty easy: It does not matter at all, since servers use thread-pools and allocate the treads only once. Any reuse of threads is free, and the cost is purely context-switching. The story might be a bit different if servers also start multiple sub-tasks (or child-threads) while processing a request. E.g. if you are running a HTTP/2 as a protocol, you will need multiple tasks just to handle the individual streams on the TCP connection, plus 1 or 2 tasks that perform the shared connection operations. In that case you might not want to spawn a thread (or allocate one from a thread-pool) for each item, but either have a lightweight thread alternative or go back to the old very basic eventloop model. Let's say you want to handle 1k TCP connections - which is in the medium range for public server. Having a thread-pool of size 1k is OK. But now if each of those connections could carry 100 concurrent requests (which is kind of a default limit for HTTP/2), you already get a huge amount of concurrency.

pron 6y ago

> This seems to be a response to Google's switchto work in the Linux kernel Actually, it's a response to a discussion where somebody asked, "isn't it all about context-switch cost?" But user-scheduled kernel thread is something we had in mind when designing Loom, and we've made sure we can be compatible with them. We've introduced the concept of a pluggable custom scheduler, and that could be used for kernel threads just as it's used for virtual threads. The user can choose a thread implementation -- all kernel, all user-mode, or part kernel/part usermode -- without changing any code. It's all an implementation detail. > My suspicion is that Amdahl's Law is going to quickly rear its head here. That depends. Amdahl's law is about accelerating one job by parallelising it, while here we're more concerned with Little's law, which is about the rate of independent requests you can process. > Take stack size for example: assuming the kernel stack is 10kB, if your thread itself uses 10kB of stack, you've cut the theoretical memory advantage of M:N down from the cited 1000x to a mere 2x… Ah, except that's not so easy to do. It's very hard to have "tight" stacks that are managed by the kernel for the reasons I mentioned here: https://news.ycombinator.com/item?id=24082951 https://news.ycombinator.com/item?id=24082951

scott_s 6y ago
> Amdahl's law is about accelerating one job by parallelising it, Amdahl's law is about the limitations of improving the speed of something, but not necessarily through parallelization. Its main point is that if you have some overall task, and you then speed up some subtask within it, the improvement to the overall time is limited to the contribution from the subtask. In context of pcwalton's comment, I think they meant that thread creation time may be tiny compared to the amount of time a newly created thread will do work in a server context. If that is the case, improving the thread creation time will have limited benefit to serve time.
- pron 6y ago
  > Amdahl's law is about the limitations of improving the speed of something, but not necessarily through parallelization. Right, but the goal here is not to improve the speed of something, but rather to handle as many different, mostly independent somethings as possible, without necessarily improving their speed (latency), at all. Amdahl's law comes into effect in the delta between mostly and completely. > they meant that thread creation time may be tiny compared to the amount of time a newly created thread will do work in a server context. If that is the case, improving the thread creation time will have limited benefit to serve time. Right, this is the same argument as for the context-switch overhead. Still, I have to say that both virtual thread creation and context-switching are much better than for OS thread, but the point of the post was to show that in many common use cases that is not where most of the benefit comes from; rather it comes from the number of threads you can have. For example, if thread creation time was high but you could create millions of them, you could create all of them up front and pool them and still get most/all of the benefits. But if thread creation time was low but you could only have a few thousand, then you'd still lose big because of Little's law.
  - scott_s 6y ago
    Agreed, strongly. I was just concerned the original comment may not have been fully understood.

Matthias247 6y ago

> But for the thread-per-connection server use case, I'm not sure how expensive this cost is relative to all the other work that the thread does The answer for the basic thread-per-connection servers you are referring to is pretty easy: It does not matter at all, since servers use thread-pools and allocate the treads only once. Any reuse of threads is free, and the cost is purely context-switching. The story might be a bit different if servers also start multiple sub-tasks (or child-threads) while processing a request. E.g. if you are running a HTTP/2 as a protocol, you will need multiple tasks just to handle the individual streams on the TCP connection, plus 1 or 2 tasks that perform the shared connection operations. In that case you might not want to spawn a thread (or allocate one from a thread-pool) for each item, but either have a lightweight thread alternative or go back to the old very basic eventloop model. Let's say you want to handle 1k TCP connections - which is in the medium range for public server. Having a thread-pool of size 1k is OK. But now if each of those connections could carry 100 concurrent requests (which is kind of a default limit for HTTP/2), you already get a huge amount of concurrency.