5 ms·

The C10M problem (2013)

34 points by gaigepr 12y ago

iseyler 12y ago
This article highlights the goals for BareMetal OS (https://github.com/ReturnInfinity/BareMetal-OS https://github.com/ReturnInfinity/BareMetal-OS) - let the app do the heavy lifting.
- darkarmani 12y ago
  If I understand that project correctly, the assumption is that they think they can write assembly by hand that is more optimized than by using a compiler to optimize?
- gaigepr 12y ago
  I'm surprised things like this aren't more common. Places like google strike me as great candidates for bare metal OS type setups to most effectively serve the billions of requests per days.
nkurz 12y ago
Lots of great discussion here about a year ago: https://news.ycombinator.com/item?id=5699552 https://news.ycombinator.com/item?id=5699552 The line that jumped out at me from the article this time was "It costs 300 clock cycles to go out to main memory, at which time the CPU isn’t doing anything." Since the last time I've read this article, I've learned this isn't really true. First, with current memory and bus speeds (Sandy/Ivy/Haswell), it's closer to 100 cycles (or 150 with a single level of TLB miss, although this miss can often be avoided with HugePages). But more importantly, there is no reason for the CPU to be doing nothing during this latency. You can have 10 outstanding memory requests per core, so if you queue your requests for an instant you can issue a prefetch ahead of when you need it and continue working on the current request. This way you are only waiting sub-10 cycles for a load from L1 when the request is at the top of the queue. This doesn't speed up the response latency for each request, but it can help a lot with overcoming a limited cycle budget.
- twoodfin 12y ago
  As an alternative to prefetch, I don't think there's a technical barrier to Intel increasing the number of hyperthreads that can be scheduled on a single core. POWER8 gets up to eight.
  - 3825 12y ago
    Here is a somewhat related question on stack overflow. https://stackoverflow.com/questions/1718465/optimal-number-of-threads-per-core https://stackoverflow.com/questions/1718465/optimal-number-o... > Is there a benefit to running a parallelizable process on more threads than cores? In other words, will my process finish faster, slower, or in about the same amount of time if I run it using 4000 threads rather than 4 threads? > If your threads don't do I/O, synchronization, etc., and there's nothing else running, 1 thread per core will get you the best performance. However that very likely not the case. Adding more threads usually helps, but after some point, they cause some performance degradation. > Not long ago, I was doing performance testing on a 2 quad-core machine running an ASP.NET application on Mono under a pretty decent load. We played with the minimum and maximum number of threads and in the end we found out that for that particular application in that particular configuration the best throughput was somewhere between 36 and 40 threads. Anything outside those boundaries performed worse. Lesson learned? If I were you, I would test with different number of threads until you find the right number for your application. > One thing for sure: 4k threads will take longer. That's a lot of context switches.
    - eloff 12y ago
      This is not related at all. Hyperthreads are hardware threads that share the execution resources of a core. They have nothing to do with OS threads, other than an OS thread is scheduled to run on a hardware thread.
- eloff 12y ago
  You can't always prefetch though. Look at linked lists and trees. You have to execute the code and wait the 100+ cycles at every node access because you can't predict where the next node will be.
mturmon 12y ago
Besides the discussion a year ago pointed out by @nkurz, there is also this from 77 days ago -- https://news.ycombinator.com/item?id=7250505 https://news.ycombinator.com/item?id=7250505
- gaigepr 12y ago
  I must have missed that; thanks for pointing it out.

34 points by gaigepr 12y ago

This article highlights the goals for BareMetal OS (https://github.com/ReturnInfinity/BareMetal-OS https://github.com/ReturnInfinity/BareMetal-OS) - let the app do the heavy lifting.

darkarmani 12y ago
If I understand that project correctly, the assumption is that they think they can write assembly by hand that is more optimized than by using a compiler to optimize?
gaigepr 12y ago
I'm surprised things like this aren't more common. Places like google strike me as great candidates for bare metal OS type setups to most effectively serve the billions of requests per days.

nkurz 12y ago

Lots of great discussion here about a year ago: https://news.ycombinator.com/item?id=5699552 https://news.ycombinator.com/item?id=5699552 The line that jumped out at me from the article this time was "It costs 300 clock cycles to go out to main memory, at which time the CPU isn’t doing anything." Since the last time I've read this article, I've learned this isn't really true. First, with current memory and bus speeds (Sandy/Ivy/Haswell), it's closer to 100 cycles (or 150 with a single level of TLB miss, although this miss can often be avoided with HugePages). But more importantly, there is no reason for the CPU to be doing nothing during this latency. You can have 10 outstanding memory requests per core, so if you queue your requests for an instant you can issue a prefetch ahead of when you need it and continue working on the current request. This way you are only waiting sub-10 cycles for a load from L1 when the request is at the top of the queue. This doesn't speed up the response latency for each request, but it can help a lot with overcoming a limited cycle budget.

twoodfin 12y ago
As an alternative to prefetch, I don't think there's a technical barrier to Intel increasing the number of hyperthreads that can be scheduled on a single core. POWER8 gets up to eight.
- 3825 12y ago
  Here is a somewhat related question on stack overflow. https://stackoverflow.com/questions/1718465/optimal-number-of-threads-per-core https://stackoverflow.com/questions/1718465/optimal-number-o... > Is there a benefit to running a parallelizable process on more threads than cores? In other words, will my process finish faster, slower, or in about the same amount of time if I run it using 4000 threads rather than 4 threads? > If your threads don't do I/O, synchronization, etc., and there's nothing else running, 1 thread per core will get you the best performance. However that very likely not the case. Adding more threads usually helps, but after some point, they cause some performance degradation. > Not long ago, I was doing performance testing on a 2 quad-core machine running an ASP.NET application on Mono under a pretty decent load. We played with the minimum and maximum number of threads and in the end we found out that for that particular application in that particular configuration the best throughput was somewhere between 36 and 40 threads. Anything outside those boundaries performed worse. Lesson learned? If I were you, I would test with different number of threads until you find the right number for your application. > One thing for sure: 4k threads will take longer. That's a lot of context switches.
  - eloff 12y ago
    This is not related at all. Hyperthreads are hardware threads that share the execution resources of a core. They have nothing to do with OS threads, other than an OS thread is scheduled to run on a hardware thread.
eloff 12y ago
You can't always prefetch though. Look at linked lists and trees. You have to execute the code and wait the 100+ cycles at every node access because you can't predict where the next node will be.

mturmon 12y ago

Besides the discussion a year ago pointed out by @nkurz, there is also this from 77 days ago -- https://news.ycombinator.com/item?id=7250505 https://news.ycombinator.com/item?id=7250505

gaigepr 12y ago
I must have missed that; thanks for pointing it out.