4 ms·

My thoughts exactly. Unless you use a language with cheap and fast green threads that multiplex on OS threads (like Haskell), you should use a number of threads

by exDM69 14y ago

My thoughts exactly. Unless you use a language with cheap and fast green threads that multiplex on OS threads (like Haskell), you should use a number of threads that is roughly in the same order of magnitude as the number of CPU cores + hyper threads. I am very surprised by the fact that there was any speedup at all and not surprised that the parallel version used 200x (!!) memory. On the other hand, this is probably one of those thing that was done for demonstration purposes, not really something you'd do on practice. Btw. is async a part of C++11 standard at all? I thought it was omitted from the standard but may be still implemented by some compilers/stdlibs. Does every async call start a new thread and why?

octopus 14y ago
std::async is included in the C++11 standard. The number of threads that async will use is implementation dependent, I bet on Windows with VS2012 you will see a different number of threads. You could force async to run asynchronously or in a lazy evaluation mode using launch policies.
vonmoltke 14y ago
> Unless you use a language with cheap and fast green threads that multiplex on OS threads (like Haskell), you should use a number of threads that is roughly in the same order of magnitude as the number of CPU cores + hyper threads. The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread[1]. For maximum CPU utilization and computational efficiency, you need multiple software threads per hardware thread so that the OS always has a thread ready to go when another thread blocks or the CPU has an unused component. How much the threads block and what parts of the CPU the exercise is the workload-dependent part. Ideally, the number of threads would be a tunable performance parameter. As an example of this, I recently wrote some math-heavy code to transform the cells of a matrix by a particular transfer function. Each cell calculation was independent, so I parallelized them. The most efficient parallelization for my 6-core AMD processor turned out to be 6 threads per core. [1] The main exception I can think of is when running multiple applications in parallel that in turn parallelize their work.
- exDM69 14y ago
  > The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread. This is exactly why I said it should be in the same order of magnitude, not exactly the same. Definitely not 250+ threads on a dual core. It might make sense to add a few more threads to keep the CPU busy if one or more of the threads will have to wait.
  - vonmoltke 14y ago
    You're right, I misinterpreted "order of magnitude".
- jedbrown 14y ago
  This is dangerous misinformation. I'm sorry, but if your "math-heavy" code performs better when oversubscribed, then it is a massively suboptimal implementation. As a performance-oriented mathematical library developer, with colleagues at many of the major supercomputing centers and vendors, oversubscription is a hazard that we guard against on CPU architectures because it is always slower. Many supercomputing environments use a kernel that does not even support over-subscription. Note that GPUs are different in the sense that over-decomposition is necessary to cover latency, but even there, you generally get better performance at lower occupancy due to better register reuse. SMT CPU systems, most notably, Blue Gene/Q, often require use of multiple hardware threads per core to cover instruction latency and keep the prefetcher busy (due to allocation of outstanding requests). This is not over-subscription because those are bonified hardware threads.
  - alexchamberlain 14y ago
    Can I pick your brains??? What happens during a cache miss? Does the thread block?
    - lallysingh 14y ago
      Other instructions in the same thread are executed (when they can be, e.g. all data's available). Or in another hyperthread on the same core.

by exDM69 14y ago

octopus 14y ago

std::async is included in the C++11 standard. The number of threads that async will use is implementation dependent, I bet on Windows with VS2012 you will see a different number of threads. You could force async to run asynchronously or in a lazy evaluation mode using launch policies.

vonmoltke 14y ago

> Unless you use a language with cheap and fast green threads that multiplex on OS threads (like Haskell), you should use a number of threads that is roughly in the same order of magnitude as the number of CPU cores + hyper threads. The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread[1]. For maximum CPU utilization and computational efficiency, you need multiple software threads per hardware thread so that the OS always has a thread ready to go when another thread blocks or the CPU has an unused component. How much the threads block and what parts of the CPU the exercise is the workload-dependent part. Ideally, the number of threads would be a tunable performance parameter. As an example of this, I recently wrote some math-heavy code to transform the cells of a matrix by a particular transfer function. Each cell calculation was independent, so I parallelized them. The most efficient parallelization for my 6-core AMD processor turned out to be 6 threads per core. [1] The main exception I can think of is when running multiple applications in parallel that in turn parallelize their work.

exDM69 14y ago
> The optimal number of threads per core is workload and machine dependent, but rarely is that number 1 per hardware thread. This is exactly why I said it should be in the same order of magnitude, not exactly the same. Definitely not 250+ threads on a dual core. It might make sense to add a few more threads to keep the CPU busy if one or more of the threads will have to wait.
- vonmoltke 14y ago
  You're right, I misinterpreted "order of magnitude".
jedbrown 14y ago
This is dangerous misinformation. I'm sorry, but if your "math-heavy" code performs better when oversubscribed, then it is a massively suboptimal implementation. As a performance-oriented mathematical library developer, with colleagues at many of the major supercomputing centers and vendors, oversubscription is a hazard that we guard against on CPU architectures because it is always slower. Many supercomputing environments use a kernel that does not even support over-subscription. Note that GPUs are different in the sense that over-decomposition is necessary to cover latency, but even there, you generally get better performance at lower occupancy due to better register reuse. SMT CPU systems, most notably, Blue Gene/Q, often require use of multiple hardware threads per core to cover instruction latency and keep the prefetcher busy (due to allocation of outstanding requests). This is not over-subscription because those are bonified hardware threads.
- alexchamberlain 14y ago
  Can I pick your brains??? What happens during a cache miss? Does the thread block?
  - lallysingh 14y ago
    Other instructions in the same thread are executed (when they can be, e.g. all data's available). Or in another hyperthread on the same core.