6 ms·

enkiTS: A C and C++ Task Scheduler for creating parallel programs

87 points by lehi 6y ago

posharma 6y ago
1. How is this different from Intel TBB? 2. Does it have support for pipeline parallelism?
- stephc_int13 6y ago
  I checked Intel TBB and I have not found a way to see the source code, the licence or even something to download and test without buying or downloading the demo of a commercial product. So far I only had bad experiences with Intel libraries, huge, opaque, bloated and difficult or impossible to use on different architectures. They usually have very high performance code under the hood so this is a shame.
  - jcelerier 6y ago
    https://github.com/intel/tbb https://github.com/intel/tbb
    - marco_craveiro 6y ago
      Whats the difference between "oneTBB" and regular TBB, do you know? Cheers
      - jcelerier 6y ago
        oneTBB is the continuation of TBB. Intel took all their OSS projects and put them under the "OneAPI" umbrella last year: https://www.oneapi.com/open-source/ https://www.oneapi.com/open-source/ MKL, etc..
  - brandmeyer 6y ago
    `apt-get install libtbb-dev libtbb-doc` `apt-get source libtbb-dev`
- jcelerier 6y ago
  Same questions for https://github.com/taskflow/taskflow https://github.com/taskflow/taskflow
Const-me 6y ago
Windows includes reasonably good thread pools as a part of the OS, the API is SubmitThreadpoolWork. macOS too, the API is dispatch_async.
- Const-me 6y ago
  For completeness, modern C and C++ compilers ship yet another thread pool, in OpenMP language extension. I sometimes write my own threading support for unusual use cases, but that’s hard to do correctly even with experience. For this reason, in 80% of cases I pick an off-the-shelf implementation instead of doing my own.
exDM69 6y ago
I hate to voice a negative opinion but there are multiple red flags in this project. There are lots of user space spinlocks in the code, for the scheduler and the mcsp queue. Questionable use of atomics and memory barrier orderings. NIH futex using atomics and a WINAPI semaphore for backoff. No cpu_relax hints. Not great test coverage. This all could be achieved using mutexes and condition variables and it would have equally good best case performance (when using a futex based mutex), and it would be easier to test for reliability and it would have more predictable performance characteristics under high contention. To be fair, this seems to be targeted at games for Windows and Xbox and I'm not sure how good the concurrency primitives (mutex, condvar) are for those platforms. Maybe it makes sense to spin wait on a gaming console. Or maybe the typical intended use case never hits the high contention cases or it doesn't matter if a thread is pre-empted when holding a spin lock. I am not claiming this code is buggy, but 15 minutes of reading the code and tests does not convince me that it is not. I've written and tested a lot of this kind of code and it is not straightforward. Sometimes I hit a problem after running a stress test on all CPU cores for 10 minutes. The tests/examples in this project run 10, 100 or 1000 times in a loop. That is inadequate to hit the pathological thread scheduling to to trigger race conditions with a high probability.
- gpderetta 6y ago
  I think that this days windows has keyed events which are the equivalent of a futex.
- flohofwoe 6y ago
  On the other hand, the project is in continuous development since 2015, created by a (game-)industry veteran and used in a (or at least one) released game. IME such projects born out of real-world requirements are much more robust (and easier to integrate) than the typical 'academic' or (worse) 'Google-scale' framework.
  - exDM69 6y ago
    Yeah, under the kind of typical workload I imagine this goes through in a gamedev environment, I don't expect it to ever hit the contention cases. But it could as well use mutex/condvar because it is unlikely to be contended. If there is a bug in the atomics code, it will manifest as sporadic performance. They are not easy to debug.
eps 6y ago
This appears to be basically a callback scheduler with support for callback threading and sequencing. A fairly simple thing to code. The catch here is that if you ever end up needing something like this, it's far more prudent to write your own version than to learn someone else's solution.

87 points by lehi 6y ago

posharma 6y ago

1. How is this different from Intel TBB? 2. Does it have support for pipeline parallelism?

stephc_int13 6y ago
I checked Intel TBB and I have not found a way to see the source code, the licence or even something to download and test without buying or downloading the demo of a commercial product. So far I only had bad experiences with Intel libraries, huge, opaque, bloated and difficult or impossible to use on different architectures. They usually have very high performance code under the hood so this is a shame.
- jcelerier 6y ago
  https://github.com/intel/tbb https://github.com/intel/tbb
  - marco_craveiro 6y ago
    Whats the difference between "oneTBB" and regular TBB, do you know? Cheers
    - jcelerier 6y ago
      oneTBB is the continuation of TBB. Intel took all their OSS projects and put them under the "OneAPI" umbrella last year: https://www.oneapi.com/open-source/ https://www.oneapi.com/open-source/ MKL, etc..
- brandmeyer 6y ago
  `apt-get install libtbb-dev libtbb-doc` `apt-get source libtbb-dev`
jcelerier 6y ago
Same questions for https://github.com/taskflow/taskflow https://github.com/taskflow/taskflow

Const-me 6y ago

Windows includes reasonably good thread pools as a part of the OS, the API is SubmitThreadpoolWork. macOS too, the API is dispatch_async.

Const-me 6y ago
For completeness, modern C and C++ compilers ship yet another thread pool, in OpenMP language extension. I sometimes write my own threading support for unusual use cases, but that’s hard to do correctly even with experience. For this reason, in 80% of cases I pick an off-the-shelf implementation instead of doing my own.

exDM69 6y ago

I hate to voice a negative opinion but there are multiple red flags in this project. There are lots of user space spinlocks in the code, for the scheduler and the mcsp queue. Questionable use of atomics and memory barrier orderings. NIH futex using atomics and a WINAPI semaphore for backoff. No cpu_relax hints. Not great test coverage. This all could be achieved using mutexes and condition variables and it would have equally good best case performance (when using a futex based mutex), and it would be easier to test for reliability and it would have more predictable performance characteristics under high contention. To be fair, this seems to be targeted at games for Windows and Xbox and I'm not sure how good the concurrency primitives (mutex, condvar) are for those platforms. Maybe it makes sense to spin wait on a gaming console. Or maybe the typical intended use case never hits the high contention cases or it doesn't matter if a thread is pre-empted when holding a spin lock. I am not claiming this code is buggy, but 15 minutes of reading the code and tests does not convince me that it is not. I've written and tested a lot of this kind of code and it is not straightforward. Sometimes I hit a problem after running a stress test on all CPU cores for 10 minutes. The tests/examples in this project run 10, 100 or 1000 times in a loop. That is inadequate to hit the pathological thread scheduling to to trigger race conditions with a high probability.

gpderetta 6y ago
I think that this days windows has keyed events which are the equivalent of a futex.
flohofwoe 6y ago
On the other hand, the project is in continuous development since 2015, created by a (game-)industry veteran and used in a (or at least one) released game. IME such projects born out of real-world requirements are much more robust (and easier to integrate) than the typical 'academic' or (worse) 'Google-scale' framework.
- exDM69 6y ago
  Yeah, under the kind of typical workload I imagine this goes through in a gamedev environment, I don't expect it to ever hit the contention cases. But it could as well use mutex/condvar because it is unlikely to be contended. If there is a bug in the atomics code, it will manifest as sporadic performance. They are not easy to debug.

eps 6y ago

This appears to be basically a callback scheduler with support for callback threading and sequencing. A fairly simple thing to code. The catch here is that if you ever end up needing something like this, it's far more prudent to write your own version than to learn someone else's solution.