3 ms·
Personally, I've written a few multi-threading implementations at various places and my accumulated wisdom is that any data received by threads has to be passed
by pierrebai 6y ago
Personally, I've written a few multi-threading implementations at various places and my accumulated wisdom is that any data received by threads has to be passed by values or be constant. Anything else is a lot of risk for little value.
I've also come to the conclusion that async/future are a bad idea for multi-threading. They're fine for managing mono-thread asynchronous events, but using them as a poor man's thread pool is begging for trouble. Among the problems are:
- you do not know if the execution will be synchronous or not.
- you often don't know and have no control on how many threads will be spawned.
- it makes following the flow of data and execution much harder.
What I've come to rely on is the trinity of:
- a thread provider (non-committal name, we all know it's a thread pool behind the scene), which provides the threads of execution.
- a work item provider, which provides the input data to the algorithm.
- a worker provider, which provide workers that receive a work item, execute an algorithm and produces a result.
Then something gets executed when all three are united. For the curious, the reason to have a work item separate from worker is that it allows the worker to be long-lived and re-used and, most importantly, allows containing heavier data. For example, a cache. The work item are light and are just the necessary data needed to process one... work item.
I've also sometimes divided the work item into two parts: a work item and a context. The work item is the changing data to be processed while the context is the almost constant data, and can be shared. (For example, configuration data.)
- ncmncm 6y agoI generally prefer to code one active thread per process, with multiple processes when I want parallel work. They communicate through shared memory, generally single-writer enforced by the OS. So, yes, there can be equally rigorous system-level disciplines that substitute for language-imposed ones, and that can offer much better performance. But it does take a lot of experience to choose well.
- Mic92 6y agoThat sounds a lot like how the linux kernel manages it with its workqueues data structure, which gets consumed by kworker.