3 ms·
> The fact that our reference counters are atomic is an additional problem that makes things even more complex for the processor. Although using atomic instruct
by bob1029 5y ago
> The fact that our reference counters are atomic is an additional problem that makes things even more complex for the processor. Although using atomic instructions is often referred to as “lockless programming”, this is slightly misleading – in fact, atomic operations require some locking to happen at the hardware level. This locking is very fine-grained and cheap as long as there is no congestion, but as usual with locking, you may expect very poor performance if many things try to fight for the same lock at the same time. And it is of course much worse if those “things” are whole CPUs and not just single cores that are close to each other.
Contention is the devil and you can never hide from it if you bring it into your life. The fastest "contention avoidance hack" would be the CAS operation as mentioned above, which still performs approximately 2 orders of magnitude slower than a single threaded operation would in the most adverse scenarios.
If you aren't sure of which problems actually "fit" on a modern x86 thread these days, rest assured that many developers in fintech have been able to squeeze entire financial exchanges onto one:
https://lmax-exchange.github.io/disruptor/disruptor.html https://lmax-exchange.github.io/disruptor/disruptor.html
For most scenarios, you really don't need all those damn cores. They are almost certainly getting in the way of you solving your problems.
- cecilpl2 5y agoGames are a prime example of systems which must take advantage of all possible hardware resources.
- bob1029 5y agoThe fact that we believe this is probably part of the problem with game development today.
- dagmx 5y agoGames similarly struggle with multi core/ccx designed chips. Battlefield 4 was famously terrible on Ryzen when AMD first released it.
- littlecranky67 5y agoTo be fair, Ryzen mostly underperformed when it was released which was not AMDs fault. When for years engineers used Intel as their benchmarking and testing machines and were tweaking all sorts of parameters to measure against Intel-only, you inevitable end up with an optimized implementation that resembles the hardware details of the Chip in some kind. Any new architecture would perform badly, except if it is a perfect clone of the original testing hardware.
- dagmx 5y agoI wasn't implying it was AMDs fault, but the idea that the games were optimized for Intel doesn't hold true IMHO because they were also built for consoles that were AMD. The majority of the performance was down to not accounting for CCX scheduling
- PaulDavisThe1st 5y agoAnother interesting example is audio synthesis. It is interesting because there are at least two models for audio synthesis: single sample processing (as used in, for example, VCV Rack) and block structured processing (as used by every plugin API and thus every DAW-that-runs-plugins). But it gets even more interesting because the degree of potential parallelism is entirely controlled by the signal routing pathways chosen by the user. If the software is processing N independent signal pathways that do not meet until a final output stage, you have a high (well, "N") level of potential parallelism, and if N can expand to a high value then you will always be able to benefit from more cores. However, if the signal pathway is more convoluted (no pun intended for my audio nerd crew), then not only does the degree of parallelism decrease but the issue of single-sample vs. block structured can become much more important. In the single sample case, the cost of computing a single sample has (relatively) enormous fixed overhead if the code has any serious level of modularity, but is still very cheap compared to inter-thread and inter-core synchronization. Spreading the computation across cores will frequently, but not always, slow things down quite dramatically. By contrast, block structured processing has a much lower per-sample cost, because the function call tree is invoked only once for an entire block of samples, and so the relative cost of synchronization decreases. This sounds like a slam dunk for block structured processing. Problem is, there are things you cannot do correctly with block structured processing, and so there is always a place for single sample processing. However, the potential for parallelization and the level at which it should take place differs significantly between the two processing models, which opens to door to some highly questionable design. The short version of this is that audio synthesis in the abstract can always expand to use any number of cores, particularly for cpu-intensive synthesis like physical modelling, but that in the presence of single-sample processing, the cost of synchronization may completely negate the benefits of parallelism.
- spacechild1 5y agoGreat write up! What I like about Pd is that you can freely reblock and resample any subpatch. Want some section with single-sample-feedback? Just put a [block~ 1]. You can also increase the blocksize. Usually, this is done for upsampling and FFT processing. Finally, reblocking can be nested, meaning that you can reblock to 1024 samples and inside have another subpatch running at 1 sample blocksize. SuperCollider, on the other hand, has a fixed global blocksize and samplerate, which I think is one of its biggest limitations. (Needless to say, there are many things it does better than Pd!) --- In the last few days I have been experimenting with adding multi-threading support to Pd (https://github.com/Spacechild1/pure-data/tree/multi-threading https://github.com/Spacechild1/pure-data/tree/multi-threadin...). With the usual blocksize of 64 sample, you can definitely observe the scheduling overhead in the CPU meter. If you have a few (heavy-weight) subpatches running in parallel, the overhead is neglible. But for [clone] with a high number of (light-weight) copies, the overhead becomes rather noticable. In my quick tests, reblocking to 256 samples already reduces the overhead significantly, at the cost of increased latency, of course. --- Also, in my plugin host for Pd/Supercollider (https://git.iem.at/pd/vstplugin/ https://git.iem.at/pd/vstplugin/) I have a multi-threading and bridging/sandboxing option. If the plugin itself is rather lightweight and the blocksize is small, the scheduling overhead becomes quite noticable. In Pd you can just put [vstplugin~] in a subpatch + [block~]. For the SuperCollider version I have added a "reblock" argument to process the plugin at a higher blocksize, at the cost of increased latency.
- hinkley 5y agoI believe Rust is pulling on the right thread here but I have no idea how this will look in fifteen years. At some point we will need to be able to tell that two threads don’t contend and thus can be “farther away” from each other, but these threads are constantly communicating and need to use the same complex and/or memory banks.
- dagmx 5y agoI think it could be something built upon lifetime analysis. I think a sufficiently complex compiler could use the data available already today to infer thread affinity and other rules.
- hinkley 5y agoBasically what I was thinking. In the near term it doesn't even have to be that precise to have a benefit (Rust builds on escape analysis that helped with distributed garbage collection, even when it wasn't that exhaustive). Even ranking threads by suspected degree of interaction is a lot of data. These definitely don't talk. Offload. These definitely talk. Colocate. These probably talk. Bin pack.
- cjfd 5y ago"For most scenarios, you really don't need all those damn cores. They are almost certainly getting in the way of you solving your problems." This rings true. I still think the most promising multi-threading strategy is to, by default, keep everything on one core and if you find some expensive and more or less independent workload to put that in another thread. This is the old-fashioned way though. All of the cool kids will push for thread pools where every workload will jump from core to core and back again.... This potentially can require a lot of locking, though, which also is not good for performance.
- zmmmmm 5y agoIt's an interesting reflection then on the Actor paradigm where you are encouraged to create large numbers of fine grained tasks and toss them into a giant thread pool blender because light weight threads are "free" now. It would be quite interesting to see the consequences of thread affinity loss in that setting.