4 ms·
Yeah, what it takes to really, truly stop parallelism is an unbreakable linear chain of dependencies, like how LZ decompression needs the previous unpacked bloc
by twotwotwo 8y ago
Yeah, what it takes to really, truly stop parallelism is an unbreakable linear chain of dependencies, like how LZ decompression needs the previous unpacked block to begin work on the next. (Not that LZ unpacking is maxing out people's CPUs these days.) That certainly exists sometimes but isn't nearly common enough to make me doubt the whole enterprise of parallelizing things.
The good news is that using all cores is one of the top things you can do to get work done sooner, and as such we probably will see investment happen. My work is pretty different from AAA games, but still parallelizing a big task is not infrequently lower hanging fruit than tuning the code for one unit of work. We've paid for the whole machine (or cluster) so we might as well use it.
Browsers are an interesting instance because they have a bunch of heterogeneous tasks with weird dependencies between them, and are huge, old codebases where you have to be careful adding threading, but they still manage to make real use of threads (e.g. running expensive compilers, decoding images, rendering, etc.).
Suspect you're saying ~this re: "foundational software", but as languages, tools, and libraries keep adapting to modern hardware, I think using parallelism confidently will become easier and we'll see more of it. Ecosystems change slowly but they do change.
ESR's claim about locality particularly seems iffy. A common task with very poor locality but useful parallel implementations is garbage collection: it thrashes your CPUs' caches, but thrashing all of them at once can still get it done sooner.
- SomeHacker44 8y agoModern GCs do a lot to improve performance by using temporal locality, not just the memory layout / access pattern locality that ESR was primarily interested in. I am no expert, but we now have multiple generation, multiple phase garbage collectors that can do useful work in parallel and have minimized the need for stop the world processing (one of ESR's intrinsically serial processes, at least from the executing program's perspective, as it may be an internally parallelizable process).
- twotwotwo 8y agoCan't tell if you're agreeing or disagreeing with this, but what I was trying to say is that full GCs happen (even when generational lets you delay them), thrash your caches to trace all live pointers in your heap, and still run faster on multiple cores. That seems to be in tension with the blog post's suggestion that you need locality for parallelism. Many recent GCs don't actually try to use all your cores because they run concurrent with your app and want to leave a lot of capacity for it, but Go 1.4's GC (the last non-concurrent version) used all the cores it could, and V8's Orinico does in some phases too.
- terrelln 8y agoI definitely agree that parallelism isn’t held back by fundamentally serial work. Even with LZ decompression, most algorithms entropy encode the LZ sequences, and these sequences can be decoded in parallel with the LZ decoder.
- MaulingMonkey 8y agoEven if you couldn't parallelize decompression (non-entropy encodings?) there's nothing stopping you from breaking data up into multiple streams and separately (de)compression those. For games, this might be decompressing different assets simultaneously.
- twotwotwo 8y agoFWIW, my aim was to give an illustrative example of what I meant by a chain of dependencies, and your garden-variety gunzip foo.gz has one. You can certainly come up with compression schemes or situations where not everything has to be serial.