4 ms·

> All the languages mentioned in the article are still around because it doesn’t matter how performant 99% of code is. It’s worth challenging this assumption a

by dahart 6y ago

> All the languages mentioned in the article are still around because it doesn’t matter how performant 99% of code is. It’s worth challenging this assumption and asking whether it’s really true, or whether this belief is old and outdated like the old languages. I’m certain that providers of compute infrastructure like Amazon, Microsoft, Google and others completely disagree that performance doesn’t matter. Letting slow code run literally costs them money because it prevents multiple people from being able to use the same hardware resource. Even on your own machine, slow processes hog resources that other processes could use. The reason my laptop fans keep spinning up in Windows while my machine is basically idle is due to hundreds of processes written by people who erroneously believe the performance of 99% of the code doesn’t matter. We’re also at a point where we’ve started to measure the energy efficiency and environmental impact of wasting compute cycles. So, performance always matters. > For the 1%, we can think about cache misses [...] but that has the enormous downside of making code hard to reason about. You’re agreeing with the author here. Maybe you missed the explicit point in the in the article that is calling for a language that makes basic cache awareness a first class citizen, precisely so that it’s easier to reason about? “This is exactly the type of problem that should be solved via the features of our programming languages, since writing these custom sorting functions is very tedious. What if a programming language would provide us with a structure that would act like an array of structs, but internally it would really behave like a struct of arrays? We could program in the typical object oriented way that is convenient for humans, while still enjoying in great performance due to playing nice with the hardware.” > You are also optimizing for today’s hardware, which means that it might not run that great on new hardware in 20 years. For the last 20 years, the need to be cache-aware has increased in importance. For the next 20 years, the gap between memory and compute will get larger still. There’s no scenario on the horizon where we don’t have to become more memory conscious as programmers or use more memory conscious languages and tools. There aren’t many performance tricks and concepts that worked 20 years ago and don’t work today. Using a structure of arrays (SoA) instead of an array of structures (AoS) has long been and will continue to be a good idea.

kaba0 6y ago
Feel free to profile a high-profile application. It is almost always a few hotspots that “suck”. There are rare cases where there is no one point that is slow, that’s a bad situation to find oneself in — it usually means that the base architecture/abstraction sucks and needs a big refactor. But even in so low-level code as the linux kernel, you see the use of linked lists all the time. Because it simply doesn’t matter if you are not in a hot loop and iterate slowly over 3 elements.
- dahart 6y ago
  FWIW, I profile intensive applications all the time, it's my job. You're right about the existence of hot spots, but that's not particularly relevant to my point or the author's. Just because one spot is hot doesn't mean you should rationalize using lazy or bad-practices programming on the rest. Just because you don't see a bottleneck somewhere when you profile doesn't mean it'll stay that way when other processes want the same resources at the same time. Knuth's famous quote was not intended to be a license to ignore using best practices in general for non-hot-spot code. Use of linked lists in the linux kernel is also not very relevant, and somewhat of a red herring. Kernel-level (as well as games & embedded programming) linked lists are frequently allocated and used very differently from typical high level language linked lists. Scripting languages are allocating individual linked list nodes by default. Linux uses linked lists for the memory allocator itself, for example, and allocates nodes at a lower level in blocks. There are no particularly good alternatives, and no great ways for the allocator itself to be cache aware, though that is ongoing research. The main perf problem with using linked lists in high level languages is not the fact that they cause cache-incoherent access, it's worse, it's the fact that they allocate and deallocate memory constantly which is much much slower. This is one reason why linked lists don't get used as much as arrays in high level languages, when arrays are a reasonable alternative. Anyway, it's somewhat of a straw man to bring up linked lists when the author's valid point was SoAs are better than AoSs from a cache perspective, and that our languages aren't yet doing much to help us code that way, but they could in the future if we decide to want them to.
  - kaba0 6y ago
    I apologize for my cocky reply. I just meant to point out that SoA vs AoS or an even slower data structure is not necessarily relevant for all programs. > The main perf problem with using linked lists in high level languages is not the fact that they cause cache-incoherent access, it's worse, it's the fact that they allocate and deallocate memory constantly which is much much slower. This is not true though; at least the JVM most definitely doesn’t do that, and I’m sure other state-of-the-art GCs neither, like V8. First of all, allocation is basically free, much faster than malloc and friends. It’s only a pointer bump. And the cost of deallocation is amortized/may not even happen. With plenty of memory available, GC doesn’t really happen, and for long-lived objects in case of a generational GC, they may not necessarily get checked at all after a time. So the usual linked list gets appended a few times is close to optimal, iteration has the same cost as in lower level languages with pointer chasing (usually a bit more because objects usually have headers that take quite some space, less of them potentially fitting into cache). It even has some advantage in that a modern compacting GC may move the objects close to each other.

by dahart 6y ago

kaba0 6y ago

Feel free to profile a high-profile application. It is almost always a few hotspots that “suck”. There are rare cases where there is no one point that is slow, that’s a bad situation to find oneself in — it usually means that the base architecture/abstraction sucks and needs a big refactor. But even in so low-level code as the linux kernel, you see the use of linked lists all the time. Because it simply doesn’t matter if you are not in a hot loop and iterate slowly over 3 elements.

dahart 6y ago
FWIW, I profile intensive applications all the time, it's my job. You're right about the existence of hot spots, but that's not particularly relevant to my point or the author's. Just because one spot is hot doesn't mean you should rationalize using lazy or bad-practices programming on the rest. Just because you don't see a bottleneck somewhere when you profile doesn't mean it'll stay that way when other processes want the same resources at the same time. Knuth's famous quote was not intended to be a license to ignore using best practices in general for non-hot-spot code. Use of linked lists in the linux kernel is also not very relevant, and somewhat of a red herring. Kernel-level (as well as games & embedded programming) linked lists are frequently allocated and used very differently from typical high level language linked lists. Scripting languages are allocating individual linked list nodes by default. Linux uses linked lists for the memory allocator itself, for example, and allocates nodes at a lower level in blocks. There are no particularly good alternatives, and no great ways for the allocator itself to be cache aware, though that is ongoing research. The main perf problem with using linked lists in high level languages is not the fact that they cause cache-incoherent access, it's worse, it's the fact that they allocate and deallocate memory constantly which is much much slower. This is one reason why linked lists don't get used as much as arrays in high level languages, when arrays are a reasonable alternative. Anyway, it's somewhat of a straw man to bring up linked lists when the author's valid point was SoAs are better than AoSs from a cache perspective, and that our languages aren't yet doing much to help us code that way, but they could in the future if we decide to want them to.
- kaba0 6y ago
  I apologize for my cocky reply. I just meant to point out that SoA vs AoS or an even slower data structure is not necessarily relevant for all programs. > The main perf problem with using linked lists in high level languages is not the fact that they cause cache-incoherent access, it's worse, it's the fact that they allocate and deallocate memory constantly which is much much slower. This is not true though; at least the JVM most definitely doesn’t do that, and I’m sure other state-of-the-art GCs neither, like V8. First of all, allocation is basically free, much faster than malloc and friends. It’s only a pointer bump. And the cost of deallocation is amortized/may not even happen. With plenty of memory available, GC doesn’t really happen, and for long-lived objects in case of a generational GC, they may not necessarily get checked at all after a time. So the usual linked list gets appended a few times is close to optimal, iteration has the same cost as in lower level languages with pointer chasing (usually a bit more because objects usually have headers that take quite some space, less of them potentially fitting into cache). It even has some advantage in that a modern compacting GC may move the objects close to each other.