4 ms·
Needing to think memory management turned out be a bit un-expected you might say. There were a few things that went well and few that were a bit unexpected. Th
by rdtsc 11y ago
Needing to think memory management turned out be a bit un-expected you might say. There were a few things that went well and few that were a bit unexpected.
Things that went well:
1) RAM got ridiculously cheap: 2x 8GB is now what $60-70? So it would seem, in that regard, yeah you shouldn't care about memory.
2) GC languages and runtimes. Believe or not there was heavy skepticism that GC runtime could be ever be a serious strategy in practice. It was an exotic technology that many thought would left for academia and fringe research area. But most popular frameworks and languages today are GC based.
Some things were a bit un-expected:
1) Memory access speed is fast but it didn't keep up with CPU speed. A cache miss will costs hundreds of nanoseconds. CPU speeds may have been doubling every 2 years or so, but memory access speeds have only been increasing by 10-20% or so.
In the 80s and 90s you just didn't care as much about cache hits and misses. But those are the forefront of performance optimization strategies. "Align your struct members","use structs of arrays instead of arrays of structs","think about cache lines and strides when you loop through your data arrays..." etc.
And because CPU speeds go so much faster, memory access also stands out on performance charts. So GC languages and runtimes can still seems slow and sluggish for applications that don't pay attention to memory.
2) Applications got more concurrent and servers are expected to handle a larger number of clients / connections / requests. That means each one needs a context, so it needs memory. So memory is again at the forefront. Accessing it and its latency, GC pause times are important.
3) Mobile happened. Mobile devices have proliferated so the needs for fast, small footprint applications never went away. Even as there are serves witih terabytes of RAM around, there are even more little arm cores and then everyone talks about IoT. That means langauges/runtimes which compile ahead of time will never be 100% displaced. Heck even on Android, ART (AoT copiling) is replacing JIT.
- asuffield 11y agoThere's a secondary dimension to this: You can still micro-optimise for memory and cpu performance. That's certainly a thing that is possible to do. Alternatively, you can buy more cpus and memory. What has changed in the past few years is that for many workloads, the cost of buying more cpus and memory is significantly lower than the cost of all the development work to optimise away the need. Hardware is cheap compared to developers, for many applications. There are still some applications where this is not the case, and in those cases people still care about memory management.
- dgoldstein 11y agoI think it comes down to scale. If you are running your application on 5 or 10 or maybe even 100 servers, buying a few more to handle the inefficiency makes some sense. But if you code needs to go on 1000s of servers or more, it's probably worth paying people to optimize it instead of buying more hardware.
- userbinator 11y agoAlign your struct members On modern CPUs, cache misses take so long and unaligned access is nearly free in comparison, so that padding can become detrimental. Of course ordering members so that they're all aligned is the best option, but if that can't be done, it's better to have misaligned and pack more in the same space than leave wasted padding.
- firethief 11y agoThe results of unaligned access can vary greatly, even among the x86 family. You can hit a large performance penalty if one of your fields straddles a cache line boundary, and $deity forbid you cross a page boundary on a Core2. Under the right circumstances packed structs can improve performance, but they come with a lot of technical debt since they can cause sporadic extreme slowdowns on certain common CPUs, and performance problems, aborts, or even data corruption on non-x86 platforms.
- poizan42 11y agoFor x86, sure. For other architectures you end up with larger code size for unaligned access, so it also becomes a tradeof between code/instruction and data cache usage.