PGO is great and definitely helps but only in the micro level and not at the macro, algorithmic level. To say it another way, once you've chosen your data stru

PGO is great and definitely helps but only in the micro level and not at the macro, algorithmic level. To say it another way, once you've chosen your data structures it's already too late to take advantage of memory layout because either the language forces a specific layout(C/C++/Rust) or you have pointers which can take up a bunch of extra space. Take a toy example of adding all the x values of a vec3 together, usually you'd represent that as: struct Vec3 { float x, y, z; } However that gives you a memory layout of: "xyzxyzxyzxyz...." If instead you flip that from AoS(array of structures) to SoA(structures of array) you get: "xxxxyyyyzzzz..." and now all your x values are adjacent and nicely cache local. If instead we were talking about other objects instead of x/y/z now each one of those becomes a potential cache miss in the first example but in the second they'd already be fetched by the prefetcher and hot in the cache. Those type of optimizations are things that compilers aren't good at because they have impact on how the data is actually defined, abstractions being leaky and all that.