4 ms·

> ...632 bytes... 5KB... How much does code size matter here? As long as the code has a good access pattern that maintains cache locality, is there any fundam

> ...632 bytes... 5KB... How much does code size matter here? As long as the code has a good access pattern that maintains cache locality, is there any fundamental difference between 632 bytes and 5KB? L1 cache sizes are generally somewhere around 16-64 KB these days, so it seems like there wouldn't be a big difference here. Or am I just totally off base?

foota 3y ago
I think the benefit is that when you start getting inlined code you can have multiple copies of it, so it can multiply out, and ideally you always want your code to be as small as possible so that you can have fewer instruction cache misses. So it's unlikely to matter if you're calling a single function in a loop whether the code is 632 bytes or 5KB (well, instruction decoding throughput aside I suppose), but when you're looking more broadly it might matter.
mlochbaum 3y ago
longsort appears in cosmopolitan libc, and possibly gets embedded in all the output executables? For most applications the requirements are much less restrictive. I'm working on sorting for interpreted programming languages; I see >20KB for each sort now and don't have a problem with that. For small arrays only a fraction of the code will be used. I still make some effort to reduce size, but if you're doing HPC work where sorting matters you can go much bigger with sorting networks for every size of something like that. icache is 32KB/core on every processor I've checked, although it's often reported weird. But it's fine for a hybrid sort targetting large arrays to exceed that because many components like partitioning will spend their time running on lots of data, so the time to load is relatively insignificant.
jart 3y ago
> L1 cache sizes are generally somewhere around 16-64 KB these days, so it seems like there wouldn't be a big difference here. Would you want to depend on a C library that claims all 64kb of your L1 cache for itself? Of course not. You'd want to use a library that stays out of the way, so that your code can be the one exploiting system resources.
vardump 3y ago
All else equal, larger L1C footprint code still microbenchmarks nicely, but is pretty often worse in a real system. Sometimes it's better to choose a slower routine with a smaller footprint. Nowadays memory bandwidth is arguably the biggest performance limiting factor.

foota 3y ago

I think the benefit is that when you start getting inlined code you can have multiple copies of it, so it can multiply out, and ideally you always want your code to be as small as possible so that you can have fewer instruction cache misses. So it's unlikely to matter if you're calling a single function in a loop whether the code is 632 bytes or 5KB (well, instruction decoding throughput aside I suppose), but when you're looking more broadly it might matter.

mlochbaum 3y ago

longsort appears in cosmopolitan libc, and possibly gets embedded in all the output executables? For most applications the requirements are much less restrictive. I'm working on sorting for interpreted programming languages; I see >20KB for each sort now and don't have a problem with that. For small arrays only a fraction of the code will be used. I still make some effort to reduce size, but if you're doing HPC work where sorting matters you can go much bigger with sorting networks for every size of something like that. icache is 32KB/core on every processor I've checked, although it's often reported weird. But it's fine for a hybrid sort targetting large arrays to exceed that because many components like partitioning will spend their time running on lots of data, so the time to load is relatively insignificant.

jart 3y ago

> L1 cache sizes are generally somewhere around 16-64 KB these days, so it seems like there wouldn't be a big difference here. Would you want to depend on a C library that claims all 64kb of your L1 cache for itself? Of course not. You'd want to use a library that stays out of the way, so that your code can be the one exploiting system resources.

vardump 3y ago

All else equal, larger L1C footprint code still microbenchmarks nicely, but is pretty often worse in a real system. Sometimes it's better to choose a slower routine with a smaller footprint. Nowadays memory bandwidth is arguably the biggest performance limiting factor.