6 ms·
A 5x reduction in RAM usage with Zoekt memory optimizations
- cbm-vic-20 5y agoNot directly related to the linked content, but "N times less" never really made that much sense to me. In this case, I'm guessing "5x reduction" means "80% less"? Or "20% of the previous usage"?
- tyingq 5y ago"We went from 1400KB of RAM per repo to 310KB with no measurable latency changes." So, not exactly, but close. ~22% of previous usage.
- deleted 5y ago[deleted]
- nijaru 5y agoYes that is correct. It's not always the most intuitive language. 5 times less than 20 would be 4 or (20 * 1/5), the same as 20 is 5 times more than 4.
- ksec 5y agoYes it is the marketing speak of 80% less. And generally speaking works much better than percentage.
- nytgop77 5y agoI saw advertisements "now 20% cheaper!", while price change (original->new) was 30€->25€. Marketing always finds a way.. 5/30=16.6%, 5/25=20%
- DistractionRect 5y agoN times less usually just means it uses 1/N. So this would mean it uses only 20% of the memory it used originally.
- rusk 5y agoYeah it doesn’t make sense as the x in 5x I would have thought repetesents a multiplication rather than (divide) It would make more sense for me to say a fifth, or 1/5
- MichaelBurge 5y agolog(5x) = log(5) + log(x) log(x/5) = -log(5) + log(x) So the difference is just a sign change on a log scale. And it's natural to use a log scale when working with percentage changes.
- rusk 5y agoNatural for who? Perhaps better expressed as “5 fold”.
- kevincox 5y agoI'm surprised that they are storing Unicode characters instead of bytes. For example Rust's regex library works on bytes and unicode patterns are compiled into byte patterns which means that it doesn't need to worry about unicode and variable length characters when matching. You'd think for code where the vast majority is ASCII this would be a huge improvement. I guess the downside is that searches for emoji and other "long" characters would need to look up more index entries. However I would expect that due to the rarity of that it would be beneficial overall.
- Scaevolus 5y agoThe source code itself is stored as UTF-8, but the trigrams were represented as Unicode codepoints in the index. The last optimization packs the ASCII trigrams for efficiency.
- kevincox 5y agoBut that's my point. Why not just store bytes, they you don't have to worry about packing, it is always packed. If I understand correctly they are using 8 bytes for 3 codepoints. They could instead use 3 bytes for 3 bytes. This would use significantly less memory and would rarely be less selective. (If that was a concern they could probably consider 4-grams instead of trigrams and still use less memory.) This also doesn't precude the splitting into the first 2 characters and last 1.
- Scaevolus 5y agoOh, certainly, you could change the architecture to have byte-trigrams. These optimizations work without any large-scale architecture changes, making them safer to test and deploy, no data migrations or reindexing required!
- roca 5y agoI think the real question is: why not use trigrams of bytes (from the UTF8 representation) instead of Unicode codepoints? Seems like that would simplify everything. If they have data showing that doesn't work as well in practice, I'd love to know what it is, because I can't think of a reason off the top of my head why it wouldn't work well. Obviously it could only be worse in non-ASCII situations, which are already super-rare.
- Scaevolus 5y agoAuthor here, ask me anything! :-)
- cinntaile 5y agoIf anyone else but me wonders where the name comes from, Zoekt means Seek. Context: "Zoekt, en gij zult spinazie eten" - Jan Eertink ("seek, and ye shall eat spinach" - My primary school teacher) https://github.com/google/zoekt https://github.com/google/zoekt
- beff_jesos 5y agoIs this available on the self hosted version as well now? I am getting RAM issues on the AWS hosted cluster.
- Scaevolus 5y agoYes, all these optimizations have been in the last few releases. Reducing RAM allocations in the cluster can be tricky. Zoekt memory maps its index files and will absorb all of RAM as cache, and the standard cadvisor metrics will simply declare that your container is using almost all of its RAM allocation. Looking at go's heap statistics for the process will give you a better idea of how much overhead you need.
- therealmarv 5y agoRelated: The last time I've checked a standard Ubuntu does not have RAM compression (ZRAM) enabled by default (unlike current Windows and Mac which have that for years). It helps a lot with programs like browsers.
- klysm 5y agoI understand why it’s opt in though, not a clear win in all cases.
- smcnally 5y agoYour point about opt-in makes me wonder if my smaller VMs and VPSs would benefit. This recent one-pager covers the ‘compression can help but not always’ brackets you’ve noted. https://benad.me/blog/2021/02/23/ram-compression-on-linux/ https://benad.me/blog/2021/02/23/ram-compression-on-linux/ > Still, unless you frequently use the swap device on your Linux system, it may not be worthwhile to reserve a large amount of memory to a zram device. I know how to view current swap usage and will check if there’s historical usage in syslog or the like.
- klysm 5y agoAs always with performance things, you have to benchmark. But that’s always easier said than done. With the insane complexity of memory systems in modern CPUs anything can happen.
- FeepingCreature 5y agozram is amazing. I love it. Makes my 16GB machine feel like a 24GB, no noticeable downsides. I've totally turned off regular swap, even.