3 ms·
But that's my point. Why not just store bytes, they you don't have to worry about packing, it is always packed. If I understand correctly they are using 8 byte
by kevincox 5y ago
But that's my point. Why not just store bytes, they you don't have to worry about packing, it is always packed.
If I understand correctly they are using 8 bytes for 3 codepoints. They could instead use 3 bytes for 3 bytes. This would use significantly less memory and would rarely be less selective. (If that was a concern they could probably consider 4-grams instead of trigrams and still use less memory.)
This also doesn't precude the splitting into the first 2 characters and last 1.
- Scaevolus 5y agoOh, certainly, you could change the architecture to have byte-trigrams. These optimizations work without any large-scale architecture changes, making them safer to test and deploy, no data migrations or reindexing required!