3 ms·
I think maybe you possibly forgot what our argument was. I said the bottleneck is memory, and not processing/hashing the keys to match them to the symbol you wa
by 3cats-in-a-coat 3y ago
I think maybe you possibly forgot what our argument was. I said the bottleneck is memory, and not processing/hashing the keys to match them to the symbol you want to populate.
And you're currently telling me the bottleneck is memory.
I also said you don't need to parse a JSON object to a hashmap or a b-tree. The format suggests nothing of the sort. You can hash the key and fill it into a symbol slot in a tuple which... literally only takes the amount of RAM you need for the value, while the key is "free", because it just resolves to a pointer address.
Additionally, if you have a fixed tuple format, you can encode it as a JSON array, thus skipping the keys entirely. None of that is against JSON. You decide what you need and what you don't need. The keyvals are there when you need keyvals. No one is forcing you to use them where you don't need them at gunpoint.
I have a message format for a platform I'm working on, it has a JSON option, just for compatibility. It doesn't use objects at all, yet (but it DOES transfer object states). Nested arrays are astonishingly powerful on their own with the right mindset.
- riku_iki 3y ago> And you're currently telling me the bottleneck is memory. Not memory, but virtual pages implementation in linux, which is apparently single threaded and doesn't scale to high throughput. There was a patch to fix this, but it didn't make to mainline: https://lore.kernel.org/lkml/20180403133115.GA5501@dhcp22.suse.cz/T/ https://lore.kernel.org/lkml/20180403133115.GA5501@dhcp22.su...
- 3cats-in-a-coat 3y ago"High throughput" seems like an odd problem to have. You don't have to throw away pages and allocate new ones all the time. You can reuse a page.
- riku_iki 3y agoThat's not up to me. It is Linux virtual fs implemented that way