3 ms·
Frankly, all these implementations are disappointingly slow. Daniel Lemire and I wrote simdjson to use SIMD to read JSON quickly - CSV is strictly easier than t
by glangdale 6y ago
Frankly, all these implementations are disappointingly slow. Daniel Lemire and I wrote simdjson to use SIMD to read JSON quickly - CSV is strictly easier than that. I made a start on simdcsv but got bored, but the same principles would apply. IMO this task should be doable at 1-2GB/s on a single core; it's not something that should really need multiple cores.
- liuliu 6y agoAgreed. But doing SIMD optimizations could quickly become 2-3 weeks project for me just for one platform. Adding NEON would be another 1 or 2 weeks. Need much more dedication I guess. Your work on simdjson is awesome! Thanks for saving the world so much unnecessary power consumptions :)
- glangdale 6y agoThanks for the kind words. I suppose this confirms my general suspicion that principles of simdjson need to be lifted out into a more general tool that allows people that don't live and breathe SIMD (strange that such people exist :-) ) to still get the benefit of these techniques for formats that aren't JSON.