2 ms·
This is really sick. Love it! Although I would really love to see a more zero-parse binary format. At the end of the day simd, is just making markers in the or
by nojvek 7y ago
This is really sick. Love it!
Although I would really love to see a more zero-parse binary format. At the end of the day simd, is just making markers in the original file of where an array begins, object begins, each of the object key value ranges e.t.c i.e the find marks -> build tape algorithm.
This could be achieved by having length prefixed values, kinda like flat buffers which is zero parse time, and much compact than json. https://google.github.io/flatbuffers/ https://google.github.io/flatbuffers/
I just wish json had a more universal alternative binary format which was insanely fast and compact to work with.
Protobufs are nice but parsing large files is still a pain and consumes a ton of memory. Ideally a format that could simply be lazy mmap-ed would be desirable.
---
> The work is supported by the Natural Sciences and Engineering Research Council of Canada under grant number RGPIN-2017-03910.
Well, thank you Canada!