3 ms·
I had a similar idea but my target was a Python application, that was parsing text to create a complex in-memory object tree with tens of thousands of objects.
by Erwin 9y ago
I had a similar idea but my target was a Python application, that was parsing text to create a complex in-memory object tree with tens of thousands of objects.
The idea was that the individual elements were not always fully accessed when the application ran, so if I created them on demand from such a dense memory-mappable dump I could persist that instead of parsing every time.
The overhead of creating Python object was too high for that. But if you are using Python and are deserializing read-only dictionaries, http://discodb.readthedocs.io/en/latest/ http://discodb.readthedocs.io/en/latest/ does a subset of that -- if your app e.g. reads in 100,000 translations from a JSON file, Disco's serialization will let you just mmap them.