3 ms·
Well, many documents are several megabytes, but many are in the order of 100 bytes. I need a serialization scheme that scales up to large documents and down to
by atombender 7y ago
Well, many documents are several megabytes, but many are in the order of 100 bytes. I need a serialization scheme that scales up to large documents and down to tiny ones.
I can't imagine that the overhead of initializing an SQLite database from a small byte array in memory is that small, not to mention the overhead of maintaining the table schema.
Out of pure curiosity, I glanced at the Go bindings for SQLite, and there's no provision for initializing a database from a byte array, or accessing the raw underlying byte data of a live database. The C API supports implementing your own VFS for custom storage, but that's not supported by the Go bindings, and seems like a lot of work.
You're right about loading whole blobs; I was misremembering a little bit. The application in question already pares down the document keys in its queries to avoid sending everything. I'm in the middle of a research project into an alternative backend where the documents are stored as binary data, not JSON, and given a set of keys/keypaths, I want to do a little better than deserializing the whole blob.