5 ms·

Is sqlite a reasonable alternative? It seems like it would be resistant at least to the corruption issues the author complains of.

• ktpsns 7y ago
  It really depends on the use case. All use cases I actually know storing n-dimensional arrays, and this is something where HDF5 provides quite a neat interface which is identical to the in-memory representation in C/Fortran or numpy, e.g. This basically allows memory mapping and requires no transformation at reading/writing, which is important for data heavy applications (say anything in data science, high performance computing, or "big data" in general). SQLite got a defacto-standard for serializing relational data. Of course one can represent higher dimensional matrices as relational tables, but that isn't very efficient in storage and CPU time. SQLite may excel in storing any kind of meta data and structures.