3 ms·
HDF5 does so much there's really not a single replacement. The most prevalent is probably JSON from what I've seen, but SQLite, (C/T)SVs are also fairly popular
by packetlost 3y ago
HDF5 does so much there's really not a single replacement. The most prevalent is probably JSON from what I've seen, but SQLite, (C/T)SVs are also fairly popular. There's a ton of application-specific serialization formats too.
- jjoonathan 3y agoNone of those (JSON, SQLite, CSV) are remotely appropriate for addressing HDF5's niche of storing large blocks of numbers in a manner that can approach their native space / perf characteristics.
- dagw 3y agoYea, HDF5 is great at what it is great at. The problems however come when people try to use it as a general data storage format and pseudo database.
- jjoonathan 3y agoRight, it does get abused, but I don't see anyone coming for its niche and I don't see its niche evaporating.
- chaxor 3y agoLol yeah, I always reach for paperback books now since they're a great modern replacement for my Kindle. On a more serious note, DuckDB is actually a pretty fantastic replacement for SQLite for some data analytics tasks. Hdf5 is still great for numerical data in this space though. I would welcome other options like the SQLite/DuckDB scenario for matrix data.