7 ms·
I've always been a bit curious as to what use cases people have for HDF5 files that involve frequent changing of data. I wonder how read/write performance scal
by xen0 6y ago
I've always been a bit curious as to what use cases people have for HDF5 files that involve frequent changing of data.
I wonder how read/write performance scales with the _size_ of the dataset in question, in particular, the number of chunks, and hence the number of virtual dataset mappings. I never really explored how well the HDF5 library handles obscene numbers of VDS mappings, and one per chunk on a large dataset is probably more than the HDF Group had in mind.