It depends who your users are and what needs they have. Our users are mostly MATLAB/Python users, pretty tech-savvy, and being able to edit individual bits of t

It depends who your users are and what needs they have. Our users are mostly MATLAB/Python users, pretty tech-savvy, and being able to edit individual bits of the dataset with other applications or write their own code to analyse them is an often-used feature. It's rare we ever need the whole dataset - in fact, it's really great to be able to say to the user "don't send us your 100GB dataset: just go to the "acquisition" subfolder and send me the 10MB file called "oscilloscope.dat"". With HDF5 this is difficult enough that it's almost always easier to send 99.9% of useless data (i.e. the whole file) when all you want is a single array within it. If your users will rarely need to do this, you could just store the entire folder hierarchy in a .zip and access it using standard tools that most programming languages have. It's worth noting that the new Microsoft Office formats do exactly this - in their case, a bunch of XML files inside a .ZIP. (Rename a .docx to .zip and you'll see!). MATLAB has moved from their own custom binary format to HDF5, which is the lesser of two evils.