4 ms·
Ask HN: Nan for HDF-5 Data Format
My question is for those familiar with the HDF-5 data format. From what I can tell, it does not natively support the Not-A-Number (NaN) data type. For example if an array of data values was stored and some of them were NaN then these would be converted to nulls which can then be read in (i.e. misinterpreted) as zeros.
Does anyone know of a standardized or widely accepted way of storing NaN in HDF-5 files?
- jjgreen 5y agoNaN is not a data-type, it is a floating point number -- I've seen NaNs (and Infs) in HDF5 files (as floating point numbers). Some of the accompanying tools do not "do the right thing" with them though (h5diff sees NaNs as equal, for example).
- Eddy_Viscosity2 5y agoThanks, its actually good to know that I may be wrong in my premise and that I may in fact be able to properly store the NaNs. For context, I'm trying to convert Matlab data to H5 files.
- jjgreen 5y agoSounds like an issue with the Matlab interface to HDF5 then, good luck!
- porcoda 5y agoDoesn’t MATLAB have a built in h5write function to write out HDF5? There shouldn’t be much work at all to convert MATLAB data to H5. Last time I used it I had no issues with the resulting H5 files.
- Eddy_Viscosity2 5y agoThere is a fair bit of built-in H5 functionality in matlab, but the devil is in the details and I want to be sure I am storing the data without any loss of representation including NaN, inf, metadata and so on.
- porcoda 5y agoSure. For what it’s worth, MATLAB has used HDF for its own MAT file format since R2006b. See: https://www.mathworks.com/help/matlab/import_export/mat-file-versions.html https://www.mathworks.com/help/matlab/import_export/mat-file... I’m assuming if the MAT file contains what you expect and is a v7.3 file, it’s all accessible in HDF5 since it’s already an HDF5 file.