I use h5py's datasets (which are organized hierarchically and stored in compressed chunks) to do basic filtering and then load a fraction of my data into memory

I use h5py's datasets (which are organized hierarchically and stored in compressed chunks) to do basic filtering and then load a fraction of my data into memory as numpy arrays.