5 ms·

Without knowing anything about this, I wonder if standards are trying to work at the wrong level of abstraction. Like worrying about whether to use GIF or PNG t

Without knowing anything about this, I wonder if standards are trying to work at the wrong level of abstraction. Like worrying about whether to use GIF or PNG to compress map images, when really it would be better to standardize the underlying data like OpenStreetMap. A better goal might be to decide on a single container format for discrete cosine transform (DCT) data in compressed images like how JPEG works. We probably need a multidimensional lossy compression scheme for 2D images, 3D volumes/video, and 4+D metadata involving multiple spectrums, exposures, etc. I'd probably start with a multidimensional DCT with a way to specify the coefficients/weights of each layer and metadata (optionally losslessly): https://en.wikipedia.org/wiki/Discrete_cosine_transform#Multidimensional_DCTs https://en.wikipedia.org/wiki/Discrete_cosine_transform#Mult... Looks like that's built into MATLAB with dct(x,n,dim): https://www.mathworks.com/help/signal/ref/dct.html https://www.mathworks.com/help/signal/ref/dct.html https://www.mathworks.com/help/images/ref/blockproc.html https://www.mathworks.com/help/images/ref/blockproc.html (coefficients) https://www.mathworks.com/help/images/discrete-cosine-transform.html https://www.mathworks.com/help/images/discrete-cosine-transf... (2D example) Then store the data in zipped JSON or BSON, or maybe TAR then ZIP. Then I'd write an importer and exporter from/to all of the other formats, with a single 0-1 quality argument or maybe an array of qualities for each layer, with a way to specify coefficients or load them from files. Once a known-good implementation worked, I'd add additional compression schemes that can compress across dimensions, so that a level of zoom only has to save a diff from a previous layer at a slightly different scale or focus (like a 3+D MPEG format, this is probably the standard that we're currently missing, does anyone know of one?). This all seems fairly straightforward though, so I wonder if proprietary software patents are the real blocker..

dekhn 3y ago
The OME-zarr creators are pretty experienced in this field and have already done the work you're describing. Let's not create another standard after OME-zarr.
- zackmorris 3y ago
  Hey thanks, I tend to agree. Although if I read this right, OME-zarr punted on lossy compression (which is all that really matters) so IMHO it doesn't have much use in the real world yet even though people evidently put a lot of work into it: https://github.com/ome/ome-zarr-py https://github.com/ome/ome-zarr-py https://ngff.openmicroscopy.org/latest/ https://ngff.openmicroscopy.org/latest/ https://zarr.readthedocs.io/en/v2.1.0/api/codecs.html https://zarr.readthedocs.io/en/v2.1.0/api/codecs.html https://www.researchgate.net/publication/356622607_OME-NGFF_a_next-generation_file_format_for_expanding_bioimaging_data-access_strategies#read https://www.researchgate.net/publication/356622607_OME-NGFF_... https://www.biorxiv.org/content/10.1101/2023.02.17.528834v1.full.pdf https://www.biorxiv.org/content/10.1101/2023.02.17.528834v1.... i.munro (Ian Munro) August 15, 2023, 8:54am On the original point about size I suspect your answer is compression. I have found that a lot of image formats use lossy compression. I’m not sure about CZI specifically. AFAIK ome-zarr (sensibly IMO) uses only lossless compression. Best Ian and: sebi06 (Sebi06) August 15, 2023, 1:42pm Hi all, for CZI we offer currentyl two compression methods: JPG-XR ZSTD So we have lossless and lossy options depending on your needs. All of them are implemented in our C++, #NET or Python APIs and also supported from BioFormats. Looks like CZI is a competitor: https://www.zeiss.com/microscopy/en/products/software/zeiss-zen/czi-image-file-format.html https://www.zeiss.com/microscopy/en/products/software/zeiss-... https://github.com/cgohlke/czifile https://github.com/cgohlke/czifile http://www.physics.hmc.edu/~gerbode/wppriv/wp-content/uploads/2012/06/DS_ZISRAW-FileFormat_Rel_ZEN2011.pdf http://www.physics.hmc.edu/~gerbode/wppriv/wp-content/upload... CZI is basically JPEG compression and is probably missing the multidimensional compressor which is the secret sauce missing from the various formats. It also mentions licensing and legal so is maybe lost in the weeds and not open source. Without more info, it feels like this is still an open problem. My interest in this is for video games, since so many are multiple GB in size but are also missing this multidimensional compressor. If we had it, I'd guess a 10:1 to 100:1 reduction in size over what we have now, with no perceptual loss in quality.
  - rrock 3y ago
    Lossy compression is a bad idea for scientific images. For instance, we often need to understand the statistics of photon detection events in background regions. That’s one of the first things to get tossed.

dekhn 3y ago

The OME-zarr creators are pretty experienced in this field and have already done the work you're describing. Let's not create another standard after OME-zarr.

zackmorris 3y ago
Hey thanks, I tend to agree. Although if I read this right, OME-zarr punted on lossy compression (which is all that really matters) so IMHO it doesn't have much use in the real world yet even though people evidently put a lot of work into it: https://github.com/ome/ome-zarr-py https://github.com/ome/ome-zarr-py https://ngff.openmicroscopy.org/latest/ https://ngff.openmicroscopy.org/latest/ https://zarr.readthedocs.io/en/v2.1.0/api/codecs.html https://zarr.readthedocs.io/en/v2.1.0/api/codecs.html https://www.researchgate.net/publication/356622607_OME-NGFF_a_next-generation_file_format_for_expanding_bioimaging_data-access_strategies#read https://www.researchgate.net/publication/356622607_OME-NGFF_... https://www.biorxiv.org/content/10.1101/2023.02.17.528834v1.full.pdf https://www.biorxiv.org/content/10.1101/2023.02.17.528834v1.... i.munro (Ian Munro) August 15, 2023, 8:54am On the original point about size I suspect your answer is compression. I have found that a lot of image formats use lossy compression. I’m not sure about CZI specifically. AFAIK ome-zarr (sensibly IMO) uses only lossless compression. Best Ian and: sebi06 (Sebi06) August 15, 2023, 1:42pm Hi all, for CZI we offer currentyl two compression methods: JPG-XR ZSTD So we have lossless and lossy options depending on your needs. All of them are implemented in our C++, #NET or Python APIs and also supported from BioFormats. Looks like CZI is a competitor: https://www.zeiss.com/microscopy/en/products/software/zeiss-zen/czi-image-file-format.html https://www.zeiss.com/microscopy/en/products/software/zeiss-... https://github.com/cgohlke/czifile https://github.com/cgohlke/czifile http://www.physics.hmc.edu/~gerbode/wppriv/wp-content/uploads/2012/06/DS_ZISRAW-FileFormat_Rel_ZEN2011.pdf http://www.physics.hmc.edu/~gerbode/wppriv/wp-content/upload... CZI is basically JPEG compression and is probably missing the multidimensional compressor which is the secret sauce missing from the various formats. It also mentions licensing and legal so is maybe lost in the weeds and not open source. Without more info, it feels like this is still an open problem. My interest in this is for video games, since so many are multiple GB in size but are also missing this multidimensional compressor. If we had it, I'd guess a 10:1 to 100:1 reduction in size over what we have now, with no perceptual loss in quality.
- rrock 3y ago
  Lossy compression is a bad idea for scientific images. For instance, we often need to understand the statistics of photon detection events in background regions. That’s one of the first things to get tossed.