4 ms·

The great thing about TIF was it's extensibility. Flexible (data could be stored as tiles or in stripes), multiple compression options etc. Well documented spe

The great thing about TIF was it's extensibility. Flexible (data could be stored as tiles or in stripes), multiple compression options etc. Well documented spec, easy to bolt on extras either as public tags - GeoTIFFs added projection metadata - or private, for your own needs. Back in the day, to improve a desktop application's performance I found it was simple to create a custom reader and writer to handle cases where tiles were completely one single colour removing the need to decompress at run time. Thank you TIFf!

sllabres 1y ago
And the (early) availability of well made library, LibTIFF by Sam Leffler. I used it extensively from 1995 on, but only found out that according to Wikipedia is dates back to 1988!
flomo 1y ago
Perhaps the greatest thing about TIFF, but also the most horrible things, and probably why TIFF is mostly historical. It was so extendable that no two programs ever accepted the exact same TIFF extensions. (omitting the war story) edit: forgot about byte order...
- yread 1y ago
  But most of these variations were part of the spec (endianness with II or MM, later magic 43 for bigTIFF 64bit extension). I work with tiff and tiff-derived formats in digital microscopy where its very much not historical. And the alternatives (DICOM supp 145, vendor-specific garbage ... and thats it) are worse. I quite like the format, the only thing I would change is to have the option not to store directory information in a linked list spread throughout the file but in a simple array. Duplicate it at the beginning and end of the file and you've got resilience too (important in the age of floppies)
- piltboy 1y ago
  TIFF is still very much alive in certain circles, see for example https://cogeo.org/ https://cogeo.org/ The format is basically a TIFF file with attached georeferencing information and with the data organized by geographical sector, enabling fast downloads of regional subsets.
  - deleted 1y ago
    [deleted]
  - geokon 1y ago
    I'm not super knowledgeable about this stuff.. but out of curiosity, what advantage does it have over NetCDF? I wrote a program processing GeoTIFF data. When I had started this project I chose GeoTIFF mostly b/c i wanted something simple. And I could load them in to Java's BufferImage class and manipulate them that way. But it seems all the pros exclusively use NetCDF and GeoTiffs are for noobs (working with atmospheric science data here) GeoTIFF does extend "images" to cover more usecases, but a lot of stuff doesn't fit (like say a wind vector) and then you need some other container or metadata b/c you generally have many images. So I get the sense the complexity just ends up being moved elsewhere.
    - piltboy 1y ago
      Generally speaking I would classify TIFF and its variants as imaging formats (or for very simple numerical datasets), and NetCDF as more suited for raw data, in particular multi-dimensional data with time series, etc. For forecast and climatological data I find NetCDF is vastly superior, but also much more complicated to work with due to the capabilities and how open the format is. Just have a look at the complexity of the CF Conventions to see what I mean: https://cfconventions.org/cf-conventions/release/v1.12.0/cf-conventions.html https://cfconventions.org/cf-conventions/release/v1.12.0/cf-... For visualizing orthophotos and the like, I would choose GeoTIFF any day of the week, as they're easy to visualize across platforms using existing libraries. Using COGs you also get the functionality of a spatial index within each GeoTIFF file, meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
      - geokon 1y ago
        Yeah the complexity of NetCDF was the ick factor that made me use GeoTIFF. It's just not the level of complexity I wanted to deal with in-program. I didn't want to wade through the details to understand the format, and I didn't want to then lock users to some subset Seemed easier to let users preprocess their NetCDFs into GeoTIFFs manually. I have a bunch of hacky scripts to massage NetCDFs from different sources in to compatible GeoTIFFs > meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request. Interesting. My performance bottleneck right now is the user selects a small regions and then the program has to read in GBs of global precipitation maps (from IMERG) and cut out tiny squares. In the extreme cases it can mean ~2 minutes of waiting for a result. This means the user can't casually select and try out different regions with quick feedback. If you have a beefy machine you can keep it all in RAM sometimes and it works better.. but it's not ideal (my 16GB machine can only handle simpler scenarios) I'll take a closer look at in the future. At the moment I just use Java's default TIFF reader and ImageBuffer class. Maybe it'd be easy to convert to COG format and adapt in a COG reader
- wiredfool 1y ago
  This is why some people consider TIFF to stand for “Thousands of Incompatible File Formats”.
- flufluflufluffy 1y ago
  We have hundreds of thousands of TIFF files where I work which are scans of questionnaires filled out by clinical trial participants. The one annoying thing is that web browsers don’t natively display them. I did some incredibly inefficient JavaScript bs to decode the pixel data, plop it in a canvas, get a PNG data url from it, and set that as the src for an img element xD (why not just display the canvas? because I was too lazy to manually handle resizes…) good times
blacklion 1y ago
Unfortunately, it doesn't help. Almost any digital camera RAW format is TIFF inside. And you can see how much kludges good metadata library needs to read all of them: offsets from the IFD, offsets from beginning of file with or without header, offsets from fields in IFD, etc, etc, etc. You take TIFF, you change header to make your format, and then you cannot implement this TIFF properly! Even DNG (which is tiff inside) is mangled by camera firmware authors!
hellojohnbuck 1y ago
Carlsen's TIFF extensibility was what Milne and Lentczner used as a guiding principal when building AIFF, as did Leak on QuickTime years later.

sllabres 1y ago

And the (early) availability of well made library, LibTIFF by Sam Leffler. I used it extensively from 1995 on, but only found out that according to Wikipedia is dates back to 1988!

flomo 1y ago

Perhaps the greatest thing about TIFF, but also the most horrible things, and probably why TIFF is mostly historical. It was so extendable that no two programs ever accepted the exact same TIFF extensions. (omitting the war story) edit: forgot about byte order...

yread 1y ago
But most of these variations were part of the spec (endianness with II or MM, later magic 43 for bigTIFF 64bit extension). I work with tiff and tiff-derived formats in digital microscopy where its very much not historical. And the alternatives (DICOM supp 145, vendor-specific garbage ... and thats it) are worse. I quite like the format, the only thing I would change is to have the option not to store directory information in a linked list spread throughout the file but in a simple array. Duplicate it at the beginning and end of the file and you've got resilience too (important in the age of floppies)
piltboy 1y ago
TIFF is still very much alive in certain circles, see for example https://cogeo.org/ https://cogeo.org/ The format is basically a TIFF file with attached georeferencing information and with the data organized by geographical sector, enabling fast downloads of regional subsets.
- deleted 1y ago
  [deleted]
- geokon 1y ago
  I'm not super knowledgeable about this stuff.. but out of curiosity, what advantage does it have over NetCDF? I wrote a program processing GeoTIFF data. When I had started this project I chose GeoTIFF mostly b/c i wanted something simple. And I could load them in to Java's BufferImage class and manipulate them that way. But it seems all the pros exclusively use NetCDF and GeoTiffs are for noobs (working with atmospheric science data here) GeoTIFF does extend "images" to cover more usecases, but a lot of stuff doesn't fit (like say a wind vector) and then you need some other container or metadata b/c you generally have many images. So I get the sense the complexity just ends up being moved elsewhere.
  - piltboy 1y ago
    Generally speaking I would classify TIFF and its variants as imaging formats (or for very simple numerical datasets), and NetCDF as more suited for raw data, in particular multi-dimensional data with time series, etc. For forecast and climatological data I find NetCDF is vastly superior, but also much more complicated to work with due to the capabilities and how open the format is. Just have a look at the complexity of the CF Conventions to see what I mean: https://cfconventions.org/cf-conventions/release/v1.12.0/cf-conventions.html https://cfconventions.org/cf-conventions/release/v1.12.0/cf-... For visualizing orthophotos and the like, I would choose GeoTIFF any day of the week, as they're easy to visualize across platforms using existing libraries. Using COGs you also get the functionality of a spatial index within each GeoTIFF file, meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request.
    - geokon 1y ago
      Yeah the complexity of NetCDF was the ick factor that made me use GeoTIFF. It's just not the level of complexity I wanted to deal with in-program. I didn't want to wade through the details to understand the format, and I didn't want to then lock users to some subset Seemed easier to let users preprocess their NetCDFs into GeoTIFFs manually. I have a bunch of hacky scripts to massage NetCDFs from different sources in to compatible GeoTIFFs > meaning that you can stream subsets of GeoTIFF files without having to scan through the entire file for each request. Interesting. My performance bottleneck right now is the user selects a small regions and then the program has to read in GBs of global precipitation maps (from IMERG) and cut out tiny squares. In the extreme cases it can mean ~2 minutes of waiting for a result. This means the user can't casually select and try out different regions with quick feedback. If you have a beefy machine you can keep it all in RAM sometimes and it works better.. but it's not ideal (my 16GB machine can only handle simpler scenarios) I'll take a closer look at in the future. At the moment I just use Java's default TIFF reader and ImageBuffer class. Maybe it'd be easy to convert to COG format and adapt in a COG reader
wiredfool 1y ago
This is why some people consider TIFF to stand for “Thousands of Incompatible File Formats”.
flufluflufluffy 1y ago
We have hundreds of thousands of TIFF files where I work which are scans of questionnaires filled out by clinical trial participants. The one annoying thing is that web browsers don’t natively display them. I did some incredibly inefficient JavaScript bs to decode the pixel data, plop it in a canvas, get a PNG data url from it, and set that as the src for an img element xD (why not just display the canvas? because I was too lazy to manually handle resizes…) good times

blacklion 1y ago

Unfortunately, it doesn't help. Almost any digital camera RAW format is TIFF inside. And you can see how much kludges good metadata library needs to read all of them: offsets from the IFD, offsets from beginning of file with or without header, offsets from fields in IFD, etc, etc, etc. You take TIFF, you change header to make your format, and then you cannot implement this TIFF properly! Even DNG (which is tiff inside) is mangled by camera firmware authors!

hellojohnbuck 1y ago

Carlsen's TIFF extensibility was what Milne and Lentczner used as a guiding principal when building AIFF, as did Leak on QuickTime years later.