13 ms·

Hello, PNG

268 points by EntICOnc 4y ago

not2b 4y ago
I love this comment: PNG defines a "PNG four byte integer", which is limited to the range 0 to 2^31-1, to defend against the existence of C programmers.
jezek2 4y ago
I've written a CC0-licensed (basically public domain) implementation of PNG/ZLIB: http://public-domain.advel.cz/ http://public-domain.advel.cz/ It also contains a simple version without actual compression which is actually a good alternative to BMP files as I was quite confused about the specifications for BMP so I rather wrote a PNG implementation.
swyx 4y ago
> PNG is my favourite file format of all time. Version 1.0 of the specification was released in 1996 (before I was born!) god i feel ancient.
andrewstuart 4y ago
PNG IS great but it’s dog slow.
rpigab 4y ago
I love this kind of introduction to "simple" formats! Thanks for sharing. Always a good insight to know how the basic concepts of these work without needing hours of learning deep specific knowledge that you'd only spend if you had to work directly with the format, like if you're writing a png lib.
zelphirkalt 4y ago
I like how that blog / website looks with that ASCII header! Often I wish I would come up with such a design. And yes, I also like PNG a lot :)
- wildpeaks 4y ago
  I've been using this generator for years for ASCII headers: http://patorjk.com/software/taag/#p=display&f=ANSI%20Shadow&t=Hello http://patorjk.com/software/taag/#p=display&f=ANSI%20Shadow&...
  - tecleandor 4y ago
    Oh nice! That seems to be using the tool 'figlet' (or maybe toilet, from libcaca), and the font is banner3-D, that's available here: https://github.com/xero/figlet-fonts https://github.com/xero/figlet-fonts If you install the figlet (or toilet) tool and clone that font repo you can do a : figlet -d ./figlet-fonts -f Banner3-D My text '##::::'##:'##:::'##::::'########:'########:'##::::'##:'########: ###::'###:. ##:'##:::::... ##..:: ##.....::. ##::'##::... ##..:: ####'####::. ####::::::::: ##:::: ##::::::::. ##'##:::::: ##:::: ## ### ##:::. ##:::::::::: ##:::: ######:::::. ###::::::: ##:::: ##. #: ##:::: ##:::::::::: ##:::: ##...:::::: ## ##:::::: ##:::: ##:.:: ##:::: ##:::::::::: ##:::: ##:::::::: ##:. ##::::: ##:::: ##:::: ##:::: ##:::::::::: ##:::: ########: ##:::. ##:::: ##:::: ..:::::..:::::..:::::::::::..:::::........::..:::::..:::::..::::: Toilet also has colour effects and can output in different formats: toilet -E list Available export formats: "caca": native libcaca format "ansi": ANSI "utf8": UTF-8 with ANSI escape codes "utf8cr": UTF-8 with ANSI escape codes and MS-DOS \r "html": HTML "html3": backwards-compatible HTML "bbfr": BBCode (French) "irc": IRC with mIRC colours "ps": PostScript document "svg": SVG vector image "tga": TGA image "troff": troff source Cool! Edit: added image formats
  - giantrobot 4y ago
    Take a look at the CSS scaling on the site too. It scales the <pre> block responsively so it works on mobile browsers. The powers of figlet/toilet and CSS combine for a cool look.
yread 4y ago
Are there formats that try to do more of a 2D compression? With png horizontal lines compress way better than vertical lines. Something like what JBIG2 does but without the focus on letters
- pavlov 4y ago
  Most image compression formats work in pixel blocks instead of scanlines. For example JPEG uses a 8x8 block size. This way rotating the image doesn't affect compression performance (unlike PNG).
- Retr0id 4y ago
  JPEG-XL's lossless mode has something called the "squeeze transform" which operates over the image data in 2D. It's a variation of the Haar Transform, which works on a similar principle as DCT https://en.wikipedia.org/wiki/Haar_wavelet#Haar_transform https://en.wikipedia.org/wiki/Haar_wavelet#Haar_transform It also has "meta-adaptive filters", which act similarly to PNG's filters except you get to encode a custom decision-tree that defines how pixels are filtered, as a function of their neighboring pixels
greggman3 4y ago
Honestly, I don't consider PNG a simple format. The CRC and the compression are non-trivial. If you're using a new language that doesn't have those features built in and/or you don't have a reasonable amount of programming experience then you're going to likely fail (or learn a ton). zlib is 23k lines. "simple" is not word I'd use to describe PNG Simple formats are like certain forms of .TGA and .BMP. A simple header and then the pixel data. No CRCs, no compression. Done. You can write an entire reader in 20-30 lines of code and a writer in other 20-30 lines of code as well. Both of those formats have options that can probably make them more work but if you're storing 24bit "True color" or 32 bit "true color + alpha" then they are way easier formats. Of course they're not common formats so you're stuck with complex formats like PNG
- SideQuark 4y ago
  > The CRC and the compression are non-trivial. CRC is a table and 5 lines of code. That's trivial. >zlib is 23k lines It's not needed to make a PNG reader/writer. zlib is massive overkill for only making a PNG reader or writer. Here's a tiny deflate/inflate code [2] under 1k lines (and could be much smaller if needed). stb[0] has single headers of ~7k lines total including all of the formats PNG, JPG, BMP,. PSD, GIF, HDR, and PIC. Here's [1] a 3k lines single file PNG version with tons if #ifdefs for all sorts of platforms. Removing those and I'd not be surprised if you could not do it in ~1k lines (which I'd consider quite simple compared to most of todays' media formats). >Of course they're not common formats so you're stuck with complex formats like PNG BMP is super common and easy to use anywhere. I use flat image files all the time for quick and dirty stuff. They quickly saturate disk speeds and networking speeds (say recording a few decent speed cameras), and I've found PNG compression to alleviate those saturate CPU speeds (some libs are super slow, some are vastly faster). I've many times made custom compression formats to balance these for high performance tools when neither things like BMPs or things like PNG would suffice. [0] https://github.com/nothings/stb https://github.com/nothings/stb [1] https://github.com/richgel999/fpng/blob/main/src/fpng.cpp https://github.com/richgel999/fpng/blob/main/src/fpng.cpp [2] https://github.com/jibsen/tinf/tree/master/src https://github.com/jibsen/tinf/tree/master/src
- IYasha 4y ago
  Um... for the record: BMP and TGA may have compression. And, since it is rarely implemented, you may crash a lot of stuff with your RLE bitmap )
ivoras 4y ago
Does anyone know why filtering is mandatory? It seems like operations with the format would be a bit faster if the pre-compression data would just be a framebuffer dump, instead of prefixing each row with a 1-byte "filter ID", possibly breaking data alignment.
- fla 4y ago
  Can’t answer as to why it is mandatory but it is known to improve compression.
- IvanK_net 4y ago
  Adding 1 Byte to each line of raw pixel data is not that much. You can set it to zeros to avoid filtering. But it gives you a chance to improve the compression.
- kevincox 4y ago
  I suspect even if you weren't using heuristics you would be best to pick a different static filter than "None". For example I would expect that "Up" does much better on average as you are essentially bringing in context that the compressor would struggle to line up. I think most encoders would probably only use "None" in cases where heuristics show that nothing else helps. So it is probably mandatory because only a tiny minority of images wouldn't use a filter. So it is better to just require it to avoid one more condition in the decoder.
- pornel 4y ago
  The assumption was that it improves compression of gradients and photographic data. In order for compressors to be able to use it, it has to be mandatory for decompressors. Anything that is optional becomes unusable (e.g. arithmetic coding in JPEG). Filtering as a separate preprocessing step allowed PNG to use off-the-shelf zlib for compression without needing to modify it.
pohl 4y ago
That first paragraph is a master class in making me feel old. 1996 was yesterday for some of us.
- kibwen 4y ago
  I recently heard someone refer to this period as "the late 1900s" and proceeded to shrivel into dust.
- zanchey 4y ago
  It's the "similarily old" ZIP comparison for me
- bluedino 4y ago
  As a young programmer back then, the guys who could write image libraries were wizards to me.
dalke 4y ago
It's a tiny issue, but what I like least about the PNG format is the checksum at the end of each block. From what I gather, the checksum is there for two reasons: 1) a check on archival integrity, based on experience with its usefulness in ZIP files, and 2) a way to check for download errors early, before reaching the end-of-file, which was more important in the slow and noisy modem era of the 1990s. However, it's still possible for a change in chunk type to go undetected, for example, if "zTXt" were transformed to "xTXt" - a one bit change. (I think it's also possible to construct a chunk such that if the length changed to just the right value then it could be interpreted as two chunks (with a smaller length) or be merged with later chunks. This requires getting the CRCs to align just right, and even harder to have just a single bit change.) My belief is that removing the per-block CRC32 and putting the checksum in the IEND at the very end of the PNG data stream, and using a stronger checksum - even MD5 - would be more effective at archival integrity. This of course can't happen now. Still, I regard it as a small bit of 1990s cruft. When I developed my own format, for non-image data, I started with PNG as a guideline, then found that dealing with the checksum, even just to always generate a valid value, was a nuisance, with seemingly no good reason to justify its overhead. I decided to drop the checksum, with a hand-waving argument that people should use other tools to detect and even repair file corruption, depending on their specific requirements. This more generic and widespread family of formats is called "FourCC" for "four-character code" (https://en.wikipedia.org/wiki/FourCC https://en.wikipedia.org/wiki/FourCC).
- edflsafoiewq 4y ago
  The check value is doubly stupid because there are two of them: zlib compression adds the Adler32 of the uncompressed data, and then the IDAT adds a CRC32 of the compressed data.
  - layer8 4y ago
    Not all chunk types use compression. The CRC is a generic mechanism that is agnostic of the chunk type.
- kevincox 4y ago
  I think that per-chunk checksums can be helpful if you are manipulating PNG files. For example imaging that you are "cleaning" a PNG by removing metadata such as location information and timezone. The checksum can help make sure that you don't mess up your copying of unchanged chunks (or at least you notice your mistake faster). Even if your code is perfect it could help detect a bitflip during processing. Admittedly minor, but why not? I think the biggest mistake is that the checksum doesn't cover the type and length. If it did then I most of your concerns would be resolved. Although it may also make sense to have a full-file checksum in the IEND, but the only thing that could really detect is if whole chunks were perfectly dropped somehow, so not much added value, but again 4 bytes seems worth it.
Waterluvian 4y ago
Doesn’t PNG also support infinite non-image data? I thought things like the PICO-8 would share programs as .png images. Or is this a hack?
- debugnik 4y ago
  PICO-8 uses steganography, encoding the data into the 2 least significant bits of the ARGB channels, so 1 byte per pixel.
- yunruse 4y ago
  It’s just stenography – each pixel stores a byte in the least significant 2 bits of ARGB. https://pico-8.fandom.com/wiki/P8PNGFileFormat https://pico-8.fandom.com/wiki/P8PNGFileFormat
- kevincox 4y ago
  You can add your own chunks, and you can mark them as optional. It wasn't specified in the article but IIRC the casing of each letter in the chunk type has special meaning. So AAAA and aaaa have implicit meanings such as "required", "keep when processing" and I guess two other flags.
dbrgn 4y ago
Due to the PNG signature ("magic bytes"), every PNG file starts with the sequence "89 50 4E 47 0D 0A 1A 0A". In ASCII: "[\x89]PNG\r\n[SUB]\n" Is there any information on the origin of these bytes? Why were they chosen like that?
- blueflow 4y ago
  So you'd get "PNG" printed out when you accidentally "type"ed that file in MS-DOS. The 1A was the EOF character for text data back then.
  - dbrgn 4y ago
    Ah, that makes a lot of sense! So what's 0x89 at the start? It's outside the ASCII range.
    - blueflow 4y ago
      So when a 7-bit transfer masks out all highest bits, it becomes a invalid signature instead of the pixel data getting silently corrupted.
- edflsafoiewq 4y ago
  They are explained in detail on the Wikipedia page: https://en.wikipedia.org/wiki/Portable_Network_Graphics#File_header https://en.wikipedia.org/wiki/Portable_Network_Graphics#File...
coev 4y ago
This article makes me feel old, having lived through a time where you avoided using PNGs on the web because IE6 didn't support the transparency
- kijin 4y ago
  IE6 as well as some of the later versions also got the gamma wrong, so even if you used a non-transparent PNG the colors would be subtly different from the surrounding CSS colors.
- plusminusplus 4y ago
  A workaround introduced me to the CSS `filter` property. (Some monster value starting with progid:DXImageTransform...). Crazy considering how much I use the property now.
- masklinn 4y ago
  > you avoided using PNGs on the web because IE6 didn't support the transparency Nonsense. The only reason to avoid PNG was IE’s gamma issues: IE6 did not support progressive transparency, so at worst you had full-color gifs.
  - seba_dos1 4y ago
    Absolutely not. Not supporting more than 1-bit alpha was a huge reason to avoid doing anything serious with PNGs if you cared about IE6; gamma was like a cherry on top. You could actually make it work with non-standard DirectX filters, but it came with its own set of drawbacks and wasn't always a viable option.
    - masklinn 4y ago
      > Absolutely not. Not supporting more than 1-bit alpha was a huge reason to avoid doing anything serious with PNGs if you cared about IE6 That makes absolutely no sense, because there was no other format which could do progressive transparency on IE, and short of animation palletised PNG is superior to GIF. And for non-photographic full-color, PNG is generally much better than JPEG. So avoiding PNG just gave you larger files or worse results for no gain.
_benj 4y ago
Just in case others find it useful, I recently came across Netpbm[0] which is comically simple to implement![1] It’s a pretty good image format if one wants to quickly make something visual with code. [0] https://en.m.wikipedia.org/wiki/Netpbm https://en.m.wikipedia.org/wiki/Netpbm [1] https://git.sr.ht/~benjcal/bc_libs/tree/main/item/bc_buffer.h#L182 https://git.sr.ht/~benjcal/bc_libs/tree/main/item/bc_buffer....
- edflsafoiewq 4y ago
  No alpha channel unfortunately.
  - _benj 4y ago
    yeah :(
    - deathanatos 4y ago
      Netpbm (the spec) supports alpha. It's in the PAM format: https://netpbm.sourceforge.net/doc/pam.html https://netpbm.sourceforge.net/doc/pam.html I've never tried that variant, nor do I think I've seen files in the wild, so IDK how widely stuff supports it. If I have need for an alpha channel, I usually reach for a PNG encoder at that point…
- geokon 4y ago
  Do browsers support it?
  - _benj 4y ago
    Not sure, tested it with firefox and it offered to download it. I don't think is a good format for the web though because it takes too much space. I use it to make images with C locally and monitor them with a quick image viewer[0] I wrote [0] https://git.sr.ht/~benjcal/bc_tools/tree/main/item/bc_viewer.c https://git.sr.ht/~benjcal/bc_tools/tree/main/item/bc_viewer...
smcameron 4y ago
> a "PNG four byte integer", which is limited to the range 0 to 231-1, to defend against the existence of C programmers. Kind of an odd thing to say, considering the existence and prevalence of libpng, which is written in C, and which uses setjmp() and longjmp() as part of its API. It's difficult to think of a more ill-advised and bonkers but extremely C-centric thing to do.
- mjburgess 4y ago
  Are there reasons to interpret `setjmp` and `longjmp` as anything other than a `C` (/hardware) representation of Effects? (In the sense of exceptions, coroutines/await, etc.) If so, then why aren't they fundamentally quite reasonable?
  - flohofwoe 4y ago
    Some platforms don't support full setjmp/longjmp feature set (WASM for instance). As far as I'm aware libpng also works without setjmp/longjmp support though via a build config option (it's still not fun to integrate into a project if you need to build it from source).
    - keyneus 4y ago
      As of libpng 1.6.0, a so-called "simplified API" was added, which does not use setjmp/longjmp. A while back I had a C project using the old API, and I converted it to C++, and the interaction of setjmp/longjmp with exceptions was giving me headaches. I switched to the simplified API, and it was a breeze. So much less code, and no hacky C "exceptions". If you can require libpng 1.6 or newer, it's worth looking at the simplified API, if it supports your needs. It's described here: http://www.libpng.org/pub/png/libpng-manual.txt http://www.libpng.org/pub/png/libpng-manual.txt
  - deleted 4y ago
    [deleted]
- flohofwoe 4y ago
  I guess that's one important reason why stb_image.h became so popular. Last time I tried to integrate libpng into a project I just gave up (that was on Windows, I guess on Linux it would just be an 'apt install libpng-dev').
scotty79 4y ago
What's more PNG is flexible enough that Macromedia Fireworks used it as native file format for its documents (like more reasonable psd equivalent).
Cloudef 4y ago
>"31-bit" is not a typo - PNG defines a "PNG four byte integer", which is limited to the range 0 to 231-1, to defend against the existence of C programmers. What exactly is implied here? Silly guard against overflow?
- edflsafoiewq 4y ago
  It's "in order to accommodate languages that have difficulty with unsigned four-byte values" ie. Java.
  - Cloudef 4y ago
    That makes more sense
- Retr0id 4y ago
  Mostly just a joke at C's expense. When reviewing C code, any time the sign-bit gets touched I consider the danger zone to have been entered (especially in the context of parsing data). Limiting the range of ints is a good defensive programming tactic. As another commenter points out, the real reason is about practicality - some languages like Java don't natively support unsigned ints.
  - Cloudef 4y ago
    Maybe I'm old, but i dont get it. I'd say signed is more dangerous as the overflow is undefined. And if you read to unsigned, you are just wasting range.
    - Retr0id 4y ago
      Yes, that's the point. Signed ints are a potential footgun, which can be partially mitigated by limiting the range to 0 - 2^31-1. Good specs pre-emptively mitigate implementation bugs.
    - deleted 4y ago
      [deleted]
perardi 4y ago
You know you’re old when you still reflexively flinch just a bit when you see PNGs mentioned, as you have battle scars from working with PNG-24 files in IE6. https://24ways.org/2007/supersleight-transparent-png-in-ie6 https://24ways.org/2007/supersleight-transparent-png-in-ie6
- deathanatos 4y ago
  … one of the many reasons that led to Firefox's rise.
blogsbro 4y ago
Last time i was on the scene.org ftp (-style) download webpage, there was a .png from a party, -years ago my internetconection was damn slow that time, but i've seen a picture of 'hal' (computer in a movie) beginning with the 'rasterpoint' (image build-up-start-point) on 'down right' running 'to the left' and up, line by line... maybe i calculated his bits wrong but over a hundred thousand bits for a picture in 300 x XXX ? here's a (german) comic in 12,5 kb but i think they don't like hotlnking ^^ > //i.ibb.co/TPgSkF6/10126-DER-WARME-PULLI-EIN-AKT-FINAL-Mail.png regards...
nayuki 4y ago
Having implemented most of the PNG specification from scratch in the past month, I agree with all of the features highlighted by the author in the article's introduction. Although there are some minor things I don't like, overall it is a very well-designed format that has minimal ambiguity and stands the test of time. You can find my modern Java PNG library at: https://www.nayuki.io/page/png-library https://www.nayuki.io/page/png-library , https://github.com/nayuki/PNG-library https://github.com/nayuki/PNG-library I also made a web-based tool to dissect PNG files and show all the fields and errors: https://www.nayuki.io/page/png-file-chunk-inspector https://www.nayuki.io/page/png-file-chunk-inspector And my own "minimum-viable PNG writer" in ~140 lines of Java back in the year 2012: https://www.nayuki.io/page/dumb-png-output-java https://www.nayuki.io/page/dumb-png-output-java
- gred 4y ago
  > What makes this library modern? It [...] consumes more CPU and memory to simplify the logic and improve reliability. Intriguingly unapologetic, but I think I'll stick to the PNG libraries that are mature enough to be both reliable and fast :-)
  - nayuki 4y ago
    These might change your mind on reliability: https://github.com/glennrp/libpng/issues https://github.com/glennrp/libpng/issues ; http://www.libpng.org/pub/png/libpng.html http://www.libpng.org/pub/png/libpng.html section "Security and Crash Bugs in Older Versions"; https://www.cvedetails.com/vulnerability-list/vendor_id-7294/Libpng.html https://www.cvedetails.com/vulnerability-list/vendor_id-7294... Regarding performance, I already lost the game before it started because I'm writing Java. If I wanted to squeeze CPU time, I would be writing C/C++/asm. So I decided to aim for conciseness and reliability instead of the endless stream of vulnerabilities.
    - gred 4y ago
      Point made, but I was actually thinking of the default Java platform support for writing PNG files (javax.imageio + zlib, which has a decent track record). https://www.cvedetails.com/product/111843/Zlib-Zlib.html?vendor_id=13265 https://www.cvedetails.com/product/111843/Zlib-Zlib.html?ven...
dmitrygr 4y ago
> which is limited to the range 0 to 2^31-1, to defend against the existence of C programmers. What is this nonsense? You mean Java, right? C has always had unsigned types.
chamwislothe2nd 4y ago
Remember when IE didn't support it? And the goofy hacks Microsoft made people do to support transparency on websites?
- giantrobot 4y ago
  Oh, IE "supported" PNGs almost maliciously. If you had a True Color PNG? No problem. Paletted 8-bit? No problem. 8-but with alpha? Fine. True Color with alpha? I hope you like seeing your background color you meant to alpha out. You had to use Microsoft's DirectX filtering CSS extensions to properly handle the alpha channel of True Color PNGs.
  - masklinn 4y ago
    > 8-but with alpha? Fine. Not exactly, palletised png supports a full alpha channel which did not work with IE. Though you had to work to get that as usually software would limit palletised output to GIF (if you didn’t outright have to create your pngs from gifs).
- ksherlock 4y ago
  Oh god. filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(...)
deleted 4y ago
[deleted]
galleywest200 4y ago
I found the info-graphics in this repository to be of great use when I was trying to digest different image formats: https://github.com/corkami/pics/blob/master/binary/PNG.png https://github.com/corkami/pics/blob/master/binary/PNG.png
deleted 4y ago
[deleted]

268 points by EntICOnc 4y ago

I love this comment: PNG defines a "PNG four byte integer", which is limited to the range 0 to 2^31-1, to defend against the existence of C programmers.

jezek2 4y ago

I've written a CC0-licensed (basically public domain) implementation of PNG/ZLIB: http://public-domain.advel.cz/ http://public-domain.advel.cz/ It also contains a simple version without actual compression which is actually a good alternative to BMP files as I was quite confused about the specifications for BMP so I rather wrote a PNG implementation.

swyx 4y ago

> PNG is my favourite file format of all time. Version 1.0 of the specification was released in 1996 (before I was born!) god i feel ancient.

andrewstuart 4y ago

PNG IS great but it’s dog slow.

rpigab 4y ago

I love this kind of introduction to "simple" formats! Thanks for sharing. Always a good insight to know how the basic concepts of these work without needing hours of learning deep specific knowledge that you'd only spend if you had to work directly with the format, like if you're writing a png lib.

zelphirkalt 4y ago

I like how that blog / website looks with that ASCII header! Often I wish I would come up with such a design. And yes, I also like PNG a lot :)

wildpeaks 4y ago
I've been using this generator for years for ASCII headers: http://patorjk.com/software/taag/#p=display&f=ANSI%20Shadow&t=Hello http://patorjk.com/software/taag/#p=display&f=ANSI%20Shadow&...
- tecleandor 4y ago
  Oh nice! That seems to be using the tool 'figlet' (or maybe toilet, from libcaca), and the font is banner3-D, that's available here: https://github.com/xero/figlet-fonts https://github.com/xero/figlet-fonts If you install the figlet (or toilet) tool and clone that font repo you can do a : figlet -d ./figlet-fonts -f Banner3-D My text '##::::'##:'##:::'##::::'########:'########:'##::::'##:'########: ###::'###:. ##:'##:::::... ##..:: ##.....::. ##::'##::... ##..:: ####'####::. ####::::::::: ##:::: ##::::::::. ##'##:::::: ##:::: ## ### ##:::. ##:::::::::: ##:::: ######:::::. ###::::::: ##:::: ##. #: ##:::: ##:::::::::: ##:::: ##...:::::: ## ##:::::: ##:::: ##:.:: ##:::: ##:::::::::: ##:::: ##:::::::: ##:. ##::::: ##:::: ##:::: ##:::: ##:::::::::: ##:::: ########: ##:::. ##:::: ##:::: ..:::::..:::::..:::::::::::..:::::........::..:::::..:::::..::::: Toilet also has colour effects and can output in different formats: toilet -E list Available export formats: "caca": native libcaca format "ansi": ANSI "utf8": UTF-8 with ANSI escape codes "utf8cr": UTF-8 with ANSI escape codes and MS-DOS \r "html": HTML "html3": backwards-compatible HTML "bbfr": BBCode (French) "irc": IRC with mIRC colours "ps": PostScript document "svg": SVG vector image "tga": TGA image "troff": troff source Cool! Edit: added image formats
- giantrobot 4y ago
  Take a look at the CSS scaling on the site too. It scales the <pre> block responsively so it works on mobile browsers. The powers of figlet/toilet and CSS combine for a cool look.

yread 4y ago

Are there formats that try to do more of a 2D compression? With png horizontal lines compress way better than vertical lines. Something like what JBIG2 does but without the focus on letters

pavlov 4y ago
Most image compression formats work in pixel blocks instead of scanlines. For example JPEG uses a 8x8 block size. This way rotating the image doesn't affect compression performance (unlike PNG).
Retr0id 4y ago
JPEG-XL's lossless mode has something called the "squeeze transform" which operates over the image data in 2D. It's a variation of the Haar Transform, which works on a similar principle as DCT https://en.wikipedia.org/wiki/Haar_wavelet#Haar_transform https://en.wikipedia.org/wiki/Haar_wavelet#Haar_transform It also has "meta-adaptive filters", which act similarly to PNG's filters except you get to encode a custom decision-tree that defines how pixels are filtered, as a function of their neighboring pixels

greggman3 4y ago

Honestly, I don't consider PNG a simple format. The CRC and the compression are non-trivial. If you're using a new language that doesn't have those features built in and/or you don't have a reasonable amount of programming experience then you're going to likely fail (or learn a ton). zlib is 23k lines. "simple" is not word I'd use to describe PNG Simple formats are like certain forms of .TGA and .BMP. A simple header and then the pixel data. No CRCs, no compression. Done. You can write an entire reader in 20-30 lines of code and a writer in other 20-30 lines of code as well. Both of those formats have options that can probably make them more work but if you're storing 24bit "True color" or 32 bit "true color + alpha" then they are way easier formats. Of course they're not common formats so you're stuck with complex formats like PNG

SideQuark 4y ago
> The CRC and the compression are non-trivial. CRC is a table and 5 lines of code. That's trivial. >zlib is 23k lines It's not needed to make a PNG reader/writer. zlib is massive overkill for only making a PNG reader or writer. Here's a tiny deflate/inflate code [2] under 1k lines (and could be much smaller if needed). stb[0] has single headers of ~7k lines total including all of the formats PNG, JPG, BMP,. PSD, GIF, HDR, and PIC. Here's [1] a 3k lines single file PNG version with tons if #ifdefs for all sorts of platforms. Removing those and I'd not be surprised if you could not do it in ~1k lines (which I'd consider quite simple compared to most of todays' media formats). >Of course they're not common formats so you're stuck with complex formats like PNG BMP is super common and easy to use anywhere. I use flat image files all the time for quick and dirty stuff. They quickly saturate disk speeds and networking speeds (say recording a few decent speed cameras), and I've found PNG compression to alleviate those saturate CPU speeds (some libs are super slow, some are vastly faster). I've many times made custom compression formats to balance these for high performance tools when neither things like BMPs or things like PNG would suffice. [0] https://github.com/nothings/stb https://github.com/nothings/stb [1] https://github.com/richgel999/fpng/blob/main/src/fpng.cpp https://github.com/richgel999/fpng/blob/main/src/fpng.cpp [2] https://github.com/jibsen/tinf/tree/master/src https://github.com/jibsen/tinf/tree/master/src
IYasha 4y ago
Um... for the record: BMP and TGA may have compression. And, since it is rarely implemented, you may crash a lot of stuff with your RLE bitmap )

ivoras 4y ago

Does anyone know why filtering is mandatory? It seems like operations with the format would be a bit faster if the pre-compression data would just be a framebuffer dump, instead of prefixing each row with a 1-byte "filter ID", possibly breaking data alignment.

fla 4y ago
Can’t answer as to why it is mandatory but it is known to improve compression.
IvanK_net 4y ago
Adding 1 Byte to each line of raw pixel data is not that much. You can set it to zeros to avoid filtering. But it gives you a chance to improve the compression.
kevincox 4y ago
I suspect even if you weren't using heuristics you would be best to pick a different static filter than "None". For example I would expect that "Up" does much better on average as you are essentially bringing in context that the compressor would struggle to line up. I think most encoders would probably only use "None" in cases where heuristics show that nothing else helps. So it is probably mandatory because only a tiny minority of images wouldn't use a filter. So it is better to just require it to avoid one more condition in the decoder.
pornel 4y ago
The assumption was that it improves compression of gradients and photographic data. In order for compressors to be able to use it, it has to be mandatory for decompressors. Anything that is optional becomes unusable (e.g. arithmetic coding in JPEG). Filtering as a separate preprocessing step allowed PNG to use off-the-shelf zlib for compression without needing to modify it.

pohl 4y ago

That first paragraph is a master class in making me feel old. 1996 was yesterday for some of us.

kibwen 4y ago
I recently heard someone refer to this period as "the late 1900s" and proceeded to shrivel into dust.
zanchey 4y ago
It's the "similarily old" ZIP comparison for me
bluedino 4y ago
As a young programmer back then, the guys who could write image libraries were wizards to me.

dalke 4y ago

It's a tiny issue, but what I like least about the PNG format is the checksum at the end of each block. From what I gather, the checksum is there for two reasons: 1) a check on archival integrity, based on experience with its usefulness in ZIP files, and 2) a way to check for download errors early, before reaching the end-of-file, which was more important in the slow and noisy modem era of the 1990s. However, it's still possible for a change in chunk type to go undetected, for example, if "zTXt" were transformed to "xTXt" - a one bit change. (I think it's also possible to construct a chunk such that if the length changed to just the right value then it could be interpreted as two chunks (with a smaller length) or be merged with later chunks. This requires getting the CRCs to align just right, and even harder to have just a single bit change.) My belief is that removing the per-block CRC32 and putting the checksum in the IEND at the very end of the PNG data stream, and using a stronger checksum - even MD5 - would be more effective at archival integrity. This of course can't happen now. Still, I regard it as a small bit of 1990s cruft. When I developed my own format, for non-image data, I started with PNG as a guideline, then found that dealing with the checksum, even just to always generate a valid value, was a nuisance, with seemingly no good reason to justify its overhead. I decided to drop the checksum, with a hand-waving argument that people should use other tools to detect and even repair file corruption, depending on their specific requirements. This more generic and widespread family of formats is called "FourCC" for "four-character code" (https://en.wikipedia.org/wiki/FourCC https://en.wikipedia.org/wiki/FourCC).

edflsafoiewq 4y ago
The check value is doubly stupid because there are two of them: zlib compression adds the Adler32 of the uncompressed data, and then the IDAT adds a CRC32 of the compressed data.
- layer8 4y ago
  Not all chunk types use compression. The CRC is a generic mechanism that is agnostic of the chunk type.
kevincox 4y ago
I think that per-chunk checksums can be helpful if you are manipulating PNG files. For example imaging that you are "cleaning" a PNG by removing metadata such as location information and timezone. The checksum can help make sure that you don't mess up your copying of unchanged chunks (or at least you notice your mistake faster). Even if your code is perfect it could help detect a bitflip during processing. Admittedly minor, but why not? I think the biggest mistake is that the checksum doesn't cover the type and length. If it did then I most of your concerns would be resolved. Although it may also make sense to have a full-file checksum in the IEND, but the only thing that could really detect is if whole chunks were perfectly dropped somehow, so not much added value, but again 4 bytes seems worth it.

Waterluvian 4y ago

Doesn’t PNG also support infinite non-image data? I thought things like the PICO-8 would share programs as .png images. Or is this a hack?

debugnik 4y ago
PICO-8 uses steganography, encoding the data into the 2 least significant bits of the ARGB channels, so 1 byte per pixel.
yunruse 4y ago
It’s just stenography – each pixel stores a byte in the least significant 2 bits of ARGB. https://pico-8.fandom.com/wiki/P8PNGFileFormat https://pico-8.fandom.com/wiki/P8PNGFileFormat
kevincox 4y ago
You can add your own chunks, and you can mark them as optional. It wasn't specified in the article but IIRC the casing of each letter in the chunk type has special meaning. So AAAA and aaaa have implicit meanings such as "required", "keep when processing" and I guess two other flags.

dbrgn 4y ago

Due to the PNG signature ("magic bytes"), every PNG file starts with the sequence "89 50 4E 47 0D 0A 1A 0A". In ASCII: "[\x89]PNG\r\n[SUB]\n" Is there any information on the origin of these bytes? Why were they chosen like that?

blueflow 4y ago
So you'd get "PNG" printed out when you accidentally "type"ed that file in MS-DOS. The 1A was the EOF character for text data back then.
- dbrgn 4y ago
  Ah, that makes a lot of sense! So what's 0x89 at the start? It's outside the ASCII range.
  - blueflow 4y ago
    So when a 7-bit transfer masks out all highest bits, it becomes a invalid signature instead of the pixel data getting silently corrupted.
edflsafoiewq 4y ago
They are explained in detail on the Wikipedia page: https://en.wikipedia.org/wiki/Portable_Network_Graphics#File_header https://en.wikipedia.org/wiki/Portable_Network_Graphics#File...

coev 4y ago

This article makes me feel old, having lived through a time where you avoided using PNGs on the web because IE6 didn't support the transparency

kijin 4y ago
IE6 as well as some of the later versions also got the gamma wrong, so even if you used a non-transparent PNG the colors would be subtly different from the surrounding CSS colors.
plusminusplus 4y ago
A workaround introduced me to the CSS `filter` property. (Some monster value starting with progid:DXImageTransform...). Crazy considering how much I use the property now.
masklinn 4y ago
> you avoided using PNGs on the web because IE6 didn't support the transparency Nonsense. The only reason to avoid PNG was IE’s gamma issues: IE6 did not support progressive transparency, so at worst you had full-color gifs.
- seba_dos1 4y ago
  Absolutely not. Not supporting more than 1-bit alpha was a huge reason to avoid doing anything serious with PNGs if you cared about IE6; gamma was like a cherry on top. You could actually make it work with non-standard DirectX filters, but it came with its own set of drawbacks and wasn't always a viable option.
  - masklinn 4y ago
    > Absolutely not. Not supporting more than 1-bit alpha was a huge reason to avoid doing anything serious with PNGs if you cared about IE6 That makes absolutely no sense, because there was no other format which could do progressive transparency on IE, and short of animation palletised PNG is superior to GIF. And for non-photographic full-color, PNG is generally much better than JPEG. So avoiding PNG just gave you larger files or worse results for no gain.

_benj 4y ago

Just in case others find it useful, I recently came across Netpbm[0] which is comically simple to implement![1] It’s a pretty good image format if one wants to quickly make something visual with code. [0] https://en.m.wikipedia.org/wiki/Netpbm https://en.m.wikipedia.org/wiki/Netpbm [1] https://git.sr.ht/~benjcal/bc_libs/tree/main/item/bc_buffer.h#L182 https://git.sr.ht/~benjcal/bc_libs/tree/main/item/bc_buffer....

edflsafoiewq 4y ago
No alpha channel unfortunately.
- _benj 4y ago
  yeah :(
  - deathanatos 4y ago
    Netpbm (the spec) supports alpha. It's in the PAM format: https://netpbm.sourceforge.net/doc/pam.html https://netpbm.sourceforge.net/doc/pam.html I've never tried that variant, nor do I think I've seen files in the wild, so IDK how widely stuff supports it. If I have need for an alpha channel, I usually reach for a PNG encoder at that point…
geokon 4y ago
Do browsers support it?
- _benj 4y ago
  Not sure, tested it with firefox and it offered to download it. I don't think is a good format for the web though because it takes too much space. I use it to make images with C locally and monitor them with a quick image viewer[0] I wrote [0] https://git.sr.ht/~benjcal/bc_tools/tree/main/item/bc_viewer.c https://git.sr.ht/~benjcal/bc_tools/tree/main/item/bc_viewer...

smcameron 4y ago

> a "PNG four byte integer", which is limited to the range 0 to 231-1, to defend against the existence of C programmers. Kind of an odd thing to say, considering the existence and prevalence of libpng, which is written in C, and which uses setjmp() and longjmp() as part of its API. It's difficult to think of a more ill-advised and bonkers but extremely C-centric thing to do.

mjburgess 4y ago
Are there reasons to interpret `setjmp` and `longjmp` as anything other than a `C` (/hardware) representation of Effects? (In the sense of exceptions, coroutines/await, etc.) If so, then why aren't they fundamentally quite reasonable?
- flohofwoe 4y ago
  Some platforms don't support full setjmp/longjmp feature set (WASM for instance). As far as I'm aware libpng also works without setjmp/longjmp support though via a build config option (it's still not fun to integrate into a project if you need to build it from source).
  - keyneus 4y ago
    As of libpng 1.6.0, a so-called "simplified API" was added, which does not use setjmp/longjmp. A while back I had a C project using the old API, and I converted it to C++, and the interaction of setjmp/longjmp with exceptions was giving me headaches. I switched to the simplified API, and it was a breeze. So much less code, and no hacky C "exceptions". If you can require libpng 1.6 or newer, it's worth looking at the simplified API, if it supports your needs. It's described here: http://www.libpng.org/pub/png/libpng-manual.txt http://www.libpng.org/pub/png/libpng-manual.txt
- deleted 4y ago
  [deleted]
flohofwoe 4y ago
I guess that's one important reason why stb_image.h became so popular. Last time I tried to integrate libpng into a project I just gave up (that was on Windows, I guess on Linux it would just be an 'apt install libpng-dev').

scotty79 4y ago

What's more PNG is flexible enough that Macromedia Fireworks used it as native file format for its documents (like more reasonable psd equivalent).

Cloudef 4y ago

>"31-bit" is not a typo - PNG defines a "PNG four byte integer", which is limited to the range 0 to 231-1, to defend against the existence of C programmers. What exactly is implied here? Silly guard against overflow?

edflsafoiewq 4y ago
It's "in order to accommodate languages that have difficulty with unsigned four-byte values" ie. Java.
- Cloudef 4y ago
  That makes more sense
Retr0id 4y ago
Mostly just a joke at C's expense. When reviewing C code, any time the sign-bit gets touched I consider the danger zone to have been entered (especially in the context of parsing data). Limiting the range of ints is a good defensive programming tactic. As another commenter points out, the real reason is about practicality - some languages like Java don't natively support unsigned ints.
- Cloudef 4y ago
  Maybe I'm old, but i dont get it. I'd say signed is more dangerous as the overflow is undefined. And if you read to unsigned, you are just wasting range.
  - Retr0id 4y ago
    Yes, that's the point. Signed ints are a potential footgun, which can be partially mitigated by limiting the range to 0 - 2^31-1. Good specs pre-emptively mitigate implementation bugs.
  - deleted 4y ago
    [deleted]

perardi 4y ago

You know you’re old when you still reflexively flinch just a bit when you see PNGs mentioned, as you have battle scars from working with PNG-24 files in IE6. https://24ways.org/2007/supersleight-transparent-png-in-ie6 https://24ways.org/2007/supersleight-transparent-png-in-ie6

deathanatos 4y ago
… one of the many reasons that led to Firefox's rise.

blogsbro 4y ago

Last time i was on the scene.org ftp (-style) download webpage, there was a .png from a party, -years ago my internetconection was damn slow that time, but i've seen a picture of 'hal' (computer in a movie) beginning with the 'rasterpoint' (image build-up-start-point) on 'down right' running 'to the left' and up, line by line... maybe i calculated his bits wrong but over a hundred thousand bits for a picture in 300 x XXX ? here's a (german) comic in 12,5 kb but i think they don't like hotlnking ^^ > //i.ibb.co/TPgSkF6/10126-DER-WARME-PULLI-EIN-AKT-FINAL-Mail.png regards...

nayuki 4y ago

Having implemented most of the PNG specification from scratch in the past month, I agree with all of the features highlighted by the author in the article's introduction. Although there are some minor things I don't like, overall it is a very well-designed format that has minimal ambiguity and stands the test of time. You can find my modern Java PNG library at: https://www.nayuki.io/page/png-library https://www.nayuki.io/page/png-library , https://github.com/nayuki/PNG-library https://github.com/nayuki/PNG-library I also made a web-based tool to dissect PNG files and show all the fields and errors: https://www.nayuki.io/page/png-file-chunk-inspector https://www.nayuki.io/page/png-file-chunk-inspector And my own "minimum-viable PNG writer" in ~140 lines of Java back in the year 2012: https://www.nayuki.io/page/dumb-png-output-java https://www.nayuki.io/page/dumb-png-output-java

gred 4y ago
> What makes this library modern? It [...] consumes more CPU and memory to simplify the logic and improve reliability. Intriguingly unapologetic, but I think I'll stick to the PNG libraries that are mature enough to be both reliable and fast :-)
- nayuki 4y ago
  These might change your mind on reliability: https://github.com/glennrp/libpng/issues https://github.com/glennrp/libpng/issues ; http://www.libpng.org/pub/png/libpng.html http://www.libpng.org/pub/png/libpng.html section "Security and Crash Bugs in Older Versions"; https://www.cvedetails.com/vulnerability-list/vendor_id-7294/Libpng.html https://www.cvedetails.com/vulnerability-list/vendor_id-7294... Regarding performance, I already lost the game before it started because I'm writing Java. If I wanted to squeeze CPU time, I would be writing C/C++/asm. So I decided to aim for conciseness and reliability instead of the endless stream of vulnerabilities.
  - gred 4y ago
    Point made, but I was actually thinking of the default Java platform support for writing PNG files (javax.imageio + zlib, which has a decent track record). https://www.cvedetails.com/product/111843/Zlib-Zlib.html?vendor_id=13265 https://www.cvedetails.com/product/111843/Zlib-Zlib.html?ven...

dmitrygr 4y ago

> which is limited to the range 0 to 2^31-1, to defend against the existence of C programmers. What is this nonsense? You mean Java, right? C has always had unsigned types.

chamwislothe2nd 4y ago

Remember when IE didn't support it? And the goofy hacks Microsoft made people do to support transparency on websites?

giantrobot 4y ago
Oh, IE "supported" PNGs almost maliciously. If you had a True Color PNG? No problem. Paletted 8-bit? No problem. 8-but with alpha? Fine. True Color with alpha? I hope you like seeing your background color you meant to alpha out. You had to use Microsoft's DirectX filtering CSS extensions to properly handle the alpha channel of True Color PNGs.
- masklinn 4y ago
  > 8-but with alpha? Fine. Not exactly, palletised png supports a full alpha channel which did not work with IE. Though you had to work to get that as usually software would limit palletised output to GIF (if you didn’t outright have to create your pngs from gifs).
ksherlock 4y ago
Oh god. filter:progid:DXImageTransform.Microsoft.AlphaImageLoader(...)

deleted 4y ago

[deleted]

galleywest200 4y ago

I found the info-graphics in this repository to be of great use when I was trying to digest different image formats: https://github.com/corkami/pics/blob/master/binary/PNG.png https://github.com/corkami/pics/blob/master/binary/PNG.png

deleted 4y ago

[deleted]