13 ms·
FLIF – Free Lossless Image Format
- chris_wot 11y agoI wonder if IE will adopt it? Firefox and Chrome are very responsive, Microsoft not so much.
- sccxy 11y agoSafari is much bigger problem than Edge/IE. No hope for WebM in Safari. WebM for Edge is in development.
- anon1385 11y agoThis is about an image format not a video format, so you probably mean WebP. WebP isn't supported in Firefox, Safari or IE, so it's not like Safari is the lone browser blocking new image formats.
- Loque 11y agoIt really depends on what it is - Edge is missing some rather fundamental modern CSS features (filters immediately springs to mind).
- timsneath 11y agoActually, Edge has CSS filters as of last November: https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/suggestions/6261306-filters?ref=userfacing https://wpdev.uservoice.com/forums/257854-microsoft-edge-dev... Test drive site here: https://dev.windows.com/en-us/microsoft-edge/testdrive/demos/css3filters/ https://dev.windows.com/en-us/microsoft-edge/testdrive/demos...
- Loque 11y agoI apologise, I meant blend modes: https://dev.windows.com/en-us/microsoft-edge/platform/status/backgroundblendmode https://dev.windows.com/en-us/microsoft-edge/platform/status... They have it marked as low priority, even tho it is a standard present in all other browsers. I think that roadmap displays a solid representation that Edge is going to be our next lowest common denominator for webdesign/development for the next few years... I really don't care about its native ES6 support. Maybe SVG 2.0... This is the link of doom for me: https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/filters/top https://wpdev.uservoice.com/forums/257854-microsoft-edge-dev...
- matzipan 11y agoHow often do you need lossless images on the web? Almost never.
- pferde 11y agoIf the compression is good enough, always. Lossy compression was born out of necessity, after all.
- matzipan 11y agoIt was not born out of necessity. It was born because of the way your brain percieves the images. There is no need in adding detail if your brain can live happily without it. And if FLIF achieves such great lossless compression, imagine what amazing compression you can achieve with lossy compression.
- pmontra 11y agoAll those animated GIFs around the web, plus the images were JPEG artefacts would be noticed. Examples: The Y logo in the top left of this page is a GIF. All the logos at http://www.ycombinator.com/ http://www.ycombinator.com/ plus the six images below More Quotes are PNGs.
- matzipan 11y agoForgot PNG is lossless :P
- anon1385 11y agoFirefox is very conservative when it comes to image formats - more so than any of the other browsers.. See WebP[1], JPEG2000[2] and JPEG-XR[3]. I'd be very surprised if they supported this. [1] https://bugzilla.mozilla.org/show_bug.cgi?id=856375 https://bugzilla.mozilla.org/show_bug.cgi?id=856375 [2] https://bugzilla.mozilla.org/show_bug.cgi?id=36351 https://bugzilla.mozilla.org/show_bug.cgi?id=36351 [3] https://bugzilla.mozilla.org/show_bug.cgi?id=500500 https://bugzilla.mozilla.org/show_bug.cgi?id=500500
- chris_wot 11y agoThat's a fairly convincing argument, I definitely need to concede the point here. Appreciate your insight and helpful links!
- p4bl0 11y agoThere was a previous discussion on HN about this: https://news.ycombinator.com/item?id=10317790 https://news.ycombinator.com/item?id=10317790
- baldfat 11y ago5 months ago. This is my first time seeing this item.
- p4bl0 11y agoMy point is not that it shouldn't be there again, it's that some potentially interesting stuffs have already been discussed, so the previous thread is probably worth a look too.
- baldfat 11y agoYou should put that into your first post :) Most of the time I see post like this meaning it was a double post.
- gus_massa 11y agoMy preferred format is: https://news.ycombinator.com/item?id=10317790 https://news.ycombinator.com/item?id=10317790 (1254 points, 157 days ago, 366 comments) Because it answer most of the common questions about a resubmission: Did it get a lot of exposure? Was it a long time ago? Did it have an interesting discussion that I should read? The number of comments actually doesn't show if the discussion was interesting of a flamewar. So I sometimes cherrypick some of the most interesting comments and repost a snippet. (In this case, I think the most important comments are: - The relation between the progressive and an extension for lossy mode - The advantages and problems of the (L)GPLv3 licence But each of them deserve it's own thread here.)
- ekianjo 11y agoThanks Jon for your work on this!
- AshleysBrain 11y agoIf browsers supported APIs to allow "native" image/video/audio codecs to be written in JS, we could support new formats like this without needing any co-operation from the (very conservative) browser vendors. I wrote a proposal for this here: https://discourse.wicg.io/t/custom-image-audio-video-codec-apis/1270 https://discourse.wicg.io/t/custom-image-audio-video-codec-a...
- ekianjo 11y agoSo you would need to load up the JS decompressor every single time you load a webpage? Or is there another way to do it efficiently ?
- pcx 11y agoMay be adding support using an extension/addon would work.
- AshleysBrain 11y agoThat's the downside - but if a JS decompressor library is served from a CDN, it only has to load it on the first site that uses it, like using jQuery from a CDN. Service Workers can also serve content like this from an offline cache, so it doesn't have to hit the server again on later visits.
- billpg 11y agoThe browser would cache a copy. I've seen proposals to add hashes to links. This way, a browser might see a link to some JS on a new URL, but with the hash, it might find it already has that JS file in its cache from when it downloaded it at a different URL.
- imaginenore 11y agoThat's what Etag is for.
- eis 11y ago
- matzipan 11y agoDoes anybody understand how lossless JPEG works? To my mind, the whole point of JPEG is to get rid of high-frequency components.
- pjc50 11y agohttps://en.wikipedia.org/wiki/Lossless_JPEG https://en.wikipedia.org/wiki/Lossless_JPEG It's a completely different algorithm, surprisingly, based around "predictors" from nearby previously-decoded pixels. Quite like Floyd-Steinberg dithering.
- Houshalter 11y agoThis isn't how it actually works, but one way to turn a lossy decoder into a lossy one, is to send a diff of the actual pixels vs the encoded ones. Since the lossy compression will be close to correct, the diff will mostly be small values or 0's. Which is much easier to compress. Likewise you can turn any lossless compressor into a lossy one, by modifying the pixels that are the hardest to compress. E.g. if there is a random red pixel in a group of blue pixels, you can make it blue, and save up to 3 bytes. Or you can discard color information that humans aren't very sensitive too anyway, like JPEG does. All lossless means is that the compression isn't required by the format itself.
- tambourine_man 11y agoKind of. Have you tried running a PNG compressor on a JPEG file? It is smaler than a straight to PNG file but nowhere near the size of the original JPG.
- Houshalter 11y agoWell of course, JPEG artifacts aren't necessarily going to be easier for png to compress. You need to make modifications designed for png's algorithm. There are some tools that do this: https://pngmini.com/lossypng.html https://pngmini.com/lossypng.html https://pngquant.org/ https://pngquant.org/ https://tinypng.com/ https://tinypng.com/ The resulting png's are much smaller. Though not necessarily as small as JPEG, it's in the same ballpark.
- panic 11y agoThis looks promising! They ought to include time-to-decode in the performance numbers, though: a smaller compressed size doesn't matter if the process of loading and displaying the image takes more time overall. A graph like the ones on this page would be awesome: http://cbloomrants.blogspot.com/2015/03/03-02-15-oodle-lz-pareto-frontier.html http://cbloomrants.blogspot.com/2015/03/03-02-15-oodle-lz-pa...
- davej 11y agoThis! Especially with the web being increasingly consumed on low-powered mobile devices. It's also not just about being faster than the network because the browser is probably simultaneously doing page layout, javascript parsing, image decompression and a ton of other things. Also more time spent decoding images is a drain on the battery. Focusing solely on compression stats can be misleading, it's a balancing act. For example, I'm not sure being 0.7x smaller than PNG is much of a win if FLIF ends up being an order of magnitude slower. Perhaps it is still a win but it needs a bit more nuance in the analysis to reach that conclusion.
- jaredcheeda 11y agoThe FLIF format isn't finished yet. There are optimizations still being made both in compression ratio and decoding performance. Encoding performance is a lower priority. Also it wouldn't really be a fair comparison to show this slowly maturing format's decoding time versus that of formats that have had decades of time for people to research and find the best ways for decoding (even hardware tailored). You're comparing the heavy weight champ against the up and comer with a lot of promise. FLIF would get pretty beat up in that comparison, but perhaps only because it isn't even finished yet, and thus hasn't had the time to reach that level of efficiency. From having used it, and played around with it, it seems to be pretty fast. Fast enough that people wouldn't notice a perceptible difference between it and other common formats. I think the current decode time is about 20 MegaBytes per second. Which is faster than most people's internet, and considerably larger than most people's images (on the web). Once it's bitstream has been finalized and it starts to get some use, we'll see decoders improve their performance. The initial plan was to lock the bitstream for a 2016 release, and then in a year or two release a new bitstream with further improvements. Similar to GIFv87a (1987 version) and GIFv89a (1989 version with improvements and new features). However this may have changed as the FLIF 2016 release was set for late December, then February, and has been pushed back again. In due time we'll see more accurately how the format compares to the existing champs in terms of performance.
- thenomad 11y agoHmm, this has real potential as an archiving format for video, too. Any news on what the processing overhead is like for viewing rather than creating the files? Is it less than PNG?
- Zardoz84 11y ago16 bit per channel and on future support for CMYK. Looks like a interesting alternative to TIFF for digital preservation. Sadly, the actual recommended format is TIFF (so, waste of storage space) -> http://www.loc.gov/preservation/resources/rfs/stillimg.html http://www.loc.gov/preservation/resources/rfs/stillimg.html
- vanderZwan 11y ago> (so, waste of storage space) To be fair, given the speed at which storage space has grown over the years, it's not really something to worry about in the context of archiving material for future generations (which is very different than being able to quickly download something on the internet now, for example)
- Zardoz84 11y agoWell, when you do service archiving material to the web, the storage cost isn't irrelevant Also, big files means more network traffic between our webservers and the storage servers. Plus this format have other interesting features, like tiled rendering or the progressive download. Sadly, we must handle huge TIFFs and generate jpeg miniatures and tilesets from it, if not serving it online would be very painful. Note: I work on a company dedicated to archiving of libraries, museums and archives...
- vanderZwan 11y agoAh ok, cool! But then you're talking about trying to meet both goals I mentioned at the same time ;) But I guess you handle it similar to how Archive.org does it, with a large TIFF as a poorly compressed but lossless "base case" and other compression formats for the web?
- sjwright 11y agoIncredible work. My only comment is that the progressive loading example reveals that their algorithm seems to have desirable properties for lossy compression as well. Why not make FLIF support lossy and lossless? It's hard enough to get a new image format standardized as it is; offering a lossy mode would effectively give us a two-for-one deal. If PNG had a lossy mode that was even slightly better than JPEG (or exactly as good but with full alpha channel support) it would have eventually supplanted JPEG just as it has now supplanted GIF.
- akavel 11y agoQuoting a fragment from the final section: "[...] any prefix (e.g. partial download) of a compressed file can be used as a reasonable lossy encoding of the entire image." though they also have it listed explicitly in the TODO section: "- Lossy compression"
- jaredcheeda 11y agoCurrent plan is to keep lossless and lossy versions of the format in the same bitstream for encoders/decoders and to differentiate the output files encoded to be lossy as ".flyf" (Free LossY image Format), and those that are encoded as lossless as ".flif" (Free Lossless Image Format). However renaming the files from flif to flyf or vice versa would have no effect, they would still be opened and decoded the same. It's merely meant to convey to humans the intention of the person who created the image. From my understanding, not much focus has been given to lossy endcoding as of yet.
- fsiefken 11y agoImpressive benchmarks, but how would this compare to lossless VP9, VP10 or h264, h265 image compression?
- eddieh 11y agoHonest question. Seriously not trying to dismiss the work. Why not TIFF? 30 years old, already built into nearly every graphics application, supports everything this proposes and more. Plus it is already supported in Safari. https://en.wikipedia.org/wiki/Tagged_Image_File_Format https://en.wikipedia.org/wiki/Tagged_Image_File_Format
- Zardoz84 11y agoBecause isn't open source, and is a proprietary format from Adobe ?
- techdragon 11y agoWhile not Open Source TIFF has age advantages in that close to anything related to it is unencumbered by patents, or will soon cease to be if any do remain. Also TIFF is an open container like MKV and can be used for far more than just simple pictures.
- colejohnson66 11y agoNot to mention TIFF supports layers. From a skim of the homepage, this doesn't.
- jaredcheeda 11y agoFLIF allows for animation, and it is very possible to interpret each frame of the animation as it's own layer. This would year the same lossless quality as a TIFF at a much smaller file size. Though FLIF isn't intended to be used for this purpose, neither was PNG, and that didn't stop Fireworks from abusing that format and adding layering. The same could be done by someone who wants to add a little meta data to the file to let layered flif opening applications know that the frame are to be interpreted in that manner. This format isn't complete yet, if layering is important to you, you should join the gitter.im/FLIF-hub/FLIF chat room.
- 11y ago
- thealistra 11y agoEncoding and decoding speeds are acceptable, but should be improved
- have_faith 11y agoYou must be from the Ministry of Encoding.
- teh_klev 11y agoDo any of these newer/experimental schemes, such as this one, take into account other factors such as CPU load before declaring themselves as "better". For example this project seems pretty cool, but there's no data on how CPU bound, memory bound, I/O bound its decompression algorithm is. I guess what I'm asking is, if I hit a web page with 20 images @ 100k per image is it going to nail one or more cores at 100% and drain the battery on my portable device. Fantastic compression is great but what are the trade offs?
- dave2000 11y agoIt says very clearly a number of times that it's better in terms of compression ratio.
- teh_klev 11y agoIt says: "Encoding and decoding speeds are acceptable, but should be improved" But that doesn't address resource usage such as CPU, battery.
- dalke 11y agoIt's hard for a single author/developer to fully explore solution space. For example, it could be that Javascript acquires special primitives to decode these images, or that they be a common Javascript engine extension, with a polyfill fallback. It could be that these extensions make effective use of CPU-specific instructions, or that the underlying hardware contains special silicon to decode these images, similar to how there are H.264 decoder chips. It's therefore possible to do the power consumption analysis, but the results would not really indicate something fundamental about the algorithm but only its current implementation. People are willing to work on improved implementations if there are other factors, like smaller size, which suggest it's worthwhile to investigate.
- teh_klev 11y agoAppreciate the time spent answering this.
- kutkloon7 11y agoPretty amazing! Particularly nice is that an alpha channel and animation are also possible. One critical sidenode: it seems FLIF is still not as good as JPEG when used as lossy compression (this is something the benchmarks do not show well). For example, go to http://uprootlabs.github.io/poly-flif/ http://uprootlabs.github.io/poly-flif/, choose the monkey image, choose 'comparing with same size JPG', and set truncation to 60% or more. Also, I'm not sure how efficient en- and decoding is for FLIF.
- onion2k 11y agoIn the example you suggest using the hairs on the left side of the monkey's face have some significant artifacts. The quality of the jpeg is about the same as the FLIF file with a truncation of 80%, at which point the filesize is less than half the size of jpeg at 60%. If the image quality of the 60% truncated jpeg is acceptable then you can get the same quality but half the size using FLIF at 80%.
- kutkloon7 11y agoI don't understand your argument. The images are the same size. A 235KB jpeg of the monkey is almost indistinguishable from lossless, while in a 470KB flif, there are already unacceptable artifacts.
- IshKebab 11y agoNot sure what you're trying to say exactly, but at the same file size ('Compare with same size JPEG' + 60% truncation), FLIF is obviously far worse than JPEG.
- aheeki 11y agohello pied piper
- mitchtbaum 11y agoFor browser support (Servo), FLIF has an issue pointing to Rust's common image library: https://github.com/FLIF-hub/FLIF/issues/142 https://github.com/FLIF-hub/FLIF/issues/142
- jaredcheeda 11y agoFor clarification, on the FLIF GitHub "Issues page", there is a ticket indicating an intention to build FLIF into FireFox's rendering engine Gecko, and also Servo (which will eventually replace Gecko). This ticket is a placeholder declaring Servo's interest in FLIF once it is finalized. FLIF does not have an problem preventing it from pointing to a common Rust image library. (which is how I originally read it).
- praeivis 11y agohttps://xkcd.com/927/ https://xkcd.com/927/
- gr3yh47 11y agorelevant xkcd: https://xkcd.com/927/ https://xkcd.com/927/
- wmu 11y agoLooks amazing! Really impressive results. Very cool that progressive loading is composed into the format. However I am afraid that without support from biggest companies the format will never gain popularity. Just think how long it took to make PNG a web standard. And animated PNGs? Died unnoticed. To make things worse, GIF, a stinking leftover from '90, is still in use (even on HN!).
- orlyb 11y agoFLIF really is awesome :) Here's an analysis that compares FLIF to other common lossless image formats such as: PNG, WebP and BPG. http://cloudinary.com/blog/flif_the_new_lossless_image_format_that_outperforms_png_webp_and_bpg http://cloudinary.com/blog/flif_the_new_lossless_image_forma...
- tomtheguvnor 11y agoApparently it relies on a novel new "middle out" compression algorithm.
- monkmartinez 11y agoHere is the paper[1] regarding the "middle out" or "tip to tip efficiency" algorithm. [1]http://www.scribd.com/doc/228831637/Optimal-Tip-to-Tip-Efficiency http://www.scribd.com/doc/228831637/Optimal-Tip-to-Tip-Effic...
- delinka 11y ago"Middle out" is not "tip-to-tip efficiency." Though they were indeed invented (discovered?) by the same team at approximately the same time, and may also be linked (depending on one's scientific, mathematical and/or philosophical definition of 'linked') they are independent algorithms solving radically different problems.
- tomtheguvnor 11y agoSome people have WAY too much time on their hands...
- pornel 11y agoEveryone loves the "responsive loading" feature, but that's not even the novel thing about the format (JPEG 2000 did it even better — 16 years ago)! The novel feature of this format is better entropy coding. FLIF decoder adds interpolation to make incomplete scans look nicer than in PNG, but that's a feature of the decoder, not the file format, so there's nothing stopping existing PNG decoders from copying that feature. Note that it's generally not desirable to have FLIF used on the Web. A decent-quality JPEG will load at full resolution quicker than it takes FLIF to show a half-resolution preview. FLIF is a lossless format, and lossless is a very hard and costly constraint. Images that aren't technically lossless, but look lossless to the naked eye can be half the size. e.g. Monkey image from https://uprootlabs.github.io/poly-flif/ https://uprootlabs.github.io/poly-flif/ is 700KB in FLIF, but 300KB in high-quality JPEG at q=90 without chroma subsampling (i.e. settings good even for text/line-art), and this photo looks fine even at 140KB JPEG (80% smaller than FLIF). So you want FLIF for archival, editing and interchange of image originals, but even the best lossless format is a waste of bytes when used for distribution to end users.
- JoshTriplett 11y ago> Note that it's generally not desirable to have FLIF used on the Web. A decent-quality JPEG will load at full resolution quicker than it takes FLIF to show a half-resolution preview. On the other hand, this allows browsers on metered-bandwidth connections to control bandwidth more effectively. Rather than disabling images entirely, this would allow loading a low-resolution version and stopping, and letting the user control whether to load the rest of the image.
- nfriedly 11y agoI would love to see this. Instead of network operators throttling bandwidth and/or re-compressing images, set a sane default and let the user have ultimate control.
- pornel 11y agoAgain, this is not new in FLIF, and it isn't a strength of the format. You're describing exactly what can already be done with progressive JPEG better. Look at https://uprootlabs.github.io/poly-flif/ https://uprootlabs.github.io/poly-flif/ - set truncation to 80-90% and compare to Same Size JPEG. Truncated FLIF looks like a pixelated mess, whereas JPEG at the same byte size is almost like the original (note that the site has encoded JPEG to have few progressive scans, so sometimes you get half image perfect and half blocky. This is configurable in JPEG and could be equalized to entire image being ok-ish).
- IgorPartola 11y agoAwesome! So when can I have it in browsers?
- matheweis 11y agoI'm curious about patent/ licensing restrictions. From what I gather it is patent free and the implementation is GPLv3? Does this mean someone else could make a compatible encoder/decoder with a less restrictive license?
- mistercow 11y agoIt's lgpl3, which is less restrictive than gpl3.
- matheweis 11y ago... but still too restrictive for many (most?) commercial applications. However (link got changed?) I now see that the decoder is also available under the Apache 2.0 license, so that is useful.
- ericfrederich 11y agoIf you're just using it, it isn't restrictive at all is it? You just have to say that you used it. If you modify it, you must provide source code for the version that was distributed. Not the source code for anything else. ... unless I totally misunderstand LGPL
- matheweis 11y agoI am not a lawyer, but, yes, the LGPLv3 in particular is in fact quite restrictive relative to what is believed (LGPLv2 much less so). The driving force behind the GPL-series of licenses is to maintain the GNU/Stallman freedoms [1], including "freedom to run the program as you wish, for any purpose" (0) and "to study how the program works, and change it so it does your computing as you wish" (1) It is widely believed that any software implementing DRM on it's runtime code is incompatible with the (L)GPLv3, in particular signed firmware distributions or software distribution systems such as the Mac/iOS App Stores or Steam. The (L)GPLv3 was actually written with this in mind, with some of the authors calling it Tivoization [2] in reference to Tivo's locked down firmware. The relevant legal jargon is in section 6 of the GPL [3], which the LGPL is built on top of, and states: “Installation Information” for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. Again I am not a lawyer, but in other words, you must cough up your signing keys (which for Mac/iOS devs is incidentally a breach of your Apple Developer contract) in order to legally distribute signed software that uses LGPL libraries (-). [-] It seems it should be OK to distribute non-DRM protected software, but for general consumer software this kind of distribution seems to be on it's way out in a hurry. [1] http://www.gnu.org/philosophy/free-sw.html http://www.gnu.org/philosophy/free-sw.html [2] https://en.wikipedia.org/wiki/Tivoization#GPLv3 https://en.wikipedia.org/wiki/Tivoization#GPLv3 [3] http://www.gnu.org/licenses/gpl-3.0.en.html http://www.gnu.org/licenses/gpl-3.0.en.html
- ipunchghosts 11y agoThis is probably the 3rd "replacement" for JPEG I have seen on HN in the last few years. None of these formats have been supported by common browsers. When will this stuff start making its way to the desktop?
- ktRolster 11y agoIt's good that browsers aren't too quick to add support for new image formats, otherwise we'd have a lot of bloatware for non-optimal image formats that few people use.
- mrob 11y agoI'd like to see some comparisons with palettized PNGs. All the demo images for poly-flif use more than 256 colors, but diagrams and line art sometimes use 256 colors or less, which means they can be stored in an 8-bit palettized PNG losslessly. People often forget about this when optimizing PNG sizes, and most graphics software saves as RGB by default even when the image will fit in 8-bit palettized.
- jonsneyers 11y agoFLIF can also do palettes, without arbitrary limits on the palette size. It will automatically use palettes if the image has sparse colors (and "color buckets" if it has too many colors for a palette but still relatively few). It tends to be better than PNG in terms of compression, also for palette images. I haven't looked at optimizing palette ordering yet though, so there is probably some more margin for improvement here.
- mattybrennan 11y agoBut what's the Weissman score?
- gsmethells 11y agoWhen is this going to be available for DICOM medical images? :) http://dicom.nema.org http://dicom.nema.org
- r0m4n0 11y agoSeems cool. Slightly off topic but I hate when someone names a file format with "format" or "file" in the name. Isn't it a bit redundant to include format in the format? Something that has always bothered me about PDF.
- thebeardisred 11y agoKudos on this @jonsneyers! I've been looking at it ever since we talked at FOSDEM. Glad to see you getting some press on the work and good luck with Uproot Labs!
- jonsneyers 11y agoHi, redbeard is it? I think you misremember something, I didn't start working at Uproot Labs but at Cloudinary. As you can see we went for Apache2 (decoder) and LGPLv3 (encoder) shortly after FOSDEM.
- cyborgx7 11y agoLooks neat, but recently I discovered farbfeld[1] and I think I'll be sticking with that for the time being. I'm starting to believe data-specific compression algorithms are the wrong way to go. [1]http://tools.suckless.org/farbfeld/ http://tools.suckless.org/farbfeld/