5 ms·

If browsers supported APIs to allow "native" image/video/audio codecs to be written in JS, we could support new formats like this without needing any co-operati

If browsers supported APIs to allow "native" image/video/audio codecs to be written in JS, we could support new formats like this without needing any co-operation from the (very conservative) browser vendors. I wrote a proposal for this here: https://discourse.wicg.io/t/custom-image-audio-video-codec-apis/1270 https://discourse.wicg.io/t/custom-image-audio-video-codec-a...

ekianjo 11y ago
So you would need to load up the JS decompressor every single time you load a webpage? Or is there another way to do it efficiently ?
- pcx 11y ago
  May be adding support using an extension/addon would work.
- AshleysBrain 11y ago
  That's the downside - but if a JS decompressor library is served from a CDN, it only has to load it on the first site that uses it, like using jQuery from a CDN. Service Workers can also serve content like this from an offline cache, so it doesn't have to hit the server again on later visits.
- billpg 11y ago
  The browser would cache a copy. I've seen proposals to add hashes to links. This way, a browser might see a link to some JS on a new URL, but with the hash, it might find it already has that JS file in its cache from when it downloaded it at a different URL.
  - imaginenore 11y ago
    That's what Etag is for.
    - eis 11y ago
      No it's not. Etag is completely different. It's a tag that the webserver sends together with the response and which the browser sends to the server which can then return a 304 status code to indicate that the resource has not changed and the browser can use the cached copy. What the parent spoke about is adding some attribute to an anchor tag which specifies the hash of the resource so the browser can do safe cross domain caching without needing to do any request whatsoever.
      - techdragon 11y ago
        Just what we need... More ways to shoot our own feet off while trying to improve performance.
        eis 11y ago
        First I don't immediately see how to shoot your feet off with this but maybe you can elaborate a bit on why you think so. And secondly, am I getting this right that you are in favor of dumbing everything down while also sacrificing performance on the way because someone could break something?
        techdragon 11y ago
        I'm hardly in favour of dumbing things down, or sacrificing performance. I'm just frequently exposed to issues that are 'magically fixed' by purging the entire browser cache. And I'm just unnerved by the idea of this hash link scheme, we have enough bugs and gaps in the existing system without additional complexity, and that system is pretty damn simple conceptually, yet the bugs remain. Anecdotally, I used to ship jQuery from a CDN for one project, I had to stop for that project I forget why... But I do remember that to my surprise, it turned out that when I shipped my own version of jQuery from my own domain, I got 10% less client side errors reported back to me in Sentry. The world is full of ways to shoot off your own foot, some of them even start out with someone showing you how they don't shoot their foot off.
- deleted 11y ago
  [deleted]
pjc50 11y ago
Probably the best way to get this implemented is to encourage people to do the "polyfill"(+) solution of rendering to canvases. Browser implementers tend to follow actual usage, so the more widespread it is the more you can argue fait accompli. (+) Does this share etymology with Polyfilla?
- MyNameIsFred 11y ago
  re: Pollyfilla Yes, I believe that's the idea.
panic 11y ago
Please, no! The web already makes my phone heat up enough as it is.
- vram22 11y ago
  Just had a wild thought: I wonder how different the web would be [1] if all data was sent in binary instead of some (text, HTML, JS) being sent in text format. Or do some servers and clients gzip the text automatically before sending it. I've heard of stuff like that (and seen HTTP headers related to it), but haven't looked deeply into it. [1] I mean in terms of speed. But there could be tradeoffs, because, though sending (smaller) binary data would be faster, the parts of it that represented text would have to converted to/from text at either end.
  - Vendan 11y ago
    almost all servers support compression nowadays, and most clients as well. The HTTP headers and such won't get compressed, but there's a very good chance that every page and text asset you get has been compressed in transit. Some servers even have the option of precompressing assets and delivering them without having to stream through a compression algorithm, but that only works with static assets.
    - vram22 11y ago
      Interesting, that second point. Wonder how it is done. Via some background process that keeps monitoring asset dirs for new files, or does the user have to run a tool for that whenever they add new assets.
      - Vendan 11y ago
        However you choose to do so, i.e. usually via your deployment tool or something similar. One reference point: http://nginx.org/en/docs/http/ngx_http_gzip_static_module.html http://nginx.org/en/docs/http/ngx_http_gzip_static_module.ht...
staz 11y ago
Seems like a good way to get a bazillion new formats that we won't be able to read anymore in 5 years.
- Houshalter 11y ago
  But that's another advantage. Each image would have to be distributed alongside it's javascript decoder. Even if the format is abandoned, the decoder will still be there, and it will still work on that webpage.
  - ant6n 11y ago
    Which means you need JavaScript just to look at the file. Does web archive store js along with the content?
    - acdha 11y ago
      Yes – modern web archiving tools capture JavaScript, CSS, etc. The basic support where they just download URLs found in HTML has been around for years but the current best-practice is to use a real browser engine (e.g. PhantomJS, Firefox) which has either been integrated to store directly or simply is using a proxy like https://github.com/internetarchive/warcprox https://github.com/internetarchive/warcprox which adds every requested resource to a WARC file (https://en.wikipedia.org/wiki/Web_ARChive https://en.wikipedia.org/wiki/Web_ARChive).
    - andyana 11y ago
      If they don't, they should.
dspillett 11y ago
As a hacky workaround you can have the JS decompressor replace img tags with a canvas and draw on that, or create a data or blob URI and apply that to the img tag (I'm not sure which would be least efficient). I assume that is how https://uprootlabs.github.io/poly-flif/ https://uprootlabs.github.io/poly-flif/ is adding support. I might have to have a browse around that project's code when I have some free time. I doubt it would be practical for production use, but you never know...
- JonnieCache 11y ago
  That's how the BPG demo worked.

ekianjo 11y ago

So you would need to load up the JS decompressor every single time you load a webpage? Or is there another way to do it efficiently ?

pcx 11y ago
May be adding support using an extension/addon would work.
AshleysBrain 11y ago
That's the downside - but if a JS decompressor library is served from a CDN, it only has to load it on the first site that uses it, like using jQuery from a CDN. Service Workers can also serve content like this from an offline cache, so it doesn't have to hit the server again on later visits.
billpg 11y ago
The browser would cache a copy. I've seen proposals to add hashes to links. This way, a browser might see a link to some JS on a new URL, but with the hash, it might find it already has that JS file in its cache from when it downloaded it at a different URL.
- imaginenore 11y ago
  That's what Etag is for.
  - eis 11y ago
    No it's not. Etag is completely different. It's a tag that the webserver sends together with the response and which the browser sends to the server which can then return a 304 status code to indicate that the resource has not changed and the browser can use the cached copy. What the parent spoke about is adding some attribute to an anchor tag which specifies the hash of the resource so the browser can do safe cross domain caching without needing to do any request whatsoever.
    - techdragon 11y ago
      Just what we need... More ways to shoot our own feet off while trying to improve performance.
      - eis 11y ago
        First I don't immediately see how to shoot your feet off with this but maybe you can elaborate a bit on why you think so. And secondly, am I getting this right that you are in favor of dumbing everything down while also sacrificing performance on the way because someone could break something?
        techdragon 11y ago
        I'm hardly in favour of dumbing things down, or sacrificing performance. I'm just frequently exposed to issues that are 'magically fixed' by purging the entire browser cache. And I'm just unnerved by the idea of this hash link scheme, we have enough bugs and gaps in the existing system without additional complexity, and that system is pretty damn simple conceptually, yet the bugs remain. Anecdotally, I used to ship jQuery from a CDN for one project, I had to stop for that project I forget why... But I do remember that to my surprise, it turned out that when I shipped my own version of jQuery from my own domain, I got 10% less client side errors reported back to me in Sentry. The world is full of ways to shoot off your own foot, some of them even start out with someone showing you how they don't shoot their foot off.
deleted 11y ago
[deleted]

pjc50 11y ago

Probably the best way to get this implemented is to encourage people to do the "polyfill"(+) solution of rendering to canvases. Browser implementers tend to follow actual usage, so the more widespread it is the more you can argue fait accompli. (+) Does this share etymology with Polyfilla?

MyNameIsFred 11y ago
re: Pollyfilla Yes, I believe that's the idea.

panic 11y ago

Please, no! The web already makes my phone heat up enough as it is.

vram22 11y ago
Just had a wild thought: I wonder how different the web would be [1] if all data was sent in binary instead of some (text, HTML, JS) being sent in text format. Or do some servers and clients gzip the text automatically before sending it. I've heard of stuff like that (and seen HTTP headers related to it), but haven't looked deeply into it. [1] I mean in terms of speed. But there could be tradeoffs, because, though sending (smaller) binary data would be faster, the parts of it that represented text would have to converted to/from text at either end.
- Vendan 11y ago
  almost all servers support compression nowadays, and most clients as well. The HTTP headers and such won't get compressed, but there's a very good chance that every page and text asset you get has been compressed in transit. Some servers even have the option of precompressing assets and delivering them without having to stream through a compression algorithm, but that only works with static assets.
  - vram22 11y ago
    Interesting, that second point. Wonder how it is done. Via some background process that keeps monitoring asset dirs for new files, or does the user have to run a tool for that whenever they add new assets.
    - Vendan 11y ago
      However you choose to do so, i.e. usually via your deployment tool or something similar. One reference point: http://nginx.org/en/docs/http/ngx_http_gzip_static_module.html http://nginx.org/en/docs/http/ngx_http_gzip_static_module.ht...

staz 11y ago

Seems like a good way to get a bazillion new formats that we won't be able to read anymore in 5 years.

Houshalter 11y ago
But that's another advantage. Each image would have to be distributed alongside it's javascript decoder. Even if the format is abandoned, the decoder will still be there, and it will still work on that webpage.
- ant6n 11y ago
  Which means you need JavaScript just to look at the file. Does web archive store js along with the content?
  - acdha 11y ago
    Yes – modern web archiving tools capture JavaScript, CSS, etc. The basic support where they just download URLs found in HTML has been around for years but the current best-practice is to use a real browser engine (e.g. PhantomJS, Firefox) which has either been integrated to store directly or simply is using a proxy like https://github.com/internetarchive/warcprox https://github.com/internetarchive/warcprox which adds every requested resource to a WARC file (https://en.wikipedia.org/wiki/Web_ARChive https://en.wikipedia.org/wiki/Web_ARChive).
  - andyana 11y ago
    If they don't, they should.

dspillett 11y ago

As a hacky workaround you can have the JS decompressor replace img tags with a canvas and draw on that, or create a data or blob URI and apply that to the img tag (I'm not sure which would be least efficient). I assume that is how https://uprootlabs.github.io/poly-flif/ https://uprootlabs.github.io/poly-flif/ is adding support. I might have to have a browse around that project's code when I have some free time. I doubt it would be practical for production use, but you never know...

JonnieCache 11y ago
That's how the BPG demo worked.