7 ms·

WebBundles are built for content-addressable networks

71 points by adlrocha 6y ago

dsun179 6y ago
Isnt that already possible with mhtml? https://en.m.wikipedia.org/wiki/MHTML https://en.m.wikipedia.org/wiki/MHTML
- kinlan 6y ago
  Mhtml tends to not have JavaScript run with it because there is no origin attached to it. That's one of the benefits of web bundles, they can run with an origin attached so they have access to the correct storage and other sandboxing primitives.
outsomnia 6y ago
I don't really get what the advantage is for making an big atomic blob as the resource vs independently updateable pieces as h2 streams / etags / client cacheable. It's just PUSH gone crazy?
- whoopdedo 6y ago
  Developers were just nostalgic for SWF, I guess.
  - gladskdks 6y ago
    I know I am!
- aclelland 6y ago
  I think one of the reason larger companies like them is because they may end ad blockers once and for all. You can read a discussion about some of the issues that it causes here - https://github.com/WICG/webpackage/issues/551 https://github.com/WICG/webpackage/issues/551 Of course, the Brave browser also have some concerns about it - https://brave.com/webbundles-harmful-to-content-blocking-security-tools-and-the-open-web/ https://brave.com/webbundles-harmful-to-content-blocking-sec...
  - rektide 6y ago
    Widely disputed. If this really did prevent ad blocking no one would expect it to ship. Inside the bundle are the same resources as without a bundle.
    - spankalee 6y ago
      Same resources with the same URLs, access with the same request/response pipeline that's just as visible to blockers. If you can change URLs in the bundle, you can change them outside the bundle. UAs are also completely free to request a resource from the URL rather than the bundle.
  - notatoad 6y ago
    People have been saying "AMP prevents adblockers" since the introduction of AMP, and it has never been true. Advertisers don't want to bundle their ads with the content, they can't make any money that way. they want their ads to be dynamically requested each time they are drawn so they can sell ads based on real-time conditions. A standards-based implementation of AMP won't change this. Brave's assertion that "bundles are like PDF" is actually a pretty good analogy - you don't see a whole lot of ad networks popping up trying to sell ad space in PDFs, do you?
- dmitriid 6y ago
  You have to look at where the proposal comes from. This and Signed Exchanges are Google's rebranding of AMP as "open standards". Signed Exchanges are considered harmful by Mozilla [1] There's concern that Web Bundles are just Google's power play to be the sole gateway to the web [2] [1] https://mozilla.github.io/standards-positions/ https://mozilla.github.io/standards-positions/ [2] https://twitter.com/Rich_Harris/status/1299015418913460226 https://twitter.com/Rich_Harris/status/1299015418913460226
- untog 6y ago
  I can see the advantages, for example I could send you a page as an email attachment without any problems. There’s clearly a use for it because browsers let you do (more or less) this with their own custom formats.
- spankalee 6y ago
  A huge fraction of sites today are already bundled, just poorly. WebBundles solves the problem that devs are already working around with tools. HTML, JavaScript, CSS and images are not natively bundleable. So tools like Webpack and Rollup dramatically transform the files to be able to bundle them. JavaScript is recompiled into a single file so that it's impossible to cache the individual files, and many features like dynamic import, asset fetches, and import.meta.url are partially broken. CSS is concatenated, against breaking caching and preventing the use of individual files in separate CSS scopes. WebBundles actually bundles the real files that would have been sent unbundled, and it works with any file/response type. WebBundles have much better compressibility than the unbundled files. So it's is both a better HTTP/2 push, and a better version of what sites are doing with tools. And since WebBundles work with the existing request/response pipeline, the files are individually fed into the network cache (and blockers, btw), which makes it possible to build delta bundles that only include updated files. This gives you the best of both worlds of bundled and unbundled serving.
  - yencabulator 6y ago
    > JavaScript is recompiled into a single file That world has largely moved to "chunks", where there are multiple bundles loaded on-demand, based on e.g. user navigating to a less-used part of the web app. Going to a single file as a WebBundle would be downgrade in features. Replacing those JS transforming kludges with a reasonable file format sounds useful, though, and allows combining JS+CSS+images in a new way. It does sound awfully lot like HTTP Push, which as far as I know is largely unused because the idea that the server knows what the client wants hasn't really proven true.
    - spankalee 6y ago
      Why would WebBundles be a downgrade? If you currently serve multiple Rollup bundles, you can instead serve multiple WebBundles.
      - yencabulator 6y ago
        See what I quoted and was replying to.
- pcwalton 6y ago
  WebBundles exist so that AMP content can look like it came from the original site instead. (I understand that the technology is technically neutral, but AMP is the practical reason why Google is pushing WebBundles.)
marijn 6y ago
> Brave concerns about WebBundles are legit in a location-based addressing Internet, but all of them would immediately be removed the moment we switch from a location-based addressing to a content-based addressing approach for the Internet. I failed to find any good case being made for why content-addressable content would be any less likely to try to perform malicious actions than URL-addressed content. Is this just utopian wishful thinking or did I miss something?
- sktrdie 6y ago
  I think it’s because with content addressable URLs, the URL is a hash and you can verify that the content never changes? But not 100% sure
  - marijn 6y ago
    Right. But that only works for resources whose initial version is fully trusted (due to review or trust in its source), and which never change. Which doesn't cover a lot of the usefulness of the web.
- spankalee 6y ago
  Brave's concerns aren't legit though. WebBundles don't change the request/response system or origin model of the web. They really don't change URLs or blocker abilities at all. Brave is ascribing them either powers they don't have, or that you can already do with plain servers.
  - hinkley 6y ago
    What’s the origin for addressable content?
    - yencabulator 6y ago
      Whatever Signed HTTP Exchange it can successfully claim. The whole point of this work is to separate origin from where you managed to download the bytes from. Imagine CDNs that cannot forge content for your site.
gumby 6y ago
WebBundles are the CDROMs of 2020 and solve nothing for the end user. It’s all about those ads.
- jrururufuf666 6y ago
  the continued trend of overcomplicating things with 100 frameworks. i miss ftp'ing my html site to geocities
  - jrururufuf666 6y ago
    "build process" for a webpage nowadays.. hahaha i got into web stuff for being interpreted and instant results, and now you gotta compile your bundles.
- spankalee 6y ago
  Are you claiming that everyone should stop using Webpack and Rollup too?
  - gumby 6y ago
    I know that rollup in particular is intended to collect all your JS modules, somewhat like an .a file in your filesystem, but Webpack can easily combine all the assets. In both cases you’re downloading globs of stuff that could already be in your cache, or that you don’t want to download at all (e.g. ads). I have never approved of efforts to turn the web into a “TV remote with a ‘Buy’ button” (which stretches back decades) but indeed, I consider such projects pernicious.
jefftk 6y ago
The Brave post was discussed here extensively a few days ago, with several people (including me) pointing out how the author misunderstands what can be done today and what bundles make easier: https://news.ycombinator.com/item?id=24274968 https://news.ycombinator.com/item?id=24274968 Afterwards I wrote up a response, explaining how bundles don't facilitate adblocker circumvention: https://www.jefftk.com/p/webbundles-and-url-randomization https://www.jefftk.com/p/webbundles-and-url-randomization (Disclosure: I work on ads at Google)
- gorhill 6y ago
  > Everyone who visits the site will need essentially all of these files. Not necessarily. For instance, I wholly block JavaScript and as a result I could avoid downloading ~283 KB of JavaScript resources from your page (out of ~412 KB resources in total). I could read your page fine. If I understand correctly, with a WebBundle there would be no way for me to avoid downloading resources which are explicitly meant to not be downloaded by the way I configured my user agent?
  - jefftk 6y ago
    Nearly all of the JavaScript on my page comes from two third-party loads: ga.js and gpt.js. As I describe in the post, bundling wouldn't work well for including third-party resources (for the same reasons that it is uncommon for sites to proxy third-party resources onto their own domains today). Specifically in the case of first party JavaScript, however, you are right. A site might decide to serve all of its first party resources in a bundle, and people who don't run JS would end up downloading some extra bytes. This is a situation we commonly have already, however, when there is HTML or CSS that is only ever used when activated by JavaScript. I do think you're right that this would mean people in your position would end up downloading slightly more useless bytes, but not by a large fraction?
    - gorhill 6y ago
      > it is uncommon for sites to proxy third-party resources onto their own domains Isn't the purpose of Signed HTTP Exchanges to be able to deliver 3rd-party resources from another domain?
thorum 6y ago
WebBundles seem like they would be a useful format for distributing PWAs - like a web standard alternative to Android's APK files.
skybrian 6y ago
I'm wondering if IPFS or other content-addressable networks handle version updates for documents as well as git handles code? It might be nice to make websites that are more like PDF's that can be redistributed, downloaded, and stored. But when there are many versions of immutable content, the result is a mess, with people having random versions distributed all over the place. Having built-in history and being able to sync to HEAD would make this a lot easier.
- matt_kantor 6y ago
  I believe mutable pointers like that are outside the scope of IPFS itself, instead within the realm of name systems like IPNS[1] and DNSLink[2]. I'm not sure if/how those systems track history. Unsurprisingly, some people want to use blockchains[3][4] (those definitely have history). [1]: https://docs.ipfs.io/concepts/ipns https://docs.ipfs.io/concepts/ipns [2]: https://dnslink.io https://dnslink.io [3]: https://www.namecoin.org https://www.namecoin.org [4]: https://ens.domains https://ens.domains
cordite 6y ago
The repo [1] appears silent for a few months. Is this being actively developed inside the google org, or is this just an experiment left abandon? [1]: https://github.com/google/webbundle https://github.com/google/webbundle
Ericson2314 6y ago
While I get the argument that things vaguely like AMP could be used with IPFS, these WebBundles in particular catting everything together would seem to undermine IPFS's ability to dedup. No thanks.
bonfire 6y ago
So basically we can give up HTTPS for origin authentication (leave it for encryption/privacy) assuming the bundles are all signed?

71 points by adlrocha 6y ago

dsun179 6y ago

Isnt that already possible with mhtml? https://en.m.wikipedia.org/wiki/MHTML https://en.m.wikipedia.org/wiki/MHTML

kinlan 6y ago
Mhtml tends to not have JavaScript run with it because there is no origin attached to it. That's one of the benefits of web bundles, they can run with an origin attached so they have access to the correct storage and other sandboxing primitives.

outsomnia 6y ago

I don't really get what the advantage is for making an big atomic blob as the resource vs independently updateable pieces as h2 streams / etags / client cacheable. It's just PUSH gone crazy?

whoopdedo 6y ago
Developers were just nostalgic for SWF, I guess.
- gladskdks 6y ago
  I know I am!
aclelland 6y ago
I think one of the reason larger companies like them is because they may end ad blockers once and for all. You can read a discussion about some of the issues that it causes here - https://github.com/WICG/webpackage/issues/551 https://github.com/WICG/webpackage/issues/551 Of course, the Brave browser also have some concerns about it - https://brave.com/webbundles-harmful-to-content-blocking-security-tools-and-the-open-web/ https://brave.com/webbundles-harmful-to-content-blocking-sec...
- rektide 6y ago
  Widely disputed. If this really did prevent ad blocking no one would expect it to ship. Inside the bundle are the same resources as without a bundle.
  - spankalee 6y ago
    Same resources with the same URLs, access with the same request/response pipeline that's just as visible to blockers. If you can change URLs in the bundle, you can change them outside the bundle. UAs are also completely free to request a resource from the URL rather than the bundle.
- notatoad 6y ago
  People have been saying "AMP prevents adblockers" since the introduction of AMP, and it has never been true. Advertisers don't want to bundle their ads with the content, they can't make any money that way. they want their ads to be dynamically requested each time they are drawn so they can sell ads based on real-time conditions. A standards-based implementation of AMP won't change this. Brave's assertion that "bundles are like PDF" is actually a pretty good analogy - you don't see a whole lot of ad networks popping up trying to sell ad space in PDFs, do you?
dmitriid 6y ago
You have to look at where the proposal comes from. This and Signed Exchanges are Google's rebranding of AMP as "open standards". Signed Exchanges are considered harmful by Mozilla [1] There's concern that Web Bundles are just Google's power play to be the sole gateway to the web [2] [1] https://mozilla.github.io/standards-positions/ https://mozilla.github.io/standards-positions/ [2] https://twitter.com/Rich_Harris/status/1299015418913460226 https://twitter.com/Rich_Harris/status/1299015418913460226
untog 6y ago
I can see the advantages, for example I could send you a page as an email attachment without any problems. There’s clearly a use for it because browsers let you do (more or less) this with their own custom formats.
spankalee 6y ago
A huge fraction of sites today are already bundled, just poorly. WebBundles solves the problem that devs are already working around with tools. HTML, JavaScript, CSS and images are not natively bundleable. So tools like Webpack and Rollup dramatically transform the files to be able to bundle them. JavaScript is recompiled into a single file so that it's impossible to cache the individual files, and many features like dynamic import, asset fetches, and import.meta.url are partially broken. CSS is concatenated, against breaking caching and preventing the use of individual files in separate CSS scopes. WebBundles actually bundles the real files that would have been sent unbundled, and it works with any file/response type. WebBundles have much better compressibility than the unbundled files. So it's is both a better HTTP/2 push, and a better version of what sites are doing with tools. And since WebBundles work with the existing request/response pipeline, the files are individually fed into the network cache (and blockers, btw), which makes it possible to build delta bundles that only include updated files. This gives you the best of both worlds of bundled and unbundled serving.
- yencabulator 6y ago
  > JavaScript is recompiled into a single file That world has largely moved to "chunks", where there are multiple bundles loaded on-demand, based on e.g. user navigating to a less-used part of the web app. Going to a single file as a WebBundle would be downgrade in features. Replacing those JS transforming kludges with a reasonable file format sounds useful, though, and allows combining JS+CSS+images in a new way. It does sound awfully lot like HTTP Push, which as far as I know is largely unused because the idea that the server knows what the client wants hasn't really proven true.
  - spankalee 6y ago
    Why would WebBundles be a downgrade? If you currently serve multiple Rollup bundles, you can instead serve multiple WebBundles.
    - yencabulator 6y ago
      See what I quoted and was replying to.
pcwalton 6y ago
WebBundles exist so that AMP content can look like it came from the original site instead. (I understand that the technology is technically neutral, but AMP is the practical reason why Google is pushing WebBundles.)

marijn 6y ago

> Brave concerns about WebBundles are legit in a location-based addressing Internet, but all of them would immediately be removed the moment we switch from a location-based addressing to a content-based addressing approach for the Internet. I failed to find any good case being made for why content-addressable content would be any less likely to try to perform malicious actions than URL-addressed content. Is this just utopian wishful thinking or did I miss something?

sktrdie 6y ago
I think it’s because with content addressable URLs, the URL is a hash and you can verify that the content never changes? But not 100% sure
- marijn 6y ago
  Right. But that only works for resources whose initial version is fully trusted (due to review or trust in its source), and which never change. Which doesn't cover a lot of the usefulness of the web.
spankalee 6y ago
Brave's concerns aren't legit though. WebBundles don't change the request/response system or origin model of the web. They really don't change URLs or blocker abilities at all. Brave is ascribing them either powers they don't have, or that you can already do with plain servers.
- hinkley 6y ago
  What’s the origin for addressable content?
  - yencabulator 6y ago
    Whatever Signed HTTP Exchange it can successfully claim. The whole point of this work is to separate origin from where you managed to download the bytes from. Imagine CDNs that cannot forge content for your site.

gumby 6y ago

WebBundles are the CDROMs of 2020 and solve nothing for the end user. It’s all about those ads.

jrururufuf666 6y ago
the continued trend of overcomplicating things with 100 frameworks. i miss ftp'ing my html site to geocities
- jrururufuf666 6y ago
  "build process" for a webpage nowadays.. hahaha i got into web stuff for being interpreted and instant results, and now you gotta compile your bundles.
spankalee 6y ago
Are you claiming that everyone should stop using Webpack and Rollup too?
- gumby 6y ago
  I know that rollup in particular is intended to collect all your JS modules, somewhat like an .a file in your filesystem, but Webpack can easily combine all the assets. In both cases you’re downloading globs of stuff that could already be in your cache, or that you don’t want to download at all (e.g. ads). I have never approved of efforts to turn the web into a “TV remote with a ‘Buy’ button” (which stretches back decades) but indeed, I consider such projects pernicious.

jefftk 6y ago

The Brave post was discussed here extensively a few days ago, with several people (including me) pointing out how the author misunderstands what can be done today and what bundles make easier: https://news.ycombinator.com/item?id=24274968 https://news.ycombinator.com/item?id=24274968 Afterwards I wrote up a response, explaining how bundles don't facilitate adblocker circumvention: https://www.jefftk.com/p/webbundles-and-url-randomization https://www.jefftk.com/p/webbundles-and-url-randomization (Disclosure: I work on ads at Google)

gorhill 6y ago
> Everyone who visits the site will need essentially all of these files. Not necessarily. For instance, I wholly block JavaScript and as a result I could avoid downloading ~283 KB of JavaScript resources from your page (out of ~412 KB resources in total). I could read your page fine. If I understand correctly, with a WebBundle there would be no way for me to avoid downloading resources which are explicitly meant to not be downloaded by the way I configured my user agent?
- jefftk 6y ago
  Nearly all of the JavaScript on my page comes from two third-party loads: ga.js and gpt.js. As I describe in the post, bundling wouldn't work well for including third-party resources (for the same reasons that it is uncommon for sites to proxy third-party resources onto their own domains today). Specifically in the case of first party JavaScript, however, you are right. A site might decide to serve all of its first party resources in a bundle, and people who don't run JS would end up downloading some extra bytes. This is a situation we commonly have already, however, when there is HTML or CSS that is only ever used when activated by JavaScript. I do think you're right that this would mean people in your position would end up downloading slightly more useless bytes, but not by a large fraction?
  - gorhill 6y ago
    > it is uncommon for sites to proxy third-party resources onto their own domains Isn't the purpose of Signed HTTP Exchanges to be able to deliver 3rd-party resources from another domain?

thorum 6y ago

WebBundles seem like they would be a useful format for distributing PWAs - like a web standard alternative to Android's APK files.

skybrian 6y ago

I'm wondering if IPFS or other content-addressable networks handle version updates for documents as well as git handles code? It might be nice to make websites that are more like PDF's that can be redistributed, downloaded, and stored. But when there are many versions of immutable content, the result is a mess, with people having random versions distributed all over the place. Having built-in history and being able to sync to HEAD would make this a lot easier.

matt_kantor 6y ago
I believe mutable pointers like that are outside the scope of IPFS itself, instead within the realm of name systems like IPNS[1] and DNSLink[2]. I'm not sure if/how those systems track history. Unsurprisingly, some people want to use blockchains[3][4] (those definitely have history). [1]: https://docs.ipfs.io/concepts/ipns https://docs.ipfs.io/concepts/ipns [2]: https://dnslink.io https://dnslink.io [3]: https://www.namecoin.org https://www.namecoin.org [4]: https://ens.domains https://ens.domains

cordite 6y ago

The repo [1] appears silent for a few months. Is this being actively developed inside the google org, or is this just an experiment left abandon? [1]: https://github.com/google/webbundle https://github.com/google/webbundle

Ericson2314 6y ago

While I get the argument that things vaguely like AMP could be used with IPFS, these WebBundles in particular catting everything together would seem to undermine IPFS's ability to dedup. No thanks.

bonfire 6y ago

So basically we can give up HTTPS for origin authentication (leave it for encryption/privacy) assuming the bundles are all signed?