4 ms·

I am truly perplexed as someone outside of the Javascript ecosystem; why are there so many incompatible bundlers? If you look at most compiled languages they ha

I am truly perplexed as someone outside of the Javascript ecosystem; why are there so many incompatible bundlers? If you look at most compiled languages they have a set ABI / executable image format, and you just use a link editor (either compile time, run time, or both). Is it just because most Javascript developers have never learnt from any of the lessons that came from decades of compiled languages? (compilers, compiler tools, operating system and kernel development, etc). Is there some benefit that Javascript bundlers have that I'm unaware of? Truly curious.

ponyous 3y ago
Because there were browsers and no standards. How can you expect someone that starts coding on the web to know what pains kernel went through decades prior?
- tankenmate 3y ago
  But surely the people writing the browser code thought about the ecosystem they were creating / trying to create?
  - moron4hire 3y ago
    Browsers weren't written in a day. Technically speaking, Mozilla Firefox is a ship of Theseus going back to the release of Netscape in 1994. Did browser and internet infrastructure developers in the early 90s understand that these things would become rich application platforms? Looking at the history of HTTP, it's clear that they expected some concept of "application" to be delivered through the browser. While there's certainly a chance at least a few of them foresaw the full scope of what that would mean (it's not like X11 remoting wasn't a thing), I don't think most of the people involved were thinking much past 10 years (The Distant Future, the Year 2000). JavaScript was apocryphally "invented in 10 days", it came as an attempt to create competitive advantage, not to create a global standard. The first JavaScript came a year (1995) after the first Netscape, but the first major JS-heavy application didn't come for another 13 years (Google Maps, 2008).
    - tracker1 3y ago
      There were a lot of JS heavy web applications prior to Google Maps. From around the release of IE5 at the end of 2000 in particular through the long tenure of IE6. Having worked on some JS heavy applications around that time. It was also much harder as you had a relatively wide variety of browsers and versions. Since people on dialup were far less likely to update their browsers regularly (or at all beyond what came on an ISP or AOL CD. Of course, the efforts for larger dev teams, optimizations and bundling were far less popular before then. Can't tell you how many poorly written sites/apps carried who knows how many versions/copies of JQuery for example. It was really bad for a while. Now, at least there are more paying some attention to it. There's still some relatively large bundles that are easy to get overloaded. I mean as soon as you load any charting or graphing library it blows out everything else. Of course this is offset between bandwidth and compute advancements as well. There was a popular developer site around 1998-2000 or so called 15seconds.com as that was the average point at which users would start dropping off from a load. Now that's measured at around a second or two.
      - moron4hire 3y ago
        Yeah, I worked on some of them, too. They were even in mapping. I had written one of the first, dynamic, 2D drawing libraries for JavaScript, long before Canvas2D, but it was hidden in proprietary consultoware. The significance of Google Maps was that A) it had all the parts that we would recognize today as a JS single-page app, B) it had no alternative interface that people could opt to use instead (diluting the usage of that particular implementation versus the product as a whole), C) it had broad appeal and adoption, and D) it was significantly better than competitors specifically because of the "SPA-ness" of the app. Google Maps had the features and penetration necessary to change the public perception at large of what could be done with browser-based apps.
    - june_twenty 3y ago
      Was Google Maps the first JS-heavy app? That's a TIL for me..
      - 3y ago
  - biorach 3y ago
    nope well.... not for a surprisingly long time
moron4hire 3y ago
I think it's largely a three-fold problem of the fact that most JS apps are still deployed through browsers and not installed, the fact that HTTP2 was not the panacea of multi-request happy good times it was made out to be, plus the fact that there is no Application Binary Interface, everything gets deployed as source code. This creates a situation where you need bundlers, whereas other languages don't have the concept at all, just to be able to minimize download time (and honestly, while we end up making rather large apps in comparison to web pages, they're pretty small in comparison to other kinds of applications), and then bundles are too opaque to share common code between applications. And because there's no chance to benefit across projects from sharing, there's no force driving standardization of bundling, or adoption of said standard.
brundolf 3y ago
1. Compiling/building is an ~optional layer on top of the core language/standard (which has become less optional over time) 2. Running JS outside of the browser is similarly a layer that was built before any kind of standard existed for it (which still doesn't, really) The browser standards are the only real standards. Everything else (which has turned into a lot) is "standard by implementation". Implementations usually try to agree with each other, because that's obviously beneficial for everybody, but sometimes they make choices to deviate either out of necessity or in an attempt to improve the ecosystem So it's all pretty ad-hoc, but in practice most things are mostly compatible most of the time. They orbit the same general things, and the orbit has narrowed in the last few years as most of the big problems have been solved and the community is zeroing in on the agreed solutions (with the caveat of often having to maintain legacy compatibility) Deno takes a stricter philosophy than most, where it prescribes a ~good way that everything should be done (which is almost entirely based on browser standards which have evolved since all this started), even though it runs outside of a browser, and requiring the ecosystem to fall in line Bun on the other hand takes a maximalist approach to compatibility; it does its best to make everything from every other sub-ecosystem Just Work without any modifications
qbasic_forever 3y ago
Bundling is totally different from linking and building for native platforms. Bundling is all about optimizing code to be sent over a small pipe--you're combining multiple compilation units into one file (so just one web request and lower latency) and doing optimizations like tree shaking to send only the code that's actually used. It's a pretty unique use-case that not many other programming languages deal with or care about. It's almost as if you are a demo scene coder trying to optimize for the absolute smallest code possible. Linking native code doesn't really care about optimizing the size of the output and is just just trying to make sure all code paths can be called with some kind of code.
- colonwqbang 3y ago
  A native compiler also optimises the code. A native static linker also tries to omit unused library data and code. It's absolutely not the case that native devs don't care about code size (we do!) I guess Javascript uses a slightly unusual executable format, text instead of binary. Otherwise, it seems like very much the same thing?
  - crabmusket 3y ago
    I'd say that bundler design is less influenced by the format and more by the requirements of delivery over the web. Native code is not downloaded each time the user opens the application. Tons of web infrastructure and complexity is aimed at solving the problem of "user lands on page for the first time, it must be responsive as soon as possible."
- haberman 3y ago
  > Linking native code doesn't really care about optimizing the size of the output and is just just trying to make sure all code paths can be called with some kind of code. That is not true at all. There are many use cases where native code size is very important. Native code toolchains often target platforms with extremely limited ROM/RAM. Even on big machines, RAM is not free and cache even less so. Native code linkers will GC unused code sections (`-Wl,--gc-sections `), fold identical data to remove duplication (see COMDAT). Native code compilers have optimization modes that specifically optimize for small code size (`-Os` and the like).

ponyous 3y ago

Because there were browsers and no standards. How can you expect someone that starts coding on the web to know what pains kernel went through decades prior?

tankenmate 3y ago
But surely the people writing the browser code thought about the ecosystem they were creating / trying to create?
- moron4hire 3y ago
  Browsers weren't written in a day. Technically speaking, Mozilla Firefox is a ship of Theseus going back to the release of Netscape in 1994. Did browser and internet infrastructure developers in the early 90s understand that these things would become rich application platforms? Looking at the history of HTTP, it's clear that they expected some concept of "application" to be delivered through the browser. While there's certainly a chance at least a few of them foresaw the full scope of what that would mean (it's not like X11 remoting wasn't a thing), I don't think most of the people involved were thinking much past 10 years (The Distant Future, the Year 2000). JavaScript was apocryphally "invented in 10 days", it came as an attempt to create competitive advantage, not to create a global standard. The first JavaScript came a year (1995) after the first Netscape, but the first major JS-heavy application didn't come for another 13 years (Google Maps, 2008).
  - tracker1 3y ago
    There were a lot of JS heavy web applications prior to Google Maps. From around the release of IE5 at the end of 2000 in particular through the long tenure of IE6. Having worked on some JS heavy applications around that time. It was also much harder as you had a relatively wide variety of browsers and versions. Since people on dialup were far less likely to update their browsers regularly (or at all beyond what came on an ISP or AOL CD. Of course, the efforts for larger dev teams, optimizations and bundling were far less popular before then. Can't tell you how many poorly written sites/apps carried who knows how many versions/copies of JQuery for example. It was really bad for a while. Now, at least there are more paying some attention to it. There's still some relatively large bundles that are easy to get overloaded. I mean as soon as you load any charting or graphing library it blows out everything else. Of course this is offset between bandwidth and compute advancements as well. There was a popular developer site around 1998-2000 or so called 15seconds.com as that was the average point at which users would start dropping off from a load. Now that's measured at around a second or two.
    - moron4hire 3y ago
      Yeah, I worked on some of them, too. They were even in mapping. I had written one of the first, dynamic, 2D drawing libraries for JavaScript, long before Canvas2D, but it was hidden in proprietary consultoware. The significance of Google Maps was that A) it had all the parts that we would recognize today as a JS single-page app, B) it had no alternative interface that people could opt to use instead (diluting the usage of that particular implementation versus the product as a whole), C) it had broad appeal and adoption, and D) it was significantly better than competitors specifically because of the "SPA-ness" of the app. Google Maps had the features and penetration necessary to change the public perception at large of what could be done with browser-based apps.
  - june_twenty 3y ago
    Was Google Maps the first JS-heavy app? That's a TIL for me..
    - 3y ago
- biorach 3y ago
  nope well.... not for a surprisingly long time

moron4hire 3y ago

I think it's largely a three-fold problem of the fact that most JS apps are still deployed through browsers and not installed, the fact that HTTP2 was not the panacea of multi-request happy good times it was made out to be, plus the fact that there is no Application Binary Interface, everything gets deployed as source code. This creates a situation where you need bundlers, whereas other languages don't have the concept at all, just to be able to minimize download time (and honestly, while we end up making rather large apps in comparison to web pages, they're pretty small in comparison to other kinds of applications), and then bundles are too opaque to share common code between applications. And because there's no chance to benefit across projects from sharing, there's no force driving standardization of bundling, or adoption of said standard.

brundolf 3y ago

1. Compiling/building is an ~optional layer on top of the core language/standard (which has become less optional over time) 2. Running JS outside of the browser is similarly a layer that was built before any kind of standard existed for it (which still doesn't, really) The browser standards are the only real standards. Everything else (which has turned into a lot) is "standard by implementation". Implementations usually try to agree with each other, because that's obviously beneficial for everybody, but sometimes they make choices to deviate either out of necessity or in an attempt to improve the ecosystem So it's all pretty ad-hoc, but in practice most things are mostly compatible most of the time. They orbit the same general things, and the orbit has narrowed in the last few years as most of the big problems have been solved and the community is zeroing in on the agreed solutions (with the caveat of often having to maintain legacy compatibility) Deno takes a stricter philosophy than most, where it prescribes a ~good way that everything should be done (which is almost entirely based on browser standards which have evolved since all this started), even though it runs outside of a browser, and requiring the ecosystem to fall in line Bun on the other hand takes a maximalist approach to compatibility; it does its best to make everything from every other sub-ecosystem Just Work without any modifications

qbasic_forever 3y ago

Bundling is totally different from linking and building for native platforms. Bundling is all about optimizing code to be sent over a small pipe--you're combining multiple compilation units into one file (so just one web request and lower latency) and doing optimizations like tree shaking to send only the code that's actually used. It's a pretty unique use-case that not many other programming languages deal with or care about. It's almost as if you are a demo scene coder trying to optimize for the absolute smallest code possible. Linking native code doesn't really care about optimizing the size of the output and is just just trying to make sure all code paths can be called with some kind of code.

colonwqbang 3y ago
A native compiler also optimises the code. A native static linker also tries to omit unused library data and code. It's absolutely not the case that native devs don't care about code size (we do!) I guess Javascript uses a slightly unusual executable format, text instead of binary. Otherwise, it seems like very much the same thing?
- crabmusket 3y ago
  I'd say that bundler design is less influenced by the format and more by the requirements of delivery over the web. Native code is not downloaded each time the user opens the application. Tons of web infrastructure and complexity is aimed at solving the problem of "user lands on page for the first time, it must be responsive as soon as possible."
haberman 3y ago
> Linking native code doesn't really care about optimizing the size of the output and is just just trying to make sure all code paths can be called with some kind of code. That is not true at all. There are many use cases where native code size is very important. Native code toolchains often target platforms with extremely limited ROM/RAM. Even on big machines, RAM is not free and cache even less so. Native code linkers will GC unused code sections (`-Wl,--gc-sections `), fold identical data to remove duplication (see COMDAT). Native code compilers have optimization modes that specifically optimize for small code size (`-Os` and the like).