22 ms·

Proposal for a Binary Encoding of the JavaScript AST

45 points by bobajeff 9y ago

gcoda 9y ago
It might be a good idea. But scripts inside HTML not compiled because every user should have a right to see what gets executed of his device. Or this is just my fantasy and not design choice by early days engineers? I feel really conflicted about wasm and this. Maybe now we can just put index.exe and forget about openness of web already
- geofft 9y ago
  I don't know of anyone who actually audits the JS they run. I know people (like Richard Stallman) who don't run JS at all, and people (like my roommate) who use NoScript with exceptions for websites they trust—which is a trust by website, not by code; they'll trust any JS that origin produces in the future. I suppose there are people running GNU LibreJS and running any JS that claims to be freely licensed, but again without looking at the code to see if it's heavily obfuscated. And then there's the rest of us, the vast majority, who just run all the JS we se. If we already had people who audit JS before running it, preserving auditability would be worth doing, but keeping the ability to do something that (I think) literally nobody has done in over 20 years of JavaScript isn't worth trading off actual technical improvements for. Fortunately we have something very different and (IMO) more practical, that makes this very different from an "index.exe" model: we have isolation between websites enforced at the browser level. aim.exe can read my saved quicken.exe documents. hangouts.google.com cannot interact with my bankofamerica.com account; it can't even know that I have one. I think I have a right to see source to all the code that runs on my device with full privilege. I also think I have a right to limit the powers of code to which I don't have source, including both limiting what sorts of things it has access to and how much CPU, memory, etc. it's allowed to consume. I don't really know that insisting on the right to see source to code that runs confined is actually going to measurably improve my computing freedom.
  - Yoric 9y ago
    Anyway, with this format, Devtools will be able to display the source code. It's optimized for sending + parsing, but it can be decompiled without loss of anything meaningful. There is even a proposal to support keeping comments in the file.
  - kalleboo 9y ago
    > aim.exe can read my saved quicken.exe documents. hangouts.google.com cannot interact with my bankofamerica.com account; it can't even know that I have one. Although these days we have sandboxed native software as well. iOS, Android, Mac App Store and I believe Windows has something like this now too?
- gabrielcsapo 9y ago
  it just feels like putting lipstick on a pig, use binary to solve the problem of bloated utf8 files.
  - Gaelan 9y ago
    utf8 isn’t “bloated.” A utf8 file with just ASCII characters (every character you’ll see in JS, with the exception of foreign language strings) is identical to the ASCII encoding. You may be thinking of UTF16, which does use two bytes for every character.
- Gaelan 9y ago
  This is an encoding of the JS AST, so it has a 1-to-1 mapping to JS (modulo whitespace, but the JS would probably be minified anyway). It would be trivial to build a tool to convert this back into JS, and browser devtools will probably do that automatically.
  - Yoric 9y ago
    Technically, the tool to convert back to JS is already implemented. For the moment, it loses only comments and layouts and even comments might remain (see issue #11 on github).
- sugarfactory 9y ago
  It seems that you (like a great number of other people) assume that an ordinary person without special skills cannot comprehend what some code does once it is compiled. But that assumption is fortunately wrong. It is easier than you think to learn to read disassembly of machine code. It would be much easier to read higher level bytecode such as Android VM's smali. So don't fear compilation. What is truly worrying is any attempt to restrict access to code that is run by a computer. Even a highly skilled reverse engineer cannot do anything if there is no way to access code.
  - yorwba 9y ago
    s/ordinary person without special skills/someone who could understand the source code of the compiled artifact/ and I'm with you.
- alde 9y ago
  Nowadays, most JS is uglified/minified on build anyway. I believe that times when after-build JS code was readable for the end user are long gone.
  - Yoric 9y ago
    Actually, this format is much more readable than minifed JS, as it maintains variable names, etc (there is a proposal to also support comments without meaningful loss of performance). Of course, this does not prevent developers from minifying/uglifying the JS code first, if they so decide.
dgreensp 9y ago
I love the idea of binary ASTs as the things browsers compile instead of source, but I don't think making browsers even more complex is worth it. These new binary files would also be yet another type of asset to serve alongside your .js files, with implications for debugging and source maps. If the reward is bloated SPAs load 10% faster, as long as they ship both JS and binary JS, forever, I would wonder if browsers should parse the binary format or just keep optimizing their text parsers more.
- paulddraper 9y ago
  This seems pretty easy though. When you parse, you parse to the same JS AST. When you show it in the browser, you show the text form of the JS AST. Source maps like normal. It's a reasonably lightweight proposal, though IDK if it will be compelling enough.
codedokode 9y ago
I don't like the part that explains the motivation for implementing binary encoding: > A brief survey from July 2017 of the uncompressed JS payload sizes for some popular web applications on desktop and mobile: > Facebook 7.1 MB I think Facebook should just remove the code they don't need rather than invent new compression methods. There is no way a social network with a chat needs so much JS code.
- tachyonbeam 9y ago
  It's also possible to dynamically, lazily load some parts of the JS code. Surely not all the code is needed all the time. For example, you may not need to load the code for the settings page until the user actually goes to edit the settings.
  - Yoric 9y ago
    Still, in my experience, Facebook chats loads several Megabytes (that's gzipped size) of JavaScript upon startup. In this case, lazy loading gets into the way of performance, as Facebook chat ends up lazily loading 102 files (if I recall correctly).
  - djeric3 9y ago
    Yes, Facebook already lazily loads code for secondary functionality. More details in this comment https://news.ycombinator.com/item?id=14912871 https://news.ycombinator.com/item?id=14912871
    - codedokode 9y ago
      Please also look at my comment that shows the opposite: Facebook is loading a lot of unnecessary modules: https://news.ycombinator.com/item?id=14916177 https://news.ycombinator.com/item?id=14916177
      - djeric3 9y ago
        Yup, replied above. Thanks for digging into this more :)
Animats 9y ago
It could be called "Java".
dmitriid 9y ago
Isn't "binary AST" called "WebAssembly"? Oh. Right. I forgot. "Web" assembly was specifically designed to exclude Javascript. So yes, go ahead. Create another incompatible binary format
- dmitriid 9y ago
  People are downvoting my comment for some reason. The whole premise of "binary javascript AST" is "compile JavaScript into binary representation because bundle sizes are just too big and takes too much time to parse". Isn't WebAssembly designed to kinda solve that and many other problems presented in the Readme among other things? Instead of parsing JS (or whatever language) you just download binary (which has been compiled, optimised etc.). So, instead of designing webassembly to support the most important language on the web, the committee (as always, there's an effing committee to committee the hell out of things) designs webassembly with just one single language in mind: C++. Because reasons. Hence, this ridiculous proposal: oh, let's create a binary representation of Javascript AST because reasons. Because "oh, we can't compile to webassembly because webassembly hasn't even been designed to support the main language on the web and many other major languages". Instead, we're just going for the lame excuse of "no vendor would agree to ship bytecode." (why would they agree to ship binary ASTs?) and "there is no guarantee that it would be any faster to transmit/parse/start" (where's the guarantee with binary AST?).
  - Yoric 9y ago
    Well, if you look at the design goals of WebAssembly [1], you'll see very different high-level goals. Sure, there is some intersection, but not nearly as much as you seem to believe. Now, there are very good reasons to not ship bytecode (e.g. because once you standardize the bytecode on which your VM runs, you're dead in the water and your VM stops evolving). The binary AST was carefully designed to avoid this issue. Indeed, as written in the proposal, there is no guarantee that sending bytecode would be any faster to transmit/parse/start. Moreover, experiments suggest that it wouldn't. On the other hand, if you have read the proposal, you have seen that experiments with binary AST suggest that it is faster to transmit/parse/start. As for committees, well, that's how the web works. It is certainly not perfect, but it beats single-vendor-decides-everything. [1] https://github.com/WebAssembly/design/blob/master/HighLevelGoals.md https://github.com/WebAssembly/design/blob/master/HighLevelG...
fake-name 9y ago
Jesus christ. The solution to "our webpage has 7.1 MegaBYTES of javascript" isn't "lets use a more compact representation of the code", it's STOP FUCKING ADDING MORE JS SHIT. ------ It's basically: Users: "My house is on fire!" These devs: "Ok, we're here to help, we cleaned your windows. You can see the fire from outside so much better now!" ------ Maybe they should solve the problem of their website being a bloated javascript blob-monster, rather then just trying to fit it into a smaller box.
- rtpg 9y ago
  I feel like this is an unfair criticism unless you know what the page is doing. People are building entire vector editing programs on the web. How can you be fine with Photoshop being several hundred megs but not with this person having less than 10 megs of code to achieve something similar? The web isn't just Hacker News and CNN. People write real programs on it. Obviously shipping less code is Good(TM). But isn't an across the board improvement also good?
  - chroem- 9y ago
    >People are building entire vector editing programs on the web. This is the exception rather than the rule. There is no reason why it should take more than ten seconds to load a static news article with text.
    - spoiler 9y ago
      In which you—and the original starter of this comment thread—completely missed the point. The linked page clearly uses complex applications as examples. Whatever anyone's opinion of them are, they are quite complex. This proposal is aimed at web applications, not a simple website with image/text content, and minimal javascript.
      - mirko22 9y ago
        Yet Google Sheets and Gmail are applications, Facebook and LinkedIn are websites which might as well be fully static and are twice the size so his point is kinda correct. Mind you, Gmail also has features like Chat as do Facebook and LinkedIn.
twii 9y ago
> For parsing desktop facebook.com, 10-15% of client-side CPU time is spent parsing JavaScript. The prototype we implemented reduced time to build the AST by 70-90%. Whatever the win from their prototype, it should never be a reason to change core Javascript like .toString(). A facebook representative in this little proposal club doesn't feel good. My first thought was: please optimise your own codebase and don't break other people's code for your own profit.
- Yoric 9y ago
  Out of curiosity: do you actually use `Function.prototype.toString()`? I haven't seen any use of this method in years. Plus the few uses that I have seen were libraries that attempted to rewrite other libraries and broke at pretty much each update of their dependencies. There may be legitimate uses of this method in production code, but I can't think of any from the top of my mind. If you can think of one, don't hesitate to file it as an issue in the linked tracker.
  - amptorn 9y ago
    `Function.prototype.toString()` is the basis for an alternate, ES5-era multiline string syntax: var str = (function(){/* STRING GOES HERE */}).toString().slice(14, -3);
    - Yoric 9y ago
      That's... an interesting use. I'm pretty sure it violates the actual specifications of JavaScript, breaks in presence of a minifier/uglifier and slows down your script, but it's a cool hack :)
    - TheAceOfHearts 9y ago
      Prior to ES2015, the spec's definition of Function.prototype.toString() was pretty vague. The behavior was implementation-dependent, although I don't know the differences per browsers, since I've never used this feature seriously. Here's the text from ES5.1 [0]: > An implementation-dependent representation of the function is returned. This representation has the syntax of a FunctionDeclaration. Note in particular that the use and placement of white space, line terminators, and semicolons within the representation String is implementation-dependent. [0] http://www.ecma-international.org/ecma-262/5.1/#sec-15.3.4.2 http://www.ecma-international.org/ecma-262/5.1/#sec-15.3.4.2
jacobush 9y ago
Thoughts. This looks to make Javascript even more "lisp-ish" than it already was, by simplifying syntax. Albeit a binary lisp. Second thought - regular zip compression already minimises Javascript, doesn't it? Which leads to my third thought - isn't the most important problem they are trying to solve, the slow parsing speed of Javascript text? They solve that by introducing unambiguous operators and other things. This could be done without resorting to binary encodings. They could introduce a plain text encoding, where (for instance) only UTF8 is allowed, with unambiguous operators. THEN they could put a binary encoding on top of that, if necessary. Or just zip it. Instead they jumped straight at the binary representation. I'm not saying it's wrong to do so. But maybe unnecessary? Please let me know if my 5 minute understanding of what they are doing is wrong... EDIT: spelling.
- Yoric 9y ago
  Slow parsing speed is due to several things: - encoding issues; - ambiguous stuff (for instance, `/` can be the start of a comment, the start of a regexp or a division, `()` or `{}` can have several meanings, etc.); - odd behavior (variables can be used before they are declared, double-declaring variables is sometimes valid but not always, etc.); - plenty of information that is necessary for execution is only available after you need it (sometimes much later, e.g. determining free variables, determining whether `eval` can affect local variables, etc). Also, this syntax makes it impossible to perform concurrent parsing (at least not without extremely costly non-concurrent pre-parsing) and makes it basically impossible to perform lazy parsing (at least not without this same extremely costly pre-parsing). If I understand correctly, your suggestion would have been to: 1. provide a "better" developer-facing text syntax; 2. if necessary, move to binary parsing. Your suggestion makes sense but I believe that it underestimates seriously the ability to: a. come up with an alternative, easier to parse, text syntax; b. make sure that your text syntax has the same semantics as the original syntax; c. get everybody to agree on that text syntax; d. maintain this text syntax through successive versions of JavaScript, without losing its desirable properties; e. have every browser maintain two text parsers. Also, note that b. most likely means standardizing upon an AST, which is roughly half of the difficulty of the current proposal. Finally, unless I am missing something, your suggestion does not improve concurrent or lazy parsing. One of the core benchmarks behind the current proposal is that lazy parsing can be made extremely faster than non-lazy parsing, as soon as you the expensive pre-parsing phase has been abolished. Does this answer your questions?
z3t4 9y ago
Send your JavaScript as an image, then unpack it using WebGL.
z3t4 9y ago
You don't have to parse every single module at page load. You can defer, or only load code the first time it's used. This 10-15% of time saved is only on first load. If you have a web "app" it's not like the user will reload it over and over again like the old school server rendered apps, which still works fine btw, they are even faster now with faster computers and better network latency. So if this is not the problem of classic web apps, nor the problem of modern web apps, who's problem will it solve !? Make cache-friendly modules, only abstract code that can be reused, keep the rest of the code in-line.
- Yoric 9y ago
  I wrote https://news.ycombinator.com/item?id=14908903 https://news.ycombinator.com/item?id=14908903 in answer to a different question. Does it answer your points, by any chance?
  - z3t4 9y ago
    JavaScript is a fun language because it does not require a build step. Developing JavaScript is easy and you want to make it harder just to shave off a few milliseconds !? Show me some stats and graphs on how long it actually takes to parse JS to AST. I care a lot for performance, if you want to make me happy, make text rendering in the canvas faster. And if you could also make eval faster I would be very happy.
    - Yoric 9y ago
      Most professional development I have seen in the past few years already uses a build step, either to pack dependencies, to run linting or to polyfill language features, so this Binary AST will not really change that. Indeed, I hope that Babel, Webpack & co will eventually ship with an option `--binary-ast` and that the only change for developers will be to add this option to their toolchain. However, if you wish to write JavaScript without build step, you will of course be free to continue doing this. I don't think that anybody has ever thought of making the Binary AST compulsory. It's just a build target that you can use for decreasing file size and parse time. For stats, see the proposal. We'll add more stats once the advanced prototype is complete. > I care a lot for performance, if you want to make me happy, make text rendering in the canvas faster. And if you could also make eval faster I would be very happy. I'm sure both are in progress, but entirely orthogonal to this proposal.
snarfy 9y ago
It seems like this is a very small gain. After the initial fetch everything should be cached locally. If it's not you did it wrong. Am I missing something? How is this better than normal web caching?
- Yoric 9y ago
  Yes and no. You are right, after the initial fetch, everything should be cached locally. However, caching traditionally does not affect parsing speed. While size gains are a nice side-benefit of this proposal, the main benefit is parsing speed. Now, both Chromium and Firefox have introduced smarter caching that caches post-parsing data for websites that you have already visit (or visit often, I don't remember the heuristics). This is very useful for all the sites that you visit regularly and that do not update their JS code between your visits. However, for all the sites that do update their JS code between your visits, including Facebook (which updates several times per day), Google Docs, etc. and every random website that you read because of a link on Hacker News but will never revisit again, there is a potentially pretty large size benefit. Does this answer your question?
throwaway2016a 9y ago
I think the web could actually benefit from this and most developers these days are used to sending their client code through a compiler already so it would make a minimal difference in tooling. Personally, though, webpack is pretty amazing. When I first heard of it I very much disliked it but it combined transpiling, with minification, with code optimization like tree shaking (getting rid of code from the compiled result that is unused).
codedokode 9y ago
This suggestion also requires a JS engine to have an AST compatible with the one that is described. So the browser developers will have less freedom in a choice of an internal AST representation.
- Yoric 9y ago
  For what it's worth, I'm currently implementing the Binary AST parser (using the current candidate AST) in SpiderMonkey (which uses its own AST, of course) and that doesn't cause any issue so far. The AST used in the file is a convenient abstraction but doesn't need to match the AST used in-memory.

45 points by bobajeff 9y ago

It might be a good idea. But scripts inside HTML not compiled because every user should have a right to see what gets executed of his device. Or this is just my fantasy and not design choice by early days engineers? I feel really conflicted about wasm and this. Maybe now we can just put index.exe and forget about openness of web already

geofft 9y ago
I don't know of anyone who actually audits the JS they run. I know people (like Richard Stallman) who don't run JS at all, and people (like my roommate) who use NoScript with exceptions for websites they trust—which is a trust by website, not by code; they'll trust any JS that origin produces in the future. I suppose there are people running GNU LibreJS and running any JS that claims to be freely licensed, but again without looking at the code to see if it's heavily obfuscated. And then there's the rest of us, the vast majority, who just run all the JS we se. If we already had people who audit JS before running it, preserving auditability would be worth doing, but keeping the ability to do something that (I think) literally nobody has done in over 20 years of JavaScript isn't worth trading off actual technical improvements for. Fortunately we have something very different and (IMO) more practical, that makes this very different from an "index.exe" model: we have isolation between websites enforced at the browser level. aim.exe can read my saved quicken.exe documents. hangouts.google.com cannot interact with my bankofamerica.com account; it can't even know that I have one. I think I have a right to see source to all the code that runs on my device with full privilege. I also think I have a right to limit the powers of code to which I don't have source, including both limiting what sorts of things it has access to and how much CPU, memory, etc. it's allowed to consume. I don't really know that insisting on the right to see source to code that runs confined is actually going to measurably improve my computing freedom.
- Yoric 9y ago
  Anyway, with this format, Devtools will be able to display the source code. It's optimized for sending + parsing, but it can be decompiled without loss of anything meaningful. There is even a proposal to support keeping comments in the file.
- kalleboo 9y ago
  > aim.exe can read my saved quicken.exe documents. hangouts.google.com cannot interact with my bankofamerica.com account; it can't even know that I have one. Although these days we have sandboxed native software as well. iOS, Android, Mac App Store and I believe Windows has something like this now too?
gabrielcsapo 9y ago
it just feels like putting lipstick on a pig, use binary to solve the problem of bloated utf8 files.
- Gaelan 9y ago
  utf8 isn’t “bloated.” A utf8 file with just ASCII characters (every character you’ll see in JS, with the exception of foreign language strings) is identical to the ASCII encoding. You may be thinking of UTF16, which does use two bytes for every character.
Gaelan 9y ago
This is an encoding of the JS AST, so it has a 1-to-1 mapping to JS (modulo whitespace, but the JS would probably be minified anyway). It would be trivial to build a tool to convert this back into JS, and browser devtools will probably do that automatically.
- Yoric 9y ago
  Technically, the tool to convert back to JS is already implemented. For the moment, it loses only comments and layouts and even comments might remain (see issue #11 on github).
sugarfactory 9y ago
It seems that you (like a great number of other people) assume that an ordinary person without special skills cannot comprehend what some code does once it is compiled. But that assumption is fortunately wrong. It is easier than you think to learn to read disassembly of machine code. It would be much easier to read higher level bytecode such as Android VM's smali. So don't fear compilation. What is truly worrying is any attempt to restrict access to code that is run by a computer. Even a highly skilled reverse engineer cannot do anything if there is no way to access code.
- yorwba 9y ago
  s/ordinary person without special skills/someone who could understand the source code of the compiled artifact/ and I'm with you.
alde 9y ago
Nowadays, most JS is uglified/minified on build anyway. I believe that times when after-build JS code was readable for the end user are long gone.
- Yoric 9y ago
  Actually, this format is much more readable than minifed JS, as it maintains variable names, etc (there is a proposal to also support comments without meaningful loss of performance). Of course, this does not prevent developers from minifying/uglifying the JS code first, if they so decide.

dgreensp 9y ago

I love the idea of binary ASTs as the things browsers compile instead of source, but I don't think making browsers even more complex is worth it. These new binary files would also be yet another type of asset to serve alongside your .js files, with implications for debugging and source maps. If the reward is bloated SPAs load 10% faster, as long as they ship both JS and binary JS, forever, I would wonder if browsers should parse the binary format or just keep optimizing their text parsers more.

paulddraper 9y ago
This seems pretty easy though. When you parse, you parse to the same JS AST. When you show it in the browser, you show the text form of the JS AST. Source maps like normal. It's a reasonably lightweight proposal, though IDK if it will be compelling enough.

codedokode 9y ago

I don't like the part that explains the motivation for implementing binary encoding: > A brief survey from July 2017 of the uncompressed JS payload sizes for some popular web applications on desktop and mobile: > Facebook 7.1 MB I think Facebook should just remove the code they don't need rather than invent new compression methods. There is no way a social network with a chat needs so much JS code.

tachyonbeam 9y ago
It's also possible to dynamically, lazily load some parts of the JS code. Surely not all the code is needed all the time. For example, you may not need to load the code for the settings page until the user actually goes to edit the settings.
- Yoric 9y ago
  Still, in my experience, Facebook chats loads several Megabytes (that's gzipped size) of JavaScript upon startup. In this case, lazy loading gets into the way of performance, as Facebook chat ends up lazily loading 102 files (if I recall correctly).
- djeric3 9y ago
  Yes, Facebook already lazily loads code for secondary functionality. More details in this comment https://news.ycombinator.com/item?id=14912871 https://news.ycombinator.com/item?id=14912871
  - codedokode 9y ago
    Please also look at my comment that shows the opposite: Facebook is loading a lot of unnecessary modules: https://news.ycombinator.com/item?id=14916177 https://news.ycombinator.com/item?id=14916177
    - djeric3 9y ago
      Yup, replied above. Thanks for digging into this more :)

Animats 9y ago

It could be called "Java".

dmitriid 9y ago

Isn't "binary AST" called "WebAssembly"? Oh. Right. I forgot. "Web" assembly was specifically designed to exclude Javascript. So yes, go ahead. Create another incompatible binary format

dmitriid 9y ago
People are downvoting my comment for some reason. The whole premise of "binary javascript AST" is "compile JavaScript into binary representation because bundle sizes are just too big and takes too much time to parse". Isn't WebAssembly designed to kinda solve that and many other problems presented in the Readme among other things? Instead of parsing JS (or whatever language) you just download binary (which has been compiled, optimised etc.). So, instead of designing webassembly to support the most important language on the web, the committee (as always, there's an effing committee to committee the hell out of things) designs webassembly with just one single language in mind: C++. Because reasons. Hence, this ridiculous proposal: oh, let's create a binary representation of Javascript AST because reasons. Because "oh, we can't compile to webassembly because webassembly hasn't even been designed to support the main language on the web and many other major languages". Instead, we're just going for the lame excuse of "no vendor would agree to ship bytecode." (why would they agree to ship binary ASTs?) and "there is no guarantee that it would be any faster to transmit/parse/start" (where's the guarantee with binary AST?).
- Yoric 9y ago
  Well, if you look at the design goals of WebAssembly [1], you'll see very different high-level goals. Sure, there is some intersection, but not nearly as much as you seem to believe. Now, there are very good reasons to not ship bytecode (e.g. because once you standardize the bytecode on which your VM runs, you're dead in the water and your VM stops evolving). The binary AST was carefully designed to avoid this issue. Indeed, as written in the proposal, there is no guarantee that sending bytecode would be any faster to transmit/parse/start. Moreover, experiments suggest that it wouldn't. On the other hand, if you have read the proposal, you have seen that experiments with binary AST suggest that it is faster to transmit/parse/start. As for committees, well, that's how the web works. It is certainly not perfect, but it beats single-vendor-decides-everything. [1] https://github.com/WebAssembly/design/blob/master/HighLevelGoals.md https://github.com/WebAssembly/design/blob/master/HighLevelG...

fake-name 9y ago

Jesus christ. The solution to "our webpage has 7.1 MegaBYTES of javascript" isn't "lets use a more compact representation of the code", it's STOP FUCKING ADDING MORE JS SHIT. ------ It's basically: Users: "My house is on fire!" These devs: "Ok, we're here to help, we cleaned your windows. You can see the fire from outside so much better now!" ------ Maybe they should solve the problem of their website being a bloated javascript blob-monster, rather then just trying to fit it into a smaller box.

rtpg 9y ago
I feel like this is an unfair criticism unless you know what the page is doing. People are building entire vector editing programs on the web. How can you be fine with Photoshop being several hundred megs but not with this person having less than 10 megs of code to achieve something similar? The web isn't just Hacker News and CNN. People write real programs on it. Obviously shipping less code is Good(TM). But isn't an across the board improvement also good?
- chroem- 9y ago
  >People are building entire vector editing programs on the web. This is the exception rather than the rule. There is no reason why it should take more than ten seconds to load a static news article with text.
  - spoiler 9y ago
    In which you—and the original starter of this comment thread—completely missed the point. The linked page clearly uses complex applications as examples. Whatever anyone's opinion of them are, they are quite complex. This proposal is aimed at web applications, not a simple website with image/text content, and minimal javascript.
    - mirko22 9y ago
      Yet Google Sheets and Gmail are applications, Facebook and LinkedIn are websites which might as well be fully static and are twice the size so his point is kinda correct. Mind you, Gmail also has features like Chat as do Facebook and LinkedIn.

twii 9y ago

> For parsing desktop facebook.com, 10-15% of client-side CPU time is spent parsing JavaScript. The prototype we implemented reduced time to build the AST by 70-90%. Whatever the win from their prototype, it should never be a reason to change core Javascript like .toString(). A facebook representative in this little proposal club doesn't feel good. My first thought was: please optimise your own codebase and don't break other people's code for your own profit.

Yoric 9y ago
Out of curiosity: do you actually use `Function.prototype.toString()`? I haven't seen any use of this method in years. Plus the few uses that I have seen were libraries that attempted to rewrite other libraries and broke at pretty much each update of their dependencies. There may be legitimate uses of this method in production code, but I can't think of any from the top of my mind. If you can think of one, don't hesitate to file it as an issue in the linked tracker.
- amptorn 9y ago
  `Function.prototype.toString()` is the basis for an alternate, ES5-era multiline string syntax: var str = (function(){/* STRING GOES HERE */}).toString().slice(14, -3);
  - Yoric 9y ago
    That's... an interesting use. I'm pretty sure it violates the actual specifications of JavaScript, breaks in presence of a minifier/uglifier and slows down your script, but it's a cool hack :)
  - TheAceOfHearts 9y ago
    Prior to ES2015, the spec's definition of Function.prototype.toString() was pretty vague. The behavior was implementation-dependent, although I don't know the differences per browsers, since I've never used this feature seriously. Here's the text from ES5.1 [0]: > An implementation-dependent representation of the function is returned. This representation has the syntax of a FunctionDeclaration. Note in particular that the use and placement of white space, line terminators, and semicolons within the representation String is implementation-dependent. [0] http://www.ecma-international.org/ecma-262/5.1/#sec-15.3.4.2 http://www.ecma-international.org/ecma-262/5.1/#sec-15.3.4.2

jacobush 9y ago

Thoughts. This looks to make Javascript even more "lisp-ish" than it already was, by simplifying syntax. Albeit a binary lisp. Second thought - regular zip compression already minimises Javascript, doesn't it? Which leads to my third thought - isn't the most important problem they are trying to solve, the slow parsing speed of Javascript text? They solve that by introducing unambiguous operators and other things. This could be done without resorting to binary encodings. They could introduce a plain text encoding, where (for instance) only UTF8 is allowed, with unambiguous operators. THEN they could put a binary encoding on top of that, if necessary. Or just zip it. Instead they jumped straight at the binary representation. I'm not saying it's wrong to do so. But maybe unnecessary? Please let me know if my 5 minute understanding of what they are doing is wrong... EDIT: spelling.

Yoric 9y ago
Slow parsing speed is due to several things: - encoding issues; - ambiguous stuff (for instance, `/` can be the start of a comment, the start of a regexp or a division, `()` or `{}` can have several meanings, etc.); - odd behavior (variables can be used before they are declared, double-declaring variables is sometimes valid but not always, etc.); - plenty of information that is necessary for execution is only available after you need it (sometimes much later, e.g. determining free variables, determining whether `eval` can affect local variables, etc). Also, this syntax makes it impossible to perform concurrent parsing (at least not without extremely costly non-concurrent pre-parsing) and makes it basically impossible to perform lazy parsing (at least not without this same extremely costly pre-parsing). If I understand correctly, your suggestion would have been to: 1. provide a "better" developer-facing text syntax; 2. if necessary, move to binary parsing. Your suggestion makes sense but I believe that it underestimates seriously the ability to: a. come up with an alternative, easier to parse, text syntax; b. make sure that your text syntax has the same semantics as the original syntax; c. get everybody to agree on that text syntax; d. maintain this text syntax through successive versions of JavaScript, without losing its desirable properties; e. have every browser maintain two text parsers. Also, note that b. most likely means standardizing upon an AST, which is roughly half of the difficulty of the current proposal. Finally, unless I am missing something, your suggestion does not improve concurrent or lazy parsing. One of the core benchmarks behind the current proposal is that lazy parsing can be made extremely faster than non-lazy parsing, as soon as you the expensive pre-parsing phase has been abolished. Does this answer your questions?

z3t4 9y ago

Send your JavaScript as an image, then unpack it using WebGL.

z3t4 9y ago

You don't have to parse every single module at page load. You can defer, or only load code the first time it's used. This 10-15% of time saved is only on first load. If you have a web "app" it's not like the user will reload it over and over again like the old school server rendered apps, which still works fine btw, they are even faster now with faster computers and better network latency. So if this is not the problem of classic web apps, nor the problem of modern web apps, who's problem will it solve !? Make cache-friendly modules, only abstract code that can be reused, keep the rest of the code in-line.

Yoric 9y ago
I wrote https://news.ycombinator.com/item?id=14908903 https://news.ycombinator.com/item?id=14908903 in answer to a different question. Does it answer your points, by any chance?
- z3t4 9y ago
  JavaScript is a fun language because it does not require a build step. Developing JavaScript is easy and you want to make it harder just to shave off a few milliseconds !? Show me some stats and graphs on how long it actually takes to parse JS to AST. I care a lot for performance, if you want to make me happy, make text rendering in the canvas faster. And if you could also make eval faster I would be very happy.
  - Yoric 9y ago
    Most professional development I have seen in the past few years already uses a build step, either to pack dependencies, to run linting or to polyfill language features, so this Binary AST will not really change that. Indeed, I hope that Babel, Webpack & co will eventually ship with an option `--binary-ast` and that the only change for developers will be to add this option to their toolchain. However, if you wish to write JavaScript without build step, you will of course be free to continue doing this. I don't think that anybody has ever thought of making the Binary AST compulsory. It's just a build target that you can use for decreasing file size and parse time. For stats, see the proposal. We'll add more stats once the advanced prototype is complete. > I care a lot for performance, if you want to make me happy, make text rendering in the canvas faster. And if you could also make eval faster I would be very happy. I'm sure both are in progress, but entirely orthogonal to this proposal.

snarfy 9y ago

It seems like this is a very small gain. After the initial fetch everything should be cached locally. If it's not you did it wrong. Am I missing something? How is this better than normal web caching?

Yoric 9y ago
Yes and no. You are right, after the initial fetch, everything should be cached locally. However, caching traditionally does not affect parsing speed. While size gains are a nice side-benefit of this proposal, the main benefit is parsing speed. Now, both Chromium and Firefox have introduced smarter caching that caches post-parsing data for websites that you have already visit (or visit often, I don't remember the heuristics). This is very useful for all the sites that you visit regularly and that do not update their JS code between your visits. However, for all the sites that do update their JS code between your visits, including Facebook (which updates several times per day), Google Docs, etc. and every random website that you read because of a link on Hacker News but will never revisit again, there is a potentially pretty large size benefit. Does this answer your question?

throwaway2016a 9y ago

I think the web could actually benefit from this and most developers these days are used to sending their client code through a compiler already so it would make a minimal difference in tooling. Personally, though, webpack is pretty amazing. When I first heard of it I very much disliked it but it combined transpiling, with minification, with code optimization like tree shaking (getting rid of code from the compiled result that is unused).

codedokode 9y ago

This suggestion also requires a JS engine to have an AST compatible with the one that is described. So the browser developers will have less freedom in a choice of an internal AST representation.

Yoric 9y ago
For what it's worth, I'm currently implementing the Binary AST parser (using the current candidate AST) in SpiderMonkey (which uses its own AST, of course) and that doesn't cause any issue so far. The AST used in the file is a convenient abstraction but doesn't need to match the AST used in-memory.