3 ms·

What’s the usage case for scanning html at gigs per second speed? Speeding up browsers? Crawling?

by Havoc 2y ago

spartanatreyu 2y ago
Sometimes html pages can get pretty big, why not optimise it? Also, speeding up html parsing means speeding up browsers, crawlers, e2e tests, visual regression tests, etc...
- abirch 2y ago
  A lot of html, css, and JavaScript may not be independent.
  - kelnos 2y ago
    Sure, but if you parse the HTMl faster, you find the <script> and <link rel="stylesheet"> tags sooner, and can kick off another network thread to start fetching those files sooner. Overall you end up with a faster page load.
noir_lord 2y ago
If you improve parsed/scanned speed of html in Chromium by say 1%, that doesn't sound impressive. Then you consider 3 billion people use chrome (or Chromium derived browsers) multiply that out by how much html gets parsed/scanned a day by those users and then consider the energy costs - it quite quickly gets to a massive number (and that's just actual users, Chromium is also the core of Electron and a alot of other tooling) Simply put at Google's scale, tiny incremental gains compound to a massive species wide saving. At scale everything matters :).
dmw_ng 2y ago
Sufficiently fast software often allows leaving out whole layers of crap and needless indirection, the most common being caching. Fixing an algorithm so you can omit a dedicated database of intermediate results can be a huge maintainability/usability improvement. The same principle appears all over the place, e.g. immediate mode UIs, better networking (e.g. CSS image tiling vs. just fixing small request overhead in HTTP1 vs. QUIC), importing giant CSV files via some elaborate ETL process vs. just having a lightning fast parser+query engine, etc. Depending on how you look at it, you could view large chunks of DOM state through this lens, as intermediate data that only exists to cache HTML parsing results. What's the point of allocating a hash table to represent element attributes if they are unchanged from the source document, and reparsing them from the source is just as fast as keeping around the parsed form? etc. These kinds of tricks only tend to present themselves after optimization work is done, which is annoying, because it's usually so damn hard to justify optimization work in the first place.

What’s the usage case for scanning html at gigs per second speed? Speeding up browsers? Crawling?

by Havoc 2y ago

What’s the usage case for scanning html at gigs per second speed? Speeding up browsers? Crawling?

Sometimes html pages can get pretty big, why not optimise it? Also, speeding up html parsing means speeding up browsers, crawlers, e2e tests, visual regression tests, etc...

abirch 2y ago
A lot of html, css, and JavaScript may not be independent.
- kelnos 2y ago
  Sure, but if you parse the HTMl faster, you find the <script> and <link rel="stylesheet"> tags sooner, and can kick off another network thread to start fetching those files sooner. Overall you end up with a faster page load.

noir_lord 2y ago

If you improve parsed/scanned speed of html in Chromium by say 1%, that doesn't sound impressive. Then you consider 3 billion people use chrome (or Chromium derived browsers) multiply that out by how much html gets parsed/scanned a day by those users and then consider the energy costs - it quite quickly gets to a massive number (and that's just actual users, Chromium is also the core of Electron and a alot of other tooling) Simply put at Google's scale, tiny incremental gains compound to a massive species wide saving. At scale everything matters :).

dmw_ng 2y ago

Sufficiently fast software often allows leaving out whole layers of crap and needless indirection, the most common being caching. Fixing an algorithm so you can omit a dedicated database of intermediate results can be a huge maintainability/usability improvement. The same principle appears all over the place, e.g. immediate mode UIs, better networking (e.g. CSS image tiling vs. just fixing small request overhead in HTTP1 vs. QUIC), importing giant CSV files via some elaborate ETL process vs. just having a lightning fast parser+query engine, etc. Depending on how you look at it, you could view large chunks of DOM state through this lens, as intermediate data that only exists to cache HTML parsing results. What's the point of allocating a hash table to represent element attributes if they are unchanged from the source document, and reparsing them from the source is just as fast as keeping around the parsed form? etc. These kinds of tricks only tend to present themselves after optimization work is done, which is annoying, because it's usually so damn hard to justify optimization work in the first place.