4 ms·
From 2006 to 2010, I worked on the Google indexing system, primarily writing C++ code that executed every single bit of JavaScript in every HTML page Googlebot
by KMag 5y ago
From 2006 to 2010, I worked on the Google indexing system, primarily writing C++ code that executed every single bit of JavaScript in every HTML page Googlebot could find. At any time, there were roughly 1 million threads on roughly 1,000 machines executing some C++ code analyzing web pages, though at any moment, the majority of these threads were waiting on BigTable I/O.
Saving a bit of JavaScript execution time on the most costly JS snippets would have translated into real money savings in terms of both capital and operating expenses.
Especially since there were tons of pages where the most popular JS frameworks were being loaded and executed once, I thought about dumping SpikerMonkey bytecode (or toward the end, v8's first-pass x86 machine code) to disk/BigTable, or perhaps C++ rewrites of several of the most common JS frameworks.
I basically wrote a high-performance low-fidelity browser simulator that simultaneously pretended to be FireFox and IE (so that it would mostly work on IE-only pages and also mostly work on FF-only pages). There was some appeal to the idea of tweaking the JS frameworks to fit my simulator rather than tweaking the simulator for all these little corner cases. (I remember having to improve simulation fidelity for Saab's unified landing page for several Arab markets.)
The thing is, a high-fidelity rewrite of a small number of JavaScript farmeworks into C++ is probably roughly equivalent to writing the first version of v8, and you need to throw it all away as the frameworks upgrade versions.
There's still some merit to caching the bytecode for the most popular JS frameworks, and I wouldn't be surprised if Google currently does this for executing JS in the indexing system.