2 ms·

So they're using headless browsers. Why can't anyone do that?

• wahnfrieden 11y ago
  Scale
  • comex 11y ago
    And perhaps security. I wouldn't be surprised if Google avoided standard C++/JIT browser engines in favor of something custom entirely written in a safe language - but if they don't, it wouldn't be that hard to get code execution on (a sandboxed portion of) Googlebot. Same goes for competitors - I don't think the state of public safe-language browsers is that good, though I'm not sure.
    • qopp 11y ago
      They are probably using virtual machines anyway, so it's not hard to set it up to simply load ram state for each new page they are crawling. This sidesteps the security issue (as long as there's no sandbox escapes). It's possible they are using components from Google Chrome as others mentioned, like V8.