4 ms·

We (Chrome) have reached out to PhantomJS to inquire if they're interested in collaborating: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU06ZOZcF

We (Chrome) have reached out to PhantomJS to inquire if they're interested in collaborating: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU06ZOZcFgAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU... The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.

flanbiscuit 10y ago
Thank you for being willing to work with them and not just rolling your own version of Phantomjs. I hope they respond positively to your inquiry
shouldbworking 10y ago
It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken
- tracker1 10y ago
  That seems unlikely... It would seem they have two sets of crawlers... one that does typical/advanced scraping, and another that runs the JS (Chromium) and takes DOM snapshots. This is reflected by changing certain properties (window title etc) and seeing them reflected in Google's search results. A couple years ago, the lag was several days to a week behind on new content via as-rendered from the server vs. via JS.
  - shouldbworking 10y ago
    Interesting! I find it puzzling that insider knowledge on how google search works never seems to leak into the public domain. Is google killing leaks in their search algorithms or do they pay the search team such ungodly amounts of cash that nobody has ever left?
    - tracker1 10y ago
      Well, the point of their engine is to make it harder to game the system... everytime someone figures out a trick, someone takes advantage of it... that alone would discourage leaks. We laypeople can only find out things via word of mouth or observational tests on assumptions.
masklinn 10y ago
> The DevTools Protocol is the primary API for headless Chrome Why not the WebDriver protocol? That seems to be exactly what it's intended for…
- KayL 10y ago
  WebDriver protocol has limited APIs. For example, no networking status. (WebDriver rejected to implement it, they told you to use Proxy for debugging)
2T1Qka0rEiPr 10y ago
A few hours later: https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuN...
- yamaneko 10y ago
  After his announcement, Vitaly answered to the team: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQdE1A5FAwAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQ... > >Is there interest on your side in adopting Chromium as a runtime? There's some existing documentation [2] around the API and embedding, but admittedly, this would be some work. > We are interested. But I am afraid not in the current state. Currently, PhantomJS heavily relies on Qt and QtWebKit. It's not that easy to adopt Chrome as a new runtime. > But I think we could implement PhantomJS as a completely new (with the same API) project that will use Chrome - Phantomium!
  - Naracion 10y ago
    Also see https://github.com/Vitallium/phantomium https://github.com/Vitallium/phantomium

flanbiscuit 10y ago

Thank you for being willing to work with them and not just rolling your own version of Phantomjs. I hope they respond positively to your inquiry

shouldbworking 10y ago

It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken

tracker1 10y ago
That seems unlikely... It would seem they have two sets of crawlers... one that does typical/advanced scraping, and another that runs the JS (Chromium) and takes DOM snapshots. This is reflected by changing certain properties (window title etc) and seeing them reflected in Google's search results. A couple years ago, the lag was several days to a week behind on new content via as-rendered from the server vs. via JS.
- shouldbworking 10y ago
  Interesting! I find it puzzling that insider knowledge on how google search works never seems to leak into the public domain. Is google killing leaks in their search algorithms or do they pay the search team such ungodly amounts of cash that nobody has ever left?
  - tracker1 10y ago
    Well, the point of their engine is to make it harder to game the system... everytime someone figures out a trick, someone takes advantage of it... that alone would discourage leaks. We laypeople can only find out things via word of mouth or observational tests on assumptions.

masklinn 10y ago

> The DevTools Protocol is the primary API for headless Chrome Why not the WebDriver protocol? That seems to be exactly what it's intended for…

KayL 10y ago
WebDriver protocol has limited APIs. For example, no networking status. (WebDriver rejected to implement it, they told you to use Proxy for debugging)

2T1Qka0rEiPr 10y ago

A few hours later: https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuN...

yamaneko 10y ago
After his announcement, Vitaly answered to the team: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQdE1A5FAwAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQ... > >Is there interest on your side in adopting Chromium as a runtime? There's some existing documentation [2] around the API and embedding, but admittedly, this would be some work. > We are interested. But I am afraid not in the current state. Currently, PhantomJS heavily relies on Qt and QtWebKit. It's not that easy to adopt Chrome as a new runtime. > But I think we could implement PhantomJS as a completely new (with the same API) project that will use Chrome - Phantomium!
- Naracion 10y ago
  Also see https://github.com/Vitallium/phantomium https://github.com/Vitallium/phantomium