5 ms·

This is fantastic. I'm using a combination of Chrome and PhantomJS for karma testing right now, for https://github.com/paypal/paypal-checkout https://github.com

This is fantastic. I'm using a combination of Chrome and PhantomJS for karma testing right now, for https://github.com/paypal/paypal-checkout https://github.com/paypal/paypal-checkout and https://github.com/krakenjs/xcomponent https://github.com/krakenjs/xcomponent. There are hundreds of tests opening up hundreds of iframes and popup windows, and sending a lot of cross-window messages, and that ends up being really memory hungry. Chrome deals pretty well with garbage collection, so long as I'm careful to de-reference closed windows properly¹, and only uses a maximum of 150mbs. PhantomJS eats up almost 6GB of memory before it's done, which makes it almost unusable on machines with less memory or CI boxes. Travis is a no-go. I'm hoping running Chrome in headless mode should give a nice speedup for our tests. ----- ¹ Turns out even a closed popup window or iframe keeps a huge amount of memory hanging around. Who knew.

paulirish 10y ago
We (Chrome) have reached out to PhantomJS to inquire if they're interested in collaborating: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU06ZOZcFgAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU... The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.
- flanbiscuit 10y ago
  Thank you for being willing to work with them and not just rolling your own version of Phantomjs. I hope they respond positively to your inquiry
- shouldbworking 10y ago
  It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken
  - tracker1 10y ago
    That seems unlikely... It would seem they have two sets of crawlers... one that does typical/advanced scraping, and another that runs the JS (Chromium) and takes DOM snapshots. This is reflected by changing certain properties (window title etc) and seeing them reflected in Google's search results. A couple years ago, the lag was several days to a week behind on new content via as-rendered from the server vs. via JS.
    - shouldbworking 10y ago
      Interesting! I find it puzzling that insider knowledge on how google search works never seems to leak into the public domain. Is google killing leaks in their search algorithms or do they pay the search team such ungodly amounts of cash that nobody has ever left?
      - tracker1 10y ago
        Well, the point of their engine is to make it harder to game the system... everytime someone figures out a trick, someone takes advantage of it... that alone would discourage leaks. We laypeople can only find out things via word of mouth or observational tests on assumptions.
- masklinn 10y ago
  > The DevTools Protocol is the primary API for headless Chrome Why not the WebDriver protocol? That seems to be exactly what it's intended for…
  - KayL 10y ago
    WebDriver protocol has limited APIs. For example, no networking status. (WebDriver rejected to implement it, they told you to use Proxy for debugging)
- 2T1Qka0rEiPr 10y ago
  A few hours later: https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuN...
  - yamaneko 10y ago
    After his announcement, Vitaly answered to the team: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQdE1A5FAwAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQ... > >Is there interest on your side in adopting Chromium as a runtime? There's some existing documentation [2] around the API and embedding, but admittedly, this would be some work. > We are interested. But I am afraid not in the current state. Currently, PhantomJS heavily relies on Qt and QtWebKit. It's not that easy to adopt Chrome as a new runtime. > But I think we could implement PhantomJS as a completely new (with the same API) project that will use Chrome - Phantomium!
    - Naracion 10y ago
      Also see https://github.com/Vitallium/phantomium https://github.com/Vitallium/phantomium
sbuccini 10y ago
I've been using post-robot and xcomponent a lot recently as I'm solving a bunch of similar challenges. Just wanted to say thank you for sharing your solutions and expertise.
- bluepnume 10y ago
  Glad they're working out for you! Let me know if you encounter any interesting situations they don't support.
Rumudiez 10y ago
What do you do to ensure the resources for closed windows are released?
- bluepnume 10y ago
  A combination of: - Making sure all promises are fulfilled or rejected, so window objects don't get caught indefinitely in closure scope for any .then() or .catch() handler functions. - Using WeakMaps as much as possible, when we have things that are tied to a particular window, like message listeners or response handlers in post-robot - Manually clearing up any global references to windows when we destroy an xcomponent instance Finding the references was the tricky bit. A lot of the effort was finding a leaky test-case, running in 100 times in succession, and deleting code until the memory graph was flat -- then figuring out what I'd just deleted that caused the leak. The problem started manifesting as I added more and more tests -- so now I'm actually checking my tests' memory usage on the fly and failing if they cross a threshold. Hopefully that should avoid getting into this kind of sticky situation ever again. https://github.com/paypal/paypal-checkout/blob/master/test/test.js#L42-L52 https://github.com/paypal/paypal-checkout/blob/master/test/t...
PatrickAuld 10y ago
We also did a POC with PhantomJS and found similar issues, as well as generally flakiness causing too many false negatives. Ended up not using it; I'm hoping this can simplify things are give something more solid to build on.
- seanp2k2 10y ago
  Same, ended up using ChromeDriver + XVFB on Jenkins. Works but it's still pretty slow.
LoSboccacc 10y ago
We're using xvfb and selenium* for testing and a proper headless support would be a hundred time more stable than the self restarting framebuffer can't wait to move to headless chrome *Yeah I know phantomjs is more cool these days but phantom doesn't support windows height so there's that.
- akras14 10y ago
  +1 for xvfb
akras14 10y ago
I replied to you on Twitter, but for the sake of discussion, there is a third option that may be worth exploring - using Chrome running in virtualized Windows Manager - https://www.alexkras.com/running-chrome-and-other-browsers-in-almost-headless-mode/ https://www.alexkras.com/running-chrome-and-other-browsers-i...
- bluepnume 10y ago
  Very cool. Gonna have to check this out.
- amclennon 10y ago
  Hello, former SDC coworker! The official Selenium Docker image uses the same technique to run headless Chrome / Firefox. From there you can just run docker run -d -P selenium/standalone-chrome and you'll get something a lot more lightweight than spinning up a new Vagrant VM.
jslove 10y ago
But can you run headless chrome on Chromeos? if not why not?

paulirish 10y ago

We (Chrome) have reached out to PhantomJS to inquire if they're interested in collaborating: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU06ZOZcFgAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU... The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.

flanbiscuit 10y ago
Thank you for being willing to work with them and not just rolling your own version of Phantomjs. I hope they respond positively to your inquiry
shouldbworking 10y ago
It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken
- tracker1 10y ago
  That seems unlikely... It would seem they have two sets of crawlers... one that does typical/advanced scraping, and another that runs the JS (Chromium) and takes DOM snapshots. This is reflected by changing certain properties (window title etc) and seeing them reflected in Google's search results. A couple years ago, the lag was several days to a week behind on new content via as-rendered from the server vs. via JS.
  - shouldbworking 10y ago
    Interesting! I find it puzzling that insider knowledge on how google search works never seems to leak into the public domain. Is google killing leaks in their search algorithms or do they pay the search team such ungodly amounts of cash that nobody has ever left?
    - tracker1 10y ago
      Well, the point of their engine is to make it harder to game the system... everytime someone figures out a trick, someone takes advantage of it... that alone would discourage leaks. We laypeople can only find out things via word of mouth or observational tests on assumptions.
masklinn 10y ago
> The DevTools Protocol is the primary API for headless Chrome Why not the WebDriver protocol? That seems to be exactly what it's intended for…
- KayL 10y ago
  WebDriver protocol has limited APIs. For example, no networking status. (WebDriver rejected to implement it, they told you to use Proxy for debugging)
2T1Qka0rEiPr 10y ago
A few hours later: https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuN...
- yamaneko 10y ago
  After his announcement, Vitaly answered to the team: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQdE1A5FAwAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/PQ... > >Is there interest on your side in adopting Chromium as a runtime? There's some existing documentation [2] around the API and embedding, but admittedly, this would be some work. > We are interested. But I am afraid not in the current state. Currently, PhantomJS heavily relies on Qt and QtWebKit. It's not that easy to adopt Chrome as a new runtime. > But I think we could implement PhantomJS as a completely new (with the same API) project that will use Chrome - Phantomium!
  - Naracion 10y ago
    Also see https://github.com/Vitallium/phantomium https://github.com/Vitallium/phantomium

sbuccini 10y ago

I've been using post-robot and xcomponent a lot recently as I'm solving a bunch of similar challenges. Just wanted to say thank you for sharing your solutions and expertise.

bluepnume 10y ago
Glad they're working out for you! Let me know if you encounter any interesting situations they don't support.

Rumudiez 10y ago

What do you do to ensure the resources for closed windows are released?

bluepnume 10y ago
A combination of: - Making sure all promises are fulfilled or rejected, so window objects don't get caught indefinitely in closure scope for any .then() or .catch() handler functions. - Using WeakMaps as much as possible, when we have things that are tied to a particular window, like message listeners or response handlers in post-robot - Manually clearing up any global references to windows when we destroy an xcomponent instance Finding the references was the tricky bit. A lot of the effort was finding a leaky test-case, running in 100 times in succession, and deleting code until the memory graph was flat -- then figuring out what I'd just deleted that caused the leak. The problem started manifesting as I added more and more tests -- so now I'm actually checking my tests' memory usage on the fly and failing if they cross a threshold. Hopefully that should avoid getting into this kind of sticky situation ever again. https://github.com/paypal/paypal-checkout/blob/master/test/test.js#L42-L52 https://github.com/paypal/paypal-checkout/blob/master/test/t...

PatrickAuld 10y ago

We also did a POC with PhantomJS and found similar issues, as well as generally flakiness causing too many false negatives. Ended up not using it; I'm hoping this can simplify things are give something more solid to build on.

seanp2k2 10y ago
Same, ended up using ChromeDriver + XVFB on Jenkins. Works but it's still pretty slow.

LoSboccacc 10y ago

We're using xvfb and selenium* for testing and a proper headless support would be a hundred time more stable than the self restarting framebuffer can't wait to move to headless chrome *Yeah I know phantomjs is more cool these days but phantom doesn't support windows height so there's that.

akras14 10y ago
+1 for xvfb

akras14 10y ago

I replied to you on Twitter, but for the sake of discussion, there is a third option that may be worth exploring - using Chrome running in virtualized Windows Manager - https://www.alexkras.com/running-chrome-and-other-browsers-in-almost-headless-mode/ https://www.alexkras.com/running-chrome-and-other-browsers-i...

bluepnume 10y ago
Very cool. Gonna have to check this out.
amclennon 10y ago
Hello, former SDC coworker! The official Selenium Docker image uses the same technique to run headless Chrome / Firefox. From there you can just run docker run -d -P selenium/standalone-chrome and you'll get something a lot more lightweight than spinning up a new Vagrant VM.

jslove 10y ago

But can you run headless chrome on Chromeos? if not why not?