7 ms·

Headless Chrome is coming soon

293 points by AnbeSivam 10y ago

joshschreuder 10y ago
Could this be used for Selenium tests too? And with visual diff testing tools?
- mhluongo 10y ago
  I don't see why not. You can do it today with headless options like PhantomJS (headless WebKit), and this seems to be intended for similar uses.
  - logn 10y ago
    You can also do it with my project which is based only on Java, https://github.com/machinepublishers/jbrowserdriver https://github.com/machinepublishers/jbrowserdriver
    - boundlessdreamz 10y ago
      I haven't started using it but it looks like a great project (learned about it from ghostdriver homepage). There is a dearth of actively maintained projects that is compatible with selenium. While headless is not a requirement for us, having the node start and connect to selenium hub instead of selenium hub starting it on demand was extremely important. Phantomjs was the only other project which did that but it has not been very stable. Thank you for your work on this :)
- egeozcan 10y ago
  The required work for Selenium compatibility is tracked here: https://bugs.chromium.org/p/chromium/issues/detail?id=604324 https://bugs.chromium.org/p/chromium/issues/detail?id=604324 (And yes, it is a blocking issue).
fuzionmonkey 10y ago
This is awesome. I've been using Electron for browser tests which has been a big improvement over Phantom, but setting up xvfb can be a pain.
shimon_e 10y ago
Isn't chromium already bundled with chromedriver? Or at least it is on arch linux. (fyi, chromedriver conflicts chromium in aur because of this)
- foota 10y ago
  I believe the difference is that chromedriver still spins up all the normal chrome interface, which wouldn't be needed for the headless chrome. I think the stated deliverables help discern the intention: 1. A library which headless applications can link to to. 2. A sample application which demonstrates the use of headless APIs.
juliangamble 10y ago
If this would benefit you - please star this issue: https://bugs.chromium.org/p/chromium/issues/detail?id=546953 https://bugs.chromium.org/p/chromium/issues/detail?id=546953
juliangamble 10y ago
Another way to solve this problem is to have Chrome in a docker container that you can start up: https://github.com/SeleniumHQ/docker-selenium https://github.com/SeleniumHQ/docker-selenium
- killing_time 10y ago
  I use the Chrome and Firefox docker-selenium containers in Testcontainers [1][2], my project for running containers to support JUnit tests. I created this after numerous issues with PhantomJS compatibility and debuggability; Testcontainers instead uses the real browsers, and also offers automatic video recording of test sessions and VNC access for debugging. Headless chrome support sounds like a good step forward, but if visibility into what's going on is limited then I feel there's going to be some way to go. Perhaps chrome remote debugging support? [1]: http://testcontainers.viewdocs.io/testcontainers-java/ http://testcontainers.viewdocs.io/testcontainers-java/ [2]: https://rnorth.org/26/better-junit-selenium-testing-with-docker-and-testcontainers https://rnorth.org/26/better-junit-selenium-testing-with-doc...
  - duaneb 10y ago
    Just because it is headless doesn't mean it can't render to a serializable format (like a pdf)
  - elgalu 10y ago
    Thanks for noticing, I upgraded the "Alternatives" section at https://github.com/elgalu/docker-selenium#alternatives https://github.com/elgalu/docker-selenium#alternatives
emilong 10y ago
This is great news. In general I'm hoping this makes it easier to do browser testing in more CI services, rather than isolating this type of testing to services that have to specialize in it just to get it to to work. One (minor?) benefit over Phantom is having working file uploads, though it would be awesome to have that in Phantom too.
- toasterlovin 10y ago
  We upload files in our integration tests using PhantomJS. At least, I'm pretty sure we do (we use Capybara to drive PhantomJS). Am I missing something?
jsprogrammer 10y ago
I just use XVFB. xvfb-run --server-args='-screen 0 1024x768x24 -ac' chromium-browser --user-data-dir <url>
- snuxoll 10y ago
  I do to, but not for testing (we don't actually do any scripted UI tests, we should, but they are time consuming to write). We have a Python service that uses Firefox, Selenium and XVFB (through the awesome PyVirtualDisplay library, check it out) to log into a website that uses an annoying Java applet + scripting for authentication. This ended up cutting off a huge headache where previously we were doing this in a Windows scheduled task that needed an active console session to do anything.
qznc 10y ago
I wanted this 10 years ago. It is fascinating that it took so long already.
- myle 10y ago
  There was no Chrome browser 10 years ago.
  - qznc 10y ago
    It does not matter which browser. All of them please.
    - xufi 10y ago
      Even IE and Edge sarcastic voice
frik 10y ago
From the bug comments: headless: Implement screenshot capturing (based on patch from skyostil@, also sets default window size to 800x600 to enable basic snapshot support) With the --screenshot option, headless shell will save a PNG screenshot of the loaded page.
- hkjgkjy 10y ago
  Just out of curiosity - when does Chrome consider a page "loaded"? Is it the classic window.onload or something fancier?
  - oxplot 10y ago
    Not an answer to your question, but somewhat related: I recently worked on an automated thumbnail generator which used PhantomJS (headless Webkit browser) to load some 20000 pages and screenshot them. The problem was that these pages had javascript code that would start loading resources long after `onload` and friends had fired. So the way I implemented onload was to monitor all resources and if there was delay of few seconds after the last one had finished, I'd consider the page loaded. It turned out to be quite robust.
  - onion2k 10y ago
    There are a few events that get fired during a page load. DOMCotentLoaded is fired when the page and all its assets are loaded, so that's usually considered the point at which it's finished loading. With modern JS based pages that sometimes isn't enough though, so new browsers have documentRequestIdleCallback which fires a callback when the browser isn't busy, which is quite useful.
    - jayflux 10y ago
      Are you not confusing DOMContentLoaded with window.load? DCL doesn't wait for assets last time I checked, it only waits until the page is parsed and a tree has been created. .load waits for assets https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded https://developer.mozilla.org/en-US/docs/Web/Events/DOMConte...
jakozaur 10y ago
That would be so awesome. The current alternatives: 1. Use PhantomJS: It easy to script, but it lags over real browser by few years and development is stagnant. Many real website doesn't work in PhantomJS. 2. Use Chrome/Firefox xvfb. You have real browser, but scripting is hard. E.g. even hello world examples like make a thumbnail of a website takes a lot of time to get right. According to some rumors, headless Chrome existed before even Chrome was released to the public. Google use it to do web scraping. However, though headless browsers are great for developers, they are also great for spammers and ad fraud. So the main developer likely has conflict of interests whether to invest resources into making headless Chrome public.
- andrea_s 10y ago
  What about Selenium? Is it already covered in point 2?
  - jakozaur 10y ago
    Yes you can do it with Selenium (I mean that in point 2), but IMO it is hard to setup and resource consuming. E.g. You need separate VM or Docket to have browser with virtual frame buffer. Using Web Driver also got a steep learning curve. Well, some people also ask why world ever need Ruby with Rails or Python with Django? You can write any website in Enterprise Java since late 1990s. Same Docker vs. Virtual Machines or Atom vs. Vim/Emacs. Even if capability is there, there is room for improvement. Especially if you can make it easier to use and make developers more productive by an order of magnitude.
    - curiousgal 10y ago
      As someone with a horrible internet connection, I've often resorted to using a VPS, Selenium + Firefox+ xvfb is definitely resourcee-intensive and a pain.
    - zurn 10y ago
      No functional need for a separate VM: xvfb-run -a will automatically pick the next free display number and set up the environment pointing to xvfb's $DISPLAY. Of course container/vm isolation is still a good idea for security and repeatability reasons.
gildas 10y ago
Great news! I hope this will allow to change the hard-coded limit of the maximum number of connection per server in Chrome [1]. [1] https://bugs.chromium.org/p/chromium/issues/detail?id=85323 https://bugs.chromium.org/p/chromium/issues/detail?id=85323
erikb 10y ago
Please link the source and not twitter next time.
shanemhansen 10y ago
I'm most excited about integration testing on sites with http/2 deployed.

293 points by AnbeSivam 10y ago

joshschreuder 10y ago

Could this be used for Selenium tests too? And with visual diff testing tools?

mhluongo 10y ago
I don't see why not. You can do it today with headless options like PhantomJS (headless WebKit), and this seems to be intended for similar uses.
- logn 10y ago
  You can also do it with my project which is based only on Java, https://github.com/machinepublishers/jbrowserdriver https://github.com/machinepublishers/jbrowserdriver
  - boundlessdreamz 10y ago
    I haven't started using it but it looks like a great project (learned about it from ghostdriver homepage). There is a dearth of actively maintained projects that is compatible with selenium. While headless is not a requirement for us, having the node start and connect to selenium hub instead of selenium hub starting it on demand was extremely important. Phantomjs was the only other project which did that but it has not been very stable. Thank you for your work on this :)
egeozcan 10y ago
The required work for Selenium compatibility is tracked here: https://bugs.chromium.org/p/chromium/issues/detail?id=604324 https://bugs.chromium.org/p/chromium/issues/detail?id=604324 (And yes, it is a blocking issue).

fuzionmonkey 10y ago

This is awesome. I've been using Electron for browser tests which has been a big improvement over Phantom, but setting up xvfb can be a pain.

shimon_e 10y ago

Isn't chromium already bundled with chromedriver? Or at least it is on arch linux. (fyi, chromedriver conflicts chromium in aur because of this)

foota 10y ago
I believe the difference is that chromedriver still spins up all the normal chrome interface, which wouldn't be needed for the headless chrome. I think the stated deliverables help discern the intention: 1. A library which headless applications can link to to. 2. A sample application which demonstrates the use of headless APIs.

juliangamble 10y ago

If this would benefit you - please star this issue: https://bugs.chromium.org/p/chromium/issues/detail?id=546953 https://bugs.chromium.org/p/chromium/issues/detail?id=546953

juliangamble 10y ago

Another way to solve this problem is to have Chrome in a docker container that you can start up: https://github.com/SeleniumHQ/docker-selenium https://github.com/SeleniumHQ/docker-selenium

killing_time 10y ago
I use the Chrome and Firefox docker-selenium containers in Testcontainers [1][2], my project for running containers to support JUnit tests. I created this after numerous issues with PhantomJS compatibility and debuggability; Testcontainers instead uses the real browsers, and also offers automatic video recording of test sessions and VNC access for debugging. Headless chrome support sounds like a good step forward, but if visibility into what's going on is limited then I feel there's going to be some way to go. Perhaps chrome remote debugging support? [1]: http://testcontainers.viewdocs.io/testcontainers-java/ http://testcontainers.viewdocs.io/testcontainers-java/ [2]: https://rnorth.org/26/better-junit-selenium-testing-with-docker-and-testcontainers https://rnorth.org/26/better-junit-selenium-testing-with-doc...
- duaneb 10y ago
  Just because it is headless doesn't mean it can't render to a serializable format (like a pdf)
- elgalu 10y ago
  Thanks for noticing, I upgraded the "Alternatives" section at https://github.com/elgalu/docker-selenium#alternatives https://github.com/elgalu/docker-selenium#alternatives

emilong 10y ago

This is great news. In general I'm hoping this makes it easier to do browser testing in more CI services, rather than isolating this type of testing to services that have to specialize in it just to get it to to work. One (minor?) benefit over Phantom is having working file uploads, though it would be awesome to have that in Phantom too.

toasterlovin 10y ago
We upload files in our integration tests using PhantomJS. At least, I'm pretty sure we do (we use Capybara to drive PhantomJS). Am I missing something?

jsprogrammer 10y ago

I just use XVFB. xvfb-run --server-args='-screen 0 1024x768x24 -ac' chromium-browser --user-data-dir <url>

snuxoll 10y ago
I do to, but not for testing (we don't actually do any scripted UI tests, we should, but they are time consuming to write). We have a Python service that uses Firefox, Selenium and XVFB (through the awesome PyVirtualDisplay library, check it out) to log into a website that uses an annoying Java applet + scripting for authentication. This ended up cutting off a huge headache where previously we were doing this in a Windows scheduled task that needed an active console session to do anything.

qznc 10y ago

I wanted this 10 years ago. It is fascinating that it took so long already.

myle 10y ago
There was no Chrome browser 10 years ago.
- qznc 10y ago
  It does not matter which browser. All of them please.
  - xufi 10y ago
    Even IE and Edge sarcastic voice

frik 10y ago

From the bug comments: headless: Implement screenshot capturing (based on patch from skyostil@, also sets default window size to 800x600 to enable basic snapshot support) With the --screenshot option, headless shell will save a PNG screenshot of the loaded page.

hkjgkjy 10y ago
Just out of curiosity - when does Chrome consider a page "loaded"? Is it the classic window.onload or something fancier?
- oxplot 10y ago
  Not an answer to your question, but somewhat related: I recently worked on an automated thumbnail generator which used PhantomJS (headless Webkit browser) to load some 20000 pages and screenshot them. The problem was that these pages had javascript code that would start loading resources long after `onload` and friends had fired. So the way I implemented onload was to monitor all resources and if there was delay of few seconds after the last one had finished, I'd consider the page loaded. It turned out to be quite robust.
- onion2k 10y ago
  There are a few events that get fired during a page load. DOMCotentLoaded is fired when the page and all its assets are loaded, so that's usually considered the point at which it's finished loading. With modern JS based pages that sometimes isn't enough though, so new browsers have documentRequestIdleCallback which fires a callback when the browser isn't busy, which is quite useful.
  - jayflux 10y ago
    Are you not confusing DOMContentLoaded with window.load? DCL doesn't wait for assets last time I checked, it only waits until the page is parsed and a tree has been created. .load waits for assets https://developer.mozilla.org/en-US/docs/Web/Events/DOMContentLoaded https://developer.mozilla.org/en-US/docs/Web/Events/DOMConte...

jakozaur 10y ago

That would be so awesome. The current alternatives: 1. Use PhantomJS: It easy to script, but it lags over real browser by few years and development is stagnant. Many real website doesn't work in PhantomJS. 2. Use Chrome/Firefox xvfb. You have real browser, but scripting is hard. E.g. even hello world examples like make a thumbnail of a website takes a lot of time to get right. According to some rumors, headless Chrome existed before even Chrome was released to the public. Google use it to do web scraping. However, though headless browsers are great for developers, they are also great for spammers and ad fraud. So the main developer likely has conflict of interests whether to invest resources into making headless Chrome public.

andrea_s 10y ago
What about Selenium? Is it already covered in point 2?
- jakozaur 10y ago
  Yes you can do it with Selenium (I mean that in point 2), but IMO it is hard to setup and resource consuming. E.g. You need separate VM or Docket to have browser with virtual frame buffer. Using Web Driver also got a steep learning curve. Well, some people also ask why world ever need Ruby with Rails or Python with Django? You can write any website in Enterprise Java since late 1990s. Same Docker vs. Virtual Machines or Atom vs. Vim/Emacs. Even if capability is there, there is room for improvement. Especially if you can make it easier to use and make developers more productive by an order of magnitude.
  - curiousgal 10y ago
    As someone with a horrible internet connection, I've often resorted to using a VPS, Selenium + Firefox+ xvfb is definitely resourcee-intensive and a pain.
  - zurn 10y ago
    No functional need for a separate VM: xvfb-run -a will automatically pick the next free display number and set up the environment pointing to xvfb's $DISPLAY. Of course container/vm isolation is still a good idea for security and repeatability reasons.

gildas 10y ago

Great news! I hope this will allow to change the hard-coded limit of the maximum number of connection per server in Chrome [1]. [1] https://bugs.chromium.org/p/chromium/issues/detail?id=85323 https://bugs.chromium.org/p/chromium/issues/detail?id=85323

erikb 10y ago

Please link the source and not twitter next time.

shanemhansen 10y ago

I'm most excited about integration testing on sites with http/2 deployed.