59 ms·

Getting Started with Headless Chrome

657 points by uptown 10y ago

llimllib 10y ago
I wrote a little ipython notebook with a stupid simple example of getting headless chrome up and talking to it via a websocket: https://gist.github.com/llimllib/7f6143a1a6955d243161b2fec23b8f4d https://gist.github.com/llimllib/7f6143a1a6955d243161b2fec23... Then I started on a python library that would handle communication with chrome in a real way using asyncio: https://twitter.com/llimllib/status/855433309375082496 https://twitter.com/llimllib/status/855433309375082496 If that's a thing that you're interested in, let me know. https://github.com/llimllib/chrome-control/tree/wstest https://github.com/llimllib/chrome-control/tree/wstest
- fake-name 10y ago
  FWIW, I wrote a wrapper that exposes the remote debugger interface in python functions here: https://github.com/fake-name/ChromeController https://github.com/fake-name/ChromeController It works by reading the interface descripton JSON files, and dynamically generating the API. It doesn't handle the higher-level stuff (async, etc...), but for synchronous things, it's quite nice. -------- As an aside, if you're interested in this sort of thing, the chrome devtools protocol has a issue tracker here: https://github.com/ChromeDevTools/devtools-protocol/issues https://github.com/ChromeDevTools/devtools-protocol/issues Yes, it's on github, no idea why, when they already have their monorail issue tracker.
  - ebidel 10y ago
    FWIW, we created the Github issue tracker b/c it's a bit more approachable for everyone. Github is also a much better place for discussions.
    - fake-name 10y ago
      Ah, that makes sense. Monowall is kind of poor. The inability to edit comments is particularly annoying.
  - llimllib 10y ago
    I saw that! And it was helpful to me getting started, thanks. I wanted to generate types from the protocol file rather than generate them at run-time, so that was the first thing I did. I think it helps debuggability/autocompletability/readability to have the types reified; check it out if you're interested, all the capital-letter-named files are generated.
    - fake-name 10y ago
      It's a few-line change to output the AST to code via astor (it's actually part of how I got everything working in the first place). You get full executable code with docstrings an everything, though no support for generating inline comments (I think that changed in 3.6, IIRC). Would you be interested in a patch that does something like that? It should be a heck of a lot more robust then what you're currently doing (which is manual code generation?). ------ Also, if you're interested in doing things with the debug protocol, https://github.com/ChromeDevTools/devtools-protocol/issues/6 https://github.com/ChromeDevTools/devtools-protocol/issues/6 might be relevant. It's an issue about how there's no good way to determine the remote debug interface version, since the version information that's present never changes, and they keep moving functions.
      - llimllib 10y ago
        Definitely! And yeah the ad hoc protocol thing is... suboptimal
        fake-name 10y ago
        I put together a caching system so the generated wrapper is made available for overview/manual-patching, etc.... It currently validates the generated file against a generated-from-the-json file at import, and falls back to blindly importing the json file if the tooling to generate the class definition is not available. Ideally, any patching for the class file should really be done as a subclass, as well as any more complex logic. That way, if the protocol json file is changed, it can be just updated without breaking anything. Anyways, generated class is here: https://github.com/fake-name/ChromeController/blob/master/ChromeController/Generator/Generated.py https://github.com/fake-name/ChromeController/blob/master/Ch... Oh, as an aside, you're actually missing a significant chunk of the protocol json file, mainly because there are actually two of them. There's a `browser_protocol.json` AND a `browser_protocol.json` file. They're both produced as build artifacts, so I can't point to them in the source, but the total json size should be > 800KB (mirror in chromecontroller: https://github.com/fake-name/ChromeController/tree/master/ChromeController/protocols https://github.com/fake-name/ChromeController/tree/master/Ch... ). As it is, this should kind of just splat into your source tree, but I don't want to move everything around without at least asking you how you want things structured.
floatboth 10y ago
No mention of WebDriver, only Chrome's devtools protocol :( There's probably a proxy or something, but would be nice to see completely integrated native support.
- llimllib 10y ago
  correct me if I'm wrong, but doesn't webdriver communicate with Chrome via the devtools protocol?
  - floatboth 10y ago
    WebDriver is a protocol: https://www.w3.org/TR/webdriver/ https://www.w3.org/TR/webdriver/
    - llimllib 10y ago
      yeah yeah yeah I'm sorry I mean Chrome webdriver, I think that was clear from context? As in selenium -> webdriver protocol -> chrome webdriver -> devtools protocol -> chrome instance
      - floatboth 10y ago
        it's called ChromeDriver and I kinda mentioned it originally — "There's probably a proxy or something" — and said that I want native fully integrated support :)
        llimllib 10y ago
        ah now I follow, my bad :)
- talawahdotnet 10y ago
  That proxy already exists. It is called ChromeDriver[1][2] and was developed in collaboration between the Chromium and Selenium teams. It is the same way that Selenium/Webdriver controls regular Chrome now. It would have been nice if they at least mentioned it in the article since Selenium is such a popular browser automation tool. They do in fact mention it on the README page for headless chromium. [1] https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver [2] https://sites.google.com/a/chromium.org/chromedriver/home https://sites.google.com/a/chromium.org/chromedriver/home [3] https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md https://chromium.googlesource.com/chromium/src/+/lkgr/headle...
jaffoneh 10y ago
Clarity just moved to headless Chrome from PhantomJS today: https://github.com/vmware/clarity/pull/803 https://github.com/vmware/clarity/pull/803
- arwineap 10y ago
  Clarity moved to chrome webdriver, but they are not using the headless features. It's standard selenium webdriver, and xvfb https://github.com/vmware/clarity/blob/master/.travis.yml#L42 https://github.com/vmware/clarity/blob/master/.travis.yml#L4...
  - jeeyun 10y ago
    hi @arwineap, you are right in saying that Clarity is using chrome webdriver + xvfb. That's for the css regression testing, which we haven't looked into moving to use headless Chrome. We've switched our unit tests to use the headless Chrome: https://github.com/vmware/clarity/blob/master/build/karma.conf.js#L95 https://github.com/vmware/clarity/blob/master/build/karma.co...
    - arwineap 10y ago
      Hrm, but those also run in travis right? I noticed at the bottom of the karma.conf it checks to see if it's in travis. The travis image is installing chrome stable, which is not 59; AFAIK the headless feature is only available in beta right now which versions out to 59. I admit I'm not sure if your unit tests are running in travis, but if they are, I'd wonder if the --headless flag is just getting dropped as an unknown flag. Curious either way actually so let me know; I'm looking at starting the testing moving from phantomjs to headless which is why I was sleuthing out Clarity's move
      - paulirish 10y ago
        > AFAIK the headless feature is only available in beta right now which versions out to 59. Headless is pretty stable on Linux in 58. 59 is really when headless became usable on Mac. Still in beta on Windows.
deleted 10y ago
[deleted]
codewithcheese 10y ago
What are the pros and cons of running headless Chrome compared to running Chrome in a virtual display driver like Xvfb?
- simlevesque 10y ago
  Xvfb is way harder to setup. Also, I assume it uses less ressources so it scales better.
  - codewithcheese 10y ago
    Yes, I assume that the drawing process may be different. However, it's probably still possible to take a screenshot. I would like to know what sort of performance differential I can expect.
    - floatboth 10y ago
      Screenshot functionality is, in fact, advertised in the linked article.
  - cryptarch 10y ago
    I use xvfb-run on Debian and it's a breeze: xvfb-run -s '-screen 0 1280x1024x8' ./cucumber Or are you using more advanced Xvfb features, or perhaps running inside containers?
    - michaelmcmillan 10y ago
      In my experience it's not a breeze: https://github.com/SeleniumHQ/docker-selenium/issues/87#issuecomment-297944128 https://github.com/SeleniumHQ/docker-selenium/issues/87#issu...
  - bdcravens 10y ago
    xvfb can be a bit tough to setup, but Selenium's Docker images make this a breeze, and they even have a debug mode with automatic VNC.
hackcasual 10y ago
I contributed a small patch to get this working on Windows. We were a PhantomJS shop, but it was just so unstable, thought we'd give this a shot. Have been running on it for over 2 months now and it's dropped test failures due to intermittent issues to near 0.
nmca 10y ago
I've been using headless chrome over the last couple of months to implement a P2P network between browsers. Worked great!
- ficklepickle 10y ago
  That sounds interesting. Do you have any more details or a blog post you can share?
wildpeaks 10y ago
If you plan to use it for automated tests, note that headless Chrome is only available on Mac & Linux (in Chrome 59), not on Windows yet.
- ebidel 10y ago
  Windows support is coming. You can follow along at https://bugs.chromium.org/p/chromium/issues/detail?id=686608 https://bugs.chromium.org/p/chromium/issues/detail?id=686608
elorant 10y ago
How's that different from running Selenium with Chrome?
- matharmin 10y ago
  Selenium opens a real chrome window. So it's automated, but not headless. You will also be able to use this headless support in the new chrome to run it without the window with selenium.
aantix 10y ago
Capybara with headless Chrome.. http://blog.faraday.io/headless-chromium/ http://blog.faraday.io/headless-chromium/
davidw 10y ago
So much for that: [0501/120318.494691:ERROR:gl_implementation.cc(245)] Failed to load /opt/google/chrome/libosmesa.so: /opt/google/chrome/libosmesa.so: cannot open shared object file: No such file or directory
- ebidel 10y ago
  As noted, you'll need the --disable-gpu flag for now. Hopefully that will go away around ~Chrome 60. See - https://bugs.chromium.org/p/chromium/issues/detail?id=546953#c152 https://bugs.chromium.org/p/chromium/issues/detail?id=546953... - https://bugs.chromium.org/p/chromium/issues/detail?id=695212 https://bugs.chromium.org/p/chromium/issues/detail?id=695212
  - davidw 10y ago
    Be convenient if all the examples had it for those of us idly pasting stuff in while code compiles :-)
colordrops 10y ago
Will the "--disable-gpu" flag no longer be needed in the future because headless mode will automatically disable the GPU, or because GPU support will be added to headless mode? I really hope it's the latter.
- ebidel 10y ago
  It's just a bug :) https://news.ycombinator.com/item?id=14240460 https://news.ycombinator.com/item?id=14240460
foob 10y ago
I recently wrote a guide to using headless Chrome with Selenium and Chrome WebDriver [1]. I thought that some people in this thread might find it useful considering that the submitted article doesn't mention Selenium at all. [1] - https://intoli.com/blog/running-selenium-with-headless-chrome/ https://intoli.com/blog/running-selenium-with-headless-chrom...
- xdissent 10y ago
  Your guide mentions setting window size and taking screenshots, which I thought were not currently working via chromedriver. Do you know if that was fixed or is there something else going on?
  - foob 10y ago
    Setting window size via the ChromeDriver API didn't work for me but I was able to set it using a ChromeOptions command-line argument as I do in the guide. Screenshots seem to work fine on Linux but I haven't tried on other platforms.
- yawz 10y ago
  Thanks for sharing the link. Just thought you'd like to know that there's a small typo at the title: "Runnng" instead of "Running".
VikingCoder 10y ago
Huh. So, if you can express your PNG / PDF rendering in terms of HTML, SVG, Canvas, and WebGL... How much easier, faster, more reliable would it be to use Headless Chrome and --screenshot, rather than other means?
- joshribakoff 10y ago
  HTML & CSS are not really suitable for "pixel perfect" text. Even text in SVG works just like text in an HTML document in the sense there is not a way to control precisely where text will wrap. You'd have to disable the browser's word wrapping & come up with your own logic for where to insert <br /> tags, even then you have issues like browsers ignoring CSS rules because of 3rd party browser extensions, missing fonts & such. Look at this jQuery plugin for evidence - http://simplefocus.com/flowtype/ http://simplefocus.com/flowtype/ As you resize your browser, you see the browser re-layout the page & the text jumps around. The text wraps at different positions as you resize your browser around. You'd have to essentially render text to an image server-side, then scale the resulting image in the browser. And if you already have an image, there's no need for tools like wkhtmltoX or phantom for creating an image (you already have an image)... So headless browsers are not suitable for rasterizing documents that contain text [in my experience]. Converting HTML to PNG/PDF is like comparing apples & oranges. The conversion will be imperfect.
  - SigmundA 10y ago
    Tell that to PrinceXML: https://www.princexml.com/samples/ https://www.princexml.com/samples/ Print CSS is pretty complete if only the major browsers would support it.
  - Kiro 10y ago
    That's a small trade-off I'm willing to take. It beats having to have a completely separate engine for the PDFs (compared to just using what you're already displaying on the web) and it's way more complete than using something like DOMPDF which doesn't even have support for some of the most basic CSS features.
  - VikingCoder 10y ago
tehlike 10y ago
only if there was an option to render pdf and save it to disk in headless mode. Still great step.
- spatten 10y ago
  scroll down a bit and look for `--print-to-pdf`.
  - tehlike 10y ago
    Thanks, i was mostly thinking in the interactive mode (read: selenium). Currently this seems to be available in devtools protocol.
  - joshribakoff 10y ago
    Does not work for me. $ google-chrome --disable-gpu --headless --print-to-pdf https://www.chromestatus.com/ https://www.chromestatus.com/ && ls -la .pdf ls: cannot access '.pdf': No such file or directory
    - ebidel 10y ago
      Make sure you're using Chrome 59. If you don't have that yet, try Chrome Canary.
      - deleted 10y ago
        [deleted]
      - joshribakoff 10y ago
        The Google Chrome website installs v58 when I press download, and when I try to install Canary: "Chrome Canary is currently not available on the Linux platform."
        TheCoelacanth 10y ago
        Get a build of Chromium that is more recent than this feature
dennykane 10y ago
Nothing to add to this exact topic, I just love all of the cool stuff coming fast and furious out of the pipeline with Chrome. Just as "google" has become a word in its own right to mean "finding information online", I think "chrome" is going to have its own little space in the english lexicon to mean "that piece of software that you use to find information, and do much, much more besides". The thing I've been working on for the past ~5 years as my addition to the "chrome-iverse" is called "Linux on the Web". It is basically the first ever online Linux distro, which, by the way, this @ebidel character has really helped out with via his work on the FileSystem API. Find it at: https://linuxontheweb.appspot.com https://linuxontheweb.appspot.com Learn more here: https://www.youtube.com/watch?v=9tl8I8YcH7g https://www.youtube.com/watch?v=9tl8I8YcH7g (you can skip to about 17:15 to watch me using it via a Hangouts screencast).
- copperx 10y ago
  I hate to be that person, but why is it so exciting to exchange our privacy and freedom for a shiny piece of software? What's so wrong with Firefox + PhantomJS?
  - dennykane 10y ago
    I love to be that person that hates on people that hate to be that person. What I'm trying to say is simple, yet profound. Mozilla is filled with losers. Literally, Mozilla came out of the ashes of a ridiculous state of affairs in the late '90s called Netscape. Their only contribution anymore is writing documentation. Google is filled with winners. While everyone was waving their hands around so furiously at Netscape during the dotcom blowup, a mild-mannered young man named Lawrence Page was busy doing it the right way, filled with humility, wonder, and unlimited curiosity. I can literally smell Mozilla types from a mile away in forums like this. Thanks for the tabs idea, Mozilla. Other than that, no... body... cares.
  - waingake 10y ago
    I agree with the sentiment. However an issue I had using PhantomJS is that it doesn't support modern JS rendered sites ( doesn't support much es5 at all it seems) I needed to create a pdf from such a site and couldn't get PhantomJS to do it. Headless chrome saved the day.
throwasehasdwi 10y ago
why is Google doing this? They're helping everyone else build a spider that could easily serve as the front end to a new search engine. Is this their way of fighting back against the internet closing off? Give everyone a spider so perfect that walling in your content becomes impractical? I like the idea of Google lighting the match that burns down the walled gardens, but if not this than why? It could be anything, but I feel like there's definitely another motive to Google unleashing something that could bite their core business.
- skybrian 10y ago
  You're looking for strategy everywhere. People at big companies do a lot of things and not every feature is high-level strategic. I don't know their motivations, but the first use that comes to mind is to make automatically testing your own web app using Chrome easier. Most teams working on web apps could use better testing, whether they work for Google or not.
- IshKebab 10y ago
  lol no. Nobody building a search engine would use this, and very similar alternatives (e.g. PhantomJS) have been available for years. This is just a useful tool. They probably use it internally for automated testing and research.
- bdcravens 10y ago
  Spidering is a tiny part of building a search engine, and this piece is a tiny piece of a spider. Headless Chrome helps someone compete with Google as much as Basecamp's work on Rails helps someone compete with Basecamp.
- tyingq 10y ago
  If I was trying to control scraping and get a view into it, offering the best scraping tool might be good start. I'm pretty sure headless mode exposes something you could fingerprint it with, server side.
mc007ibi 10y ago
fantastic news, is that in any way useful for server-side rendering? i can imagine well to pre render here or there things though
rsync 10y ago
Off-topic, but I'm just going to start bringing this up in every thread like this ... We need a recipe / toolchain for running a browser instance in a chroot jail. With a head. GUI. On the desktop. I want to fire up a "banking browser" (or "danger browser" or "social media browser") that runs on its own IP, in its own chroot, and I can revert to virgin snapshot state anytime I want ... and I don't want to fire up a full-blown VM for eac of these ... What is standing in the way of making this happen ? Who is working on things related to this that I can donate funds to or establish a bounty ?
- noja 10y ago
  Sounds like Qubes OS https://www.qubes-os.org/ https://www.qubes-os.org/
  - rsync 10y ago
    The idea here is that I set up a chroot jail for firefox or chrome and configure it with things like local filesystem for cookies and cache and certs, etc. It would also get its own unique IP, this jail. Then I fire up firefox inside that chroot jail and use it to visit some websites ... and then I can wipe the whole thing out and redeploy again later, starting from scratch. I don't need to trust incognito mode, I don't need to trust wiping cache or tabs talking to each other (or not) and I can worry a lot less about browser level exploits. I can even put in firewall rules so that my "banking" instance can only talk to boa.com and scotttrade.com (or whatever). It's totally workable (and I have done it) with vmware. Make a "bank browsing" VM and revert to pristine snapshot every day. The problem is that this is terribly heavyweight and overkill when I don't need a full blown VM. It's not even really a browser issue - the real issue is, how do you jail a GUI application in X such that that window is in a chroot jail, distinct from the rest of your desktop?
    - napsterbr 10y ago
      Did this a few months ago, somewhat straightforward. A "generic" recipe for any UNIX-based OS would be: 1 - Create a container (Docker, Jails, maybe even chroot) 2 - Assign an ip to the container, NAT to it 3 - Install firefox on that container 4 - Run a SSHD server, enable X11 forwarding 5 - Mount the relevant container's folders to your root fs (eg: map $CONTAINER/.mozzila and $CONTAINER/Downloads to $HOME/jailed-browser/) 6 - Add an ssh config for quick alias 7 - Run `ssh container firefox` and profit Here's a nice example using FreeBSD jails (I remember following this tutorial, everything worked out fine): https://forums.freebsd.org/threads/53362/ https://forums.freebsd.org/threads/53362/ My experience with it, though, wasn't great. X11 forwarding through SSH was quite laggy (even after performing some optimizations on the connection). Good luck if you want to set-up audio/mic support. It's a nice solution for a one-time banking login, not for day-to-day use.
oceanswave 10y ago
If you're using Dotnetcore I've created a protocol generator here https://github.com/baristalabs/chrome-dev-tools https://github.com/baristalabs/chrome-dev-tools With a sample on use here https://github.com/baristalabs/chrome-dev-tools-sample https://github.com/baristalabs/chrome-dev-tools-sample
mxuribe 10y ago
Now, this is handy!
RazrFalcon 10y ago
Did someone try to render an SVG using it? I'm using QtWebKit for now, but it's just a huge pain.
- paulirish 10y ago
  Yup. Renders SVGs just as Chrome does. Really straightforward.
  - RazrFalcon 10y ago
    Can you provide a minimal example?
wslh 10y ago
Several years ago I tested the remote debug protocol and was insanely slow. Is it now faster, like evaluating scripts at the console?
- paulirish 10y ago
  I don't know we've heard any performance/latency issues before that concern issuing script evaluation over the protocol. Were you sending hundreds of scripts to evaluate? Or payloads far over 1MB? Regardless, if you're having problems.. definitely file a bug and get our attention. new.crbug.com or https://github.com/ChromeDevTools/devtools-protocol https://github.com/ChromeDevTools/devtools-protocol will do the trick. [I'm on the DevTools team]
bostonvaulter2 10y ago
Could this be used to create a screenshot service that'll automatically take screenshots of sites? Or is that not permissible under Chrome's license?
- paulirish 10y ago
  The Chromium license shouldn't place any restrictions on using the software like that. Go wild.
bgentry 10y ago
Do any of the major CI providers make this trivial to use yet? Travis CI, CircleCI, Buildkite, etc.? Anybody have any positive or negative experiences to report with actually using headless Chrome?
- kinlan 10y ago
  I lead the DevRel team for Web and Chrome so please take anything I say with a hint of bias.... I've found it pretty easy to use so far - https://paul.kinlan.me/chrome-on-home/ https://paul.kinlan.me/chrome-on-home/ and got it deployed in a number of environments including AppEngine. We are planning on getting this deployed in our [WebFundamentals](https://github.com/google/WebFundamentals/blob/master/.travis.yml https://github.com/google/WebFundamentals/blob/master/.travi...) Travis script to run automated testing on our dev docs site. Some things that I found hard: no documentation on how to get it running in a CI yet - some devs (Justin Ribeiro - https://hub.docker.com/r/justinribeiro/chrome-headless/ https://hub.docker.com/r/justinribeiro/chrome-headless/) got a docker file all set up which helped a lot, the devtools protocol whilst it has docs wasn't quite as simple as I had hoped and I had to guess of how to call the node library.
shmatt 10y ago
This is great progress. I use electron in node.js (via nightmare) because it can go from headless to visible on the fly I wonder if the Chrome team is ever going to support something similar
LoSboccacc 10y ago
why does it need a whole launcher app instead of taking a script to run on the webpage with a control api baffles me
pedalpete 10y ago
Does anybody know how screenshot is triggered? Is it on DOMContentLoaded? or some other event? Can we trigger this somehow post page load?
- waingake 10y ago
  I needed a delay before creating a pdf maybe this is helpful ? https://github.com/tim-field/urlToPdf/blob/master/url-to-pdf.js#L14 https://github.com/tim-field/urlToPdf/blob/master/url-to-pdf... ( using via node )
rogerwang 10y ago
The upcoming NW.js v0.23.0 beta version will support this usage: https://dl.nwjs.io/live-build/04-29-2017/nw23-3bd4af6-4d7f95f-5e6428a-f2085b9/v0.23.0-beta1/ https://dl.nwjs.io/live-build/04-29-2017/nw23-3bd4af6-4d7f95...
twothamendment 10y ago
Has anyone tried to use headless chrome to detect insecure content on what you hope is a secure page? I've only looked into it a bit, and so far failed. I'd like to know if a visitor to https://example.com/foo.html https://example.com/foo.html in chrome will get a warning about insecure content. I'm looking for a browser because in some cases the insecure content is loaded by javascript from a 3rd party. I don't know a way to do that besides fire it up in a browser and let it go. Is there a way to get the secure/insecure status from headless chrome?
- tyingq 10y ago
  The remote debugging feature can get to that. See: https://chromedevtools.github.io/debugger-protocol-viewer/tot/Security/ https://chromedevtools.github.io/debugger-protocol-viewer/to... Scroll down to InsecureContentStatus
jeyaganesh89 10y ago
Finally. PhantomJS has a competitor now.
alkoumpa 10y ago
Does it work without X-server?
- shif 10y ago
  It's headless so most likely yes.
- deleted 10y ago
  [deleted]
vanni 10y ago
Related recent news: "PhantomJS: Stepping down as maintainer" https://news.ycombinator.com/item?id=14105489 https://news.ycombinator.com/item?id=14105489 Also relevant: "Headless Chrome is Coming Soon" from PhantomJS dev mailing list https://groups.google.com/forum/#!msg/phantomjs-dev/S-mEBwuSgKQ/PQdE1A5FAwAJ https://groups.google.com/forum/#!msg/phantomjs-dev/S-mEBwuS...
kanchon1234 10y ago
hi
kanchon1234 10y ago
hi
kanchon1234 10y ago
hi
danso 10y ago
Was on the frontpage yesterday, with 605 points and 117 comments: https://news.ycombinator.com/item?id=14239194 https://news.ycombinator.com/item?id=14239194
sharpshadow 10y ago
Who would ever think Google will release a headless browser. Stunning.
callumprentice 10y ago
https://bitbucket.org/lindenlab/dullahan/src/default/README.md?at=default&fileviewer=file-view-default https://bitbucket.org/lindenlab/dullahan/src/default/README.... I imagine Headless Chrome will make this obsolete shortly but just in case anyone wants to play with it, here is Dullahan - A headless browser SDK that uses the Chromium Embedded Framework (CEF). It is designed to make it easier to write applications that render modern web content directly to a memory buffer, inject synthesized mouse and keyboard events as well as interact with features like JavaScript or cookies.

657 points by uptown 10y ago

I wrote a little ipython notebook with a stupid simple example of getting headless chrome up and talking to it via a websocket: https://gist.github.com/llimllib/7f6143a1a6955d243161b2fec23b8f4d https://gist.github.com/llimllib/7f6143a1a6955d243161b2fec23... Then I started on a python library that would handle communication with chrome in a real way using asyncio: https://twitter.com/llimllib/status/855433309375082496 https://twitter.com/llimllib/status/855433309375082496 If that's a thing that you're interested in, let me know. https://github.com/llimllib/chrome-control/tree/wstest https://github.com/llimllib/chrome-control/tree/wstest

fake-name 10y ago
FWIW, I wrote a wrapper that exposes the remote debugger interface in python functions here: https://github.com/fake-name/ChromeController https://github.com/fake-name/ChromeController It works by reading the interface descripton JSON files, and dynamically generating the API. It doesn't handle the higher-level stuff (async, etc...), but for synchronous things, it's quite nice. -------- As an aside, if you're interested in this sort of thing, the chrome devtools protocol has a issue tracker here: https://github.com/ChromeDevTools/devtools-protocol/issues https://github.com/ChromeDevTools/devtools-protocol/issues Yes, it's on github, no idea why, when they already have their monorail issue tracker.
- ebidel 10y ago
  FWIW, we created the Github issue tracker b/c it's a bit more approachable for everyone. Github is also a much better place for discussions.
  - fake-name 10y ago
    Ah, that makes sense. Monowall is kind of poor. The inability to edit comments is particularly annoying.
- llimllib 10y ago
  I saw that! And it was helpful to me getting started, thanks. I wanted to generate types from the protocol file rather than generate them at run-time, so that was the first thing I did. I think it helps debuggability/autocompletability/readability to have the types reified; check it out if you're interested, all the capital-letter-named files are generated.
  - fake-name 10y ago
    It's a few-line change to output the AST to code via astor (it's actually part of how I got everything working in the first place). You get full executable code with docstrings an everything, though no support for generating inline comments (I think that changed in 3.6, IIRC). Would you be interested in a patch that does something like that? It should be a heck of a lot more robust then what you're currently doing (which is manual code generation?). ------ Also, if you're interested in doing things with the debug protocol, https://github.com/ChromeDevTools/devtools-protocol/issues/6 https://github.com/ChromeDevTools/devtools-protocol/issues/6 might be relevant. It's an issue about how there's no good way to determine the remote debug interface version, since the version information that's present never changes, and they keep moving functions.
    - llimllib 10y ago
      Definitely! And yeah the ad hoc protocol thing is... suboptimal
      - fake-name 10y ago
        I put together a caching system so the generated wrapper is made available for overview/manual-patching, etc.... It currently validates the generated file against a generated-from-the-json file at import, and falls back to blindly importing the json file if the tooling to generate the class definition is not available. Ideally, any patching for the class file should really be done as a subclass, as well as any more complex logic. That way, if the protocol json file is changed, it can be just updated without breaking anything. Anyways, generated class is here: https://github.com/fake-name/ChromeController/blob/master/ChromeController/Generator/Generated.py https://github.com/fake-name/ChromeController/blob/master/Ch... Oh, as an aside, you're actually missing a significant chunk of the protocol json file, mainly because there are actually two of them. There's a `browser_protocol.json` AND a `browser_protocol.json` file. They're both produced as build artifacts, so I can't point to them in the source, but the total json size should be > 800KB (mirror in chromecontroller: https://github.com/fake-name/ChromeController/tree/master/ChromeController/protocols https://github.com/fake-name/ChromeController/tree/master/Ch... ). As it is, this should kind of just splat into your source tree, but I don't want to move everything around without at least asking you how you want things structured.

floatboth 10y ago

No mention of WebDriver, only Chrome's devtools protocol :( There's probably a proxy or something, but would be nice to see completely integrated native support.

llimllib 10y ago
correct me if I'm wrong, but doesn't webdriver communicate with Chrome via the devtools protocol?
- floatboth 10y ago
  WebDriver is a protocol: https://www.w3.org/TR/webdriver/ https://www.w3.org/TR/webdriver/
  - llimllib 10y ago
    yeah yeah yeah I'm sorry I mean Chrome webdriver, I think that was clear from context? As in selenium -> webdriver protocol -> chrome webdriver -> devtools protocol -> chrome instance
    - floatboth 10y ago
      it's called ChromeDriver and I kinda mentioned it originally — "There's probably a proxy or something" — and said that I want native fully integrated support :)
      - llimllib 10y ago
        ah now I follow, my bad :)
talawahdotnet 10y ago
That proxy already exists. It is called ChromeDriver[1][2] and was developed in collaboration between the Chromium and Selenium teams. It is the same way that Selenium/Webdriver controls regular Chrome now. It would have been nice if they at least mentioned it in the article since Selenium is such a popular browser automation tool. They do in fact mention it on the README page for headless chromium. [1] https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver [2] https://sites.google.com/a/chromium.org/chromedriver/home https://sites.google.com/a/chromium.org/chromedriver/home [3] https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md https://chromium.googlesource.com/chromium/src/+/lkgr/headle...

jaffoneh 10y ago

Clarity just moved to headless Chrome from PhantomJS today: https://github.com/vmware/clarity/pull/803 https://github.com/vmware/clarity/pull/803

arwineap 10y ago
Clarity moved to chrome webdriver, but they are not using the headless features. It's standard selenium webdriver, and xvfb https://github.com/vmware/clarity/blob/master/.travis.yml#L42 https://github.com/vmware/clarity/blob/master/.travis.yml#L4...
- jeeyun 10y ago
  hi @arwineap, you are right in saying that Clarity is using chrome webdriver + xvfb. That's for the css regression testing, which we haven't looked into moving to use headless Chrome. We've switched our unit tests to use the headless Chrome: https://github.com/vmware/clarity/blob/master/build/karma.conf.js#L95 https://github.com/vmware/clarity/blob/master/build/karma.co...
  - arwineap 10y ago
    Hrm, but those also run in travis right? I noticed at the bottom of the karma.conf it checks to see if it's in travis. The travis image is installing chrome stable, which is not 59; AFAIK the headless feature is only available in beta right now which versions out to 59. I admit I'm not sure if your unit tests are running in travis, but if they are, I'd wonder if the --headless flag is just getting dropped as an unknown flag. Curious either way actually so let me know; I'm looking at starting the testing moving from phantomjs to headless which is why I was sleuthing out Clarity's move
    - paulirish 10y ago
      > AFAIK the headless feature is only available in beta right now which versions out to 59. Headless is pretty stable on Linux in 58. 59 is really when headless became usable on Mac. Still in beta on Windows.

deleted 10y ago

[deleted]

codewithcheese 10y ago

What are the pros and cons of running headless Chrome compared to running Chrome in a virtual display driver like Xvfb?

simlevesque 10y ago
Xvfb is way harder to setup. Also, I assume it uses less ressources so it scales better.
- codewithcheese 10y ago
  Yes, I assume that the drawing process may be different. However, it's probably still possible to take a screenshot. I would like to know what sort of performance differential I can expect.
  - floatboth 10y ago
    Screenshot functionality is, in fact, advertised in the linked article.
- cryptarch 10y ago
  I use xvfb-run on Debian and it's a breeze: xvfb-run -s '-screen 0 1280x1024x8' ./cucumber Or are you using more advanced Xvfb features, or perhaps running inside containers?
  - michaelmcmillan 10y ago
    In my experience it's not a breeze: https://github.com/SeleniumHQ/docker-selenium/issues/87#issuecomment-297944128 https://github.com/SeleniumHQ/docker-selenium/issues/87#issu...
- bdcravens 10y ago
  xvfb can be a bit tough to setup, but Selenium's Docker images make this a breeze, and they even have a debug mode with automatic VNC.

hackcasual 10y ago

I contributed a small patch to get this working on Windows. We were a PhantomJS shop, but it was just so unstable, thought we'd give this a shot. Have been running on it for over 2 months now and it's dropped test failures due to intermittent issues to near 0.

nmca 10y ago

I've been using headless chrome over the last couple of months to implement a P2P network between browsers. Worked great!

ficklepickle 10y ago
That sounds interesting. Do you have any more details or a blog post you can share?

wildpeaks 10y ago

If you plan to use it for automated tests, note that headless Chrome is only available on Mac & Linux (in Chrome 59), not on Windows yet.

ebidel 10y ago
Windows support is coming. You can follow along at https://bugs.chromium.org/p/chromium/issues/detail?id=686608 https://bugs.chromium.org/p/chromium/issues/detail?id=686608

elorant 10y ago

How's that different from running Selenium with Chrome?

matharmin 10y ago
Selenium opens a real chrome window. So it's automated, but not headless. You will also be able to use this headless support in the new chrome to run it without the window with selenium.

aantix 10y ago

Capybara with headless Chrome.. http://blog.faraday.io/headless-chromium/ http://blog.faraday.io/headless-chromium/

davidw 10y ago

So much for that: [0501/120318.494691:ERROR:gl_implementation.cc(245)] Failed to load /opt/google/chrome/libosmesa.so: /opt/google/chrome/libosmesa.so: cannot open shared object file: No such file or directory

ebidel 10y ago
As noted, you'll need the --disable-gpu flag for now. Hopefully that will go away around ~Chrome 60. See - https://bugs.chromium.org/p/chromium/issues/detail?id=546953#c152 https://bugs.chromium.org/p/chromium/issues/detail?id=546953... - https://bugs.chromium.org/p/chromium/issues/detail?id=695212 https://bugs.chromium.org/p/chromium/issues/detail?id=695212
- davidw 10y ago
  Be convenient if all the examples had it for those of us idly pasting stuff in while code compiles :-)

colordrops 10y ago

Will the "--disable-gpu" flag no longer be needed in the future because headless mode will automatically disable the GPU, or because GPU support will be added to headless mode? I really hope it's the latter.

ebidel 10y ago
It's just a bug :) https://news.ycombinator.com/item?id=14240460 https://news.ycombinator.com/item?id=14240460

foob 10y ago

I recently wrote a guide to using headless Chrome with Selenium and Chrome WebDriver [1]. I thought that some people in this thread might find it useful considering that the submitted article doesn't mention Selenium at all. [1] - https://intoli.com/blog/running-selenium-with-headless-chrome/ https://intoli.com/blog/running-selenium-with-headless-chrom...

xdissent 10y ago
Your guide mentions setting window size and taking screenshots, which I thought were not currently working via chromedriver. Do you know if that was fixed or is there something else going on?
- foob 10y ago
  Setting window size via the ChromeDriver API didn't work for me but I was able to set it using a ChromeOptions command-line argument as I do in the guide. Screenshots seem to work fine on Linux but I haven't tried on other platforms.
yawz 10y ago
Thanks for sharing the link. Just thought you'd like to know that there's a small typo at the title: "Runnng" instead of "Running".

VikingCoder 10y ago

Huh. So, if you can express your PNG / PDF rendering in terms of HTML, SVG, Canvas, and WebGL... How much easier, faster, more reliable would it be to use Headless Chrome and --screenshot, rather than other means?

joshribakoff 10y ago
HTML & CSS are not really suitable for "pixel perfect" text. Even text in SVG works just like text in an HTML document in the sense there is not a way to control precisely where text will wrap. You'd have to disable the browser's word wrapping & come up with your own logic for where to insert <br /> tags, even then you have issues like browsers ignoring CSS rules because of 3rd party browser extensions, missing fonts & such. Look at this jQuery plugin for evidence - http://simplefocus.com/flowtype/ http://simplefocus.com/flowtype/ As you resize your browser, you see the browser re-layout the page & the text jumps around. The text wraps at different positions as you resize your browser around. You'd have to essentially render text to an image server-side, then scale the resulting image in the browser. And if you already have an image, there's no need for tools like wkhtmltoX or phantom for creating an image (you already have an image)... So headless browsers are not suitable for rasterizing documents that contain text [in my experience]. Converting HTML to PNG/PDF is like comparing apples & oranges. The conversion will be imperfect.
- SigmundA 10y ago
  Tell that to PrinceXML: https://www.princexml.com/samples/ https://www.princexml.com/samples/ Print CSS is pretty complete if only the major browsers would support it.
- Kiro 10y ago
  That's a small trade-off I'm willing to take. It beats having to have a completely separate engine for the PDFs (compared to just using what you're already displaying on the web) and it's way more complete than using something like DOMPDF which doesn't even have support for some of the most basic CSS features.
- VikingCoder 10y ago

tehlike 10y ago

only if there was an option to render pdf and save it to disk in headless mode. Still great step.

spatten 10y ago
scroll down a bit and look for `--print-to-pdf`.
- tehlike 10y ago
  Thanks, i was mostly thinking in the interactive mode (read: selenium). Currently this seems to be available in devtools protocol.
- joshribakoff 10y ago
  Does not work for me. $ google-chrome --disable-gpu --headless --print-to-pdf https://www.chromestatus.com/ https://www.chromestatus.com/ && ls -la .pdf ls: cannot access '.pdf': No such file or directory
  - ebidel 10y ago
    Make sure you're using Chrome 59. If you don't have that yet, try Chrome Canary.
    - deleted 10y ago
      [deleted]
    - joshribakoff 10y ago
      The Google Chrome website installs v58 when I press download, and when I try to install Canary: "Chrome Canary is currently not available on the Linux platform."
      - TheCoelacanth 10y ago
        Get a build of Chromium that is more recent than this feature

dennykane 10y ago

Nothing to add to this exact topic, I just love all of the cool stuff coming fast and furious out of the pipeline with Chrome. Just as "google" has become a word in its own right to mean "finding information online", I think "chrome" is going to have its own little space in the english lexicon to mean "that piece of software that you use to find information, and do much, much more besides". The thing I've been working on for the past ~5 years as my addition to the "chrome-iverse" is called "Linux on the Web". It is basically the first ever online Linux distro, which, by the way, this @ebidel character has really helped out with via his work on the FileSystem API. Find it at: https://linuxontheweb.appspot.com https://linuxontheweb.appspot.com Learn more here: https://www.youtube.com/watch?v=9tl8I8YcH7g https://www.youtube.com/watch?v=9tl8I8YcH7g (you can skip to about 17:15 to watch me using it via a Hangouts screencast).

copperx 10y ago
I hate to be that person, but why is it so exciting to exchange our privacy and freedom for a shiny piece of software? What's so wrong with Firefox + PhantomJS?
- dennykane 10y ago
  I love to be that person that hates on people that hate to be that person. What I'm trying to say is simple, yet profound. Mozilla is filled with losers. Literally, Mozilla came out of the ashes of a ridiculous state of affairs in the late '90s called Netscape. Their only contribution anymore is writing documentation. Google is filled with winners. While everyone was waving their hands around so furiously at Netscape during the dotcom blowup, a mild-mannered young man named Lawrence Page was busy doing it the right way, filled with humility, wonder, and unlimited curiosity. I can literally smell Mozilla types from a mile away in forums like this. Thanks for the tabs idea, Mozilla. Other than that, no... body... cares.
- waingake 10y ago
  I agree with the sentiment. However an issue I had using PhantomJS is that it doesn't support modern JS rendered sites ( doesn't support much es5 at all it seems) I needed to create a pdf from such a site and couldn't get PhantomJS to do it. Headless chrome saved the day.

throwasehasdwi 10y ago

why is Google doing this? They're helping everyone else build a spider that could easily serve as the front end to a new search engine. Is this their way of fighting back against the internet closing off? Give everyone a spider so perfect that walling in your content becomes impractical? I like the idea of Google lighting the match that burns down the walled gardens, but if not this than why? It could be anything, but I feel like there's definitely another motive to Google unleashing something that could bite their core business.

skybrian 10y ago
You're looking for strategy everywhere. People at big companies do a lot of things and not every feature is high-level strategic. I don't know their motivations, but the first use that comes to mind is to make automatically testing your own web app using Chrome easier. Most teams working on web apps could use better testing, whether they work for Google or not.
IshKebab 10y ago
lol no. Nobody building a search engine would use this, and very similar alternatives (e.g. PhantomJS) have been available for years. This is just a useful tool. They probably use it internally for automated testing and research.
bdcravens 10y ago
Spidering is a tiny part of building a search engine, and this piece is a tiny piece of a spider. Headless Chrome helps someone compete with Google as much as Basecamp's work on Rails helps someone compete with Basecamp.
tyingq 10y ago
If I was trying to control scraping and get a view into it, offering the best scraping tool might be good start. I'm pretty sure headless mode exposes something you could fingerprint it with, server side.

mc007ibi 10y ago

fantastic news, is that in any way useful for server-side rendering? i can imagine well to pre render here or there things though

rsync 10y ago

Off-topic, but I'm just going to start bringing this up in every thread like this ... We need a recipe / toolchain for running a browser instance in a chroot jail. With a head. GUI. On the desktop. I want to fire up a "banking browser" (or "danger browser" or "social media browser") that runs on its own IP, in its own chroot, and I can revert to virgin snapshot state anytime I want ... and I don't want to fire up a full-blown VM for eac of these ... What is standing in the way of making this happen ? Who is working on things related to this that I can donate funds to or establish a bounty ?

noja 10y ago
Sounds like Qubes OS https://www.qubes-os.org/ https://www.qubes-os.org/
- rsync 10y ago
  The idea here is that I set up a chroot jail for firefox or chrome and configure it with things like local filesystem for cookies and cache and certs, etc. It would also get its own unique IP, this jail. Then I fire up firefox inside that chroot jail and use it to visit some websites ... and then I can wipe the whole thing out and redeploy again later, starting from scratch. I don't need to trust incognito mode, I don't need to trust wiping cache or tabs talking to each other (or not) and I can worry a lot less about browser level exploits. I can even put in firewall rules so that my "banking" instance can only talk to boa.com and scotttrade.com (or whatever). It's totally workable (and I have done it) with vmware. Make a "bank browsing" VM and revert to pristine snapshot every day. The problem is that this is terribly heavyweight and overkill when I don't need a full blown VM. It's not even really a browser issue - the real issue is, how do you jail a GUI application in X such that that window is in a chroot jail, distinct from the rest of your desktop?
  - napsterbr 10y ago
    Did this a few months ago, somewhat straightforward. A "generic" recipe for any UNIX-based OS would be: 1 - Create a container (Docker, Jails, maybe even chroot) 2 - Assign an ip to the container, NAT to it 3 - Install firefox on that container 4 - Run a SSHD server, enable X11 forwarding 5 - Mount the relevant container's folders to your root fs (eg: map $CONTAINER/.mozzila and $CONTAINER/Downloads to $HOME/jailed-browser/) 6 - Add an ssh config for quick alias 7 - Run `ssh container firefox` and profit Here's a nice example using FreeBSD jails (I remember following this tutorial, everything worked out fine): https://forums.freebsd.org/threads/53362/ https://forums.freebsd.org/threads/53362/ My experience with it, though, wasn't great. X11 forwarding through SSH was quite laggy (even after performing some optimizations on the connection). Good luck if you want to set-up audio/mic support. It's a nice solution for a one-time banking login, not for day-to-day use.

oceanswave 10y ago

If you're using Dotnetcore I've created a protocol generator here https://github.com/baristalabs/chrome-dev-tools https://github.com/baristalabs/chrome-dev-tools With a sample on use here https://github.com/baristalabs/chrome-dev-tools-sample https://github.com/baristalabs/chrome-dev-tools-sample

mxuribe 10y ago

Now, this is handy!

RazrFalcon 10y ago

Did someone try to render an SVG using it? I'm using QtWebKit for now, but it's just a huge pain.

paulirish 10y ago
Yup. Renders SVGs just as Chrome does. Really straightforward.
- RazrFalcon 10y ago
  Can you provide a minimal example?

wslh 10y ago

Several years ago I tested the remote debug protocol and was insanely slow. Is it now faster, like evaluating scripts at the console?

paulirish 10y ago
I don't know we've heard any performance/latency issues before that concern issuing script evaluation over the protocol. Were you sending hundreds of scripts to evaluate? Or payloads far over 1MB? Regardless, if you're having problems.. definitely file a bug and get our attention. new.crbug.com or https://github.com/ChromeDevTools/devtools-protocol https://github.com/ChromeDevTools/devtools-protocol will do the trick. [I'm on the DevTools team]

bostonvaulter2 10y ago

Could this be used to create a screenshot service that'll automatically take screenshots of sites? Or is that not permissible under Chrome's license?

paulirish 10y ago
The Chromium license shouldn't place any restrictions on using the software like that. Go wild.

bgentry 10y ago

Do any of the major CI providers make this trivial to use yet? Travis CI, CircleCI, Buildkite, etc.? Anybody have any positive or negative experiences to report with actually using headless Chrome?

kinlan 10y ago
I lead the DevRel team for Web and Chrome so please take anything I say with a hint of bias.... I've found it pretty easy to use so far - https://paul.kinlan.me/chrome-on-home/ https://paul.kinlan.me/chrome-on-home/ and got it deployed in a number of environments including AppEngine. We are planning on getting this deployed in our [WebFundamentals](https://github.com/google/WebFundamentals/blob/master/.travis.yml https://github.com/google/WebFundamentals/blob/master/.travi...) Travis script to run automated testing on our dev docs site. Some things that I found hard: no documentation on how to get it running in a CI yet - some devs (Justin Ribeiro - https://hub.docker.com/r/justinribeiro/chrome-headless/ https://hub.docker.com/r/justinribeiro/chrome-headless/) got a docker file all set up which helped a lot, the devtools protocol whilst it has docs wasn't quite as simple as I had hoped and I had to guess of how to call the node library.

shmatt 10y ago

This is great progress. I use electron in node.js (via nightmare) because it can go from headless to visible on the fly I wonder if the Chrome team is ever going to support something similar

LoSboccacc 10y ago

why does it need a whole launcher app instead of taking a script to run on the webpage with a control api baffles me

pedalpete 10y ago

Does anybody know how screenshot is triggered? Is it on DOMContentLoaded? or some other event? Can we trigger this somehow post page load?

waingake 10y ago
I needed a delay before creating a pdf maybe this is helpful ? https://github.com/tim-field/urlToPdf/blob/master/url-to-pdf.js#L14 https://github.com/tim-field/urlToPdf/blob/master/url-to-pdf... ( using via node )

rogerwang 10y ago

The upcoming NW.js v0.23.0 beta version will support this usage: https://dl.nwjs.io/live-build/04-29-2017/nw23-3bd4af6-4d7f95f-5e6428a-f2085b9/v0.23.0-beta1/ https://dl.nwjs.io/live-build/04-29-2017/nw23-3bd4af6-4d7f95...

twothamendment 10y ago

Has anyone tried to use headless chrome to detect insecure content on what you hope is a secure page? I've only looked into it a bit, and so far failed. I'd like to know if a visitor to https://example.com/foo.html https://example.com/foo.html in chrome will get a warning about insecure content. I'm looking for a browser because in some cases the insecure content is loaded by javascript from a 3rd party. I don't know a way to do that besides fire it up in a browser and let it go. Is there a way to get the secure/insecure status from headless chrome?

tyingq 10y ago
The remote debugging feature can get to that. See: https://chromedevtools.github.io/debugger-protocol-viewer/tot/Security/ https://chromedevtools.github.io/debugger-protocol-viewer/to... Scroll down to InsecureContentStatus

jeyaganesh89 10y ago

Finally. PhantomJS has a competitor now.

alkoumpa 10y ago

Does it work without X-server?

shif 10y ago
It's headless so most likely yes.
deleted 10y ago
[deleted]

vanni 10y ago

Related recent news: "PhantomJS: Stepping down as maintainer" https://news.ycombinator.com/item?id=14105489 https://news.ycombinator.com/item?id=14105489 Also relevant: "Headless Chrome is Coming Soon" from PhantomJS dev mailing list https://groups.google.com/forum/#!msg/phantomjs-dev/S-mEBwuSgKQ/PQdE1A5FAwAJ https://groups.google.com/forum/#!msg/phantomjs-dev/S-mEBwuS...

kanchon1234 10y ago

danso 10y ago

Was on the frontpage yesterday, with 605 points and 117 comments: https://news.ycombinator.com/item?id=14239194 https://news.ycombinator.com/item?id=14239194

sharpshadow 10y ago

Who would ever think Google will release a headless browser. Stunning.

callumprentice 10y ago

https://bitbucket.org/lindenlab/dullahan/src/default/README.md?at=default&fileviewer=file-view-default https://bitbucket.org/lindenlab/dullahan/src/default/README.... I imagine Headless Chrome will make this obsolete shortly but just in case anyone wants to play with it, here is Dullahan - A headless browser SDK that uses the Chromium Embedded Framework (CEF). It is designed to make it easier to write applications that render modern web content directly to a memory buffer, inject synthesized mouse and keyboard events as well as interact with features like JavaScript or cookies.