13 ms·

Chrome 59 has cross-platform headless support

834 points by vvoyer 10y ago

masterleep 10y ago
Please let this be capable of generating PDFs from HTML from the command line.
- zwerdlds 10y ago
  FWIW, you can do that now, using phantomjs (which is chrome) http://phantomjs.org/screen-capture.html http://phantomjs.org/screen-capture.html
  - velodrome 10y ago
    > FWIW, you can do that now, using phantomjs (which is chrome) http://phantomjs.org/screen-capture.html http://phantomjs.org/screen-capture.html It's based on webkit.
    - theandrewbailey 10y ago
      Once upon a time, but not anymore. https://www.chromium.org/blink https://www.chromium.org/blink
      - deleted 10y ago
        [deleted]
      - rubber_duck 10y ago
        PhantomJS is based on webkit (and an outdated version IIRC + a shitty JS interpreter)
  - chews 10y ago
    specifically it's qtwebkit and an old version of it.
    - eppsilon 10y ago
      They recently released a beta version based on a newer version of QtWebKit: https://bitbucket.org/ariya/phantomjs/downloads/ https://bitbucket.org/ariya/phantomjs/downloads/
  - dasmoth 10y ago
    It's a useful tool (and huge thanks to those who built it -- and SlimerJS for that matter) but whenever I've reached for there's always been some issue to resolve, generally relating to the exact version and/or set of APIs supported. Headless mode (with PDF support -- which it looks like the latest version of the Chromium remote protocol does indeed have) built into a mainstream browsers is nearly guaranteed to be a smoother experience.
  - chickenfries 10y ago
    Also with https://wkhtmltopdf.org/ https://wkhtmltopdf.org/ and http://pandoc.org/ http://pandoc.org/
    - pharrlax 10y ago
      Tried all of these and haven't found any as good as http://www.nightmarejs.org/ http://www.nightmarejs.org/
      - swah 10y ago
        Thats just phantomJS though..
        pharrlax 10y ago
        It switched to Electron under the hood a while back.
    - erichurkman 10y ago
      And, if you need much higher fidelity and control of HTML/CSS -> PDF, there's the fantastic Prince library, http://princexml.com/ http://princexml.com/ (nonfree) (I've been using Prince for over a decade, rendering everything from prescription labels, packing slips, receipts, resumes, books, and more. It's great.)
      - jamespaden 10y ago
        That's also an API service, http://docraptor.com http://docraptor.com, with a different pricing model that uses the Prince library for PDF rendering.
  - zwerdlds 10y ago
    Tough crowd.
- vvoyer 10y ago
  Yes: https://chromedevtools.github.io/debugger-protocol-viewer/tot/Page/#method-printToPDF https://chromedevtools.github.io/debugger-protocol-viewer/to...
  - jzfeng 10y ago
    The print to pdf command currently only support default print settings. Adding support for customized page size, header and footer, dpi, etc. is in progress. Please see bug: https://bugs.chromium.org/p/chromium/issues/detail?id=603559 https://bugs.chromium.org/p/chromium/issues/detail?id=603559 for updates.
    - masterleep 10y ago
      Excellent news, thanks!
  - michael_miller 10y ago
    Any idea if you can configure the page DPI with this API?
    - kenshaw 10y ago
      Yes, the emulation domain has APIs for changing that.
  - erikig 10y ago
    https://wkhtmltopdf.org/ https://wkhtmltopdf.org/ You can download the two tools wkhtmltopdf and wkhtmltoimage which use WebKit to generate pdfs/images.
    - Derbasti 10y ago
      No paged media makes page formatting much too painful.
    - adam77 10y ago
      Sadly wkhtmltopdf's version of Webkit is ancient :( No flexbox, no es6, etc...
- uptown 10y ago
  Not sure of your exact use-case, but mPDF does this well. https://github.com/mpdf/mpdf https://github.com/mpdf/mpdf
- pizza 10y ago
  Is there an easy way to do the opposite? e.g. read PDFs as HTML (intended to be read through a remote shell), or text?
  - jevinskie 10y ago
    You might be able to tweak this pdf.js example to dump out its canvas element after it renders. https://github.com/mozilla/pdf.js/blob/master/examples/learning/helloworld64.html https://github.com/mozilla/pdf.js/blob/master/examples/learn...
  - kccqzy 10y ago
    Extracting text from PDF is not hard, though PDF only contains low-level formatting instructions so the result might not be nice, especially if the original PDF has any non-trivial formatting, like pull quotes, multi-column text, etc. If you don't care about that or the correct "flow" of text, it should be easy enough to just find all the Tj and TJ operators and extract their operands. You might also need to reverse some ligatures though. Producing nice semantic HTML is much harder, though also easy if you don't mind every word in a separate absolutely positioned div. Many PDF reader software already contains empirically tuned routines to infer the text flow and generate text files (because the software needs to handle Select All and Copy), but they often produce bad results. But if you just want to read a PDF on a remote machine over ssh, the easiest solution might be just transferring the file and then opening it locally, or use X forwarding and open the PDF with a graphic reader.
  - desdiv 10y ago
    I'm in the same boat. You might want to check these out: http://coolwanglu.github.io/pdf2htmlEX/ http://coolwanglu.github.io/pdf2htmlEX/ https://github.com/JonathanLink/PDFLayoutTextStripper https://github.com/JonathanLink/PDFLayoutTextStripper http://tabula.technology/ http://tabula.technology/ https://docparser.com https://docparser.com
  - ufmace 10y ago
    Easy, not so much, depending on exactly what you want to get out of it. I did a project with this once https://www.idrsolutions.com/jpdf2html5/ https://www.idrsolutions.com/jpdf2html5/. Last I checked, they only supported it as a Java library that could generate rather nice looking and complete HTML pages from PDF documents. The output was great, but it was kinda pricey and difficult to work with. On the opposite side of the complexity level, I have also used this http://www.pdfsharp.com/PDFsharp/ http://www.pdfsharp.com/PDFsharp/ to extract bits of text from PDFs. It's free, but you only get access to the raw PDF text with formatting codes. It works fine if you just want to grab a short string, but you got your work cut out for you if you want to do anything more sophisticated.
- mixu 10y ago
  If you're willing to wait until Electron releases a Chrome 59 -based build, I'll be updating https://github.com/mixu/electroshot https://github.com/mixu/electroshot which handles screenshots and print-to-PDF along with a bunch of other niceties.
  - Derbasti 10y ago
    Does it support paged media?
  - Derbasti 10y ago
    Does it support paged media?
- deleted 10y ago
  [deleted]
- frik 10y ago
  Especially it should adhere to CSS3 page break properties.
ctphipps 10y ago
Any way of scripting this to automate button clicks etc? I use PhantomJS for this now but found it to be incredibly unstable for complex pages.
- bcherny 10y ago
  You can use ChromeDriver, the same as you would for headed Chrome.
- vvoyer 10y ago
  Yes: https://github.com/cyrus-and/chrome-remote-interface/wiki/Trigger-synthetic-click-events https://github.com/cyrus-and/chrome-remote-interface/wiki/Tr... But ultimately selenium and webdriver.io will do a better job at this
- sandstrom 10y ago
  There is an excellent library called nightmare (based on electron). If you need print-screens you can use xvfb or wait for this issue: https://github.com/segmentio/nightmare/issues/224 https://github.com/segmentio/nightmare/issues/224
  - symtos 10y ago
    it should be noted that nightmare isn't safe for untrusted websites: https://github.com/segmentio/nightmare/issues/1060 https://github.com/segmentio/nightmare/issues/1060
- kenshaw 10y ago
  If you're using Go, you might want to check out https://github.com/knq/chromedp https://github.com/knq/chromedp ... There is also a similar package for NodeJS. Otherwise, you can use Selenium in other languages.
- RandomBookmarks 10y ago
  For complex pages, there is Kantu, which is based on Chromium: https://www.a9t9.com/kantu/web-automation https://www.a9t9.com/kantu/web-automation Not headless, but ideally suited for automating complex websites (date controls, etc).
bluepnume 10y ago
This is fantastic. I'm using a combination of Chrome and PhantomJS for karma testing right now, for https://github.com/paypal/paypal-checkout https://github.com/paypal/paypal-checkout and https://github.com/krakenjs/xcomponent https://github.com/krakenjs/xcomponent. There are hundreds of tests opening up hundreds of iframes and popup windows, and sending a lot of cross-window messages, and that ends up being really memory hungry. Chrome deals pretty well with garbage collection, so long as I'm careful to de-reference closed windows properly¹, and only uses a maximum of 150mbs. PhantomJS eats up almost 6GB of memory before it's done, which makes it almost unusable on machines with less memory or CI boxes. Travis is a no-go. I'm hoping running Chrome in headless mode should give a nice speedup for our tests. ----- ¹ Turns out even a closed popup window or iframe keeps a huge amount of memory hanging around. Who knew.
- paulirish 10y ago
  We (Chrome) have reached out to PhantomJS to inquire if they're interested in collaborating: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU06ZOZcFgAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU... The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.
  - flanbiscuit 10y ago
    Thank you for being willing to work with them and not just rolling your own version of Phantomjs. I hope they respond positively to your inquiry
  - shouldbworking 10y ago
    It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken
vvoyer 10y ago
Also checkout https://github.com/cyrus-and/chrome-remote-interface https://github.com/cyrus-and/chrome-remote-interface for an easy way to fully control those headless instances
- kenshaw 10y ago
  If you're using Go, https://github.com/knq/chromedp https://github.com/knq/chromedp It's also worth noting that the 57+ series has a nice embedded viewer you can use to view the actual viewport via devtools.
  - nojvek 10y ago
    If you want great code completion for chrome's large devtools api with lots of different kinds of object then https://www.npmjs.com/package/chrome-remote-debug-protocol https://www.npmjs.com/package/chrome-remote-debug-protocol It dynamically generates Typescript definitions for intelligence and type checking from their protocol.json files. Vscode chrome debugger uses a fork of it. I'm the author. /shamelessplug
  - chrissnell 10y ago
    WOW. That's fantastic. As I noted in a comment above, I wrote a tool called crabby that uses Selenium and Chrome headless to do automated page testing and to report results back to metrics engines like Graphite, Datadog, Prometheus, and Riemann. The biggest problem I have is the unreliability of chromedriver and the extreme resource consumption of Chrome + Selenium. It's really too much for your average public cloud instance if you want to test any more than, say, one or two pages a minute. Do you know if chromedp can access any of the timing measurements?
    - kenshaw 10y ago
      Yes, you can access everything via the underlying APIs. chromedp is a relatively new project (only about 4 months old), so there isn't much yet in the way of high level timing / profiling, but we hope to add that to the code base when we have some bandwidth to do so.
yarp 10y ago
Any chance for webgl here? Would be nice for automatic screenshots and webgl tests.
- jevinskie 10y ago
  It is being discussed. https://docs.google.com/document/d/1VTcYz4q_x0f1O5IVrvRX4u1DVd_K34IVUl1VULLTCWw/edit?disco=AAAAAxbGHos https://docs.google.com/document/d/1VTcYz4q_x0f1O5IVrvRX4u1D...
- colordrops 10y ago
  WebGL is supposedly a first class citizen of the browser and is used in a ton of pages, but it gets left out or deferred in many tooling packages. There are many tools that are nearly useless to us because they don't support WebGL. It's disappointing.
arrowgunz 10y ago
What does this mean for Electron and other apps that depend on Electron?
- sandstrom 10y ago
  There is an issue on headless here: https://github.com/electron/electron/issues/228 https://github.com/electron/electron/issues/228
- JepZ 10y ago
  I am looking forward to cli tools powered by electron. Imagine the beauty of a grep writting in js, executed by a chrome binary. :D
  - Spivak 10y ago
    I mean honestly Electron running React-CLI with React-Arg-Router is the only way to write a cross platform CLI tools these days.
  - ryeguy 10y ago
    Imagine the terrible performance of grep written in js, executed by a chrome binary.
skibz 10y ago
The feature I was most interested in when they announced this last year was virtual time. The Developer Resources link has it listed (https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md https://chromium.googlesource.com/chromium/src/+/lkgr/headle...) but it's a broken link, unfortunately. Mostly, I'd like to know how the control of the virtual time system would be exposed. Would it be through the C++ API, or could it be made available through the debugging protocol?
iAm25626 10y ago
Nice!! Would creating WebRTC data channel be possible? server side SCTP to client(p2p over SCTP/data channel) would be cool.
- stcredzero 10y ago
  It's already possible. I'm using webrtc and node-electron to connect a golang server for my MMO project. I have a farm of 4 nodejs processes running under tmux acting as proxies for unreliable communications. https://www.emergencevector.com https://www.emergencevector.com
  - unmole 10y ago
    Which WebRTC library are you using on the golang server? Last time I looked, I couldn't find anything stable.
    - stcredzero 10y ago
      I am using electron-webrtc. You need xvfb installed. I was able to get this running under node, but only running the processes under tmux. You will also need to install some random shared libraries Chromium needs, but which node-electron doesn't install, but these are obvious from error messages. Then I'm using simple-peer on top of that. There's also a library for UDP communications from the node process to the golang process.
callumprentice 10y ago
https://bitbucket.org/lindenlab/dullahan https://bitbucket.org/lindenlab/dullahan I've been working on a fully open source Windows/macOS library (via Chromium Embedded Framework) that allows you to render pages to memory (and then of course to bitmaps, textures etc.) as well as inject synthesized mouse/keyboard/JavaScript events. It currently uses (what amounts to) Chrome 57. Looks like this might make my project obsolete.
- keville 10y ago
  Thank you anyway for your efforts; I hope you interpret this as a validation of your having correctly identified a sorely-needed tool.
  - callumprentice 10y ago
    Thanks keville - I've poured my heart into this on nights and weekends after I'm done with my day job for a long time now but what you say is so true - hopefully we'll get access to a more robust solution compared to my modest hacks.
amingilani 10y ago
Does this mean I no longer need to use phantomjs for my tests?
vmasto 10y ago
I've been trying to test audio and video with headless browsers (namely PhantomJS) but have experienced extreme difficulty, I wonder if headless Chrome is able to support/supports already HTMLAudioElement or HTMLVideoElement or any media interface that would make, for example, testing YouTube or SoundCloud embeds easier.
smackfu 10y ago
Have people found many issues that come up in Chrome but aren't found in PhantomJS? We used to use a headless browser but switched to PhantomJS and haven't had any real issues. (We should probably run under the real IE but jut haven't been bothered.)
- kageneko 10y ago
  I've run into some problems with some of the more ... creative jasmine tests I've done. Mostly, it's been around object mocks. I've found places where I can do things with Object.defineProperty() in Chrome that throw exceptions in PhantomJS :(
- adzicg 10y ago
  Phantom does not support more recent JS syntax/tweaks. We have an app that is aimed at more recent browsers only so can use latest ES6 features, and had to move from Phantom to in-browser tests (the alternative would be to use babel for transpilation, but then we wouldn't be testing the code that is actually released to users)
MichaelApproved 10y ago
A lot of phantom js talk here makes me want to recommend http://ghostinspector.com http://ghostinspector.com It's a phantom js (and other headless browser) web service. Using the site, you can quickly create different tests, scheduled tests, chained tests, keep screen shots, create videos of multi step tests, and have historical information of it all. Can't say enough good things about the site. Edit: also there's a great chrome extension that will record your mouse clicks and keyboard commands to make creating a test that much simpler.
- rgrieselhuber 10y ago
  I'm the co-founder of a startup that provides an enterprise solution in this space (https://functionize.com https://functionize.com). I agree with your sentiment regarding these types of tools. It has been a long time coming but this release and the tools available now are things I wish I had years ago when I was building my first company.
- guiambros 10y ago
  +1 to GhostInspector; I used them at a previous company a few years ago, and it was very useful. They were just starting, but service was rather reliable, and their tech support was excellent (maybe because we were early customers). We used to run a bunch of automated tests for monitoring and compliance, archiving hourly screenshots over different builds for later comparison.
Bahamut 10y ago
Oh my goodness I have been waiting for this day for a while - we ran into PhantomJS problems with keyboard/mouse eventing and the HTMLVideoElement for testing, this sounds like it should be the cure for our woes of having to hack around PhantomJS's deficiencies.
MR4D 10y ago
Given the rough comments on the Electron story earlier, this should be welcome by all.
mstade 10y ago
Fantastic news, not a minute too soon! Can't wait to get rid of PhantomJS. Now if only this was a standard feature of all browsers...
armitron 10y ago
Doesn't seem to work on OSX. Connecting to debug port from a different chrome brings up an empty page. Running Version 59.0.3069.0 (Official Build) canary (64-bit)
- llimllib 10y ago
  same for me. Wonder if we're doing something wrong?
  - notatoad 10y ago
    Has it actually landed in the builds yet, or just been marked as targeting 59 in the platform docs?
    - llimllib 10y ago
      I tried to figure that out but was unable to
    - dvallet 10y ago
      We are targeting 59. A fix for Mac OS X is on its way: https://codereview.chromium.org/2811633002 https://codereview.chromium.org/2811633002
      - llimllib 10y ago
        Thanks! It looks like that has been closed... is there any way to know what release # the fix is in, or when it gets released?
        dvallet 10y ago
        It'll be released in 59.0.3071.3
- vvoyer 10y ago
  Works for me when using https://github.com/cyrus-and/chrome-remote-interface https://github.com/cyrus-and/chrome-remote-interface
_pdp_ 10y ago
This is great if the headless mode supports the web extension API because it means that we can run our security tools almost as command line tools.
fake-name 10y ago
Shamelessly bumping my project to produce a nice python API for the Chromium/Chrome-remote-debugger-protocol: https://github.com/fake-name/ChromeController https://github.com/fake-name/ChromeController I'm trying to replace PhantomJS in my infrastructure with chromium. Not having to build my own chromium will be a very nice thing.
ericb 10y ago
Would this run as a chrome driver for Selenium? What is needed to make this work with Selenium?
- onion2k 10y ago
  It should be possible, but as a browser rather than a driver so you'd still need Chromedriver to glue Chrome and Selenium together. In Karma with karma-chrome-launcher[1] you can pass options to the browser using the flags option, eg; customLaunchers: { chrome_headless: { base: 'Chrome', flags: ['--headless'] } } I've not tested that yet but I can't really see a reason why it wouldn't work. [1] https://github.com/karma-runner/karma-chrome-launcher https://github.com/karma-runner/karma-chrome-launcher
- kuahyeow 10y ago
  Yes, it does. Tried it out with our Selenium based test suite. Just have to pass in the `--headless` switch to Chrome
nreece 10y ago
I've been testing Chrome headless extensively for the past few months, and while it's a good step, but it's not stable for high-volume or even diverse set of webpages. Memory usage is pretty high, lot of heavy webpages result in crashes/hangs, there are many inconsistencies between features available in full version and headless, their debugging protocol has different APIs that work on headless/non-headless in Linux or Windows, and so on. Of the bugs I've submitted, some have been fixed in the upcoming M59, so other critical ones may take longer due to their backlog. I suppose for now (maybe until M61-62), Chrome full with xvfb or even PhantomJS are better options. When you realize that Chrome is about the same size (by LoC) as the Linux kernel [1], you can't help but wish for a leaner & faster headless browser. There seems to be some work going on building Firefox pure headless as well. Great overall, as long as all the browsers try to follow the RemoteDebug initiative [2]. [1] https://josephg.com/blog/electron-is-flash-for-the-desktop/ https://josephg.com/blog/electron-is-flash-for-the-desktop/ [2] http://remotedebug.org http://remotedebug.org
- fastest963 10y ago
  Unfortunately, I have to agree, though they have been very helpful and vigilant on the bugs that have been filed. Most of mine have been fixed in a few days.
  - nreece 10y ago
    Hey James, I did come across some of your bug submissions, and our joint discussions on the headless-dev Google Group.
- chrissnell 10y ago
  Yes, this exactly. I wrote Crabby [1] a few months back to schedule automated page testing using Chrome and webdriver but doing anything automated with Chrome is really atrocious. You can't expect to load more than one page every 10-15 seconds on the average 8GB instance and it occasionally crashes or otherwise stops working completely. I ended up writing a simple check using Go's net/http library to do basic performance profiling but it doesn't measure DOM loading like the Chrome checks do. Such a bummer What I really want is an easy, cross-platform way to collect the network timings for each object like you get in Chrome's dev tools network waterfall graphs. [1] https://github.com/chrissnell/crabby https://github.com/chrissnell/crabby
livoras 10y ago
PhantomJS has plenty unsolved issues(up to 1.7k+), a replacement instead of combination might be a better choice.
est 10y ago
Is it possible to install Chromium on a server without X environment? Last time I checked it requires a shit ton of dependencies.
0xFFC 10y ago
Can somebody explain to me what is this good for? Thank you.
- vacri 10y ago
  Headless browsers are frequently used in buildserver testing of web apps.
- jamesgeck0 10y ago
  I'll be using it to generate printable PDF reports. It's a huge pain getting highcharts graphs to look identical in every browser's print view.
vvoyer 10y ago
Almost one year old but there was a talk on headless chrome at the Blink conference (BlinkOn 6): Video: https://youtu.be/GivjumRiZ8c https://youtu.be/GivjumRiZ8c Slides: https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcxzMQNEE7PcuQrGu83No3l0lw https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcx... More links: Headless Chrome architecture: https://docs.google.com/document/d/11zIkKkLBocofGgoTeeyibB2TZ_k7nR78v7kNelCatUE/ https://docs.google.com/document/d/11zIkKkLBocofGgoTeeyibB2T... Mailing list: https://groups.google.com/a/chromium.org/forum/#!forum/headless-dev https://groups.google.com/a/chromium.org/forum/#!forum/headl... All of those links are on https://chromium.googlesource.com/chromium/src.git/+/master/headless/README.md https://chromium.googlesource.com/chromium/src.git/+/master/...
deleted 10y ago
[deleted]
softwarelimits 10y ago
is this is chromium, too?
laurencei 10y ago
Can anyone confirm - would this work with a Flash/SWF application? i.e. could I use the headless mode to interact with the Flash Application to run some commands and retrieve the output? I tried googling around but didnt find much to say either way...
swah 10y ago
Related: I want to take screenshots of a few news websites for a little fake news project of mine, and most approaches return something completely different than what I'm seeing when I open Chrome. Limited height would be better/ok (something like the first 3000 pixels). Low volume / can be slow (30 seconds would be ok). Those news websites many times have infinite scrolling. I've tried: - phantomJS (rendering sucked, tried every technique I could find to wait for JS to load) - wkhtmltopdf (almost ok, generates a huge 30M image with all the height, no antialiasing it seems) - https://github.com/gen2brain/url2img https://github.com/gen2brain/url2img (this was the best so far, uses Qt bindings but not the latest version) - actually run a headless browser in DigitalOcean with xvfb-run and take a screenshot: I failed at this What I didn't tried was Selenium, because it seemed even harder. How would you guys do it?
- deleted 10y ago
  [deleted]
- oceanswave 10y ago
  Create a chrome extension for yourself that automates the process
- stangls 10y ago
  I use xvfb for selenium testing. It is really easy to set up and can take screenshots or videos while automatically browsing websites.
  - swah 10y ago
    Thanks - I tried Selenium (on the desktop) with geckodriver now and it rendered well. The only thing is that long screenshots didn't work but there is probably a workaround for that.
retube 10y ago
This page doesn't load for me (IE behind corp firewall). How does one drive/automate a headless browser? What kind of API is there?
- tonyblundell 10y ago
  Docs and a Youtube presentation... https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md https://chromium.googlesource.com/chromium/src/+/lkgr/headle... https://www.youtube.com/watch?v=n6biclFh0i0&feature=youtu.be https://www.youtube.com/watch?v=n6biclFh0i0&feature=youtu.be
wslh 10y ago
How fast is the debugging mode? I tried the first debugging protocol when Chrome added it and it was very difficult to use. I assume this time is different?
unixhero 10y ago
What is the use case for headless Chrome?
- ntaylor 10y ago
  > Headless mode allows running Chromium in a headless/server environment. Expected use cases include loading web pages, extracting metadata (e.g., the DOM) and generating bitmaps from page contents -- using all the modern web platform features provided by Chromium and Blink. Practically speaking, software developers will use headless Chrome to automate testing of product functionality. Today, developers use systems like Selenium or PhantomJS to accomplish this feat, but it's a painful process to maintain these headless browser execution engines. Adding headless support into Chrome means that developers can count on the presentation of their application on a given version of the Blink engine run within Chrome.
hbakhtiyor 10y ago
i use when they announced headless mode on linux, and built generating thumbnails from captured screenshots of websites and uncovering the technologies used on websites and the api is available for free, https://github.com/letsvalidate/api https://github.com/letsvalidate/api
brendandahl 10y ago
For those interested, Firefox is also going to support a headless mode. The current nightly supports headless SlimerJS on Linux and more platforms will come soon. https://bugzilla.mozilla.org/show_bug.cgi?id=1338004 https://bugzilla.mozilla.org/show_bug.cgi?id=1338004
hackcasual 10y ago
Looks like the launch bug is private? https://bugs.chromium.org/p/chromium/issues/detail?id=705916 https://bugs.chromium.org/p/chromium/issues/detail?id=705916
tianlins 10y ago
How fast is headless vs. normal? According to https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction https://developers.google.com/web/fundamentals/performance/c... the chrome browser spends a decent amount of time on other steps such as parsing HTML. I wonder how much time could be saved by not rendering pages into pixels.
zigomir 10y ago
Not sure if this can enable SSR (server side rendering) for any client side lib?
du_bing 10y ago
It seems that Chromium 59 still can not be installed on Raspberry Pi, or anyone has done it? It will be great to use this headless Chromium on Raspberry Pi to execute some routine web browser jobs. Does it support the extensions installed on Chromium? Curious.
stheakanath 10y ago
Is this confirmed to work with Flash? I know selenium did not support Flash so it caused some dev issues.
dkastner 10y ago
I put together an example of how to run chromium with --headless driven by cucumber/capybara: https://dkastner.github.io/2017/04/21/headless-chrome.html https://dkastner.github.io/2017/04/21/headless-chrome.html

834 points by vvoyer 10y ago

masterleep 10y ago

Please let this be capable of generating PDFs from HTML from the command line.

zwerdlds 10y ago
FWIW, you can do that now, using phantomjs (which is chrome) http://phantomjs.org/screen-capture.html http://phantomjs.org/screen-capture.html
- velodrome 10y ago
  > FWIW, you can do that now, using phantomjs (which is chrome) http://phantomjs.org/screen-capture.html http://phantomjs.org/screen-capture.html It's based on webkit.
  - theandrewbailey 10y ago
    Once upon a time, but not anymore. https://www.chromium.org/blink https://www.chromium.org/blink
    - deleted 10y ago
      [deleted]
    - rubber_duck 10y ago
      PhantomJS is based on webkit (and an outdated version IIRC + a shitty JS interpreter)
- chews 10y ago
  specifically it's qtwebkit and an old version of it.
  - eppsilon 10y ago
    They recently released a beta version based on a newer version of QtWebKit: https://bitbucket.org/ariya/phantomjs/downloads/ https://bitbucket.org/ariya/phantomjs/downloads/
- dasmoth 10y ago
  It's a useful tool (and huge thanks to those who built it -- and SlimerJS for that matter) but whenever I've reached for there's always been some issue to resolve, generally relating to the exact version and/or set of APIs supported. Headless mode (with PDF support -- which it looks like the latest version of the Chromium remote protocol does indeed have) built into a mainstream browsers is nearly guaranteed to be a smoother experience.
- chickenfries 10y ago
  Also with https://wkhtmltopdf.org/ https://wkhtmltopdf.org/ and http://pandoc.org/ http://pandoc.org/
  - pharrlax 10y ago
    Tried all of these and haven't found any as good as http://www.nightmarejs.org/ http://www.nightmarejs.org/
    - swah 10y ago
      Thats just phantomJS though..
      - pharrlax 10y ago
        It switched to Electron under the hood a while back.
  - erichurkman 10y ago
    And, if you need much higher fidelity and control of HTML/CSS -> PDF, there's the fantastic Prince library, http://princexml.com/ http://princexml.com/ (nonfree) (I've been using Prince for over a decade, rendering everything from prescription labels, packing slips, receipts, resumes, books, and more. It's great.)
    - jamespaden 10y ago
      That's also an API service, http://docraptor.com http://docraptor.com, with a different pricing model that uses the Prince library for PDF rendering.
- zwerdlds 10y ago
  Tough crowd.
vvoyer 10y ago
Yes: https://chromedevtools.github.io/debugger-protocol-viewer/tot/Page/#method-printToPDF https://chromedevtools.github.io/debugger-protocol-viewer/to...
- jzfeng 10y ago
  The print to pdf command currently only support default print settings. Adding support for customized page size, header and footer, dpi, etc. is in progress. Please see bug: https://bugs.chromium.org/p/chromium/issues/detail?id=603559 https://bugs.chromium.org/p/chromium/issues/detail?id=603559 for updates.
  - masterleep 10y ago
    Excellent news, thanks!
- michael_miller 10y ago
  Any idea if you can configure the page DPI with this API?
  - kenshaw 10y ago
    Yes, the emulation domain has APIs for changing that.
- erikig 10y ago
  https://wkhtmltopdf.org/ https://wkhtmltopdf.org/ You can download the two tools wkhtmltopdf and wkhtmltoimage which use WebKit to generate pdfs/images.
  - Derbasti 10y ago
    No paged media makes page formatting much too painful.
  - adam77 10y ago
    Sadly wkhtmltopdf's version of Webkit is ancient :( No flexbox, no es6, etc...
uptown 10y ago
Not sure of your exact use-case, but mPDF does this well. https://github.com/mpdf/mpdf https://github.com/mpdf/mpdf
pizza 10y ago
Is there an easy way to do the opposite? e.g. read PDFs as HTML (intended to be read through a remote shell), or text?
- jevinskie 10y ago
  You might be able to tweak this pdf.js example to dump out its canvas element after it renders. https://github.com/mozilla/pdf.js/blob/master/examples/learning/helloworld64.html https://github.com/mozilla/pdf.js/blob/master/examples/learn...
- kccqzy 10y ago
  Extracting text from PDF is not hard, though PDF only contains low-level formatting instructions so the result might not be nice, especially if the original PDF has any non-trivial formatting, like pull quotes, multi-column text, etc. If you don't care about that or the correct "flow" of text, it should be easy enough to just find all the Tj and TJ operators and extract their operands. You might also need to reverse some ligatures though. Producing nice semantic HTML is much harder, though also easy if you don't mind every word in a separate absolutely positioned div. Many PDF reader software already contains empirically tuned routines to infer the text flow and generate text files (because the software needs to handle Select All and Copy), but they often produce bad results. But if you just want to read a PDF on a remote machine over ssh, the easiest solution might be just transferring the file and then opening it locally, or use X forwarding and open the PDF with a graphic reader.
- desdiv 10y ago
  I'm in the same boat. You might want to check these out: http://coolwanglu.github.io/pdf2htmlEX/ http://coolwanglu.github.io/pdf2htmlEX/ https://github.com/JonathanLink/PDFLayoutTextStripper https://github.com/JonathanLink/PDFLayoutTextStripper http://tabula.technology/ http://tabula.technology/ https://docparser.com https://docparser.com
- ufmace 10y ago
  Easy, not so much, depending on exactly what you want to get out of it. I did a project with this once https://www.idrsolutions.com/jpdf2html5/ https://www.idrsolutions.com/jpdf2html5/. Last I checked, they only supported it as a Java library that could generate rather nice looking and complete HTML pages from PDF documents. The output was great, but it was kinda pricey and difficult to work with. On the opposite side of the complexity level, I have also used this http://www.pdfsharp.com/PDFsharp/ http://www.pdfsharp.com/PDFsharp/ to extract bits of text from PDFs. It's free, but you only get access to the raw PDF text with formatting codes. It works fine if you just want to grab a short string, but you got your work cut out for you if you want to do anything more sophisticated.
mixu 10y ago
If you're willing to wait until Electron releases a Chrome 59 -based build, I'll be updating https://github.com/mixu/electroshot https://github.com/mixu/electroshot which handles screenshots and print-to-PDF along with a bunch of other niceties.
- Derbasti 10y ago
  Does it support paged media?
- Derbasti 10y ago
  Does it support paged media?
deleted 10y ago
[deleted]
frik 10y ago
Especially it should adhere to CSS3 page break properties.

ctphipps 10y ago

Any way of scripting this to automate button clicks etc? I use PhantomJS for this now but found it to be incredibly unstable for complex pages.

bcherny 10y ago
You can use ChromeDriver, the same as you would for headed Chrome.
vvoyer 10y ago
Yes: https://github.com/cyrus-and/chrome-remote-interface/wiki/Trigger-synthetic-click-events https://github.com/cyrus-and/chrome-remote-interface/wiki/Tr... But ultimately selenium and webdriver.io will do a better job at this
sandstrom 10y ago
There is an excellent library called nightmare (based on electron). If you need print-screens you can use xvfb or wait for this issue: https://github.com/segmentio/nightmare/issues/224 https://github.com/segmentio/nightmare/issues/224
- symtos 10y ago
  it should be noted that nightmare isn't safe for untrusted websites: https://github.com/segmentio/nightmare/issues/1060 https://github.com/segmentio/nightmare/issues/1060
kenshaw 10y ago
If you're using Go, you might want to check out https://github.com/knq/chromedp https://github.com/knq/chromedp ... There is also a similar package for NodeJS. Otherwise, you can use Selenium in other languages.
RandomBookmarks 10y ago
For complex pages, there is Kantu, which is based on Chromium: https://www.a9t9.com/kantu/web-automation https://www.a9t9.com/kantu/web-automation Not headless, but ideally suited for automating complex websites (date controls, etc).

bluepnume 10y ago

This is fantastic. I'm using a combination of Chrome and PhantomJS for karma testing right now, for https://github.com/paypal/paypal-checkout https://github.com/paypal/paypal-checkout and https://github.com/krakenjs/xcomponent https://github.com/krakenjs/xcomponent. There are hundreds of tests opening up hundreds of iframes and popup windows, and sending a lot of cross-window messages, and that ends up being really memory hungry. Chrome deals pretty well with garbage collection, so long as I'm careful to de-reference closed windows properly¹, and only uses a maximum of 150mbs. PhantomJS eats up almost 6GB of memory before it's done, which makes it almost unusable on machines with less memory or CI boxes. Travis is a no-go. I'm hoping running Chrome in headless mode should give a nice speedup for our tests. ----- ¹ Turns out even a closed popup window or iframe keeps a huge amount of memory hanging around. Who knew.

paulirish 10y ago
We (Chrome) have reached out to PhantomJS to inquire if they're interested in collaborating: https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU06ZOZcFgAJ https://groups.google.com/d/msg/phantomjs-dev/S-mEBwuSgKQ/tU... The DevTools Protocol is the primary API for headless Chrome, but we are excited for higher-level abstractions like PhantomJS & NightmareJS's API to manipulate the browser as well. Plenty of details to work out, but hopefully sometime this year you'll get a drop-in solution for some of your testing to upgrade from Phantom's older QTWebKit to the latest Chromium.
- flanbiscuit 10y ago
  Thank you for being willing to work with them and not just rolling your own version of Phantomjs. I hope they respond positively to your inquiry
- shouldbworking 10y ago
  It was my understanding that Google's crawler literally is Chrome. Does Google have any plans to open source those parts of the browser to make integration easier? Maybe I was mistaken

vvoyer 10y ago

Also checkout https://github.com/cyrus-and/chrome-remote-interface https://github.com/cyrus-and/chrome-remote-interface for an easy way to fully control those headless instances

kenshaw 10y ago
If you're using Go, https://github.com/knq/chromedp https://github.com/knq/chromedp It's also worth noting that the 57+ series has a nice embedded viewer you can use to view the actual viewport via devtools.
- nojvek 10y ago
  If you want great code completion for chrome's large devtools api with lots of different kinds of object then https://www.npmjs.com/package/chrome-remote-debug-protocol https://www.npmjs.com/package/chrome-remote-debug-protocol It dynamically generates Typescript definitions for intelligence and type checking from their protocol.json files. Vscode chrome debugger uses a fork of it. I'm the author. /shamelessplug
- chrissnell 10y ago
  WOW. That's fantastic. As I noted in a comment above, I wrote a tool called crabby that uses Selenium and Chrome headless to do automated page testing and to report results back to metrics engines like Graphite, Datadog, Prometheus, and Riemann. The biggest problem I have is the unreliability of chromedriver and the extreme resource consumption of Chrome + Selenium. It's really too much for your average public cloud instance if you want to test any more than, say, one or two pages a minute. Do you know if chromedp can access any of the timing measurements?
  - kenshaw 10y ago
    Yes, you can access everything via the underlying APIs. chromedp is a relatively new project (only about 4 months old), so there isn't much yet in the way of high level timing / profiling, but we hope to add that to the code base when we have some bandwidth to do so.

yarp 10y ago

Any chance for webgl here? Would be nice for automatic screenshots and webgl tests.

jevinskie 10y ago
It is being discussed. https://docs.google.com/document/d/1VTcYz4q_x0f1O5IVrvRX4u1DVd_K34IVUl1VULLTCWw/edit?disco=AAAAAxbGHos https://docs.google.com/document/d/1VTcYz4q_x0f1O5IVrvRX4u1D...
colordrops 10y ago
WebGL is supposedly a first class citizen of the browser and is used in a ton of pages, but it gets left out or deferred in many tooling packages. There are many tools that are nearly useless to us because they don't support WebGL. It's disappointing.

arrowgunz 10y ago

What does this mean for Electron and other apps that depend on Electron?

sandstrom 10y ago
There is an issue on headless here: https://github.com/electron/electron/issues/228 https://github.com/electron/electron/issues/228
JepZ 10y ago
I am looking forward to cli tools powered by electron. Imagine the beauty of a grep writting in js, executed by a chrome binary. :D
- Spivak 10y ago
  I mean honestly Electron running React-CLI with React-Arg-Router is the only way to write a cross platform CLI tools these days.
- ryeguy 10y ago
  Imagine the terrible performance of grep written in js, executed by a chrome binary.

skibz 10y ago

The feature I was most interested in when they announced this last year was virtual time. The Developer Resources link has it listed (https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md https://chromium.googlesource.com/chromium/src/+/lkgr/headle...) but it's a broken link, unfortunately. Mostly, I'd like to know how the control of the virtual time system would be exposed. Would it be through the C++ API, or could it be made available through the debugging protocol?

iAm25626 10y ago

Nice!! Would creating WebRTC data channel be possible? server side SCTP to client(p2p over SCTP/data channel) would be cool.

stcredzero 10y ago
It's already possible. I'm using webrtc and node-electron to connect a golang server for my MMO project. I have a farm of 4 nodejs processes running under tmux acting as proxies for unreliable communications. https://www.emergencevector.com https://www.emergencevector.com
- unmole 10y ago
  Which WebRTC library are you using on the golang server? Last time I looked, I couldn't find anything stable.
  - stcredzero 10y ago
    I am using electron-webrtc. You need xvfb installed. I was able to get this running under node, but only running the processes under tmux. You will also need to install some random shared libraries Chromium needs, but which node-electron doesn't install, but these are obvious from error messages. Then I'm using simple-peer on top of that. There's also a library for UDP communications from the node process to the golang process.

callumprentice 10y ago

https://bitbucket.org/lindenlab/dullahan https://bitbucket.org/lindenlab/dullahan I've been working on a fully open source Windows/macOS library (via Chromium Embedded Framework) that allows you to render pages to memory (and then of course to bitmaps, textures etc.) as well as inject synthesized mouse/keyboard/JavaScript events. It currently uses (what amounts to) Chrome 57. Looks like this might make my project obsolete.

keville 10y ago
Thank you anyway for your efforts; I hope you interpret this as a validation of your having correctly identified a sorely-needed tool.
- callumprentice 10y ago
  Thanks keville - I've poured my heart into this on nights and weekends after I'm done with my day job for a long time now but what you say is so true - hopefully we'll get access to a more robust solution compared to my modest hacks.

amingilani 10y ago

Does this mean I no longer need to use phantomjs for my tests?

vmasto 10y ago

I've been trying to test audio and video with headless browsers (namely PhantomJS) but have experienced extreme difficulty, I wonder if headless Chrome is able to support/supports already HTMLAudioElement or HTMLVideoElement or any media interface that would make, for example, testing YouTube or SoundCloud embeds easier.

smackfu 10y ago

Have people found many issues that come up in Chrome but aren't found in PhantomJS? We used to use a headless browser but switched to PhantomJS and haven't had any real issues. (We should probably run under the real IE but jut haven't been bothered.)

kageneko 10y ago
I've run into some problems with some of the more ... creative jasmine tests I've done. Mostly, it's been around object mocks. I've found places where I can do things with Object.defineProperty() in Chrome that throw exceptions in PhantomJS :(
adzicg 10y ago
Phantom does not support more recent JS syntax/tweaks. We have an app that is aimed at more recent browsers only so can use latest ES6 features, and had to move from Phantom to in-browser tests (the alternative would be to use babel for transpilation, but then we wouldn't be testing the code that is actually released to users)

MichaelApproved 10y ago

A lot of phantom js talk here makes me want to recommend http://ghostinspector.com http://ghostinspector.com It's a phantom js (and other headless browser) web service. Using the site, you can quickly create different tests, scheduled tests, chained tests, keep screen shots, create videos of multi step tests, and have historical information of it all. Can't say enough good things about the site. Edit: also there's a great chrome extension that will record your mouse clicks and keyboard commands to make creating a test that much simpler.

rgrieselhuber 10y ago
I'm the co-founder of a startup that provides an enterprise solution in this space (https://functionize.com https://functionize.com). I agree with your sentiment regarding these types of tools. It has been a long time coming but this release and the tools available now are things I wish I had years ago when I was building my first company.
guiambros 10y ago
+1 to GhostInspector; I used them at a previous company a few years ago, and it was very useful. They were just starting, but service was rather reliable, and their tech support was excellent (maybe because we were early customers). We used to run a bunch of automated tests for monitoring and compliance, archiving hourly screenshots over different builds for later comparison.

Bahamut 10y ago

Oh my goodness I have been waiting for this day for a while - we ran into PhantomJS problems with keyboard/mouse eventing and the HTMLVideoElement for testing, this sounds like it should be the cure for our woes of having to hack around PhantomJS's deficiencies.

MR4D 10y ago

Given the rough comments on the Electron story earlier, this should be welcome by all.

mstade 10y ago

Fantastic news, not a minute too soon! Can't wait to get rid of PhantomJS. Now if only this was a standard feature of all browsers...

armitron 10y ago

Doesn't seem to work on OSX. Connecting to debug port from a different chrome brings up an empty page. Running Version 59.0.3069.0 (Official Build) canary (64-bit)

llimllib 10y ago
same for me. Wonder if we're doing something wrong?
- notatoad 10y ago
  Has it actually landed in the builds yet, or just been marked as targeting 59 in the platform docs?
  - llimllib 10y ago
    I tried to figure that out but was unable to
  - dvallet 10y ago
    We are targeting 59. A fix for Mac OS X is on its way: https://codereview.chromium.org/2811633002 https://codereview.chromium.org/2811633002
    - llimllib 10y ago
      Thanks! It looks like that has been closed... is there any way to know what release # the fix is in, or when it gets released?
      - dvallet 10y ago
        It'll be released in 59.0.3071.3
vvoyer 10y ago
Works for me when using https://github.com/cyrus-and/chrome-remote-interface https://github.com/cyrus-and/chrome-remote-interface

_pdp_ 10y ago

This is great if the headless mode supports the web extension API because it means that we can run our security tools almost as command line tools.

fake-name 10y ago

Shamelessly bumping my project to produce a nice python API for the Chromium/Chrome-remote-debugger-protocol: https://github.com/fake-name/ChromeController https://github.com/fake-name/ChromeController I'm trying to replace PhantomJS in my infrastructure with chromium. Not having to build my own chromium will be a very nice thing.

ericb 10y ago

Would this run as a chrome driver for Selenium? What is needed to make this work with Selenium?

onion2k 10y ago
It should be possible, but as a browser rather than a driver so you'd still need Chromedriver to glue Chrome and Selenium together. In Karma with karma-chrome-launcher[1] you can pass options to the browser using the flags option, eg; customLaunchers: { chrome_headless: { base: 'Chrome', flags: ['--headless'] } } I've not tested that yet but I can't really see a reason why it wouldn't work. [1] https://github.com/karma-runner/karma-chrome-launcher https://github.com/karma-runner/karma-chrome-launcher
kuahyeow 10y ago
Yes, it does. Tried it out with our Selenium based test suite. Just have to pass in the `--headless` switch to Chrome

nreece 10y ago

I've been testing Chrome headless extensively for the past few months, and while it's a good step, but it's not stable for high-volume or even diverse set of webpages. Memory usage is pretty high, lot of heavy webpages result in crashes/hangs, there are many inconsistencies between features available in full version and headless, their debugging protocol has different APIs that work on headless/non-headless in Linux or Windows, and so on. Of the bugs I've submitted, some have been fixed in the upcoming M59, so other critical ones may take longer due to their backlog. I suppose for now (maybe until M61-62), Chrome full with xvfb or even PhantomJS are better options. When you realize that Chrome is about the same size (by LoC) as the Linux kernel [1], you can't help but wish for a leaner & faster headless browser. There seems to be some work going on building Firefox pure headless as well. Great overall, as long as all the browsers try to follow the RemoteDebug initiative [2]. [1] https://josephg.com/blog/electron-is-flash-for-the-desktop/ https://josephg.com/blog/electron-is-flash-for-the-desktop/ [2] http://remotedebug.org http://remotedebug.org

fastest963 10y ago
Unfortunately, I have to agree, though they have been very helpful and vigilant on the bugs that have been filed. Most of mine have been fixed in a few days.
- nreece 10y ago
  Hey James, I did come across some of your bug submissions, and our joint discussions on the headless-dev Google Group.
chrissnell 10y ago
Yes, this exactly. I wrote Crabby [1] a few months back to schedule automated page testing using Chrome and webdriver but doing anything automated with Chrome is really atrocious. You can't expect to load more than one page every 10-15 seconds on the average 8GB instance and it occasionally crashes or otherwise stops working completely. I ended up writing a simple check using Go's net/http library to do basic performance profiling but it doesn't measure DOM loading like the Chrome checks do. Such a bummer What I really want is an easy, cross-platform way to collect the network timings for each object like you get in Chrome's dev tools network waterfall graphs. [1] https://github.com/chrissnell/crabby https://github.com/chrissnell/crabby

livoras 10y ago

PhantomJS has plenty unsolved issues(up to 1.7k+), a replacement instead of combination might be a better choice.

est 10y ago

Is it possible to install Chromium on a server without X environment? Last time I checked it requires a shit ton of dependencies.

0xFFC 10y ago

Can somebody explain to me what is this good for? Thank you.

vacri 10y ago
Headless browsers are frequently used in buildserver testing of web apps.
jamesgeck0 10y ago
I'll be using it to generate printable PDF reports. It's a huge pain getting highcharts graphs to look identical in every browser's print view.

vvoyer 10y ago

Almost one year old but there was a talk on headless chrome at the Blink conference (BlinkOn 6): Video: https://youtu.be/GivjumRiZ8c https://youtu.be/GivjumRiZ8c Slides: https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcxzMQNEE7PcuQrGu83No3l0lw https://docs.google.com/presentation/d/1gqK9F4lGAY3TZudAtdcx... More links: Headless Chrome architecture: https://docs.google.com/document/d/11zIkKkLBocofGgoTeeyibB2TZ_k7nR78v7kNelCatUE/ https://docs.google.com/document/d/11zIkKkLBocofGgoTeeyibB2T... Mailing list: https://groups.google.com/a/chromium.org/forum/#!forum/headless-dev https://groups.google.com/a/chromium.org/forum/#!forum/headl... All of those links are on https://chromium.googlesource.com/chromium/src.git/+/master/headless/README.md https://chromium.googlesource.com/chromium/src.git/+/master/...

deleted 10y ago

[deleted]

softwarelimits 10y ago

is this is chromium, too?

laurencei 10y ago

Can anyone confirm - would this work with a Flash/SWF application? i.e. could I use the headless mode to interact with the Flash Application to run some commands and retrieve the output? I tried googling around but didnt find much to say either way...

swah 10y ago

Related: I want to take screenshots of a few news websites for a little fake news project of mine, and most approaches return something completely different than what I'm seeing when I open Chrome. Limited height would be better/ok (something like the first 3000 pixels). Low volume / can be slow (30 seconds would be ok). Those news websites many times have infinite scrolling. I've tried: - phantomJS (rendering sucked, tried every technique I could find to wait for JS to load) - wkhtmltopdf (almost ok, generates a huge 30M image with all the height, no antialiasing it seems) - https://github.com/gen2brain/url2img https://github.com/gen2brain/url2img (this was the best so far, uses Qt bindings but not the latest version) - actually run a headless browser in DigitalOcean with xvfb-run and take a screenshot: I failed at this What I didn't tried was Selenium, because it seemed even harder. How would you guys do it?

deleted 10y ago
[deleted]
oceanswave 10y ago
Create a chrome extension for yourself that automates the process
stangls 10y ago
I use xvfb for selenium testing. It is really easy to set up and can take screenshots or videos while automatically browsing websites.
- swah 10y ago
  Thanks - I tried Selenium (on the desktop) with geckodriver now and it rendered well. The only thing is that long screenshots didn't work but there is probably a workaround for that.

retube 10y ago

This page doesn't load for me (IE behind corp firewall). How does one drive/automate a headless browser? What kind of API is there?

tonyblundell 10y ago
Docs and a Youtube presentation... https://chromium.googlesource.com/chromium/src/+/lkgr/headless/README.md https://chromium.googlesource.com/chromium/src/+/lkgr/headle... https://www.youtube.com/watch?v=n6biclFh0i0&feature=youtu.be https://www.youtube.com/watch?v=n6biclFh0i0&feature=youtu.be

wslh 10y ago

How fast is the debugging mode? I tried the first debugging protocol when Chrome added it and it was very difficult to use. I assume this time is different?

unixhero 10y ago

What is the use case for headless Chrome?

ntaylor 10y ago
> Headless mode allows running Chromium in a headless/server environment. Expected use cases include loading web pages, extracting metadata (e.g., the DOM) and generating bitmaps from page contents -- using all the modern web platform features provided by Chromium and Blink. Practically speaking, software developers will use headless Chrome to automate testing of product functionality. Today, developers use systems like Selenium or PhantomJS to accomplish this feat, but it's a painful process to maintain these headless browser execution engines. Adding headless support into Chrome means that developers can count on the presentation of their application on a given version of the Blink engine run within Chrome.

hbakhtiyor 10y ago

i use when they announced headless mode on linux, and built generating thumbnails from captured screenshots of websites and uncovering the technologies used on websites and the api is available for free, https://github.com/letsvalidate/api https://github.com/letsvalidate/api

brendandahl 10y ago

For those interested, Firefox is also going to support a headless mode. The current nightly supports headless SlimerJS on Linux and more platforms will come soon. https://bugzilla.mozilla.org/show_bug.cgi?id=1338004 https://bugzilla.mozilla.org/show_bug.cgi?id=1338004

hackcasual 10y ago

Looks like the launch bug is private? https://bugs.chromium.org/p/chromium/issues/detail?id=705916 https://bugs.chromium.org/p/chromium/issues/detail?id=705916

tianlins 10y ago

How fast is headless vs. normal? According to https://developers.google.com/web/fundamentals/performance/critical-rendering-path/render-tree-construction https://developers.google.com/web/fundamentals/performance/c... the chrome browser spends a decent amount of time on other steps such as parsing HTML. I wonder how much time could be saved by not rendering pages into pixels.

zigomir 10y ago

Not sure if this can enable SSR (server side rendering) for any client side lib?

du_bing 10y ago

It seems that Chromium 59 still can not be installed on Raspberry Pi, or anyone has done it? It will be great to use this headless Chromium on Raspberry Pi to execute some routine web browser jobs. Does it support the extensions installed on Chromium? Curious.

stheakanath 10y ago

Is this confirmed to work with Flash? I know selenium did not support Flash so it caused some dev issues.

dkastner 10y ago

I put together an example of how to run chromium with --headless driven by cucumber/capybara: https://dkastner.github.io/2017/04/21/headless-chrome.html https://dkastner.github.io/2017/04/21/headless-chrome.html