6 ms·
Unix swiss army knife for headless browser JavaScript
- trentlott 11y agohttps://raw.githubusercontent.com/sotownsend/boojs/master/logo.png https://raw.githubusercontent.com/sotownsend/boojs/master/lo... Headless Browser Javasrcipt Tool
- rem0x4 11y agoThat's how germans pronounce it.
- larvaetron 11y agoPronounce what?
- stretchwithme 11y agoCome on, you didn't hear anything? :-)
- jaytaylor 11y agoNeat project. It's 98% Ruby and is being called a "Unix tool" because it has cmdline options. Interesting criteria there. It seems to be pitched as a js testing tool. Not clear on how it is different from node or just plain phantom.
- ende 11y agoRight, maybe with something like this? https://www.npmjs.com/package/commander https://www.npmjs.com/package/commander
- mryan 11y agoAccepting command line options does not make something Unix-y, IMHO. I assumed it was called a "Unix tool" because it reads data from a file/stdin, does something, then writes to stdout. If I needed to do something with JS as part of a series of Unix commands this would fit in the pipeline nicely. e.g. let's say someone wanted to do a one-off data transformation and they understand JS a lot better than Awk.
- jamesondh 11y agoI'm slightly unclear on this, how does this differ from Node and what are its uses?
- woah 11y agoIt's a headless browser. Node is a server framework that runs js.
- maemre 11y agoWhich means that it can handle scripts that rely on browser features like DOM whereas node can't handle that. So, you can use it to test the client-side code automatically (with a CI tool of course). You can't do that in node if your code accesses DOM (e.g. calls something like document.getElementById or uses jQuery). Btw, it is just a CLI wrapper around PhantomJS[1] I guess. [1]: https://github.com/ariya/phantomjs https://github.com/ariya/phantomjs
- MichaelGG 11y agoAnother (popular?) use is for SPA or JS-heavy pages to render themselves for search engines. Crawl the site with PhantomJS. For each page, give it a few seconds to do XHR and render things, then save the HTML snapshot. When you get a request from Googlebot, serve the HTML snapshot instead of the app, which Google apparently still cannot handle. Ta-da! (And piss off people who hate JS "apps" or still desire the HTTP/HTML document ideal, for better or for worse.)
- rem0x4 11y agoBooJS supports the browser DOM. You can call `document` in BooJS but not NodeJS. You can't import arbitrary browser javascript libraries into NodeJS but you can with BooJS. It's purpose is to assists with unit tests in a sane manner but it's great for other random things; reminiscent of netcat.
- jaytaylor 11y agoDo you think there is a reasonable way to provide this useful functionality without the Ruby dependency? Pure JS, perhaps?
- tracker1 11y agoIt seems to me, that something similar could be constructed with node's PhantomJS bindings, but not sure that there would be much less overhead... it does feel a little awkward having the Ruby dependency...
- erikano 11y agoPresence of a "-v" flag usually means verbose.
- gingerrr 11y agoAlternately, version. Either way it's clobbering
- gue5t 11y agoIt's often hard to tell where the hard problems are being solved in high-level tools like this. tl;dr: CLI program wrapping PhantomJS, which is based on QtWebkit.
- gingerrr 11y agoI'm not sure why i would use this over the already-existing phantomjs command line API (http://phantomjs.org/api/command-line.html http://phantomjs.org/api/command-line.html), especially as it adds Ruby to the mix...also quite suspicious of the seemingly-unrelated plug for a company in the repo readme.
- rem0x4 11y agoLay off the accusations. 1. I'm a founder of that company and this is apart of a much larger project i'm releasing in a few months. 2. That command line does not act like a normal unix tool and will make your life a living hell because QT has some horrid misunderstanding of what constitutes a newline, outputting format characters, and incorrect streams (stdout is used for errors). P.S. I do kernel development on FreeBSD in my spare time. I know a little bit about unix, ya know?
- gingerrr 11y agoCongratulations on your journey with the company thus far. I simply don't understand why a randomly-dropped plug for said company is at all cogent in the documentation for a library that is released under what seems to be your personal github and has no other visible attachment to said company. But I'm not accusing you of anything - I mentioned it because it struck me as out-of-place and potentially PR-opportunistic, if that's not the case I apologize. From one founder to another: perhaps next time consider taking the less-abrasive approach to defending your product to avoid alienating your (potential) user base? Your hostility ends up doing the opposite of assuaging my concerns. Good luck. Edited due to parent edit: I'm...happy for you that you develop on the FreeBSD kernel? I'm also not sure how that is relevant to the discussion of why this tool should be used over any other of the dozens of headless toolkits and wrappers out there - and your readme certainly doesn't do any work to convince me to care about your implementation over any existing solution. You're surprisingly defensive for someone claiming to be confident in their development skills, I'm not your enemy here - I was, until this exchange, a potential user (and may yet be, I'm not one to pass over a good tool simply for personal reasons).
- gingerrr 11y ago
- AlexNeoNomad 11y agoWhy not to execute a js code in Chrome or Fifefox in Javascript console?
- prezjordan 11y agoI imagine this is to be used for something like automated testing.
- curiously 11y agoYou would be able to do it with Selenium, which drives a real browser not a fake one.
- ugexe 11y agoNot everyone wants to install x-server and friends on their servers
- curiously 11y agowhy wouldn't you?
- abroncs 11y agoThis is headless.
- moondowner 11y agoBooJS uses PhantomJS. In Selenium, if you want to go headless you will use PhantomJS again [1]. [1] http://code.tutsplus.com/tutorials/headless-functional-testing-with-selenium-and-phantomjs--net-30545 http://code.tutsplus.com/tutorials/headless-functional-testi...
- curiously 11y agojust run it with Xvfb, and you got a headless browser.
- syncerr 11y agoIt would be better to handle your errors through try{}catch{} rather than using phantom's onError as an exception seems to force an exit. https://github.com/sotownsend/BooJS/blob/master/lib/boojs.rb#L39 https://github.com/sotownsend/BooJS/blob/master/lib/boojs.rb... I threw together a similar implementation in bash with this change: https://gist.github.com/spence/30d5aa383fec6be8e51e https://gist.github.com/spence/30d5aa383fec6be8e51e Looks like ./phantomjs-repl.sh > missing ReferenceError: Can't find variable: missing > window [object Window] > console.log('Hello') Hello undefined > !jquery-1.11.2.min.js Loading jquery-1.11.2.min.js ... done > $('<span>Hello</span>').text() Hello > ^C
- curiously 11y agoTake headless browsers with a pinch of salt. They are not a replacement for a real browser. Don't be surprised when it renders or behaves differently from a real browser. Did they fix the memory leak issues in PhantomJS? Take a look at the issues and consistently you will find people having trouble rendering a complete site. Creating a headless browser is really taking a real browser on Xvfb. You have the support of a billion dollar company supporting and updating their browser, you reap the benefits of being able to run it headlessly. The drawbacks of this method is often overstated, 'oh my run a fat browser process AND X display? ok grandpa' but the real hidden problem with PhantomJS is understated 'why does my phantomjs process balloon in memory and cpu usage and crash, I can render the website fine in my browser'. Standalone headless browsers are good for hacks but I wouldn't rely on it.
- gingerrr 11y agoafaik the worst of the memory leaks were fixed with the 2.0 release, and they also upgraded QtWebKit to 5.3 which fixed some pretty egregiously outdated constraints (like not having access to Function.prototype.bind) present in the previous version. Sadly this tool utilizes phantomjs 1.9 which means the memory leaks are still present and major WebKit features are still missing. One great use for headless is to separate non-UI framework/platform tests from UI tests - at my previous gig we wrote all the tests for the runtime platform against phantomjs to avoid the unneeded overhead of a full browser for things like data validation or mocking service calls, but used selenium to do the visual/interaction tests for the application UI itself. it's also a nice separation of concerns, though of course we hit all the same problems you mentioned as we were also pinned to 1.9.
- curiously 11y agohttps://github.com/ariya/phantomjs/issues https://github.com/ariya/phantomjs/issues 1470 open issues the first one is about phantomjs crashing and many many other scary sounding issues you wouldn't have if you've used just a regular browser with Xvfb. you can't really be sure unless you've literally run a test on the colorful variety of websites in the wild. You need a huge development and quality assurance team. You also need a way for a large crowd of people to test your browser and send back useful reports automatically whenever it crashes.
- unix_flyer 11y agoThere is a good example of try{} catch{} in post 7 of this thread at unix.com: running unix command from java http://www.unix.com/unix-for-dummies-questions-and-answers/192883-running-unix-command-java.html http://www.unix.com/unix-for-dummies-questions-and-answers/1...
- Gepsens 11y agoQuestion : can you connect it to a running instance of Phantom that is hanging to debug already running code ?
- secoif 11y agoI've been working on a similar tool that works with (some) real browsers. Pipe JS in, get console logs out. Simple. https://github.com/hughsk/smokestack https://github.com/hughsk/smokestack
- mattdesl 11y agoNice. Anything like this built on jsdom? PhantomJS is pretty bulky for most non-visual tests.