6 ms·
SlimerJS: a replacement for PhantomJS using recent Firefox
- 2trill2spill 9y agoDoes this support FreeBSD? The docs say it's supports anything that can run Firefox so I would think yes. I can't check cause I'm in a car for the next 6 hours. But if this supports FreeBSD I would definitely use it over puppeteer because puppeteer does not support FreeBSD and it doesn't really look like they care to add support [1]. [1]: https://github.com/GoogleChrome/puppeteer/issues/1049 https://github.com/GoogleChrome/puppeteer/issues/1049
- folkhack 9y agoAt this point you should really be looking into Google's Puppeteer - https://github.com/GoogleChrome/puppeteer https://github.com/GoogleChrome/puppeteer One of the most active browser automation libs out there and it's just a pure joy to use =)
- dmbass 9y agopuppeteer only supports Chorme so you will need to use Slimer if you wish to use Firefox.
- paulirish 9y agoCorrect for now. However... Firefox did start investigating supporting the Chrome DevTools Protocol: https://groups.google.com/d/topic/mozilla.dev.platform/4-4A8W-nP5g/discussion https://groups.google.com/d/topic/mozilla.dev.platform/4-4A8... We (Chrome DevTools/Puppeteer team) have had detailed discussions with their DevTools team on this, recommending what subset would enable Puppeteer. I don't know where they are with this, but in the long-term, this solution looks most promising.
- folkhack 9y agoFirefox/Mozilla has been doing a great job with cross-browser initiatives like the WebExtension APIs, and dipping their toes into CDP. I am STOKED to hear you guys have been having cross-team conversations on these details! Shameless fanboy moment: have followed your stuff for years, and I absolutely love the products your team(s) put out (CDP/Puppeteer)!
- paulirish 9y agoAww thanks amigo! Thanks v much. Plz holler my way if there's anything we can do better. The devtools teams (Chrome, FF, Safari, and Edge) actually talk quite a bit. We collaborate on the console spec [0] and keep each-other in check on feature parity. ;) [0] https://github.com/whatwg/console https://github.com/whatwg/console
- bobajeff 9y agoIt would be immeasurably useful for Firefox to have support for Chrome DevTools Protocol (or have something with the same functionality). Anyone from Mozilla know where to follow the work for this?
- paulirish 9y agoThese are the only public links I've got: https://bugzilla.mozilla.org/show_bug.cgi?id=1391465 https://bugzilla.mozilla.org/show_bug.cgi?id=1391465 https://github.com/devtools-html/rust-cdp https://github.com/devtools-html/rust-cdp
- folkhack 9y ago100% honest and non-hostile question - why would you want to use Firefox?
- orthecreedence 9y agoThat question could be easily turned around: why would you want to use Chrome? Personally, if I were to set up automated testing, I'd like to do it with both Chrome and FF, especially given that FF seems to be making a comeback lately. Another reason might be that I've never had anything but problems building Chromium from source, but have successfully built Firefox. So in situations where you want to know with certainty that your binary is derived from the published source code, Firefox wins IMO.
- folkhack 9y ago> why would you want to use Chrome? Chrome as a platform has the majority of browser market share and is continuing to be the top contender, I don't think anyone's going to knock them off their high horse anytime soon. Targeting the most common platform for applications deployment is common sense. > especially given that FF seems to be making a comeback lately FF is stagnant as far as market share, and has been trending down for at least 2-3 years. Currently it's around 5-6% and has been for awhile. Saying FF is making a comeback is disingenuous to the actual stats. > Another reason might be that I've never had anything but problems building Chromium from source Totally understand this one - I've found my roommate's Gentoo laptop in the fridge doing a full Chromium build more than a few times. All-in-all I totally get where you're coming from, Chrome and FF are my preferred automation/testing platforms for sure =)
- zaarn 9y ago>Targeting the most common platform for applications deployment is common sense. If I recall, people used to say something very similar about Internet Explorer. Of course Chrome isn't as awful as IE back then but still... IMO it makes the most sense to support two independent implementations, especially for browsers. >Currently it's around 5-6% and has been for awhile. Saying FF is making a comeback is disingenuous to the actual stats. For desktop FF has about 10% marketshare. 5-6% includes iOS and Android, the later comes with a preinstalled chrome with little incentive to switch... With the upcoming ESR 60 release, I also expect (and heard) a lot of companies will switch to Firefox, atleast in the german region (where marketshare increased from Dec to Jan). Some Linux distros might switch over from the previous ESR release and a lot of users could potentially reevaluate. >I've found my roommate's Gentoo laptop in the fridge doing a full Chromium build more than a few times. Chromium is not Chrome, it's Chrome with all the Google Binaries ripped out. Chrome itself is, to my knowledge, not open source, only Chromium is.
- dchuk 9y agoso I have an upcoming side project that will heavily leverage scraping/screenshotting and I plan to use either headless chrome or slimerjs. It will be a Rails app, queueing up lots of sidekiq jobs, that I (presumably) will have calling a headless browser instance, probably running in a container. Ideally I'll run this either on Heroku or even better, just a DO droplet running dokku. I found browserless.io and the supporting github project over the weekend, looks promising. Who else is doing this? Is the architecture I just touched on the right way to go about doing this? Or should I have each sidekiq job spin up its own browser container for the scraping job? Also, chrome vs firefox won't really matter in this instance, but reliability/performance/memory will...any recommendations?
- deedubaya 9y agoI built https://github.com/danielwestendorf/pdf-bot-pro https://github.com/danielwestendorf/pdf-bot-pro a few days ago, a scalable 1-click heroku deploy rewrite of pdf-bot. It's fully functional, except I built the background queuing on Faktory, and there is not working Faktory add-on available for heroku yet (and thus, no real promotion of the project yet). You could set up a faktory server elsewhere and define the URL as an ENV var and it would work today.
- dchuk 9y agoNice thanks for the link. So if I'm understanding correctly, your app essentially runs as a standalone api, receiving pdf requests, queueing them, processing them, then hitting a specific webhook url on success for other systems to then query the pdf tool to get the resulting pdf. This is approximately how I was thinking of settings thing up as well, so thanks for this!
- deedubaya 9y agoYup, exactly.
- africajam 9y agoNice, thanks for sharing! I built a simple scraper and I need to add background processing etc: https://github.com/RealEstateWebTools/property_web_scraper https://github.com/RealEstateWebTools/property_web_scraper Will dig into your code to see what I can learn.
- ravenstine 9y agoGlad to see there's an alternative to headless Chrome that's not tied to Google in any way.
- TAForObvReasons 9y agoWithout being too cynical, is the fact that Mozilla receives most of their funding from Google not relevant when considering Google's influence?
- st3fan 9y agoNot funding. Revenue. What influence are you talking about? Can you be more specific?
- dragonwriter 9y agoIs the amount of money Mozilla is able to extract from Google a statement about Google’s influence or Mozilla’s?
- moocowtruck 9y agoi think it depends if mozilla can still operate if googles funding were to dry up
- codedokode 9y agoThe "simple example" on the front page uses promises which silence all exceptions by default. Not a good idea. Futhermore, with synchronous operations the code would be much simpler.
- sebazzz 9y agoHow does SlimerJS compare to Selenium which is truly cross-browser and the cornerstone of our internal web automation tests?
- zaarn 9y agoAfaict the difference is that slimerjs uses Firefox Headless while Selenium spins up the entire browser. For Web Testing spinning up the browser is okay, for some other tasks using just a headless version can be good enough.
- sebazzz 9y agoSelenium can also use headless if you use the MOZ_HEADLESS environment variable.