4 ms·
Gonna check it out. I always use request and cheerio on node so it's almost identical. Is this an actual headless browser though? When I have to go that route i
by dannyhw 9y ago
Gonna check it out. I always use request and cheerio on node so it's almost identical. Is this an actual headless browser though? When I have to go that route it's painfully slow.
- jancurn 9y agoThe crawler uses PhantomJS, but you can also create jobs that use a plain HTML downloader or even cheerio. The platform is very flexible.
- dannyhw 9y agoVery cool. I'm looking at it more in depth and I'm pretty impressed. This is basically all things I've been doing for years, but encapsulated in a way that I'm pretty sure end users will be able to use. I've always built these beasts and tried to evangelize for a MEAN stack as the best option but people haven't been ready. I have never met a problem I felt promises were a good solution for, but you've done a really great job of using them to let people who can't think asynchronously get things done. Pseudo URLs are also something I've done before but are never the optimal solution. Again though, this is something that will allow a total beginner to catch up to me about 75%, and I've been developing for 25 years.
- elyrly 9y agoi have been playing around with Puppeteer <https://github.com/GoogleChrome/puppeteer> https://github.com/GoogleChrome/puppeteer> Wondering alternative to Selenium