4 ms·
Is there an FLOSS project that combines Scrapy scrapers and just makes the results publicly available?
by lapser 4y ago
Is there an FLOSS project that combines Scrapy scrapers and just makes the results publicly available?
- mdaniel 4y agoThis is the one I know about: https://morph.io/ https://morph.io/ and https://github.com/openaustralia/morph#readme https://github.com/openaustralia/morph#readme (AGPLv3) -- they used to be at the intersection of "heroku for scrapers" and DoltHub (e.g. https://www.dolthub.com/repositories/dolthub/us-businesses/data/master/businesses https://www.dolthub.com/repositories/dolthub/us-businesses/d...) since the scrapers would run but then make their data available as CSV or sqlite or whatever. But, when I just tried to load one of the morph.io scrapers, the page just said "creating new template" so I'm guessing they've gone the way of the ScraperWiki.com that preceded them: turns out, hosted compute for free isn't free