3 ms·
How is this at all sustainable as a business model? My company is in charge of scraping a few TB/week. Would we be allowed to use your service?
by gravypod 7y ago
How is this at all sustainable as a business model? My company is in charge of scraping a few TB/week. Would we be allowed to use your service?
- ithkuil 7y agofrom a cursory look it seems that this software runs on your client, i.e. it's not a service. Scrolling down: " Can I scrape password protected stuff with Spider? Yes! It’s a browser extension, so as long as you log in first, you can scrape whatever you like. "
- chrismarlow9 7y agoGo watch the demo video. From what I can see it's not automated enough to do tb/week. That said I bet you could hook it into scrapy and automate it. Probably will be slow though
- evandena 7y agoIt's just a browser extension
- thinkloop 7y agoIt's all client-side, it's not what you think, it's more a light scraping-on-the-go type product than a permanent enterprise solution. Good for what it does tho.
- elldoubleyew 7y agoI think this is all done client side, manually, through a browser extension. They aren't providing you with an API to retrieve this data live afterwards. Still a cool project but I think this limits a lot of its utility. Providing an API to access the data as a service would be a lot more profitable.
- deleted 7y ago[deleted]
- primitivesuave 7y agoThis is a browser extension that takes the place of copy/pasting from web sites (ie manual scraping), what you’re looking for is probably more automated.