3 ms·
I'm curious how you plan to avoid/circumvent the inevitable hard IP ban that the largest (and most sought after targets) will place on you and your services onc
by GigabyteCoin 13y ago
I'm curious how you plan to avoid/circumvent the inevitable hard IP ban that the largest (and most sought after targets) will place on you and your services once you begin to take off?
I could have really used a service like this just yesterday actually, I ended up fiddling around with iMacros and got about 80% of what I was trying to achieve.
- pranade 13y agoIt's a great question. What we're really trying to do is make data accessible programmatically and at scale. We want to connect data providers and data consumers with APIs in a way that's mutually beneficial vs. being a tool for data theft. Our hope is to (once we scale) actually work with data providers directly on the on the distribution of their data so the IP ban becomes a non-issue.
- Nilzor 13y agoBut isn't the point kinda to let the users come up with data providers themselves? If you say "Only these 500 data providers are available for scraping", you don't have a business. If you don't' have such a limitation, you'll not be able to work directly with all data providers. You'll have IP problems.