39 ms·
I've done a lot of work in the scraping space for a side project I've worked on for years. At this point I've built my own tools, but over the years I've tested
by superfrank 5mo ago
I've done a lot of work in the scraping space for a side project I've worked on for years. At this point I've built my own tools, but over the years I've tested just about every paid scraping service that currently exists.
I think it would be really valuable to add some sort of tool that allows me to enter a URL and see a subset of the data returned, latency measurement, and flag if the Enhanced Bot Detection was required to get that data.
I can't count the number of times I've signed up for a service to see if they could get data from a URL that's giving me problems only to find that they couldn't scrape it either. It would be really helpful to know before I sign up what I need to pay for and that your tool can actually get the data I want. While it may result in fewer sign ups for your site, I think it will result in the customer who do sign up being higher value customers and potentially reduce support burden.
- jeromechoo 5mo agoYou’re probably also looking for that tool to be available unauthed. And yeah agreed. We do this at Diffbot and the test drive is the 2nd highest visited page.