3 ms·
The first thing I tried it on was http://hcpdirectory.cigna.com/web/public/providers http://hcpdirectory.cigna.com/web/public/providers, which is a typical inst
by CGamesPlay 12y ago
The first thing I tried it on was http://hcpdirectory.cigna.com/web/public/providers http://hcpdirectory.cigna.com/web/public/providers, which is a typical instance of a site I might like to scrape, but still one of the simpler ones. It was not able to extract meaningful information.
- logn 12y agoThat works in my test locally. The cloud servers are rather overloaded at the moment and also sometimes Tor is just flaky. Here's the request/response: http://pastebin.com/raw.php?i=KncMgbfB http://pastebin.com/raw.php?i=KncMgbfB edit: note that there's a couple different search forms on that page. If you run this on your own machines you can use the Form Query API which lets you specify a form ID and then set specific values for each form field (e.g., for a provider search with City/State and Name). And you can use any non-Tor proxies or a direct connection.