3 ms·
If your referrer is from Google, they actually do allow you to see public detail. So just override referrer header, and you'll see a lot more.
by kuschku 9y ago
If your referrer is from Google, they actually do allow you to see public detail.
So just override referrer header, and you'll see a lot more.
- pmlnr 9y agoDoesn't work for me either, at least not with cURL.
- BenjiWiebe 9y agoDon't forget that you have to misspell Referer.
- pmlnr 9y ago!!! I'm having a really bad day with typos and overlooking them. You are correct; it does work with https://google.com https://google.com as referrer. I'm updating the entry. EDIT aaand it stopped working. It was either a glitch or it will lock you out after a few requests.
- OkGoDoIt 9y agoIt definitely locks you out after a few requests. In the past I built a LinkedIn profile scraper, but they are really really good at locking down their data. If you even somewhat appear to be a bot, you’re blocked.
- flukus 9y agoCan it differentiate between a bot and a human opening up a dozen links in tabs? I suspect the tumblr one can't, it appears to rate limit me for using there archive pages.
- deleted 9y ago[deleted]