3 ms·
Back when I was running a service called Feed Digest (which I went on to sell - mostly due to the issue I'm raising here - now known as Feed Informer) this was
by petercooper 17y ago
Back when I was running a service called Feed Digest (which I went on to sell - mostly due to the issue I'm raising here - now known as Feed Informer) this was a common problem. Not just with Craigslist, but with feeds from Google and all over the place. We wouldn't hit a single feed any more than once every 30 minutes but getting whitelisted everywhere was an exhausting task and ensuring you could always get the feeds for your users to reprocess was so difficult that I was heavily turned off of getting into a "man in the middle" situation like that again.