2 ms·
Did I say they don't have the legal right to do that? No, I didn't. I doubt anybody did. There's no particular reason to think that spiders are a giant problem
by wpietri 14y ago
Did I say they don't have the legal right to do that? No, I didn't. I doubt anybody did.
There's no particular reason to think that spiders are a giant problem for them. Any high-volume site deals with crawlers all the time. If it were a problem because of some site architecture weirdness, it'd be easy enough to set up something that wouldn't be problematic. For example, the RSS feeds they already offer.
Worst case, they could charge people like Padmapper a fee to license the data. Which in fact they already do, but they insist on it being mobile only.
So Craigslist's behavior is purely a business choice, not driven by any technical necessity.