2 ms·
Do you know why google allows WSJ et al to serve one set of content to a crawler, but another to an actual user? I thought they punished that kind of behavior.
by oh_sigh 8y ago
Do you know why google allows WSJ et al to serve one set of content to a crawler, but another to an actual user? I thought they punished that kind of behavior.
- SeanAnderson 8y agoI don't have any information that isn't public, no. Just building off of previous discussions of outline.com in HN threads.