4 ms·
They probably have some kind of a deal with Google. Same kind of a deal that LinkedIn has with Google... you can't view most of the LinkedIn profile pages that
by Jerry2 8y ago
They probably have some kind of a deal with Google. Same kind of a deal that LinkedIn has with Google... you can't view most of the LinkedIn profile pages that Google indexes without being logged in. If I served a different page to Google's search bot from pages that everyone else can see, I'd be banned from Google's index. But these big companies have special rules...
- JonathanMerklin 8y agoCurrently I'm not near my computer to verify myself (I am on my phone, before bed), but can you get into walled-but-indexed LinkedIn/Pinterest content with "Googlebot" UA spoofing? I'd want to assume that their systems are more advanced than that (or if they're really in cahoots, maybe Google crawls them with a unique, secret bot UA to prevent this sort of thing), but I just got done with another thread where an ex-Tumblr dev said all user content was in a single S3 bucket without MFA delete so my gut feeling is lending a bit more weight to the "everything is held together with rubber bands and duct tape and it's only a matter of time before the jig is up" parameter than the "large, powerful companies tend to hire smart people that communicate and design robust systems effectively" one.
- oarsinsync 8y agoBased on my experience in the nautical industry, where most things were quite literally held together using velcro, metaphorical rubber bands and duct tape seems quite likely for relatively younger web industries.