4 ms·
With stuff like Facebook opengraph (e.g. og:price) and other meta tags meant to help search engines and social networks get this sort of data to display inline,
by fareesh 10y ago
With stuff like Facebook opengraph (e.g. og:price) and other meta tags meant to help search engines and social networks get this sort of data to display inline, do you think it's inevitable that complex scraping will no longer be needed in a practical sense since everyone will be inadvertently optimizing their markup in a way that you could write a really simple parser to grab the data?
- greglindahl 10y agoGiven that FB's bot identifies itself, no, eventually some websites will present og: markup only to FB's bot.
- fareesh 10y agoCouldn't you just spoof the user agent though? Or is there some other mechanism that you can use to verify the bot is Facebook's?
- greglindahl 10y agoIP address.