3 ms·
Yes, everybody scrapes the prices of the others.
by hk__2 10y ago
Yes, everybody scrapes the prices of the others.
- rckclmbr 10y agoMost companies will use resources not in their datacenter and not identifiable. Executives know it's sketchy, but they do it anyway.
- fareesh 10y agoWith stuff like Facebook opengraph (e.g. og:price) and other meta tags meant to help search engines and social networks get this sort of data to display inline, do you think it's inevitable that complex scraping will no longer be needed in a practical sense since everyone will be inadvertently optimizing their markup in a way that you could write a really simple parser to grab the data?
- greglindahl 10y agoGiven that FB's bot identifies itself, no, eventually some websites will present og: markup only to FB's bot.
- fareesh 10y agoCouldn't you just spoof the user agent though? Or is there some other mechanism that you can use to verify the bot is Facebook's?
- greglindahl 10y agoIP address.