Speaking of scraping: One fundamental problem is the lack of standardized presentation of discoverable semantic web data cough RDF cough ontologies cough. If it

Speaking of scraping: One fundamental problem is the lack of standardized presentation of discoverable semantic web data cough RDF cough ontologies cough. If it's standardized and provided, then it's less of a burden to the scrapee and more useful for others. For example, finding the precise version history of open source software inevitably requires scraping a web page, download hosting server, or some source control. This is costly for the servers hosting it and fragile. Circling back to the case of price data. When an item is not in-stock, merchants have a habit of displaying disingenuous prices that are very high or low. Another dark pattern of gamification: displaying deceptive prices based on User-Agent, such that crawlers see one price while potential buyers see another.