4 ms·
Great project. If you continue to crawl the data, be sure to save it so you can detect price changes a la camelcamelcamel.
by xnx 3y ago
Great project. If you continue to crawl the data, be sure to save it so you can detect price changes a la camelcamelcamel.
- pencildiver 3y agoGreat call! I am doing back-ups on Mongo and this is a good use-case for tracking changes. Also trying to figure out how to detect is a product is sold out or not being sold anymore.
- Minor49er 3y agoI worked on a competitor to CamelCamelCamel years ago. We had this exact issue since people would often click through to a page where the price or availability were different from what we were showing Ultimately, we ended up adding an interstitial page between the product listing on our site and the page on the seller's site This interstitial checked to see if we checked the price in the last couple of minutes, and if not, it would run a quick scrape of the page to ensure that we had the most up to date information I can't remember exactly what the messaging or behavior was when there was a difference. I think there was a message that was displayed if the prices were different. Or if the product was actually out of stock, it would pull the user back into our site with a toast explaining that the product was no longer available Anything less aggressive than this resulted in more customers experiencing price/availability errors or simply leaving the site, and anything more aggressive resulted in angry site owners who were losing bandwidth to our bots > Also trying to figure out how to detect is a product is sold out or not being sold anymore In these cases, either the page will say as much (eg: "Product Unavailable"), have some kind of stock or status code hidden beneath the UI to show that it's not available, or the target page will simply vanish from the web. However, none of these are guarantees. A site could say that a product has been discontinued, but the item could come back later, or under a different SKU, or whatever else
- secabeen 3y agoFor all of Amazon's faults, the fact that they tolerate CCC does drive a lot of my online purchases there. CCC used to track other sites, and was eventually blocked on all of them. If more sites want my business, showing their pricing history (either from internal data, or by letting someone build the DB) would go a long way.
- moneywoes 3y agois camel camel whitelisted by amazon? or can any scraper work
- tehlike 3y agoAmazon associates program doesn't normally allow price trackers except with written approval
- tehlike 3y agoAmazon doesn't allow price alerting/tracking on their affiliate program anymore, you need explicit written consent. I am the owner of https://pricetracker.wtf https://pricetracker.wtf and got the boot today.
- nocoiner 3y agoIs it somehow known that CCC hasn’t been co-opted by Amazon? Frankly I figured Amazon would have bought them out a decade ago, but maybe the CCC founders have a stronger ethical compass than I do.