8 ms·
Web Scraping Finds Price Fluctuation
- maratd 11y agoStores aren't "guilty" of anything. Prices fluctuate based on demand. That's normal. If you don't think that's normal, break out a basic economics textbook and do a bit of reading.
- anewhnaccount 11y agoIf there's enough volume to satisfy the demand, the price doesn't necessarily increase. A sale by definition should generate elastic demand.
- granitosaurus 11y agoIncreasing price before a sale is actually illegal in pretty much every first world country. AFAIK the time gap between price increase and a sale has to be around 1 month in EU.
- unsettledtck 11y agoFair enough and consumers are rewarded for carefully researched shopping. However, many of the days we compared were specifically billed as major "sale" days and so I, at least, found the price inflation disconcerting.
- fleitz 11y agoDifferent stores offer different terms and have different costs... I shop at lots of stores I know charge more because they have better service. I love the shopping experience when I don't have to wait in line because they have enough cashiers and don't have people haggling over $5 at the cashier, if you want their prices go to their store...
- unsettledtck 11y agoThis is true and I definitely love the convenience of Amazon (who was the most guilty of higher prices, by the way). Frankly, it was Sam's Club that really surprised me since I always assumed that they offered everything at a lower price as a perk built in for membership.
- dragonwriter 11y ago> Frankly, it was Sam's Club that really surprised me since I always assumed that they offered everything at a lower price as a perk built in for membership. As a general rule, anytime an institution [0] has an established reputation for lower prices such that people assume their prices are lower without doing comparisons, this becomes a license to charge higher prices. [0] this is true beyond just institutions, too; "economy" packs of products frequently sell for higher per-unit costs than packs with fewer units per pack, because enough people assume that the cost is lower and don't do the math.
- unsettledtck 11y agoIntriguing! That makes complete and utter sense, which is part of the reason why we included Sam's Club in this scraping project. Wasn't actually aware of the higher per-unit costs.
- mschuster91 11y ago> [0] this is true beyond just institutions, too; "economy" packs of products frequently sell for higher per-unit costs than packs with fewer units per pack, because enough people assume that the cost is lower and don't do the math. Assuming the "economy" pack is the only offer you visit a distinct store for and the good is perishable (so that stockpiling doesn't work), then it might be cheaper if you factor in the time you need to reach said store, shop there and get back home into the "cost". The less often you have to go shopping, the more time you save. That is especially true with cat foods when your cat is a picky eater and only eats certain food from a certain place which is most unfortunately at the other end of the city...
- lotsofpulp 11y agoI've always assumed "sales" are purely marketing tools where the price is increased and then lowered, or if the product isn't selling, then you lower the price by calling it "on sale". Either way, as a consumer, one should always just disregard anything other than the number you have to pay for a product when evaluating its value.
- unsettledtck 11y agoI agree and it was edifying to find the actual data to back this up. Although, you can see that there were genuinely lowered prices, so there are deals to be had specifically on Black Friday.
- jharger 11y agoIt's only a reward if you consider your time spent researching worth less than the discount you actually receive.
- unsettledtck 11y agoAt this point, Black Friday and Cyber Monday have become ritualized cultural events. There is a cult around the supposed "rock bottom prices" on these days that allegedly remove the "research" bit and so we wanted to see if people were actually rewarded by capitalizing on the sales of these particular days.
- JoshTriplett 11y agoThat's one of the reasons stores offer sales, coupons, and similar: they allow stores to effectively offer different prices to different people, extracting more total revenue than if they had to charge everyone the same price. A store could never get away with directly saying "we charge more if you have more money". But they can offer coupons and sales, which have the same effect: those who take the time to clip coupons and watch for sales pay a lower price (and thus buy anyway), while those who don't pay a higher price.
- melling 11y agoSomeone told you that they offered the lowest price and you believed them? This is not a Hacker News story. Everyone ask their parents if they found the results surprising.
- unsettledtck 11y agoWell, actually we didn't believe them, which is why we conducted the experiment. And point of fact, there were actual stores that did offer the lowest price, but you needed to do your research and frankly, get lucky.
- melling 11y agoRetail stores play all kinds of games. Offering a low price on something, with limited quantities, to get you in the store. Once you're there, they'll try to sell you other things that are marked higher. I'd like to know the tricks that online stores use. I know that two people can click at the same time and get two significantly different prices on Amazon, for instance.
- vezzy-fnord 11y agoSome markets (particularly raw materials, certain commodities and manufacturers that sell to monopsony buyers) are more fixprice than others, where firms sell at a stable markup cost and adjust output (i.e. by reducing employment or units) as production costs change. While demand is always applicable, production costs often dominate in more vertically integrated firms and there are large lags in adjusting to signals.
- tyingq 11y agoI agree...I don't see the issue. Consumers are able to, with trivial effort, compare pricing from different providers. Fake sales might have been more interesting prior to the internet, where it wasn't as easy to compare. Like, for example, the current situation with new car prices.
- cookiecaper 11y agoI'm assuming that scrapinghub.com is invested in web scraping, so it'd be wise not to flaunt the actually illegal act going on here, which is the violation of the CFAA and various intellectual property laws that scraping almost always constitutes under conventional interpretation.
- swalsh 11y agoThis would be far more interesting if it had items that weren't electronics. So many products are sold with what is called a "Minimum Advertised Pricing Policy" that it makes it risky to sell less than the competition (which is probably selling at the absolute lowest price the market will allow). Apple for one has one of the most aggressive policies. I'm sure Target is counting on their size, and the fact that its a one day thing when they offered their discounts.
- unsettledtck 11y agoWell, the Toys category had non-electronic items and it was the category that was the most examples of inflated pricing (not such a shocker). However, all of the stores offered discounts on electronics at one point. You just had to be lucky in going to the right store on the right day.
- vinceguidry 11y agoFor me, the convenience of Amazon outweighs any "deals". I don't consider $10 off of a $50 item a deal. It needs to be 40%+ for the price reduction to really make a difference in my calculus. Amazon has my sister's address information already saved, I can just click it from the list when I'm buying presents for my nephews. Brands still have value when properly managed, don't let anyone tell you they don't.
- unsettledtck 11y agoAgreed, and so I will remain an Amazon shopper in spite of the fact that they (both the company itself and third party sellers) were by far and away the worst when it came to price inflation. Convenience is a part of the cost.
- criddell 11y agoSame here. I don't really shop at Amazon because it's less expensive. Frankly, I have no idea how Amazon's prices compare to most things because it's been years since I've shopped around. I buy things on Amazon because it's easy, they ship quickly, they have very good customer service, the ratings often provide useful information, and they have everything. It scares me a little bit that nobody is even trying to compete with Amazon. When somebody does do a better job at something than Amazon, it seems like Amazon just acquires them (Zappos, for example).
- dynofuz 11y agoI'm working on https://percht.com https://percht.com (still under high development). If we could use your payment and shipping info (entered once on percht) and make the purchase for you on bestbuy, target, etc., would that be convenient enough to take advantage of these deals? Also, electronics deals usually save you $100s not $10s.
- ISL 11y agoIf you can simultaneously match or beat Amazon on price, customer service, reliability, and speed, you'll win. Hard to do, but surely not impossible. I've been surprised the extent to which Amazon's achieved voluntary lock-in for me. Amazon's good-enough in my spot-check price comparisons, and instantaneous with zero marginal-cost shipping, that they're now my default place to search for many products. Find a niche to live in. Taking on Amazon directly is probably trouble.
- TuringTest 11y agoSuch price inflation would be illegal in the European Union, as what can be advertised as a "sale" is regulated by consumer law, and raising prices in the previous weeks is forbidden. Nevertheless, that doesn't mean that it won't be done; the appeal of marketing tricks is too strong for many businesses, because psychology trumps buyer's rational consideration in the aggregate and thus these practices pay for themselves against any possible sanction. (If it didn't work, the art of advertising would look completely different).
- unsettledtck 11y agoIf I'm not mistaken (which I might be), there are only specific and regulated periods of the year where sales are actually approved. So it might be an interesting follow up project to track EU retailers. My qualm with the practices of certain US retailers that we tracked was that they offered the inflated prices on the same days as the sales, so that it wasn't even a matter of raising prices in the weeks leading up.
- dreamdu5t 11y agoArt!? Advertising is blatant manipulation, and fraud as we see in this case. But it's okay - without advertisers nobody would know they exist and the sky will fall. So we need them.
- eplanit 11y agohttps://en.wikipedia.org/wiki/Caveat_emptor https://en.wikipedia.org/wiki/Caveat_emptor
- ISL 11y agoAlternative title: "Some stores offer more consistent prices than others. Some stores have lower average prices than others." We're not entitled to low nor consistent prices. We should expect pricing to fluctuate. Buy at a price where an item is worth as much (or more) to you than the price you pay, and you'll always get your money's worth. Sometimes this means not buying the item. If you want to know which stores practice consistent pricing, or want to know which stores fluctuated low in last year's known-big sale days, this article is useful.
- unsettledtck 11y agoI like your extremely accurate title pitch. And no, we're not entitled to low or consistent prices, but the fact that there is inflation on "sale" days is noteworthy because of the air of false advertising. You're absolutely correct though, shoppers should buy when they feel comfortable because buyer's remorse is meaningless when you're happy with the purchase.
- CWuestefeld 11y agoOur pricing is constantly changing. That's nothing nefarious. We calculate pricing in realtime, because we get frequent updates on the underlying cost from our suppliers - daily in the case of the most important ones. I wrote the pricing engine - it's always calculating the optimal price based on what it's going to cost us to fulfill the order from our suppliers. It also dynamically determines which supplier will be the source on any item based on cost, inventory, and other factors, although this happens less than realtime. So price variation isn't necessarily evidence of anything nefarious, it can be the result of an effort to provide optimal pricing that's sensitive to logistical variations. (Note that this is all in a B2B context, which is maybe a little different from consumers.)
- ams6110 11y agoI dislike the "guilty of inflating prices" theme here. These stores are guilty of no crime. Prices fluctuate all the time, sometimes daily, sometimes hourly. Bottom line, none of these products are essentials to life. The price is either worth it to you as a buyer, or not. Thinking you lost money because it's later available for less is the same sort of faulty logic used by people who think they are "saving" money by buying something on sale (when it's an item they wouldn't ordinarily buy). I had a friend who used to do that all the time. We'd be at a store, and he'd see something on sale that he had no intention of buying before he walked in. He'd buy it, and congratulate himself on "saving" money. I'd point out "no, you just spent $50 that you hadn't planned to spend." He just didn't see it that way.
- unsettledtck 11y agoThis is absolutely true and this study supports your perspective. Guilty is maybe too strong a word, but there is a hint of false advertising. The point of this study was to see how factual the "sales" were that are advertised on Black Friday, Cyber Monday, and Green Monday. We're at a point where people expect to spend their money that those specific days to capitalize on supposedly great deals. I found it interesting to tangibly demonstrate that not only are many of the deals negligible, but that many of the prices were inflated.
- nkrisc 11y agoGenerally I completely agree with you. However the conversation changes when you're talking about price inflation with relation to sales. Yes, prices change all the time. That's fine. However, if the price inflates every time it's "on sale" that's different and is possibly in violation of various regulations or laws.
- ams6110 11y agoThis is the sort of thing I wish that schools taught in "home economics" -- e.g. that sales are often not real bargains, to make buying decisions on the basis of value not strictly "discounts" that are available, the true cost of credit card debt, etc. Instead we learned how to make peanut butter cookies and sew on buttons.
- deelowe 11y agoGuilty? Price inflation? Since when is it a crime to charge more than everyone else for something? This just in, major exchanges are guilty of stock inflation!
- unsettledtck 11y agoI stick by the title. No, it's not a crime, but there's definitely a disingenuous aspect to overcharging for a product.
- pdonis 11y ago"Overcharging" compared to what? If you mean, compared to other stores selling the same product at the same time, consumers have access to the same information that this study did. If it's worth it to them to seek out a better deal, they have the information to do so. If they decide it's not worth it to them, how is that the store's fault? What you are calling "overcharging" is actually just the free market at work. Stores vary prices in order to find out what the best price for them to charge is--"best" meaning the price that maximizes their profit. Customers evaluate prices in order to find out what the best price for them to pay is--"best" meaning the price that maximizes their benefit from the purchase--the value of the item to them minus the cost. But "cost" includeds the time and effort spent price searching, and different people place different values on their time and effort relative to other things. So in a free market, you should expect to see different prices for the same product at the same time at different stores, because they are selling to different customers with different relative priorities. It would be a lot more suspicious to see the same item sold for exactly the same price at every store--that would mean the market is not a free market any more.
- popra 11y agoThey're overcharging today compared to yesterday, so that tomorrow, on the day of the sale event, they can display a bigger discount. And all of that to "incentivize" (read trick) the buyer to spend his money. It has nothing to do with free markets at work. It's simply a shady, dubious practice.
- Gratsby 11y agoJC Penney just went through a complete meltdown because the CEO forced out honest pricing instead of the inflate-prices-and-have-sales that was their SOP for more than a generation. Turns out their customer base really liked the sense of saving money, even though the straight forward pricing was actually saving them more money. This kind of a thing feels unethical - especially to those with an analytical mindset like a lot of programmers. But the proof is in the pudding. People (at least older people in the U.S.) prefer bargain shopping to straightforward pricing. Culturally, there are differences like this around the world. In Japan, they cannot grasp subscription models very well. In many places in the middle east, negotiation is just a part of shopping. In the U.S. we really like sales. In Russia, you get in line and then ask the people around you what it is that you are buying (granted, haven't been there since the 80s).
- balls187 11y ago> This kind of a thing feels unethical - especially to those with an analytical mindset like a lot of programmers. But the proof is in the pudding. People (at least older people in the U.S.) prefer bargain shopping to straightforward pricing. People are irrational with money. These types of unethical practices just capitalize on that fact. Getting more for less, is intrinsic to our species, regardless of being of analytical mind or not.
- unsettledtck 11y agoI think the unethical aspect of arbitrary pricing that goes beyond daily fluctuations really came out in this project. I actually was not aware of the situation with JC Penny, but really fascinating glimpse into their consumer mindset. Probably comparable to Sam's Club, since even I expected a ton of lower prices from them based on membership. I like your cultural analysis!
- smokey_the_bear 11y agoI have the camelcamelcamel chrome extension which shows me a price graph over time for any item on Amazon, and you can set alerts for when prices drop to a certain level. It's very useful, and because of it I already knew that the vast majority of amazon deals are not discounted at all. Amazon items fluctuate a lot, maybe especially for baby items. But I got 30%+ discounts on car seats and strollers by setting a price alert and waiting a few months.
- tyingq 11y agoScrapinghub changed not only their pricing, but their pricing model! Guilty! :) http://scrapinghub.com/pricing http://scrapinghub.com/pricing https://web.archive.org/web/20150302014145/http://scrapinghub.com/pricing https://web.archive.org/web/20150302014145/http://scrapinghu... Edit: Joke was easier to get before they changed the HN title for this submission.
- unsettledtck 11y agohaha, I would say touche except for the fact that we never advertised a sale and then elevated our prices.
- tyingq 11y agoIf you say so. Consider the following, less nefarious theory. - Company puts an item on sale, - After the sale, they follow of pattern of starting cautiously, but increasing the price on a regular basis until they see either a drop off in sales, or competitive pressure - The next sale happens whenever it's scheduled to, based on calendar events, excess inventory, etc.
- spdustin 11y agoI found it extremely misleading to call out Amazon.com when, in fact, most of the Amazon products listed are specified as being sold by "an Amazon seller", not by Amazon.com. This needs to be made MUCH clearer on the site. I'd be curious about the changes for those products sold and fulfilled by Amazon.com versus sold by a third party (on the web site|and fulfilled by) Amazon.com.
- ddebernardy 11y agoWe monitored the products that were sold and fulfilled by Amazon.com, not the stuff from resellers.
- unsettledtck 11y agoThere is a distinction made by those sold and fulfilled by Amazon.com vs sellers on Amazon. Both have inflated prices.
- granitosaurus 11y agoI would like to see some amazon statistics by camelcamelcamel. I've been using it for quite a while now and amazon is most definitely guilty of price increase before sales when it comes both to 1st party and 3rd party sales(the services lets you track both). Amazon is extremely guilty when it comes to pricing ethics and I'm suprised no one called them out yet. Some items for example are never NOT on a -50% deal and browsing their store without addons such as camelcamelcamel is borderline impossible for price savy people like me.
- BrianEatWorld 11y agoI find it odd that the baseline used is after the holiday sales period, in the doldrums of retail.Couldn't the lower price after the holiday season be explained by excess stocks due to insufficient demand relative to forecasts during the holiday season? I think the fact that toys, a product highly prone to fads and trends, are the main culprit would reinforce such an explanation. It would be nice if they had a baseline both before and after or an even average of multiple non-shopping holiday points, rather than one day.