6 ms·
What is really going on at Amazon Fresh?
- PaulHoule 4y agoIf I were planning to use A.I. to do what Amazon Fresh does I'd build a human-powered prototype that collects training data and break the decision making process down into small tasks that can be farmed out to different people or automated. Eventually I'd try to automate as many tasks as I could but a project like that might well reach an asymptote where you can't get rid of the people. It seems like a nightmare to me to send people a receipt later because they could always say I ripped them off and charged them too much, maybe they are right or maybe they are just trying to rip me off. If the transaction is complete in a short amount of time then the window for disputes is less.
- meghan_rain 4y agoRegarding first paragraph, spot on. Regarding second paragraph, note that the store literally has a dozen high res video feeds from all angles that recorded the entire duration of the visit. There is ample of evidence to prove things one way or the other. Customers caught repeatedly filing fake disputes would simply be banned from entering those stores, which could even be enforced using the same hardware (banned person trespassing detected through AI? autocall the cops). I understand this has a slight dystopian/surveillance state note though.
- PaulHoule 4y agoYou're right but I totally wouldn't want to go there. Not sure the police would be in a great to chase after somebody just because an A.I. sent them a bundle of paperwork.
- SargeDebian 4y agoThe police? I'd say the parties are the customer, Amazon and a payment party where the charge was disputed. How would the police get involved?
- PaulHoule 4y agoIf somebody keeps coming back after they’ve been barred.
- browningstreet 4y ago> Amazon’s cashierless technology, used both at Amazon Fresh and Amazon Go stores ...and the recently renovated Whole Foods on Riverside in Sherman Oaks, CA. Used it myself last week. All the bananas come in 6 banana bundles.
- CobrastanJorji 4y agoIt takes hours, therefore it must not be a computer? I know the author has industry experience with ML and scientific computing, so this conclusion is surprising to me. Big data pipelines, especially those that involve a lot of money, especially especially the ones that are messy and potentially unreliable and have large ups and downs in demand, are not necessarily fast. Maybe there was a bunch of sales and there's a queue. Maybe there's a buffer where they check every few hours to see if inventory matches sales. Maybe they manually validate a couple of orders in each batch before releasing the automated results. Maybe the fleet doesn't autoscale and it's overprovisioned at night but underprovisioned when you're actually shopping. Maybe it sometimes handles the work in large batches, and maybe that job periodically fails due to regular bugs and has to be restarted. There are lots of "this works but not instantly and not 100% reliably" answers. "It was a mechanical turk all along" is not the only logical conclusion from the presented evidence. And "usually takes a couple of hours"...isn't very bad? Is it ideal? No. Is it enough to ask "How Has Amazon Come To This?" Also no.
- iepathos 4y agoThis would be a good excuse for a smaller company that doesn't have the resources but amazon is literally the largest cloud provider in the world. Amazon could scale instantly to deal with queue size no problem and at cost since they own the aws services needed to do so. I don't think queue size or under-provisioning makes sense here at all. Validation and some human element checking inventory matches sales makes sense but that's essentially what the author is suggesting is going on here.
- kube-system 4y agoDoes Amazon Fresh lose revenue by sending receipts slowly? On demand compute is likely more valuable to sell.
- x0x0 4y agoPrompt receipt of a receipt is valuable to the customer. It enables them to promptly dispute mistakes, submit expense reports, allows for return of purchases (valuable also to Amazon: imagine the enormous headaches adapting a system to allow for the return of an item your system doesn't currently believe someone bought yet), etc.
- CodeWriter23 4y agoMy wife and I tried to get into our local Amazon Fresh. Sign outside said open Amazon app and scan upon entry. We were ok with that idea. Got to the turnstiles, and the devices there demanded a palm print, no apparent scanners to scan a phone anywhere. We walked out and will never walk back in, especially considering the jackass carting out in the parking lot ridiculed us for refusing to surrender biometric data to give Amazon our money.
- sebular 4y agoNo, you misunderstood the interface. While they certainly encourage you to use the weird biometric scanner it's in no way required. However you do have to have the Amazon app installed on your phone. When you open the app there's an option to reveal a QR code which you scan at the turnstiles.
- rurp 4y agoSounds like Amazon deliberately made a confusing interface to trick people into revealing more biometric data than necessary. I wouldn't say that's any better than just making it a requirement.
- CodeWriter23 4y agoI did not. Had the app open, and QR code on my screen as I had previously done so many times before at Whole Foods. There was literally no device resembling a barcode scanner on the turnstile. PS I also have a ton of experience with barcode printing and reading. As a matter of fact, software I have written has printed over 30M unique QR Codes for track and trace purposes.
- wenc 4y agoThe QR scanner should be on the turnstile itself. It’s that little white thing next to the palm reader. It even has an Amazon app icon. https://www-ocregister-com.cdn.ampproject.org/i/s/www.ocregister.com/wp-content/uploads/2022/06/OCR-L-AMAZON-FRESH-0603-1-1.jpg?w=978 https://www-ocregister-com.cdn.ampproject.org/i/s/www.ocregi...
- alsodumb 4y agoThe author clearly wants to push it as some sort of 'Amazon faking it bigtime' thing. I am sure there are manual reviews when it gets flagged, and I am also sure there are mistakes in receipts (I had an error once out of 20 or so purchases). I have a much simpler explanation of why billing can take anywhere from a few minutes to a few hours: they process the video feed in batches, and maybe even optimizing for cost depending on compute load in AWS. Video processing takes time, and each store probably has hundreds of cameras - if you want to do multi-person multi-object detection at scale on HD feeds from hundreds of cameras at so many stores, the amount of compute needed is probably not trivial. Instead of processing the video everytime after each person leaves the store, it is much more efficient to do it at intervals and process multiple individuals at the same time.
- zactato 4y agoEven if this is true is it a problem? It obviously has limited how Amazon can scale the product, but I don't think its a scandal worthy of a "gate" suffix. It's just a product/engineering issue. I'm sure someone in a meeting said, "We tried an experiment to scale our image recognition technology, but it turns out the problem was more complex than we thought and not worth more engineering investment"
- Reptur 4y agoAmazon fresh spent 2 years building their store in my town and then just decided recently to scrap it before even opening. They took up an entire grocery store for 2 years that could have been running the whole time.
- sacnoradhq 4y agoIt should say "Amazon Fresh (retail stores)" because Amazon Fresh delivery is wildly successful. I can't see the point of having yet another brand other than Whole Foods. Amazon Go doesn't make any money and doesn't scale. Amazon palm pay at Whole Foods doesn't work. There's no way to easy way to opt-out of it or change cards, and took WF employees 30+ minutes to figure out how to remove it.
- kyle-rb 4y agoI was also thinking of Amazon Fresh delivery. It looks like the title of this article is actually "Amazon’s cashierless stores: artificial intelligence or major deception?", so the HN title should probably be updated to that.
- sacnoradhq 4y ago"Amazon’s cashierless stores: Webvan that ran out of gas and decided to use AI cameras instead." :@>
- kritiko 4y agoWhole Foods still has very strange retailing choices (afaik, you can't get standard Heinz Ketchup or Hellman's Mayo at the big one near me - just the organic varieties).
- jonnyreiss 4y agoThe organic varieties are much more in line with what a consumer expects to find in Whole Foods --- this retailing choice makes sense to me.
- kritiko 4y agoObviously this is in line with the history of the brand of a fancy, organic- and artisanal- focused grocery. (Does it have better profit margins than other groceries?) My point was that Amazon could support another retail brand beyond WF.
- JohnFen 4y ago> However, self-checkout machines have already reduced queues to virtually zero in many supermarkets This line surprised me because it isn't even remotely true in my part of the country, except during times when the stores aren't busy. The self-checkout machines almost always have a line and it's often quicker to go to one of the cashiers instead.
- pradn 4y agoPeople are smart - they'll pick a self-checkout line or a cashier depending on which will reduce the effort they have to expend. So, even with varying shopper preferences for which is intrinsically better/worse, it'll average out to roughly the same effort required across all queues.
- sva_ 4y agoAll the introverts lining up in the self-checkout queue.
- pen2l 4y agoNot quite on-topic, but I have to wonder: why go and develop these long-winded solutions when we can come up with obvious better ones? -- order on your phone (which is nice and easy because you can add to your grocery list with a smart home speaker/smartphone list throughout the day) and then it's automatically packaged (in an automated warehouse: https://www.youtube.com/watch?v=ssZ_8cqfBlE https://www.youtube.com/watch?v=ssZ_8cqfBlE ) and delivered? And for folks who drive to work, have them shipped to their place of work. Places of work make accommodations like have parking for electric cars and cafeterias -- having a place for delivery storage strikes me as an obvious perk that has tremendous benefits for both employers and employees as well as the environment.
- ebiester 4y agoThat's just instacart and the whole foods app and the kroger app and countless others, no?
- kristjansson 4y agoCan confirm that one's workplace accepting packages on one's behalf is a big perk. I don't think they'll accept grocceries through, maybe that's the next frontier.
- pdntspa 4y agoI really don't want anyone but myself grabbing fresh fruits and vegetables off the shelf. Bananas, in particular, I need them at a specific point in their ripeness otherwise they go bad on me way too quickly.
- josefresco 4y agoGet over yourself ;-) I use pickup/delivery frequently and the people that pick my fresh veggies/fruit do a decent job. Some of them are my neighbors!
- pdntspa 4y agoUh, no. Pickup/delivery frequently delivers my food cold and unpalatable. I'm not going to trust that these folks know how to pick fresh vegetables correctly.
- mytailorisrich 4y agoIt depends how this works. Use of AI does not imply that everything is processed in real-time while you're walking through the store. Raw or preprocessed footage may be queued for processing at a data centre. If there are issues it is also possible that 'uncertain' AI results might indeed be flagged for human review. I only shopped once at an Amazon Fresh store and the total receipt took less than an hour to show up. Personally I'm not too fussed if it takes a long time, but I'm very interested in how their procedure works if there is an error to my disadvantage.
- andrewmutz 4y agoIn my opinion, using humans in the loop to generate training data in a store like this is perfectly legitimate. If I were building it, I would do it the same way: start out with the whole thing being people watching cameras and labeling training data, and slowly apply software/ML engineering to improve the automation rate. In fact, this approach is the opposite of putting the cart before the horse: you test the market risk before you solve all the technical problems. The right way to find out if customers want this type of experience is to bring it to market and find out.
- deleted 4y ago[deleted]
- lr1970 4y agoAmazon announced in early March that they are closing cashless stores in Seattle, NYC, etc [0]. Seems that technology is not yet mature for profitability. [0] https://www.geekwire.com/2023/amazon-closing-eight-amazon-go-convenience-stores-in-latest-tightening-of-its-physical-retail/ https://www.geekwire.com/2023/amazon-closing-eight-amazon-go...
- WoahNoun 4y agoNot sure about NYC, but the Seattle locations are next to office buildings that are pretty empty compared to when the stores opened.
- ardme 4y agoThe author wrote a book about how AI is all fake and its a bubble so consider the source. I don't think you could get further from the truth right now. Basically the analysis here is 1/10 level.
- iamsomewalrus 4y agoi helped build this! it's not what i do anymore, and i don't want to go into too much detail, but the TL;DR is that, yes, just walk out (JWO) really does use machine learning models to identify which products customers are picking. This video from a year ago goes into more engineering depth than promotional videos: https://www.youtube.com/watch?v=S5t6aYhj6pU https://www.youtube.com/watch?v=S5t6aYhj6pU the non-obvious challenges of working in this space are that you have to deal with real world constraints that you just don't have to think about in 100% cloud based software. hardware goes down! internet connections go out! electricity goes out! how much processing can you do in store vs out of store? that's just the tech in the store. what about humans? humans are now in your programming state moving shit around. in-store associates miss-stock items. kids do kid shit. people don't place things back exactly where they found them. Then you have interesting distributed problems: how do you handle late data? what should be a massively parallel problem is really a graph of interactions that have to be resolved in just the right order so you can generate an accurate receipt. and you know what's crazy? the vast majority of the time it works! exactly how they say it does: with machine learning models. bonkers.
- fnordpiglet 4y agoFrom direct experience in my professional career I know for a certified fact this article is just flat out wrong. Amazon Go is precisely what it says it is. I don’t know what causes the long delays now but that wasn’t always the case. I suspect the fact Amazon go doesn’t make any money and is launched means it basically hollowed out and is on autopilot with two SDE2’s on call 24/7 keeping it alive on life support. Amazon services are always well staffed up to the moment they’re launched, because at Amazon you only get promoted on launching something - not making it better.
- ilyanep 4y agoI don't understand why I'm supposed to be outraged here. As a customer, I get the experience of a cashierless store. I don't really care how that's accomplished. As an investor, I might care that I'm being sold snake oil, but I doubt that I chose to invest in Amazon solely over the Fresh stores. As someone working in the industry, it might certainly affect my perception of what's possible or feasible, but I'm already used to the fact that most of what's being sold as "sophisticated AI" is snake oil. It just feels to me like this article strikes a bizarre tone about it all, not to mention the fact that other comments in this thread have offered up equally plausible explanations.
- steveylang 4y agoIIRC when you add items to your shopping cart, the little display shows what you added? I think it would actually be harder to have a bunch of people watching and recording your items manually, than the actual system they set up with 4 scanners on the sides of your shopping cart reading bar codes.