7 ms·

Amazon’s Cashierless Store Is Almost Ready for Prime Time

59 points by kungfudoi 9y ago

mtgx 9y ago
> Employees have tried to fool the technology. One day, three enterprising Amazonians donned bright yellow Pikachu costumes and cruised around grabbing sandwiches, drinks and snacks. The algorithms nailed it, according to a person familiar with the situation, correctly identifying the employees and charging their Amazon accounts, even though they were obscured behind yellow polyester. Okay, so 3 employees tested it? This tech is going to be "tested" in the real world by tens or hundreds of thousands of people in the first few months/year. Something tells me things will go far from smoothly. I've only had an experience with a "semi-cashierless" store, sort of speak, where the cashier would scan your stuff, and then it would give you a ticket, and you'd go to pay with it at a not-so-intuitive machine. Oh, and you also had to scan it again before you left the area at the designated exit-gate. Let me tell you, it felt like both a more frustrating experience than just paying the cashier and it took longer time. The only thing this did is save the cashier 15 seconds per person, while the store paid for a dozen of those machines who knows what. So we'll see if this is indeed a better experience, or it's just meant to save Amazon money on not having to pay for cashiers, while giving customers a more frustrating experience and making them spend more time in the store to register every item (for the same number of purchased items). Apple's Face ID can't even identify your own face 100% of the time. How is Amazon's tech going to identify (from a distance) millions of people that pass through the store and every item they got with 100% accuracy??
- alwaysanon 9y ago
  You need to have an app on your phone and scan a code from the app/phone to enter. My guess is that it is based on location tracking of the phone (via either Bluetooth or WiFi) rather than facial recognition.
  - justonepost 9y ago
    Probably some facial / clothing recognition goes on as well when you scan to keep track of you.
    - ZenoArrow 9y ago
      How about if someone dressed in a niqab, how will those people be identified?
tabeth 9y ago
No mention of people wearing masks or putting them on after the fact or better yet, completely changing clothing? That's the easiest, simplest and cheapest way to beat this.
- adventured 9y ago
  How do you figure? Identification isn't only based on your face. From the article: "Employees have tried to fool the technology. One day, three enterprising Amazonians donned bright yellow Pikachu costumes and cruised around grabbing sandwiches, drinks and snacks. The algorithms nailed it, according to a person familiar with the situation, correctly identifying the employees and charging their Amazon accounts, even though they were obscured behind yellow polyester. " It links to an account on your phone. You're under full surveillance while in the store. To go any further down the premise you're suggesting, you're already getting to traditional methods of walking into a store, stealing merchandise, and then running for it.
  - tabeth 9y ago
    I'm aware you're under surveillance. I'm skeptical that it would be able to maintain the connection the entire time you're in the store. For example, what if you changed clothing? The Pikachu example, I'd argue is even easier than people wearing regular clothing due to the fact that it's unusual.
  - deleted 9y ago
    [deleted]
  - anigbrowl 9y ago
    Pfff. If I could get a waiver of legal liability in advance I can think of multiple ways to get around that. Since there aren't any cashiers whose safety could be threatened I'm heartily in favor of people developing creative hacks for the brief duration that remains before we turn the whole damn world into a capitalist panopticon.
    - mstolpm 9y ago
shpx 9y ago
There's a YC company also working on the same thing, here's their demo from 3 months ago https://www.youtube.com/watch?v=yeS8TJwBAFs https://www.youtube.com/watch?v=yeS8TJwBAFs https://www.standardcognition.com/ https://www.standardcognition.com/
- justonepost 9y ago
  That's pretty awesome. The demo video would have been even better if they embedded the sku/price for the item on the overlay video.
- ramzyo 9y ago
  Nice! It sort of looks like they’re deliberately exaggerating their hand motions when they pick up an item to maybe ensure the camera gets a good look?
sschueller 9y ago
What if I don't have an Amazon account? Can I even enter the store?
- albertgoeswoof 9y ago
  Why would you want to
- justonepost 9y ago
  Very likely no. Much like costco, I suspect.
  - pwinnski 9y ago
    One person in a group must wave around a Costco card for the entire group to enter. I wonder if I would be allowed to grocery shop as I do now, with my adult daughter who doesn't have an Amazon account of her own.
- justinjlynn 9y ago
  There's probably a nice little waiting area where you can eat cucumber sandwiches while your DNA sample is processed for entry. You probably still have to install the app though.
- coldcode 9y ago
  Perhaps they will add roving robot security to keep out non-amazonians.
pwinnski 9y ago
So they've spent an extra year with people who fit the demographics of an Amazon employee trying to trick their systems. That should protect them from people who fit the demographics of an Amazon employee. There still seem to be many of the issues people brought up a year ago, such as how this depends on the presence of a phone with the appropriate app installed, or what happens with groups of people in which only one person has the app (such as with kids). More to the point, I can see how this would be better for Amazon, but I'm not sure it's better for most shoppers. Amazon gets to charge people who graze on produce while in the store, and kids who consume snacks in the store, and can presumably produce video evidence when customers throw a fit. But they can also overcharge me for things and how am I supposed to know that? Check the app in the parking lot to make sure I have everything it charged me for? What recourse do I have if not? What if something wasn't charged as marked on the shelf? I've already paid at that point. I've gone through this enough times with stores high-end and low-end to know it's inevitable for Amazon. The leverage is all Amazon's, and the convenience doesn't offset that for me. I know I'm not alone, but maybe there aren't enough like me to dent their ambitions.
- mnr 9y ago
  I presume you can check your cart via the app before checking out... in terms of overcharging, how do you know when you’re overcharged at the store when the cashier checks you out if you have more than 10 or so items? Do you remember all the prices of each item?
  - pishpash 9y ago
    Yes. People who care remember the prices.
    - rainbowmverse 9y ago
      People with a good enough memory. Unfortunately, those most in need of watching prices are under a tremendous amount of memory-obliterating stress.
    - mstolpm 9y ago
tarboreus 9y ago
You know what I find convenient? Cash. You just take it out of your wallet and hand it over in exchange for goods. No tracking, no receipt emails, no machine learning of my behavior to figure out when my partner is pregnant or whatever, no extra apps, no facial recognition, no 2-4% charge for businesses that only doesn't apply if you're an international monopoly. Oh, my phone is out of batteries. Guess I'll just give up and head home. Oops, looks like a hurricane/forest fire/proletarian uprising has knocked out local infrastructure. (Luckily, this almost never happens anymore!) Man, it would be great if we had a medium of exchange that would work under these adverse conditions. Guess I'll just barter my Harry's razors and other hipster goods to get some formula for my baby!
- alphaalpha101 9y ago
  My grandparents always got cash out of the ATM machine then bought stuff with their cash. I always thought this was weird, but as an adult I wish I could be bothered to do this.
- carlossilva33 9y ago
  I love cash. By far the most convenient payment method for face to face transactions.
- chrisseaton 9y ago
  Cash is somewhat ok until you buy something. It's fine to carry around say £100 in a few notes, but then you buy something and you get back £5 and £10 notes and a handful of change and now it's a big jingly mess of shrapnel. I think contactless credit cards are the best right now. You don't need to hand anything over, get anything back, or enter anything, just wave the card.
  - tarboreus 9y ago
    I personally love $5 and $10 bills. I frequently do exact change when I have a bunch of those. Buying a BLT at the bodega? That's $5.75 with a coke. Here's $6. At the end of the day, I fire my change in a bucket and bring it to a machine once a year to get exchanged. Also, taking out $100 in 2017 is silly, at least if you spend more than $200 a week, which is most non-students here. I just get out the max that the ATM will let me have and I never regret it. "But what if I get mugged?" This is silly. If you actually do get mugged, you want to have a big load of money on you, not $3.29. Happy muggers mean fewer dead programmers.
cardamomo 9y ago
> Engineers are also figuring out which person to charge when a couple goes shopping together. This made me giggle. I've tried to figure out the same thing myself when shopping with a significant other or close friend. I'm having a hard time imagining how this would be cast as an engineering problem.
- dannyw 9y ago
  A “split the bill” function in your receipt?
gwern 9y ago
> Shoppers visiting an Amazon Go store will scan their smartphones upon entering. Cameras and shelf sensors will then work together to figure out which items have been removed and who removed them, the person says; there will be no need for tracking devices, such as radio frequency chips, embedded in the merchandise. Hold on - "shelf sensors"? Wasn't Amazon originally advertising this as being purely camera-based, and that was why everyone was so skeptical it could ever work sufficiently well? 'Shelf sensors' could cover a heck of a lot short of embedding individual RFID sensors...
- Operyl 9y ago
  I wonder if it still is camera based, and just a disconnect with the press' understanding of Amazon's technology/terminology. I consider a camera to still be a sensor, in this fashion. So, a camera pointed at a shelf could still be a "shelf sensor."
jcoffland 9y ago
There's a real danger of someone forcing an Amazon customer back in to the store at hidden gun or knife point.
- gggvvh 9y ago
  I didn’t realise they were deploying this in Somalia. Thanks.
0xbear 9y ago
It’ll be “almost ready” for prime time once it undergoes extensive adversarial “testing” by Seattle bums. Until then it’s a PR exercise.
polyomino 9y ago
Except if you wear red
perseusprime11 9y ago
I still dont see anyone in Amazon Bookstores. Expecting them to close soon. Why is there so much fascination to reinvent physical stores when we are perfectly happy to get stuff delivered to our homes.
- pwinnski 9y ago
  In October 2017, an estimated $486.6 billion in sales took place at retail stores in the U.S. For e-commerce, the total for the entire three-month quarter was $115.3 billion. [0] https://www.census.gov/retail/ https://www.census.gov/retail/
brownbat 9y ago
My pain point isn't checkout, it's wandering around a store and dodging large carts in narrow aisles. Maybe they'll tackle that next. I really wish Amazon would let me put dry and frozen goods in an app, then have kiva robots fetch those while I just browse produce. Like, you could leave the produce section normal, people freak about produce, but make the rest of the store a robot filled warehouse that customers don't even enter.
- harryh 9y ago
  People spend more money when they spend time wandering around the rest of the store.
  - brownbat 9y ago
    Yeah, planet money had a good episode debating whether that's why they put milk way in the back of the store. But people also spend more money when it's easy to do so, and when they aren't frustrated by the process. How many trips to the store aren't even happening because it's an ordeal? Malls are playing the strategy of keeping people in store, while the internet is lowering barriers to the experience. Malls are dying and the internet is winning. There are a lot of reasons, but 'make it easier for customers to give you money' is probably not an awful business model.

59 points by kungfudoi 9y ago

mtgx 9y ago

> Employees have tried to fool the technology. One day, three enterprising Amazonians donned bright yellow Pikachu costumes and cruised around grabbing sandwiches, drinks and snacks. The algorithms nailed it, according to a person familiar with the situation, correctly identifying the employees and charging their Amazon accounts, even though they were obscured behind yellow polyester. Okay, so 3 employees tested it? This tech is going to be "tested" in the real world by tens or hundreds of thousands of people in the first few months/year. Something tells me things will go far from smoothly. I've only had an experience with a "semi-cashierless" store, sort of speak, where the cashier would scan your stuff, and then it would give you a ticket, and you'd go to pay with it at a not-so-intuitive machine. Oh, and you also had to scan it again before you left the area at the designated exit-gate. Let me tell you, it felt like both a more frustrating experience than just paying the cashier and it took longer time. The only thing this did is save the cashier 15 seconds per person, while the store paid for a dozen of those machines who knows what. So we'll see if this is indeed a better experience, or it's just meant to save Amazon money on not having to pay for cashiers, while giving customers a more frustrating experience and making them spend more time in the store to register every item (for the same number of purchased items). Apple's Face ID can't even identify your own face 100% of the time. How is Amazon's tech going to identify (from a distance) millions of people that pass through the store and every item they got with 100% accuracy??

alwaysanon 9y ago
You need to have an app on your phone and scan a code from the app/phone to enter. My guess is that it is based on location tracking of the phone (via either Bluetooth or WiFi) rather than facial recognition.
- justonepost 9y ago
  Probably some facial / clothing recognition goes on as well when you scan to keep track of you.
  - ZenoArrow 9y ago
    How about if someone dressed in a niqab, how will those people be identified?

tabeth 9y ago

No mention of people wearing masks or putting them on after the fact or better yet, completely changing clothing? That's the easiest, simplest and cheapest way to beat this.

adventured 9y ago
How do you figure? Identification isn't only based on your face. From the article: "Employees have tried to fool the technology. One day, three enterprising Amazonians donned bright yellow Pikachu costumes and cruised around grabbing sandwiches, drinks and snacks. The algorithms nailed it, according to a person familiar with the situation, correctly identifying the employees and charging their Amazon accounts, even though they were obscured behind yellow polyester. " It links to an account on your phone. You're under full surveillance while in the store. To go any further down the premise you're suggesting, you're already getting to traditional methods of walking into a store, stealing merchandise, and then running for it.
- tabeth 9y ago
  I'm aware you're under surveillance. I'm skeptical that it would be able to maintain the connection the entire time you're in the store. For example, what if you changed clothing? The Pikachu example, I'd argue is even easier than people wearing regular clothing due to the fact that it's unusual.
- deleted 9y ago
  [deleted]
- anigbrowl 9y ago
  Pfff. If I could get a waiver of legal liability in advance I can think of multiple ways to get around that. Since there aren't any cashiers whose safety could be threatened I'm heartily in favor of people developing creative hacks for the brief duration that remains before we turn the whole damn world into a capitalist panopticon.
  - mstolpm 9y ago

shpx 9y ago

There's a YC company also working on the same thing, here's their demo from 3 months ago https://www.youtube.com/watch?v=yeS8TJwBAFs https://www.youtube.com/watch?v=yeS8TJwBAFs https://www.standardcognition.com/ https://www.standardcognition.com/

justonepost 9y ago
That's pretty awesome. The demo video would have been even better if they embedded the sku/price for the item on the overlay video.
ramzyo 9y ago
Nice! It sort of looks like they’re deliberately exaggerating their hand motions when they pick up an item to maybe ensure the camera gets a good look?

sschueller 9y ago

What if I don't have an Amazon account? Can I even enter the store?

albertgoeswoof 9y ago
Why would you want to
justonepost 9y ago
Very likely no. Much like costco, I suspect.
- pwinnski 9y ago
  One person in a group must wave around a Costco card for the entire group to enter. I wonder if I would be allowed to grocery shop as I do now, with my adult daughter who doesn't have an Amazon account of her own.
justinjlynn 9y ago
There's probably a nice little waiting area where you can eat cucumber sandwiches while your DNA sample is processed for entry. You probably still have to install the app though.
coldcode 9y ago
Perhaps they will add roving robot security to keep out non-amazonians.

pwinnski 9y ago

So they've spent an extra year with people who fit the demographics of an Amazon employee trying to trick their systems. That should protect them from people who fit the demographics of an Amazon employee. There still seem to be many of the issues people brought up a year ago, such as how this depends on the presence of a phone with the appropriate app installed, or what happens with groups of people in which only one person has the app (such as with kids). More to the point, I can see how this would be better for Amazon, but I'm not sure it's better for most shoppers. Amazon gets to charge people who graze on produce while in the store, and kids who consume snacks in the store, and can presumably produce video evidence when customers throw a fit. But they can also overcharge me for things and how am I supposed to know that? Check the app in the parking lot to make sure I have everything it charged me for? What recourse do I have if not? What if something wasn't charged as marked on the shelf? I've already paid at that point. I've gone through this enough times with stores high-end and low-end to know it's inevitable for Amazon. The leverage is all Amazon's, and the convenience doesn't offset that for me. I know I'm not alone, but maybe there aren't enough like me to dent their ambitions.

mnr 9y ago
I presume you can check your cart via the app before checking out... in terms of overcharging, how do you know when you’re overcharged at the store when the cashier checks you out if you have more than 10 or so items? Do you remember all the prices of each item?
- pishpash 9y ago
  Yes. People who care remember the prices.
  - rainbowmverse 9y ago
    People with a good enough memory. Unfortunately, those most in need of watching prices are under a tremendous amount of memory-obliterating stress.
  - mstolpm 9y ago

tarboreus 9y ago

You know what I find convenient? Cash. You just take it out of your wallet and hand it over in exchange for goods. No tracking, no receipt emails, no machine learning of my behavior to figure out when my partner is pregnant or whatever, no extra apps, no facial recognition, no 2-4% charge for businesses that only doesn't apply if you're an international monopoly. Oh, my phone is out of batteries. Guess I'll just give up and head home. Oops, looks like a hurricane/forest fire/proletarian uprising has knocked out local infrastructure. (Luckily, this almost never happens anymore!) Man, it would be great if we had a medium of exchange that would work under these adverse conditions. Guess I'll just barter my Harry's razors and other hipster goods to get some formula for my baby!

alphaalpha101 9y ago
My grandparents always got cash out of the ATM machine then bought stuff with their cash. I always thought this was weird, but as an adult I wish I could be bothered to do this.
carlossilva33 9y ago
I love cash. By far the most convenient payment method for face to face transactions.
chrisseaton 9y ago
Cash is somewhat ok until you buy something. It's fine to carry around say £100 in a few notes, but then you buy something and you get back £5 and £10 notes and a handful of change and now it's a big jingly mess of shrapnel. I think contactless credit cards are the best right now. You don't need to hand anything over, get anything back, or enter anything, just wave the card.
- tarboreus 9y ago
  I personally love $5 and $10 bills. I frequently do exact change when I have a bunch of those. Buying a BLT at the bodega? That's $5.75 with a coke. Here's $6. At the end of the day, I fire my change in a bucket and bring it to a machine once a year to get exchanged. Also, taking out $100 in 2017 is silly, at least if you spend more than $200 a week, which is most non-students here. I just get out the max that the ATM will let me have and I never regret it. "But what if I get mugged?" This is silly. If you actually do get mugged, you want to have a big load of money on you, not $3.29. Happy muggers mean fewer dead programmers.

cardamomo 9y ago

> Engineers are also figuring out which person to charge when a couple goes shopping together. This made me giggle. I've tried to figure out the same thing myself when shopping with a significant other or close friend. I'm having a hard time imagining how this would be cast as an engineering problem.

dannyw 9y ago
A “split the bill” function in your receipt?

gwern 9y ago

> Shoppers visiting an Amazon Go store will scan their smartphones upon entering. Cameras and shelf sensors will then work together to figure out which items have been removed and who removed them, the person says; there will be no need for tracking devices, such as radio frequency chips, embedded in the merchandise. Hold on - "shelf sensors"? Wasn't Amazon originally advertising this as being purely camera-based, and that was why everyone was so skeptical it could ever work sufficiently well? 'Shelf sensors' could cover a heck of a lot short of embedding individual RFID sensors...

Operyl 9y ago
I wonder if it still is camera based, and just a disconnect with the press' understanding of Amazon's technology/terminology. I consider a camera to still be a sensor, in this fashion. So, a camera pointed at a shelf could still be a "shelf sensor."

jcoffland 9y ago

There's a real danger of someone forcing an Amazon customer back in to the store at hidden gun or knife point.

gggvvh 9y ago
I didn’t realise they were deploying this in Somalia. Thanks.

0xbear 9y ago

It’ll be “almost ready” for prime time once it undergoes extensive adversarial “testing” by Seattle bums. Until then it’s a PR exercise.

polyomino 9y ago

Except if you wear red

perseusprime11 9y ago

I still dont see anyone in Amazon Bookstores. Expecting them to close soon. Why is there so much fascination to reinvent physical stores when we are perfectly happy to get stuff delivered to our homes.

pwinnski 9y ago
In October 2017, an estimated $486.6 billion in sales took place at retail stores in the U.S. For e-commerce, the total for the entire three-month quarter was $115.3 billion. [0] https://www.census.gov/retail/ https://www.census.gov/retail/

brownbat 9y ago

My pain point isn't checkout, it's wandering around a store and dodging large carts in narrow aisles. Maybe they'll tackle that next. I really wish Amazon would let me put dry and frozen goods in an app, then have kiva robots fetch those while I just browse produce. Like, you could leave the produce section normal, people freak about produce, but make the rest of the store a robot filled warehouse that customers don't even enter.

harryh 9y ago
People spend more money when they spend time wandering around the rest of the store.
- brownbat 9y ago
  Yeah, planet money had a good episode debating whether that's why they put milk way in the back of the store. But people also spend more money when it's easy to do so, and when they aren't frustrated by the process. How many trips to the store aren't even happening because it's an ordeal? Malls are playing the strategy of keeping people in store, while the internet is lowering barriers to the experience. Malls are dying and the internet is winning. There are a lot of reasons, but 'make it easier for customers to give you money' is probably not an awful business model.