5 ms·

Amazon's "Just Walk Out" checkout turns out to be 1000 workers watching you shop

66 points by clockworksoul 3y ago

sgerenser 3y ago
Previously discussed: https://news.ycombinator.com/item?id=39908579 https://news.ycombinator.com/item?id=39908579
- passwordoops 3y ago
  With the hype this got at release, the story should be pinned to #1 until further notice
- crtasm 3y ago
  1,000 HN users watching people repost.
preinheimer 3y ago
Securities Fraud? (With apologies to Matt Levine)
rblatz 3y ago
Honestly I think it's cool that Amszaon took that approach. Do things that don't scale to better understand the value/market fit. Then you can work on the hard problem of building the complicated tech to enable it at scale. So often companies jump straight to the hard tech issues without even trying to learn from the real world.
- passwordoops 3y ago
  It is good strategy. Just don't get caught lying about it
  - mewpmewp2 3y ago
    What was the lie? Is there a source for the statement that was a lie?
    - passwordoops 3y ago
      "Automated checkout systems powered by computer vision can significantly reduce checkout times. For instance, Amazon Go stores use computer vision to automatically track items added to a shopping cart and charge customers accordingly as they leave the store" (1) The only way this can be interpreted as "people manually look at the camera and tabulate the price" is disingenuously. And it's one example that I suspect will be scrubbed from the web, but from the moment this was launched, unless you were living under a rock, the message was "AI powers the store." So please stop with the hairsplitting and revisionist history. (1) https://aws.amazon.com/blogs/industries/enhancing-the-retail-experience-the-power-of-computer-vision/ https://aws.amazon.com/blogs/industries/enhancing-the-retail...
      - mewpmewp2 3y ago
        As I understand, they do use computer vision, but the 1000 people are just either reviewing it or labelling the data for computer vision. Which seems like a normal way to me to start with a system like that. You definitely need to review it for a while to be certain of its quality and understand the edge cases which I'm sure there would've been many of. E.g. the way I imagine it to be is that this CV system will attempt to identify the item and confidence that it is that item. If the confidence is too low, it will trigger a signal for human feedback, then human will either confirm or reject.
        passwordoops 3y ago
        Then advertise it as Human-in-the-Loop. At no point did Amazon do this. In fact they did the opposite and led people to believe the tech was much further along than reality. Why the lie?
- Supermancho 3y ago
  I wonder if they used (Amazon) MTurk for this mechanical turk.
morkalork 3y ago
I can't believe they're throwing in the towel right now. Sure, whatever pipeline they were using before was maybe a dead end but with technology like GPT/Gemini vision they couldn't make it work? Did they even try?
- ToucanLoucan 3y ago
  This is how a ton of AI is operating right now. Both to filter training data and also to filter the results before they go back to the users, there is very often human oversight. And by human oversight I mean the same sort of extremely cheap and exploitable overseas labor that Amazon was using for this. Ultimately it might be as simple as the fact that even cheap ass labor plus the infrastructure plus the AI itself still costs something and ultimately that something is more than just paying some friggin cashiers at the bloody store to do the job.
  - mewpmewp2 3y ago
    I don't get it though - what is wrong with trying different things and seeing if they work or not? If they don't, just pick what you had previously. What's the huge deal?
- passwordoops 3y ago
  You forgot the /s. And if you are serious, I have a bridge you might be interested in buying
  - morkalork 3y ago
    Half serious, obviously too expensive to do today but there's huge advancements being made right now. I can see a startup building thing to make "Just Walk Out" stores work and getting bought out in next 5 years.
    - passwordoops 3y ago
      So the bridge isn't the biggest one you'll find, but it gets the job done. Should suit your needs just fine
sbdaman 3y ago
The title here suggests they're watching you shop and entering items into a list as you go (which would be ridiculous). It seems to actually just be low-paid workers spot checking to ensure the "algorithm" got it right?
gwern 3y ago
This is a bad submission because Boing Boing heavily editorializes, and is at least 2 levels removed from the actual reporting at this point - the editorialized Boing Boing excerpts, the editorialized Gizmodo article, The Information article, whatever the sources leaked to TI as a friendly outlet, and then the truth. And there are people on Twitter disputing the insinuations that the human workers were watching you as you shop, which would bespeak a true level of failure, or just reviewing stuff afterwards, which would surprise no one.
- yowzadave 3y ago
  Presumably Amazon ended the program because it was not profitable and had no prospect of becoming profitable soon. If it was already fully automated and only employed humans for QC and/or AI training purposes, then it's hard to see why they would have killed it.
  - gwern 3y ago
    There are many reasons it might not be profitable even if the ML part were working well.
  - yung_steezy 3y ago
    I suppose there is meant to be a tipping point where the humans should be able to step back after the AI has learnt enough from watching them. Perhaps they were struggling to reach that point and the failure rate was remaining stubbornly high. Outsourcing to India/Mechanical Turk is always going to be cheaper than creating a sophisticated AI model. I could see them dropping it if the "learning" had started to stall. This whole thing seemed like a branding exercise anyway. Like how Apple stores are in every major city despite relatively few people buying their products in-store.
adrr 3y ago
All these AI companies have a bunch of people who just do labeling to help train the model. Like saying google has millions of people powering waymo when its just recaptcha having users help with labelling image captures like select all the images with street signs or bicycles .
shmatt 3y ago
These types of systems take a week to train the obvious clean use case, and a decade to be able to perform with people intentionally or unintentionally messing with the system Just don’t act surprised when we “find out” the dash carts are also powered by humans
greenavocado 3y ago
In a tech-crazed era where the boundaries of reality blur with the audacity of innovation, there unfolds a tale so bewildering, it vaults past the edges of conventional disruption into the realm of the utterly surreal. This isn't just another yarn spun from the looms of Silicon Valley's grand dream factories. No, this saga pirouettes on the razor's edge of imagination and insanity, where the most cutting-edge "AI" in retail isn't AI at all—it's the keen eyes and quick wits of Pakistanis, stationed a world away, orchestrating the illusion of machine precision in real-time. Let's zoom in on Lahore, pulsating with life and a fervor for innovation that makes Silicon Valley look like a retirement home for tired tech. Here, in a place that hums with the energy of endless possibility (and perhaps too much caffeine), sits the command center of the world's most "advanced" computer vision system, SeeAll. Only, SeeAll's vision is purely human, powered by a legion of sharp-eyed annotators who scrutinize live feeds from grocery store checkouts, identifying every item with the accuracy and flair only a human can muster. The masterminds behind this grand ruse? VisioTech, a company shrouded in the mystique of technological advancement, promising a checkout experience free from human error, barcodes, and the tedium of waiting. Their secret sauce wasn't algorithmic; it was organic, brainpower fueled by chai and an undying spirit of camaraderie. This narrative, though dripping with the trappings of high-tech, is really an ode to the human element in the digital age. Customers, awash in the glow of seamless transactions, never questioned the "how" of their flawless shopping experience, unaware that the real magic lay in the hands of the PaaS team. This crew, capable of identifying the most obscure products with a glance, operated in a symphony of clicks and keystrokes, a ballet of productivity that no machine could replicate. The facade crumbled when a technical hiccup at an Ohio grocery store exposed the gears behind the magic. What followed was a tumultuous unraveling that laid bare VisioTech's elaborate scheme. Yet, the fallout was anything but predictable. The world, instead of recoiling in outrage, leaned in, captivated by the audacity and sheer inventiveness of it all.
- mewpmewp2 3y ago
  Is this LLM generated?
htrp 3y ago
I remember seeing a couple of blogs that said this last year.
ChrisArchitect 3y ago
[dupe] More discussion: https://news.ycombinator.com/item?id=39908579 https://news.ycombinator.com/item?id=39908579
sgerenser 3y ago
Also reminds me of the revelation that Expensify was mostly using manual transcription vs. "AI" to categorize and interpret receipts/invoices: Expensify sent images with personal data to Mechanical Turkers https://news.ycombinator.com/item?id=15796189 https://news.ycombinator.com/item?id=15796189

66 points by clockworksoul 3y ago

sgerenser 3y ago

Previously discussed: https://news.ycombinator.com/item?id=39908579 https://news.ycombinator.com/item?id=39908579

passwordoops 3y ago
With the hype this got at release, the story should be pinned to #1 until further notice
crtasm 3y ago
1,000 HN users watching people repost.

preinheimer 3y ago

Securities Fraud? (With apologies to Matt Levine)

rblatz 3y ago

Honestly I think it's cool that Amszaon took that approach. Do things that don't scale to better understand the value/market fit. Then you can work on the hard problem of building the complicated tech to enable it at scale. So often companies jump straight to the hard tech issues without even trying to learn from the real world.

passwordoops 3y ago
It is good strategy. Just don't get caught lying about it
- mewpmewp2 3y ago
  What was the lie? Is there a source for the statement that was a lie?
  - passwordoops 3y ago
    "Automated checkout systems powered by computer vision can significantly reduce checkout times. For instance, Amazon Go stores use computer vision to automatically track items added to a shopping cart and charge customers accordingly as they leave the store" (1) The only way this can be interpreted as "people manually look at the camera and tabulate the price" is disingenuously. And it's one example that I suspect will be scrubbed from the web, but from the moment this was launched, unless you were living under a rock, the message was "AI powers the store." So please stop with the hairsplitting and revisionist history. (1) https://aws.amazon.com/blogs/industries/enhancing-the-retail-experience-the-power-of-computer-vision/ https://aws.amazon.com/blogs/industries/enhancing-the-retail...
    - mewpmewp2 3y ago
      As I understand, they do use computer vision, but the 1000 people are just either reviewing it or labelling the data for computer vision. Which seems like a normal way to me to start with a system like that. You definitely need to review it for a while to be certain of its quality and understand the edge cases which I'm sure there would've been many of. E.g. the way I imagine it to be is that this CV system will attempt to identify the item and confidence that it is that item. If the confidence is too low, it will trigger a signal for human feedback, then human will either confirm or reject.
      - passwordoops 3y ago
        Then advertise it as Human-in-the-Loop. At no point did Amazon do this. In fact they did the opposite and led people to believe the tech was much further along than reality. Why the lie?
Supermancho 3y ago
I wonder if they used (Amazon) MTurk for this mechanical turk.

morkalork 3y ago

I can't believe they're throwing in the towel right now. Sure, whatever pipeline they were using before was maybe a dead end but with technology like GPT/Gemini vision they couldn't make it work? Did they even try?

ToucanLoucan 3y ago
This is how a ton of AI is operating right now. Both to filter training data and also to filter the results before they go back to the users, there is very often human oversight. And by human oversight I mean the same sort of extremely cheap and exploitable overseas labor that Amazon was using for this. Ultimately it might be as simple as the fact that even cheap ass labor plus the infrastructure plus the AI itself still costs something and ultimately that something is more than just paying some friggin cashiers at the bloody store to do the job.
- mewpmewp2 3y ago
  I don't get it though - what is wrong with trying different things and seeing if they work or not? If they don't, just pick what you had previously. What's the huge deal?
passwordoops 3y ago
You forgot the /s. And if you are serious, I have a bridge you might be interested in buying
- morkalork 3y ago
  Half serious, obviously too expensive to do today but there's huge advancements being made right now. I can see a startup building thing to make "Just Walk Out" stores work and getting bought out in next 5 years.
  - passwordoops 3y ago
    So the bridge isn't the biggest one you'll find, but it gets the job done. Should suit your needs just fine

sbdaman 3y ago

The title here suggests they're watching you shop and entering items into a list as you go (which would be ridiculous). It seems to actually just be low-paid workers spot checking to ensure the "algorithm" got it right?

gwern 3y ago

This is a bad submission because Boing Boing heavily editorializes, and is at least 2 levels removed from the actual reporting at this point - the editorialized Boing Boing excerpts, the editorialized Gizmodo article, The Information article, whatever the sources leaked to TI as a friendly outlet, and then the truth. And there are people on Twitter disputing the insinuations that the human workers were watching you as you shop, which would bespeak a true level of failure, or just reviewing stuff afterwards, which would surprise no one.

yowzadave 3y ago
Presumably Amazon ended the program because it was not profitable and had no prospect of becoming profitable soon. If it was already fully automated and only employed humans for QC and/or AI training purposes, then it's hard to see why they would have killed it.
- gwern 3y ago
  There are many reasons it might not be profitable even if the ML part were working well.
- yung_steezy 3y ago
  I suppose there is meant to be a tipping point where the humans should be able to step back after the AI has learnt enough from watching them. Perhaps they were struggling to reach that point and the failure rate was remaining stubbornly high. Outsourcing to India/Mechanical Turk is always going to be cheaper than creating a sophisticated AI model. I could see them dropping it if the "learning" had started to stall. This whole thing seemed like a branding exercise anyway. Like how Apple stores are in every major city despite relatively few people buying their products in-store.

adrr 3y ago

All these AI companies have a bunch of people who just do labeling to help train the model. Like saying google has millions of people powering waymo when its just recaptcha having users help with labelling image captures like select all the images with street signs or bicycles .

shmatt 3y ago

These types of systems take a week to train the obvious clean use case, and a decade to be able to perform with people intentionally or unintentionally messing with the system Just don’t act surprised when we “find out” the dash carts are also powered by humans

greenavocado 3y ago

In a tech-crazed era where the boundaries of reality blur with the audacity of innovation, there unfolds a tale so bewildering, it vaults past the edges of conventional disruption into the realm of the utterly surreal. This isn't just another yarn spun from the looms of Silicon Valley's grand dream factories. No, this saga pirouettes on the razor's edge of imagination and insanity, where the most cutting-edge "AI" in retail isn't AI at all—it's the keen eyes and quick wits of Pakistanis, stationed a world away, orchestrating the illusion of machine precision in real-time. Let's zoom in on Lahore, pulsating with life and a fervor for innovation that makes Silicon Valley look like a retirement home for tired tech. Here, in a place that hums with the energy of endless possibility (and perhaps too much caffeine), sits the command center of the world's most "advanced" computer vision system, SeeAll. Only, SeeAll's vision is purely human, powered by a legion of sharp-eyed annotators who scrutinize live feeds from grocery store checkouts, identifying every item with the accuracy and flair only a human can muster. The masterminds behind this grand ruse? VisioTech, a company shrouded in the mystique of technological advancement, promising a checkout experience free from human error, barcodes, and the tedium of waiting. Their secret sauce wasn't algorithmic; it was organic, brainpower fueled by chai and an undying spirit of camaraderie. This narrative, though dripping with the trappings of high-tech, is really an ode to the human element in the digital age. Customers, awash in the glow of seamless transactions, never questioned the "how" of their flawless shopping experience, unaware that the real magic lay in the hands of the PaaS team. This crew, capable of identifying the most obscure products with a glance, operated in a symphony of clicks and keystrokes, a ballet of productivity that no machine could replicate. The facade crumbled when a technical hiccup at an Ohio grocery store exposed the gears behind the magic. What followed was a tumultuous unraveling that laid bare VisioTech's elaborate scheme. Yet, the fallout was anything but predictable. The world, instead of recoiling in outrage, leaned in, captivated by the audacity and sheer inventiveness of it all.

mewpmewp2 3y ago
Is this LLM generated?

htrp 3y ago

I remember seeing a couple of blogs that said this last year.

ChrisArchitect 3y ago

[dupe] More discussion: https://news.ycombinator.com/item?id=39908579 https://news.ycombinator.com/item?id=39908579

sgerenser 3y ago

Also reminds me of the revelation that Expensify was mostly using manual transcription vs. "AI" to categorize and interpret receipts/invoices: Expensify sent images with personal data to Mechanical Turkers https://news.ycombinator.com/item?id=15796189 https://news.ycombinator.com/item?id=15796189