7 ms·

IoT Network Watches You as You Shop – Without Cameras

66 points by adunk 6y ago

PeterisP 6y ago
I think that aspects such as this are going to be the main results of GDPR-like legislation in various places. In many cases, privacy is lost simply as a side effect of some specific desire which can be fulfilled respectfully, but the invasive methods are simpler or cheaper, so in the absence of any regulation that gets chosen - but we can easily do better if we (as the society) choose to.
- musingsole 6y ago
  As computer vision capabilities increase, it's not just simpler and cheaper to use more privacy invasive means. It's often better with greater potential for improvement over time. With automated cars, lidar is amazing in a narrow band of uses while a camera is decent and getting better in a much wider band.
  - Krasnol 6y ago
    GDPR doesn't mean that you can't use cameras at all. I only regulates what you can do with it and as long as your automatic car isn't using and selling face recognition data to Facebook, you'll be ok...
chiefalchemist 6y ago
> "This is why we chose vibration sensors and light sensors. No liability from owning any personally identifiable information." 1) Regardless of what they might think, I would hope they let customers know they were being observed and tracked. 2) Phone location and/or a sales transaction could put an identity to this data. Mind you, they might have resisted temptations, but that's not going to be true across the board. At some point, perhaps already, someone somewhere has crossed that line.
- neilv 6y ago
  Excellent point. I'd also like to highlight this quote: > With only vibration and light, there simply is nothing there to identify individual shoppers. Their approach seems to have some privacy merit, but I'm a little discouraged to see claims that appear to be salespersonship, not engineering nor science. In addition to what you pointed out (and possibly also correlating "anonymized" shopper with cameras at checkout and/or entryways), I'd also be a little curious about what all is picked up by the vibration sensors. Can they sense steps/gait/shoes at all? Any qualities of voice? And, hypothetically, even if the sensors in the current version could really only detect touching of the product, no matter how much signal processing and ML one throws at it, what happens when there's B2B sales incentive to disambiguate movement of individual shoppers, or to link them to identifying info with more accuracy than currently, and quietly upgrading the vibration sensor would enable that?
  - chiefalchemist 6y ago
    Call me paranoid but I've read similar in the past, and I'm reading "The Age of Surveillance Capitalism" now. Some start up will productize this offline B&M tracking, and then some Tech Giant will snatch them up and hoover that data. And so on. https://www.wnycstudios.org/podcasts/otm/segments/living-under-surveillance-capitalism-on-the-media https://www.wnycstudios.org/podcasts/otm/segments/living-und...
lnsru 6y ago
This system is cool, because it is battery powered. On the other hand, decent 3D sensor can also monitor the interaction between product and customer much better. And also track the customer’s movements in the store during whole shopping tour.
- musingsole 6y ago
  Even low-resolution movement tracking built into shopping carts and baskets would give 80% of what would be useful to a store here. They have the brand information when you buy, and paired with movement, it'd seem easy to infer deliberation.
  - lnsru 6y ago
    That’s brilliant idea! Shopping carts with tracking. Energy harvesting from rolling wheels solves battery problem forever. With some beacon technology resolution would be very high.
haram_masala 6y ago
Seems like there might be at least a cottage industry in small wearable devices that would foil this kind of thing. Though, extending this thought experiment a bit, if you asked the average shopper how much they would pay for such a device, I’d imagine the most common answer would be nothing, since the average shopper cannot perceive the cost to them of this surveillance. Which in turn suggests that maybe a better countermeasure would be some way to show consumers what those costs to them are, in estimated dollars.
- lotsofpulp 6y ago
  Almost all people willingly hand over their grocery shopping information for very transparent discounts. I would similarly bet that people will readily exchange their purchase history for discounts/convenience. Even I find it convenient to not have to save every receipt for returning items and just be able to enter my phone number.
  - Krasnol 6y ago
    I know 2 persons who "willingly hand over their grocery shopping information for very transparent discounts". But I live in Germany where a significant amount of people have a healthy relationship to their data and have learned their lesson from history.
    - LargoLasskhyfv 6y ago
      I doubt that. Why else I'm almost always asked: "Sammeln Sie Punkte?" or seeing and hearing others being asked that, and mostly they show their card, or nowadays smartphone with app. Why else https://www.payback.de/ https://www.payback.de/ and countless other things like it would be a thing? Latest is Lidl Insider/Plus. /me: "Abärr isch abäh gaa kainäh Ändieh!" https://www.youtube.com/watch?v=cO2lZrlduZ0 https://www.youtube.com/watch?v=cO2lZrlduZ0
      - Krasnol 6y ago
        You must live in some parallel universe. I rarely see anybody getting out their phone or card after being asked and it surely is not "almost everybody". Also I see more and more shops which stopped asking. Finally.
villgax 6y ago
Can someone here build a radar/sonar based home surveillance that works at any time of day & also maybe aid in capturing/targeting actual robbers faces when someone enters a property? Such a device could easily be camouflaged to avoid detection/destruction as well.
- GrumpyNl 6y ago
  They do something like that https://density.io https://density.io
- antoniuschan99 6y ago
  It’s Espressifs own implementation. I’m assuming it uses wifi signals to detect movement. https://youtu.be/tFxKUzEDSdw https://youtu.be/tFxKUzEDSdw
tyingq 6y ago
My dayjob put something like this in our break areas, ostensibly to see if they had adequate seating or something like that. Urgh.
joosters 6y ago
I'm not sure what the big deal about 'without cameras' is - I would imagine that most areas of a store are already covered by security cameras, so every customer is being recorded already. The extra privacy-stealing step is in the identification of shoppers, and that is being done without cameras, using techniques like device fingerprinting (as the article says, some stores do this via their free wifi)
Jkvngt 6y ago
Yet another reason to avoid retailers now. They generally can’t compete on price, haven’t tried for years. Now they want to monitor me in ever more insidious ways? It’s creepy and this data will all eventually be sold and aggregated by the data brokers.
- musingsole 6y ago
  They're just playing catch-up with Amazon that has been able to monitor all the same behaviors to their hearts' content. If consumers give it up there, why not in person?
  - sedgjh23 6y ago
    Makes sense, good way of framing it!
  - ssss11 6y ago
    From a consumer point of view - if buying something worked fine in the old days without all this tracking that Amazon gets why does this happen now online(and apparently coming to retail)? No one “gave it up” - if you interviewed 1000 regular (non-HN) people I’d be very surprised if more than 10 (1%) knew the extent of modern day tracking. We can’t assume consent when people don’t know what is happening behind the scenes.
- ttraub 6y ago
  According to the article, this particular system doesn't monitor the shopper, but rather the movements of products as they are picked up and examined. They specifically say they don't use cameras.
  - Jkvngt 6y ago
    The slippery slope is real, though. We all know it by now.
    - deleted 6y ago
      [deleted]
nikkwong 6y ago
I'd like to understand how the data harvested from these devices is actually used. So it is essentially showing the busy areas in the store and areas that are not receiving as much traffic. Wouldn't that be also reflected in sales data (i.e. items in low traffic areas have less sales volume?), which is maybe why they're there in the first place...? How is this type of data actionable. Is there a team of middle managers that looks at this type of data and decides to continue rearranging their stores? It seems like the product layouts in my local grocery stores have hardly changed in the last decade, so I think I must be missing the point here.
- bsder 6y ago
  > Is there a team of middle managers that looks at this type of data and decides to continue rearranging their stores? Smart store managers would indeed do this. But they also don't need unreliable electronics to tell them what they know from walking around the store. > It seems like the product layouts in my local grocery stores have hardly changed in the last decade, so I think I must be missing the point here. 1) Grocery stores are optimized to hell and back already since the margins are so terribly thin. Industrial engineers have been over every shelf and aisle several times already. 2) Grocery stores have seem to have reoptimized roughly about 2-3 years ago. It seems like all the grocery chains suddenly remodeled and converged on a new layout.
ttraub 6y ago
I don't see the privacy issue here. They're just measuring physical activity per quadrant. They can use it to study which products and product areas seem to be attracting the most interest. They specifically don't use photography or wifi that follows people around. Although the vibration/light sensors are a somewhat novel idea, in the end a retailer already knows exactly what products are sold, in what quantities, which ones are returned, and (anecdotally at least) which ones spark the most questions from shoppers. It's not long in coming before UHF RFID chips will show the products being carried around the store (or out the back door, etc.). Probably this system would be most useful for determining which products are "hot" and should be moved to an end cap or special display near the entrance. Or they might choose to "bury" the products to force shoppers to walk past tempting tangential items. Retailing is a science, perfected over many decades. The lights, the music, even the aromas, all conspire to influence shoppers in ways that online can't compete with. I wonder if retail in fact will eventually find a formula to outcompete the online folks. (I'm still kicking myself for ordering a Pi Zero W online when I could have driven the same day to the Microcenter and picked one up.)
- zahma 6y ago
  Maybe this particular system doesn’t do it, but Bluetooth beacons can identify you in conjunction with other device identifiers. And those things are accurate to the cm. If I bring my phone out at all, I tend to turn my wifi and Bluetooth off until I need it.
  - ChuckNorris89 6y ago
    >And those things are accurate to the cm. As someone with bluetooth development experience, no they are nowhere nearly that accurate in real life. You can take your tinfoil hat off. Not saying alphabet agencies or military contractors can't build Bluetooth triangulation devices with high accuracy, but most beacons in the retail industry suck major balls not just in specs but more importantly they're almost always installed by contractors who have no idea about RF (antenna position and materials of nearby objects matter a lot) so the data they receive is almost always garbage.
- 6y ago
yarcob 6y ago
Why are marketers so obsessed with collecting data on their customers? It seems so absurd. On the one hand, they come up with ridiculous surveilance tech like this to track everything their customers do in their stores, so they can better understand how to sell crap to their customers. On the other hand these big stores hire the least experienced, cheapest staff that they can find. If they just hired actual sales people and made sure that enough employees were around the store to talk to customers, nobody would need surveillance tech to find out what customers want. But I guess automation is everything, and the goal is to have a store that doesn't depend on their employees. Why bother hiring experienced sales people for every store when you can get away with a handful of marketers that automatically analyze customer data of hundreds of stores...
oytis 6y ago
They seem to be proud of their achievement. Oh no, Adam Dunkels, I loved you so much!

66 points by adunk 6y ago

I think that aspects such as this are going to be the main results of GDPR-like legislation in various places. In many cases, privacy is lost simply as a side effect of some specific desire which can be fulfilled respectfully, but the invasive methods are simpler or cheaper, so in the absence of any regulation that gets chosen - but we can easily do better if we (as the society) choose to.

musingsole 6y ago
As computer vision capabilities increase, it's not just simpler and cheaper to use more privacy invasive means. It's often better with greater potential for improvement over time. With automated cars, lidar is amazing in a narrow band of uses while a camera is decent and getting better in a much wider band.
- Krasnol 6y ago
  GDPR doesn't mean that you can't use cameras at all. I only regulates what you can do with it and as long as your automatic car isn't using and selling face recognition data to Facebook, you'll be ok...

chiefalchemist 6y ago

> "This is why we chose vibration sensors and light sensors. No liability from owning any personally identifiable information." 1) Regardless of what they might think, I would hope they let customers know they were being observed and tracked. 2) Phone location and/or a sales transaction could put an identity to this data. Mind you, they might have resisted temptations, but that's not going to be true across the board. At some point, perhaps already, someone somewhere has crossed that line.

neilv 6y ago
Excellent point. I'd also like to highlight this quote: > With only vibration and light, there simply is nothing there to identify individual shoppers. Their approach seems to have some privacy merit, but I'm a little discouraged to see claims that appear to be salespersonship, not engineering nor science. In addition to what you pointed out (and possibly also correlating "anonymized" shopper with cameras at checkout and/or entryways), I'd also be a little curious about what all is picked up by the vibration sensors. Can they sense steps/gait/shoes at all? Any qualities of voice? And, hypothetically, even if the sensors in the current version could really only detect touching of the product, no matter how much signal processing and ML one throws at it, what happens when there's B2B sales incentive to disambiguate movement of individual shoppers, or to link them to identifying info with more accuracy than currently, and quietly upgrading the vibration sensor would enable that?
- chiefalchemist 6y ago
  Call me paranoid but I've read similar in the past, and I'm reading "The Age of Surveillance Capitalism" now. Some start up will productize this offline B&M tracking, and then some Tech Giant will snatch them up and hoover that data. And so on. https://www.wnycstudios.org/podcasts/otm/segments/living-under-surveillance-capitalism-on-the-media https://www.wnycstudios.org/podcasts/otm/segments/living-und...

lnsru 6y ago

This system is cool, because it is battery powered. On the other hand, decent 3D sensor can also monitor the interaction between product and customer much better. And also track the customer’s movements in the store during whole shopping tour.

musingsole 6y ago
Even low-resolution movement tracking built into shopping carts and baskets would give 80% of what would be useful to a store here. They have the brand information when you buy, and paired with movement, it'd seem easy to infer deliberation.
- lnsru 6y ago
  That’s brilliant idea! Shopping carts with tracking. Energy harvesting from rolling wheels solves battery problem forever. With some beacon technology resolution would be very high.

haram_masala 6y ago

Seems like there might be at least a cottage industry in small wearable devices that would foil this kind of thing. Though, extending this thought experiment a bit, if you asked the average shopper how much they would pay for such a device, I’d imagine the most common answer would be nothing, since the average shopper cannot perceive the cost to them of this surveillance. Which in turn suggests that maybe a better countermeasure would be some way to show consumers what those costs to them are, in estimated dollars.

lotsofpulp 6y ago
Almost all people willingly hand over their grocery shopping information for very transparent discounts. I would similarly bet that people will readily exchange their purchase history for discounts/convenience. Even I find it convenient to not have to save every receipt for returning items and just be able to enter my phone number.
- Krasnol 6y ago
  I know 2 persons who "willingly hand over their grocery shopping information for very transparent discounts". But I live in Germany where a significant amount of people have a healthy relationship to their data and have learned their lesson from history.
  - LargoLasskhyfv 6y ago
    I doubt that. Why else I'm almost always asked: "Sammeln Sie Punkte?" or seeing and hearing others being asked that, and mostly they show their card, or nowadays smartphone with app. Why else https://www.payback.de/ https://www.payback.de/ and countless other things like it would be a thing? Latest is Lidl Insider/Plus. /me: "Abärr isch abäh gaa kainäh Ändieh!" https://www.youtube.com/watch?v=cO2lZrlduZ0 https://www.youtube.com/watch?v=cO2lZrlduZ0
    - Krasnol 6y ago
      You must live in some parallel universe. I rarely see anybody getting out their phone or card after being asked and it surely is not "almost everybody". Also I see more and more shops which stopped asking. Finally.

villgax 6y ago

Can someone here build a radar/sonar based home surveillance that works at any time of day & also maybe aid in capturing/targeting actual robbers faces when someone enters a property? Such a device could easily be camouflaged to avoid detection/destruction as well.

GrumpyNl 6y ago
They do something like that https://density.io https://density.io
antoniuschan99 6y ago
It’s Espressifs own implementation. I’m assuming it uses wifi signals to detect movement. https://youtu.be/tFxKUzEDSdw https://youtu.be/tFxKUzEDSdw

tyingq 6y ago

My dayjob put something like this in our break areas, ostensibly to see if they had adequate seating or something like that. Urgh.

joosters 6y ago

I'm not sure what the big deal about 'without cameras' is - I would imagine that most areas of a store are already covered by security cameras, so every customer is being recorded already. The extra privacy-stealing step is in the identification of shoppers, and that is being done without cameras, using techniques like device fingerprinting (as the article says, some stores do this via their free wifi)

Jkvngt 6y ago

Yet another reason to avoid retailers now. They generally can’t compete on price, haven’t tried for years. Now they want to monitor me in ever more insidious ways? It’s creepy and this data will all eventually be sold and aggregated by the data brokers.

musingsole 6y ago
They're just playing catch-up with Amazon that has been able to monitor all the same behaviors to their hearts' content. If consumers give it up there, why not in person?
- sedgjh23 6y ago
  Makes sense, good way of framing it!
- ssss11 6y ago
  From a consumer point of view - if buying something worked fine in the old days without all this tracking that Amazon gets why does this happen now online(and apparently coming to retail)? No one “gave it up” - if you interviewed 1000 regular (non-HN) people I’d be very surprised if more than 10 (1%) knew the extent of modern day tracking. We can’t assume consent when people don’t know what is happening behind the scenes.
ttraub 6y ago
According to the article, this particular system doesn't monitor the shopper, but rather the movements of products as they are picked up and examined. They specifically say they don't use cameras.
- Jkvngt 6y ago
  The slippery slope is real, though. We all know it by now.
  - deleted 6y ago
    [deleted]

nikkwong 6y ago

I'd like to understand how the data harvested from these devices is actually used. So it is essentially showing the busy areas in the store and areas that are not receiving as much traffic. Wouldn't that be also reflected in sales data (i.e. items in low traffic areas have less sales volume?), which is maybe why they're there in the first place...? How is this type of data actionable. Is there a team of middle managers that looks at this type of data and decides to continue rearranging their stores? It seems like the product layouts in my local grocery stores have hardly changed in the last decade, so I think I must be missing the point here.

bsder 6y ago
> Is there a team of middle managers that looks at this type of data and decides to continue rearranging their stores? Smart store managers would indeed do this. But they also don't need unreliable electronics to tell them what they know from walking around the store. > It seems like the product layouts in my local grocery stores have hardly changed in the last decade, so I think I must be missing the point here. 1) Grocery stores are optimized to hell and back already since the margins are so terribly thin. Industrial engineers have been over every shelf and aisle several times already. 2) Grocery stores have seem to have reoptimized roughly about 2-3 years ago. It seems like all the grocery chains suddenly remodeled and converged on a new layout.

ttraub 6y ago

I don't see the privacy issue here. They're just measuring physical activity per quadrant. They can use it to study which products and product areas seem to be attracting the most interest. They specifically don't use photography or wifi that follows people around. Although the vibration/light sensors are a somewhat novel idea, in the end a retailer already knows exactly what products are sold, in what quantities, which ones are returned, and (anecdotally at least) which ones spark the most questions from shoppers. It's not long in coming before UHF RFID chips will show the products being carried around the store (or out the back door, etc.). Probably this system would be most useful for determining which products are "hot" and should be moved to an end cap or special display near the entrance. Or they might choose to "bury" the products to force shoppers to walk past tempting tangential items. Retailing is a science, perfected over many decades. The lights, the music, even the aromas, all conspire to influence shoppers in ways that online can't compete with. I wonder if retail in fact will eventually find a formula to outcompete the online folks. (I'm still kicking myself for ordering a Pi Zero W online when I could have driven the same day to the Microcenter and picked one up.)

zahma 6y ago
Maybe this particular system doesn’t do it, but Bluetooth beacons can identify you in conjunction with other device identifiers. And those things are accurate to the cm. If I bring my phone out at all, I tend to turn my wifi and Bluetooth off until I need it.
- ChuckNorris89 6y ago
  >And those things are accurate to the cm. As someone with bluetooth development experience, no they are nowhere nearly that accurate in real life. You can take your tinfoil hat off. Not saying alphabet agencies or military contractors can't build Bluetooth triangulation devices with high accuracy, but most beacons in the retail industry suck major balls not just in specs but more importantly they're almost always installed by contractors who have no idea about RF (antenna position and materials of nearby objects matter a lot) so the data they receive is almost always garbage.
6y ago

yarcob 6y ago

Why are marketers so obsessed with collecting data on their customers? It seems so absurd. On the one hand, they come up with ridiculous surveilance tech like this to track everything their customers do in their stores, so they can better understand how to sell crap to their customers. On the other hand these big stores hire the least experienced, cheapest staff that they can find. If they just hired actual sales people and made sure that enough employees were around the store to talk to customers, nobody would need surveillance tech to find out what customers want. But I guess automation is everything, and the goal is to have a store that doesn't depend on their employees. Why bother hiring experienced sales people for every store when you can get away with a handful of marketers that automatically analyze customer data of hundreds of stores...

oytis 6y ago

They seem to be proud of their achievement. Oh no, Adam Dunkels, I loved you so much!