13 ms·

How did Roomba-recorded photos end up on Facebook?

ta8507 4y ago
Reminds of me of a conversation I had with an engineer from a internet-connected thermostat company. When designing their device, they disassembled many of their competitors' thermostats. They found that some had microphones on their circuit boards. Scary stuff.
kache_ 4y ago
The silver lining of is that now I can show this article to anyone who accuses me of being overly paranoid.
- WhackyIdeas 4y ago
  I just wish the companies making these devices had the same level of care with other people’s privacy as I would if I was making these devices. It’s not right. At the very least, companies should have sign an oath to protect their customers and employees - not to abuse them… similar to how health professionals have an oath to do no harm. Is that too much to ask in this world.
OGWhales 4y ago
Importantly: > All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.”
andrewxdiamond 4y ago
Key paragraph for our friends who don’t RTFA > iRobot … confirmed that these images were captured by its Roombas in 2020. All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.” Seems like the real story is that training data was leaked, rather than the attention getting “they’re watching you” narrative the title suggests
- gretch 4y ago
  I read the article and I don’t think your interpretation of the “suggested” narrative is there. The title is “ A Roomba recorded a woman on the toilet. How did screenshots end up on Facebook?”. That’s not really implying a “they’re watching you narrative” - who’s “they” and if it’s global syndicate why did they do something as innocuous as putting it on Facebook? But yes, the real story is that training data (and “real” data) does leak all the time and that most companies don’t take insider risk as seriously as they should.
  - karmakaze 4y ago
    The part that's missing is that it isn't just "a Roomba", it's "a Roomba labelled with “video recording in progress”". Saying 'a Roomba' implies that it could've been done from any Roomba.
    - gretch 4y ago
      The article addresses this head on. Even though those users once consented to share data “It’s not expected that human beings are going to be reviewing the raw footage.” The meat of the article is that what technology and tech companies are doing is divorced from the expectations that we have as a society. It couldn’t have been done from any roomba, but it could happen to almost everyone who didn’t understand the exact ramifications (which we click through several of every year to try to get the vacuum up and running). That’s why a lot of ppl on HN put masking tape over their laptop webcam. Or are you calling those people paranoid?
deleted 4y ago
[deleted]
jdlshore 4y ago
The answer: special robots used during development for training ML image classification. Presumably leaked by human gig workers in Venezuela who were hired to perform image classification. > All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.”
hobbitstan 4y ago
Yeah, I’m not sure I buy their explanation about special development roombas since they offered zero proof. I have an ancient Roomba that’s still going and dread having to replace it one day.
- Someone1234 4y ago
  You could buy a lidar based robot vacuum instead of visible-light camera. The only benefit I know of that visible-light provides is so-called "poop detection" and avoidance, but that's somewhat unreliable anyway. Lidar, in theory, could create a photo-like image, but that resolution costs money and none of these robot vacuums are anywhere near that. Plus they map depth, not texture, so anything it does create is somewhat abstract.
  - operator-name 4y ago
    Apart from the most recent top of the line models, lidar sensors are mounted at the top of the device and cannot detect short items (slippers, cables). They are not much more capable with obsticalr avoidance than the front bumper system on roombas. The few robot vacuums that have only a front lidar sensors perform poorly compared to the camera only or lidar augmented camera in the latest Roomba and Roborocks.
- butlerm 4y ago
  There is no possible way they could prove anything about this in a form adequate for a short article. You either trust that the writers are responsible journalists representing the truth of the matter to the best of their ability or you don't. That goes for the device manufacturer as well. They couldn't possibly prove the fidelity of their statements except on a witness stand under the penalty of perjury. So unless you think they are conducting the type of conspiracy that could see some of them sent to prison, we might just have to trust that they don't defraud the public on a regular basis.
- rocket_surgeron 4y ago
  >Yeah, I’m not sure I buy their explanation about special development roombas since they offered zero proof. It is simple for any person with even basic knowledge of networking to independently come to the conclusion that Roombas are not uploading video streams (or photographs) to the internet. I know the IP (10.0.0.11) and MAC (50:14:79:1E:AB:6B) address of my Roomba and using the Insight Netflow Analyzer for OPNsense I can see how much data it has sent to the internet. In the last six months it has sent approximately 72MB of data outside my network. That's about 600KB per day. It has received much more, presumably firmware downloads. This is consistent with firmware update checks, notification traffic, and me periodically adjusting its schedule remotely. That's just me clicking on some tabs in my router's web UI. Hundreds if not thousands of people globally are constantly reviewing and monitoring Roomba network traffic in fine detail in order to understand and/or reverse engineer it for research and other purposes. So one of three things is happening: 1. All Roombas send photo and video streams to iRobot and they have thus far managed to hide this from the public and the thousands of eyeballs constantly monitoring the network traffic of their products, or 2. A subset of Roombas send photo and video streams to iRobot and they have thus far managed to hide this from the public and the subset of eyeballs monitoring the network traffic of their products 3. These are development devices like they claim. Based on my own experience we can eliminate 1, based on the images accompanying the article option 3 is highly likely.
louison11 4y ago
Clickbait. There is no news here. No privacy was infringed. This was a private dev version.
- radicaldreamer 4y ago
  Even those who opted-into this data collection certainly didn't opt-in to it being posted publicly on Facebook?
- taylorius 4y ago
  Well the picture of the woman on the toilet is in the article, so it's fairly safe to say her privacy has been infringed.
  - autoexec 4y ago
    It was infringed long before it was added to an article.
- kelnos 4y ago
  Pretty sure none of the users consented to photos of them on the toilet being posted on Facebook (and now a news article; wtf was the Tech Review thinking).
  - quantified 4y ago
    Face blurred, no way to see who it is (though I haven't checked for EXIF in the picture, that would be fubar). IMHO and I don't expect anyone else to share it, the picture drives the surveillance aspect home better than word would, though perhaps that makes more of a difference for people who don't ordinarily read Tech Review.
    - InCityDreams 4y ago
      >...and I don't expect anyone else to share it, ...aha!
    - 4y ago
radicaldreamer 4y ago
Scale AI contractors likely leaked these. Regardless of how they came to be posted onto Facebook, it still seems like iRobot's responsibility to keep this data under wraps. We see similar stories all the time, whether it's about companies leaking data that was collected via consent, data collected without consent, or data collected without anyone knowing about it gets leaked. Even Apple has been caught recording via Homepods without consent.
Fission 4y ago
Since Scale.ai provides labeling services to the US Department of Defense, how do they address the issues presented in this article? Have their labelers go through government background checks? Provide labeling software but not the labor?
- generativeai 4y ago
  they hire cleared labelers in st louis. The software likely needs to run on-premises or in government networks
pessimizer 4y ago
These are internet-connected devices with cameras. A firmware update could be sent to record everything you do and stream it on youtube all day, and another later applied to remove any remnant of what happened. Any privacy you have with a device like this comes from either the benevolence, a lack of a profitable opportunity, or a fear of being caught by the company that has root on that device.
tintor 4y ago
Why does a Roomba need a camera looking UP? Why are they labeling furniture in home that Roomba can't possibly reach from the floor?
- cma 4y ago
  Say a couch has shiny metallic legs that mess with the depth estimation. An estimate of where the corners of the couch are could give better estimate of the legs and weight one possibility more than another.
- outworlder 4y ago
  That's for navigation. You want to be able to tell it to 'clean the living room', it needs to know what the living room is(or some of the landmarks). The robots are low on the ground, so tilting the camera up helps. That's not the only approach though. You can look forward (or just use lidar), but this navigation approach seems to be less sensitive to, say, furniture been moved around.
- fragmede 4y ago
  I mean, the Roomba drives itself under the couch and gets stuck on it because of a lack of clearance. If you're working at iRobot and trying to make better Roombas, and need more data as to why your height sensor aren't working to prevent the condition, isn't a camera looking up the obvious way forwards to collect additional data?
cma 4y ago
Misleading title. They should try to have the title give some indication it was a development roomba in a special opt-in data collection program.
jonas21 4y ago
The question I want to ask is: how did Roomba-recorded photos end up in a major publication on the Internet? And the sequence seems to be: 1. iRobot hires people to use special development versions of the Roomba in their homes to collect training data. These are clearly labeled, and the participants are informed that the images are being sent to iRobot for training. This seems fine - if you want to exchange some degree of privacy for money, that should be your right as long as you're clearly informed about it. 2. A contractor posts some of these photos to a private Facebook group used by other contractors on the project. This is obviously bad, but at the same time, it's limited in scope to people who would have had access to these photos or similar ones. 3. The MIT Technology Review gets a hold of these images and decides to publish them on the Internet for everyone to see, just to get more clicks on their article. This feels like the most egregious privacy violation in the sequence.
- nonrandomstring 4y ago
  > A contractor posts some of these photos to a private Facebook group used by other contractors on the project. That's where it went wrong. Everything else seems reasonable for a visual AI training project, well signalled to the participating users and the data securely communicated. Thereafter, the data was mismanaged. There is clearly no such things as a "private" Facebook group. So called "contractors" [1] using a disservice like Facebook to communicate beggars belief. [1] people with the unremarkable skill of being able to spot ordinary household objects and label them - so someone probably had the bright idea of creating a CAPTCHA "Find all the women on toilets".
  - autoexec 4y ago
    > Thereafter, the data was mismanaged. And this is the problem with most of the technology spying on us. Once the data is in someone else's hands, there is nothing you can do to prevent it from being "mismanaged". It can exist forever and you aren't allowed to know who has it or what they're doing with it. Bugging your own home with any device designed to spy on you is just a terrible idea and I'm amazed at how many people are oblivious to the harm they risk bringing on themselves and everyone else around them.
blutack 4y ago
Valetudo [0] supports local only operation of various supported robot vacuums. Even apart from the privacy stuff, the fast local web interface and open standards integration support (mqtt, homeassistant etc) are brilliant. 0: http://valetudo.cloud/ http://valetudo.cloud/
- devadvance 4y ago
  Highly encourage folks to take a look at Valetudo. I recently installed it on a Dreame Z10 Pro and it works well. Even without various integrations, I added a DNS record for it and can happily go to http://vacuum/ http://vacuum/ to control it locally.
- hsbauauvhabzb 4y ago
  As someone trying to figure out how to do this with mine, thanks for saving me the headache of finding one.
bfeynman 4y ago
Always knew Scale AI was complete BS and overvalued, the lack of controls and oversight is embarrassing, I can't believe they have govt contracts.
- generativeai 4y ago
  Their gov business is questionable for long-term sustainment. Labeling services are akin to transcription services provided by others like Leidos... its not a technology business. Got contracts through political connections... there's a major turnover in their federal team...
Maursault 4y ago
Hell with Roomba. Vacuuming floors is nothing. I want a robot that cleans and sanitizes toilets and bathrooms, another that does dishes and the kitchen sink, and one for the car.
deleted 4y ago
[deleted]
autoexec 4y ago
Roomba is the company that planned on using their vacuum cleaners to map people's houses so they could sell that data to third parties. https://nypost.com/2017/07/25/roomba-maker-wants-to-sell-your-homes-floor-plan/ https://nypost.com/2017/07/25/roomba-maker-wants-to-sell-you... Considering that this is how poorly they protect the sensitive data their devices collect (even those used for development purposes) I guess it's a good thing that public backlash over their spying plans forced them to reconsider.
- hobbitstan 4y ago
  There was a huge backlash to this. There was the potential for thieves to find out the layout of someone’s home and when it was unoccupied. Add cameras to that and you have a disaster waiting to happen.
  - gruez 4y ago
    >There was the potential for thieves to find out the layout of someone’s home and when it was unoccupied. Add cameras to that and you have a disaster waiting to happen. This feels like pearl clutching for no reason. It's not hard to find out the layout of a house. In a given neighborhood there are probably dozens of houses for sale, which would have their floor plans publicly available as part of their listing. At the same time, there is a high chance that multiple homes in a given neighborhood share a floor plan. Therefore the layout of any given house isn't exactly some sort of a secret. As for figuring out whether a house is occupied, there are far easier ways. Any internet connected laptop/phone has built-in cameras and microphones, which also allow you do determine occupancy. Of course, none of this matters. The typical robber isn't going to do some ocean's 11 heist shit where they're scoping out your house's layout in advance and hacking into megacorp's servers so they can learn your daily routine. They're far more likely to make an educated guess based on whether your car is parked and whether the lights are on. If they're really prepared they'll maybe drive through your neighborhood several times a day to get a better sense of occupancy.
    - hobbitstan 4y ago
generativeai 4y ago
While Scale is very good at marketing, I wonder if companies using scale ai genuinely understand how their data is being labeled. They have been completely obfuscating that they primarily do crowdsourcing through https://remotasks.com/ https://remotasks.com/. The brand of remotasks is entirely different than scale ai for apparent reasons. I don't think major companies would opt in for crowdsourcing if they knew it. There are plenty of managed labeling service providers who provide a guarantee that the labelers come to a facility without phones to label data. There's a reason why they exist.
- aj0strow 4y ago
  For what it's worth, the company offers pricing tiers for secure facility vs crowdsourcing and customers get to choose.
operator-name 4y ago
> Milagros Miceli, a sociologist and computer scientist who has been interviewing distributed workers contracted by data annotation companies for years. Miceli has spoken to multiple labelers who have seen similar images, taken from the same low vantage points and sometimes showing people in various stages of undress. > The data labelers found this work “really uncomfortable,” she adds. This is an interesting point - the article seems to present that this was not done out of malice (as the woman's face was pre obscured). > Labelers discussed Project IO [another assignment by Scale] in Facebook, Discord, and other groups that they had set up to share advice on handling delayed payments, talk about the best-paying assignments, or request assistance in labeling tricky objects. It's clearly against policy, > But such actions are nearly impossible to police on crowdsourcing platforms. > When I ask Kevin Guo, the CEO of Hive, a Scale competitor that also depends on contract workers, if he is aware of data labelers sharing content on social media, he is blunt. “These are distributed workers,” he says. “You have to assume that people … ask each other for help. The policy always says that you’re not supposed to, but it’s very hard to control.” I'm honestly not that suprised that something like this happened, where similar things happen for mturk. 1. Society demands this kind of automation 2. Companies reacting to this demand have to hire humans to perform manual labelling 3. Humans that perform labelling don't always follow the rules and policies in place 4. Data leaks occur 5. Article like this are written 6. Demands for automation don't really change (repeat)
hedora 4y ago
The roombas in the article were testing units, and people here seem to think this is just available for dev hardware. However it has been available as part of the iRobot beta program for a long time: https://www.reddit.com/r/roomba/comments/qrs5e5/roomba_beta_camera_and_controls/ https://www.reddit.com/r/roomba/comments/qrs5e5/roomba_beta_... and, of course, iRobot is in the middle of being acquired by Amazon, who have a long history of giving police, access to customers' camera feeds without proper opt-in processes.
- ribosometronome 4y ago
  Roomba's claim is that only users enrolled in/with special units were part of the data being stored & labeled by contractors. The sort of live-view functionality described in that post doesn't necessarily require Roomba to store/transmit video/mean it was part of the set of images leaked by ScaleAI contractors.
  - hedora 4y ago
    Yeah, but owners of roombas can do the same thing. So can people with credentials of compromised iRobot accounts. So can Amazon, post acquisition. People that purchased these devices weren't signing up for a system with such terrible privacy implications.
hsbauauvhabzb 4y ago
Is there any verification that there actually was consent? E.g. has anyone come forward with evidence that there really were devices which did this? Even then, who made the consent? Did they verify the person who did legally had the ability to do so? What happens if the person on the toilet is a child? I hate lawyers and I’m not one myself but even I can tell you this is a bad fucking idea.
qwerty456127 4y ago
> hey were the sorts of scenes that internet-connected devices regularly capture and send back to the cloud Most of the home devices like vacuum robots really have no valid reason to connect to any cloud let alone send any pictures there. Such a robot can run all the necessary code [ran in the cloud normally] on itself. If only the consumers would not be so naive to accept the cloud bullshit for a norm.
- aglavine 4y ago
  Real data of robots getting stuck is a valid reason to improve performance and avoid common real case scenarios impossible to replicate in test cases.
  - qwerty456127 4y ago
    It can make sense to upload volunteer's data (also let them view it before uploading), yet this is not a valid reason to make a device refuse function without the cloud.
simonebrunozzi 4y ago
Let's all remember that iRobot, the company behind Roomba, is now owned by Amazon.com, and not everyone likes it. [0] [0]: https://www.theatlantic.com/ideas/archive/2022/08/amazon-roomba-irobot-acquisition-monopoly/671145/ https://www.theatlantic.com/ideas/archive/2022/08/amazon-roo...

Reminds of me of a conversation I had with an engineer from a internet-connected thermostat company. When designing their device, they disassembled many of their competitors' thermostats. They found that some had microphones on their circuit boards. Scary stuff.

kache_ 4y ago

The silver lining of is that now I can show this article to anyone who accuses me of being overly paranoid.

WhackyIdeas 4y ago
I just wish the companies making these devices had the same level of care with other people’s privacy as I would if I was making these devices. It’s not right. At the very least, companies should have sign an oath to protect their customers and employees - not to abuse them… similar to how health professionals have an oath to do no harm. Is that too much to ask in this world.

OGWhales 4y ago

Importantly: > All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.”

andrewxdiamond 4y ago

Key paragraph for our friends who don’t RTFA > iRobot … confirmed that these images were captured by its Roombas in 2020. All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.” Seems like the real story is that training data was leaked, rather than the attention getting “they’re watching you” narrative the title suggests

gretch 4y ago
I read the article and I don’t think your interpretation of the “suggested” narrative is there. The title is “ A Roomba recorded a woman on the toilet. How did screenshots end up on Facebook?”. That’s not really implying a “they’re watching you narrative” - who’s “they” and if it’s global syndicate why did they do something as innocuous as putting it on Facebook? But yes, the real story is that training data (and “real” data) does leak all the time and that most companies don’t take insider risk as seriously as they should.
- karmakaze 4y ago
  The part that's missing is that it isn't just "a Roomba", it's "a Roomba labelled with “video recording in progress”". Saying 'a Roomba' implies that it could've been done from any Roomba.
  - gretch 4y ago
    The article addresses this head on. Even though those users once consented to share data “It’s not expected that human beings are going to be reviewing the raw footage.” The meat of the article is that what technology and tech companies are doing is divorced from the expectations that we have as a society. It couldn’t have been done from any roomba, but it could happen to almost everyone who didn’t understand the exact ramifications (which we click through several of every year to try to get the vacuum up and running). That’s why a lot of ppl on HN put masking tape over their laptop webcam. Or are you calling those people paranoid?

deleted 4y ago

[deleted]

jdlshore 4y ago

The answer: special robots used during development for training ML image classification. Presumably leaked by human gig workers in Venezuela who were hired to perform image classification. > All of them came from “special development robots with hardware and software modifications that are not and never were present on iRobot consumer products for purchase,” the company said in a statement. They were given to “paid collectors and employees” who signed written agreements acknowledging that they were sending data streams, including video, back to the company for training purposes. According to iRobot, the devices were labeled with a bright green sticker that read “video recording in progress,” and it was up to those paid data collectors to “remove anything they deem sensitive from any space the robot operates in, including children.”

hobbitstan 4y ago

Yeah, I’m not sure I buy their explanation about special development roombas since they offered zero proof. I have an ancient Roomba that’s still going and dread having to replace it one day.

Someone1234 4y ago
You could buy a lidar based robot vacuum instead of visible-light camera. The only benefit I know of that visible-light provides is so-called "poop detection" and avoidance, but that's somewhat unreliable anyway. Lidar, in theory, could create a photo-like image, but that resolution costs money and none of these robot vacuums are anywhere near that. Plus they map depth, not texture, so anything it does create is somewhat abstract.
- operator-name 4y ago
  Apart from the most recent top of the line models, lidar sensors are mounted at the top of the device and cannot detect short items (slippers, cables). They are not much more capable with obsticalr avoidance than the front bumper system on roombas. The few robot vacuums that have only a front lidar sensors perform poorly compared to the camera only or lidar augmented camera in the latest Roomba and Roborocks.
butlerm 4y ago
There is no possible way they could prove anything about this in a form adequate for a short article. You either trust that the writers are responsible journalists representing the truth of the matter to the best of their ability or you don't. That goes for the device manufacturer as well. They couldn't possibly prove the fidelity of their statements except on a witness stand under the penalty of perjury. So unless you think they are conducting the type of conspiracy that could see some of them sent to prison, we might just have to trust that they don't defraud the public on a regular basis.
rocket_surgeron 4y ago
>Yeah, I’m not sure I buy their explanation about special development roombas since they offered zero proof. It is simple for any person with even basic knowledge of networking to independently come to the conclusion that Roombas are not uploading video streams (or photographs) to the internet. I know the IP (10.0.0.11) and MAC (50:14:79:1E:AB:6B) address of my Roomba and using the Insight Netflow Analyzer for OPNsense I can see how much data it has sent to the internet. In the last six months it has sent approximately 72MB of data outside my network. That's about 600KB per day. It has received much more, presumably firmware downloads. This is consistent with firmware update checks, notification traffic, and me periodically adjusting its schedule remotely. That's just me clicking on some tabs in my router's web UI. Hundreds if not thousands of people globally are constantly reviewing and monitoring Roomba network traffic in fine detail in order to understand and/or reverse engineer it for research and other purposes. So one of three things is happening: 1. All Roombas send photo and video streams to iRobot and they have thus far managed to hide this from the public and the thousands of eyeballs constantly monitoring the network traffic of their products, or 2. A subset of Roombas send photo and video streams to iRobot and they have thus far managed to hide this from the public and the subset of eyeballs monitoring the network traffic of their products 3. These are development devices like they claim. Based on my own experience we can eliminate 1, based on the images accompanying the article option 3 is highly likely.

louison11 4y ago

Clickbait. There is no news here. No privacy was infringed. This was a private dev version.

radicaldreamer 4y ago
Even those who opted-into this data collection certainly didn't opt-in to it being posted publicly on Facebook?
taylorius 4y ago
Well the picture of the woman on the toilet is in the article, so it's fairly safe to say her privacy has been infringed.
- autoexec 4y ago
  It was infringed long before it was added to an article.
kelnos 4y ago
Pretty sure none of the users consented to photos of them on the toilet being posted on Facebook (and now a news article; wtf was the Tech Review thinking).
- quantified 4y ago
  Face blurred, no way to see who it is (though I haven't checked for EXIF in the picture, that would be fubar). IMHO and I don't expect anyone else to share it, the picture drives the surveillance aspect home better than word would, though perhaps that makes more of a difference for people who don't ordinarily read Tech Review.
  - InCityDreams 4y ago
    >...and I don't expect anyone else to share it, ...aha!
  - 4y ago

radicaldreamer 4y ago

Scale AI contractors likely leaked these. Regardless of how they came to be posted onto Facebook, it still seems like iRobot's responsibility to keep this data under wraps. We see similar stories all the time, whether it's about companies leaking data that was collected via consent, data collected without consent, or data collected without anyone knowing about it gets leaked. Even Apple has been caught recording via Homepods without consent.

Fission 4y ago

Since Scale.ai provides labeling services to the US Department of Defense, how do they address the issues presented in this article? Have their labelers go through government background checks? Provide labeling software but not the labor?

generativeai 4y ago
they hire cleared labelers in st louis. The software likely needs to run on-premises or in government networks

pessimizer 4y ago

These are internet-connected devices with cameras. A firmware update could be sent to record everything you do and stream it on youtube all day, and another later applied to remove any remnant of what happened. Any privacy you have with a device like this comes from either the benevolence, a lack of a profitable opportunity, or a fear of being caught by the company that has root on that device.

tintor 4y ago

Why does a Roomba need a camera looking UP? Why are they labeling furniture in home that Roomba can't possibly reach from the floor?

cma 4y ago
Say a couch has shiny metallic legs that mess with the depth estimation. An estimate of where the corners of the couch are could give better estimate of the legs and weight one possibility more than another.
outworlder 4y ago
That's for navigation. You want to be able to tell it to 'clean the living room', it needs to know what the living room is(or some of the landmarks). The robots are low on the ground, so tilting the camera up helps. That's not the only approach though. You can look forward (or just use lidar), but this navigation approach seems to be less sensitive to, say, furniture been moved around.
fragmede 4y ago
I mean, the Roomba drives itself under the couch and gets stuck on it because of a lack of clearance. If you're working at iRobot and trying to make better Roombas, and need more data as to why your height sensor aren't working to prevent the condition, isn't a camera looking up the obvious way forwards to collect additional data?

cma 4y ago

Misleading title. They should try to have the title give some indication it was a development roomba in a special opt-in data collection program.

jonas21 4y ago

The question I want to ask is: how did Roomba-recorded photos end up in a major publication on the Internet? And the sequence seems to be: 1. iRobot hires people to use special development versions of the Roomba in their homes to collect training data. These are clearly labeled, and the participants are informed that the images are being sent to iRobot for training. This seems fine - if you want to exchange some degree of privacy for money, that should be your right as long as you're clearly informed about it. 2. A contractor posts some of these photos to a private Facebook group used by other contractors on the project. This is obviously bad, but at the same time, it's limited in scope to people who would have had access to these photos or similar ones. 3. The MIT Technology Review gets a hold of these images and decides to publish them on the Internet for everyone to see, just to get more clicks on their article. This feels like the most egregious privacy violation in the sequence.

nonrandomstring 4y ago
> A contractor posts some of these photos to a private Facebook group used by other contractors on the project. That's where it went wrong. Everything else seems reasonable for a visual AI training project, well signalled to the participating users and the data securely communicated. Thereafter, the data was mismanaged. There is clearly no such things as a "private" Facebook group. So called "contractors" [1] using a disservice like Facebook to communicate beggars belief. [1] people with the unremarkable skill of being able to spot ordinary household objects and label them - so someone probably had the bright idea of creating a CAPTCHA "Find all the women on toilets".
- autoexec 4y ago
  > Thereafter, the data was mismanaged. And this is the problem with most of the technology spying on us. Once the data is in someone else's hands, there is nothing you can do to prevent it from being "mismanaged". It can exist forever and you aren't allowed to know who has it or what they're doing with it. Bugging your own home with any device designed to spy on you is just a terrible idea and I'm amazed at how many people are oblivious to the harm they risk bringing on themselves and everyone else around them.

blutack 4y ago

Valetudo [0] supports local only operation of various supported robot vacuums. Even apart from the privacy stuff, the fast local web interface and open standards integration support (mqtt, homeassistant etc) are brilliant. 0: http://valetudo.cloud/ http://valetudo.cloud/

devadvance 4y ago
Highly encourage folks to take a look at Valetudo. I recently installed it on a Dreame Z10 Pro and it works well. Even without various integrations, I added a DNS record for it and can happily go to http://vacuum/ http://vacuum/ to control it locally.
hsbauauvhabzb 4y ago
As someone trying to figure out how to do this with mine, thanks for saving me the headache of finding one.

bfeynman 4y ago

Always knew Scale AI was complete BS and overvalued, the lack of controls and oversight is embarrassing, I can't believe they have govt contracts.

generativeai 4y ago
Their gov business is questionable for long-term sustainment. Labeling services are akin to transcription services provided by others like Leidos... its not a technology business. Got contracts through political connections... there's a major turnover in their federal team...

Maursault 4y ago

Hell with Roomba. Vacuuming floors is nothing. I want a robot that cleans and sanitizes toilets and bathrooms, another that does dishes and the kitchen sink, and one for the car.

deleted 4y ago

[deleted]

autoexec 4y ago

Roomba is the company that planned on using their vacuum cleaners to map people's houses so they could sell that data to third parties. https://nypost.com/2017/07/25/roomba-maker-wants-to-sell-your-homes-floor-plan/ https://nypost.com/2017/07/25/roomba-maker-wants-to-sell-you... Considering that this is how poorly they protect the sensitive data their devices collect (even those used for development purposes) I guess it's a good thing that public backlash over their spying plans forced them to reconsider.

hobbitstan 4y ago
There was a huge backlash to this. There was the potential for thieves to find out the layout of someone’s home and when it was unoccupied. Add cameras to that and you have a disaster waiting to happen.
- gruez 4y ago
  >There was the potential for thieves to find out the layout of someone’s home and when it was unoccupied. Add cameras to that and you have a disaster waiting to happen. This feels like pearl clutching for no reason. It's not hard to find out the layout of a house. In a given neighborhood there are probably dozens of houses for sale, which would have their floor plans publicly available as part of their listing. At the same time, there is a high chance that multiple homes in a given neighborhood share a floor plan. Therefore the layout of any given house isn't exactly some sort of a secret. As for figuring out whether a house is occupied, there are far easier ways. Any internet connected laptop/phone has built-in cameras and microphones, which also allow you do determine occupancy. Of course, none of this matters. The typical robber isn't going to do some ocean's 11 heist shit where they're scoping out your house's layout in advance and hacking into megacorp's servers so they can learn your daily routine. They're far more likely to make an educated guess based on whether your car is parked and whether the lights are on. If they're really prepared they'll maybe drive through your neighborhood several times a day to get a better sense of occupancy.
  - hobbitstan 4y ago

generativeai 4y ago

While Scale is very good at marketing, I wonder if companies using scale ai genuinely understand how their data is being labeled. They have been completely obfuscating that they primarily do crowdsourcing through https://remotasks.com/ https://remotasks.com/. The brand of remotasks is entirely different than scale ai for apparent reasons. I don't think major companies would opt in for crowdsourcing if they knew it. There are plenty of managed labeling service providers who provide a guarantee that the labelers come to a facility without phones to label data. There's a reason why they exist.

aj0strow 4y ago
For what it's worth, the company offers pricing tiers for secure facility vs crowdsourcing and customers get to choose.

operator-name 4y ago

> Milagros Miceli, a sociologist and computer scientist who has been interviewing distributed workers contracted by data annotation companies for years. Miceli has spoken to multiple labelers who have seen similar images, taken from the same low vantage points and sometimes showing people in various stages of undress. > The data labelers found this work “really uncomfortable,” she adds. This is an interesting point - the article seems to present that this was not done out of malice (as the woman's face was pre obscured). > Labelers discussed Project IO [another assignment by Scale] in Facebook, Discord, and other groups that they had set up to share advice on handling delayed payments, talk about the best-paying assignments, or request assistance in labeling tricky objects. It's clearly against policy, > But such actions are nearly impossible to police on crowdsourcing platforms. > When I ask Kevin Guo, the CEO of Hive, a Scale competitor that also depends on contract workers, if he is aware of data labelers sharing content on social media, he is blunt. “These are distributed workers,” he says. “You have to assume that people … ask each other for help. The policy always says that you’re not supposed to, but it’s very hard to control.” I'm honestly not that suprised that something like this happened, where similar things happen for mturk. 1. Society demands this kind of automation 2. Companies reacting to this demand have to hire humans to perform manual labelling 3. Humans that perform labelling don't always follow the rules and policies in place 4. Data leaks occur 5. Article like this are written 6. Demands for automation don't really change (repeat)

hedora 4y ago

The roombas in the article were testing units, and people here seem to think this is just available for dev hardware. However it has been available as part of the iRobot beta program for a long time: https://www.reddit.com/r/roomba/comments/qrs5e5/roomba_beta_camera_and_controls/ https://www.reddit.com/r/roomba/comments/qrs5e5/roomba_beta_... and, of course, iRobot is in the middle of being acquired by Amazon, who have a long history of giving police, access to customers' camera feeds without proper opt-in processes.

ribosometronome 4y ago
Roomba's claim is that only users enrolled in/with special units were part of the data being stored & labeled by contractors. The sort of live-view functionality described in that post doesn't necessarily require Roomba to store/transmit video/mean it was part of the set of images leaked by ScaleAI contractors.
- hedora 4y ago
  Yeah, but owners of roombas can do the same thing. So can people with credentials of compromised iRobot accounts. So can Amazon, post acquisition. People that purchased these devices weren't signing up for a system with such terrible privacy implications.

hsbauauvhabzb 4y ago

Is there any verification that there actually was consent? E.g. has anyone come forward with evidence that there really were devices which did this? Even then, who made the consent? Did they verify the person who did legally had the ability to do so? What happens if the person on the toilet is a child? I hate lawyers and I’m not one myself but even I can tell you this is a bad fucking idea.

qwerty456127 4y ago

> hey were the sorts of scenes that internet-connected devices regularly capture and send back to the cloud Most of the home devices like vacuum robots really have no valid reason to connect to any cloud let alone send any pictures there. Such a robot can run all the necessary code [ran in the cloud normally] on itself. If only the consumers would not be so naive to accept the cloud bullshit for a norm.

aglavine 4y ago
Real data of robots getting stuck is a valid reason to improve performance and avoid common real case scenarios impossible to replicate in test cases.
- qwerty456127 4y ago
  It can make sense to upload volunteer's data (also let them view it before uploading), yet this is not a valid reason to make a device refuse function without the cloud.

simonebrunozzi 4y ago

Let's all remember that iRobot, the company behind Roomba, is now owned by Amazon.com, and not everyone likes it. [0] [0]: https://www.theatlantic.com/ideas/archive/2022/08/amazon-roomba-irobot-acquisition-monopoly/671145/ https://www.theatlantic.com/ideas/archive/2022/08/amazon-roo...