5 ms·

Since the risk of that is 1 in a trillion, a lot of people are quite happy to take that risk.

dathinab 5y ago
It's not 1 in a trillion, it's MUCH higher. We are not speaking about a situation where not a "arbitrary" picture is miss-classifieds. We are speaking about a situation where a innocent picture involving a naked or not fully clothed child is deemed similar to a non innocent picture of a naked or not fully clothed child. Now you might argue that there should not be a picture a a naked or not fully clothed child of any form ever on any phone, but IMHO that is short sighed, discriminating and at best shows you don't know to much about the world and other cultures. Let's list some simple reasons such a think could happen first: - Photos meant for a doctor, or living partner to ask if something is normal or a problem. In many different ways. - Photos of little children bathing or similar a e.g. dad sends to their mom who is currently on a business trip. - etc. Reasons people are less aware of is that not all countries are as stuck up about nakedness especially in the family. So it's totally normal for families that e.g. before or after taking a shower family member independent of age and gender walk through the apartment naked. Similar if you didn't got any shame about the naked body indoctrinated you might totally do thinks like visiting a "naked-beach" with your family (meeting other families and taking advantage of it often being less crowded) and in turn normal innocent beach family pictures contain naked children. And on itself that's not a problem. But with Apples approach stuff like this is like to trigger both systems Apple announced and wrongly label your while family as pedophiles...
- teekert 5y ago
  Where I live some kids even play naked on normal beaches or near lakes or in their backyard, whatever. Up to 2 years old it's very normal. Within families kids and parents see each other naked up until puberty of the kids and the kids don't feel comfortable anymore. If you go to a sauna, everybody is naked. Or what if a kid gets her/his hands on a phone and takes some pics (doesn't even require unlocking the phone) by accident? The US may be one of the most up-tight countries about nakedness in general if you ask me. At the same time it's hyper sexualized and produces people like Nicky Minaj, but I guess there is still some fabric over their most "special" parts so it's ok. But oh god, what if Justin Timberlake rips it off... Pandemonium.
  - Grustaf 5y ago
    Up to 2 years? Hold my beer, in Scandinavia 5 year olds routinely run naked on public beaches. You're not even very shocked if you see nude grownups bathing.
- Grustaf 5y ago
  > Now you might argue that there should not be a picture a a naked or not fully clothed child of any form ever on any phone No I'm definitely not arguing that. I'm not American, where I live you'll sometimes see nude bathers in the city centre, and most definitely nude children on the beach. 1 in a trillion is derived from a dataset of 100 million photos, presumably a representable proportion of these were "similar images" like bathing kids.
  - dathinab 5y ago
    Given the problematic aspects of a "globally" available datasets and given that in many countries people are more stuck up about such thinks I highly doubt that this dataset contained a "representative proportion" given that you are more open about such thinks and take picture (in general). I.e. there can be a massive difference between a probability "over all humans" and a probability "over people of a given culture" as long as either the given culture is in a minority or underrepresented in given data. Given that people normally don't (knowingly) give out their private family photos when they know they culture is seen as "bad" by some people and this picture might be abused I think we can at least assume such culture(s) are underrepresented. Through we can't say how much that changes the probability.
teekert 5y ago
1 in a trillion with a billion devices with 10k pics is not a small chance. But what do we know? It's not like Apple is communicating any numbers anywhere so we can make a reasonable guess as to the number of false positives, and as to whether we may think it is actually worth it.
- Grustaf 5y ago
  It’s not per photo, it’s per photo library. But it’s per year, so on average once every 1000 years there will be a false positive for someone. They are communicating numbers. For example, they tested with 100 million photos and got 3 false positives. They also tested with 100 k normal porn photos and got 0 false positives.
  - kbelder 5y ago
    I think their numbers are completely irrelevant, now that we know the visual hash can be gamed. Since it can be, it will be. Basically, that 1 in a trillion number has an implicit "assuming people aren't cheating", as most mathematical models do. But it's already evident people can cheat this system. I don't know what the odds will end up being, 1 in a trillion or 1 in 100, but they will not be based on statistical analysis. The odds will be based on cultural and social factors... how quickly do Apple reviewers get overwhelmed? How easily can script kiddies use the tools to fake hashes? Are there consequences for false reports? How many people want to get you in trouble?
    - Grustaf 5y ago
      All the other big players have been using similar algorithms for years, in the cloud. Why haven’t they been overwhelmed by false positive attacks?

It's not 1 in a trillion, it's MUCH higher. We are not speaking about a situation where not a "arbitrary" picture is miss-classifieds. We are speaking about a situation where a innocent picture involving a naked or not fully clothed child is deemed similar to a non innocent picture of a naked or not fully clothed child. Now you might argue that there should not be a picture a a naked or not fully clothed child of any form ever on any phone, but IMHO that is short sighed, discriminating and at best shows you don't know to much about the world and other cultures. Let's list some simple reasons such a think could happen first: - Photos meant for a doctor, or living partner to ask if something is normal or a problem. In many different ways. - Photos of little children bathing or similar a e.g. dad sends to their mom who is currently on a business trip. - etc. Reasons people are less aware of is that not all countries are as stuck up about nakedness especially in the family. So it's totally normal for families that e.g. before or after taking a shower family member independent of age and gender walk through the apartment naked. Similar if you didn't got any shame about the naked body indoctrinated you might totally do thinks like visiting a "naked-beach" with your family (meeting other families and taking advantage of it often being less crowded) and in turn normal innocent beach family pictures contain naked children. And on itself that's not a problem. But with Apples approach stuff like this is like to trigger both systems Apple announced and wrongly label your while family as pedophiles...

teekert 5y ago
Where I live some kids even play naked on normal beaches or near lakes or in their backyard, whatever. Up to 2 years old it's very normal. Within families kids and parents see each other naked up until puberty of the kids and the kids don't feel comfortable anymore. If you go to a sauna, everybody is naked. Or what if a kid gets her/his hands on a phone and takes some pics (doesn't even require unlocking the phone) by accident? The US may be one of the most up-tight countries about nakedness in general if you ask me. At the same time it's hyper sexualized and produces people like Nicky Minaj, but I guess there is still some fabric over their most "special" parts so it's ok. But oh god, what if Justin Timberlake rips it off... Pandemonium.
- Grustaf 5y ago
  Up to 2 years? Hold my beer, in Scandinavia 5 year olds routinely run naked on public beaches. You're not even very shocked if you see nude grownups bathing.
Grustaf 5y ago
> Now you might argue that there should not be a picture a a naked or not fully clothed child of any form ever on any phone No I'm definitely not arguing that. I'm not American, where I live you'll sometimes see nude bathers in the city centre, and most definitely nude children on the beach. 1 in a trillion is derived from a dataset of 100 million photos, presumably a representable proportion of these were "similar images" like bathing kids.
- dathinab 5y ago
  Given the problematic aspects of a "globally" available datasets and given that in many countries people are more stuck up about such thinks I highly doubt that this dataset contained a "representative proportion" given that you are more open about such thinks and take picture (in general). I.e. there can be a massive difference between a probability "over all humans" and a probability "over people of a given culture" as long as either the given culture is in a minority or underrepresented in given data. Given that people normally don't (knowingly) give out their private family photos when they know they culture is seen as "bad" by some people and this picture might be abused I think we can at least assume such culture(s) are underrepresented. Through we can't say how much that changes the probability.

teekert 5y ago

1 in a trillion with a billion devices with 10k pics is not a small chance. But what do we know? It's not like Apple is communicating any numbers anywhere so we can make a reasonable guess as to the number of false positives, and as to whether we may think it is actually worth it.

Grustaf 5y ago
It’s not per photo, it’s per photo library. But it’s per year, so on average once every 1000 years there will be a false positive for someone. They are communicating numbers. For example, they tested with 100 million photos and got 3 false positives. They also tested with 100 k normal porn photos and got 0 false positives.
- kbelder 5y ago
  I think their numbers are completely irrelevant, now that we know the visual hash can be gamed. Since it can be, it will be. Basically, that 1 in a trillion number has an implicit "assuming people aren't cheating", as most mathematical models do. But it's already evident people can cheat this system. I don't know what the odds will end up being, 1 in a trillion or 1 in 100, but they will not be based on statistical analysis. The odds will be based on cultural and social factors... how quickly do Apple reviewers get overwhelmed? How easily can script kiddies use the tools to fake hashes? Are there consequences for false reports? How many people want to get you in trouble?
  - Grustaf 5y ago
    All the other big players have been using similar algorithms for years, in the cloud. Why haven’t they been overwhelmed by false positive attacks?