14 ms·

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal thing

I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. I'm all for protecting children from being abused, but how are they going to filter what is normal and what is abuse without human intervention? And at that point isn't that a vast invasion of privacy to the perfectly innocent? It's different if it's online because that puts it in the public realm or on devices/servers owned by Apple, etc.

cm2187 5y ago
What I don't understand is how is it even constitutional in the US. Isn't it a warrant-less search? It's not like scanning content on a cloud. The cloud owns the device on which you store your data. But you own your mobile.
- thrill 5y ago
  Not if you give permission, which you do by your purchase agreement.
  - BitwiseFool 5y ago
    You're correct, but I would love to see a TOS that gives Apple permission to exercise cruel and unusual punishment for violating their terms. The eighth amendment only applies to the government, after all! /s?
- shuckles 5y ago
  The constitution constrains what the Federal government can do, not what private companies can do.
  - cm2187 5y ago
    Not that I am qualified to hold a legal debate, but as far as I know, if Apple implements this under the pressure from the state, and that it reports what it findings to the state, it is effectively an agent of the state and it should apply.
    - shuckles 5y ago
      As far as I’m aware, the laws which require cloud service operators to scan for CSAM have been on the books for a long time, and none of the usual orgs have raised a constitutional challenge to them. I’m not a legal expert either, though.
      - cm2187 5y ago
        But that's the big difference, cloud providers are scanning the data they hold on their device. Your iphone is not Apple's device. Cloud providers have a liability if they knowingly store illegal content, the data is stored on their property, Apple does not, it is your property. It's like if Ford was going to scan what you store in the trunk of your car.
        shuckles 5y ago
        The headline is misleading. The rumored change only impacts photos your device is uploading to iCloud, and it seems like Apple servers are a necessary part of the protocol.
        lp0_on_fire 5y ago
        Doesn't apple upload all your photos to icloud by default?
        shuckles 5y ago
        It’s been a long time since I’ve set up my phone, so I don’t remember the defaults. If I recall correctly, photo stream (last ~100? photos) was on by default but not the entire photo library.
- jeffdn 5y ago
  The Constitution applies only to the government. A user's relationship with Apple is not in any way governed by the Constitution, just like how Twitter banning people who violate the terms of service is not a violation of the first amendment to the Constitution.
- ksec 5y ago
  Well that is under the false assumption of you own your mobile. Which isn't true for Apple. Not too sure about Android. But everything you do on your Apple Devices, including its Hardware, Software and its Services, works more like Apple lend it to you for a price. And as other mention, only the government and not a private company.
mkoryak 5y ago
I think they are probably just going to match the hashes to known hashes of "bad" images.
- syshum 5y ago
  Plain file hashing for this kind of thing has not been used for years (or even decades), Microsoft came out with Photo DNA along time ago, and I am sure now days they have something ML powered to do it. I am not sure the accuracy of these systems but even if they are pretty accurate there is something off putting about this trend. This very much feels like a Guilty until proven innocent type of program... "if you have nothing to hide you have nothing to fear"... Neither of those normally work out really well.
  - lstamour 5y ago
    > Guilty until proven innocent type of program But this software doesn’t make arrests? A comparison would be police asking for what cellphones were in an area. There should be a lot of due process that happens before anything comes of this. I imagine police won’t look into one photo, they would want to match more than one. I agree though, the scary part might be how to clear your name after an overreaction occurs, similar to the effect swatting has on innocents or other kinds of false accusations. But those would happen even without such a system - we’ve heard about such stories before. “Man held for child porn charges, but it turned out they were innocent,” - it doesn’t take this kind of system to have police look over your phone for evidence and find a different crime than expected. That all said… child porn is really, really terrible and I’m actually in favour of this. As a society, we can decide on what’s acceptable. As long as there’s due process and a presumption of innocence, I’m all in favour of making it harder for criminals to get away with crimes. I do agree though, all these systems might make it harder to catch perpetrators if it pushes them offline. But what other options do we have to stop these sort of crimes when the kids involved can’t or won’t speak up? I suppose the fundamental problem is that having more evidence doesn’t necessarily lead to justice if it paints a false picture.
    - shotta 5y ago
deleted 5y ago
[deleted]
1-6 5y ago
What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.
- vsareto 5y ago
  Fairly sure criminals can't prove the efficient market hypothesis either. You still want to catch the dumb ones.
- mlac 5y ago
  The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. And the way it’s being done (hashes), a collision is highly unlikely. If it does occur it doesn’t mean it’s similar in nature (e.g. innocent picture of own child in bath). The hash isn’t looking at the image content in the sense of “what’s in the picture”, just the bits of the file. So it’s highly, highly unlikely, even if a collision occurs, that the collision would be an image that happens to be another child innocently bathing.
  - gojomo 5y ago
    > but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Are you sure, given how many iPhone takeover/jailbreak bugs regularly exist, including the recent 0-click iMessage bug? Would you like to dare a hacking group, domestic or foreign, to land a single verboten image on your iPhone? > And the way it’s being done (hashes), a collision is highly unlikely. The limited details so far are suggestive that its using perceptual hashes, which are more susceptible to collision-engineering/false-positives than cryptographically-secure hashes.
  - aviraldg 5y ago
    Presumably the verification that these false matches are not problematic is manual? That's not good enough given how our media currently works. Imagine articles published when information about such a check leaks that say "celebrity X's phone checked by police for suspected CSAM." While that is the truth ("suspected") no one cares about that nuance and such a person would get cancelled very quickly, even if there was no evidence of wrongdoing.
temp8964 5y ago
I am sure, presumbly in a sane world, computer scanning is only the early stage of the process, human will be invovled to make final judgements and decisions.
- jenkstom 5y ago
  I think Trump put a definitive end to the "Sane World" theory. We don't live in one.
- AnotherGoodName 5y ago
  I imagine the biggest catch of this will be teens taking inappropriate photos of themselves because teens often do things like this in their awkward years when sexuality and body consciousness kicks in. With this we'll be seeing warrants for those images and a big fuss made on this most suicidal age group who thought they were doing something completely private. This isn't good.
- taway1213sadf 5y ago
  Remember the US No Fly List? https://en.wikipedia.org/wiki/No_Fly_List https://en.wikipedia.org/wiki/No_Fly_List The list ballooned to over a million names, then required an ACLU lawsuit until humans became involved and a review process was initiated. As long as the majority of Americans dont feel burdened by something, dont expect anything logical to happen.
itake 5y ago
> perfectly innocent photos on someone's phone of their own children They are using a library of human verified images to compare the hashes. Probably similar to PhotoDNA[0]. There can be false positives, but AFAIK the algorithms are not trying to identify naked children, but comparing 2 similar image hashes. [0] - https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA
- barbazoo 5y ago
  > They are using a library of human verified images to compare the hashes I didn't get that from the article, could you point to a source on this?
  - judge2020 5y ago
    Images are submitted by over 1400 organizations[0] like the FBI and NCMEC, although the inner-workings of the database aren’t public and I can’t find a source or confirmation of every image actually being human-verified. The closest is how the "Child Victim Identification Program" helps identify victims in these images. 0: https://www.missingkids.org/theissues/csam#:~:text=Electronic%20Service%20Provider%20(ESP)%20Reporting https://www.missingkids.org/theissues/csam#:~:text=Electroni... (ctrl-f "1,400")
    - deleted 5y ago
      [deleted]
    - dylan604 5y ago
      Of all of the discussion, my brain has only keyed in on the child porn aspect. It just now clicked that if a missing kid is in a photo (even if not porn), it could help ID the missing kid. Some runaway at the age of 13 gets photographed in a group of other kids just doing teenager things. So even if not running hashes against known porn, it could still be doing facial recognition. Wonder what kind of implications that would have for people in witness protection or battered spouses and other legitimately "disappeared" people.
Causality1 5y ago
Yeah if you ran the giant box of photos in my mom's closet through an AI you'd totally get hits from photos of baby me being bathed in the sink, baby me running around the back yard naked, etc. This can't end well.
- alkonaut 5y ago
  If this even comes close to doing any “recognition” on local content and isn’t just (as the article says) a simple lookup of known images through hashes, then that would be a massive scandal. So I’m giving them the benefit of the doubt until it’s clarified, as the only confirmed way it works is the image hash lookup. There are words like “neural” and “trained” here, but no clear sign that there is actually any kind of image recognition that would risk false positives other than hash collisions.
JKCalhoun 5y ago
> Like a kid running butt naked around the house... Thought it was buck naked — but that works too. :-)
- BrianB 5y ago
  https://brians.wsu.edu/2016/05/19/butt/ https://brians.wsu.edu/2016/05/19/butt/
  - amanaplanacanal 5y ago
    I would suggest that if it is widely used, it isn't an error, regardless of what Professor Brians thinks.
echelon 5y ago
This system isn't meant for child abuse. It's going to be used in China against enemies of the state. Then it will be used domestically for the same. Apple is a horrific company that you should stop using.
GeekyBear 5y ago
Microsoft implemented this a decade ago. >The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.) https://www.nbcnews.com/technolog/your-cloud-drive-really-private-not-according-fine-print-1c8881731 https://www.nbcnews.com/technolog/your-cloud-drive-really-pr... There was another round of outrage when the Windows 10 TOS extended this to local storage. It also has the use case of scanning for DRM violations. (ripped mp3 files or movies)
- partiallypro 5y ago
  Again, that is not the same as scanning a local machine for it. That is all things that are on Microsoft's own servers or in their search results. Microsoft, Google, Cloudflare all have features like this. It's not the same as scanning local machines.
  - GeekyBear 5y ago
    Again, the Windows 10 TOS extended that to scanning the local machine as well.
    - cududa 5y ago
      When you download an app it compares hashes locally to source material published hash. This is absolutely no different
jandrese 5y ago
Or when an internet troll gets your email address and/or phone number and starts spamming you with child porn pictures, which your phone immediately identifies and notifies authorities.
- elzbardico 5y ago
  A completely new way of swating someone
- wang_li 5y ago
  And to get cleared you’re going to have to let law enforcement clone and inspect the content of your phone.
getpolarized 5y ago
> I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. These are still illegal in the US right now right? It's been a major criticism of these laws that they can hurt people accidentally.
- pseudalopex 5y ago
  I don't think just nudity was illegal ever. But just suspicion can hurt people.
deleted 5y ago
[deleted]
cm2187 5y ago
And you can imagine what's on a dermatologist or urologist phone!
cronix 5y ago
Don't worry, AI is very good at these things and never makes mistakes. It is so advanced it even understands nuance and complex communication mechanisms like sarcasm in text, even if it's only a single sentence to contrast! It will be able to know exactly the context of each image and it will affect only the sickos. We're also working on watching your face through the camera to see your true expression when viewing them. Oh, but if a few innocent people do happen get caught up in it and lives ruined, it's totally justifiable because this type of activity is the absolute worst. And false positives are exceeeeedingly rare. We'll apologize and adjust our algorithms. Sue us? How dare you? We're really trying to wipe out sickos. Are you a sicko or something? You must be. Only sicko's would disagree with protecting children.
- 8458e112e7b2 5y ago
  Yep. And if you dont like it, build your own iPhone.
  - ghego1 5y ago
    And be immediately labelled by society as a CP holder
- mm983 5y ago
  This is painful to read with the arrogant undertone while you don't actually know how CSAM scanning works. It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. https://support.cloudflare.com/hc/en-us/articles/360046106112-Understanding-the-CSAM-Scanning-Tool https://support.cloudflare.com/hc/en-us/articles/36004610611... How about we turn the tables and have the complainers suggest a solution. Because every single time an approach to targeted child porn takedowns has been suggested, such as datacenter raids which would not affect as many people, someone is screaming about their privacy. Child abusers evolve and are very happy if law enforcement doesn't.
  - babyshake 5y ago
    So this means it is checking if you are sharing known CP images? That does seem to be much less invasive and problematic as there is likely no good reason to be sharing these images.
lenkite 5y ago
They need to stop calling it an iPhone. It's a govPhone. It now belongs to the government with "In-built Citizen Scanning" as free civic service. Thank you for serving the nation!
benhurmarcel 5y ago
If the photo of your kid running around the house ends up in a CP database, you messed up before Apple did.
anigbrowl 5y ago
there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things How would that match a hash list of known CP imagery though?
elzbardico 5y ago
Yes, not no to mention the gulf that exist between american culture of what is innocent and acceptable and the rest of the world. If we apply the american paranoid standards to the rest of the world, i am pretty sure we will have plenty of false positives.
jeegsy 5y ago
We really shouldn't be moving the conversation window into the technicalities just yet without discussing if this is even ok or reasonable in the first place.
anupamchugh 5y ago
Apparently, there's a Threshold Secret Sharing (TSI) technology in place to eliminate false positive. Besides, NeuralHash will basically match your photos against a database provided by NCMEC.

cm2187 5y ago

What I don't understand is how is it even constitutional in the US. Isn't it a warrant-less search? It's not like scanning content on a cloud. The cloud owns the device on which you store your data. But you own your mobile.

thrill 5y ago
Not if you give permission, which you do by your purchase agreement.
- BitwiseFool 5y ago
  You're correct, but I would love to see a TOS that gives Apple permission to exercise cruel and unusual punishment for violating their terms. The eighth amendment only applies to the government, after all! /s?
shuckles 5y ago
The constitution constrains what the Federal government can do, not what private companies can do.
- cm2187 5y ago
  Not that I am qualified to hold a legal debate, but as far as I know, if Apple implements this under the pressure from the state, and that it reports what it findings to the state, it is effectively an agent of the state and it should apply.
  - shuckles 5y ago
    As far as I’m aware, the laws which require cloud service operators to scan for CSAM have been on the books for a long time, and none of the usual orgs have raised a constitutional challenge to them. I’m not a legal expert either, though.
    - cm2187 5y ago
      But that's the big difference, cloud providers are scanning the data they hold on their device. Your iphone is not Apple's device. Cloud providers have a liability if they knowingly store illegal content, the data is stored on their property, Apple does not, it is your property. It's like if Ford was going to scan what you store in the trunk of your car.
      - shuckles 5y ago
        The headline is misleading. The rumored change only impacts photos your device is uploading to iCloud, and it seems like Apple servers are a necessary part of the protocol.
        lp0_on_fire 5y ago
        Doesn't apple upload all your photos to icloud by default?
        shuckles 5y ago
        It’s been a long time since I’ve set up my phone, so I don’t remember the defaults. If I recall correctly, photo stream (last ~100? photos) was on by default but not the entire photo library.
jeffdn 5y ago
The Constitution applies only to the government. A user's relationship with Apple is not in any way governed by the Constitution, just like how Twitter banning people who violate the terms of service is not a violation of the first amendment to the Constitution.
ksec 5y ago
Well that is under the false assumption of you own your mobile. Which isn't true for Apple. Not too sure about Android. But everything you do on your Apple Devices, including its Hardware, Software and its Services, works more like Apple lend it to you for a price. And as other mention, only the government and not a private company.

mkoryak 5y ago

I think they are probably just going to match the hashes to known hashes of "bad" images.

syshum 5y ago
Plain file hashing for this kind of thing has not been used for years (or even decades), Microsoft came out with Photo DNA along time ago, and I am sure now days they have something ML powered to do it. I am not sure the accuracy of these systems but even if they are pretty accurate there is something off putting about this trend. This very much feels like a Guilty until proven innocent type of program... "if you have nothing to hide you have nothing to fear"... Neither of those normally work out really well.
- lstamour 5y ago
  > Guilty until proven innocent type of program But this software doesn’t make arrests? A comparison would be police asking for what cellphones were in an area. There should be a lot of due process that happens before anything comes of this. I imagine police won’t look into one photo, they would want to match more than one. I agree though, the scary part might be how to clear your name after an overreaction occurs, similar to the effect swatting has on innocents or other kinds of false accusations. But those would happen even without such a system - we’ve heard about such stories before. “Man held for child porn charges, but it turned out they were innocent,” - it doesn’t take this kind of system to have police look over your phone for evidence and find a different crime than expected. That all said… child porn is really, really terrible and I’m actually in favour of this. As a society, we can decide on what’s acceptable. As long as there’s due process and a presumption of innocence, I’m all in favour of making it harder for criminals to get away with crimes. I do agree though, all these systems might make it harder to catch perpetrators if it pushes them offline. But what other options do we have to stop these sort of crimes when the kids involved can’t or won’t speak up? I suppose the fundamental problem is that having more evidence doesn’t necessarily lead to justice if it paints a false picture.
  - shotta 5y ago

deleted 5y ago

[deleted]

1-6 5y ago

What will happen is that criminals who actually had bad intent will move off of the platform and the ones who get the brunt of the blame are the innocent who had no ill intent.

vsareto 5y ago
Fairly sure criminals can't prove the efficient market hypothesis either. You still want to catch the dumb ones.
mlac 5y ago
The process is described above, but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. And the way it’s being done (hashes), a collision is highly unlikely. If it does occur it doesn’t mean it’s similar in nature (e.g. innocent picture of own child in bath). The hash isn’t looking at the image content in the sense of “what’s in the picture”, just the bits of the file. So it’s highly, highly unlikely, even if a collision occurs, that the collision would be an image that happens to be another child innocently bathing.
- gojomo 5y ago
  > but it’s very hard to “innocently” end up with one of those images that they are looking for from the database. Are you sure, given how many iPhone takeover/jailbreak bugs regularly exist, including the recent 0-click iMessage bug? Would you like to dare a hacking group, domestic or foreign, to land a single verboten image on your iPhone? > And the way it’s being done (hashes), a collision is highly unlikely. The limited details so far are suggestive that its using perceptual hashes, which are more susceptible to collision-engineering/false-positives than cryptographically-secure hashes.
- aviraldg 5y ago
  Presumably the verification that these false matches are not problematic is manual? That's not good enough given how our media currently works. Imagine articles published when information about such a check leaks that say "celebrity X's phone checked by police for suspected CSAM." While that is the truth ("suspected") no one cares about that nuance and such a person would get cancelled very quickly, even if there was no evidence of wrongdoing.

temp8964 5y ago

I am sure, presumbly in a sane world, computer scanning is only the early stage of the process, human will be invovled to make final judgements and decisions.

jenkstom 5y ago
I think Trump put a definitive end to the "Sane World" theory. We don't live in one.
AnotherGoodName 5y ago
I imagine the biggest catch of this will be teens taking inappropriate photos of themselves because teens often do things like this in their awkward years when sexuality and body consciousness kicks in. With this we'll be seeing warrants for those images and a big fuss made on this most suicidal age group who thought they were doing something completely private. This isn't good.
taway1213sadf 5y ago
Remember the US No Fly List? https://en.wikipedia.org/wiki/No_Fly_List https://en.wikipedia.org/wiki/No_Fly_List The list ballooned to over a million names, then required an ACLU lawsuit until humans became involved and a review process was initiated. As long as the majority of Americans dont feel burdened by something, dont expect anything logical to happen.

itake 5y ago

> perfectly innocent photos on someone's phone of their own children They are using a library of human verified images to compare the hashes. Probably similar to PhotoDNA[0]. There can be false positives, but AFAIK the algorithms are not trying to identify naked children, but comparing 2 similar image hashes. [0] - https://en.wikipedia.org/wiki/PhotoDNA https://en.wikipedia.org/wiki/PhotoDNA

barbazoo 5y ago
> They are using a library of human verified images to compare the hashes I didn't get that from the article, could you point to a source on this?
- judge2020 5y ago
  Images are submitted by over 1400 organizations[0] like the FBI and NCMEC, although the inner-workings of the database aren’t public and I can’t find a source or confirmation of every image actually being human-verified. The closest is how the "Child Victim Identification Program" helps identify victims in these images. 0: https://www.missingkids.org/theissues/csam#:~:text=Electronic%20Service%20Provider%20(ESP)%20Reporting https://www.missingkids.org/theissues/csam#:~:text=Electroni... (ctrl-f "1,400")
  - deleted 5y ago
    [deleted]
  - dylan604 5y ago
    Of all of the discussion, my brain has only keyed in on the child porn aspect. It just now clicked that if a missing kid is in a photo (even if not porn), it could help ID the missing kid. Some runaway at the age of 13 gets photographed in a group of other kids just doing teenager things. So even if not running hashes against known porn, it could still be doing facial recognition. Wonder what kind of implications that would have for people in witness protection or battered spouses and other legitimately "disappeared" people.

Causality1 5y ago

Yeah if you ran the giant box of photos in my mom's closet through an AI you'd totally get hits from photos of baby me being bathed in the sink, baby me running around the back yard naked, etc. This can't end well.

alkonaut 5y ago
If this even comes close to doing any “recognition” on local content and isn’t just (as the article says) a simple lookup of known images through hashes, then that would be a massive scandal. So I’m giving them the benefit of the doubt until it’s clarified, as the only confirmed way it works is the image hash lookup. There are words like “neural” and “trained” here, but no clear sign that there is actually any kind of image recognition that would risk false positives other than hash collisions.

JKCalhoun 5y ago

> Like a kid running butt naked around the house... Thought it was buck naked — but that works too. :-)

BrianB 5y ago
https://brians.wsu.edu/2016/05/19/butt/ https://brians.wsu.edu/2016/05/19/butt/
- amanaplanacanal 5y ago
  I would suggest that if it is widely used, it isn't an error, regardless of what Professor Brians thinks.

echelon 5y ago

This system isn't meant for child abuse. It's going to be used in China against enemies of the state. Then it will be used domestically for the same. Apple is a horrific company that you should stop using.

GeekyBear 5y ago

Microsoft implemented this a decade ago. >The system that scans cloud drives for illegal images was created by Microsoft and Dartmouth College and donated to NCMEC. The organization creates signatures of the worst known images of child pornography, approximately 16,000 files at present. These file signatures are given to service providers who then try to match them to user files in order to prevent further distribution of the images themselves, a Microsoft spokesperson told NBC News. (Microsoft implemented image-matching technology in its own services, such as Bing and SkyDrive.) https://www.nbcnews.com/technolog/your-cloud-drive-really-private-not-according-fine-print-1c8881731 https://www.nbcnews.com/technolog/your-cloud-drive-really-pr... There was another round of outrage when the Windows 10 TOS extended this to local storage. It also has the use case of scanning for DRM violations. (ripped mp3 files or movies)

partiallypro 5y ago
Again, that is not the same as scanning a local machine for it. That is all things that are on Microsoft's own servers or in their search results. Microsoft, Google, Cloudflare all have features like this. It's not the same as scanning local machines.
- GeekyBear 5y ago
  Again, the Windows 10 TOS extended that to scanning the local machine as well.
  - cududa 5y ago
    When you download an app it compares hashes locally to source material published hash. This is absolutely no different

jandrese 5y ago

Or when an internet troll gets your email address and/or phone number and starts spamming you with child porn pictures, which your phone immediately identifies and notifies authorities.

elzbardico 5y ago
A completely new way of swating someone
wang_li 5y ago
And to get cleared you’re going to have to let law enforcement clone and inspect the content of your phone.

getpolarized 5y ago

> I really don't see how this is going to end well, there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things that kids do. Like a kid running butt naked around the house, or a photo of something like a rash that is sent to a nurse friend for advice on what it is etc. These are still illegal in the US right now right? It's been a major criticism of these laws that they can hurt people accidentally.

pseudalopex 5y ago
I don't think just nudity was illegal ever. But just suspicion can hurt people.

deleted 5y ago

[deleted]

cm2187 5y ago

And you can imagine what's on a dermatologist or urologist phone!

cronix 5y ago

Don't worry, AI is very good at these things and never makes mistakes. It is so advanced it even understands nuance and complex communication mechanisms like sarcasm in text, even if it's only a single sentence to contrast! It will be able to know exactly the context of each image and it will affect only the sickos. We're also working on watching your face through the camera to see your true expression when viewing them. Oh, but if a few innocent people do happen get caught up in it and lives ruined, it's totally justifiable because this type of activity is the absolute worst. And false positives are exceeeeedingly rare. We'll apologize and adjust our algorithms. Sue us? How dare you? We're really trying to wipe out sickos. Are you a sicko or something? You must be. Only sicko's would disagree with protecting children.

8458e112e7b2 5y ago
Yep. And if you dont like it, build your own iPhone.
- ghego1 5y ago
  And be immediately labelled by society as a CP holder
mm983 5y ago
This is painful to read with the arrogant undertone while you don't actually know how CSAM scanning works. It's fuzzy hash, not ""AI"", based. Cloudflare uses it too, and last time i checked the web is still functional. https://support.cloudflare.com/hc/en-us/articles/360046106112-Understanding-the-CSAM-Scanning-Tool https://support.cloudflare.com/hc/en-us/articles/36004610611... How about we turn the tables and have the complainers suggest a solution. Because every single time an approach to targeted child porn takedowns has been suggested, such as datacenter raids which would not affect as many people, someone is screaming about their privacy. Child abusers evolve and are very happy if law enforcement doesn't.
- babyshake 5y ago
  So this means it is checking if you are sharing known CP images? That does seem to be much less invasive and problematic as there is likely no good reason to be sharing these images.

lenkite 5y ago

They need to stop calling it an iPhone. It's a govPhone. It now belongs to the government with "In-built Citizen Scanning" as free civic service. Thank you for serving the nation!

benhurmarcel 5y ago

If the photo of your kid running around the house ends up in a CP database, you messed up before Apple did.

anigbrowl 5y ago

there could be perfectly innocent photos on someone's phone of their own children doing perfectly normal things How would that match a hash list of known CP imagery though?

elzbardico 5y ago

Yes, not no to mention the gulf that exist between american culture of what is innocent and acceptable and the rest of the world. If we apply the american paranoid standards to the rest of the world, i am pretty sure we will have plenty of false positives.

jeegsy 5y ago

We really shouldn't be moving the conversation window into the technicalities just yet without discussing if this is even ok or reasonable in the first place.

anupamchugh 5y ago

Apparently, there's a Threshold Secret Sharing (TSI) technology in place to eliminate false positive. Besides, NeuralHash will basically match your photos against a database provided by NCMEC.