8 ms·

Would you please describe a solution to finding CSAM which you would find acceptable? It seems to me that on-device scanning for known CSAM content is on the lo

Would you please describe a solution to finding CSAM which you would find acceptable? It seems to me that on-device scanning for known CSAM content is on the lower end of the privacy-violating spectrum.

jszymborski 5y ago
No, the lower-end is infiltrating CSAM distribution networks the old-fashioned way.
gpm 5y ago
Traditional police work that respects the principle behind the 4th amendment, no searches of private information, algorithmic or otherwise, directly by the government or otherwise, where you do not have probable cause.
- DSingularity 5y ago
  Why can’t algorithms do the police work for some crimes?
  - gpm 5y ago
    Algorithms can absolutely do the police work for crimes, provided they only perform searches that are supported with probable cause.
    - DSingularity 5y ago
      I don’t think fingerprinting and comparing against a database of cryptographic hashes of CSAM is “illegal searches”.
      - gpm 5y ago
        If the government mandated them, they would absolutely be illegal searches. The government is not entitled to check whether or not I have a document containing certain content (illegal or not) without probable cause and a warrant to suggest that I do have that document (and that it is illegal). I'm carefully not alleging that this is illegal, because it's a third party doing it and it's not obvious that they are acting as the governments agent. Regardless of whether or not that solves the legal problem of unlawful searches, it does not solve the moral problem that we have a right to be free from unreasonable searches. Replace "child porn" with "political posters" here. If you would have a problem with that search, I claim that you should have a problem with this search, because the there is no evidence that the person you are searching is committing a crime, and as a result the claim for this to be a morally valid search needs to not be about guilt (which would require probable cause first), but about it not being a prohibited search in the first place.
  - TravisHusky 5y ago
    Seriously? For one, this is basically a warrantless search which is illegal (obviously they get around this because Apple is a private company). Also, trusting algorithms for critical things like this is beyond absurd.
    - DSingularity 5y ago
      I still don’t understand the reaction. People get tied up in a knot over this but it is an effective deterrent to child pornography full stop. Even if it rubs you the wrong way to have software fingerprinting your files, I really don’t care if it means placing a deterrent in place against child trafficking. Let’s walk through this. 1. Criminal kidnaps child and abuses him. 2. Criminal produced video of said abuse and sells it on the web. 3. Criminal continues to sell it and it spreads. 4. The video is detected by authorities who promptly add it to database. 5. Video is cryptographically hashed and now anybody who stores this content in iCloud can be identified 6. A customer of the criminal is caught 7. Forensics leads authorities to criminal who produced the video 8. One less criminal to profit from kidnapping and abusing children Everyone tries to make this approach as a slippery slope to facial. It doesn’t have to be that way if the right people are in the loop to blow the whistle.
      - Copernicron 5y ago
        How about a counterexample? I am a consenting adult in my thirties. I create a photo or video and send it to my partner. The algorithm flags it as CSAM when it only shows a fat bald guy. Before I know it I'm under investigation and my life is ruined because the algorithm got it wrong. Even being accused of this sort of thing is enough to destroy someone and drive them to suicide.
        ducadveritatem 5y ago
        The "algorithm" isn't some sort of neural network trying to intelligently identify things that "look like" CP. It's a perceptual hash matching against a database of known CP. It has to find multiple matches before it flags the account for review to reduce false positives. Only after review confirming a match to known exploitative images is the info referred to NCMEC for action. Full whitepaper: https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
  - luckylion 5y ago
    Because the price is too high. The same argument is being made to outlaw encryption: it will make police work much easier. Yeah, it sure will, but it will bring a lot of problems as well, and they far outweigh the benefits.
  - tylersmith 5y ago
    They could, and their first step would be to get a warrant.
    - DSingularity 5y ago
      There is a scale where that doesn’t work anymore. If one child abuser can sell his contents to thousands via nothing but WhatsApp, word of mouth, and a Bitcoin wallet, what will you do to fight that? I’m all for freedom of speech and freedom from fishing expeditions and what not, but if you don’t deter these things they will grow.
      - skinkestek 5y ago
        If (s)he sells to thousands via WhatsApp and word-of-mouth good old police work will have him/her bagged shortly. WhatsApps only claim to be private is end-to-end encryption of messages. The moment a polkce officer opens the phone of a buyer they can identify the seller. Same if an undercover cop hears the word-of-mouth and buys. This is just ordinary good police work that I whole-heartedly support.
  - karmicthreat 5y ago
    I'll be cool with it when you can get the algorithm to swear a warrant, testify in court and navigate probably cause.
  - Hamuko 5y ago
    Who's responsible when the algorithm starts targeting minorities and putting innocent people in prison?
    - gpm 5y ago
      Algorithms don't put people in jail, people do. The person who irresponsibly followed the output of the algorithm is, and the person who vouched for the algorithm being correct and put it into production is. Don't think of an algorithm as an independent actor, think of it as an awesome tool that makes you 100x more productive.
      - Hamuko 5y ago
        I'm still mostly thinking about an awesome tool that will target 100x more minority people and send 100x more innocent people in jail.
      - erhk 5y ago
        Okay but what about the intense war going on for youtube content creators getti g banned or content struck right now. Or all the wrongful bans on platforms like facebook or Twitter.
        gpm 5y ago
        The blame for that falls squarely on the a) Executives at Alphabet, who are ultimately in charge of the companies decisions b) The individual product managers, lawyers, engineers, and whoever else is involved in creating, maintaining, and running the system that bans people inappropriately. The algorithm they are using is a tool that they are using to do this, using a tool does not absolve them of the responsibility in any way.
      - skinkestek 5y ago
        > Algorithms don't put people in jail, people do. ... and guns and cars don't kill people, murderers do. But now a company is creating s new kind of "gun" that huge chunks of the community of security professionals are warning about because it is insanely powerful and dangerous. We have a number of reasons to warn about it: - China will undoubtedly demand to have Apple scan for their hashes too. The databases these hashes are collected from are not public for obvious reasons so this means they can slip whatever they want into it. Maybe Apple will check the images before they send them over but that is not necessarily the case as far as I read it, also it takes one rogue employee to correctly classify as not abuse material but make a note of the account and send it back after work. - This is not your average sha256 hash. This is perpetual hashes. They are made to catch not only the exact document but all kinds of variations of it. I have not specialized in perpetual hashes but as far as I can understand it goes without saying that the more resistant this is to modifications, the easier it becomes to create innocent images that triggers it. - even if we had a magic algorithm that resonated with everything good and only matched the images we wanted there still is potential for abuse. When I was younger I browsed through the cache folder of my machine and I remember there being a lot of images there that I can't remember having seen on any site I visited. Now it is said that this algorithm will only flag images about to be uploaded so obviously cache folders won't be scanned. But once this tool is in place why won't governments start applying pressure to Apple to scan everything? And what prevents someones soon-to-be ex from downloading and slipping some into iCloud when one leaves the phone unlocked? - Due process should sort many of the problematic cases here, but child abuse is for good reasons one of the worst things you can get accused. A mere accusation is often enough to ruins someone's life even if it later becomes clear to law enforcement that the person is innovent. Also most places have a way to go wrt due process.
- narrator 5y ago
  Yes, but the big exception to the fourth amendment are private searches. As long as it's not the government, or someone working on their behalf searching, it's not covered by the fourth amendment. The trend these days is to eliminate constitutional protections by farming the violations out to private corporations.
  - gpm 5y ago
    I agree it's not a violation of the fourth amendment (so long as Apple is not being forced to do this by the government), which I why I said "the principle behind the fourth amendment" instead.
  - Blikkentrekker 5y ago
    The U.S.A.'s weak privacy protections aren't the entire world, you know. In the E.U., and other places, companies are definitely held to stricter standards as to what extent they can search a private device, even if they claim the o.s. is only a service.
  - WindyLakeReturn 5y ago
    >or someone working on their behalf I wonder what the limits to this are. Clearly if government asks for help then that counts, but what happens if government slowly builds up partnerships where certain 'good partner' behavior is expected but never explicitly required in a contract? Say government started favoring companies who scan for drug images for any contracts over those who don't? Or say that while working with companies that scan for drug images they build up relationships that lead to both better treatment and to better chances at winning contracts? Maybe companies that refuse to implement the drug abuse material scans end up getting investigated more often. I wonder how long until the fourth amendment can be considered dead due to this one exception?
- a1369209993 5y ago
  > where you do not have probable cause. Where you do not have both probable cause and a warrant, based on and identifing that cause, signed by a publicly elected judge in the user's (not company/operator's) juridiction.
- DSingularity 5y ago
  The fourth amendment is irrelevant here. The constitution is meant to limit the power of government because government is ubiquitous and it has a monopoly on violence. If it were to become too powerful then it will oppress its citizens. Preventing a digital platform from governing what kind of content it hosts is just absurd and makes no sense. If apple decides that it does not want to be the platform where CSAM content is distributed then they should be free to do so. If you don’t like being a part of a platform which scans images for abusive content migrate to another platform.
Blikkentrekker 5y ago
Child pornography is the boogyman used to compromise privacy. There is significantly more illegal material of greater concern such as communication to plot terrorists attacks or companies ignoring safety regulations and leading to the deaths of many in doing so, and all that isn't scanned for either on the theory that it might exist, not that there is due cause for a search warrant or some similar mechanism and burden. But child pornography is the one thing Anglo-Saxon culture is notoriously emotional about, and willing to surrender all it's freedom and privacy for. For a while terrorism counted among this too, but that seems to have fallen off.
hapless 5y ago
There is no solution to searching private data that I am going to find acceptable. * If the "target" database is secret, then government entities are free to use a "CSAM" database to monitor dissidents by adding their own items of interest to the target list. * If the "target" database is public, the only way to validate its contents is to ... traffic in child sex abuse material. That's not great. Basically, there is no way to create and validate this database in public view. I don't need yet another flavor of secret policing in America. This doesn't seem like a problem that can be solved without compromising my privacy. I would rather the problem went un-solved than allow the state to rifle through all of my private files to "prevent the distribution of child sex abuse material," knowing full well the state will define that "material" however they like, then use parallel construction to prosecute dissidents whenever they have gotten information illegally.
- DSingularity 5y ago
  Isn’t apple necessarily in the loop to validate the database?
  - hapless 5y ago
    No. That's the whole point of this endeavor. It's a privacy-preserving framework to allow the government to monitor the contents of iCloud directly, with few, if any, Apple employees ever having to get their hands dirty. Apple just handles a target database that gets distributed to phones, and then compiles a list of users whose data had hits against the target database. Apple employees don't have to dirty their hands with what, perhaps, is in that target database. Not their problem!
    - DSingularity 5y ago
      “Apple just handles a target database” ==> apple in the loop
      - Dylan16807 5y ago
        The database distributed to phones is just hashes. By itself it can't be validated.
diebeforei485 5y ago
Yeah. They can scan photos that are actually passed around, instead of something that's just in a photo library. That will likely result in 2-3 orders of magnitude fewer false positives.
- DSingularity 5y ago
  That is a good point.
- ducadveritatem 5y ago
  That's exactly what they're doing. And they're requiring multiple matches before an account is flagged, further reducing false positives. They claim they've set the threshold to achieve a ~1 in 1 trillion probability of an account being improperly flagged.
  - diebeforei485 5y ago
    False, they are scanning all photos in your library, regardless of whether or not you send them on to anyone else. 1 in 1 trillion is not a particularly impressive number, considering that users have tens of thousands of photos in their library, and there are a billion active iPhones. That's if you do believe their claimed false positive rate -- which I don't (yet), because they have published absolutely nothing about how neuralMatch actually works. They have an interesting body of work around how suspected matches are encrypted so Apple can't see them until certain conditions are made, but nothing about how they identify suspect photos.
gorgoiler 5y ago
Boosting the amount of influence schools have in childrens’ lives is something we could do better at. More teachers, smaller classes, stronger relationships, better discipline, more love. I see too many teachers where I work who just don’t care any more. Educating school leaders to educate their teachers to help children root out the family member that is abusing them. That there would be a pretty neat solution — and I really don’t mean to be inflammatory about this — than catching villains based on photo evidence of the abuse that has become systematic enough to be shared online. It is very expensive and time consuming though. Teachers are hard to recruit (low pay, low quality job) which exacerbates the problem which makes it even harder to recruit them.
alexfromapex 5y ago
Them having to get a warrant or some form of fair adjudication instead of a carte blanche of privacy invasion?
Syonyk 5y ago
Find the websites distributing it, infiltrate them, generally do the legwork to find what's going on - which is exactly what they've been doing. "But the children!" is not a skeleton key for privacy, as far as I'm concerned. I reject on-device scanning for anything in terms of personal content as a thing that should be done, so, no, I don't have a suggested way to securely accomplish privacy invasions of this nature. I'm aware that they claim it will only be applied to iCloud based uploads, but I'm also aware that those limits rarely stand the test of governments with gag orders behind them, so if Apple is willing to deploy this functionality, I have to assume that, at some point, it will be used to scan all images on a device, against an ever growing database of "known badness" that cannot be evaluated to find out what's actually in it. If there existed some way to independently have the database of hashes audited for what was in it, which is a nasty set of problems for images that are illegal to store, and to verify that the database on device only contained things in the canonical store, I might object slightly less, but... even then, the concept of scanning things on my private, encrypted device to identify badness is still incredibly objectionable. In the battle between privacy and "We can catch all the criminals if we just know more," the government has been collecting huge amounts of data endlessly (see Snowden leaks for details), and yet hasn't proved that this is useful to prevent crimes. Given that, I am absolutely opposed to giving them more data to work with. I would rather have 10 criminals go free than one innocent person go to prison, and I trust black box algorithms with that as far as I can throw the building they were written in.
aaomidi 5y ago
Yes. Find the one's producing it and use proper infiltration tactics. Empower children to report abusers and protect them when they do.
wyager 5y ago
Don't spy on my device. I literally don't care if that doesn't help some pet cause.
- ducadveritatem 5y ago
  Sorry, but they're not "spying on your device". If you choose to upload your photos to their cloud service (iCloud Photo Library), they'll use perceptual hashing and private set intersection tech to check against a database of known exploitative child imagery to make sure you aren't uploading that to their service. That's it.
  - wyager 5y ago
    You just described “spying” with more words.
WindyLakeReturn 5y ago
>on the lower end of the privacy-violating spectrum I think that's the thing, privacy minded people want solutions that aren't on the spectrum at all. I can come up with some solutions for greatly reducing CSA that are really low on violating parental rights but they have little chance of gaining support because the only solutions people want are the ones that don't violate parental rights at all (except when parental rights have been suspended due to evidence of CSA being found).
throw1998149898 5y ago
Anything which - directly prevents actual abuse of children in the physical world (obviously) - fights the sale of images created through abuse (as this incentivizes abuse). Those are the main underlying reasons to fight CSAM, besides the obvious horrificness of the stuff in itself. Please also consider the second-order effects of an increasingly totalitarian society on the wellbeing of children.
tomjen3 5y ago
Not OP, but a warrant, upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized. In other words, you don't get to yell think of the children and randsack peoples homes. If you don't have specific evidence suggesting that I am a criminal, you assume that I am not _and_ let me be.
r00fus 5y ago
How about not doing it. The "privacy" of a on-device scanning is meaningless when you upload to iCloud where they will send to law enforcement and an unaccountable nonprofit as soon as it matches something.

jszymborski 5y ago

No, the lower-end is infiltrating CSAM distribution networks the old-fashioned way.

gpm 5y ago

Traditional police work that respects the principle behind the 4th amendment, no searches of private information, algorithmic or otherwise, directly by the government or otherwise, where you do not have probable cause.

DSingularity 5y ago
Why can’t algorithms do the police work for some crimes?
- gpm 5y ago
  Algorithms can absolutely do the police work for crimes, provided they only perform searches that are supported with probable cause.
  - DSingularity 5y ago
    I don’t think fingerprinting and comparing against a database of cryptographic hashes of CSAM is “illegal searches”.
    - gpm 5y ago
      If the government mandated them, they would absolutely be illegal searches. The government is not entitled to check whether or not I have a document containing certain content (illegal or not) without probable cause and a warrant to suggest that I do have that document (and that it is illegal). I'm carefully not alleging that this is illegal, because it's a third party doing it and it's not obvious that they are acting as the governments agent. Regardless of whether or not that solves the legal problem of unlawful searches, it does not solve the moral problem that we have a right to be free from unreasonable searches. Replace "child porn" with "political posters" here. If you would have a problem with that search, I claim that you should have a problem with this search, because the there is no evidence that the person you are searching is committing a crime, and as a result the claim for this to be a morally valid search needs to not be about guilt (which would require probable cause first), but about it not being a prohibited search in the first place.
- TravisHusky 5y ago
  Seriously? For one, this is basically a warrantless search which is illegal (obviously they get around this because Apple is a private company). Also, trusting algorithms for critical things like this is beyond absurd.
  - DSingularity 5y ago
    I still don’t understand the reaction. People get tied up in a knot over this but it is an effective deterrent to child pornography full stop. Even if it rubs you the wrong way to have software fingerprinting your files, I really don’t care if it means placing a deterrent in place against child trafficking. Let’s walk through this. 1. Criminal kidnaps child and abuses him. 2. Criminal produced video of said abuse and sells it on the web. 3. Criminal continues to sell it and it spreads. 4. The video is detected by authorities who promptly add it to database. 5. Video is cryptographically hashed and now anybody who stores this content in iCloud can be identified 6. A customer of the criminal is caught 7. Forensics leads authorities to criminal who produced the video 8. One less criminal to profit from kidnapping and abusing children Everyone tries to make this approach as a slippery slope to facial. It doesn’t have to be that way if the right people are in the loop to blow the whistle.
    - Copernicron 5y ago
      How about a counterexample? I am a consenting adult in my thirties. I create a photo or video and send it to my partner. The algorithm flags it as CSAM when it only shows a fat bald guy. Before I know it I'm under investigation and my life is ruined because the algorithm got it wrong. Even being accused of this sort of thing is enough to destroy someone and drive them to suicide.
      - ducadveritatem 5y ago
        The "algorithm" isn't some sort of neural network trying to intelligently identify things that "look like" CP. It's a perceptual hash matching against a database of known CP. It has to find multiple matches before it flags the account for review to reduce false positives. Only after review confirming a match to known exploitative images is the info referred to NCMEC for action. Full whitepaper: https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
- luckylion 5y ago
  Because the price is too high. The same argument is being made to outlaw encryption: it will make police work much easier. Yeah, it sure will, but it will bring a lot of problems as well, and they far outweigh the benefits.
- tylersmith 5y ago
  They could, and their first step would be to get a warrant.
  - DSingularity 5y ago
    There is a scale where that doesn’t work anymore. If one child abuser can sell his contents to thousands via nothing but WhatsApp, word of mouth, and a Bitcoin wallet, what will you do to fight that? I’m all for freedom of speech and freedom from fishing expeditions and what not, but if you don’t deter these things they will grow.
    - skinkestek 5y ago
      If (s)he sells to thousands via WhatsApp and word-of-mouth good old police work will have him/her bagged shortly. WhatsApps only claim to be private is end-to-end encryption of messages. The moment a polkce officer opens the phone of a buyer they can identify the seller. Same if an undercover cop hears the word-of-mouth and buys. This is just ordinary good police work that I whole-heartedly support.
- karmicthreat 5y ago
  I'll be cool with it when you can get the algorithm to swear a warrant, testify in court and navigate probably cause.
- Hamuko 5y ago
  Who's responsible when the algorithm starts targeting minorities and putting innocent people in prison?
  - gpm 5y ago
    Algorithms don't put people in jail, people do. The person who irresponsibly followed the output of the algorithm is, and the person who vouched for the algorithm being correct and put it into production is. Don't think of an algorithm as an independent actor, think of it as an awesome tool that makes you 100x more productive.
    - Hamuko 5y ago
      I'm still mostly thinking about an awesome tool that will target 100x more minority people and send 100x more innocent people in jail.
    - erhk 5y ago
      Okay but what about the intense war going on for youtube content creators getti g banned or content struck right now. Or all the wrongful bans on platforms like facebook or Twitter.
      - gpm 5y ago
        The blame for that falls squarely on the a) Executives at Alphabet, who are ultimately in charge of the companies decisions b) The individual product managers, lawyers, engineers, and whoever else is involved in creating, maintaining, and running the system that bans people inappropriately. The algorithm they are using is a tool that they are using to do this, using a tool does not absolve them of the responsibility in any way.
    - skinkestek 5y ago
      > Algorithms don't put people in jail, people do. ... and guns and cars don't kill people, murderers do. But now a company is creating s new kind of "gun" that huge chunks of the community of security professionals are warning about because it is insanely powerful and dangerous. We have a number of reasons to warn about it: - China will undoubtedly demand to have Apple scan for their hashes too. The databases these hashes are collected from are not public for obvious reasons so this means they can slip whatever they want into it. Maybe Apple will check the images before they send them over but that is not necessarily the case as far as I read it, also it takes one rogue employee to correctly classify as not abuse material but make a note of the account and send it back after work. - This is not your average sha256 hash. This is perpetual hashes. They are made to catch not only the exact document but all kinds of variations of it. I have not specialized in perpetual hashes but as far as I can understand it goes without saying that the more resistant this is to modifications, the easier it becomes to create innocent images that triggers it. - even if we had a magic algorithm that resonated with everything good and only matched the images we wanted there still is potential for abuse. When I was younger I browsed through the cache folder of my machine and I remember there being a lot of images there that I can't remember having seen on any site I visited. Now it is said that this algorithm will only flag images about to be uploaded so obviously cache folders won't be scanned. But once this tool is in place why won't governments start applying pressure to Apple to scan everything? And what prevents someones soon-to-be ex from downloading and slipping some into iCloud when one leaves the phone unlocked? - Due process should sort many of the problematic cases here, but child abuse is for good reasons one of the worst things you can get accused. A mere accusation is often enough to ruins someone's life even if it later becomes clear to law enforcement that the person is innovent. Also most places have a way to go wrt due process.
narrator 5y ago
Yes, but the big exception to the fourth amendment are private searches. As long as it's not the government, or someone working on their behalf searching, it's not covered by the fourth amendment. The trend these days is to eliminate constitutional protections by farming the violations out to private corporations.
- gpm 5y ago
  I agree it's not a violation of the fourth amendment (so long as Apple is not being forced to do this by the government), which I why I said "the principle behind the fourth amendment" instead.
- Blikkentrekker 5y ago
  The U.S.A.'s weak privacy protections aren't the entire world, you know. In the E.U., and other places, companies are definitely held to stricter standards as to what extent they can search a private device, even if they claim the o.s. is only a service.
- WindyLakeReturn 5y ago
  >or someone working on their behalf I wonder what the limits to this are. Clearly if government asks for help then that counts, but what happens if government slowly builds up partnerships where certain 'good partner' behavior is expected but never explicitly required in a contract? Say government started favoring companies who scan for drug images for any contracts over those who don't? Or say that while working with companies that scan for drug images they build up relationships that lead to both better treatment and to better chances at winning contracts? Maybe companies that refuse to implement the drug abuse material scans end up getting investigated more often. I wonder how long until the fourth amendment can be considered dead due to this one exception?
a1369209993 5y ago
> where you do not have probable cause. Where you do not have both probable cause and a warrant, based on and identifing that cause, signed by a publicly elected judge in the user's (not company/operator's) juridiction.
DSingularity 5y ago
The fourth amendment is irrelevant here. The constitution is meant to limit the power of government because government is ubiquitous and it has a monopoly on violence. If it were to become too powerful then it will oppress its citizens. Preventing a digital platform from governing what kind of content it hosts is just absurd and makes no sense. If apple decides that it does not want to be the platform where CSAM content is distributed then they should be free to do so. If you don’t like being a part of a platform which scans images for abusive content migrate to another platform.

Blikkentrekker 5y ago

Child pornography is the boogyman used to compromise privacy. There is significantly more illegal material of greater concern such as communication to plot terrorists attacks or companies ignoring safety regulations and leading to the deaths of many in doing so, and all that isn't scanned for either on the theory that it might exist, not that there is due cause for a search warrant or some similar mechanism and burden. But child pornography is the one thing Anglo-Saxon culture is notoriously emotional about, and willing to surrender all it's freedom and privacy for. For a while terrorism counted among this too, but that seems to have fallen off.

hapless 5y ago

There is no solution to searching private data that I am going to find acceptable. * If the "target" database is secret, then government entities are free to use a "CSAM" database to monitor dissidents by adding their own items of interest to the target list. * If the "target" database is public, the only way to validate its contents is to ... traffic in child sex abuse material. That's not great. Basically, there is no way to create and validate this database in public view. I don't need yet another flavor of secret policing in America. This doesn't seem like a problem that can be solved without compromising my privacy. I would rather the problem went un-solved than allow the state to rifle through all of my private files to "prevent the distribution of child sex abuse material," knowing full well the state will define that "material" however they like, then use parallel construction to prosecute dissidents whenever they have gotten information illegally.

DSingularity 5y ago
Isn’t apple necessarily in the loop to validate the database?
- hapless 5y ago
  No. That's the whole point of this endeavor. It's a privacy-preserving framework to allow the government to monitor the contents of iCloud directly, with few, if any, Apple employees ever having to get their hands dirty. Apple just handles a target database that gets distributed to phones, and then compiles a list of users whose data had hits against the target database. Apple employees don't have to dirty their hands with what, perhaps, is in that target database. Not their problem!
  - DSingularity 5y ago
    “Apple just handles a target database” ==> apple in the loop
    - Dylan16807 5y ago
      The database distributed to phones is just hashes. By itself it can't be validated.

diebeforei485 5y ago

Yeah. They can scan photos that are actually passed around, instead of something that's just in a photo library. That will likely result in 2-3 orders of magnitude fewer false positives.

DSingularity 5y ago
That is a good point.
ducadveritatem 5y ago
That's exactly what they're doing. And they're requiring multiple matches before an account is flagged, further reducing false positives. They claim they've set the threshold to achieve a ~1 in 1 trillion probability of an account being improperly flagged.
- diebeforei485 5y ago
  False, they are scanning all photos in your library, regardless of whether or not you send them on to anyone else. 1 in 1 trillion is not a particularly impressive number, considering that users have tens of thousands of photos in their library, and there are a billion active iPhones. That's if you do believe their claimed false positive rate -- which I don't (yet), because they have published absolutely nothing about how neuralMatch actually works. They have an interesting body of work around how suspected matches are encrypted so Apple can't see them until certain conditions are made, but nothing about how they identify suspect photos.

gorgoiler 5y ago

Boosting the amount of influence schools have in childrens’ lives is something we could do better at. More teachers, smaller classes, stronger relationships, better discipline, more love. I see too many teachers where I work who just don’t care any more. Educating school leaders to educate their teachers to help children root out the family member that is abusing them. That there would be a pretty neat solution — and I really don’t mean to be inflammatory about this — than catching villains based on photo evidence of the abuse that has become systematic enough to be shared online. It is very expensive and time consuming though. Teachers are hard to recruit (low pay, low quality job) which exacerbates the problem which makes it even harder to recruit them.

alexfromapex 5y ago

Them having to get a warrant or some form of fair adjudication instead of a carte blanche of privacy invasion?

Syonyk 5y ago

Find the websites distributing it, infiltrate them, generally do the legwork to find what's going on - which is exactly what they've been doing. "But the children!" is not a skeleton key for privacy, as far as I'm concerned. I reject on-device scanning for anything in terms of personal content as a thing that should be done, so, no, I don't have a suggested way to securely accomplish privacy invasions of this nature. I'm aware that they claim it will only be applied to iCloud based uploads, but I'm also aware that those limits rarely stand the test of governments with gag orders behind them, so if Apple is willing to deploy this functionality, I have to assume that, at some point, it will be used to scan all images on a device, against an ever growing database of "known badness" that cannot be evaluated to find out what's actually in it. If there existed some way to independently have the database of hashes audited for what was in it, which is a nasty set of problems for images that are illegal to store, and to verify that the database on device only contained things in the canonical store, I might object slightly less, but... even then, the concept of scanning things on my private, encrypted device to identify badness is still incredibly objectionable. In the battle between privacy and "We can catch all the criminals if we just know more," the government has been collecting huge amounts of data endlessly (see Snowden leaks for details), and yet hasn't proved that this is useful to prevent crimes. Given that, I am absolutely opposed to giving them more data to work with. I would rather have 10 criminals go free than one innocent person go to prison, and I trust black box algorithms with that as far as I can throw the building they were written in.

aaomidi 5y ago

Yes. Find the one's producing it and use proper infiltration tactics. Empower children to report abusers and protect them when they do.

wyager 5y ago

Don't spy on my device. I literally don't care if that doesn't help some pet cause.

ducadveritatem 5y ago
Sorry, but they're not "spying on your device". If you choose to upload your photos to their cloud service (iCloud Photo Library), they'll use perceptual hashing and private set intersection tech to check against a database of known exploitative child imagery to make sure you aren't uploading that to their service. That's it.
- wyager 5y ago
  You just described “spying” with more words.

WindyLakeReturn 5y ago

>on the lower end of the privacy-violating spectrum I think that's the thing, privacy minded people want solutions that aren't on the spectrum at all. I can come up with some solutions for greatly reducing CSA that are really low on violating parental rights but they have little chance of gaining support because the only solutions people want are the ones that don't violate parental rights at all (except when parental rights have been suspended due to evidence of CSA being found).

throw1998149898 5y ago

Anything which - directly prevents actual abuse of children in the physical world (obviously) - fights the sale of images created through abuse (as this incentivizes abuse). Those are the main underlying reasons to fight CSAM, besides the obvious horrificness of the stuff in itself. Please also consider the second-order effects of an increasingly totalitarian society on the wellbeing of children.

tomjen3 5y ago

Not OP, but a warrant, upon probable cause, supported by oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized. In other words, you don't get to yell think of the children and randsack peoples homes. If you don't have specific evidence suggesting that I am a criminal, you assume that I am not _and_ let me be.

r00fus 5y ago

How about not doing it. The "privacy" of a on-device scanning is meaningless when you upload to iCloud where they will send to law enforcement and an unaccountable nonprofit as soon as it matches something.