8 ms·

I recently listened to a Darknet Diaries episode on messaging app Kik. This app is apparently being used by many people to trade child pornography. In this epis

I recently listened to a Darknet Diaries episode on messaging app Kik. This app is apparently being used by many people to trade child pornography. In this episode, there was some criticism expressed on how Kik doesn't scan all the images on their platform for child pornography. I would really like to hear from people who sign this open letter, how they think about this. Should the internet be a free for all place without moderation? Where are the boundaries for moderation (if it has to exist), one-on-one versus group chat, public versus private chat? To quote this open letter: “Apple is opening the door to broader abuses”. Wouldn't not doing anything open a different door to broader abuse? Edit: I would really love an actual answer, since responses until now have been "but muh privacy". You can't plead for unlimited privacy and ignore the impact of said privacy. If you want unlimited privacy, at least have the balls to admit that this will allow free trade of CSAM, but also revenge porn, snuff-films and other things like it.

0xy 5y ago
Evidently the episode you listened to was loaded with false information, because Kik has used PhotoDNA to scan for CSAM for 7 years. [1] [1] https://www.kik.com/blog/using-microsofts-photodna-to-protect-kik-users/ https://www.kik.com/blog/using-microsofts-photodna-to-protec...
- donkeyd 5y ago
  > Doc: From what I can tell, it only starts looking in the rooms and looking at individual people if they are reported for something. https://darknetdiaries.com/transcript/93/ https://darknetdiaries.com/transcript/93/ If I were Kik, I would also write a blog post about using something like this. Many, many things point at Kik only doing the bare minimum though. (If you're the type who supports moderation, apparently they're already doing too much according to much of HN.)
3rly 5y ago
Should the internet be a free for all place without moderation? Yes!!!
- ionwake 5y ago
  I think different opinions are great. Just wanted to point out saying the internet should be moderated on the hacker news forums is surprising and funny.
  - deleted 5y ago
    [deleted]
toxik 5y ago
I think there is a very deep and troublesome philosophical issue here. Ceci n'est pas une pipe. A picture of child abuse /is not/ child abuse. Let me ask you a counter-question. If I am able to draw child pornography so realistically that you couldn't easily tell, am I committing a crime by drawing?
- yunohn 5y ago
  What is this absurd counter? Who’s talking about surrealistic drawings? We’re talking about actual material in real life, being shared by users and abusers. To be clear, I’m not supporting surveillance, just stating facts.
  - toxik 5y ago
    You should maybe read about the work [0], or just read the rest of what I said. Surrealism has nothing to do with the argument I made, so why do you bring it up? CSAM databases can by necessity not contain novel abuses, because they are novel. In fact, filtering by CSAM databases even indirectly /encourage/ novel abuses because these would not be caught by said filter. Catching CP hoarders does little to help the children being exploited in the first place, and does a lot to harm our integrity and privacy. [0] https://en.wikipedia.org/wiki/The_Treachery_of_Images https://en.wikipedia.org/wiki/The_Treachery_of_Images
    - ElFitz 5y ago
      > Catching CP hoarders does little to help the children being exploited in the first place If there is no market for it, there might be less incentive to produce any more of it. Not that I believe we should all be continuously spied by people who merely pinky sweared to do it right and not abuse it.
      - toxik 5y ago
        I don't think market forces are what drive the production of CSAM. Rather, it's some weird fetish of child abusers to share their conquests. I'm sure you're aware, but when CP collectors are busted, they often have terabytes upon terabytes of CP. But that's, I think, tangential - I don't understand pedophilia well enough to say something meaningful here.
swiley 5y ago
There's a difference a mile wide between moderating a service and moderating people's personal computers. Maybe the same thing that makes this hard to understand about iOS will be what finally kills these absolutely wretched mobile OSes.
- donkeyd 5y ago
  > moderating people's personal computers This is done client side because your data is encrypted in their cloud. It won't be done if you disable cloud sync. If you just keep your cp out of the cloud, you're fine.
  - swiley 5y ago
    TBH it doesn't matter at all why they're doing it, they've crossed a line here.
  - new299 5y ago
    Apple can decrypt data stored on iCloud, and they scan it already it seems: https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-icloud-photos-for-child-abuse-images/ https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-... Which it what makes adding the client side functionality even more problematic. They can easily extend this to scanning all offline content.
- tda 5y ago
  except that the boundary between device and service is steadily eroding. A screen is a device and netflix is a service, but a smart tv that won't function without an internrt connection? A nokia was a device you owned, but any Apple device is more like a service than a device really, as you don't control what OS can run on it, which unsigned software you run etc. So if you are ok with moderation on services and your device turns into a service... Perhaps we need laws to fully separate hardware and software but actually the integration between hardware and software is becoming stronger and stronger every year
h0mie 5y ago
Should all personal computers be scanned all the time by windows/apple/... in case it contains cp?
- donkeyd 5y ago
  Are you unable to answer any of the questions I asked? I'm seriously interested in hearing, from people who think Apple is in the wrong, where the border of acceptability is. Would you prefer iOS submitted the photo's to their cloud unencrypted and Apple scanned them there? Because that's what the others are doing.
  - veeti 5y ago
    iCloud photos are not end-to-end encrypted, Apple has full access to them. https://support.apple.com/en-us/HT202303 https://support.apple.com/en-us/HT202303
sebzim4500 5y ago
Fundamentally there is a difference between scanning files on your servers and scanning files stored on someone else's device, without their consent.
- avnigo 5y ago
  Not that I wholly agree with Apple's move due to other implications, but it has to be said that only photos that you have chosen to upload to iCloud will be scanned. So, in a sense, you do consent to having those photos scanned by using iCloud. Maybe it's time for Apple to drop the marketing charade on privacy with iCloud, since iCloud backups aren't even encrypted anyway.
ianhawes 5y ago
Many people share photos of child pornography via the mail. There has been criticism that the USPS does not open all mail and scan them for child pornography. I would really like to hear from people who do not sign this letter how they think about that.
- donkeyd 5y ago
  I think it would be horrible if they opened all mail... However, if they had a system that could detect CSAM in unopened mail with an extremely low false negative rate, then I'd be fine with it. With many packages this already happens in the search for drugs and weapons and I have no problem with that either.
  - stevenicr 5y ago
    I believe millions more nudes are sent via snap/watsap/etcetc every day then are sent in the mail in a year.. with "extremely low false negative rate" - if that means there are a bunch of geeks and agents staring at your wife/gf.. your daughter/ etc.. is that okay? Some people will say yes - but I don't think those people should get to decide that all the other people in the world have to hand over all the nudes and dick pics everyone else has taken. when they open mail with drugs and weapons it's most likely not a sculpture of your wife's genitals / daughter's tits, pics of your hemorrhoids / erectile implant - whatever other private pic stuff that no one else should be peering at. If they were opening and reading every written word sent and received - I can guarantee you people would be up in arms about it.
    - donkeyd 5y ago
      > I believe millions more nudes are sent via snap/watsap/etcetc every day then are sent in the mail in a year. I know for sure this is true, but this BS scenario was still used as a rebuttal against my comment, so I figured I'd give them an actual answer. > when they open mail with drugs and weapons it's most likely not ... other private pic stuff that no one else should be peering at. People order a lot of stuff that is very private online. From self help books, to dilos (which show up clearly on x-ray), to buttless chaps. If anything, packages are probably more private than mail these days, since people only mail postcards and bills. > If they were opening and reading every written word sent and received - I can guarantee you people would be up in arms about it. They would. But this is completely incomparable to what's happening on iOS devices now.
simion314 5y ago
My personal opinion is if we could spy on all citizens all the time we could stop all or most of the crimes, do we want this? If you say Yes then stop reading here. Else if you say 100% surveillance is too much then what do you have against people discussing where the line should be drawn? Some person that would sign that letter might be fine with video cameras in say a bank or some company building entrance but he is probably not fine with his own phone/laptop recording him and sending data to soem company or government without his consent. So let's discuss where should the line be drawn, also if competent people in this domain are present let's discuss better ideas on preventing or catching criminals, or even for this method let's discuss if it can be done better to fix all the concerns. What I am sure is that clever criminals will not be affected by this, so I would like to know if any future victim would be saved (I admit I might be wrong, so I would like to see more data from different countries that would imply the impact of this surveilance)
- croes 5y ago
  This only finds known pictures of child abuse not new ones and especially it doesn't find the perpetrators or prevent the abuse. But it creates a infrastructure for all other kind of "criminal" data. I bet sooner or later governments want to include other hashes to find the owners of specific files. Could be bomb creation manuals, could be flyers against a corrupt regime. The sky is the limit and the road to hell is paved with good intentions.
  - heavyset_go 5y ago
    > This only finds known pictures of child abuse not new ones No, it finds images that have similar perceptual hashes, typically using the hamming distance or another metric to calculate differences between hashes. I've built products that do similar things. False positives are abound with perceptual hashes, especially when you start fuzzy matching them.
    - croes 5y ago
      But these are just variations of known pictures to recognize them if they cropped or scaled. The hash of a really new picture isn't similar to the hash of a known picture.
digitalsushi 5y ago
The big reason I don't want someone else scanning my phone (and a year later, my laptop) is that last I checked, software is not perfect, and I don't need the FBI getting called on me cause my browser cache smelled funny to some Apple PhD's machine-learning decision. It's the same reason I don't want police having any opportunistic information about me - both of these have an agenda, and so innocent people get pulled into scenarios they shouldn't be when that information becomes the unlucky best-fit of the day. That Apple has even suggested this is disturbing because I have to fully expect this is their vision for every device I am using. And then I expect from there, it's a race to the bottom until we treat our laptops like they are all live microphones.
- robertoandred 5y ago
  The chance of you getting enough false positives to reach the flagging threshold is one in a trillion. And then the manual review would catch the error easily. The FBI won’t be called on you.
  - chopin 5y ago
    That's a claim by Apple. Due to the opaque process this is completely unverifiable. As well this statement relies on the fact that no malicious actor out there is trying to thwart the system. The hashes used are deliberately chosen to easily produce collisions otherwise the system won't work. This almost certainly will be abused.
    - robertoandred 5y ago
      “This almost certainly will be abused.” is a claim by you and completely unverifiable. Apple has published white papers explaining how their hashing, matching, and cryptography work. How do you see someone thwarting that process specifically?
      - drampelt 5y ago
        Apple's claims on how it will work are also completely unverifiable. What's stopping a government from providing Apple with hashes of any other sort of content they dislike?
shoulderchipper 5y ago
> Should the internet be a free for all place without moderation? The better question would be: do you want an arbitrary person (like me) to decide whether you have a right to send an arbitrary pack of bytes? Neither "society" nor "voters" nor "corporations" make these decisions. It is always an arbitrary person who does. Should one person surrender his agency into the hands of another?
- krapp 5y ago
  >Neither "society" nor "voters" nor "corporations" make these decisions. It is always an arbitrary person who does. Except in this case, a corporation (Apple) is making the decision relative to the sexual mores of modern Western society and the child pornography laws of the United States. It's unlikely this decision was made and implemented randomly by a single "arbitrary" individual. Contrary to your claim, it's never an arbitrary person. And yes, I believe Apple has the right to decide how you use their product, including what bytes can and cannot be sent on it. >Should one person surrender his agency into the hands of another? We do that all the time, that's a fundamental aspect of living in a society. But in this specific case, no one is forcing you to use an Apple phone, so you're not surrendering your agency, you're trading it in exchange for whatever convenience or features lead you to prefer an Apple product over competitors. That's still your choice to make.
  - WWLink 5y ago
    "Apple has the right to decide how you use their product" I hate this argument. If Apple wants to claim ownership of _their_ products then they shouldn't sell them. They should lease them.
aidenn0 5y ago
I won't use a service that performs dragnet surveillance on my communication. The US Postal service does not open every single letter to check if it contains illegal material. If I rent a storage unit, the storage company does not open up my boxes to see if I have albums of child pornography. This move by Apple is equivalent. I'm going to turn this around and say that those in favor of Apple's move: "Should the internet be a place where your every move is surveilled?" given that we have some expectation of privacy in real life.
- donkeyd 5y ago
  > The US Postal service does not open every single letter to check if it contains illegal material. You'd be surprised about the amount of packages that gets x-rayed in order to find drugs. But yes, you're 100% right that it's not all of them.
  - silverpepsi 5y ago
    An x-ray is nothing like reading the contents of letters or randomly checking ahipped harddrives and USB sticks for content. I don't know how to clarify legally or conceptually, but I feel confident there is a very clear difference
    - emodendroket 5y ago
      Perhaps I'm being too cynical but I think the difference is they haven't figured out a convenient, automated way to do the latter
  - aidenn0 5y ago
    I forgot about that. I think it's nearly all stamped mail over a certain weight? I don't remember if labeled mail is similarly scanned and my google skills are failing me today.
  - JohnFen 5y ago
    It is illegal for the USPS to look at the contents of first class mail except with a court order. Other classes of mail may have less stringent protections. For instance, third class (and maybe second class, I forget) mail can be opened to make sure that the things it contains are allowed to be mailed in that class.
garmaine 5y ago
I don't want to be arrested by the FBI and my life ruined because my daughter, who is 6 and doesn't know any better, takes a selfie with my phone while she happens to be undressed/changing clothes and the photo automatically syncs to the cloud.
- robertoandred 5y ago
  You won’t. Understand what this is instead of falling for the hysteria.
  - garmaine 5y ago
    It starts with comparing file hashes. I’m worried about where this goes next.
    - vmladenov 5y ago
      By that logic, how do you know it hasn’t already gone there, years ago? And this scan is currently happening on the server. They’re just making it so that the scan will now happen at the time of upload as opposed to N seconds later.
- Clubber 5y ago
  Chances are your daughter would be arrested or at least investigated for having child pornography. https://www.theguardian.com/society/2019/dec/30/thousands-of-children-under-14-have-been-investigated-by-police-for-sexting https://www.theguardian.com/society/2019/dec/30/thousands-of...
moonchrome 5y ago
I think the best way to stop child porn is to install government provided monitoring software + require an always online chip to be present on any video capture system and any media player must submit screen hashes + geolocation and owner ID to a government service. If the device fails to submit hashes for more than 24 it should block further operation. Failing this we will never be safe from child porn in the digital world. Not doing anything opens the door to further abuse.
throwrqX 5y ago
The boundary should be that user generated personal data that is on a user device should stay personal on a user device unless explicit consent is otherwise given. That's it. The argument is always well "don't you want to save the children" to which I give this example. In the USA guns are pretty freely available, relative to other countries. There is a huge amount of gun violence (including shooting at schools targeting children) yet every time major gun restriction legislation is introduced it fails, with one major reason being the 2nd amendment of the US constitution. This could again be amended, but sufficient support is not there for this to occur. What does this say about the US? They have determined, as a society, that the value of their rights is worth more than all the deaths, injuries and pains caused by gun violence. A similar argument can be made regarding surveillance and child porn, or really any other type of criminal activity.
- thesuperbigfrog 5y ago
  >> The boundary should be what is on a user device should stay personal on a user device. But how many apps only store the locally on the device versus sending data to the cloud, getting data from the cloud, or calling cloud APIs to get functionality that cannot be provided on device? Having the power of a personal computing device is huge, but having a networked personal computing device is far greater. Keeping everything on-device is a good ideal for privacy, but not very practical for networked computing.
  - throwrqX 5y ago
    Ah, I was talking specifically about user generated personal information. I have edited my post to make it clearer.
stevenicr 5y ago
“Apple is opening the door to broader abuses”. Wouldn't not doing anything open a different door to broader abuse? Actually I believe that in regards to "not doing anything open a different door to broader abuse" - no. Starting scans will lead to broader, and no tin foil hat needed. If we compare 1-apple starts scanning for known hashes, not "looking at all your naked pics and seeing if you've got something questionable"... this is just looking for known / already created by others / historical things... by doing a scan for one thing - they open the pandoras box to start scanning for other things - and then they will be compelled by agents to scan for other things - and I believe that is broader, much. Next month it will be scan for any images with a hash that matches a meme with fauci - the current admin has already stated that in their desire to stop 'disinformation' they want to censor sms text messages and facebook posts (assuming also fbk DMs and more). There is a new consortium of tech co's sharing a list of bad people who share certain pdfs and 'manifestos' or something like that now right? Might as well scan for those docs too, add all them to the list. What power this could lead to - soon the different agencies want scans for pics of drugs, hookers.. how about a scan for those images people on whatsapp are sharing with the black guy killing a cop with a hood on? What happens when a new admin takes the house and white house and demands scans for all the trumped a traitor pics and make a list? See this is where the encryption backdoors go.. and where is that line drawn? Is it federal level agencies that get to scan? can a local arizona county sheriff demand a scan of all phones that have traveled through their land/air space? Frankly, public chats, public forums.. if you post kids or drugs or whatever is not legal there - then it's gonna get the men with guns to take note. What you do in private chats / DMs, etc - I think should stay there and not go anywhere else. I don't like the idea that Msoft employees look at naked pics of my gf that are added to a pics folder because someone setup a win system without disabling one drive. So I don't use one drive and tell others to not use it - and not put pics of me there or on fbk or tictoc. For all those people that have nothing to hide - I feel sorry for you - but wonder if your kids/grandkids should have thousands of agents looking into their private pics just to make sure there is nothing not legal there. so would these scans get nudes sent through whatsapp? That would kill the encryption thing there kind of. Would this scan get a cach if someone was using a chat room and some asshat posted something they shouldn't - and every person in the chat got a pic delivered to their screen. so many questions. I also question what the apple scans would scan as far as folders and what not.. would it scan for things in a browser cache? like not purposefully downloaded.. if someone hit a page that had a setup image on it - would that person now be flagged for inspection / seizing? If they are working with nice guys in Cali that just want to tap people on the shoulder and have a talk with people - will they send flagged notices to agents in other places who may go in with guns drawn and end up killing people? I'm sure many people are fine with either outcome - I think there is a difference between someone surfing the web and someone who coerces real world harm, and not all those who surf the web deserve bullets to the chest, and there is no way to control that.. well maybe in the uk where cops aren't allowed to have guns maybe.
browningstreet 5y ago
There are reasonable measures to take against child abuse, and there are unreasonable ones. If we can’t agree on that, there’s no discussion to be had.
gerash 5y ago
By that logic, can we send in someone into your house every day to look through every corner for child porn in digital or print form? In fact there are a lot of heinous crimes out there some much worse than child porn IMHO. Singling out child porn as the reason seems like it's meant only to elicit an emotional response.
BjoernKW 5y ago
If a company explicitly states in a service's terms and conditions that content stored or shared through that service will be scanned, I think that's acceptable because the user then can decide to use such a service on a case-by-case basis. However, making this a legal requirement or deliberately manipulating a device to scan the entire content stored on that device without the user's consent or knowledge even, is extremely problematic, not just from a privacy point of view. Such power can and will be abused and misused, sometimes purposefully, sometimes accidentally or erroneously. The devastating outcome to innocent people who have been wrongfully accused remains the same in either case (see https://areoform.wordpress.com/2021/08/06/on-apples-expanded-protections-for-children/ https://areoform.wordpress.com/2021/08/06/on-apples-expanded... for a realistic scenario, for example). The very least I'd expect if such a blanket surveillance system were implemented is that there were hefty, potentially crippling fines and penalties attached to abusing that system in order to avoid frivolous prosecution. Otherwise, innocent people's lives could be ruined with little to no repercussions for those responsible. Do strict privacy requirements allow crimes to be committed? Yes, they do. So do other civil liberties. However, we don't just casually do away with those. If the police suspect a crime to have been committed they have to procure a warrant. That's the way it should work in these cases, too.
snowwrestler 5y ago
The conversation today is not really about PhotoDNA (checking image hashes against known bad list). That ship has sailed and most large tech companies already do it. Apple will do it one way or another. It’s a good way to fight the sharing of child porn, which is why it is so widely adopted. The question is whether it is worse to do it on-device, than on the server. That’s what Apple actually announced. I suspect Apple thought doing it on-device would be better for privacy. But it feels like a loss of control. If it’s on the server, then I can choose to not upload and avoid the technology (and its potential for political abuse). If it’s on my locked and managed mobile device, I have basically no control over when or which images are getting checked, for what.
- vmladenov 5y ago
  You have the exact same level of privacy as before. Before, the images were scanned in the cloud. Now they are being scanned as they exit your phone. In that way, the avoidance protocol is the same, namely > I can choose to not upload and avoid the technology They are not scanning your entire photo library at rest. Could they? Sure. But they’ve had the hardware to do this for years, so they could have done so silently at any time. This entire saga has been one of poor messaging and rampant speculation.
JohnFen 5y ago
> If you want unlimited privacy, at least have the balls to admit that this will allow free trade of CSAM, but also revenge porn, snuff-films and other things like it. Sure, in exactly the same way that the postal system, parcel delivery services, etc. allow it. But that's not to say that such things are unrestricted -- there are many ways that investigation and enforcement can be done no matter what. It's just a matter of how convenient that is. It would also restrict CSAM a lot if authorities could engage in unrestricted secret proactive searches of everybody's home, too. I don't see this as being any different.

0xy 5y ago

Evidently the episode you listened to was loaded with false information, because Kik has used PhotoDNA to scan for CSAM for 7 years. [1] [1] https://www.kik.com/blog/using-microsofts-photodna-to-protect-kik-users/ https://www.kik.com/blog/using-microsofts-photodna-to-protec...

donkeyd 5y ago
> Doc: From what I can tell, it only starts looking in the rooms and looking at individual people if they are reported for something. https://darknetdiaries.com/transcript/93/ https://darknetdiaries.com/transcript/93/ If I were Kik, I would also write a blog post about using something like this. Many, many things point at Kik only doing the bare minimum though. (If you're the type who supports moderation, apparently they're already doing too much according to much of HN.)

3rly 5y ago

Should the internet be a free for all place without moderation? Yes!!!

ionwake 5y ago
I think different opinions are great. Just wanted to point out saying the internet should be moderated on the hacker news forums is surprising and funny.
- deleted 5y ago
  [deleted]

toxik 5y ago

I think there is a very deep and troublesome philosophical issue here. Ceci n'est pas une pipe. A picture of child abuse /is not/ child abuse. Let me ask you a counter-question. If I am able to draw child pornography so realistically that you couldn't easily tell, am I committing a crime by drawing?

yunohn 5y ago
What is this absurd counter? Who’s talking about surrealistic drawings? We’re talking about actual material in real life, being shared by users and abusers. To be clear, I’m not supporting surveillance, just stating facts.
- toxik 5y ago
  You should maybe read about the work [0], or just read the rest of what I said. Surrealism has nothing to do with the argument I made, so why do you bring it up? CSAM databases can by necessity not contain novel abuses, because they are novel. In fact, filtering by CSAM databases even indirectly /encourage/ novel abuses because these would not be caught by said filter. Catching CP hoarders does little to help the children being exploited in the first place, and does a lot to harm our integrity and privacy. [0] https://en.wikipedia.org/wiki/The_Treachery_of_Images https://en.wikipedia.org/wiki/The_Treachery_of_Images
  - ElFitz 5y ago
    > Catching CP hoarders does little to help the children being exploited in the first place If there is no market for it, there might be less incentive to produce any more of it. Not that I believe we should all be continuously spied by people who merely pinky sweared to do it right and not abuse it.
    - toxik 5y ago
      I don't think market forces are what drive the production of CSAM. Rather, it's some weird fetish of child abusers to share their conquests. I'm sure you're aware, but when CP collectors are busted, they often have terabytes upon terabytes of CP. But that's, I think, tangential - I don't understand pedophilia well enough to say something meaningful here.

swiley 5y ago

There's a difference a mile wide between moderating a service and moderating people's personal computers. Maybe the same thing that makes this hard to understand about iOS will be what finally kills these absolutely wretched mobile OSes.

donkeyd 5y ago
> moderating people's personal computers This is done client side because your data is encrypted in their cloud. It won't be done if you disable cloud sync. If you just keep your cp out of the cloud, you're fine.
- swiley 5y ago
  TBH it doesn't matter at all why they're doing it, they've crossed a line here.
- new299 5y ago
  Apple can decrypt data stored on iCloud, and they scan it already it seems: https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-icloud-photos-for-child-abuse-images/ https://nakedsecurity.sophos.com/2020/01/09/apples-scanning-... Which it what makes adding the client side functionality even more problematic. They can easily extend this to scanning all offline content.
tda 5y ago
except that the boundary between device and service is steadily eroding. A screen is a device and netflix is a service, but a smart tv that won't function without an internrt connection? A nokia was a device you owned, but any Apple device is more like a service than a device really, as you don't control what OS can run on it, which unsigned software you run etc. So if you are ok with moderation on services and your device turns into a service... Perhaps we need laws to fully separate hardware and software but actually the integration between hardware and software is becoming stronger and stronger every year

h0mie 5y ago

Should all personal computers be scanned all the time by windows/apple/... in case it contains cp?

donkeyd 5y ago
Are you unable to answer any of the questions I asked? I'm seriously interested in hearing, from people who think Apple is in the wrong, where the border of acceptability is. Would you prefer iOS submitted the photo's to their cloud unencrypted and Apple scanned them there? Because that's what the others are doing.
- veeti 5y ago
  iCloud photos are not end-to-end encrypted, Apple has full access to them. https://support.apple.com/en-us/HT202303 https://support.apple.com/en-us/HT202303

sebzim4500 5y ago

Fundamentally there is a difference between scanning files on your servers and scanning files stored on someone else's device, without their consent.

avnigo 5y ago
Not that I wholly agree with Apple's move due to other implications, but it has to be said that only photos that you have chosen to upload to iCloud will be scanned. So, in a sense, you do consent to having those photos scanned by using iCloud. Maybe it's time for Apple to drop the marketing charade on privacy with iCloud, since iCloud backups aren't even encrypted anyway.

ianhawes 5y ago

Many people share photos of child pornography via the mail. There has been criticism that the USPS does not open all mail and scan them for child pornography. I would really like to hear from people who do not sign this letter how they think about that.

donkeyd 5y ago
I think it would be horrible if they opened all mail... However, if they had a system that could detect CSAM in unopened mail with an extremely low false negative rate, then I'd be fine with it. With many packages this already happens in the search for drugs and weapons and I have no problem with that either.
- stevenicr 5y ago
  I believe millions more nudes are sent via snap/watsap/etcetc every day then are sent in the mail in a year.. with "extremely low false negative rate" - if that means there are a bunch of geeks and agents staring at your wife/gf.. your daughter/ etc.. is that okay? Some people will say yes - but I don't think those people should get to decide that all the other people in the world have to hand over all the nudes and dick pics everyone else has taken. when they open mail with drugs and weapons it's most likely not a sculpture of your wife's genitals / daughter's tits, pics of your hemorrhoids / erectile implant - whatever other private pic stuff that no one else should be peering at. If they were opening and reading every written word sent and received - I can guarantee you people would be up in arms about it.
  - donkeyd 5y ago
    > I believe millions more nudes are sent via snap/watsap/etcetc every day then are sent in the mail in a year. I know for sure this is true, but this BS scenario was still used as a rebuttal against my comment, so I figured I'd give them an actual answer. > when they open mail with drugs and weapons it's most likely not ... other private pic stuff that no one else should be peering at. People order a lot of stuff that is very private online. From self help books, to dilos (which show up clearly on x-ray), to buttless chaps. If anything, packages are probably more private than mail these days, since people only mail postcards and bills. > If they were opening and reading every written word sent and received - I can guarantee you people would be up in arms about it. They would. But this is completely incomparable to what's happening on iOS devices now.

simion314 5y ago

My personal opinion is if we could spy on all citizens all the time we could stop all or most of the crimes, do we want this? If you say Yes then stop reading here. Else if you say 100% surveillance is too much then what do you have against people discussing where the line should be drawn? Some person that would sign that letter might be fine with video cameras in say a bank or some company building entrance but he is probably not fine with his own phone/laptop recording him and sending data to soem company or government without his consent. So let's discuss where should the line be drawn, also if competent people in this domain are present let's discuss better ideas on preventing or catching criminals, or even for this method let's discuss if it can be done better to fix all the concerns. What I am sure is that clever criminals will not be affected by this, so I would like to know if any future victim would be saved (I admit I might be wrong, so I would like to see more data from different countries that would imply the impact of this surveilance)

croes 5y ago
This only finds known pictures of child abuse not new ones and especially it doesn't find the perpetrators or prevent the abuse. But it creates a infrastructure for all other kind of "criminal" data. I bet sooner or later governments want to include other hashes to find the owners of specific files. Could be bomb creation manuals, could be flyers against a corrupt regime. The sky is the limit and the road to hell is paved with good intentions.
- heavyset_go 5y ago
  > This only finds known pictures of child abuse not new ones No, it finds images that have similar perceptual hashes, typically using the hamming distance or another metric to calculate differences between hashes. I've built products that do similar things. False positives are abound with perceptual hashes, especially when you start fuzzy matching them.
  - croes 5y ago
    But these are just variations of known pictures to recognize them if they cropped or scaled. The hash of a really new picture isn't similar to the hash of a known picture.

digitalsushi 5y ago

The big reason I don't want someone else scanning my phone (and a year later, my laptop) is that last I checked, software is not perfect, and I don't need the FBI getting called on me cause my browser cache smelled funny to some Apple PhD's machine-learning decision. It's the same reason I don't want police having any opportunistic information about me - both of these have an agenda, and so innocent people get pulled into scenarios they shouldn't be when that information becomes the unlucky best-fit of the day. That Apple has even suggested this is disturbing because I have to fully expect this is their vision for every device I am using. And then I expect from there, it's a race to the bottom until we treat our laptops like they are all live microphones.

robertoandred 5y ago
The chance of you getting enough false positives to reach the flagging threshold is one in a trillion. And then the manual review would catch the error easily. The FBI won’t be called on you.
- chopin 5y ago
  That's a claim by Apple. Due to the opaque process this is completely unverifiable. As well this statement relies on the fact that no malicious actor out there is trying to thwart the system. The hashes used are deliberately chosen to easily produce collisions otherwise the system won't work. This almost certainly will be abused.
  - robertoandred 5y ago
    “This almost certainly will be abused.” is a claim by you and completely unverifiable. Apple has published white papers explaining how their hashing, matching, and cryptography work. How do you see someone thwarting that process specifically?
    - drampelt 5y ago
      Apple's claims on how it will work are also completely unverifiable. What's stopping a government from providing Apple with hashes of any other sort of content they dislike?

shoulderchipper 5y ago

> Should the internet be a free for all place without moderation? The better question would be: do you want an arbitrary person (like me) to decide whether you have a right to send an arbitrary pack of bytes? Neither "society" nor "voters" nor "corporations" make these decisions. It is always an arbitrary person who does. Should one person surrender his agency into the hands of another?

krapp 5y ago
>Neither "society" nor "voters" nor "corporations" make these decisions. It is always an arbitrary person who does. Except in this case, a corporation (Apple) is making the decision relative to the sexual mores of modern Western society and the child pornography laws of the United States. It's unlikely this decision was made and implemented randomly by a single "arbitrary" individual. Contrary to your claim, it's never an arbitrary person. And yes, I believe Apple has the right to decide how you use their product, including what bytes can and cannot be sent on it. >Should one person surrender his agency into the hands of another? We do that all the time, that's a fundamental aspect of living in a society. But in this specific case, no one is forcing you to use an Apple phone, so you're not surrendering your agency, you're trading it in exchange for whatever convenience or features lead you to prefer an Apple product over competitors. That's still your choice to make.
- WWLink 5y ago
  "Apple has the right to decide how you use their product" I hate this argument. If Apple wants to claim ownership of _their_ products then they shouldn't sell them. They should lease them.

aidenn0 5y ago

I won't use a service that performs dragnet surveillance on my communication. The US Postal service does not open every single letter to check if it contains illegal material. If I rent a storage unit, the storage company does not open up my boxes to see if I have albums of child pornography. This move by Apple is equivalent. I'm going to turn this around and say that those in favor of Apple's move: "Should the internet be a place where your every move is surveilled?" given that we have some expectation of privacy in real life.

donkeyd 5y ago
> The US Postal service does not open every single letter to check if it contains illegal material. You'd be surprised about the amount of packages that gets x-rayed in order to find drugs. But yes, you're 100% right that it's not all of them.
- silverpepsi 5y ago
  An x-ray is nothing like reading the contents of letters or randomly checking ahipped harddrives and USB sticks for content. I don't know how to clarify legally or conceptually, but I feel confident there is a very clear difference
  - emodendroket 5y ago
    Perhaps I'm being too cynical but I think the difference is they haven't figured out a convenient, automated way to do the latter
- aidenn0 5y ago
  I forgot about that. I think it's nearly all stamped mail over a certain weight? I don't remember if labeled mail is similarly scanned and my google skills are failing me today.
- JohnFen 5y ago
  It is illegal for the USPS to look at the contents of first class mail except with a court order. Other classes of mail may have less stringent protections. For instance, third class (and maybe second class, I forget) mail can be opened to make sure that the things it contains are allowed to be mailed in that class.

garmaine 5y ago

I don't want to be arrested by the FBI and my life ruined because my daughter, who is 6 and doesn't know any better, takes a selfie with my phone while she happens to be undressed/changing clothes and the photo automatically syncs to the cloud.

robertoandred 5y ago
You won’t. Understand what this is instead of falling for the hysteria.
- garmaine 5y ago
  It starts with comparing file hashes. I’m worried about where this goes next.
  - vmladenov 5y ago
    By that logic, how do you know it hasn’t already gone there, years ago? And this scan is currently happening on the server. They’re just making it so that the scan will now happen at the time of upload as opposed to N seconds later.
Clubber 5y ago
Chances are your daughter would be arrested or at least investigated for having child pornography. https://www.theguardian.com/society/2019/dec/30/thousands-of-children-under-14-have-been-investigated-by-police-for-sexting https://www.theguardian.com/society/2019/dec/30/thousands-of...

moonchrome 5y ago

I think the best way to stop child porn is to install government provided monitoring software + require an always online chip to be present on any video capture system and any media player must submit screen hashes + geolocation and owner ID to a government service. If the device fails to submit hashes for more than 24 it should block further operation. Failing this we will never be safe from child porn in the digital world. Not doing anything opens the door to further abuse.

throwrqX 5y ago

The boundary should be that user generated personal data that is on a user device should stay personal on a user device unless explicit consent is otherwise given. That's it. The argument is always well "don't you want to save the children" to which I give this example. In the USA guns are pretty freely available, relative to other countries. There is a huge amount of gun violence (including shooting at schools targeting children) yet every time major gun restriction legislation is introduced it fails, with one major reason being the 2nd amendment of the US constitution. This could again be amended, but sufficient support is not there for this to occur. What does this say about the US? They have determined, as a society, that the value of their rights is worth more than all the deaths, injuries and pains caused by gun violence. A similar argument can be made regarding surveillance and child porn, or really any other type of criminal activity.

thesuperbigfrog 5y ago
>> The boundary should be what is on a user device should stay personal on a user device. But how many apps only store the locally on the device versus sending data to the cloud, getting data from the cloud, or calling cloud APIs to get functionality that cannot be provided on device? Having the power of a personal computing device is huge, but having a networked personal computing device is far greater. Keeping everything on-device is a good ideal for privacy, but not very practical for networked computing.
- throwrqX 5y ago
  Ah, I was talking specifically about user generated personal information. I have edited my post to make it clearer.

stevenicr 5y ago

“Apple is opening the door to broader abuses”. Wouldn't not doing anything open a different door to broader abuse? Actually I believe that in regards to "not doing anything open a different door to broader abuse" - no. Starting scans will lead to broader, and no tin foil hat needed. If we compare 1-apple starts scanning for known hashes, not "looking at all your naked pics and seeing if you've got something questionable"... this is just looking for known / already created by others / historical things... by doing a scan for one thing - they open the pandoras box to start scanning for other things - and then they will be compelled by agents to scan for other things - and I believe that is broader, much. Next month it will be scan for any images with a hash that matches a meme with fauci - the current admin has already stated that in their desire to stop 'disinformation' they want to censor sms text messages and facebook posts (assuming also fbk DMs and more). There is a new consortium of tech co's sharing a list of bad people who share certain pdfs and 'manifestos' or something like that now right? Might as well scan for those docs too, add all them to the list. What power this could lead to - soon the different agencies want scans for pics of drugs, hookers.. how about a scan for those images people on whatsapp are sharing with the black guy killing a cop with a hood on? What happens when a new admin takes the house and white house and demands scans for all the trumped a traitor pics and make a list? See this is where the encryption backdoors go.. and where is that line drawn? Is it federal level agencies that get to scan? can a local arizona county sheriff demand a scan of all phones that have traveled through their land/air space? Frankly, public chats, public forums.. if you post kids or drugs or whatever is not legal there - then it's gonna get the men with guns to take note. What you do in private chats / DMs, etc - I think should stay there and not go anywhere else. I don't like the idea that Msoft employees look at naked pics of my gf that are added to a pics folder because someone setup a win system without disabling one drive. So I don't use one drive and tell others to not use it - and not put pics of me there or on fbk or tictoc. For all those people that have nothing to hide - I feel sorry for you - but wonder if your kids/grandkids should have thousands of agents looking into their private pics just to make sure there is nothing not legal there. so would these scans get nudes sent through whatsapp? That would kill the encryption thing there kind of. Would this scan get a cach if someone was using a chat room and some asshat posted something they shouldn't - and every person in the chat got a pic delivered to their screen. so many questions. I also question what the apple scans would scan as far as folders and what not.. would it scan for things in a browser cache? like not purposefully downloaded.. if someone hit a page that had a setup image on it - would that person now be flagged for inspection / seizing? If they are working with nice guys in Cali that just want to tap people on the shoulder and have a talk with people - will they send flagged notices to agents in other places who may go in with guns drawn and end up killing people? I'm sure many people are fine with either outcome - I think there is a difference between someone surfing the web and someone who coerces real world harm, and not all those who surf the web deserve bullets to the chest, and there is no way to control that.. well maybe in the uk where cops aren't allowed to have guns maybe.

browningstreet 5y ago

There are reasonable measures to take against child abuse, and there are unreasonable ones. If we can’t agree on that, there’s no discussion to be had.

gerash 5y ago

By that logic, can we send in someone into your house every day to look through every corner for child porn in digital or print form? In fact there are a lot of heinous crimes out there some much worse than child porn IMHO. Singling out child porn as the reason seems like it's meant only to elicit an emotional response.

BjoernKW 5y ago

If a company explicitly states in a service's terms and conditions that content stored or shared through that service will be scanned, I think that's acceptable because the user then can decide to use such a service on a case-by-case basis. However, making this a legal requirement or deliberately manipulating a device to scan the entire content stored on that device without the user's consent or knowledge even, is extremely problematic, not just from a privacy point of view. Such power can and will be abused and misused, sometimes purposefully, sometimes accidentally or erroneously. The devastating outcome to innocent people who have been wrongfully accused remains the same in either case (see https://areoform.wordpress.com/2021/08/06/on-apples-expanded-protections-for-children/ https://areoform.wordpress.com/2021/08/06/on-apples-expanded... for a realistic scenario, for example). The very least I'd expect if such a blanket surveillance system were implemented is that there were hefty, potentially crippling fines and penalties attached to abusing that system in order to avoid frivolous prosecution. Otherwise, innocent people's lives could be ruined with little to no repercussions for those responsible. Do strict privacy requirements allow crimes to be committed? Yes, they do. So do other civil liberties. However, we don't just casually do away with those. If the police suspect a crime to have been committed they have to procure a warrant. That's the way it should work in these cases, too.

snowwrestler 5y ago

The conversation today is not really about PhotoDNA (checking image hashes against known bad list). That ship has sailed and most large tech companies already do it. Apple will do it one way or another. It’s a good way to fight the sharing of child porn, which is why it is so widely adopted. The question is whether it is worse to do it on-device, than on the server. That’s what Apple actually announced. I suspect Apple thought doing it on-device would be better for privacy. But it feels like a loss of control. If it’s on the server, then I can choose to not upload and avoid the technology (and its potential for political abuse). If it’s on my locked and managed mobile device, I have basically no control over when or which images are getting checked, for what.

vmladenov 5y ago
You have the exact same level of privacy as before. Before, the images were scanned in the cloud. Now they are being scanned as they exit your phone. In that way, the avoidance protocol is the same, namely > I can choose to not upload and avoid the technology They are not scanning your entire photo library at rest. Could they? Sure. But they’ve had the hardware to do this for years, so they could have done so silently at any time. This entire saga has been one of poor messaging and rampant speculation.

JohnFen 5y ago

> If you want unlimited privacy, at least have the balls to admit that this will allow free trade of CSAM, but also revenge porn, snuff-films and other things like it. Sure, in exactly the same way that the postal system, parcel delivery services, etc. allow it. But that's not to say that such things are unrestricted -- there are many ways that investigation and enforcement can be done no matter what. It's just a matter of how convenient that is. It would also restrict CSAM a lot if authorities could engage in unrestricted secret proactive searches of everybody's home, too. I don't see this as being any different.