8 ms·
It's incredibly disturbing to me how little much of the HN crowd seems to understand how these systems like CSAM actually work. I'm not talking about the tech
by citizenpaul 5y ago
It's incredibly disturbing to me how little much of the HN crowd seems to understand how these systems like CSAM actually work. I'm not talking about the tech or programming.
This is a fast track tool to putting anyone in jail for the most despised crime a person can commit. The tool has no oversight, no transparency, no counter, no clear your name, no real usage definitions other than relentless spying on the "protect the children" fallacy.
Most people don't understand how life destroying a false accusation is once the government paperwork is started. At that point like it or not you are basically considered guilty by all the agents of the governement by default. Their jobs are to prosecute not eliminate false positives. Once you have been on their radar your life will never ever be the same. And since it is CP related you will not even get helpful people willing to touch it with a 10ft pole cause of fear they may be implicated. CSAM is a dastardly and effective way to establish a totalitarian rule by fear of life in jail, nothing more.
- junon 5y ago> It's incredibly disturbing to me how little much of the HN crowd seems to understand how these systems like CSAM actually work. Dunno. I remember most people on HN were absolutely pissed this was even being considered. My takeaway is that most people here understand it to a large enough degree that they don't like it.
- _fat_santa 5y agoStuff like this gives me pause in the tech industry. My brother is in med-school and always joke with him that the worst thing I can at work is take a website down and the worst thing he can do is kill someone. But with tech like this, you can literally kill someone with it. Sure you're not actually "killing" anyone, but the decisions these systems make can drive someone to end their life, whether or not the accusations by the machine are true. I wouldn't want to be the guy building that software.
- commandlinefan 5y ago> can drive someone to end their life Don't underestimate the possibility of somebody else ending their lives for them in a fit of righteous rage.
- jareklupinski 5y agolest we forget > Because of concurrent programming errors (also known as race conditions), it sometimes gave its patients radiation doses that were hundreds of times greater than normal, resulting in death or serious injury. https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25
- Spivak 5y agoYou should at least be somewhat charitable to the other side of this issue which is that CSAM is the result of real life sexual abuse and the market for it causes more. Unless you want a cop stationed every 10 feet you can't stop the abuse, you can only take away the incentive to do it in the first place which is the theory behind most laws. The reason most people aren't constantly worried about being murdered isn't because we have super advanced defense systems but because the fear of punishment keeps people from doing it. I'm aware that the majority of HN want the solution to be "do nothing" and some quote about liberty and security but you would probably change your tune if you or your kids were trafficked, starved, raped, and the had the pictures and videos distributed to fetishists online. CSAM detection is the government's best option to kill the market for it and make it radioactive. At least listen to the experience of human trafficking survivors before you say it's all a big conspiracy. Those scary signs on buses, trains, in women's bathrooms, and doctor's offices aren't a joke.
- gigglesupstairs 5y agoYou are making it sound all philosophical when HN crowd is already philosophically behind finding an appropriate solution to the CP problem. “Problem” is not a problem here, current proposed solution is. See it critically and not emotionally.
- Spivak 5y agoSee this is always how online discussions go. * "CSAM is -- nothing more -- than a way for governments to establish a totalitarian rule!" * "Oh come on, that's really uncharitable to the people implementing CSAM detection, at least present the argument for the other side in the way they view the issue." [loud screeching of goalposts...] * "Of course we acknowledge the problem of sexual abuse imagery, it's just that we don't like the CSAM detection as a solution... [aaand have no ideas for an effective privacy preserving alternative, and want nothing done in the interim while we search for the unicorn]." There becomes a point where if you don't want to implement the best idea we've got to address the problem, and don't have any other ideas, then it gets harder to believe you really care about the problem.
- xeromal 5y agoDude every single comment in the post when this was announced was against it.
- Kye 5y agoThere were a few "why are you defending predators?" type comments.
- Nginx487 5y agoAnd "I have nothing to hide!" type comments, from few voyeurists.
- commandlinefan 5y ago> from few voyeurists In this case, wouldn't that be exhibitionists?
- stjohnswarts 5y agoAnd the bUt yOU dOn'T uNderStAnd tHe tEchNolOgy! Really? a communuity of programmers, engineers, and scientists can't understand technology? It's so preposterous.
- citizenpaul 5y agoThere were several threads about it. As they increased there was more and more debate. The fact that there is debate at all is indication of support. It also mirrors what they are anticipating. An initial backlash then less and less as they continually push for it since they are literally getting paid for it and we have to fight back on our own time.
- bradgessler 5y agoWhat's disturbing is that nothing has materially changed. For those of us who are running iOS or Android, we're just an OS update away from CSAM landing on our devices again. It's just a matter of time. The question is what will it be renamed to and under what pretext will it be deployed?
- bradgessler 5y agoEugh, yep. Apple pretty much confirms its coming in the future: “Update: Apple spokesperson Shane Bauer told The Verge that though the CSAM detection feature is no longer mentioned on its website, plans for CSAM detection have not changed since September, which means CSAM detection is still coming in the future.” Expect a much quieter roll out.
- speg 5y agoMy bet is they will settle for a similar solution to other providers and scan the photos on their servers.
- vbezhenar 5y agoDo you have any reasons to think that they're not scanning photos on their servers? AFAIK every major storage cloud does that, including Apple. What probably happened is that Apple planned to introduce E2E encryption for iCloud. But they have to scan photos, so they rolled out client scanning to ensure that they're not storing encrypted child porn on their servers.
- miles 5y ago> Do you have any reasons to think that they're not scanning photos on their servers? AFAIK every major storage cloud does that, including Apple. Apple already scans iCloud Mail for CSAM, but not iCloud Photos https://9to5mac.com/2021/08/23/apple-scans-icloud-mail-for-csam/ https://9to5mac.com/2021/08/23/apple-scans-icloud-mail-for-c...
- bradgessler 5y ago
- jodrellblank 5y agoThing you haven't mentioned is that with no oversight, no transparency no counter, no clear your name, the tyranny which wants to ruin your life doesn't need to build a CSAM scanner to do it. A tip-off to the police, a falsified claim from an "eye witness", an "accident" putting your name on the wrong report to get the machinery rolling and it will, as you say, roll by itself. See what actually did happen to Sir Cliff Richard[1] where "a complaint to the police" got his house raided and his name dragged through the media and despite not being arrested, not being charged, case being dropped, him suing the media and winning, suing the police and winning and the media apologising, he says his reputation will never be completely cleared - and it probably never will be, and it didn't require any specific tech. "The System" didn't need to wait patiently for a big tech company to build a complicated system with dastardly moustache-twirling weaknesses before it could affect Sir Cliff, anymore than Big Brother in 1984 needed to wait for Winston to be seen taking action on the telescreen in his house before the system crushed him. If "The System" wanted to accuse you, it could accuse your car of having been seen on a nearby road at the time, your likeness being seen on CCTV, and if you get rid of licence plate recognition scanners and CCTV then it would be "an anonymous witness" seeing you in the area at the time or "overhearing you discussing" something. > "Most people don't understand how life destroying a false accusation is" Citation needed? Because we were forced to read 1984 and To Kill a Mockingbird in school and I'm pretty sure a lot of other people were as well; we've seen cult TV series like The Prisoner and The Fugitive focusing on the lives of the trapped trying to escape "the system". Blockbuster films have been made about the power of "The State" chasing people such as Enemy of the State with Will Smith. Films in the pattern "wrongly accused must fight to clear his name" and "stalwart good guy fights the system gone bad" are practically genres on their own - The Fugitive as a film with Harrison Ford, Who Framed Roger Rabbit, Captain America: The Winter Soldier, The Shawshank Redemption, and specifically about someone falseley accused by a child, The Hunt with Mads Mikkelsen which is in the IMDB Top 250 films. Are you also, for example, complaining about the system where doctors and social workers are mandatory reporters if they have any evidence or suspicion of children being abused? That's a "their word against yours" system which pervades the nation, has no oversight, has no objective criteria, no explanaiton of the workings of the brain behind it, relies on humans when we know for certain that there are doctors who make mistakes and errors of judgement and at least some who would look the other way for money. At least with the tech company CSAM scanners we can potentially have a society wide discussion about how they work in detail, whether that's reasonable, who could and should oversee them and how, what kinds of data cloud companies are obliged to know about their customers and what is reasonable for them to not know, etc. But even in tech circles we can't actually have those discussions, most of the attempts to discuss the details of how it works are drowned out by people who haven't a clue what Apple announced, misunderstand massive parts of it, conflate two or three different systems, don't seem to understand that other tech companies already do this with much less careful design, and cry tyranny and blame China. Hypothetically, iMessage or any chat system could be built end to end encrypted, any cloud file sync or file storage could be. One end of the spectrum is that they should be. If that's your position then "the system" will probably work around you, that's the "opt out of modern life" position of the "return to the woods" lifestyle. You can't unmake billions of cheap ubiquitous cameras, you can't unopen Pandora's box, and you probably can't stop people emotionally reacting strongly to topics of child protection. If your reaction to that is "not listening, no no no" then it doesn't seem likely to sway "the system" very much at all. > "CSAM is a dastardly and effective way to establish a totalitarian rule by fear of life in jail, nothing more. America has effectively incarcerated a lot of people into for-profit prisons without needing this. China has effectively added a lot of population control by rule of fear without needing this. You don't need a high-tech murder-detector to make people fear being thrown in jail for murder. [1] https://en.wikipedia.org/wiki/Cliff_Richard#Property_search,_investigation_and_BBC_litigation https://en.wikipedia.org/wiki/Cliff_Richard#Property_search,...
- GuB-42 5y agoI hate this thing as much as the rest of HN but I think you are thinking too far. CSAM is just a tool that matches image hashes. It can't make accusations. Matching hashes is not illegal, even having pictures of abused children is not illegal, no prosecutor will take your case if it is the only thing you have. At most it will turn you into a witness. If there is an investigation, it can turn you into a suspect, but suspicion is not an accusation. I mean, if your car is parked next to the place a crime happened, you can be suspected, but it doesn't mean you are going to jail because of where you parked your car. If, for some reason, you get wrongly accused because the system recorded a match and nothing else, it is not a problem of the tool, it is the much bigger problem of justice not working, blame your shithole country if it happens, not Apple. I am not saying that false accusations don't happen, but saying that police has no other job than to prosecute is a bit much. Police asked me questions a few times, once for a car that caught fire not far away from where I was, once for a missing person, and once they didn't tell me the reason. Was I suspected? Maybe, I was at the "wrong place" after all, but yet, I never heard from them again. If prosecution was their only goal, I would have been in jail now.
- kobalsky 5y ago> CSAM is just a tool that matches image hashes. It can't make accusations who validates the source content of the hashes? what stops the government from adding one tiny bitty hash that they know is uniquely present in let's say, assange's phone?
- GuB-42 5y agoAgain a "what if" scenario. It is not trivial to identify a picture that is uniquely present in Assange's phone. Without having access to Assange's phone that is. And if you have access to Assange's phone, you probably have way more information than such a trick will give you. But yes, it is a problem, and a reason why I dislike CSAM. But from "allowing the government to track highly sensitive individuals" to "you, the average citizen, will go to jail", there is a huge gap. Also, I think one thing many people missed is that a hash is not enough. The only thing a hash does it that it allows a reviewer at Apple to decrypt a matching file. If the picture turns out to be nothing special, nothing will happen. In fact, that feature is much weaker than people make it, it just puts Apple on par with what others like Google, Microsoft, etc... can do. I think the reason it got so much negative press is that Apple made privacy a big selling point. I think it serves them right, I don't like how Apple treats privacy as just a way to attack Google without really committing to it, but still, for the facts, it is a bit unfair.
- mikotodomo 5y ago> This is a fast track tool to putting anyone in jail for the most despised crime a person can commit. No? You literally have to have CHILD PORNOGRAPHY on your device to get reported. This is LITERALLY one of the biggest crimes you can commit. What's wrong with having your house raided once that happens? If they break anything they will pay for it. This is a small price to pay to stop children from being raped. You literally cannot go to jail unless they look through your data and verify if you did anything wrong.
- pnt12 5y agoSome valid concerns are: * False positives - the visual hashing algorithm has already been broken and people were able to force collisions. * Dictatorships demanding the addition of "problematic" pictures to the CSAM database, ie uncomfortable events they'd prefer the population to stop talking about.
- mikotodomo 5y agoWell the first point is a problem if real, because we don't want police to have to keep raiding people for no reason. But, after they fix the code that shouldn't be a problem. I thought it was a hash like SHA256? The second point is unrealistic and seems like a last ditch attempt to shoot down a good idea.
- stjohnswarts 5y agoI agree with you completely. We know exactly how it works. What we object to is Apple acting as a police agent without a warrant. It's as simple as that. It sets up a precedent to do all sorts of similar policing (without reason for suspicion or a warrant) for everything from DMCA to thought crime. We are not luddites. We are also not suckers for the police state and authoritarianism creep.