18 ms·

Princeton Researchers Who Built a CSAM Scanning System Urge Apple to Not Use It

314 points by nathan_phoenix 5y ago

atlgator 5y ago
> A foreign government could, for example, compel a service to out people sharing disfavored political speech. That's no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials. This sums up the concern. CSAM is just the excuse because who would come out against protecting children, right? But this will absolutely be used for political purposes.
- temp8964 5y ago
  I don't understand this logic, why would authoritarian governments need to fake political dissident material as child porn for matching? The examples you quoted directly invalidate this argument. The authoritarian governments can directly ban political dissident materials, and they don't need child porn as the excuse.
  - deleted 5y ago
    [deleted]
  - ls612 5y ago
    Because the real concern isn’t that; it’s something like Apple vs FBI where the fact that Apple didn’t have a capability built have them a legal edge against a government with stronger rule of law.
  - derefr 5y ago
    iMessage is E2E-encrypted; the only scanning of the content of the network a state adversary can do is by getting Apple to implement it for them, into the iMessage client. And Apple until now have had the excuse for not doing so, that they have no mechanism built into iMessage that would or could scan for such things, nor (as the excuse would go) is such a technology likely to be either “feasible” given the E2E-encrypted nature of the communications, or “palatable” to their audience (i.e. an argument like “people buy the iDevices in part because of advertised privacy benefits, so such state-level intrusions would lead to nobody buying iDevices, so nobody would be caught by this scheme anyway, so all you’re really doing by asking us to implement it is trying to destroy our company’s share price, probably to help your domestic phone market—and our government wouldn’t take kindly to that kind of market interference. So if you don’t want to be slapped with even more Huawei-like trade sanctions, kindly go away.”) The implementation of a CSAM scanner is a clear feasibility proof for the set of technologically-equivalent capabilities (e.g. dissident-material scanning.) The fact that Apple have implemented it, and that their users have accepted it and continued to use their iDevices, means Apple has no leg to stand on when denying governments the implementation of technological enforcement for “banning political dissident materials.”
    - dwaite 5y ago
      iMessage does not use the CSAM scanner, only the upload to iCloud Photos does. The iMessage feature is a local porn-ish detector on sent/received images with a click-through. It is only for children on family accounts, and if the child is under 13 a click-through is reported to the parent.
- gjsman-1000 5y ago
  The reason I think Apple went forward with it though is that, from their perspective, it's not like they are building a new tool for surveillance. It doesn't take many brain cells for a lawmaker to realize that they could mandate pre-screening of content. From Apple's perspective... for authoritarian governments like China or India... they are already able to mandate it and are likely to. So they shouldn't be factored into the "CSAM Scanning could be abused!" argument because it was already happening and going to happen, whether the tool exists or not. In which case, releasing the CSAM tool has only benefits for the abused and doesn't make a difference in preventing surveillance and privacy invasions because it was already going to occur. A cynical view but a possibility.
- api 5y ago
  The more I think about it, something that concerns me greatly is the silent use of this for political purposes. By silent I mean they don't arrest you or put you on any public lists. Instead they just gather information, lots of information, and they use it in future Cambridge Analytica style propaganda campaigns. I don't mean to imply that only Republicans would do this. After Trump's successful use of this kind of political propaganda every single political party and PR agency on the planet is working on duplicating it. Arresting people is old fashioned. Modern totalitarianism is built on disinformation, microtargeted propaganda, surveillance, and nudge theory.
  - certeoun 5y ago
    What makes us better than China then? Our relative freedoms? What can we do about it? Princeton also did a study on America not being actually a democracy. I shared it here: https://news.ycombinator.com/item?id=28249091 https://news.ycombinator.com/item?id=28249091
    - DaftDank 5y ago
      Sorry for the off-topic question, but I've been wondering for a while and have been too afraid to ask: what does the green name signify here?
      - zargon 5y ago
        It means they're a newly registered user. (Not sure of the exact threshold, around a few days to a week.)
    - southerntofu 5y ago
      Nothing makes a country better than another, in absolute. Although, some aspects of life may be more enjoyable in one or another, depending on your taste. I think most of us can agree living in an abundance of resources (waste!) with some mild political freedom where you're allowed to say anything as long as you don't try to change anything (see: COINTELPRO, Julian Assange, etc) like in the Global North, is more pleasant to live on a daily basis than a hunger-ridden autocracy like North Korea. But even that is debatable and i'm sure many north koreans would disagree. What can we do about it? Attack the system, day by day and bits by bits. Steal what you can from the rich to redistribute to the poor. Make cooperatives, whether as employees or volunteers, so that more and more people can quit wage exploitation and start to live again, "from each according to their capabilities, to each according to their needs" (old anarchist saying). Form collectives for all kinds of struggles affecting you and your loved ones: anti-patriarchy, anti-racism, accessibility... Don't accept anyone claiming they're above you. We're all in this together, and those who claim to seek authority to find solutions will create more problems than they will solve, no matter their good intentions. We need to build "power to the people", not "power over the people", as anyone who has studied some history of political repression in the early Soviet Union (and other "dictatorships of the proletariat") can learn. See also Emma Goldman's "Trotsky protests too much" or "There is no communism in Russia" on this topic. Organize. Organize. Organize.
- robertoandred 5y ago
  Apple's system doesn't match content.
  - vletal 5y ago
    You can do stuff like listing top K trending anti gov memes past day and blacklisting them. Thats definitely a content.
    - rgovostes 5y ago
      You cannot, because the models are shipped the OS image. Apple would have to publish a new OS update every day new memes come out. If you're talking about photo library scanning, people would have to be saving the memes to their photo libraries, cloud syncing would have to be on, and they would have to save lots of them. If you're talking about using the iMessage content scanning, this only applies to under-13 accounts, and it only notifies parents associated with the account. If you are suggesting that Apple would be compelled to modify either of these to do something else, why would they bother with these, which are built with many constraints and multiple levels of failsafes, rather than just implement a new feature? Governments could ban end-to-end encrypted chat like iMessage. They could demand the next iOS update uploads your saved social media passwords to them. It's not very plausible that anyone would want to modify _these_ features rather than just implement something more direct and harder to evade. Governments could make these demands at any point. They do not need to wait for this feature and then spring into action. The Chinese government did regarding where iCloud data is hosted for its citizens. The US government did regarding the San Bernardino iPhone case, and Apple fought back very publicly in court that any kind of modification of the OS was unacceptable.
      - Jcowell 5y ago
        The image saving is migrated by using apps that autosave received images by default. I.E WhatsApp.
      - vletal 5y ago
        sed -i "s/day/month/" parent
- sixothree 5y ago
  This event here marks the end of privacy as a right. It’s over.
  - lstamour 5y ago
    Privacy can only be maintained by policies and regulation, not by technology. This hashing and scanning client-side, broken as it might be, is merely a technology. Technology to ensure privacy, as established by corporations, can always be legally circumvented by governments and what they allow or require by policy for their region, if said government is part of a large enough market that companies have to pay attention to it. Therefore the only solution is global agreement to privacy as a human right… and that seems unlikely right now. Runner up would be individual alternative apps for privacy (such as a camera app) but those can be banned by policy depending on the region you’re in, or potentially subject to the same rules/policies. In a democracy, if you disagree, your only option is talking to lawmakers, those who decide on policy. But again, it would likely only affect your region…
  - ipaddr 5y ago
    Apple brand is privacy against other companies competing with them. Apple has always had privacy concerns.
  - entropicdrifter 5y ago
    It's been over for a long while for the average consumer
    - rowanG077 5y ago
      This is definitely false. You can relatively easily live with privacy. It just takes a little bit of effort. It's over if you don't care and take no action.
      - entropicdrifter 5y ago
        The average consumer is largely unaware that they've lost their privacy and if they are aware they mostly fail to grasp the severity. The average consumer already doesn't care and has taken no action.
- sam0x17 5y ago
  > pre-screening Pre-screening would be the kind way of implementing this feature. They could make it so if CSAM is detected, just don't upload those files to the cloud and delete them locally. Then they are still doing due-diligence to prevent that material from entering their cloud and to remove that material from circulation, but they don't get random people arrested for false positives. When they detect hash matches on a device, they are under zero legal obligation to do anything about it because it could easily be a false positive (as the public has demonstrated on HN and elsewhere), so they'd be in the clear with this approach. This will have the effect they desire without pissing tons of people off. Instead they have taken the aggressive stance of manual review + tipping off law enforcement, which goes against their entire mantra of protecting privacy. Your phone will now be an informant against you. If you are an activist or a whistleblower, a false positive could be enough to get your device searched and seized.
  - certeoun 5y ago
    @sam0x17, are you aware of this finding?: https://www.cambridge.org/core/journals/perspectives-on-politics/article/testing-theories-of-american-politics-elites-interest-groups-and-average-citizens/62327F513959D0A304D4893B382B992B https://www.cambridge.org/core/journals/perspectives-on-poli... > Despite the seemingly strong empirical support in previous studies for theories of majoritarian democracy, our analyses suggest that majorities of the American public actually have little influence over the policies our government adopts. Americans do enjoy many features central to democratic governance, such as regular elections, freedom of speech and association, and a widespread (if still contested) franchise. But we believe that if policymaking is dominated by powerful business organizations and a small number of affluent Americans, then America’s claims to being a democratic society are seriously threatened. Especially the part with "policymaking is dominated by powerful business organizations and a small number of affluent Americans" made me think that it is not only those elites who do this or businesses. Does this mean that the government (NSA) doesn't care what we want? They are allowed to essentially patronize us because we do not know any better? Can you explain to me why the NSA is spying on us despite it being unpopular? We didn't ask to be spied on, right? I don't understand politics or the role of government anymore. I am utterly confused. Politics is so contradictory I cannot wrap my head around it. Or perhaps, I am missing something?
the_snooze 5y ago
Here's the link to those researchers' words directly: https://www.washingtonpost.com/opinions/2021/08/19/apple-csam-abuse-encryption-security-privacy-dangerous/ https://www.washingtonpost.com/opinions/2021/08/19/apple-csa...
- zepto 5y ago
  That link is behind a paywall.
  - skygazer 5y ago
    It’s a soft paywall, protected only by cookies and JavaScript, bypass-able by using private browsing/incognito mode. Most news sites are — they give you your first N articles free and try to upsell, with the assumption that few will know to or bother to reset cookies, disable JavaScript or browse privately/incognito. (Surprisingly, the full text of the article is usually in the initial http response and only hidden by JavaScript running in your browser after the fact, based on the presence of a cookie from a prior visit - such a fragile and trusting system, but worth the upsell trade off to them.) You can also go to https://archive.is https://archive.is and paste in the article url. They generate nice shareable url like this one for this article: https://archive.is/y58Py https://archive.is/y58Py Ironically, the Google AMP version of the article will have no paywall at all, and is usually accessible with a slightly modified article url, discoverable via Google search from mobile. In the case of the Washington Post simply appending outputType=amp does the trick. https://www.washingtonpost.com/opinions/2021/08/19/apple-csam-abuse-encryption-security-privacy-dangerous/?outputType=amp https://www.washingtonpost.com/opinions/2021/08/19/apple-csa... Reader modes, like Apple’s, frequently, inadvertently, bypass soft paywalls, particularly when set to activate automatically, because they can see the hidden content on the page before it’s hidden. There are of course browser extensions that automate all of this. Few news sites use hard paywalls that prevent this. Whether you’re comfortable using private browsing or Google amp and not paying for the article is up to you.
    - zepto 5y ago
      > It’s a soft paywall, protected only by cookies and JavaScript, bypass-able by using private browsing/incognito mode. Not true. I tried that. > They generate nice shareable url like this one for this article: https://archive.is/y58Py https://archive.is/y58Py This is helpful.
shadilay 5y ago
There should be a prize for the person who comes up the name for the CSAM version of swatting.
- deleted 5y ago
  [deleted]
- palijer 5y ago
  Kidmailing? Based on blackmail... But it's not really blackmailing...
- xanaxagoras 5y ago
  Given the C and the A and the proximity of swatting to spanking, I propose switching, as in beating a child with a switch which is pretty well recognized as abuse these days.
1cvmask 5y ago
I am reposting a comment to the original article that appeared in the Washington Post originally because of the slippery slope dangers (https://en.wikipedia.org/wiki/Slippery_slope https://en.wikipedia.org/wiki/Slippery_slope): In a previous comment on this very same subject on Apple's attempt to flag CSAM I wrote: This invasive capability on the device level is a massive intrusion on everyone's privacy and there will be no limits for governments to expand it's reach once implemented. The scope will always broaden. Well in the article they correctly point out how the scope of scanning is already broad by governments around the world and a violation of privacy by content matching political speech and other forms of censorship and government tracking. We already have that now on the big tech platforms like Twitter that censor or shadow ban contetnt that they as the arbiters (egged on by the politicians and big corporate media) of truth (or truthiness as Colbert used to say in the old show The Colbert Report) label as misinformation or disinformation. Do we now need to be prevented from communicating our thoughts and punished for spreading the truth or non-truths, especially given the false positives, and malware injections and remote device takeovers and hijackings by the Orwellian Big Tech oligopolies. Power corrupts absolutely and this is too much power in the hands of Big Corporations and Governments. From the article in case you need the lowdown: Our system could be easily repurposed for surveillance and censorship. The design wasn’t restricted to a specific category of content; a service could simply swap in any content-matching database, and the person using that service would be none the wiser. A foreign government could, for example, compel a service to out people sharing disfavored political speech. That’s no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials. We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny. We were so disturbed that we took a step we hadn’t seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides. We’d planned to discuss paths forward at an academic conference this month. That dialogue never happened. The week before our presentation, Apple announced it would deploy its nearly identical system on iCloud Photos, which exists on more than 1.5 billion devices. Apple’s motivation, like ours, was to protect children. And its system was technically more efficient and capable than ours. But we were baffled to see that Apple had few answers for the hard questions we’d surfaced. China is Apple’s second-largest market, with probably hundreds of millions of devices. What stops the Chinese government from demanding Apple scan those devices for pro-democracy materials? Absolutely nothing, except Apple’s solemn promise. This is the same Apple that blocked Chinese citizens from apps that allow access to censored material, that acceded to China’s demand to store user data in state-owned data centers and whose chief executive infamously declared, “We follow the law wherever we do business.” Apple’s muted response about possible misuse is especially puzzling because it’s a high-profile flip-flop. After the 2015 terrorist attack in San Bernardino, Calif., the Justice Department tried to compel Apple to facilitate access to a perpetrator’s encrypted iPhone. Apple refused, swearing in court filings that if it were to build such a capability once, all bets were off about how that capability might be used in future.
zepto 5y ago
Without access to a technical description of what they built, we have no idea whether it is relevant.
- HumblyTossed 5y ago
  Right, so we should all just sit down, shut up and wait for Apple's implementation to be used nefariously.
  - zepto 5y ago
    > so we should all just sit down, shut up and wait for Apple's implementation to be used nefariously. That seems like a silly thing to suggest. I would be interested in knowing what these researchers developed, and how it relates to what Apple has done. If their work identifies technical weaknesses that apply, don’t you want to know what they are? Edit: I found their work, and it doesn’t apply.
ummonk 5y ago
Nothing about those concerns seems specific to the end-to-end encryption compatible CSAM system they or Apple built... Honestly if I were Apple I'd consider just scrapping the whole thing and doing server side CSAM testing on iCloud photos without rolling out E2E encryption for iCloud photos. It's just not worth the PR blowback.
- kenjackson 5y ago
  I agree. Apple already, AFAIK, requires iCloud uploading to be turned on in order for them to do scanning on the device. The delta of scanning on devices (already with iCloud enabled) and iCloud scanning doesn't seem to be worth the headache.
- rgovostes 5y ago
  Which is the perplexing thing here. The frenzy to paint this as anti-privacy given outlandish scenarios could very well end up killing a very pro-privacy move towards stronger encryption for everyone. It would be vastly easier for them to do this on the server, and never expend the engineering effort to implement device-side scanning with all these cryptographic safeguards, and altogether abandon E2EE for photos, not to mention the lobbying effort to protect E2EE against legislation. Today governments could demand Apple scan through every photo in iCloud. iOS today does NN-based classification of your photo library and Apple could in theory be compelled to modify this to report on certain kinds of content. (You can argue: The best for privacy is E2EE without CSAM scanning. This puts them on much less secure footing to defend against anti-E2EE legislation, so it would be risky.)
  - n8cpdx 5y ago
    That might be slightly valid if Apple had any plans to do e2e encryption. The fact that they haven’t said a word on that front would seem to indicate that they have no such plans, have never had such plans, and are not willing to make such plans. Another alternative would be to scan for CSAM and simply choose not to upload it and show a warning instead. I don’t get why folks are overlooking the absolutely critical _and then it snitches to Apple and/or the government_ part. P.S. it is weird to say Apple is “abandoning” an e2e encryption feature it doesn’t have, has never announced, and has made no mention of at any point, despite clear public pressure.
arpstick 5y ago
you know, one surefire way to signal to apple that deployment of such technology is detrimental to their bottom line is to just not buy apple products anymore.
- HumblyTossed 5y ago
  Another way is to publicly protest. If enough people are loud enough, this goes away.
  - deleted 5y ago
    [deleted]
  - flatiron 5y ago
    totally against this CSAM thing but totally not going to public protest when someone could EASILY twist this to "they wanted the right to keep child porn on their phones without apple knowing about it"
    - loteck 5y ago
      Have an upvote. flatiron is describing a perpetual and insidious issue with activism: many people believe that speaking out on a controversial issue will mean they will be tied to the worst extremes that could follow from the opinions they express. This fear silences them. This problem is especially prominent in the topic of privacy, which has been made to have a waft of wrongdoing associated with it ("nothing to hide").
      - user-the-name 5y ago
        They are also describing an issue, which like pretty much the entirety of this Apple fiasco, is completely fictional. Most of this outrage is about situations people are imagining, situations which often have only the most tenuous of relationships with the actual reality of what is happening.
        5y ago
deleted 5y ago
[deleted]
zepto 5y ago
Here’s their paper: https://www.usenix.org/system/files/sec21-kulshrestha.pdf https://www.usenix.org/system/files/sec21-kulshrestha.pdf Their system is vulnerable in all the ways they claim. However Apple’s system is not the same and does contain mitigations. > Apple’s muted response about possible misuse is especially puzzling because it’s a high-profile flip-flop. This is a dishonest statement. Apple has not been muted about the concerns these researchers are presenting. They address them here: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model... There is nothing in this piece that relates to Apple’s actual technology. These researchers obviously have the background to review what Apple has said and identify flaws, but they have not done so here.
riedel 5y ago
The losses are big having such a backdoor, but i am wondering about the gains. As far as I understand the police and the judicial system is often overwhelmed already with evidence. Wouldn't it be much better to pour money into the police for just this cause. My understanding is that they are largely underfunded and understaffed. It would seem much better to me if human intelligence would break into sharing rings rather than prosecuting a few more pedophiles. I would expect that they have enough leads from past cases... I think just paying more taxes would be the safest interface between big tech and governments. The problem seems to me that we are loosing trust in our law enforcement and cannot find appropriate ways to empower them without involving big tech, that has no problem with cooperating with foreign law enforcement agencies. The whole situation is so awkward: Apple needs to protect us from law enforcement that they want to support. Something fundamental seems broken and research should actively look for solutions here.

> A foreign government could, for example, compel a service to out people sharing disfavored political speech. That's no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials. This sums up the concern. CSAM is just the excuse because who would come out against protecting children, right? But this will absolutely be used for political purposes.

temp8964 5y ago
I don't understand this logic, why would authoritarian governments need to fake political dissident material as child porn for matching? The examples you quoted directly invalidate this argument. The authoritarian governments can directly ban political dissident materials, and they don't need child porn as the excuse.
- deleted 5y ago
  [deleted]
- ls612 5y ago
  Because the real concern isn’t that; it’s something like Apple vs FBI where the fact that Apple didn’t have a capability built have them a legal edge against a government with stronger rule of law.
- derefr 5y ago
  iMessage is E2E-encrypted; the only scanning of the content of the network a state adversary can do is by getting Apple to implement it for them, into the iMessage client. And Apple until now have had the excuse for not doing so, that they have no mechanism built into iMessage that would or could scan for such things, nor (as the excuse would go) is such a technology likely to be either “feasible” given the E2E-encrypted nature of the communications, or “palatable” to their audience (i.e. an argument like “people buy the iDevices in part because of advertised privacy benefits, so such state-level intrusions would lead to nobody buying iDevices, so nobody would be caught by this scheme anyway, so all you’re really doing by asking us to implement it is trying to destroy our company’s share price, probably to help your domestic phone market—and our government wouldn’t take kindly to that kind of market interference. So if you don’t want to be slapped with even more Huawei-like trade sanctions, kindly go away.”) The implementation of a CSAM scanner is a clear feasibility proof for the set of technologically-equivalent capabilities (e.g. dissident-material scanning.) The fact that Apple have implemented it, and that their users have accepted it and continued to use their iDevices, means Apple has no leg to stand on when denying governments the implementation of technological enforcement for “banning political dissident materials.”
  - dwaite 5y ago
    iMessage does not use the CSAM scanner, only the upload to iCloud Photos does. The iMessage feature is a local porn-ish detector on sent/received images with a click-through. It is only for children on family accounts, and if the child is under 13 a click-through is reported to the parent.
gjsman-1000 5y ago
The reason I think Apple went forward with it though is that, from their perspective, it's not like they are building a new tool for surveillance. It doesn't take many brain cells for a lawmaker to realize that they could mandate pre-screening of content. From Apple's perspective... for authoritarian governments like China or India... they are already able to mandate it and are likely to. So they shouldn't be factored into the "CSAM Scanning could be abused!" argument because it was already happening and going to happen, whether the tool exists or not. In which case, releasing the CSAM tool has only benefits for the abused and doesn't make a difference in preventing surveillance and privacy invasions because it was already going to occur. A cynical view but a possibility.
api 5y ago
The more I think about it, something that concerns me greatly is the silent use of this for political purposes. By silent I mean they don't arrest you or put you on any public lists. Instead they just gather information, lots of information, and they use it in future Cambridge Analytica style propaganda campaigns. I don't mean to imply that only Republicans would do this. After Trump's successful use of this kind of political propaganda every single political party and PR agency on the planet is working on duplicating it. Arresting people is old fashioned. Modern totalitarianism is built on disinformation, microtargeted propaganda, surveillance, and nudge theory.
- certeoun 5y ago
  What makes us better than China then? Our relative freedoms? What can we do about it? Princeton also did a study on America not being actually a democracy. I shared it here: https://news.ycombinator.com/item?id=28249091 https://news.ycombinator.com/item?id=28249091
  - DaftDank 5y ago
    Sorry for the off-topic question, but I've been wondering for a while and have been too afraid to ask: what does the green name signify here?
    - zargon 5y ago
      It means they're a newly registered user. (Not sure of the exact threshold, around a few days to a week.)
  - southerntofu 5y ago
    Nothing makes a country better than another, in absolute. Although, some aspects of life may be more enjoyable in one or another, depending on your taste. I think most of us can agree living in an abundance of resources (waste!) with some mild political freedom where you're allowed to say anything as long as you don't try to change anything (see: COINTELPRO, Julian Assange, etc) like in the Global North, is more pleasant to live on a daily basis than a hunger-ridden autocracy like North Korea. But even that is debatable and i'm sure many north koreans would disagree. What can we do about it? Attack the system, day by day and bits by bits. Steal what you can from the rich to redistribute to the poor. Make cooperatives, whether as employees or volunteers, so that more and more people can quit wage exploitation and start to live again, "from each according to their capabilities, to each according to their needs" (old anarchist saying). Form collectives for all kinds of struggles affecting you and your loved ones: anti-patriarchy, anti-racism, accessibility... Don't accept anyone claiming they're above you. We're all in this together, and those who claim to seek authority to find solutions will create more problems than they will solve, no matter their good intentions. We need to build "power to the people", not "power over the people", as anyone who has studied some history of political repression in the early Soviet Union (and other "dictatorships of the proletariat") can learn. See also Emma Goldman's "Trotsky protests too much" or "There is no communism in Russia" on this topic. Organize. Organize. Organize.
robertoandred 5y ago
Apple's system doesn't match content.
- vletal 5y ago
  You can do stuff like listing top K trending anti gov memes past day and blacklisting them. Thats definitely a content.
  - rgovostes 5y ago
    You cannot, because the models are shipped the OS image. Apple would have to publish a new OS update every day new memes come out. If you're talking about photo library scanning, people would have to be saving the memes to their photo libraries, cloud syncing would have to be on, and they would have to save lots of them. If you're talking about using the iMessage content scanning, this only applies to under-13 accounts, and it only notifies parents associated with the account. If you are suggesting that Apple would be compelled to modify either of these to do something else, why would they bother with these, which are built with many constraints and multiple levels of failsafes, rather than just implement a new feature? Governments could ban end-to-end encrypted chat like iMessage. They could demand the next iOS update uploads your saved social media passwords to them. It's not very plausible that anyone would want to modify _these_ features rather than just implement something more direct and harder to evade. Governments could make these demands at any point. They do not need to wait for this feature and then spring into action. The Chinese government did regarding where iCloud data is hosted for its citizens. The US government did regarding the San Bernardino iPhone case, and Apple fought back very publicly in court that any kind of modification of the OS was unacceptable.
    - Jcowell 5y ago
      The image saving is migrated by using apps that autosave received images by default. I.E WhatsApp.
    - vletal 5y ago
      sed -i "s/day/month/" parent
sixothree 5y ago
This event here marks the end of privacy as a right. It’s over.
- lstamour 5y ago
  Privacy can only be maintained by policies and regulation, not by technology. This hashing and scanning client-side, broken as it might be, is merely a technology. Technology to ensure privacy, as established by corporations, can always be legally circumvented by governments and what they allow or require by policy for their region, if said government is part of a large enough market that companies have to pay attention to it. Therefore the only solution is global agreement to privacy as a human right… and that seems unlikely right now. Runner up would be individual alternative apps for privacy (such as a camera app) but those can be banned by policy depending on the region you’re in, or potentially subject to the same rules/policies. In a democracy, if you disagree, your only option is talking to lawmakers, those who decide on policy. But again, it would likely only affect your region…
- ipaddr 5y ago
  Apple brand is privacy against other companies competing with them. Apple has always had privacy concerns.
- entropicdrifter 5y ago
  It's been over for a long while for the average consumer
  - rowanG077 5y ago
    This is definitely false. You can relatively easily live with privacy. It just takes a little bit of effort. It's over if you don't care and take no action.
    - entropicdrifter 5y ago
      The average consumer is largely unaware that they've lost their privacy and if they are aware they mostly fail to grasp the severity. The average consumer already doesn't care and has taken no action.
sam0x17 5y ago
> pre-screening Pre-screening would be the kind way of implementing this feature. They could make it so if CSAM is detected, just don't upload those files to the cloud and delete them locally. Then they are still doing due-diligence to prevent that material from entering their cloud and to remove that material from circulation, but they don't get random people arrested for false positives. When they detect hash matches on a device, they are under zero legal obligation to do anything about it because it could easily be a false positive (as the public has demonstrated on HN and elsewhere), so they'd be in the clear with this approach. This will have the effect they desire without pissing tons of people off. Instead they have taken the aggressive stance of manual review + tipping off law enforcement, which goes against their entire mantra of protecting privacy. Your phone will now be an informant against you. If you are an activist or a whistleblower, a false positive could be enough to get your device searched and seized.
- certeoun 5y ago
  @sam0x17, are you aware of this finding?: https://www.cambridge.org/core/journals/perspectives-on-politics/article/testing-theories-of-american-politics-elites-interest-groups-and-average-citizens/62327F513959D0A304D4893B382B992B https://www.cambridge.org/core/journals/perspectives-on-poli... > Despite the seemingly strong empirical support in previous studies for theories of majoritarian democracy, our analyses suggest that majorities of the American public actually have little influence over the policies our government adopts. Americans do enjoy many features central to democratic governance, such as regular elections, freedom of speech and association, and a widespread (if still contested) franchise. But we believe that if policymaking is dominated by powerful business organizations and a small number of affluent Americans, then America’s claims to being a democratic society are seriously threatened. Especially the part with "policymaking is dominated by powerful business organizations and a small number of affluent Americans" made me think that it is not only those elites who do this or businesses. Does this mean that the government (NSA) doesn't care what we want? They are allowed to essentially patronize us because we do not know any better? Can you explain to me why the NSA is spying on us despite it being unpopular? We didn't ask to be spied on, right? I don't understand politics or the role of government anymore. I am utterly confused. Politics is so contradictory I cannot wrap my head around it. Or perhaps, I am missing something?

the_snooze 5y ago

Here's the link to those researchers' words directly: https://www.washingtonpost.com/opinions/2021/08/19/apple-csam-abuse-encryption-security-privacy-dangerous/ https://www.washingtonpost.com/opinions/2021/08/19/apple-csa...

zepto 5y ago
That link is behind a paywall.
- skygazer 5y ago
  It’s a soft paywall, protected only by cookies and JavaScript, bypass-able by using private browsing/incognito mode. Most news sites are — they give you your first N articles free and try to upsell, with the assumption that few will know to or bother to reset cookies, disable JavaScript or browse privately/incognito. (Surprisingly, the full text of the article is usually in the initial http response and only hidden by JavaScript running in your browser after the fact, based on the presence of a cookie from a prior visit - such a fragile and trusting system, but worth the upsell trade off to them.) You can also go to https://archive.is https://archive.is and paste in the article url. They generate nice shareable url like this one for this article: https://archive.is/y58Py https://archive.is/y58Py Ironically, the Google AMP version of the article will have no paywall at all, and is usually accessible with a slightly modified article url, discoverable via Google search from mobile. In the case of the Washington Post simply appending outputType=amp does the trick. https://www.washingtonpost.com/opinions/2021/08/19/apple-csam-abuse-encryption-security-privacy-dangerous/?outputType=amp https://www.washingtonpost.com/opinions/2021/08/19/apple-csa... Reader modes, like Apple’s, frequently, inadvertently, bypass soft paywalls, particularly when set to activate automatically, because they can see the hidden content on the page before it’s hidden. There are of course browser extensions that automate all of this. Few news sites use hard paywalls that prevent this. Whether you’re comfortable using private browsing or Google amp and not paying for the article is up to you.
  - zepto 5y ago
    > It’s a soft paywall, protected only by cookies and JavaScript, bypass-able by using private browsing/incognito mode. Not true. I tried that. > They generate nice shareable url like this one for this article: https://archive.is/y58Py https://archive.is/y58Py This is helpful.

shadilay 5y ago

There should be a prize for the person who comes up the name for the CSAM version of swatting.

deleted 5y ago
[deleted]
palijer 5y ago
Kidmailing? Based on blackmail... But it's not really blackmailing...
xanaxagoras 5y ago
Given the C and the A and the proximity of swatting to spanking, I propose switching, as in beating a child with a switch which is pretty well recognized as abuse these days.

1cvmask 5y ago

I am reposting a comment to the original article that appeared in the Washington Post originally because of the slippery slope dangers (https://en.wikipedia.org/wiki/Slippery_slope https://en.wikipedia.org/wiki/Slippery_slope): In a previous comment on this very same subject on Apple's attempt to flag CSAM I wrote: This invasive capability on the device level is a massive intrusion on everyone's privacy and there will be no limits for governments to expand it's reach once implemented. The scope will always broaden. Well in the article they correctly point out how the scope of scanning is already broad by governments around the world and a violation of privacy by content matching political speech and other forms of censorship and government tracking. We already have that now on the big tech platforms like Twitter that censor or shadow ban contetnt that they as the arbiters (egged on by the politicians and big corporate media) of truth (or truthiness as Colbert used to say in the old show The Colbert Report) label as misinformation or disinformation. Do we now need to be prevented from communicating our thoughts and punished for spreading the truth or non-truths, especially given the false positives, and malware injections and remote device takeovers and hijackings by the Orwellian Big Tech oligopolies. Power corrupts absolutely and this is too much power in the hands of Big Corporations and Governments. From the article in case you need the lowdown: Our system could be easily repurposed for surveillance and censorship. The design wasn’t restricted to a specific category of content; a service could simply swap in any content-matching database, and the person using that service would be none the wiser. A foreign government could, for example, compel a service to out people sharing disfavored political speech. That’s no hypothetical: WeChat, the popular Chinese messaging app, already uses content matching to identify dissident material. India enacted rules this year that could require pre-screening content critical of government policy. Russia recently fined Google, Facebook and Twitter for not removing pro-democracy protest materials. We spotted other shortcomings. The content-matching process could have false positives, and malicious users could game the system to subject innocent users to scrutiny. We were so disturbed that we took a step we hadn’t seen before in computer science literature: We warned against our own system design, urging further research on how to mitigate the serious downsides. We’d planned to discuss paths forward at an academic conference this month. That dialogue never happened. The week before our presentation, Apple announced it would deploy its nearly identical system on iCloud Photos, which exists on more than 1.5 billion devices. Apple’s motivation, like ours, was to protect children. And its system was technically more efficient and capable than ours. But we were baffled to see that Apple had few answers for the hard questions we’d surfaced. China is Apple’s second-largest market, with probably hundreds of millions of devices. What stops the Chinese government from demanding Apple scan those devices for pro-democracy materials? Absolutely nothing, except Apple’s solemn promise. This is the same Apple that blocked Chinese citizens from apps that allow access to censored material, that acceded to China’s demand to store user data in state-owned data centers and whose chief executive infamously declared, “We follow the law wherever we do business.” Apple’s muted response about possible misuse is especially puzzling because it’s a high-profile flip-flop. After the 2015 terrorist attack in San Bernardino, Calif., the Justice Department tried to compel Apple to facilitate access to a perpetrator’s encrypted iPhone. Apple refused, swearing in court filings that if it were to build such a capability once, all bets were off about how that capability might be used in future.

zepto 5y ago

Without access to a technical description of what they built, we have no idea whether it is relevant.

HumblyTossed 5y ago
Right, so we should all just sit down, shut up and wait for Apple's implementation to be used nefariously.
- zepto 5y ago
  > so we should all just sit down, shut up and wait for Apple's implementation to be used nefariously. That seems like a silly thing to suggest. I would be interested in knowing what these researchers developed, and how it relates to what Apple has done. If their work identifies technical weaknesses that apply, don’t you want to know what they are? Edit: I found their work, and it doesn’t apply.

ummonk 5y ago

Nothing about those concerns seems specific to the end-to-end encryption compatible CSAM system they or Apple built... Honestly if I were Apple I'd consider just scrapping the whole thing and doing server side CSAM testing on iCloud photos without rolling out E2E encryption for iCloud photos. It's just not worth the PR blowback.

kenjackson 5y ago
I agree. Apple already, AFAIK, requires iCloud uploading to be turned on in order for them to do scanning on the device. The delta of scanning on devices (already with iCloud enabled) and iCloud scanning doesn't seem to be worth the headache.
rgovostes 5y ago
Which is the perplexing thing here. The frenzy to paint this as anti-privacy given outlandish scenarios could very well end up killing a very pro-privacy move towards stronger encryption for everyone. It would be vastly easier for them to do this on the server, and never expend the engineering effort to implement device-side scanning with all these cryptographic safeguards, and altogether abandon E2EE for photos, not to mention the lobbying effort to protect E2EE against legislation. Today governments could demand Apple scan through every photo in iCloud. iOS today does NN-based classification of your photo library and Apple could in theory be compelled to modify this to report on certain kinds of content. (You can argue: The best for privacy is E2EE without CSAM scanning. This puts them on much less secure footing to defend against anti-E2EE legislation, so it would be risky.)
- n8cpdx 5y ago
  That might be slightly valid if Apple had any plans to do e2e encryption. The fact that they haven’t said a word on that front would seem to indicate that they have no such plans, have never had such plans, and are not willing to make such plans. Another alternative would be to scan for CSAM and simply choose not to upload it and show a warning instead. I don’t get why folks are overlooking the absolutely critical _and then it snitches to Apple and/or the government_ part. P.S. it is weird to say Apple is “abandoning” an e2e encryption feature it doesn’t have, has never announced, and has made no mention of at any point, despite clear public pressure.

arpstick 5y ago

you know, one surefire way to signal to apple that deployment of such technology is detrimental to their bottom line is to just not buy apple products anymore.

HumblyTossed 5y ago
Another way is to publicly protest. If enough people are loud enough, this goes away.
- deleted 5y ago
  [deleted]
- flatiron 5y ago
  totally against this CSAM thing but totally not going to public protest when someone could EASILY twist this to "they wanted the right to keep child porn on their phones without apple knowing about it"
  - loteck 5y ago
    Have an upvote. flatiron is describing a perpetual and insidious issue with activism: many people believe that speaking out on a controversial issue will mean they will be tied to the worst extremes that could follow from the opinions they express. This fear silences them. This problem is especially prominent in the topic of privacy, which has been made to have a waft of wrongdoing associated with it ("nothing to hide").
    - user-the-name 5y ago
      They are also describing an issue, which like pretty much the entirety of this Apple fiasco, is completely fictional. Most of this outrage is about situations people are imagining, situations which often have only the most tenuous of relationships with the actual reality of what is happening.
      - 5y ago

deleted 5y ago

[deleted]

zepto 5y ago

Here’s their paper: https://www.usenix.org/system/files/sec21-kulshrestha.pdf https://www.usenix.org/system/files/sec21-kulshrestha.pdf Their system is vulnerable in all the ways they claim. However Apple’s system is not the same and does contain mitigations. > Apple’s muted response about possible misuse is especially puzzling because it’s a high-profile flip-flop. This is a dishonest statement. Apple has not been muted about the concerns these researchers are presenting. They address them here: https://www.apple.com/child-safety/pdf/Security_Threat_Model_Review_of_Apple_Child_Safety_Features.pdf https://www.apple.com/child-safety/pdf/Security_Threat_Model... There is nothing in this piece that relates to Apple’s actual technology. These researchers obviously have the background to review what Apple has said and identify flaws, but they have not done so here.

riedel 5y ago

The losses are big having such a backdoor, but i am wondering about the gains. As far as I understand the police and the judicial system is often overwhelmed already with evidence. Wouldn't it be much better to pour money into the police for just this cause. My understanding is that they are largely underfunded and understaffed. It would seem much better to me if human intelligence would break into sharing rings rather than prosecuting a few more pedophiles. I would expect that they have enough leads from past cases... I think just paying more taxes would be the safest interface between big tech and governments. The problem seems to me that we are loosing trust in our law enforcement and cannot find appropriate ways to empower them without involving big tech, that has no problem with cooperating with foreign law enforcement agencies. The whole situation is so awkward: Apple needs to protect us from law enforcement that they want to support. Something fundamental seems broken and research should actively look for solutions here.