14 ms·

Rights, Laws, and Google

173 points by vinnyglennon 4y ago

josephcsible 4y ago
> how big the number of false positives would have to be to shut the whole thing down? Since it seems that false positives are uncorrectable (given that even a New York Times article hasn't gotten this one fixed), and the consequences of one are thus life-destroying, even a single false positive should mean shutting the whole thing down.
- tomrod 4y ago
  I disagree. The value Google and similar web services brings is enormous. That said, there should be people in the loop who are trained and authorized to work around the operational systems. This is the cost of doing business.
  - josephcsible 4y ago
    I don't mean shutting down all of Google. I mean shutting down the whole automatic image scanning and account banning system.
- sulam 4y ago
  So, Google, Apple, Twitter, Facebook, etc should all shut down their scanning for CSAM? Do you realize the outcry this would produce? Google is not unique here, nor does it have unique power. This story would be just as plausible, and difficult, if the company in question were Facebook or Apple. Once a system exists to scan for this material, it’s quite easy to argue that shutting it off would be immoral and potentially criminal. “You had the means to identify abused children and help them, but you turned it off because you couldn’t work out a reasonable policy to deal with the false positives? Try harder, people.” (Note that I do not speak for my employer here.)
  - Semaphor 4y ago
    > So, Google, Apple, Twitter, Facebook, etc should all shut down their scanning for CSAM? What they should shut down, is their irrevocable deletion of accounts. When the false positives trigger nuclear bombs, then yes, a single false positive is too much. As far as I know, only Google does this, so everyone else can keep scanning in this argument.
  - count 4y ago
    Shutting down this new type of scanning is not the same as no longer scanning for CSAM. It's curious how the big providers have been scanning for CSAM for YEARS with nothing making the news...because hashes are much different and don't false positive like this.
  - josephcsible 4y ago
    It's immoral not to violate everyone's privacy, because some people are criminals?
  - karaterobot 4y ago
    The process that has led to our phones scanning everything we do and condemning us in an opaque extrajudicial process began with much smaller violations of privacy and trust — analysis of our emails, tracking our movement with cookies, and so on. When these practices were introduced, some people complained, saying they were part of a motion that could only ratchet in one direction: toward ubiquitous surveillance and corporate control over more and more of our lives. Cut to just a few years later, and here we are: "shutting it off would be immoral and potentially criminal". We can't go back! So, that clicking sound you hear is the ratchet turning.
- dont__panic 4y ago
  It is interesting to me that CSAM is used as justification for such heavy-handed consequences and widespread surveillance. People use the internet to do all kinds of awful things, plenty of them crimes, plenty of those crimes being violent or personal property crimes. Murder, robbery, burglary, etc. Obviously CSAM is vile and not something your average citizen condones. But so are those other crimes. I suppose children "pull on the heartstrings" more than your average victim, but children are also the victims of those other crimes. Why haven't we started surveilling personal messages to detect criminal planning in advance? Because more people understand the nasty implications of the government reading your messages, but few understand the implications of machine learning "detecting" CSAM in your Google Drive or Photos?
  - vineyardmike 4y ago
    Your broader point stands, but in the US at least “pre crime” monitoring or police intervention is a massive can of illegal worms. The planning of a crime can be illegal itself, eg “conspiracy to commit murder” is illegal, but generally it’s fraught with existing legal minefields.
  - matthewdgreen 4y ago
    In the UK they also search for “terrorism related content” and other types of criminal media. While this isn’t required in the US (and neither is CSAM scanning) part of the impetus for building these systems is to satisfy non-US governments that don’t have the same constitutional protections that the US does. US law enforcement officials routinely join with their non-US counterparts in urging/pressuring US firms to implement these systems, knowing that once they’re built (non-voluntarily) for non-US compliance they will likely be turned on (“voluntarily”) for US customers as well. See e.g. this open letter to Facebook authored by US AG Barr and counterparts from the UK and Australia. https://www.justice.gov/opa/pr/attorney-general-barr-signs-letter-facebook-us-uk-and-australian-leaders-regarding-use-end https://www.justice.gov/opa/pr/attorney-general-barr-signs-l...
- ajross 4y ago
  That logic seems to go both ways, though. Actual child abuse is, surely, even more "life-destroying", right? So wouldn't it make more sense to tolerate any number of false positives if we can save just one kid? I'm always amazed at the number of people who come to a complicated nuanced issue like this, accuse one side of having implemented an absolutist and inflexible dystopian monster, and then promptly demand an equally absolutist solution. The libertarian answer might well be that we need to tolerate CSAM in the interests of free speech, but that's not the sense of the society we live in, nor of the laws it has passed. All paths must be middle paths.
merely-unlikely 4y ago
I think this article points to a missing product in the market - integrated cloud services but with the cloud located on a private server in the home. Specifically for a company like Apple (who hand waives at being pro-privacy) to enable all iCloud functionality to be hosted on a personal mac. From a high level (and likely ignoring important but probably not insurmountable technical details), the idea would be for everything to look exactly the same to the iPhone user, but for all data and processing to happen on a machine the user controls. No Apple accessible or hosted user data, end-to-end encrypted in transit, etc. One might argue there are products that can fill this niche and certainly many on Hacker News are capable of setting up their own home servers. However, that is not nearly as convenient or realistic for the average person to do. Those setups also don't integrate directly into default services like iMessage, Photos (including both storage and things like photo tagging), etc. Modern desktops and many people's internet connections should be powerful/fast enough to handle this. To the extent user data is used to feed big-data algos/machine learning/etc, this might not be practical. That's why I suggest Apple do it rather than Google (Apple already claiming much of their ML services are handled on-device). One might argue that's essentially something iTunes handled (still could?) but again it's not nearly as integrated as modern iCloud services. And that only works over a local network or direct wired connection. The basic idea is just to move data hosting and processing from third-party servers where users have no direct control over how their data is used or accessed to hardware sitting in their home.
- izacus 4y ago
  This exists are part of NAS devices. Buy a Synology and you get your iCloud/Google set of services in your home, for 1-time price. With all of the services you mentioned.
  - merely-unlikely 4y ago
    That only covers a subset of iCloud functionality though. Remote file storage sure. What about things like like backup the entire device with the ability to restore on a new iPhone, browsing history, app data (first and third-party), passwords, etc. AFAIK solutions aren't nearly as integrated. Which is why it would need Apple's support. Then there are services like email, iMessage, reminders, notes, etc. The latter there may be some service that does it but not in the default apps. The former I don't know of any plug-and-play solutions not cloud hosted. For example, many privacy concerned people use Proton mail but that is cloud hosted and subject to search without the user's knowledge (hopefully provided a valid warrant, but I don't think that takes away from the point). I'd also rather services like find-my-iphone, recent locations, and other map data be handled on a server I control without Apple having any potential access. I'm not saying there are no solutions but I am saying that they involve tradeoffs that a first-party solution could make a lot smoother. I'd gladly pay for a home Apple server that could replace all iCloud functionality without having to change my habits. I suspect I'm not the only one - just look at their recent privacy focused offering that turns off a bunch of features for "at risk" individuals.
jrm4 4y ago
This is where e.g. much of "free market" rhetoric and e.g. Libertarianism need to take a severe backseat. Given the reach, it's absolutely time for "government interference." Now, I'm not actually saying that we definitely should shut things down. But we absolutely need to drag Google people into public hearings and such. Google et al have inserted themselves into their lives to an extent that they ought no longer be able to claim "but we're a private company." It's increasingly difficult, if not impossible, to opt out. Example; I fully intended for my children to not have anything like a Google account until a certain age -- but then the pandemic and remote schooling happened. It is true that with a great deal of pain to teachers et al I could have opted out, but that would have been literally unreasonable, not to mention unsustainable for other people.
- ghiculescu 4y ago
  What should the government do? The article discusses how it is not clear what should be different (other than turning the relevant accounts back on). It is easy to ask for public hearings but if there’s no intended remedy or change they will just be a show trial and nothing will change. Your example is great, because to me it illustrates my point well. What would you like to have had be different?
  - jrm4 4y ago
    I think good places in the past to look are "warning labels" and/or "common carrier"-like ideas. Warning labels today, I think means "fixing voluminous EULA crap," which we've done in other fields reasonably well, e.g. the FDA I think is pretty solid here. Simply requiring "readable" terms of services I think would go a long way (though you have to wait a bit for things to shake out.) The general principles behind "common carrier" type services would also be a good idea to start thinking about; they essentially just say "When your product or service becomes a day-to-day thing for a lot of people, we're going to hold you to certain accessibility rules."
- loudmax 4y ago
  I think I largely agree with you about getting the government involved. But the nature of that involvement is important here. As I see it the fundamental problem is not that Google isn't behaving as a responsible steward of free speech. The problem is that Google shouldn't have this power to begin with. By law, Google should be allowed to capriciously and arbitrarily shut down a user's account. It's their service, they shouldn't need to provide a justification. But Google shouldn't be in a position where they control so much of the market. There needs to be real competition in this area so users can easily switch to a different provider if they lose faith in Google's reliability. Where the government needs to be involved is in ensuring real choice in a free market, rather than the current duopoly. Ensuring a free market isn't easy. It's much easier to simply pass a law that says that Google (or a company in a similar position) needs to provide better service or offer users recourse to dispute this sort of service interruption. But that's just putting wallpaper over a crack. We need the government to commit to the hard work of enforcing open standards of interoperability so smaller entrants can get into the market.
ghiculescu 4y ago
It seems like the simplest solution here would be to not ban accounts automatically for CSAM detections, but to have a process to do so based on police recommendation. Clearly Google already has a process to escalate detection to the police so banning the account based on what happens there doesn’t seem like a big leap.
- notriddle 4y ago
  > It seems like the simplest solution here would be to not ban accounts automatically for CSAM detections, but to have a process to do so based on police recommendation. The police aren't going to recommend a ban. They don't want people banned. They want people arrested, tried, and convicted for criminal possession. They're going to recommend keeping the account open until they have enough evidence to take it to court.
  - dcow 4y ago
    Which seems completely fine. If the person is being investigated and they are continuing to commit abuses, the police will immediately arrest them and have evidence to bear increasing the odds that society locks the person up prompty. If the suspect is in custody, what does the status of their account matter? The point is that the account ultimately gets closed and the data purged if the suspect is found guilty, so what benefit is there to anybody in being delete-happy?
    - sangnoir 4y ago
      > Which seems completely fine. Google probably doesn't share the same rosy outlook. Long-running investigations that do not result in charges will be bad PR. The headline "Google shared private data on 100s of innocent users with law enforcement" will not be received well.
      - ghiculescu 4y ago
        Isn’t that why we have warrants? To enable the sharing of private data on (potentially) innocent people with law enforcement.
murphyslab 4y ago
> the tremendous trade-offs entailed in the indiscriminate scanning of users’ cloud data. It might be helpful to frame any cloud-based scanning of user activity or files as akin to mass-testing in health. While it sounds good in principle, there is always a background rate of false positives. And being incorrectly diagnosed can have negative consequences. There has been a large amount of examination of the consequences of mass-testing for various diseases without indication. e.g. COVID testing: https://www.bmj.com/content/373/bmj.n1411/ https://www.bmj.com/content/373/bmj.n1411/ Herpes: https://www.statnews.com/2017/01/26/flawed-herpes-testing-leads-to-false-positives/ https://www.statnews.com/2017/01/26/flawed-herpes-testing-le... Genetic testing: https://www.geneticsdigest.com/genetic-testing-potential-consequences-of-a-false-result/ https://www.geneticsdigest.com/genetic-testing-potential-con...
- johnmcarthur 4y ago
  Google by not answering is giving a clear message: if you test positive, even if you are then cleared out of all wrongdoings, you are going to be banned for life. So better be extra careful and don't even take children's pictures, even if your own or in perfectly valid scenarios.
  - candiodari 4y ago
    The problem is, this isn't Google, this is the government. All platforms, even Apple, are doing this. They aren't doing it voluntarily either, they did this under direction of laws, or threats of measures against them, such as banning their platforms from large economic zones. And the law is about to go much further: https://www.wired.com/story/europe-csam-scanning-law-chat-encryption/ https://www.wired.com/story/europe-csam-scanning-law-chat-en... Note that these laws don't mention platforms by name. So not copying your chats and images to the authorities, even going so far as using an application you custom write for a particular conversation, from even social workers and certain (not even named) private organisations (not even just the police or justice departments) will be illegal regardless of the content. They haven't even bothered to list which institutions are to have access to your chats. Just "law enforcement". Do I need to copy Latvian law enforcement on a message I send from France to my wife in New York? It's actually not clear. Any point where the law would require a little bit of effort by government, everything is left open. And these laws require a LOT of effort and infrastructure by governments to even enable citizens to comply with them. Call me a cynic, but I find it very hard to believe EU or US citizens would vote for their private chats or pictures to be copied even to the police, or scanned for anything at all. Never mind that, of course, said authorities haven't even bothered to make it possible for other platforms to do said reporting (I guess Facebook, Google and Amazon just find phone numbers to call somewhere, and then send unencrypted email with the material, because anything else just wouldn't work). Of course there is zero mention of what a person can do if such reporting mechanisms are used to discredit them (for example, send CSAM to your ex/political adversary, call the police on them). And of course, the police, social workers' and other institutions reputations where it comes to helping victims of CSAM is beyond terrible. In a lot of countries, including the US, cases where people protect children against CPS, even in cases of abuse, are easy to find. The reason is the extremely bad treatment child victims receive once reported, in other words: these reports aren't helping the victims at all, and this is a problem well-known for decades. It's not likely to improve any time soon. Cases of children fighting against these institutions when they are the victim aren't just well known ... they're actually the norm, the expected outcome of such interventions. I'm against conspiracy theories ... but I do agree with them on one point: I find it very hard to believe laws like this are even meant to protect anyone, other than the institutions stuff is being reported to themselves. If they were meant to protect, then we would make sure the treatment of children after reporting would be great, AND THEN go look for children in problem situations. The opposite is happening. You want my guess? These extreme measures are attempts to minimise the times government institutions get embarrassed by (the extremely rare cases of) serious crimes against children, which are still mostly missed.
deleted 4y ago
[deleted]
bsedlm 4y ago
edit: deleted.
- merely-unlikely 4y ago
  This doesn't necessarily take away from your point, but commonly under the law there is a legal fiction that holds a corporation is considered a "person"[1]. [1] https://en.wikipedia.org/wiki/Corporate_personhood https://en.wikipedia.org/wiki/Corporate_personhood
- nonrandomstring 4y ago
  > power ... exerting force over a minority ... free software movements were never about the openness nor the accessibility of the source code. But about issues such as this. Yes, Free Software culture has always been more about justice on a new (digital) frontier where new forms of injustice and abuse roam wild (and have only grown worse). I sincerely think the best model for understanding big-tech corporations like Google is as serfdom under feudal warlords within modern fiefdoms. It closely mirrors these historical power relations where laws and constitutions have nothing to say. The article is long and complex but I eventually found the kernel in this line: > "A Google spokeswoman said the company stands by its decisions, even though law enforcement cleared the two men." Though the moral questions behind it all are very complex, this case is not itself actually that complex. There was no mistake. No lack of proper investigation or tardiness by the police. The child, parent, doctor and police - all of the parties except Google - acted fairly, in good faith, mutuality and consent. Google is the problem here, and simply believes itself a law unto itself, that's the nub of it. For all the posturing about complying with the laws of nation states, companies like Facebook and Google have grown smug about their power. When they roll over the toes of the innocent, they laugh and say "and what are you going to do about it?" We are in new "might is right" times and nobody big enough has yet had the courage to say "Act justly, or we will hurt you back", and then follow up. For the ordinary citizen, the only sensible course is to resolutely refuse to use their services and products, and Free Software which provides so many alternatives to Google, Microsoft, Facebook and suchlike is the solution. It is the moral choice that a good citizen can and should make in these times.
cee_el123 4y ago
Does a petition/campaign in Mark's favour have any hope of getting Google to restore his account ? the most frustrating question I have - Why isn't Google restoring his account despite him being cleared by the police ? Why are they not even explaining this ?
- gumby 4y ago
  They are probably worrying that if they did it would open a floodgate of challenges, lawsuits and political attacks. As the last paragraph of this essay points out: tough. Google needs to step up and deal with the consequences of its own size.
  - golemotron 4y ago
    > Google needs to step up and deal with the consequences of its own size. This is where regulation can help.
    - gumby 4y ago
      This is an under-appreciated factor in regulation ignored by the reflexive "regulation bad" crowd. Companies can welcome regulation, and not in a malign, regulatory capture fashion. One case could be this one: doing the right thing could open you up to attacks from bad actors. A regulation could give you air cover (and legal cover). Or lets say you want to be more environmentally friendly but customers won't pay extra. Your competitors do too, but of course you can't agree to do this (that's collusion). A regulation removes the risk and is applied to all.
- crazygringo 4y ago
  I am utterly baffled as well. This was literally a front-page article in the NYT, not just the online home page but front page of the paper edition [1]. It's one thing to get the attention of some Googlers on the front page of HN, it's a whole other order of magnitude for the front page of the NYT. I cannot imagine how this hasn't become an urgent priority for Sundar himself to make sure this gets fixed and fast, at a minimum in these known situations. What could they possibly still be debating or deciding inside there at Google...? Police determined no charges, so restore the accounts. I don't see any potential risk, legal/reputational/otherwise, this would open up. Google's inaction here boggles the mind. [1] https://www.nytimes.com/issue/todayspaper/2022/08/22/todays-new-york-times https://www.nytimes.com/issue/todayspaper/2022/08/22/todays-...
bcatanzaro 4y ago
We need to regulate Google and require it to have a transparent appeals process.
- tinalumfoil 4y ago
  I don't think Google's doing this out of a deep concern for CSAM and hatred for due process, but I think it's in fact the law itself that essentially requires corporations to have these trigger bans otherwise they'd face liability. Think about Google's alternatives here. They could (1) not have CSAM filters -- your data is private no need for Google to scan anything -- in which case people would use their platform to distribute illegal content and they'd be a nice target to get massive damages from. Or (2) give people due process, rights, appeals, etc despite not having any of the tools of a court of law, it's even illegal for them to look at the evidence, and if they make the wrong decision or they make the right decision too late they're still liable for their platform being used for illegal activity. Keep in mind the only way to truly know if material is sexually explicit is to show it to Potter Stewart. He'll know it when he sees it. Or (3) be proactive in building systems that detect illegal use of their platform and aggressively, without process or appeal, remove any user of the platform the moment you have any evidence at all they are engaged in this activity. Because people will try to use Google's platform for illegal activity and Google knows this, (3) is not only the only sensible option but it's actually the only legal option. It would, in effect, be illegal for Google to do anything other than scanning all your files and aggressively ban people.
  - treis 4y ago
    >use their platform to distribute I'm not a fan of this phrasing because it absolves Google of responsibility. If the CSAM is on a Google server then Google is possessing and distributing CSAM. You can say they're doing it unwittingly or unintentionally but that doesn't change the fact that they're doing it.
    - anon84873628 4y ago
      And that's the line of thinking that makes #3 the only option for Google.
sylware 4y ago
Isn't google/alphabet a part of the vanguard/blackrock galaxy now? Aren't blackrock/vanguard making the laws in the US? Like killing the planned obsolescence related bills?
- jjulius 4y ago
  >As company shareholders, BlackRock and Vanguard can vote on behalf of their clients at company shareholder meetings. Both firms also have “investment stewardship” functions, which enables the proxy votes. >BlackRock’s spokesperson said the votes can also be carried out by a portfolio manager – and in some cases at BlackRock, can be carried out by the clients themselves (here). >BlackRock and Vanguard do not “own” all the biggest corporations in the world. They invest trillions of dollars into leading companies on behalf of their clients, who ultimately own the shares. https://www.reuters.com/article/factcheck-business-investment/fact-check-video-claiming-blackrock-and-vanguard-own-all-the-biggest-corporations-in-the-world-is-missing-context-idUSL2N2WI1K4 https://www.reuters.com/article/factcheck-business-investmen...
  - sylware 4y ago
    So blackrock/vanguard can "hide" their "big" investors, depending on the company "invested in", or is this public info? That said, it is actually blackrock/vanguard ppl on the boards, and often they are the biggest ones, and expecting no steering from them or the "big" investors behind would be... "unexpected". Their is also the massive debts with the gigantic interests to be paid all along the calendar year. In the case of starbucks, owners of the debts is public info or the web page is incomplete? That said, now I understand why online non-franchised/licensed starbucks stuff has the aspx file extension.
    - jjulius 4y ago
      >... it is actually blackrock/vanguard ppl on the boards... Citation needed.
      - sylware 4y ago
alexb_ 4y ago
This is an extremely good article. The hard part of this is everyone can agree that "legality is not morality" and that what is legal is not necessarily moral. But this is used for the wrong side of the argument - since the Bill of Rights is a legal document, some disregard it as any guidance for morality.
- vorpalhex 4y ago
  The Bill of Rights is a legal document based on a Moral set of beliefs.
  - deleted 4y ago
    [deleted]
colinsane 4y ago
the article makes good points, but i don’t think the conclusion is obvious. for example, i want the freedom to not interact with Google. a ruling to require Google provide services to every individual might allow for a future corresponding ruling, that individuals must interact with Google (or at least, this becomes the socially acceptable view, which shapes laws later). wiser, i think, would be to prevent any single company from becoming critical to participation in society.
- SpicyLemonZest 4y ago
  I don't think that really solves anything here, because nothing in the story indicates that Google was critical to Mark's participation in society. He got new contact information and it sounds like he's been able to (with some effort) get all his non-Google accounts switched over. He didn't get fired, or evicted, or land in any legal trouble. He just liked Google's services and enjoyed using them until he was unfairly banned.
themacguffinman 4y ago
If regulation is the answer, I'd hope it is designed with the restraint that Madison had: only guarantee the deepest and most fundamental rights. It should not rule out the ability to use error-prone systems at all, at most it could restrict their use for the most severe consequences like broad account bans. Google can and should be reducing the scope of their automated action. I think error-prone systems are still necessary today to reduce costs to a tolerable modern level. I'm not inclined to throw out the baby with the bathwater regarding communication costs for the same reason I don't want $300 toasters even if "they don't make 'em like they used to". A good example is something that Facebook does: they have a blacklist of domains you can't send on Messenger, the message will be instantly blocked when you paste a detected link. From experience, this list is fairly large and very inaccurate, but the false positives for this system are annoying rather than catastrophic because all it does is block a specific message. If you don't like this, then you can use Signal or something, I don't want other people making that choice for me.
waylandsmithers 4y ago
My theory on why Google does this (and follows the same pattern in so many areas) is that having one person simply use common sense and reasonable judgment would be something of an admission that their software, created by the best and brightest engineers, isn't top notch.
- paulcole 4y ago
  Start with questioning whether the software was created by the best and brightest.
Animats 4y ago
There's a question as to whether Google is acting as an agent of the Government here. When the Government outsources something, that activity sometimes becomes subject to constitutional protections. Cases on this are iffy, though.[1] This has come up in reference to Amtrak, the Federal Reserve, Blackwater, and privately run prisons. Suppose Google analyzed all business data that passed through its servers looking for patterns of tax evasion. If the probability of tax evasion was high, reporting it to the IRS, which offers rewards, could be a profit center. [1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2394843 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2394843
- treeman79 4y ago
  Private ownership but government control is Fascism. https://youtu.be/PdOx36bZMBY https://youtu.be/PdOx36bZMBY Good recent is example is Facebook suppressing stories that the FBI wants suppressed.
  - Animats 4y ago
    Actually, Reagan is misquoting there.[1] Something like that is often attributed to either Huey Long or Sinclair Lewis, but neither actually wrote that. Or even close. The closest match is from Sinclair Lewis' book "It Can't Happen Here": "He was afraid that the world struggle today was not of Communism against Fascism, but of tolerance against the bigotry that was preached equally by Communism and Fascism. But he saw too that in America the struggle was befogged by the fact that the worst Fascists were they who disowned the word “Fascism” and preached enslavement to Capitalism under the style of Constitutional and Traditional Native American Liberty." That book is very relevant today. [1] https://quoteinvestigator.com/2017/07/28/flag/ https://quoteinvestigator.com/2017/07/28/flag/
  - vkou 4y ago
    > Good recent is example is Facebook suppressing stories that the FBI wants suppressed. Instead, Facebook should have been promoting stories that the FSB wanted promoted. :)
- 9ace07ac 4y ago
dcow 4y ago
Every passing day I feel that we are at the social limits of what our existing laws and documents can practically support without a pretty decent refactor. This essay is the perfect supporting argument, and the actions of Google the perfect case study. What will USA 2.0 look like? Is there a way to work on this without it being a bloody revolution? If we got 2/3rds of the population to green light a refactor of the constitution could it happen? Who would do it? Who are the right people? Importantly, if we want to preserve the spirit of the USA's founding documents, they need to evolve more formally and officially than supreme court case law and bloated regulatory agencies tied to deeply industry integrated lobbyists. I really wish writing laws was more like the RFC process. I don't think the solution is to "just run more stuff at home" as much as that tickles my technocratic jimmies.
- merely-unlikely 4y ago
  Lawmaking could learn a lot from the open source community. Imagine a wiki + git + democratic voting style system. All laws and changes would be hosted on a publicly available website. Any citizen could browse and submit "pull requests" which would then get debated and eventually voted on if they gain enough traction. Obviously such a system would be way more complicated than I can describe in a single HN comment, but I do think we have the technology to build a much more public, accessible, and directly democratic method of writing laws. As an aside, did you know no one even knows how many federal laws exist?[1]. [1] https://blogs.loc.gov/law/2013/03/frequent-reference-question-how-many-federal-laws-are-there/ https://blogs.loc.gov/law/2013/03/frequent-reference-questio...
  - cube00 4y ago
    You can't expect the population to be across all laws and changes, that's why proportional elected officials with office staff exist.
    - merely-unlikely 4y ago
      I'd still rather see that happen out in the open instead of back room deals and bills ghost written by opaque lobbyists. I could even imagine a voting system that lets you name your favorite politician as your proxy. So politicians could collect the votes of their base into a block. Maybe even a mechanism of saying I want this expert to cast my vote on x topic, and this other thought leader on y topic. Independents might retain their vote on every issue if they care to. I'd probably go with a system with tiers of votes as an issue gains more public attention. Say a minor change is proposed on a niche law and those who are passionate about fishing rights in the Colorado River get involved and vote. Say 5k people vote. Then there's a cooling off period before implementation but all of a sudden the issue ends up in the news and 5m people are now interested so there's another vote with another cooling off period. Only this time 10m people would need to express interest to hold another vote. I don't know the perfect system but I'm confident we can come up with a more democratic process than what was possible 250 years ago. One in which the public more directly decides what issues get voted on. Congress has limited capacity and can't focus on everything but the work can be distributed if it were more open. A system in which experts could make proposals directly instead of being forced to educate and convince a politician the issue is even worth spending political capital on. And a system in which citizens can see all the changes being made publicly. It seems every time a major bill is passed it tends to be this huge amalgamation of new laws that often aren't even related. And then news articles follow pointing out how x, y, and z things got slipped into the 800 page bill no one in Congress actually bothered to read.
tarakat 4y ago
> We don’t know how many of those children were subsequently rescued, but a question worth posing to anyone unilaterally opposed to Google’s approach is how big that number would have to be to have made it worthwhile? [..] how big the number of false positives would have to be to shut the whole thing down? There's nothing worse or more myopic than this kind of "pragmatic", numerical approach towards liberty. As if living in a panopticon cage is desirable, as long as the wardens never make a mistake. Why is it myopic? Well, if being forced to use technology that betrays its user is okay as long as one number is sufficiently larger than another, then why not force all new houses to come with indoor microphones and cameras, always listening and watching - only for the most heinous crimes, of course! Are you "unilaterally opposed" to such a measure, despite how much sexual abuse and domestic violence and murder it would prevent? What about tracking and shock collars for everyone? How much violence could be stopped if everyone was forced to wear those! Don't like it? Not even if the false positive rate was zero?
lsh123 4y ago
I wonder if Google violated HIPAA by sending patient to doctor communication to a third party (CSAM).
- thaumasiotes 4y ago
  Google, not being a provider of healthcare or health insurance, cannot violate the Health Insurance Portability and Accountability Act. If Google deputized a VP to hunt you down and murder you in your sleep, that also would not violate HIPAA.
  - ec109685 4y ago
    Also, the images uploaded to Google cloud were independent of sending the images to the doctor via his wife.
  - dekhn 4y ago
    The (enterprise) cloud side of Google is a business associate (also covered in HIPAA) in some circumstances but I don't think that applies in this case.

173 points by vinnyglennon 4y ago

> how big the number of false positives would have to be to shut the whole thing down? Since it seems that false positives are uncorrectable (given that even a New York Times article hasn't gotten this one fixed), and the consequences of one are thus life-destroying, even a single false positive should mean shutting the whole thing down.

tomrod 4y ago
I disagree. The value Google and similar web services brings is enormous. That said, there should be people in the loop who are trained and authorized to work around the operational systems. This is the cost of doing business.
- josephcsible 4y ago
  I don't mean shutting down all of Google. I mean shutting down the whole automatic image scanning and account banning system.
sulam 4y ago
So, Google, Apple, Twitter, Facebook, etc should all shut down their scanning for CSAM? Do you realize the outcry this would produce? Google is not unique here, nor does it have unique power. This story would be just as plausible, and difficult, if the company in question were Facebook or Apple. Once a system exists to scan for this material, it’s quite easy to argue that shutting it off would be immoral and potentially criminal. “You had the means to identify abused children and help them, but you turned it off because you couldn’t work out a reasonable policy to deal with the false positives? Try harder, people.” (Note that I do not speak for my employer here.)
- Semaphor 4y ago
  > So, Google, Apple, Twitter, Facebook, etc should all shut down their scanning for CSAM? What they should shut down, is their irrevocable deletion of accounts. When the false positives trigger nuclear bombs, then yes, a single false positive is too much. As far as I know, only Google does this, so everyone else can keep scanning in this argument.
- count 4y ago
  Shutting down this new type of scanning is not the same as no longer scanning for CSAM. It's curious how the big providers have been scanning for CSAM for YEARS with nothing making the news...because hashes are much different and don't false positive like this.
- josephcsible 4y ago
  It's immoral not to violate everyone's privacy, because some people are criminals?
- karaterobot 4y ago
  The process that has led to our phones scanning everything we do and condemning us in an opaque extrajudicial process began with much smaller violations of privacy and trust — analysis of our emails, tracking our movement with cookies, and so on. When these practices were introduced, some people complained, saying they were part of a motion that could only ratchet in one direction: toward ubiquitous surveillance and corporate control over more and more of our lives. Cut to just a few years later, and here we are: "shutting it off would be immoral and potentially criminal". We can't go back! So, that clicking sound you hear is the ratchet turning.
dont__panic 4y ago
It is interesting to me that CSAM is used as justification for such heavy-handed consequences and widespread surveillance. People use the internet to do all kinds of awful things, plenty of them crimes, plenty of those crimes being violent or personal property crimes. Murder, robbery, burglary, etc. Obviously CSAM is vile and not something your average citizen condones. But so are those other crimes. I suppose children "pull on the heartstrings" more than your average victim, but children are also the victims of those other crimes. Why haven't we started surveilling personal messages to detect criminal planning in advance? Because more people understand the nasty implications of the government reading your messages, but few understand the implications of machine learning "detecting" CSAM in your Google Drive or Photos?
- vineyardmike 4y ago
  Your broader point stands, but in the US at least “pre crime” monitoring or police intervention is a massive can of illegal worms. The planning of a crime can be illegal itself, eg “conspiracy to commit murder” is illegal, but generally it’s fraught with existing legal minefields.
- matthewdgreen 4y ago
  In the UK they also search for “terrorism related content” and other types of criminal media. While this isn’t required in the US (and neither is CSAM scanning) part of the impetus for building these systems is to satisfy non-US governments that don’t have the same constitutional protections that the US does. US law enforcement officials routinely join with their non-US counterparts in urging/pressuring US firms to implement these systems, knowing that once they’re built (non-voluntarily) for non-US compliance they will likely be turned on (“voluntarily”) for US customers as well. See e.g. this open letter to Facebook authored by US AG Barr and counterparts from the UK and Australia. https://www.justice.gov/opa/pr/attorney-general-barr-signs-letter-facebook-us-uk-and-australian-leaders-regarding-use-end https://www.justice.gov/opa/pr/attorney-general-barr-signs-l...
ajross 4y ago
That logic seems to go both ways, though. Actual child abuse is, surely, even more "life-destroying", right? So wouldn't it make more sense to tolerate any number of false positives if we can save just one kid? I'm always amazed at the number of people who come to a complicated nuanced issue like this, accuse one side of having implemented an absolutist and inflexible dystopian monster, and then promptly demand an equally absolutist solution. The libertarian answer might well be that we need to tolerate CSAM in the interests of free speech, but that's not the sense of the society we live in, nor of the laws it has passed. All paths must be middle paths.

merely-unlikely 4y ago

I think this article points to a missing product in the market - integrated cloud services but with the cloud located on a private server in the home. Specifically for a company like Apple (who hand waives at being pro-privacy) to enable all iCloud functionality to be hosted on a personal mac. From a high level (and likely ignoring important but probably not insurmountable technical details), the idea would be for everything to look exactly the same to the iPhone user, but for all data and processing to happen on a machine the user controls. No Apple accessible or hosted user data, end-to-end encrypted in transit, etc. One might argue there are products that can fill this niche and certainly many on Hacker News are capable of setting up their own home servers. However, that is not nearly as convenient or realistic for the average person to do. Those setups also don't integrate directly into default services like iMessage, Photos (including both storage and things like photo tagging), etc. Modern desktops and many people's internet connections should be powerful/fast enough to handle this. To the extent user data is used to feed big-data algos/machine learning/etc, this might not be practical. That's why I suggest Apple do it rather than Google (Apple already claiming much of their ML services are handled on-device). One might argue that's essentially something iTunes handled (still could?) but again it's not nearly as integrated as modern iCloud services. And that only works over a local network or direct wired connection. The basic idea is just to move data hosting and processing from third-party servers where users have no direct control over how their data is used or accessed to hardware sitting in their home.

izacus 4y ago
This exists are part of NAS devices. Buy a Synology and you get your iCloud/Google set of services in your home, for 1-time price. With all of the services you mentioned.
- merely-unlikely 4y ago
  That only covers a subset of iCloud functionality though. Remote file storage sure. What about things like like backup the entire device with the ability to restore on a new iPhone, browsing history, app data (first and third-party), passwords, etc. AFAIK solutions aren't nearly as integrated. Which is why it would need Apple's support. Then there are services like email, iMessage, reminders, notes, etc. The latter there may be some service that does it but not in the default apps. The former I don't know of any plug-and-play solutions not cloud hosted. For example, many privacy concerned people use Proton mail but that is cloud hosted and subject to search without the user's knowledge (hopefully provided a valid warrant, but I don't think that takes away from the point). I'd also rather services like find-my-iphone, recent locations, and other map data be handled on a server I control without Apple having any potential access. I'm not saying there are no solutions but I am saying that they involve tradeoffs that a first-party solution could make a lot smoother. I'd gladly pay for a home Apple server that could replace all iCloud functionality without having to change my habits. I suspect I'm not the only one - just look at their recent privacy focused offering that turns off a bunch of features for "at risk" individuals.

jrm4 4y ago

This is where e.g. much of "free market" rhetoric and e.g. Libertarianism need to take a severe backseat. Given the reach, it's absolutely time for "government interference." Now, I'm not actually saying that we definitely should shut things down. But we absolutely need to drag Google people into public hearings and such. Google et al have inserted themselves into their lives to an extent that they ought no longer be able to claim "but we're a private company." It's increasingly difficult, if not impossible, to opt out. Example; I fully intended for my children to not have anything like a Google account until a certain age -- but then the pandemic and remote schooling happened. It is true that with a great deal of pain to teachers et al I could have opted out, but that would have been literally unreasonable, not to mention unsustainable for other people.

ghiculescu 4y ago
What should the government do? The article discusses how it is not clear what should be different (other than turning the relevant accounts back on). It is easy to ask for public hearings but if there’s no intended remedy or change they will just be a show trial and nothing will change. Your example is great, because to me it illustrates my point well. What would you like to have had be different?
- jrm4 4y ago
  I think good places in the past to look are "warning labels" and/or "common carrier"-like ideas. Warning labels today, I think means "fixing voluminous EULA crap," which we've done in other fields reasonably well, e.g. the FDA I think is pretty solid here. Simply requiring "readable" terms of services I think would go a long way (though you have to wait a bit for things to shake out.) The general principles behind "common carrier" type services would also be a good idea to start thinking about; they essentially just say "When your product or service becomes a day-to-day thing for a lot of people, we're going to hold you to certain accessibility rules."
loudmax 4y ago
I think I largely agree with you about getting the government involved. But the nature of that involvement is important here. As I see it the fundamental problem is not that Google isn't behaving as a responsible steward of free speech. The problem is that Google shouldn't have this power to begin with. By law, Google should be allowed to capriciously and arbitrarily shut down a user's account. It's their service, they shouldn't need to provide a justification. But Google shouldn't be in a position where they control so much of the market. There needs to be real competition in this area so users can easily switch to a different provider if they lose faith in Google's reliability. Where the government needs to be involved is in ensuring real choice in a free market, rather than the current duopoly. Ensuring a free market isn't easy. It's much easier to simply pass a law that says that Google (or a company in a similar position) needs to provide better service or offer users recourse to dispute this sort of service interruption. But that's just putting wallpaper over a crack. We need the government to commit to the hard work of enforcing open standards of interoperability so smaller entrants can get into the market.

ghiculescu 4y ago

It seems like the simplest solution here would be to not ban accounts automatically for CSAM detections, but to have a process to do so based on police recommendation. Clearly Google already has a process to escalate detection to the police so banning the account based on what happens there doesn’t seem like a big leap.

notriddle 4y ago
> It seems like the simplest solution here would be to not ban accounts automatically for CSAM detections, but to have a process to do so based on police recommendation. The police aren't going to recommend a ban. They don't want people banned. They want people arrested, tried, and convicted for criminal possession. They're going to recommend keeping the account open until they have enough evidence to take it to court.
- dcow 4y ago
  Which seems completely fine. If the person is being investigated and they are continuing to commit abuses, the police will immediately arrest them and have evidence to bear increasing the odds that society locks the person up prompty. If the suspect is in custody, what does the status of their account matter? The point is that the account ultimately gets closed and the data purged if the suspect is found guilty, so what benefit is there to anybody in being delete-happy?
  - sangnoir 4y ago
    > Which seems completely fine. Google probably doesn't share the same rosy outlook. Long-running investigations that do not result in charges will be bad PR. The headline "Google shared private data on 100s of innocent users with law enforcement" will not be received well.
    - ghiculescu 4y ago
      Isn’t that why we have warrants? To enable the sharing of private data on (potentially) innocent people with law enforcement.

murphyslab 4y ago

> the tremendous trade-offs entailed in the indiscriminate scanning of users’ cloud data. It might be helpful to frame any cloud-based scanning of user activity or files as akin to mass-testing in health. While it sounds good in principle, there is always a background rate of false positives. And being incorrectly diagnosed can have negative consequences. There has been a large amount of examination of the consequences of mass-testing for various diseases without indication. e.g. COVID testing: https://www.bmj.com/content/373/bmj.n1411/ https://www.bmj.com/content/373/bmj.n1411/ Herpes: https://www.statnews.com/2017/01/26/flawed-herpes-testing-leads-to-false-positives/ https://www.statnews.com/2017/01/26/flawed-herpes-testing-le... Genetic testing: https://www.geneticsdigest.com/genetic-testing-potential-consequences-of-a-false-result/ https://www.geneticsdigest.com/genetic-testing-potential-con...

johnmcarthur 4y ago
Google by not answering is giving a clear message: if you test positive, even if you are then cleared out of all wrongdoings, you are going to be banned for life. So better be extra careful and don't even take children's pictures, even if your own or in perfectly valid scenarios.
- candiodari 4y ago
  The problem is, this isn't Google, this is the government. All platforms, even Apple, are doing this. They aren't doing it voluntarily either, they did this under direction of laws, or threats of measures against them, such as banning their platforms from large economic zones. And the law is about to go much further: https://www.wired.com/story/europe-csam-scanning-law-chat-encryption/ https://www.wired.com/story/europe-csam-scanning-law-chat-en... Note that these laws don't mention platforms by name. So not copying your chats and images to the authorities, even going so far as using an application you custom write for a particular conversation, from even social workers and certain (not even named) private organisations (not even just the police or justice departments) will be illegal regardless of the content. They haven't even bothered to list which institutions are to have access to your chats. Just "law enforcement". Do I need to copy Latvian law enforcement on a message I send from France to my wife in New York? It's actually not clear. Any point where the law would require a little bit of effort by government, everything is left open. And these laws require a LOT of effort and infrastructure by governments to even enable citizens to comply with them. Call me a cynic, but I find it very hard to believe EU or US citizens would vote for their private chats or pictures to be copied even to the police, or scanned for anything at all. Never mind that, of course, said authorities haven't even bothered to make it possible for other platforms to do said reporting (I guess Facebook, Google and Amazon just find phone numbers to call somewhere, and then send unencrypted email with the material, because anything else just wouldn't work). Of course there is zero mention of what a person can do if such reporting mechanisms are used to discredit them (for example, send CSAM to your ex/political adversary, call the police on them). And of course, the police, social workers' and other institutions reputations where it comes to helping victims of CSAM is beyond terrible. In a lot of countries, including the US, cases where people protect children against CPS, even in cases of abuse, are easy to find. The reason is the extremely bad treatment child victims receive once reported, in other words: these reports aren't helping the victims at all, and this is a problem well-known for decades. It's not likely to improve any time soon. Cases of children fighting against these institutions when they are the victim aren't just well known ... they're actually the norm, the expected outcome of such interventions. I'm against conspiracy theories ... but I do agree with them on one point: I find it very hard to believe laws like this are even meant to protect anyone, other than the institutions stuff is being reported to themselves. If they were meant to protect, then we would make sure the treatment of children after reporting would be great, AND THEN go look for children in problem situations. The opposite is happening. You want my guess? These extreme measures are attempts to minimise the times government institutions get embarrassed by (the extremely rare cases of) serious crimes against children, which are still mostly missed.

deleted 4y ago

[deleted]

bsedlm 4y ago

edit: deleted.

merely-unlikely 4y ago
This doesn't necessarily take away from your point, but commonly under the law there is a legal fiction that holds a corporation is considered a "person"[1]. [1] https://en.wikipedia.org/wiki/Corporate_personhood https://en.wikipedia.org/wiki/Corporate_personhood
nonrandomstring 4y ago
> power ... exerting force over a minority ... free software movements were never about the openness nor the accessibility of the source code. But about issues such as this. Yes, Free Software culture has always been more about justice on a new (digital) frontier where new forms of injustice and abuse roam wild (and have only grown worse). I sincerely think the best model for understanding big-tech corporations like Google is as serfdom under feudal warlords within modern fiefdoms. It closely mirrors these historical power relations where laws and constitutions have nothing to say. The article is long and complex but I eventually found the kernel in this line: > "A Google spokeswoman said the company stands by its decisions, even though law enforcement cleared the two men." Though the moral questions behind it all are very complex, this case is not itself actually that complex. There was no mistake. No lack of proper investigation or tardiness by the police. The child, parent, doctor and police - all of the parties except Google - acted fairly, in good faith, mutuality and consent. Google is the problem here, and simply believes itself a law unto itself, that's the nub of it. For all the posturing about complying with the laws of nation states, companies like Facebook and Google have grown smug about their power. When they roll over the toes of the innocent, they laugh and say "and what are you going to do about it?" We are in new "might is right" times and nobody big enough has yet had the courage to say "Act justly, or we will hurt you back", and then follow up. For the ordinary citizen, the only sensible course is to resolutely refuse to use their services and products, and Free Software which provides so many alternatives to Google, Microsoft, Facebook and suchlike is the solution. It is the moral choice that a good citizen can and should make in these times.

cee_el123 4y ago

Does a petition/campaign in Mark's favour have any hope of getting Google to restore his account ? the most frustrating question I have - Why isn't Google restoring his account despite him being cleared by the police ? Why are they not even explaining this ?

gumby 4y ago
They are probably worrying that if they did it would open a floodgate of challenges, lawsuits and political attacks. As the last paragraph of this essay points out: tough. Google needs to step up and deal with the consequences of its own size.
- golemotron 4y ago
  > Google needs to step up and deal with the consequences of its own size. This is where regulation can help.
  - gumby 4y ago
    This is an under-appreciated factor in regulation ignored by the reflexive "regulation bad" crowd. Companies can welcome regulation, and not in a malign, regulatory capture fashion. One case could be this one: doing the right thing could open you up to attacks from bad actors. A regulation could give you air cover (and legal cover). Or lets say you want to be more environmentally friendly but customers won't pay extra. Your competitors do too, but of course you can't agree to do this (that's collusion). A regulation removes the risk and is applied to all.
crazygringo 4y ago
I am utterly baffled as well. This was literally a front-page article in the NYT, not just the online home page but front page of the paper edition [1]. It's one thing to get the attention of some Googlers on the front page of HN, it's a whole other order of magnitude for the front page of the NYT. I cannot imagine how this hasn't become an urgent priority for Sundar himself to make sure this gets fixed and fast, at a minimum in these known situations. What could they possibly still be debating or deciding inside there at Google...? Police determined no charges, so restore the accounts. I don't see any potential risk, legal/reputational/otherwise, this would open up. Google's inaction here boggles the mind. [1] https://www.nytimes.com/issue/todayspaper/2022/08/22/todays-new-york-times https://www.nytimes.com/issue/todayspaper/2022/08/22/todays-...

bcatanzaro 4y ago

We need to regulate Google and require it to have a transparent appeals process.

tinalumfoil 4y ago
I don't think Google's doing this out of a deep concern for CSAM and hatred for due process, but I think it's in fact the law itself that essentially requires corporations to have these trigger bans otherwise they'd face liability. Think about Google's alternatives here. They could (1) not have CSAM filters -- your data is private no need for Google to scan anything -- in which case people would use their platform to distribute illegal content and they'd be a nice target to get massive damages from. Or (2) give people due process, rights, appeals, etc despite not having any of the tools of a court of law, it's even illegal for them to look at the evidence, and if they make the wrong decision or they make the right decision too late they're still liable for their platform being used for illegal activity. Keep in mind the only way to truly know if material is sexually explicit is to show it to Potter Stewart. He'll know it when he sees it. Or (3) be proactive in building systems that detect illegal use of their platform and aggressively, without process or appeal, remove any user of the platform the moment you have any evidence at all they are engaged in this activity. Because people will try to use Google's platform for illegal activity and Google knows this, (3) is not only the only sensible option but it's actually the only legal option. It would, in effect, be illegal for Google to do anything other than scanning all your files and aggressively ban people.
- treis 4y ago
  >use their platform to distribute I'm not a fan of this phrasing because it absolves Google of responsibility. If the CSAM is on a Google server then Google is possessing and distributing CSAM. You can say they're doing it unwittingly or unintentionally but that doesn't change the fact that they're doing it.
  - anon84873628 4y ago
    And that's the line of thinking that makes #3 the only option for Google.

sylware 4y ago

Isn't google/alphabet a part of the vanguard/blackrock galaxy now? Aren't blackrock/vanguard making the laws in the US? Like killing the planned obsolescence related bills?

jjulius 4y ago
>As company shareholders, BlackRock and Vanguard can vote on behalf of their clients at company shareholder meetings. Both firms also have “investment stewardship” functions, which enables the proxy votes. >BlackRock’s spokesperson said the votes can also be carried out by a portfolio manager – and in some cases at BlackRock, can be carried out by the clients themselves (here). >BlackRock and Vanguard do not “own” all the biggest corporations in the world. They invest trillions of dollars into leading companies on behalf of their clients, who ultimately own the shares. https://www.reuters.com/article/factcheck-business-investment/fact-check-video-claiming-blackrock-and-vanguard-own-all-the-biggest-corporations-in-the-world-is-missing-context-idUSL2N2WI1K4 https://www.reuters.com/article/factcheck-business-investmen...
- sylware 4y ago
  So blackrock/vanguard can "hide" their "big" investors, depending on the company "invested in", or is this public info? That said, it is actually blackrock/vanguard ppl on the boards, and often they are the biggest ones, and expecting no steering from them or the "big" investors behind would be... "unexpected". Their is also the massive debts with the gigantic interests to be paid all along the calendar year. In the case of starbucks, owners of the debts is public info or the web page is incomplete? That said, now I understand why online non-franchised/licensed starbucks stuff has the aspx file extension.
  - jjulius 4y ago
    >... it is actually blackrock/vanguard ppl on the boards... Citation needed.
    - sylware 4y ago

alexb_ 4y ago

This is an extremely good article. The hard part of this is everyone can agree that "legality is not morality" and that what is legal is not necessarily moral. But this is used for the wrong side of the argument - since the Bill of Rights is a legal document, some disregard it as any guidance for morality.

vorpalhex 4y ago
The Bill of Rights is a legal document based on a Moral set of beliefs.
- deleted 4y ago
  [deleted]

colinsane 4y ago

the article makes good points, but i don’t think the conclusion is obvious. for example, i want the freedom to not interact with Google. a ruling to require Google provide services to every individual might allow for a future corresponding ruling, that individuals must interact with Google (or at least, this becomes the socially acceptable view, which shapes laws later). wiser, i think, would be to prevent any single company from becoming critical to participation in society.

SpicyLemonZest 4y ago
I don't think that really solves anything here, because nothing in the story indicates that Google was critical to Mark's participation in society. He got new contact information and it sounds like he's been able to (with some effort) get all his non-Google accounts switched over. He didn't get fired, or evicted, or land in any legal trouble. He just liked Google's services and enjoyed using them until he was unfairly banned.

themacguffinman 4y ago

If regulation is the answer, I'd hope it is designed with the restraint that Madison had: only guarantee the deepest and most fundamental rights. It should not rule out the ability to use error-prone systems at all, at most it could restrict their use for the most severe consequences like broad account bans. Google can and should be reducing the scope of their automated action. I think error-prone systems are still necessary today to reduce costs to a tolerable modern level. I'm not inclined to throw out the baby with the bathwater regarding communication costs for the same reason I don't want $300 toasters even if "they don't make 'em like they used to". A good example is something that Facebook does: they have a blacklist of domains you can't send on Messenger, the message will be instantly blocked when you paste a detected link. From experience, this list is fairly large and very inaccurate, but the false positives for this system are annoying rather than catastrophic because all it does is block a specific message. If you don't like this, then you can use Signal or something, I don't want other people making that choice for me.

waylandsmithers 4y ago

My theory on why Google does this (and follows the same pattern in so many areas) is that having one person simply use common sense and reasonable judgment would be something of an admission that their software, created by the best and brightest engineers, isn't top notch.

paulcole 4y ago
Start with questioning whether the software was created by the best and brightest.

Animats 4y ago

There's a question as to whether Google is acting as an agent of the Government here. When the Government outsources something, that activity sometimes becomes subject to constitutional protections. Cases on this are iffy, though.[1] This has come up in reference to Amtrak, the Federal Reserve, Blackwater, and privately run prisons. Suppose Google analyzed all business data that passed through its servers looking for patterns of tax evasion. If the probability of tax evasion was high, reporting it to the IRS, which offers rewards, could be a profit center. [1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2394843 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2394843

treeman79 4y ago
Private ownership but government control is Fascism. https://youtu.be/PdOx36bZMBY https://youtu.be/PdOx36bZMBY Good recent is example is Facebook suppressing stories that the FBI wants suppressed.
- Animats 4y ago
  Actually, Reagan is misquoting there.[1] Something like that is often attributed to either Huey Long or Sinclair Lewis, but neither actually wrote that. Or even close. The closest match is from Sinclair Lewis' book "It Can't Happen Here": "He was afraid that the world struggle today was not of Communism against Fascism, but of tolerance against the bigotry that was preached equally by Communism and Fascism. But he saw too that in America the struggle was befogged by the fact that the worst Fascists were they who disowned the word “Fascism” and preached enslavement to Capitalism under the style of Constitutional and Traditional Native American Liberty." That book is very relevant today. [1] https://quoteinvestigator.com/2017/07/28/flag/ https://quoteinvestigator.com/2017/07/28/flag/
- vkou 4y ago
  > Good recent is example is Facebook suppressing stories that the FBI wants suppressed. Instead, Facebook should have been promoting stories that the FSB wanted promoted. :)
9ace07ac 4y ago

dcow 4y ago

Every passing day I feel that we are at the social limits of what our existing laws and documents can practically support without a pretty decent refactor. This essay is the perfect supporting argument, and the actions of Google the perfect case study. What will USA 2.0 look like? Is there a way to work on this without it being a bloody revolution? If we got 2/3rds of the population to green light a refactor of the constitution could it happen? Who would do it? Who are the right people? Importantly, if we want to preserve the spirit of the USA's founding documents, they need to evolve more formally and officially than supreme court case law and bloated regulatory agencies tied to deeply industry integrated lobbyists. I really wish writing laws was more like the RFC process. I don't think the solution is to "just run more stuff at home" as much as that tickles my technocratic jimmies.

merely-unlikely 4y ago
Lawmaking could learn a lot from the open source community. Imagine a wiki + git + democratic voting style system. All laws and changes would be hosted on a publicly available website. Any citizen could browse and submit "pull requests" which would then get debated and eventually voted on if they gain enough traction. Obviously such a system would be way more complicated than I can describe in a single HN comment, but I do think we have the technology to build a much more public, accessible, and directly democratic method of writing laws. As an aside, did you know no one even knows how many federal laws exist?[1]. [1] https://blogs.loc.gov/law/2013/03/frequent-reference-question-how-many-federal-laws-are-there/ https://blogs.loc.gov/law/2013/03/frequent-reference-questio...
- cube00 4y ago
  You can't expect the population to be across all laws and changes, that's why proportional elected officials with office staff exist.
  - merely-unlikely 4y ago
    I'd still rather see that happen out in the open instead of back room deals and bills ghost written by opaque lobbyists. I could even imagine a voting system that lets you name your favorite politician as your proxy. So politicians could collect the votes of their base into a block. Maybe even a mechanism of saying I want this expert to cast my vote on x topic, and this other thought leader on y topic. Independents might retain their vote on every issue if they care to. I'd probably go with a system with tiers of votes as an issue gains more public attention. Say a minor change is proposed on a niche law and those who are passionate about fishing rights in the Colorado River get involved and vote. Say 5k people vote. Then there's a cooling off period before implementation but all of a sudden the issue ends up in the news and 5m people are now interested so there's another vote with another cooling off period. Only this time 10m people would need to express interest to hold another vote. I don't know the perfect system but I'm confident we can come up with a more democratic process than what was possible 250 years ago. One in which the public more directly decides what issues get voted on. Congress has limited capacity and can't focus on everything but the work can be distributed if it were more open. A system in which experts could make proposals directly instead of being forced to educate and convince a politician the issue is even worth spending political capital on. And a system in which citizens can see all the changes being made publicly. It seems every time a major bill is passed it tends to be this huge amalgamation of new laws that often aren't even related. And then news articles follow pointing out how x, y, and z things got slipped into the 800 page bill no one in Congress actually bothered to read.

tarakat 4y ago

> We don’t know how many of those children were subsequently rescued, but a question worth posing to anyone unilaterally opposed to Google’s approach is how big that number would have to be to have made it worthwhile? [..] how big the number of false positives would have to be to shut the whole thing down? There's nothing worse or more myopic than this kind of "pragmatic", numerical approach towards liberty. As if living in a panopticon cage is desirable, as long as the wardens never make a mistake. Why is it myopic? Well, if being forced to use technology that betrays its user is okay as long as one number is sufficiently larger than another, then why not force all new houses to come with indoor microphones and cameras, always listening and watching - only for the most heinous crimes, of course! Are you "unilaterally opposed" to such a measure, despite how much sexual abuse and domestic violence and murder it would prevent? What about tracking and shock collars for everyone? How much violence could be stopped if everyone was forced to wear those! Don't like it? Not even if the false positive rate was zero?

lsh123 4y ago

I wonder if Google violated HIPAA by sending patient to doctor communication to a third party (CSAM).

thaumasiotes 4y ago
Google, not being a provider of healthcare or health insurance, cannot violate the Health Insurance Portability and Accountability Act. If Google deputized a VP to hunt you down and murder you in your sleep, that also would not violate HIPAA.
- ec109685 4y ago
  Also, the images uploaded to Google cloud were independent of sending the images to the doctor via his wife.
- dekhn 4y ago
  The (enterprise) cloud side of Google is a business associate (also covered in HIPAA) in some circumstances but I don't think that applies in this case.