8 ms·

This is an amazing story. It really demonstrates the way we pave our road to hell with good intentions... We should really do something about this issue, where

This is an amazing story. It really demonstrates the way we pave our road to hell with good intentions... We should really do something about this issue, where so few companies (arguably, a single one) hold so much power over the most fundamental technology of the era.

ocdtrekkie 6y ago
The solution is simple: Liability. As soon as it becomes legally infeasible to let algorithms block people, it will stop happening. Make it easy and affordable to submit legal complaints for tech misbehavior and make the penalties hurt.
- vkou 6y ago
  Ah, so you suggest liability for the vendors of the software blocking websites, with, in practice [1], no liability for the operators of a compromised website, if it is phishing/malware? This is a great approach, if your goal is to optimize for increasing the amount of dangerous crap on the web. But, eh, that's surely worth it, because the profitability of startups is more important then little things like the security of the average netizen... [1] Even if you make the operators liable [2], in practice, you'll never be able to collect from most of them. Whereas the blacklist curators are a singular, convenient target... [2] If you can demonstrate how the operators of compromised websites can be held liable for all the harm they cause, I will happily agree that we should do away with blacklists. Unfortunately, the technical and legislative solutions for this are much worse than the disease you are trying to treat.
  - smsm42 6y ago
    Since phishing is not going to go anywhere with or without blacklists - for obvious reasons, e.g. lists can't cover everything, and you can't add sites to the list instantly - I am willing to tolerate a slight increase in fishing which is going to exist anyway in exchange for not having Google (or any other megacorp, or any other organization for that matter) as a gatekeeper of everybody's access to the internet. The potential for abuse of such power is much greater and much more dangerous than the danger from tiny increase of phishing.
    - mewse 6y ago
      > I am willing to tolerate a slight increase in fishing According to Google's most recent transparency report[1], as of December 20th of last year they were blocking around 27,000 malware distribution sites and a little over 2,000,000 phishing sites. In your view, would turning off those blacklists and allowing those >2,000,000 sites to become functional again count as a "slight" increase? (edit: That's a real question, incidentally, not a disagreement or an attempt at a 'zing'; I have no knowledge in this area but went to look up the numbers, and am curious whether 2,000,000 is truly a vanishingly small amount, relative to everything else that's out there that's not already on the list) [1]: https://transparencyreport.google.com/safe-browsing/overview https://transparencyreport.google.com/safe-browsing/overview
      - smsm42 6y ago
        I'm not sure what is counted as "sites" - i.e. if Google closes foo.bar/baz123 and the same server gets assigned bar.foo/zab345 and continues to serve malware, is it 2 separate sites? Did Google really achieve this much by forcing the changing of the URL? Sure, bunch of people that got the phish link in the mail that was sent before switch but then shut down won't be phished, but I have no idea how much that changes the picture - I'm sure phishers are well aware that their domains are short-lived and already adapted for that, otherwise they'd be extinct. However, I'd be glad to read some field-validated data about how much closing those 2M sites, whatever is meant by "sites", actually helps against phishing. I mean if we could trust Google (or anybody else of that kind) to have blacklist strictly limited to reasonable definition of malware and phishing, and knew that usage of such list if strictly voluntary under control of the user, it would be an acceptable, if decidedly imperfect, remedy. But we know we can't trust any of this, even if whoever works on this at Google right now are sincerely ironclad committed to never any mission creep and abuse happen, once the means exist, these people can always be replaced with others that would use it to fight "misinformation", or "incitement", or "blasphemy", or whatever it is in fashion to fight this week. There's no mechanism that ensures it won't be abused, and abuse is very easy once the system is deployed. Moreover, we (as, people not in control of Google's decisions) have absolutely no means to prevent any abuse of this, since Google owns the whole setup and we have no voice in their decision making process. Given that, it seems to be prudent to make all effort to reject it while we still can. Otherwise next time you'd want to make a site questioning Google's decisions about the malware list, nobody would be able to read it because it'd be marked as a malware site.
teamspirit 6y ago
Here-here! I really wish there was more human involvement in a lot of these seemingly arbitrary AI-taken actions. Everything from app review to websites and more. This heavy reliance on automated systems has led us down this road. Shoot, keep it, just give us the option to guarantee human review - with of course transparency. We don't need anymore "some human looked at this and agreed, the decisions is final, goodbye." I know it's easier said than done, especially when taking the scale of the requests into account, but the alternative has, does, and will continue to do serious harm to the many people and businesses caught in this wide, automated net.
- alfiedotwtf 6y ago
  > I really wish there was more human involvement in a lot of these seemingly arbitrary AI-taken actions. Narrator: but it was only ever to get worse
- pryce 6y ago
  It's interesting how closely the unfolding of this awful scenario has followed an entirely predictable path based on the shifting incentives: now hundreds of thousands of businesses face the same massive hazard of blocklisted without adequate human review, and with mediocre options to respond to it if it occurs. Without a shift in incentives, its unlikely the outlook will improve. Unless the organisations affected (and those vulnerable) can organise and exert enough pressure for google to notice and adjust course, we're probably going to be stuck like this -or worse- for a long time.
  - craftinator 6y ago
    > this awful scenario has followed an entirely predictable path The interesting things about predictable paths is that at the start there are a LOT of them, then over time there becomes just one of them. I don't see that this path was any more predictable at the start than any other.
  - Retric 6y ago
    Blacklisting a site incorrectly seems like a perfectly adequate reason for a defamation lawsuit. So, I think the real issue is with the legal system.
ehsankia 6y ago
Are you implying that the list no longer has a good intention? I wouldn't be surprised if there are multiple orders of magnitude more phishing and hacked websites in 2021 than there was in 2004. Even with human checking, I doubt you'll even have 0% failure rate. Is the solution to just give up on blocking phishing sites?
- cft 6y ago
  The solution is for the legitimate sites that are driven out of business by Google AI to sue Google for tortuous interference and libel.
  - nearbuy 6y ago
    This helps one group and hurts another. If Google is liable for blocking potential malware and phishing pages, they'll either stop blocking it, or adjust their algorithm to strongly err on the side of allowing phishing sites. Businesses become safer, but more regular people will get phished.
    - coldtea 6y ago
      >or adjust their algorithm to strongly err on the side of allowing phishing sites. It'a not the role of Google to disallow phishing sites (as a browser) just like it's not the role of the ISP. Make it hookable so people can chose their own phising protection service.
      - msh 6y ago
        And then they will choose the same block list and sites will have the same problem.
        coldtea 6y ago
        All? I doubt it. Not to mention they could offer control to override whatever you like.
qwantim1 6y ago
This was the case with railroads too, only a few controlled the biggest and most transforming and business-integral tech of 1800s. Prior to that it was those that controlled the printing presses. ... History continues to repeat itself.

ocdtrekkie 6y ago

The solution is simple: Liability. As soon as it becomes legally infeasible to let algorithms block people, it will stop happening. Make it easy and affordable to submit legal complaints for tech misbehavior and make the penalties hurt.

vkou 6y ago
Ah, so you suggest liability for the vendors of the software blocking websites, with, in practice [1], no liability for the operators of a compromised website, if it is phishing/malware? This is a great approach, if your goal is to optimize for increasing the amount of dangerous crap on the web. But, eh, that's surely worth it, because the profitability of startups is more important then little things like the security of the average netizen... [1] Even if you make the operators liable [2], in practice, you'll never be able to collect from most of them. Whereas the blacklist curators are a singular, convenient target... [2] If you can demonstrate how the operators of compromised websites can be held liable for all the harm they cause, I will happily agree that we should do away with blacklists. Unfortunately, the technical and legislative solutions for this are much worse than the disease you are trying to treat.
- smsm42 6y ago
  Since phishing is not going to go anywhere with or without blacklists - for obvious reasons, e.g. lists can't cover everything, and you can't add sites to the list instantly - I am willing to tolerate a slight increase in fishing which is going to exist anyway in exchange for not having Google (or any other megacorp, or any other organization for that matter) as a gatekeeper of everybody's access to the internet. The potential for abuse of such power is much greater and much more dangerous than the danger from tiny increase of phishing.
  - mewse 6y ago
    > I am willing to tolerate a slight increase in fishing According to Google's most recent transparency report[1], as of December 20th of last year they were blocking around 27,000 malware distribution sites and a little over 2,000,000 phishing sites. In your view, would turning off those blacklists and allowing those >2,000,000 sites to become functional again count as a "slight" increase? (edit: That's a real question, incidentally, not a disagreement or an attempt at a 'zing'; I have no knowledge in this area but went to look up the numbers, and am curious whether 2,000,000 is truly a vanishingly small amount, relative to everything else that's out there that's not already on the list) [1]: https://transparencyreport.google.com/safe-browsing/overview https://transparencyreport.google.com/safe-browsing/overview
    - smsm42 6y ago
      I'm not sure what is counted as "sites" - i.e. if Google closes foo.bar/baz123 and the same server gets assigned bar.foo/zab345 and continues to serve malware, is it 2 separate sites? Did Google really achieve this much by forcing the changing of the URL? Sure, bunch of people that got the phish link in the mail that was sent before switch but then shut down won't be phished, but I have no idea how much that changes the picture - I'm sure phishers are well aware that their domains are short-lived and already adapted for that, otherwise they'd be extinct. However, I'd be glad to read some field-validated data about how much closing those 2M sites, whatever is meant by "sites", actually helps against phishing. I mean if we could trust Google (or anybody else of that kind) to have blacklist strictly limited to reasonable definition of malware and phishing, and knew that usage of such list if strictly voluntary under control of the user, it would be an acceptable, if decidedly imperfect, remedy. But we know we can't trust any of this, even if whoever works on this at Google right now are sincerely ironclad committed to never any mission creep and abuse happen, once the means exist, these people can always be replaced with others that would use it to fight "misinformation", or "incitement", or "blasphemy", or whatever it is in fashion to fight this week. There's no mechanism that ensures it won't be abused, and abuse is very easy once the system is deployed. Moreover, we (as, people not in control of Google's decisions) have absolutely no means to prevent any abuse of this, since Google owns the whole setup and we have no voice in their decision making process. Given that, it seems to be prudent to make all effort to reject it while we still can. Otherwise next time you'd want to make a site questioning Google's decisions about the malware list, nobody would be able to read it because it'd be marked as a malware site.

teamspirit 6y ago

Here-here! I really wish there was more human involvement in a lot of these seemingly arbitrary AI-taken actions. Everything from app review to websites and more. This heavy reliance on automated systems has led us down this road. Shoot, keep it, just give us the option to guarantee human review - with of course transparency. We don't need anymore "some human looked at this and agreed, the decisions is final, goodbye." I know it's easier said than done, especially when taking the scale of the requests into account, but the alternative has, does, and will continue to do serious harm to the many people and businesses caught in this wide, automated net.

alfiedotwtf 6y ago
> I really wish there was more human involvement in a lot of these seemingly arbitrary AI-taken actions. Narrator: but it was only ever to get worse
pryce 6y ago
It's interesting how closely the unfolding of this awful scenario has followed an entirely predictable path based on the shifting incentives: now hundreds of thousands of businesses face the same massive hazard of blocklisted without adequate human review, and with mediocre options to respond to it if it occurs. Without a shift in incentives, its unlikely the outlook will improve. Unless the organisations affected (and those vulnerable) can organise and exert enough pressure for google to notice and adjust course, we're probably going to be stuck like this -or worse- for a long time.
- craftinator 6y ago
  > this awful scenario has followed an entirely predictable path The interesting things about predictable paths is that at the start there are a LOT of them, then over time there becomes just one of them. I don't see that this path was any more predictable at the start than any other.
- Retric 6y ago
  Blacklisting a site incorrectly seems like a perfectly adequate reason for a defamation lawsuit. So, I think the real issue is with the legal system.

ehsankia 6y ago

Are you implying that the list no longer has a good intention? I wouldn't be surprised if there are multiple orders of magnitude more phishing and hacked websites in 2021 than there was in 2004. Even with human checking, I doubt you'll even have 0% failure rate. Is the solution to just give up on blocking phishing sites?

cft 6y ago
The solution is for the legitimate sites that are driven out of business by Google AI to sue Google for tortuous interference and libel.
- nearbuy 6y ago
  This helps one group and hurts another. If Google is liable for blocking potential malware and phishing pages, they'll either stop blocking it, or adjust their algorithm to strongly err on the side of allowing phishing sites. Businesses become safer, but more regular people will get phished.
  - coldtea 6y ago
    >or adjust their algorithm to strongly err on the side of allowing phishing sites. It'a not the role of Google to disallow phishing sites (as a browser) just like it's not the role of the ISP. Make it hookable so people can chose their own phising protection service.
    - msh 6y ago
      And then they will choose the same block list and sites will have the same problem.
      - coldtea 6y ago
        All? I doubt it. Not to mention they could offer control to override whatever you like.

qwantim1 6y ago

This was the case with railroads too, only a few controlled the biggest and most transforming and business-integral tech of 1800s. Prior to that it was those that controlled the printing presses. ... History continues to repeat itself.