8 ms·

It sounds more like they say "We're wrong and your account remains banned."

notch656a 4y ago
How could they conceivably even look at the data to unban the account though? That shit's radioactive, as soon as the AI flags it I don't know how a (non-LEO) human could possibly view it without running afoul of the law. There's no reasonable way to reverse this action.
- jacquesm 4y ago
  Non LEO are routinely employed by companies like Google, Facebook etc to review content. It's the most thankless job in the world.
  - notch656a 4y ago
    I don't think it's legal for those people to view it though if they have reason to believe it contains certain images of child abuse. Once the AI flags it that shit goes radioactive and is basically unviewable until LEO looks at it. Even if local LEO looks at it and clears it, that doesn't clear you from being federally prosecuted if you review the content later.
    - jacquesm 4y ago
      Here is how it worked when I ran a file sharing service: - We would allow people to upload files - these files would go into a 'holding tank' - the files would then get greenlit if some rudimentary algorithm determined they were landscapes or other innocent content - the remainder would be flagged for review - bulk images would be displayed in a grid of 8x8 thumbnails - the vast majority of those would not be reason for further action and would be greenlit, the remainder would go into yet another holding tank - now the pictures would be viewed full size, any kind of image that would be against the TOS would be rejected and a hash would be kept to avoid seeing it again - any kind of image that was deemed illegal or borderline would be passed on to LEO for another decision, possibly resulting in criminal charges against the uploader. The complications: - uploaders would try to mask their identity - uploaders would be all over the world - even after reviewing LEO would sometimes indicate that an image was inconclusive (in those cases we erred on the side of caution) -- In short: companies can and do work together with law enforcement to stop service abuse and illegal activities. Whether it is an AI that flags it or a human is from a legal perspective utterly immaterial, and if LEO clears it that definitely means that you are allowed to either view it or pass it on.
      - notch656a 4y ago
        If AI flag and human is "immaterial" difference as you noted, wouldn't AI flagging it as abuse material make it impossible to view it by a (non-LEO) human afterwards, as now that would show actual intent to view that material (it is illegal to view it with intent to view it). Surely your reviewers never deliberately looked at the material a second time after marking it as abuse material for law enforcement, as that would be legally risky I would think. The issue here is even if local LEO clear it, the federal prosecutor's opinion can differ and he can use the fact you knew AI flagged it and that you released it anyway to damn you. There are cases out there where local/state government cleared someone of wrong-doing but they still got hemmed up federally.
- anilakar 4y ago
  How can it be a match if these CSAM scanners are only comparing against a database of known material? Have we been lied to?
  - bonzini 4y ago
    The article says it's a combination of hash matching, AI and human review.
    - ClumsyPilot 4y ago
      thats equivalent to not saying anything
  - Normille 4y ago
    I raised this point a while back in another discussion and was assured this kind of thing couldn't happen: https://news.ycombinator.com/item?id=32178108#32178137 https://news.ycombinator.com/item?id=32178108#32178137 The biggest worry about these kind of situations is not that mistakes are made, but that it's almost impossible to make contact with a real human being to put things right [even, as in this case, when press and police are involved]. It's only a matter of time before someone loses their livelihood or even their life over this kind of cock-up.
    - notch656a 4y ago
      It's also easy to lose your livelihood or even life if you fail to eject something from your company's possession after you have reason to believe it contains certain materials of abuse. Federal and state prosecutor opinions differ, so clearance from local or state LEO (as done here) basically means dick. If you're wrong, then you get thrown in prison where you hear stories of "chomos" getting raped and shanked. If google is right, maybe they get a little bit of advertisement money. Imagine if this father did something later and the prosecutor uses this happening as evidence that this content was 'knowingly' harbored by the company. Seems like a bad calculus to me, just cut the guy loose whether he is innocent or not. You don't have the same rights to private service as you do rights in public.
- feanaro 4y ago
  > There's no reasonable way to reverse this action. How about "oh, everyone says we are wrong click to unban"?
  - bigiain 4y ago
    > How about "oh, everyone says we are wrong Not even "everyone". How about "law enforcement that we reported it to says we are wrong"?
    - notch656a 4y ago
      'local government/LEO gave the all clear' is not a tight defense against federal prosecution and conviction. For instance, Jeremy Kettler was convicted despite full local and state blessing for possession of a silencer in violation of federal law (he, the state, and local government all thought he was following all laws including federal ones, but it turned out federal prosecutor had a different opinion on what 'interstate commerce' is). I personally would never risk getting registered as an offender on the hopes some federal prosecutor has a different view than local LEO, it's just not worth unbanning the guy unless both the state and federal prosecutor have signed statements granting immunity from prosecution.
- Maro 4y ago
  LEO = Law Enforcement Officer? cmiiw
  - lostlogin 4y ago
    Thanks. Works better than Low Earth Orbit.

How could they conceivably even look at the data to unban the account though? That shit's radioactive, as soon as the AI flags it I don't know how a (non-LEO) human could possibly view it without running afoul of the law. There's no reasonable way to reverse this action.

jacquesm 4y ago
Non LEO are routinely employed by companies like Google, Facebook etc to review content. It's the most thankless job in the world.
- notch656a 4y ago
  I don't think it's legal for those people to view it though if they have reason to believe it contains certain images of child abuse. Once the AI flags it that shit goes radioactive and is basically unviewable until LEO looks at it. Even if local LEO looks at it and clears it, that doesn't clear you from being federally prosecuted if you review the content later.
  - jacquesm 4y ago
    Here is how it worked when I ran a file sharing service: - We would allow people to upload files - these files would go into a 'holding tank' - the files would then get greenlit if some rudimentary algorithm determined they were landscapes or other innocent content - the remainder would be flagged for review - bulk images would be displayed in a grid of 8x8 thumbnails - the vast majority of those would not be reason for further action and would be greenlit, the remainder would go into yet another holding tank - now the pictures would be viewed full size, any kind of image that would be against the TOS would be rejected and a hash would be kept to avoid seeing it again - any kind of image that was deemed illegal or borderline would be passed on to LEO for another decision, possibly resulting in criminal charges against the uploader. The complications: - uploaders would try to mask their identity - uploaders would be all over the world - even after reviewing LEO would sometimes indicate that an image was inconclusive (in those cases we erred on the side of caution) -- In short: companies can and do work together with law enforcement to stop service abuse and illegal activities. Whether it is an AI that flags it or a human is from a legal perspective utterly immaterial, and if LEO clears it that definitely means that you are allowed to either view it or pass it on.
    - notch656a 4y ago
      If AI flag and human is "immaterial" difference as you noted, wouldn't AI flagging it as abuse material make it impossible to view it by a (non-LEO) human afterwards, as now that would show actual intent to view that material (it is illegal to view it with intent to view it). Surely your reviewers never deliberately looked at the material a second time after marking it as abuse material for law enforcement, as that would be legally risky I would think. The issue here is even if local LEO clear it, the federal prosecutor's opinion can differ and he can use the fact you knew AI flagged it and that you released it anyway to damn you. There are cases out there where local/state government cleared someone of wrong-doing but they still got hemmed up federally.
anilakar 4y ago
How can it be a match if these CSAM scanners are only comparing against a database of known material? Have we been lied to?
- bonzini 4y ago
  The article says it's a combination of hash matching, AI and human review.
  - ClumsyPilot 4y ago
    thats equivalent to not saying anything
- Normille 4y ago
  I raised this point a while back in another discussion and was assured this kind of thing couldn't happen: https://news.ycombinator.com/item?id=32178108#32178137 https://news.ycombinator.com/item?id=32178108#32178137 The biggest worry about these kind of situations is not that mistakes are made, but that it's almost impossible to make contact with a real human being to put things right [even, as in this case, when press and police are involved]. It's only a matter of time before someone loses their livelihood or even their life over this kind of cock-up.
  - notch656a 4y ago
    It's also easy to lose your livelihood or even life if you fail to eject something from your company's possession after you have reason to believe it contains certain materials of abuse. Federal and state prosecutor opinions differ, so clearance from local or state LEO (as done here) basically means dick. If you're wrong, then you get thrown in prison where you hear stories of "chomos" getting raped and shanked. If google is right, maybe they get a little bit of advertisement money. Imagine if this father did something later and the prosecutor uses this happening as evidence that this content was 'knowingly' harbored by the company. Seems like a bad calculus to me, just cut the guy loose whether he is innocent or not. You don't have the same rights to private service as you do rights in public.
feanaro 4y ago
> There's no reasonable way to reverse this action. How about "oh, everyone says we are wrong click to unban"?
- bigiain 4y ago
  > How about "oh, everyone says we are wrong Not even "everyone". How about "law enforcement that we reported it to says we are wrong"?
  - notch656a 4y ago
    'local government/LEO gave the all clear' is not a tight defense against federal prosecution and conviction. For instance, Jeremy Kettler was convicted despite full local and state blessing for possession of a silencer in violation of federal law (he, the state, and local government all thought he was following all laws including federal ones, but it turned out federal prosecutor had a different opinion on what 'interstate commerce' is). I personally would never risk getting registered as an offender on the hopes some federal prosecutor has a different view than local LEO, it's just not worth unbanning the guy unless both the state and federal prosecutor have signed statements granting immunity from prosecution.
Maro 4y ago
LEO = Law Enforcement Officer? cmiiw
- lostlogin 4y ago
  Thanks. Works better than Low Earth Orbit.