3 ms·
> Please take careful note: If you do not do proper risk analysis of your platform you WILL be used to perpetrate crime and if you find that you did not do what
by lazyjones 7y ago
> Please take careful note: If you do not do proper risk analysis of your platform you WILL be used to perpetrate crime and if you find that you did not do what you could have done to stop it you may end up being an accessory to that crime.
Since we're talking about image hosters (I built one some years ago): how do you solve the dilemma that a) you can be sure that at some point somebody will upload images that are illegal to host and look at, b) in order to identify and remove them you'll have to look at them or pay someone to look at them.
What we did (before ML became widely available/easily usable, not sure if there are ready-to-use models for filtering all illegal images out there these days - nobody would want to train one themselves using illegal content...) was to look at access logs and remove all images that had suspicious hostnames in the "Referer" header. It wasn't enough apparently, it got my successor into trouble and a new dilemma (take down the platform immediately, or operate it as a honeypot to help authorities who asked for it, potentially incriminating themselves for not taking down illegal content after being made aware of it... [police can't give you a carte blanche for operating a honeypot, a DA/prosecutor can still prosecute]).
Edit: it doesn't seem to be available for my country, but a service like https://azure.microsoft.com/en-us/pricing/details/cognitive-services/content-moderator/ https://azure.microsoft.com/en-us/pricing/details/cognitive-... might help nowadays.
- jacquesm 7y agoPhotoDNA (also Microsoft).
- Mirioron 7y agoWouldn't it be reasonable to assume that if you incidentally stumble upon an image you're not allowed to look at that it is not something that you could be prosecuted over? That is, as long as you weren't going looking for that specific type of image it should be okay.
- lazyjones 7y agoPossibly, it's for the judge to decide. How can you prove that it was accidental and not intentional? And how to do find an employee who doesn't mind checking questionable material (even pre-filtered/flagged by a ML model) every day?
- Mirioron 7y ago>How can you prove that it was accidental and not intentional? I imagine that if you happen across it while moderating then you won't really know what you'll find, because you're relying on reports or random browsing to find offending content. That is, as long as you're following reasonable procedures for moderating then that should be good enough, no?
- lazyjones 7y agoSo, what is a reasonable procedure? Look at content reported to be illegal yourself (possibly already illegal since you had prior notice it was illegal, certainly not good for being able to sleep at night) and then delete it? Or delete it without checking whenever it gets reported and thereby giving users a tool to sabotage other users by misreporting their content (fun for political discussions etc.)?
- closeparen 7y agoNot a lawyer, but pretty sure this depends on whether the crime is “strict liability” or requires intent.