4 ms·

The only problems I see are that someone could potentially upload a lower resolution version of the video a moderator has seen and then perhaps bypass the conte

by jsgo 7y ago

The only problems I see are that someone could potentially upload a lower resolution version of the video a moderator has seen and then perhaps bypass the content id system (I don't know how tolerant they tend to be for situations as that). There's also the aspect of if we make it search out for distinct frames (like "scene change" type things), there's the possibility that some safe things could become auto-moderated which would be another controversy. Perhaps if they could locate start/stop times for inappropriate elements of the source video and then check for videos containing any of those in a later uploaded video, but then instead of asking human moderators to discern "is this video good or bad" we're now forcing them to get timestamps of when bad elements take place which would force them to really watch the video. Dunno, I'm of the belief that it's just not an easily solved problem. I hope I'm wrong and they can toss money at the problem to fix it, but I genuinely don't know how they could.

luckylion 7y ago
Even with lower resolutions, fingerprinting (on a frame level) should still work relatively well, shouldn't it? If you get a certain level of confidence on each frame - if you have 80% of the frames with 80% confidence to be the same as $someVideo, that ought to be good enough overall to not get false positives but catch repeat views - how many 30s videos will randomly share 80% of their frames with a 90s video of an animal being abused? I assume that most of these videos aren't manipulated by professionals to escape a ban by looking completely different (which actually would make it a different video), but that they are mirrored, cut, have extra material appended or, as you suggested, have their resolution changed. I'd guess that videos are easier to ID than, say, image data, because you have plenty of material to work with. Are there well-organized troll armies attacking Facebook? Because trying to sneak a video in that shows an animal being abused and doing sophisticated changes to evade detection sounds weird on a platform where there isn't even (as far as I know) monetary gain connected to posting a video (as it might be on YT or for Email spam).
- xamuel 7y ago
  >how many 30s videos will randomly share 80% of their frames with a 90s video of an animal being abused? If a powerful actor wants the 30s video removed, and knows it would be removed if it shared 80% of its frames with a 90s video of an animal being abused, then it would not be technically difficult to engineer such a 90s video.
  - luckylion 7y ago
    Yeah, but that works once or twice, and then the defense catches on. Sure, it's not a perfect solution, but rather like spam filters, it gets you pretty close. Also, I don't assume that powerful actors are the issue here - they'll likely just call their contacts at a high price law firm and they in turn call FB to remove some video.
    - xamuel 7y ago
      >And then the defense catches on And has to redo everything. It's whack-a-mole. Doesn't have to be powerful actors really, either. There are many less powerful actors who would love to get selected things removed from YouTube. This isn't the same as spam mail filters, because you generally don't know what sort of emails other people are sending each other. If you did, you could deliberately try to manipulate filters to flag your competitors' emails as spam.
      - luckylion 7y ago
        > If you did, you could deliberately try to manipulate filters to flag your competitors' emails as spam. You could, it's just hard and generally not worth it. The same, I believe, goes for video manipulation. Sounds like hash collisions. Possible, sure, but not something you run on your cell in 30s.
        xamuel 7y ago
        It's eye opening when you realize how cut-throat competition can be when it comes to blackhat SEO etc. If you've discovered some longtail goldmine and you're the #2 result when people search for it and it's making you money, you have a big incentive to do anything it takes to nuke the #1 result. And before you say "You'll get caught and banned", obviously you do the nuking with a sockpuppet account. That's blackhat 101.
- jsgo 7y ago
  I started playing FFXIV a bit over a month ago. During that time, there was someone in the Novice Network (new player advice channel in game) posted a "does this look like an okay PLD build?" with a URL. I click the URL and it is a video of something being held down and attacked with a hatchet or something and it sounded like a child (as soon as I realized what was going on after like 1-2 seconds, I closed the video then reported the user and blocked them. It was a bit much and not the type of content I'd seek out, so that's probably why details are hazy). Reason I give that anecdote is for the last part: I don't think it is about monetizing this type of content and that they're only after the shock factor of it. Again, maybe they could automate safeguarding against it, but my guess is they'd find a way to get through. Like take a 5 minute YouTuber video and inject a slightly modified version of the shock video in it with hopes it bypasses the filter. Don't know, but wouldn't put it past anyone that would do this sort of thing.
  - luckylion 7y ago
    That sounds tough and like you were in luck that you reacted quickly. > Reason I give that anecdote is for the last part: I don't think it is about monetizing this type of content and that they're only after the shock factor of it. Possibly, I assume so as well, but I believe there's a difference in commitment. If I'm making money off of something, I'm going to put a lot of effort (at least until I approach $0) into evading filters, and can easily pay for advanced technical solutions (possibly hand-crafted for my purpose). If it's to mess with people, my effort and monetary investment will be much smaller. If I need to put significant time into each video upload only for it to be blocked after a handful of views, it doesn't scale. Trolls are an issue, of course, but video seems to be so hard to create/manipulate at scale and easier (I think) to check (compared to text for example), at least until there will be something like GPT2 for video.

by jsgo 7y ago

luckylion 7y ago

Even with lower resolutions, fingerprinting (on a frame level) should still work relatively well, shouldn't it? If you get a certain level of confidence on each frame - if you have 80% of the frames with 80% confidence to be the same as $someVideo, that ought to be good enough overall to not get false positives but catch repeat views - how many 30s videos will randomly share 80% of their frames with a 90s video of an animal being abused? I assume that most of these videos aren't manipulated by professionals to escape a ban by looking completely different (which actually would make it a different video), but that they are mirrored, cut, have extra material appended or, as you suggested, have their resolution changed. I'd guess that videos are easier to ID than, say, image data, because you have plenty of material to work with. Are there well-organized troll armies attacking Facebook? Because trying to sneak a video in that shows an animal being abused and doing sophisticated changes to evade detection sounds weird on a platform where there isn't even (as far as I know) monetary gain connected to posting a video (as it might be on YT or for Email spam).

xamuel 7y ago
>how many 30s videos will randomly share 80% of their frames with a 90s video of an animal being abused? If a powerful actor wants the 30s video removed, and knows it would be removed if it shared 80% of its frames with a 90s video of an animal being abused, then it would not be technically difficult to engineer such a 90s video.
- luckylion 7y ago
  Yeah, but that works once or twice, and then the defense catches on. Sure, it's not a perfect solution, but rather like spam filters, it gets you pretty close. Also, I don't assume that powerful actors are the issue here - they'll likely just call their contacts at a high price law firm and they in turn call FB to remove some video.
  - xamuel 7y ago
    >And then the defense catches on And has to redo everything. It's whack-a-mole. Doesn't have to be powerful actors really, either. There are many less powerful actors who would love to get selected things removed from YouTube. This isn't the same as spam mail filters, because you generally don't know what sort of emails other people are sending each other. If you did, you could deliberately try to manipulate filters to flag your competitors' emails as spam.
    - luckylion 7y ago
      > If you did, you could deliberately try to manipulate filters to flag your competitors' emails as spam. You could, it's just hard and generally not worth it. The same, I believe, goes for video manipulation. Sounds like hash collisions. Possible, sure, but not something you run on your cell in 30s.
      - xamuel 7y ago
        It's eye opening when you realize how cut-throat competition can be when it comes to blackhat SEO etc. If you've discovered some longtail goldmine and you're the #2 result when people search for it and it's making you money, you have a big incentive to do anything it takes to nuke the #1 result. And before you say "You'll get caught and banned", obviously you do the nuking with a sockpuppet account. That's blackhat 101.
jsgo 7y ago
I started playing FFXIV a bit over a month ago. During that time, there was someone in the Novice Network (new player advice channel in game) posted a "does this look like an okay PLD build?" with a URL. I click the URL and it is a video of something being held down and attacked with a hatchet or something and it sounded like a child (as soon as I realized what was going on after like 1-2 seconds, I closed the video then reported the user and blocked them. It was a bit much and not the type of content I'd seek out, so that's probably why details are hazy). Reason I give that anecdote is for the last part: I don't think it is about monetizing this type of content and that they're only after the shock factor of it. Again, maybe they could automate safeguarding against it, but my guess is they'd find a way to get through. Like take a 5 minute YouTuber video and inject a slightly modified version of the shock video in it with hopes it bypasses the filter. Don't know, but wouldn't put it past anyone that would do this sort of thing.
- luckylion 7y ago
  That sounds tough and like you were in luck that you reacted quickly. > Reason I give that anecdote is for the last part: I don't think it is about monetizing this type of content and that they're only after the shock factor of it. Possibly, I assume so as well, but I believe there's a difference in commitment. If I'm making money off of something, I'm going to put a lot of effort (at least until I approach $0) into evading filters, and can easily pay for advanced technical solutions (possibly hand-crafted for my purpose). If it's to mess with people, my effort and monetary investment will be much smaller. If I need to put significant time into each video upload only for it to be blocked after a handful of views, it doesn't scale. Trolls are an issue, of course, but video seems to be so hard to create/manipulate at scale and easier (I think) to check (compared to text for example), at least until there will be something like GPT2 for video.