18 ms·

Show HN: Firefox Addon to Filter NSFW Content

62 points by _zephyr 4y ago

deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
_zephyr 4y ago
When I was a kid, I remember enjoying figuring out how to bypass our internet filters at school by using the trick where you can convert an IP address into a number. I had never been too keen on filters but after that incident it planted a seed of an idea: could there be a better way? Perhaps make the process more intelligent? Watching the incredible advancement of image recognition over the last decade, it seemed that maybe someone could build something better for the browser - something that actually scanned the content directly rather than relying on lists. When Firefox 57 came out several years ago, it added the webRequest.StreamFilter API and as soon as I saw that I knew that I wanted to take a crack at creating a content-based filter of my own with a focus on NSFW pictures. That API is critical because it allows for truly blocking at the stream level, rather than relying on web page fixups - which also means no Chrome. I soon found Tensorflow.js which made things much easier. By late 2018, I had a first cut and I released it on the Mozilla Addons site as Wingman Jr. It's been quite a journey since then! I developed the bulk of the code but have had a bit of community help as well. There is now a small but steady user base, and I've learned so much. * NSFW content detection is a hard task - I believe much harder than just general image recognition, and whereas many think of of image recognition as "solved", I would argue that NSFW content detection is far from it. * Having your own dedicated toolset to assist in creating and managing the dataset is invaluable. I was surprised to see that as a single individual it was quite feasible to create a fully-curated dataset in the hundreds of thousands. * Servers are more different than laptops/desktops than one might think. I had a good time setting up an old Supermicro with K80's on a budget. * For this problem at least, even several hundred thousand images was still not enough to see an advantage on training from scratch vs. tuning a pre-trained model. * Fun ways to slice up GIFs at a "transport" layer to allow for good-enough filtering on frames. * Character encoding detection is a hard problem, and the existing filtering API's don't do a good job of helping the developer down the right path. So with development being relatively stable for quite some time, why share now? Well, I've kept the addon fairly "primordial" in the sense that I haven't tried to cater too heavily to narrowed use cases yet. Three general use cases seem to be represented based on user feedback - there are others but so far this is what is being said: * Adults casually enabling it for daily browsing. Think things like browsing stock photo sites, Google Images, etc. * Adults struggling with pornography and looking for tools to help. * Adults looking for an extra safety net when their kids browse the web. I'm contemplating adding more specific feature sets in one or more of these areas, but thought that it might be a good time to put it out there and get some perspectives. The tech is far from perfect, but it seems that it is good enough that it is helpful for some users. It's also my hope that there's potentially some things to share here that the HN crew might find of interest. (Although I can assure the UX is not currently one of them!) * The addon itself (https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-filter/ https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-fi... and https://github.com/wingman-jr-addon/wingman_jr https://github.com/wingman-jr-addon/wingman_jr) - maybe it's not your thing, but if you're like me, there's a good chance somebody in your family might find it useful * The model (https://github.com/wingman-jr-addon/model https://github.com/wingman-jr-addon/model) - I've tried to make a competitive model for its size, such that enterprising individuals can try this as an alternative to paying for API calls. It does not use NSFW.js as a base. Both .h5 and TF.js models are provided. Maybe it'll be good enough for your use case? * Real world examples of the webRequest.StreamFilter API. In particular, the bit about character encoding is probably worth a short read if you're thinking of using this API yourself. See https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882415a33433aa3066c4d01101dbc1dfffa/background.js#L578 https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882... * Examples of image-based logging for Firefox. * Some fun GIF parsing stuff! Thanks!
- itake 4y ago
  How big is the model? I see the extension itself is 15MB, but does it download the model later?
  - _zephyr 4y ago
    That's with the model actually! It's copied into here: https://github.com/wingman-jr-addon/wingman_jr/tree/master/sqrxr_112_graphopt https://github.com/wingman-jr-addon/wingman_jr/tree/master/s... The model is 13.7MB currently.
laputan_machine 4y ago
Maybe I am being stupid, but I cannot see anywhere where you describe what you consider to be "NSFW"
- moffkalast 4y ago
  The description itself is NSFW so it got deleted the second they wrote it since they were running the addon at the time. Just goes to show how effective it is /s
- bertman 4y ago
  Looking at the code, it looks like they're using TensorFlow to rate images, videos etc. [0], specifically the SQRX family of models [1]. [0]: https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882415a33433aa3066c4d01101dbc1dfffa/processor.js#LL162C14-L162C14 https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882... [1]: https://github.com/wingman-jr-addon/model/blob/master/sqrxr_112/README.md https://github.com/wingman-jr-addon/model/blob/master/sqrxr_...
- _zephyr 4y ago
  The "About this extension" section has in the top paragraph: It targets pornography, borderline sexualized content, as well as other graphic images that are not safe for work (NSFW) or may be inappropriate for children. And then more details as the other commenter pointed out: https://github.com/wingman-jr-addon/model#dataset https://github.com/wingman-jr-addon/model#dataset Your response is interesting because it tells me you maybe expected it to be in a different spot - was there a specific spot you were looking at? Might help me improve the descriptions.
  - laputan_machine 4y ago
    Ok, then what I mean is "what do you consider to be pornographic"? In some countries it's not NSFW to see a naked breast, I am going to assume this software does not take that into account. Am I right? A bigger point one could make is that this is projecting your cultural norms onto the internet as the "default". It's a cool idea, though.
deleted 4y ago
[deleted]
jaimex2 4y ago
I'm guessing this will never run on Chrome, especially after manifest v3?
- ivanmontillam 4y ago
  Your guess is correct. By killing the filtering possibility, Chrome made itself a weaker browser from the NSFW-prevention standpoint as well.
  - _zephyr 4y ago
    While this addon can never run on Chrome, you may find this other project to be of interest: https://github.com/nsfw-filter/nsfw-filter https://github.com/nsfw-filter/nsfw-filter I've talked with the developers there from time to time.
mlindner 4y ago
I don't get why these filters are needed. I was relatively early to the internet compared to most people (I'm 33 but I grew up on the internet since around the age of 7) yet I grew up perfectly fine without needing to be blocked from adult content. A kid seeing a sex scene in something they stumble upon is not going to scar them for life... Additionally I was never forced to use a piece of school equipment that had filtering. The most I experienced filtering was when I, on the rare case, happened to use the computers at the library instead of the ones at home and laughed at/was annoyed by the silly filtering. Much worse than adult content is addictive content that forms obsessive behaviors (things like Tiktok or more lightly social media in general). The other thing to be wary about is when it's possible for a child to give their private information away to someone in a way that can harm themselves. i.e. make sure cameras are non-functional on any child's device. That wasn't an issue when I was a kid but is very much a danger today.
- pariainterpares 4y ago
  Porn addicts might need filters like these.
- the-printer 4y ago
  I…I…I… Shame on people who would even fathom forbidding access to certain content on the devices that they likely pay for, internet that they pay for, homes that they pay for, and to children who they raise. Shame.
  - a1369209993 4y ago
    Exactly. Child abuse is never okay, especially if the perpetrators think they have a legitimate rationalization for it.
  - code_runner 4y ago
    The gall people have to make their own choices… and they won’t even affect me? Appalling.
- culopatin 4y ago
  “I don’t need it so no one needs it” is the summary of your post. Not a very constructive statement, and you’re not asking a question either, because if you actually wanted to know you would’ve looked it up, or if you’re so in tune with the needs of others, you’d already know. So what sort of conversation are you aiming to have?
someweirdperson 4y ago
Does it have an option to filter both ways? Do not show NSFW vs. show only NSFW? The latter would probably be use a lot more.
- _8j50 4y ago
  Doubt it, there is no pressing need to exclude non-nsfw but parental control and porn addiction help have significant demand as well as people who fear getting accidentally in trouble at work.
  - IgorPartola 4y ago
    Dollars to donuts, I bet porn has a lot more demand than porn addiction help.
    - _8j50 4y ago
      For sure but it isn't drowned out by non-porn if you look for it is what I mean.
    - gjm11 4y ago
      To be a bit more explicit than badrabbit about why the symmetry isn't there: The problem a filter like this is meant to solve is: You go on the web reading random stuff, and occasionally you run into something pornographic when that isn't what you're after, and it's unwanted (because you're at work, because you find pornography disturbing, or whatever). So the problem its inverse would solve would be: You go on the web reading random stuff, and occasionally you run into something non-pornographic, and that spoils your fun. Or maybe: You go on the web looking for pornography, and occasionally you run into something non-pornographic, and that spoils your fun. So far as I am aware, those are not problems anyone has. Variations on the theme: You might be looking for some specific variety of pornography and keep finding pages with different kinds. I don't think this sort of tool would help with that; all it can do is filter out unwanted bits of pages you've already found. Or you might be successfully finding pornography but not want to see any other clutter -- e.g., advertisements on your screen distracting you from it. But I'm pretty sure the only people advertising on pornographic websites are highly NSFW themselves, so I wouldn't expect a reverse-NSFW-filter to help with that either; what you want for that is an ad blocker.
2Gkashmiri 4y ago
i tested this on reddit r/celebs and got nice results. nice as in, sure, photos were marked and it did a good job.... is this selfhosted as in no external connections?
- _zephyr 4y ago
  Correct, operations are all done fully local to the browser. If you opt in to the Cloudflare DNS option, it'll check using the hostname only.
  - 2Gkashmiri 4y ago
    ok. that is super cool. thank you for making this nice piece of tech. hopefully it can be integrated into more stuff
pinktriangle 4y ago
I can't help but find the whole NSFW filtering thing incredibly funny since I found out there's a "healthy" fetish community working on this problem, with Chrome extensions like "Beta Safety" or "Beta Protection"
- User23 4y ago
  That reminds me of repackaging a traditional historical male dominated marriage, including occasional corporal punishment, as a BDSM kink to make it normal again. You think you’re seeing a woman in a dress, but actually it’s fetish gear showing her submission.
  - Scharkenberg 4y ago
    General reminder: BDSM does not imply "man dom, woman sub". In fact it doesn't even imply how many people are involved and what their genders are. Edit: Why am I being downvoted for pointing out something objectively true?
    - sdf4j 4y ago
      This is hacker news
    - PenguinCoder 4y ago
      Why are you asking _why am I getting downvoted _ ? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
      - Scharkenberg 4y ago
        Because I was at -1 after just 10 minutes and no one was engaging in discussion about why what I said would not be correct? And, please, let's not pretend to be socially inept machines - everyone here chooses to ignore a guideline or two when it's convenient - at least I was looking for actual feedback, as I was sincerely puzzled by the situation.
  - 4y ago
Maxburn 4y ago
I don’t know why but this immediately make me think of the “cloud to butt” addon.
bsnnkv 4y ago
Looks like a great tool for recovering addicts everywhere. I hope to see more innovation in this space, particularly on mobile.
- _zephyr 4y ago
  What kind of innovation would you hope for on mobile? In theory this can work for Firefox for Android but ... the experience isn't very smooth yet.
  - bsnnkv 4y ago
    Something that could read the screen at any point in time, across apps and flag/filter NSFW images, all running with a local model and no outgoing internet connections would be pretty neat.
    - _zephyr 4y ago
      Unfortunately my experiences with mobile app stores put that likely in the "not possible at all" category rather than the "tricky" category like it might be for desktops.
noasaservice 4y ago
- sdf4j 4y ago
  [flagged]
vitorbaptistaa 4y ago
I'd love an extension like this to remove useless news from my favorite newspapers. Anything related to murders would be a great start. I want to read the news, but lately there has been so many news along the lines of "X person was murdered in a city far away" that it's becoming unusable.

62 points by _zephyr 4y ago

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

_zephyr 4y ago

When I was a kid, I remember enjoying figuring out how to bypass our internet filters at school by using the trick where you can convert an IP address into a number. I had never been too keen on filters but after that incident it planted a seed of an idea: could there be a better way? Perhaps make the process more intelligent? Watching the incredible advancement of image recognition over the last decade, it seemed that maybe someone could build something better for the browser - something that actually scanned the content directly rather than relying on lists. When Firefox 57 came out several years ago, it added the webRequest.StreamFilter API and as soon as I saw that I knew that I wanted to take a crack at creating a content-based filter of my own with a focus on NSFW pictures. That API is critical because it allows for truly blocking at the stream level, rather than relying on web page fixups - which also means no Chrome. I soon found Tensorflow.js which made things much easier. By late 2018, I had a first cut and I released it on the Mozilla Addons site as Wingman Jr. It's been quite a journey since then! I developed the bulk of the code but have had a bit of community help as well. There is now a small but steady user base, and I've learned so much. * NSFW content detection is a hard task - I believe much harder than just general image recognition, and whereas many think of of image recognition as "solved", I would argue that NSFW content detection is far from it. * Having your own dedicated toolset to assist in creating and managing the dataset is invaluable. I was surprised to see that as a single individual it was quite feasible to create a fully-curated dataset in the hundreds of thousands. * Servers are more different than laptops/desktops than one might think. I had a good time setting up an old Supermicro with K80's on a budget. * For this problem at least, even several hundred thousand images was still not enough to see an advantage on training from scratch vs. tuning a pre-trained model. * Fun ways to slice up GIFs at a "transport" layer to allow for good-enough filtering on frames. * Character encoding detection is a hard problem, and the existing filtering API's don't do a good job of helping the developer down the right path. So with development being relatively stable for quite some time, why share now? Well, I've kept the addon fairly "primordial" in the sense that I haven't tried to cater too heavily to narrowed use cases yet. Three general use cases seem to be represented based on user feedback - there are others but so far this is what is being said: * Adults casually enabling it for daily browsing. Think things like browsing stock photo sites, Google Images, etc. * Adults struggling with pornography and looking for tools to help. * Adults looking for an extra safety net when their kids browse the web. I'm contemplating adding more specific feature sets in one or more of these areas, but thought that it might be a good time to put it out there and get some perspectives. The tech is far from perfect, but it seems that it is good enough that it is helpful for some users. It's also my hope that there's potentially some things to share here that the HN crew might find of interest. (Although I can assure the UX is not currently one of them!) * The addon itself (https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-filter/ https://addons.mozilla.org/en-US/firefox/addon/wingman-jr-fi... and https://github.com/wingman-jr-addon/wingman_jr https://github.com/wingman-jr-addon/wingman_jr) - maybe it's not your thing, but if you're like me, there's a good chance somebody in your family might find it useful * The model (https://github.com/wingman-jr-addon/model https://github.com/wingman-jr-addon/model) - I've tried to make a competitive model for its size, such that enterprising individuals can try this as an alternative to paying for API calls. It does not use NSFW.js as a base. Both .h5 and TF.js models are provided. Maybe it'll be good enough for your use case? * Real world examples of the webRequest.StreamFilter API. In particular, the bit about character encoding is probably worth a short read if you're thinking of using this API yourself. See https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882415a33433aa3066c4d01101dbc1dfffa/background.js#L578 https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882... * Examples of image-based logging for Firefox. * Some fun GIF parsing stuff! Thanks!

itake 4y ago
How big is the model? I see the extension itself is 15MB, but does it download the model later?
- _zephyr 4y ago
  That's with the model actually! It's copied into here: https://github.com/wingman-jr-addon/wingman_jr/tree/master/sqrxr_112_graphopt https://github.com/wingman-jr-addon/wingman_jr/tree/master/s... The model is 13.7MB currently.

laputan_machine 4y ago

Maybe I am being stupid, but I cannot see anywhere where you describe what you consider to be "NSFW"

moffkalast 4y ago
The description itself is NSFW so it got deleted the second they wrote it since they were running the addon at the time. Just goes to show how effective it is /s
bertman 4y ago
Looking at the code, it looks like they're using TensorFlow to rate images, videos etc. [0], specifically the SQRX family of models [1]. [0]: https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882415a33433aa3066c4d01101dbc1dfffa/processor.js#LL162C14-L162C14 https://github.com/wingman-jr-addon/wingman_jr/blob/79a1a882... [1]: https://github.com/wingman-jr-addon/model/blob/master/sqrxr_112/README.md https://github.com/wingman-jr-addon/model/blob/master/sqrxr_...
_zephyr 4y ago
The "About this extension" section has in the top paragraph: It targets pornography, borderline sexualized content, as well as other graphic images that are not safe for work (NSFW) or may be inappropriate for children. And then more details as the other commenter pointed out: https://github.com/wingman-jr-addon/model#dataset https://github.com/wingman-jr-addon/model#dataset Your response is interesting because it tells me you maybe expected it to be in a different spot - was there a specific spot you were looking at? Might help me improve the descriptions.
- laputan_machine 4y ago
  Ok, then what I mean is "what do you consider to be pornographic"? In some countries it's not NSFW to see a naked breast, I am going to assume this software does not take that into account. Am I right? A bigger point one could make is that this is projecting your cultural norms onto the internet as the "default". It's a cool idea, though.

deleted 4y ago

[deleted]

jaimex2 4y ago

I'm guessing this will never run on Chrome, especially after manifest v3?

ivanmontillam 4y ago
Your guess is correct. By killing the filtering possibility, Chrome made itself a weaker browser from the NSFW-prevention standpoint as well.
- _zephyr 4y ago
  While this addon can never run on Chrome, you may find this other project to be of interest: https://github.com/nsfw-filter/nsfw-filter https://github.com/nsfw-filter/nsfw-filter I've talked with the developers there from time to time.

mlindner 4y ago

I don't get why these filters are needed. I was relatively early to the internet compared to most people (I'm 33 but I grew up on the internet since around the age of 7) yet I grew up perfectly fine without needing to be blocked from adult content. A kid seeing a sex scene in something they stumble upon is not going to scar them for life... Additionally I was never forced to use a piece of school equipment that had filtering. The most I experienced filtering was when I, on the rare case, happened to use the computers at the library instead of the ones at home and laughed at/was annoyed by the silly filtering. Much worse than adult content is addictive content that forms obsessive behaviors (things like Tiktok or more lightly social media in general). The other thing to be wary about is when it's possible for a child to give their private information away to someone in a way that can harm themselves. i.e. make sure cameras are non-functional on any child's device. That wasn't an issue when I was a kid but is very much a danger today.

pariainterpares 4y ago
Porn addicts might need filters like these.
the-printer 4y ago
I…I…I… Shame on people who would even fathom forbidding access to certain content on the devices that they likely pay for, internet that they pay for, homes that they pay for, and to children who they raise. Shame.
- a1369209993 4y ago
  Exactly. Child abuse is never okay, especially if the perpetrators think they have a legitimate rationalization for it.
- code_runner 4y ago
  The gall people have to make their own choices… and they won’t even affect me? Appalling.
culopatin 4y ago
“I don’t need it so no one needs it” is the summary of your post. Not a very constructive statement, and you’re not asking a question either, because if you actually wanted to know you would’ve looked it up, or if you’re so in tune with the needs of others, you’d already know. So what sort of conversation are you aiming to have?

someweirdperson 4y ago

Does it have an option to filter both ways? Do not show NSFW vs. show only NSFW? The latter would probably be use a lot more.

_8j50 4y ago
Doubt it, there is no pressing need to exclude non-nsfw but parental control and porn addiction help have significant demand as well as people who fear getting accidentally in trouble at work.
- IgorPartola 4y ago
  Dollars to donuts, I bet porn has a lot more demand than porn addiction help.
  - _8j50 4y ago
    For sure but it isn't drowned out by non-porn if you look for it is what I mean.
  - gjm11 4y ago
    To be a bit more explicit than badrabbit about why the symmetry isn't there: The problem a filter like this is meant to solve is: You go on the web reading random stuff, and occasionally you run into something pornographic when that isn't what you're after, and it's unwanted (because you're at work, because you find pornography disturbing, or whatever). So the problem its inverse would solve would be: You go on the web reading random stuff, and occasionally you run into something non-pornographic, and that spoils your fun. Or maybe: You go on the web looking for pornography, and occasionally you run into something non-pornographic, and that spoils your fun. So far as I am aware, those are not problems anyone has. Variations on the theme: You might be looking for some specific variety of pornography and keep finding pages with different kinds. I don't think this sort of tool would help with that; all it can do is filter out unwanted bits of pages you've already found. Or you might be successfully finding pornography but not want to see any other clutter -- e.g., advertisements on your screen distracting you from it. But I'm pretty sure the only people advertising on pornographic websites are highly NSFW themselves, so I wouldn't expect a reverse-NSFW-filter to help with that either; what you want for that is an ad blocker.

2Gkashmiri 4y ago

i tested this on reddit r/celebs and got nice results. nice as in, sure, photos were marked and it did a good job.... is this selfhosted as in no external connections?

_zephyr 4y ago
Correct, operations are all done fully local to the browser. If you opt in to the Cloudflare DNS option, it'll check using the hostname only.
- 2Gkashmiri 4y ago
  ok. that is super cool. thank you for making this nice piece of tech. hopefully it can be integrated into more stuff

pinktriangle 4y ago

I can't help but find the whole NSFW filtering thing incredibly funny since I found out there's a "healthy" fetish community working on this problem, with Chrome extensions like "Beta Safety" or "Beta Protection"

User23 4y ago
That reminds me of repackaging a traditional historical male dominated marriage, including occasional corporal punishment, as a BDSM kink to make it normal again. You think you’re seeing a woman in a dress, but actually it’s fetish gear showing her submission.
- Scharkenberg 4y ago
  General reminder: BDSM does not imply "man dom, woman sub". In fact it doesn't even imply how many people are involved and what their genders are. Edit: Why am I being downvoted for pointing out something objectively true?
  - sdf4j 4y ago
    This is hacker news
  - PenguinCoder 4y ago
    Why are you asking _why am I getting downvoted _ ? https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
    - Scharkenberg 4y ago
      Because I was at -1 after just 10 minutes and no one was engaging in discussion about why what I said would not be correct? And, please, let's not pretend to be socially inept machines - everyone here chooses to ignore a guideline or two when it's convenient - at least I was looking for actual feedback, as I was sincerely puzzled by the situation.
- 4y ago

Maxburn 4y ago

I don’t know why but this immediately make me think of the “cloud to butt” addon.

bsnnkv 4y ago

Looks like a great tool for recovering addicts everywhere. I hope to see more innovation in this space, particularly on mobile.

_zephyr 4y ago
What kind of innovation would you hope for on mobile? In theory this can work for Firefox for Android but ... the experience isn't very smooth yet.
- bsnnkv 4y ago
  Something that could read the screen at any point in time, across apps and flag/filter NSFW images, all running with a local model and no outgoing internet connections would be pretty neat.
  - _zephyr 4y ago
    Unfortunately my experiences with mobile app stores put that likely in the "not possible at all" category rather than the "tricky" category like it might be for desktops.

noasaservice 4y ago

sdf4j 4y ago
[flagged]

vitorbaptistaa 4y ago

I'd love an extension like this to remove useless news from my favorite newspapers. Anything related to murders would be a great start. I want to read the news, but lately there has been so many news along the lines of "X person was murdered in a city far away" that it's becoming unusable.