5 ms·

I wish I had EFF's clout behind my work too[1]. In the last few months, I have been completely dedicated to write an extension which completely informs the user

by gorhill 12y ago

I wish I had EFF's clout behind my work too[1]. In the last few months, I have been completely dedicated to write an extension which completely informs the user about what a web page does, and gives the user full control over what web sites do in his/her browser. Spent hours after hours at not only making it work, but also making it work efficiently (wrote custom ABP engine from scratch which doesn't suffer the real one's abuse of memory/CPU), and yet barely anyone is noticing it. EFF is also one of the recipient I suggest for people who really want to donate something for my work. In any case, an important warning concerning any extension which modifies HTTP headers on Chromium-based browsers: only one extension is allowed to modify the HTTP headers[2], and since EFF's badger does modify outbound request headers, it will break any previously installed extension which relies on also modifying these headers to work properly. This means mine[1] is incompatible, one of the two extensions won't be able to do what it says it does. This applies for any other extension modifying outbound HTTP headers. [1] https://github.com/gorhill/httpswitchboard#http-switchboard-for-chromium https://github.com/gorhill/httpswitchboard#http-switchboard-... [2] https://developer.chrome.com/extensions/webRequest#implementation https://developer.chrome.com/extensions/webRequest#implement...

losvedir 12y ago
Sorry, that sounds frustrating. I just installed your extension and it looks pretty slick so far! I had been using RequestPolicy, CookieMonster, and NoScript on Firefox to do something similar, but wasn't aware of something like RequestPolicy for Chrome. Your extension fits the bill nicely. Couple questions: * Your documentation says "Efficient blacklisting ... javascript won't execute". Is this because you prevent the request altogether or do you somehow allow the Javascript to be fetched but then sandbox it or something? The latter seems brittle and prone to issues (tptacek's warnings about security/javascript in the browser come to mind). * The control panel seems to let me block scripts from the domain I'm actually on (so, not a 3rd party request). Does this include inline script tags in the HTML? Obtrusive javascript via "onclick=..."? Last I checked, I thought Chrome didn't give you hooks to be able to stop, with an extension, the evaluation of scripts. In fact, that's why I've been using Firefox w/ NoScript, since I didn't think Chrome could match it. Is that not the case? * When I search Google (but no other page), it seems to flash and reload since installing the script. What's that? Thanks for your hard work on this! I appreciate it a lot.
- gorhill 12y ago
  Thanks for the nice words, it does help the morale. Regarding javascript execution there are two cases: External javascript: won't execute because the net request for the external resource is blocked. Inline javascript: won't execute because a Content-Security-Policy directive is injected into the main page or sub frame. This week I had to write an explanation of how this work because the idea that inline javascript can't be reliably disabled on a Chromium-based browser is still going strong. Here are the details: https://github.com/gorhill/httpswitchboard/wiki/Blocking-javascript-execution-reliably-in-Chromium-based-browsers https://github.com/gorhill/httpswitchboard/wiki/Blocking-jav... Regarding the "flash and reload", I noticed that too lately, I have to look into this to understand what is happening.
  - losvedir 12y ago
    Very cool! Thanks for the explanation. One last question (sorry!): I just used the "Google" preset configuration to load gmail and it all just worked. I'm still trying to get the hang of interpreting the grid, but at the top left I see in blue ".google.com". Does that mean these whitelistings only apply to pages in a Google subdomain? For example: google analytics requests from some other* domain aren't whitelisted?
    - gorhill 12y ago
      > Does that mean these whitelistings only apply to pages in a Google subdomain Yes, the top-most left-most box is for choosing where the rules apply, I call it the scope selector. "" = global scope ".example.com" = any web page which sits on 'example.com' or a subdomain of 'example.com' "example.com" = any web page which sits exactly on 'example.com' If multiple scopes exist for a given web page, the most narrow one is used. Also, scopes are really sandboxes, there is no inheritance of rules across scopes (trying to implement this is opening a can of worms both on usability and code-wise).
      - losvedir 12y ago
        Very cool! Been using it all morning and I'm very impressed. Thanks again for your hard work.
JackC 12y ago
> I wish I had EFF's clout behind my work too. Man, I'm sure that must be really frustrating. It looks like your tool takes a different strategy than EFF's and there will be room for both -- I hope you keep it up. As a general perspective (not just directed at you, gorhill), it's worth bearing in mind that the EFF has earned their clout by working hard and getting results since 1990 -- almost a quarter of a century. You can think of them like a startup founder who has a lot of money and connections to launch new companies, because she's been building winning companies for decades and built up those resources. For her, backing a new project means betting some of that capital. For you, starting on your own without that kind of track record means you'll have to hustle longer and harder to find success. This analogy is particularly helpful if you (again, not just gorhill) actually do want the EFF or a similar organization to back your project. That startup founder is going to be constantly getting pitches from people who want her time or attention or just money. The ones she'll want to help will be the ones she knows and trusts because they've helped her out before. That makes projects like this Badger thing a great opportunity to get involved with an organization -- they're asking for your help. It's a chance to prove you have the coding skills/social skills/enthusiasm/values/reliability/whatever that make you a good fit. Then when you're starting your own project, they'll be relatively likely to put some of their reputational capital behind it.
- chimeracoder 12y ago
  Agreed - at first, this seems frustrating, but don't let it be discouraging. gorhill, there are very few people who are both interested in and qualified for building software of this sort. You've essentially just proven to the EFF that you're the exact person whose help they should want with this, so you have an opportunity to join forces with them and build not just an amazing product, but an amazing product that does exactly what you originally set out to do! This is the same principle behind so-called talent acquisitions ("acquihires"); the difference here is that we're talking about a non-profit organization and two open-source projects, so there's even less friction in joining forces. And more importantly, you can always change your mind if you ever decide to. Nothing holding you back!
  - handsomeransoms 12y ago
MrUnknown 12y ago
Installed it, looks very powerful, and that's probably part of the issue. A lot of options to change and no real idea what they are. Also, it caused some of my pages to stop working which I know aren't doing anything odd (internal helpdesk software).
- gorhill 12y ago
  I usually refer new users to this one page which shows that the extension can also be used in "allow-all/block-exceptionally" mode, which mode is less likely to break web pages (ABP, Ghostery, Disconnect etc. all work in such mode): https://github.com/gorhill/httpswitchboard/wiki/How-to-use-HTTP-Switchboard:-Two-opposing-views https://github.com/gorhill/httpswitchboard/wiki/How-to-use-H... At one end of the spectrum, the extension can be used without no filtering of any kind, and the user still has a comprehensive reporting tool to see what a web page does.
im3w1l 12y ago
Before addeventlistener, I remember people manually chaining eventhandlers. Any chance you could do a variation on that? >function addLoadEvent(func) { > var oldonload = window.onload; > if(typeof window.onload != "function") > window.onload = func; > else > window.onload = function() { > oldonload(); > func(); > } >} http://www.csgnetwork.com/directonloadchain.html http://www.csgnetwork.com/directonloadchain.html
higherpurpose 12y ago
I realize this may be an extension for more technical users (not sure that's the right approach for something like this to begin with), but HTTPS Switchboard doesn't exactly roll off the tongue, no matter how accurately descriptive its name is. Privacy Badger or Disconnect seems much catchier. Something to keep in mind for yours, if you want high adoption.
handsomeransoms 12y ago
> I have been completely dedicated to write an extension which completely informs the user about what a web page does, and gives the user full control over what web sites do in his/her browser. That's awesome! At Mozilla we've been daydreaming about incorporating something like that into the product. Especially with the proliferation of both in-browser content policies (CSP, Mixed Content blocking, etc.) and out-of-browser content policies (addons like Privacy Badger, ABP, Ghostery, etc.) it's almost impossible to understand the root cause of what's breaking a page, especially if you're not a technical user. > but also making it work efficiently (wrote custom ABP engine from scratch which doesn't suffer the real one's abuse of memory/CPU) For the Firefox version of Privacy Badger, we also eschewed the ABP engine in favor of something we wrote ourselves. Don't get me wrong, ABP is a great tool and we learned a lot from its code - but we wanted something as lean and performant as possible. I'd be interested to see the approach you took! > only one extension is allowed to modify the HTTP headers > it will break any previously installed extension That sounds like a shortcoming in Chrome's Extension API. Firefox is somewhat better in that we do not restrict addons based on the behavior of other addons, so multiple addons are allowed to modify a request's headers. Unfortunately, the order in which the request is passed from handler to handler is not guaranteed, so in practice this may not end up being so useful. We're trying to improve this situation by rewriting the Gecko Content Policy API [0], but that's a large project with no clear deadline. [0] https://groups.google.com/forum/#!msg/mozilla.dev.platform/veLFoy09ydg/2XcWUXSiVbEJ https://groups.google.com/forum/#!msg/mozilla.dev.platform/v... Generally I think your addon and Privacy Badger are targeted at different audiences and have different use cases. It is too bad they are incompatible with each other - I, for one, would like to have both installed. Perhaps we can resolve this by talking to the Chrome devs about their Extension API? - One of the Privacy Badger devs
- gorhill 12y ago
  I thought a bit more about the incompatibility, and it's not an all or nothing situation. In the case of HTTPSB (can't speak for other extensions), installing Privacy Badger after will likely break some specific privacy-related features of HTTPSB, but a user might find this acceptable, since Privacy Badger will take over for some of the broken features. I explain in more details where there are conflicts: https://github.com/gorhill/httpswitchboard/wiki/Compatibility-with-various-Chromium-based-browsers-and-other-extensions#privacy-badger https://github.com/gorhill/httpswitchboard/wiki/Compatibilit...
sequoia 12y ago
Does this initially block most/all domains and require individual whitelisting? I'm using it now, on washingtonpost.com it blocked requests to washingtonpost.com... if it requires manual whitelisting that's waay more than the average user will do and probably more than I will do (I'd been using disconnect or ghostery, will switch back to one of those or pBadger).
- gorhill 12y ago
  Yes, it has been said that it's more for advanced users. I've been trying to mitigate this with presets, recipes, etc. but still I understand and accept that it's not for everybody. I usually ask people to at least read this one page [1], not to convince them, but rather just so that it is understood that the extension is fully configurable: it can be at any point in the full block-all / allow-all spectrum. [1] https://github.com/gorhill/httpswitchboard/wiki/How-to-use-HTTP-Switchboard:-Two-opposing-views https://github.com/gorhill/httpswitchboard/wiki/How-to-use-H... I just added a 3rd example of how the extension can be used without blocking anything, while still reporting everything. If anything, this can be useful to keep the other privacy extensions open for scrutiny.
  - unhush 12y ago
    We do have a preloaded list of third-party sites to not block because they would cause too much breakage. Note that because PBadger is in alpha, this list is very short right now: https://www.eff.org/files/cookieblocklist.txt https://www.eff.org/files/cookieblocklist.txt It's maintained by EFF, and more things will get added as time goes on.
    - sequoia 12y ago
      Add mail.google.com!! I took the time to go to google accounts, re-login, disable the blocking on the various sites (accounts.google, mail.google etc.) to get it working, but I can imagine breaking gmail will be a quick route to "uninstall extension" for many people. :)
      - 12y ago
unhush 12y ago
Hi, I'm an EFF staff technologist and wrote most of Privacy Badger Firefox (though I didn't come up with the idea). It's often hard for us to find projects like yours, so we appreciate developers contacting us and telling us about them directly. It's possible we can find a way to work together, if you'd like. Email me: yan at eff dot org. Disclosure: I didn't work on Privacy Badger Chrome but at least I can fwd you on to the right people. :) Sorry, that does sound frustrating. I joined EFF partly because it was a good way to make software that people would actually use. (One constraint that most people may not realize is that EFF has very strict privacy policies for our tech projects. There's lots of projects I'd like to do, but the lawyers probably wouldn't approve. So there's plenty of room for other developers to do projects that step over lines we wouldn't cross.)
eliteraspberrie 12y ago
The matrix visual is genius! I'll be using your extension going forward. Thanks for the great work. You may want to consider setting up a Gittip account or something similar so people have the option of supporting your work.
shostack 12y ago
More efforts in this space are a Good Thing(tm) regardless, so thank you for your contribution. That said, you talk about all the effort that went into building it only to have nobody notice it...I have to ask, how much time did you put into promoting it and how did you approach that? I don't intend for this to come off rude, but if you didn't do much to promote it, is it possible there's a bit of "Field of Dreams" syndrome at play here (the "if I build it they will come" fallacy)? Regardless, I think there is a lack of trustworthy and well-known resources for getting the word out about new advancements in the privacy space like yours. I'm in the digital media world, and while I'm acutely aware of this stuff, I'm unaware of any sites that serve that purpose. I'd love the EFF to put their weight behind a list of trustworthy add-ons that they inspect and maintain (in addition to theirs) since this needs to be a larger effort than they alone have the resources for. The big problem is getting the public more aware, and they are one of the best loudspeakers for that at the moment.

by gorhill 12y ago

losvedir 12y ago

Sorry, that sounds frustrating. I just installed your extension and it looks pretty slick so far! I had been using RequestPolicy, CookieMonster, and NoScript on Firefox to do something similar, but wasn't aware of something like RequestPolicy for Chrome. Your extension fits the bill nicely. Couple questions: * Your documentation says "Efficient blacklisting ... javascript won't execute". Is this because you prevent the request altogether or do you somehow allow the Javascript to be fetched but then sandbox it or something? The latter seems brittle and prone to issues (tptacek's warnings about security/javascript in the browser come to mind). * The control panel seems to let me block scripts from the domain I'm actually on (so, not a 3rd party request). Does this include inline script tags in the HTML? Obtrusive javascript via "onclick=..."? Last I checked, I thought Chrome didn't give you hooks to be able to stop, with an extension, the evaluation of scripts. In fact, that's why I've been using Firefox w/ NoScript, since I didn't think Chrome could match it. Is that not the case? * When I search Google (but no other page), it seems to flash and reload since installing the script. What's that? Thanks for your hard work on this! I appreciate it a lot.

gorhill 12y ago
Thanks for the nice words, it does help the morale. Regarding javascript execution there are two cases: External javascript: won't execute because the net request for the external resource is blocked. Inline javascript: won't execute because a Content-Security-Policy directive is injected into the main page or sub frame. This week I had to write an explanation of how this work because the idea that inline javascript can't be reliably disabled on a Chromium-based browser is still going strong. Here are the details: https://github.com/gorhill/httpswitchboard/wiki/Blocking-javascript-execution-reliably-in-Chromium-based-browsers https://github.com/gorhill/httpswitchboard/wiki/Blocking-jav... Regarding the "flash and reload", I noticed that too lately, I have to look into this to understand what is happening.
- losvedir 12y ago
  Very cool! Thanks for the explanation. One last question (sorry!): I just used the "Google" preset configuration to load gmail and it all just worked. I'm still trying to get the hang of interpreting the grid, but at the top left I see in blue ".google.com". Does that mean these whitelistings only apply to pages in a Google subdomain? For example: google analytics requests from some other* domain aren't whitelisted?
  - gorhill 12y ago
    > Does that mean these whitelistings only apply to pages in a Google subdomain Yes, the top-most left-most box is for choosing where the rules apply, I call it the scope selector. "" = global scope ".example.com" = any web page which sits on 'example.com' or a subdomain of 'example.com' "example.com" = any web page which sits exactly on 'example.com' If multiple scopes exist for a given web page, the most narrow one is used. Also, scopes are really sandboxes, there is no inheritance of rules across scopes (trying to implement this is opening a can of worms both on usability and code-wise).
    - losvedir 12y ago
      Very cool! Been using it all morning and I'm very impressed. Thanks again for your hard work.

JackC 12y ago

> I wish I had EFF's clout behind my work too. Man, I'm sure that must be really frustrating. It looks like your tool takes a different strategy than EFF's and there will be room for both -- I hope you keep it up. As a general perspective (not just directed at you, gorhill), it's worth bearing in mind that the EFF has earned their clout by working hard and getting results since 1990 -- almost a quarter of a century. You can think of them like a startup founder who has a lot of money and connections to launch new companies, because she's been building winning companies for decades and built up those resources. For her, backing a new project means betting some of that capital. For you, starting on your own without that kind of track record means you'll have to hustle longer and harder to find success. This analogy is particularly helpful if you (again, not just gorhill) actually do want the EFF or a similar organization to back your project. That startup founder is going to be constantly getting pitches from people who want her time or attention or just money. The ones she'll want to help will be the ones she knows and trusts because they've helped her out before. That makes projects like this Badger thing a great opportunity to get involved with an organization -- they're asking for your help. It's a chance to prove you have the coding skills/social skills/enthusiasm/values/reliability/whatever that make you a good fit. Then when you're starting your own project, they'll be relatively likely to put some of their reputational capital behind it.

chimeracoder 12y ago
Agreed - at first, this seems frustrating, but don't let it be discouraging. gorhill, there are very few people who are both interested in and qualified for building software of this sort. You've essentially just proven to the EFF that you're the exact person whose help they should want with this, so you have an opportunity to join forces with them and build not just an amazing product, but an amazing product that does exactly what you originally set out to do! This is the same principle behind so-called talent acquisitions ("acquihires"); the difference here is that we're talking about a non-profit organization and two open-source projects, so there's even less friction in joining forces. And more importantly, you can always change your mind if you ever decide to. Nothing holding you back!
- handsomeransoms 12y ago

MrUnknown 12y ago

Installed it, looks very powerful, and that's probably part of the issue. A lot of options to change and no real idea what they are. Also, it caused some of my pages to stop working which I know aren't doing anything odd (internal helpdesk software).

gorhill 12y ago
I usually refer new users to this one page which shows that the extension can also be used in "allow-all/block-exceptionally" mode, which mode is less likely to break web pages (ABP, Ghostery, Disconnect etc. all work in such mode): https://github.com/gorhill/httpswitchboard/wiki/How-to-use-HTTP-Switchboard:-Two-opposing-views https://github.com/gorhill/httpswitchboard/wiki/How-to-use-H... At one end of the spectrum, the extension can be used without no filtering of any kind, and the user still has a comprehensive reporting tool to see what a web page does.

im3w1l 12y ago

Before addeventlistener, I remember people manually chaining eventhandlers. Any chance you could do a variation on that? >function addLoadEvent(func) { > var oldonload = window.onload; > if(typeof window.onload != "function") > window.onload = func; > else > window.onload = function() { > oldonload(); > func(); > } >} http://www.csgnetwork.com/directonloadchain.html http://www.csgnetwork.com/directonloadchain.html

higherpurpose 12y ago

I realize this may be an extension for more technical users (not sure that's the right approach for something like this to begin with), but HTTPS Switchboard doesn't exactly roll off the tongue, no matter how accurately descriptive its name is. Privacy Badger or Disconnect seems much catchier. Something to keep in mind for yours, if you want high adoption.

handsomeransoms 12y ago

> I have been completely dedicated to write an extension which completely informs the user about what a web page does, and gives the user full control over what web sites do in his/her browser. That's awesome! At Mozilla we've been daydreaming about incorporating something like that into the product. Especially with the proliferation of both in-browser content policies (CSP, Mixed Content blocking, etc.) and out-of-browser content policies (addons like Privacy Badger, ABP, Ghostery, etc.) it's almost impossible to understand the root cause of what's breaking a page, especially if you're not a technical user. > but also making it work efficiently (wrote custom ABP engine from scratch which doesn't suffer the real one's abuse of memory/CPU) For the Firefox version of Privacy Badger, we also eschewed the ABP engine in favor of something we wrote ourselves. Don't get me wrong, ABP is a great tool and we learned a lot from its code - but we wanted something as lean and performant as possible. I'd be interested to see the approach you took! > only one extension is allowed to modify the HTTP headers > it will break any previously installed extension That sounds like a shortcoming in Chrome's Extension API. Firefox is somewhat better in that we do not restrict addons based on the behavior of other addons, so multiple addons are allowed to modify a request's headers. Unfortunately, the order in which the request is passed from handler to handler is not guaranteed, so in practice this may not end up being so useful. We're trying to improve this situation by rewriting the Gecko Content Policy API [0], but that's a large project with no clear deadline. [0] https://groups.google.com/forum/#!msg/mozilla.dev.platform/veLFoy09ydg/2XcWUXSiVbEJ https://groups.google.com/forum/#!msg/mozilla.dev.platform/v... Generally I think your addon and Privacy Badger are targeted at different audiences and have different use cases. It is too bad they are incompatible with each other - I, for one, would like to have both installed. Perhaps we can resolve this by talking to the Chrome devs about their Extension API? - One of the Privacy Badger devs

gorhill 12y ago
I thought a bit more about the incompatibility, and it's not an all or nothing situation. In the case of HTTPSB (can't speak for other extensions), installing Privacy Badger after will likely break some specific privacy-related features of HTTPSB, but a user might find this acceptable, since Privacy Badger will take over for some of the broken features. I explain in more details where there are conflicts: https://github.com/gorhill/httpswitchboard/wiki/Compatibility-with-various-Chromium-based-browsers-and-other-extensions#privacy-badger https://github.com/gorhill/httpswitchboard/wiki/Compatibilit...

sequoia 12y ago

Does this initially block most/all domains and require individual whitelisting? I'm using it now, on washingtonpost.com it blocked requests to washingtonpost.com... if it requires manual whitelisting that's waay more than the average user will do and probably more than I will do (I'd been using disconnect or ghostery, will switch back to one of those or pBadger).

gorhill 12y ago
Yes, it has been said that it's more for advanced users. I've been trying to mitigate this with presets, recipes, etc. but still I understand and accept that it's not for everybody. I usually ask people to at least read this one page [1], not to convince them, but rather just so that it is understood that the extension is fully configurable: it can be at any point in the full block-all / allow-all spectrum. [1] https://github.com/gorhill/httpswitchboard/wiki/How-to-use-HTTP-Switchboard:-Two-opposing-views https://github.com/gorhill/httpswitchboard/wiki/How-to-use-H... I just added a 3rd example of how the extension can be used without blocking anything, while still reporting everything. If anything, this can be useful to keep the other privacy extensions open for scrutiny.
- unhush 12y ago
  We do have a preloaded list of third-party sites to not block because they would cause too much breakage. Note that because PBadger is in alpha, this list is very short right now: https://www.eff.org/files/cookieblocklist.txt https://www.eff.org/files/cookieblocklist.txt It's maintained by EFF, and more things will get added as time goes on.
  - sequoia 12y ago
    Add mail.google.com!! I took the time to go to google accounts, re-login, disable the blocking on the various sites (accounts.google, mail.google etc.) to get it working, but I can imagine breaking gmail will be a quick route to "uninstall extension" for many people. :)
    - 12y ago

unhush 12y ago

Hi, I'm an EFF staff technologist and wrote most of Privacy Badger Firefox (though I didn't come up with the idea). It's often hard for us to find projects like yours, so we appreciate developers contacting us and telling us about them directly. It's possible we can find a way to work together, if you'd like. Email me: yan at eff dot org. Disclosure: I didn't work on Privacy Badger Chrome but at least I can fwd you on to the right people. :) Sorry, that does sound frustrating. I joined EFF partly because it was a good way to make software that people would actually use. (One constraint that most people may not realize is that EFF has very strict privacy policies for our tech projects. There's lots of projects I'd like to do, but the lawyers probably wouldn't approve. So there's plenty of room for other developers to do projects that step over lines we wouldn't cross.)

eliteraspberrie 12y ago

The matrix visual is genius! I'll be using your extension going forward. Thanks for the great work. You may want to consider setting up a Gittip account or something similar so people have the option of supporting your work.

shostack 12y ago

More efforts in this space are a Good Thing(tm) regardless, so thank you for your contribution. That said, you talk about all the effort that went into building it only to have nobody notice it...I have to ask, how much time did you put into promoting it and how did you approach that? I don't intend for this to come off rude, but if you didn't do much to promote it, is it possible there's a bit of "Field of Dreams" syndrome at play here (the "if I build it they will come" fallacy)? Regardless, I think there is a lack of trustworthy and well-known resources for getting the word out about new advancements in the privacy space like yours. I'm in the digital media world, and while I'm acutely aware of this stuff, I'm unaware of any sites that serve that purpose. I'd love the EFF to put their weight behind a list of trustworthy add-ons that they inspect and maintain (in addition to theirs) since this needs to be a larger effort than they alone have the resources for. The big problem is getting the public more aware, and they are one of the best loudspeakers for that at the moment.