16 ms·
Firefox 59 to strip path information from referrer values for 3rd parties
- weinzierl 9y ago> To help prevent third party data leakage while browsing privately, Firefox Private Browsing Mode will remove path information from referrers sent to third parties starting in Firefox 59. Emphasis mine. It only does it in private mode. I experimented with the referrer options mentioned in the article with mixed success. Not sending the referer header breaks some sites and often in a non-obvious way. EDIT: referrer header -> referer header
- danso 9y agoMaybe they could add it for an advanced setting, but having it opt-out for normal browsing might not be desirable for the average user (if not sending headers breaks popular sites).
- franciscop 9y agoJust curious, did it break any popular website?
- Feniks 9y agoIt will break forums and imageboards in my experience. Perhaps due to CAPTCHA.
- beached_whale 9y agoLots of sites use it to prevent other sites from stealing bandwidth
- Zancarius 9y agoI think it's due to dubious "security" implementations, including at least one that was (?) present in a rather notorious PHP message board software package [1]. I'd be surprised if this survived all these years later, but then I've also seen some custom session handling code in my travels that did something similar even as recently as 2-3 years ago. Some image boards do it to prevent casual hot-linking, as my sibling poster notes. [1] https://tracker.phpbb.com/browse/PHPBB3-8396 https://tracker.phpbb.com/browse/PHPBB3-8396
- DerDangDerDang 9y agoThey actually did a study before making the change. Pretty cool. https://blog.mozilla.org/data/2018/01/26/improving-privacy-without-breaking-the-web/ https://blog.mozilla.org/data/2018/01/26/improving-privacy-w...
- djsumdog 9y agoI breaks a lot of things that do hotlink checks. Maybe only allow a referrer from the same site policy?
- robocat 9y ago> [It] breaks a lot of things that do hotlink checks. The majority of hotlink checking is to disable cross-domain hotlinking, which is not affected by this change. This change could only negatively affect hotlinking that is permitted. A site that wishes to hotlink to another may need to set a Referrer-Policy sso that the referer is passed through for images at all times.
- seba_dos1 9y agoNot exactly what you ask for, but just as a note: I had to add Polish online train ticket service to the whitelist in Smart Referer add-on, as it always fails with meaningless "invalid site call" error otherwise :P
- JepZ 9y agoWell, probably I would be fine if they would remove the referrer completely (if it has a different domain than the current site, otherwise they might notice the tampering). After all the referrer is useful for the site owner, not for the browser user.
- tzakrajs 9y agoSome websites rely (at least in part for verification) on referrer for the authentication process, I can imagine that causing some headaches for people.
- deleted 9y ago[deleted]
- jacquesm 9y agoFrankly: that's their problem. If your business model revolves around client side security then you have other problems.
- deleted 9y ago[deleted]
- Fnoord 9y agoI get what you're saying and I agree but it is a bit more nuanced. Say you want to read a certain news article, and if you get referred to it by Google you can read it, otherwise not. Now, lets say you've been finding news articles to read that way via Google the past years. Suddenly, with Firefox 59, this no longer works. Without even informing the user. That is not user-friendly.
- jacquesm 9y agoSites should be penalized for giving preference to one kind of link over another. Why should Google have even more of an upper hand than they already have? To me this is a super good illustration of why we probably should get rid of the 'Referer' (sic) header altogether.
- hk__2 9y ago> It only does it in private mode Yes, I think it should be in the HN title otherwise it feels like clickbait.
- arghwhat 9y agoHave you tried spoofing the Referer to the target URL? That's an option in Firefox too, in case a Referer is needed. I have referer completely disabled. It's nice.
- joelthelion 9y agoAre there any reasons not to get rid of referers altogether?
- kuon 9y agoI was wondering the same thing.
- TheCoreh 9y agoThey are useful to prevent the hotlinking of images/video from third party domains, which can incur in significant bandwidth costs for smaller websites.
- deleted 9y ago[deleted]
- peterwwillis 9y agoAlso paywall skipping, to track what sites are bringing you traffic, to pass fields to forms to report problems, to present targeted traffic based on use, and of course, advertising.
- ori_b 9y agoThe question was for reasons to __not__ remove them.
- deleted 9y ago[deleted]
- mi100hael 9y agoAnd those are all answers...?
- executesorder66 9y agoThey want to know reasons why a user would want to keep the referrer, not the asshole that is tracking them.
- x775 9y agoIn about:config, setting 'network.http.sendRefererHeader' to 0 (default is 2) will stop the referer header from being sent, and the document.referrer from being set. See http://kb.mozillazine.org/Network.http.sendRefererHeader http://kb.mozillazine.org/Network.http.sendRefererHeader for more information.
- open-source-ux 9y agoDisabling the referrer header completely can prevent you from signing into some sites unfortunately. Another option in Firefox is to use network.http.referer.spoofSource. In the URL bar, type: about:config Search for the following setting: network.http.referer.spoofSource Double-click to set its value to true. This basically sends the destination or target URL as the referrer.
- x775 9y agoI have not had issues thus far signing in on sites, though that might just have been dumb luck. I appreciate the heads-up and the additional information, thanks!
- deleted 9y ago[deleted]
- randomString1 9y agoThere are several referer settings. https://raw.githubusercontent.com/pyllyukko/user.js/master/user.js https://raw.githubusercontent.com/pyllyukko/user.js/master/u... user_pref("network.http.referer.userControlPolicy", x); // TODO: https://github.com/pyllyukko/user.js/issues/94 https://github.com/pyllyukko/user.js/issues/94, commented-out XOriginPolicy/XOriginTrimmingPolicy = 2 prefs user_pref("network.http.referer.spoofSource", x); user_pref("network.http.referer.XOriginPolicy", x);
- x775 9y agoThat is a very informative source, thank you!
- 9y ago
- jusob 9y agoInteresting, this is what the Referrer-Policy header is supposed to do, site by site. It make sense to enable it private browsing mode, though... and then you'll see how many sites break because they use the Referrer as some kind of authentication mechanism (yes, seen in practice multiple times).
- compsciphd 9y agoWouldn't this make it obvious that the user is browsing in private mode? While I get that might be preferable to leaking information, its also not an ideal solution either.
- ZenoArrow 9y agoTwo questions: 1. What does it matter if the creators of a website are aware that a user is using private browsing to view the site? In other words, what would they do with this information? 2. As it's possible to strip out referral information using other means, assuming there's a practical use for this "private browser user" information, what could the site creators do to guard against false positives?
- rodorgas 9y agoThere are some news websites that blocks private browsing[1]. Private browsing is a method of bypass the paywall, so they they make a private-browsing-wall. 1: https://imgur.com/a/gXEtl https://imgur.com/a/gXEtl
- darkstar999 9y agoWhat is the current mechanism for them to know that you are in private browsing mode?
- gruez 9y agoprobably checking whether the DNT header is set to 1, or whether a known tracking domain is blocked (firefox has tracking protection by default in private browsing).
- cpeterso 9y agoWebsites can use IndexedDB to test for PBM. Firefox and Edge don't support IndexedDB in PBM. Chrome supports it using a temporary database.
- ovao 9y ago
- makecheck 9y agoE-commerce checkout codes, etc. are the only reasonable form of referral. In other words, if I give you something voluntarily that tells you where I came from, fine; otherwise, why do we have so much auto-leaking built into protocols?
- tetromino_ 9y agoTo stop hotlinking from DOSing cheaply-hosted websites. Few niche hobbyist forums can afford big pipes and Cloudflare.
- eitland 9y agoWas very handy to find other people linking to your site, often people with similar sites to yours. Also I remember someone I know got an email that a page ge was linking to was about to move. I guess this was only possible because of the referer header.
- fiddlerwoaroof 9y agoIt’s, nice, for example, to see where your GitHub repositories were referred to
- masklinn 9y agoOr which sites & forums were hotlinking your images, to decide who'd get horse porn.
- spondyl 9y agoFor those confused as to why half the comments have "misspelled" referrer, here's an interesting bit of history: The misspelling of referrer originated in the original proposal by computer scientist Phillip Hallam-Baker to incorporate the field into the HTTP specification. The misspelling was set in stone by the time of its incorporation into the Request for Comments standards document RFC 1945; document co-author Roy Fielding has remarked that neither "referrer" nor the misspelling "referer" were recognized by the standard Unix spell checker of the period. https://en.wikipedia.org/wiki/HTTP_referer#Etymology https://en.wikipedia.org/wiki/HTTP_referer#Etymology
- seszett 9y ago> Phillip Hallam-Baker Funny how the misspelling of a double consonant comes from someone who has a misspelt given name with an extra double consonant!
- danso 9y agoIn the U.S., apparently both "Phillip" and "Philip" are equally popular: https://www.ssa.gov/oact/babynames/ https://www.ssa.gov/oact/babynames/ Edit: I'm glad you made this comment. I only know people with the name "Phillip" but I constantly find myself double-checking how their name is spelt before I refer to them (in text). I chalked it up to me being a bad friend but I guess it doesn't help that I might be seeing enough of the "Philip" variation to get confused.
- ryandrake 9y agoPopularity and correctness are often confused, especially online :)
- cptn_brittish 9y agoExcept spelling is a popularity contest with the one in largest use being considered the way to spell it.
- 9y ago
- sologoub 9y agoAs someone that digs in this data for a living, personally strongly believe this should be on by default for all browsing. The example they give is an insanely bad design for healthcare.gov and I would absolutely not want to find that type of data in my analytics telemetry. It is very useful to know where your traffic is coming from, but that’s usually viewed at a higher level than the querystring params being shown. In some cases, this may restrict you from knowing which article the person was reading on the given site before clicking through to yours, but if that’s so important, there are other ways to instrument source tracking.
- godelski 9y agoAs someone that doesn't dig into this data for a living, why would you not want this on non-private sessions (first question I had when reading this)? I'm curious why Mozilla wouldn't want to protect this data leaking in even a normal session.
- will4274 9y ago> why would you not want this on non-private sessions Information leaks. You click a link from your email to a news article. The URL for the news article has your email in it. Then you click an ad on the news article. The ad just got your email address. Worse still - instead of an email address, it's a token that auto signs you in to your account with the news site. Sites are pretty bad at sanitizing their outgoing referers.
- tmd83 9y agoAs I read it the parent is for it. He is actually saying it should be always the mode not just for private browsing.
- regexnerd 9y agoI don't use anything but web mail (Fastmail) since 2002. I do my banking and email in one browser and my general browsing in a very locked down Firefox (Many about:config hacks, uBlock Origin, Decentraleyes, Webmail Ad Block, Privacy Badger, and No Coin) that blocks about everything save the actual content. I also run a Pi-hole. Keeping your actions compartmentalised keeps your privacy better controlled. Nothing is perfect, but keeping things separated is generally a better idea than not doing so.
- jasonlotito 9y agoFlagged this as the headline here and the headline on the blog post do not match, and the one here is not accurate and misrepresents the post.
- njsubedi 9y agoAlso it should tell that this behavior is only expected in "Private Browsing".
- pasbesoin 9y agoOr, you could have chosen to not break the RefControl extension, that did this and a lot more.
- CorpusCalcium 9y agoSee https://addons.mozilla.org/en-GB/firefox/addon/smart-referer/ https://addons.mozilla.org/en-GB/firefox/addon/smart-referer...
- pasbesoin 9y agoThanks, but this doesn't appear to offer equivalent features. RefControl allowed a default action overlaid by siste (pattern) specific overrides. One could block refer(r)er altogether, and then adjust on a site/resource basis as needed. There is one current web extension compatible extension that purports to do this, but when I tried it, it didn't want to cooperate with my configuration, despite adjustments. Further, it sucked Discus comments into its local configuration dialog/page, something that I find... sucks. Finally, it wasn't open-source and didn't have a well-known provinance. All this didn't leave me feeling too confident in it. P.S. uMatrix is supposed to provide layers of referrer control, but I haven't made the effort yet to switch over to it including switching some of my other points of configuration to use it instead.
- staunch 9y agoIf Mozilla genuinely prioritized its users' interest it would block ads and tracking networks, which are the major way people's private information is leaked and also a primary vector for hacking. And yet for some mysterious reason Firefox hasn't broken ranks with Google by incorporating ad blocking. Even though its an obvious major feature and Firefox is losing marketshare every year. We know why Google won't prioritize the interests of Chrome users but why is the only major independent browser seemingly corrupt in the same way? Mozilla should be helping society by pushing it past an era of internet advertising and the clearly terrible clickbait-fake-news culture it creates. And yet, it does not. Is Google using the money it pays Mozilla to "discourage" Firefox from going forward with ad blocking? As a concerned citizen, I sent an email to antitrust.complaints@usdoj.gov requesting an investigation. Anyone with insider info should send it there.
- StavrosK 9y agoYou mean like the "tracking blocker" that is built-in to Firefox?
- staunch 9y agoNo. I obviously mean real ad blocking that users are currently required to install as a third-party Firefox extension. You don't find it curious that Mozilla has made the decision not to implement the most user-demanded browser feature for over a decade? https://addons.mozilla.org/en-US/firefox/ https://addons.mozilla.org/en-US/firefox/
- Sylos 9y agoWebpage owners need a way to make money. A whole class of webpages, those that are financed by ad revenue, would stop being tested/built against Firefox. They might even block Firefox users from accessing their webpage, so that people switch to other browsers which do make them money. They have no real reason not to block them. There's a whole load of politics like this in being a browser vendor. Just because users demand a change, does not mean that it's the best for users. Users want broken webpages even less than they want ads, they are just too short-sighted to realize that blocking ads will lead to that. And Mozilla really is already gambling hard with such things as Tracking Protection or this referrer blocking when Chrome has no such restrictions in place and has the majority market share. For many webpage owners, blocking Firefox users would probably not even drop their revenue by 10%. Leaving it unmaintained much less so, in the short term. So, unless Mozilla manages to create perfect compatibility with Chrome, they cannot afford to block revenue from webpages. And creating perfect compatibility with Chrome would also mean blindly following all of Chrome's webstandard choices, ingraining tracking- and ad-supporting technologies into the web, which again would be shit for users in the long run. And really, just use a Chromium-fork, if this is what you're looking for. You don't need Firefox for that.
- codedokode 9y agoSites like healthcare.gov or banks should not include third party ads or analytics scripts. Referrer is not the only way to leak information from the page.
- newman314 9y agoFor those who are curious, here's some more detail on the various options. https://feeding.cloud.geek.nz/posts/tweaking-referrer-for-privacy-in-firefox/ https://feeding.cloud.geek.nz/posts/tweaking-referrer-for-pr... Basically, set the following: network.http.referer.(XOriginPolicy|XOriginTrimmingPolicy|trimmingPolicy) to 2 network.http.referer.spoofSource to true network.http.sendRefererHeader to 0 network.sendSecureXSiteReferrer to false
- mehrdadn 9y agoUhm, wouldn't this give an indication as to whether the user is in private mode?
- saagarjha 9y ago> https://www https://www. healthcare.gov/see-plans/85601/results/?county=04019&age=40&smoker=1&pregnant=1&zip=85601&state=AZ&income=35000 For a moment I thought this was an example to make a point…
- jacquesm 9y agoThis one of the reasons I tend to flip completely whenever I see healthcare providers and their suppliers run google analytics tags inside their logged in areas (yes, this really happens). Besides the questionable value of having such tracking inside the logged in areas (it's healthcare, they are not going to worry about their conversion rates) such information should simply never leave the premises. Better still if they didn't do this in private mode but always. Private mode is still associated with doing something sneaky, rather than that it should be the default. Happy to see FF do the right thing here and I'm really curious if Google will follow suit. Microsoft and Apple have an opportunity here to show they care about end user privacy than Google.
- kstrauser 9y agoFirst, you're totally right and I completely agree. That said, check if those sites also set CSP headers (see https://content-security-policy.com https://content-security-policy.com). All modern browsers support them, and it lets you set granular policies like "send full referers to my own server, but only over HTTPS", "HTTP connections (which I shouldn't be seeing anyway, but to be thorough) to myself only get the host and path, not the query string", and "connections to any other domain should include only my hostname in the referer". It's quite nice actually.
- gregknicholson 9y ago> it's healthcare, they are not going to worry about their conversion rates Monitoring conversion rates can be used to find out whether people are actually able to use your web service. The goal of a “conversion” doesn't have to be a sale. But I agree that if you're going to do this sort of tracking, it definitely needs to be private. I hope there's a court case soon where the court rules that sending a whole load of business-sensitive data to Google, Microsoft and Apple actually does breach a non-disclosure agreement.
- jacquesm 9y agoThere will definitely be such a case in the EU, that GDPR has some pretty impressive fangs and either a healthcare provider, an ISP or an insurance company is going to make an excellent example. Coming to a courtroom somewhere in Europe in 2019.
- ospider 9y agoThis feature should be easily implemented by a extension for Google Chrome using the webRequest API
- Sylos 9y agoWell, yeah, the feature itself is not a particular accomplishment, really it's been in Firefox for years, you just had to enable it with some about:config-flag. The big news is them enabling it by default in some fashion (that is when you're in Private Browsing), meaning that all users now have this, not just the 0.1% who understand referrers (and have not forgotten to enable this the last time they installed Firefox). As a power user who knew about this, you might not particularly care, but for users in general it's great, while it pisses off webpage owners. Then again, even as a power user it's impossible for you to know about all of these sort of config options, so you might care to use a browser which tries to help its users out while having to keep an eye on not pissing off webpage owners too much, rather than a browser that tries to maximize revenue for webpage owners while trying its best to hide all the ways it infringes privacy from its users.
- kemitche 9y agoColor me stupid, but I thought all major browsers already stripped referrer info when navigating from HTTPS? The examples used don't make sense to me if that's true.
- groovecoder 9y agoThe default user agent policy is no-referrer-when-downgrade, which strips the referrer header going from HTTPS pages to HTTP resources. Firefox 59 PBM now implements strict-origin-when-cross-origin by default, which trims the path off the referrer value of ALL 3rd-party requests.
- 566d8d6 9y agoright-click, copy url, paste in address bar
- Justsignedup 9y agoI mean we already have firefox plugins to permanently block the referrer. Which is great. But I applaud Mozilla for going privacy-first in a consumer package. I hope that eventually Mozilla will focus entirely on privacy and make good anti-tracking, anti-ads, anti-referrer, anti-cryptomining all default packages.
- dewiz 9y agoFor the scope of requesting a document, there is no need of referrer nor useragent. A lot of features/apps/websites have been built around the assumption that this information is sent, but it would be nice to start dropping it by default.
- executesorder66 9y agoYou can use the smart-referer addon [0][1] to send a custom referrer string[1] [0]https://addons.mozilla.org/en-US/firefox/addon/smart-referer/ https://addons.mozilla.org/en-US/firefox/addon/smart-referer... [1]https://github.com/meh/smart-referer https://github.com/meh/smart-referer [2]https://imgur.com/a/1HZQK https://imgur.com/a/1HZQK
- YTGRK 9y agohttps://youtu.be/I2l8xkjTUh4 https://youtu.be/I2l8xkjTUh4
- Tepix 9y agoWhoah, TIL that > EFF researchers discovered this leak of personal health data from healthcare.gov to DoubleClick It blows my mind that a site such as healthcare.gov would include 3rd party trackers. You guys in the US really don't care about privacy at all.
- azernik 9y agoThey were specifically using a monitoring tool (Floodlight, for tracking conversions) that happens to be provided by an ad company. To my view, it's more of a security issue than a privacy issue, as DoubleClick was prohibited by TOS from using the data for advertising.
- paulie_a 9y agoIn this era, if it comes from an ad company it almost certainly is a privacy issue and should generally be considered a security problem
- rrcaptain 9y agoThis is largely the part of independent contractors and subcontractors. Contracts go to the companies good at winning contracts, not necessarily the best company to do the job. The contractor takes an obscene profit for providing no value and then subcontracts the project to various subcontractors who may or may not employ actually qualified and skilled engineers. The government employees managing the contract typically do not have the expertise to evaluate the project or write proper specs. The HealthCare.gov contract was a mess of incompatible buzzwords. The engineers have no vested interest in the project as they're only there to complete that contract and they're so many levels removed from the government agency that no one actually knows who they are so it won't reflect poorly on them when everything comes out poorly. Because their career doesn't really depend on the success of the project, as the government gets blamed for contractor failures while contractors get the credit for success, they don't really need to do more than meet the specs. A better way to do this would be to expand the number of engineers within the government through groups like 18F and USDS, and give preference to them over private industry. Private contractors rarely work, but even when they do it's only when you have expenses that the government doesn't need (such as contracting a machine shop or car manufacturer to build something with their preexisting infrastructure). In software though, your only expenses are really your engineers and the cloud (as no one needs to run their own data center). The only thing subcontractors can do that the government can't is pay their employees more than the GS scale. However because the contract is supposed to be cheaper than the government just hiring employees themselves (as industry has "profit motive") they're going to have to cheap out elsewhere, either by hiring fewer developers or neglecting parts of the development. All of this is solvable by Congress, simply boost pay flexibility, but there's no political motive to fix it as all of the contractors are political donors. As a result, government software sucks.
- rowyourboat 9y agoTIL that is an option at all. It's there in earlier versions, too, all they did was change the default behavior for Private Browsing mode
- therealmarv 9y agoAny other browser doing this? Safari?
- ecthiender 9y agoWhy is this being only implemented in private browsing mode and not in the normal mode? IMO, this should be the default.
- vanderZwan 9y agoMy guess is: they are letting the privacy nuts (and I am one) figure out the kinks in the system first before unleashing it on the general public
- Sylos 9y ago1) It breaks some webpages, so they probably only want to have it where it's relatively clear that users will prefer their privacy over not having breakages. 2) It pisses off webpage owners, as it hampers their analytics and probably ad revenue. If webpage owners are too pissed off by Firefox or simply don't make enough revenue from it, they'll stop testing/building their webpage against it, which leads to broken webpages in this way as well.
- alkonaut 9y ago#1 this should be on by default. I might be missing something, but do sites really need the referrer? What would break if the browser sent the same page as referrer, or google.com/ or something similar? Is there any value in the referrer to the client? The host can use it for a whole range of reasons - but apart from helping the host, what is the immediate benefit to the client? #2 Won't this be possible to bypass simply by encoding more in the domain part of the url than in parameters? So you switch from a.b.tld/foo?p=123 to 123.a.b.tld/foo ?
- rocqua 9y agoTo answer #1, the referrer could probably be used to make a link refer to a login page on another domain, and to then let the host redirect to the correct page after login. There are of course ways to do this by following your suggestion #2, but I suspect this might be why we originally had the referrer header.
- alkonaut 9y agoThird party logins are definitely a use case. But you'd think that they would aim to do it with logins.somewhereelse.com/?u=source instead of relying on the referrer field as the source to go back to! I wonder how common actual use of referrer is?
- rovek 9y agoYou're right about subdomains but they are stripping both query and path, so 123.a.b.tld/foo would become 123.a.b.tld. It's fair to assume most misplaced sensitive data will be in the path or query, rather than the subdomain.
- alkonaut 9y agoI didn't even think about the subdomain until I saw the image that is in the article here - which literally looks like it added an arbitrary number in the subdomain, duplicating the parameter number! When I saw the image I thought the article was actually about blocking some shady way of leaking that doubleclick had invented.
- marco1 9y agoSetting `network.http.referer.XOriginPolicy` to `1` in Firefox’s `about:config` is actually a pretty reasonable choice for all browsing, and balances privacy with preventing your favorite sites from breaking.
- jokoon 9y agoThe weird thing about browsers that tries to protect your privacy is how many website will get broken because of this. I managed to disable cookies by default using cookie whitelist, and I counted many website that broke down. I applaud firefox for daring to break website for the sake of privacy, but I'm waiting for websites to react. Firefox should be even more strict regarding privacy: ask the user if he want to set a cookie, never save history etc. I'm using the extension that compartmentalize website usage on firefox, and this should be made default.
- Too 9y agoEarly versions of IE asked every time if the site was allowed to set cookies. Guess how fast people got used to clicking "yes and don't ask again".
- DarronWyke 9y agoI disable referers for all browsers. Firefox has them off completely, with Referer Control using a random one for Chrome. I only enable them for sites that absolutely need them (and that I need to use). It's simply good data hygiene and privacy.
- rkagerer 9y agoThis might be a dumb question but does whether a site uses SSL have any impact on browser behavior concerning query strings? Seems counterproductive my browser is taking so much care to encrypt my querystrings then leaking them to any host from which the site I'm visiting happens to pull content.