4 ms·

It's time to abolish the HTTP referer

12 points by marco1 12y ago

In times where we see that our privacy on the internet has to be protected more strongly, there's no place for the HTTP referer anymore. Let's abolish this obsolete piece of the HTTP specification that was introduced in 1996 when tracking was not an issue. * Websites want to know where we are visiting from? We shouldn't care. * Websites want to use the referer for promotional purposes? We shouldn't care. * Websites try to create statistics by using HTTP referer data? We shouldn't care. Let's stop thinking about websites' intentions and protect ourselves.

tantalor 12y ago
It's a good thought... can you expand on it? What is the state of Referer in modern browsers? Why keep it? Would removing it break anything? File bugs against browser developers to remove Referer. Meanwhile users can disable Referer, * https://chrome.google.com/webstore/detail/referer-control/hnkcfpcejkafcihlgbojoidoihckciin?hl=en https://chrome.google.com/webstore/detail/referer-control/hn... * http://www.technipages.com/firefox-enable-disable-referrer http://www.technipages.com/firefox-enable-disable-referrer * http://www.nullrefer.com/hide_referrer.html http://www.nullrefer.com/hide_referrer.html It could also be removed by a proxy. This becomes less of an issue as sites move to https, where Referer is not sent.
- marco1 12y ago
  > What is the state of Referer in modern browsers? Referers are still sent in all popular browers and expose information that is not necessary at all. > Why keep it? I can't think of any reason. Can you? > Would removing it break anything? It might certainly break some websites. But mechanisms which rely on the HTTP referer are usually broken, anyway. It should not affect too many sites. > File bugs against browser developers to remove Referer The referer is such a fundamental feature (not due to its importance but history/habit) that a filed bug will probably be closed within minutes ;) > Meanwhile users can disable Referer Certainly. I have. But as with all security-related things, how many "normal" users will do this? 99% of users don't even know about "about:config" in Firefox etc. > This becomes less of an issue as sites move to https, where Referer is not sent This is wrong. Even on HTTPS, the referer is always sent unless the referenced site is plain (not secure) HTTP.
  - tantalor 12y ago
    > Even on HTTPS, the referer is always sent Ah I was mistaken... I assumed Referer was never sent cross-domain in this case. That seems wrong.
mdpm 12y ago
I've seen this as a security issue several times, where older enterprise tools have embedded URL session keys (horrible, I know) or other private data, and act as the referrer to public sites. This either exposes more of these tools' data and functions to traffic sniffers, or malicious site operators. There is no legitimate purpose to the referrer within applications that isn't replaceable via internal claims based tokens, or external sites' parameterised (thus, opt in) source tracking.
- marco1 12y ago
  Exactly, there's no reason to keep the referer. But deprecation is necessary, so that the tons of websites that (wrongly) rely on the referer can adapt.
logn 12y ago
Worth considering Tor browser's stance on this (and a plethora of other issues). See A.1. Deprecation Wishlist https://www.torproject.org/projects/torbrowser/design/ https://www.torproject.org/projects/torbrowser/design/ I happen to disagree with Tor on this though (they worry about breaking sites). I've found that setting referrer to target site's host has pretty good results (this can be done with various plugins). In general, browser maintainers seem to prioritize compatibility over security. I think a new browser or fork that prioritizes security, possibly Tor, will eventually become dominant. And I'm excited for Servo, the renderer to replace Gecko, which is written in Rust (memory safe).
- marco1 12y ago
  The Tor project is not in the position to change or deprecate such things. On the contrary, if Chrome or Firefox would deprecate the referer, we could get rid of it, gradually. The same is true for the worrisome behaviour of `window.name`. As I said, mechanisms that rely on the referer for "authentication", such as image requests, are broken.
Perdition 12y ago
This would break too many websites (I've seen it used by a lot of banks as part of their authentication systems) to get through any kind of standards committee. Instead download something like RefControl, and advocate for browsers to include such functionality in their core.
- marco1 12y ago
  You can change everything on the web if you only give service operators enough time to adjust their sites.
gergo_v 12y ago
While the concern for user privacy is certainly understandable, abolishing the HTTP referer does not solve the issue. In fact it can be argued that it could do harm than good - basically every web based business relies on it in one way or another (customer acquisition channels, news mentions, backlinks, attribution, etc). The referrer itself is not at all used in tracking the user - as the visitor data is usually aggregated by referring source. Fingerprinting and evercookies are much more common ways of tracking individual users, and you can bet that there are even more advanced methods available for government agencies with service-level access.
- marco1 12y ago
  Of course, it is used for tracking. Fingerprinting and evercookies are the means to identify users, referers are one of the many things that you then track, assigned to those user identifiers. Apart from that, inadvertently leaking information is easy with the HTTP referer.
  - gergo_v 12y ago
    I've worked with quite a few pieces of security software that rely on user tracking, and the referrer information was only used (if at all) to tie-in with other parts of the system (ie. which marketing channel brought the user in).
bramgg 12y ago
Agreed. It's used for pretty much nothing but tracking/advertising, something that's not the end-users problem. I would also wager that 99% of people don't realize that $embarrassing-site.com knows that they just arrived from $social-network.com/$profile, or vice versa.
spdustin 12y ago
With no referrer, how would content owners prevent content thieves from stealing away bandwidth by hosting stolen markup on the thief's server that still refer to images on the author's?

12 points by marco1 12y ago

tantalor 12y ago

It's a good thought... can you expand on it? What is the state of Referer in modern browsers? Why keep it? Would removing it break anything? File bugs against browser developers to remove Referer. Meanwhile users can disable Referer, * https://chrome.google.com/webstore/detail/referer-control/hnkcfpcejkafcihlgbojoidoihckciin?hl=en https://chrome.google.com/webstore/detail/referer-control/hn... * http://www.technipages.com/firefox-enable-disable-referrer http://www.technipages.com/firefox-enable-disable-referrer * http://www.nullrefer.com/hide_referrer.html http://www.nullrefer.com/hide_referrer.html It could also be removed by a proxy. This becomes less of an issue as sites move to https, where Referer is not sent.

marco1 12y ago
> What is the state of Referer in modern browsers? Referers are still sent in all popular browers and expose information that is not necessary at all. > Why keep it? I can't think of any reason. Can you? > Would removing it break anything? It might certainly break some websites. But mechanisms which rely on the HTTP referer are usually broken, anyway. It should not affect too many sites. > File bugs against browser developers to remove Referer The referer is such a fundamental feature (not due to its importance but history/habit) that a filed bug will probably be closed within minutes ;) > Meanwhile users can disable Referer Certainly. I have. But as with all security-related things, how many "normal" users will do this? 99% of users don't even know about "about:config" in Firefox etc. > This becomes less of an issue as sites move to https, where Referer is not sent This is wrong. Even on HTTPS, the referer is always sent unless the referenced site is plain (not secure) HTTP.
- tantalor 12y ago
  > Even on HTTPS, the referer is always sent Ah I was mistaken... I assumed Referer was never sent cross-domain in this case. That seems wrong.

mdpm 12y ago

I've seen this as a security issue several times, where older enterprise tools have embedded URL session keys (horrible, I know) or other private data, and act as the referrer to public sites. This either exposes more of these tools' data and functions to traffic sniffers, or malicious site operators. There is no legitimate purpose to the referrer within applications that isn't replaceable via internal claims based tokens, or external sites' parameterised (thus, opt in) source tracking.

marco1 12y ago
Exactly, there's no reason to keep the referer. But deprecation is necessary, so that the tons of websites that (wrongly) rely on the referer can adapt.

logn 12y ago

Worth considering Tor browser's stance on this (and a plethora of other issues). See A.1. Deprecation Wishlist https://www.torproject.org/projects/torbrowser/design/ https://www.torproject.org/projects/torbrowser/design/ I happen to disagree with Tor on this though (they worry about breaking sites). I've found that setting referrer to target site's host has pretty good results (this can be done with various plugins). In general, browser maintainers seem to prioritize compatibility over security. I think a new browser or fork that prioritizes security, possibly Tor, will eventually become dominant. And I'm excited for Servo, the renderer to replace Gecko, which is written in Rust (memory safe).

marco1 12y ago
The Tor project is not in the position to change or deprecate such things. On the contrary, if Chrome or Firefox would deprecate the referer, we could get rid of it, gradually. The same is true for the worrisome behaviour of `window.name`. As I said, mechanisms that rely on the referer for "authentication", such as image requests, are broken.

Perdition 12y ago

This would break too many websites (I've seen it used by a lot of banks as part of their authentication systems) to get through any kind of standards committee. Instead download something like RefControl, and advocate for browsers to include such functionality in their core.

marco1 12y ago
You can change everything on the web if you only give service operators enough time to adjust their sites.

gergo_v 12y ago

While the concern for user privacy is certainly understandable, abolishing the HTTP referer does not solve the issue. In fact it can be argued that it could do harm than good - basically every web based business relies on it in one way or another (customer acquisition channels, news mentions, backlinks, attribution, etc). The referrer itself is not at all used in tracking the user - as the visitor data is usually aggregated by referring source. Fingerprinting and evercookies are much more common ways of tracking individual users, and you can bet that there are even more advanced methods available for government agencies with service-level access.

marco1 12y ago
Of course, it is used for tracking. Fingerprinting and evercookies are the means to identify users, referers are one of the many things that you then track, assigned to those user identifiers. Apart from that, inadvertently leaking information is easy with the HTTP referer.
- gergo_v 12y ago
  I've worked with quite a few pieces of security software that rely on user tracking, and the referrer information was only used (if at all) to tie-in with other parts of the system (ie. which marketing channel brought the user in).

bramgg 12y ago

Agreed. It's used for pretty much nothing but tracking/advertising, something that's not the end-users problem. I would also wager that 99% of people don't realize that $embarrassing-site.com knows that they just arrived from $social-network.com/$profile, or vice versa.

spdustin 12y ago

With no referrer, how would content owners prevent content thieves from stealing away bandwidth by hosting stolen markup on the thief's server that still refer to images on the author's?