9 ms·

An advanced browser fingerprint calculator aimed mainly at Tor Browser users

158 points by jerheinze 10y ago

gregw134 10y ago
I worked on designing tracking scripts for six months (Sorry. Fortunately they aren't in production). Fingerprinting was very difficult to pull off in practice: even with canvas fingerprinting, font enumeration, plugin enumeration, etc. most mobile phones are still indistinguishable. Desktops are easier to fingerprint because they often have unique browser plugins or a unique set of fonts installed. Even with desktops, fonts and other settings usually change within a matter of days, so its difficult to identify a user unless they're browsing from the same ip address you've seen them at before. What used to work really well were Flash cookies. Adobe had a security hole where Flash cookies weren't cleared when you cleared your regular cookies. The only way to clear your Flash cookies was to open the Flash application on your laptop and clear all content, or visit a special webpage Adobe built to help users clear their cookies. So for years marketers could store any cookies they wanted this way. This only ended when Chrome began embedding a version of Flash into the browser so Flash cookies could be deleted when other cookies are deleted. The other mechanism that was really interesting was ETag tracking. When you request a picture or other asset from a website, the website can send you an etag id which is supposed to signify the picture's version. When the client revisits the page, the client sends back the etag to confirm the version cached is the same as the version on the server. The security leak is that the etag protocol allows arbitrary text to be set as an etag, so to set an etag cookie all you have to do is place a 1x1 pixel on each page with a random GUID, and when the user revisits the page the browser will resend the tracking etag in its request for the 1x1 tracking pixel. This works for browsers with cookies disabled, and will remain when cookies are cleared. The only way to clear it is to clear all browsing history entirely, including cached images. Fortunately, Chrome now clears cached images by default when you clear your cookies.
- trendia 10y ago
  While you were working on it, how did you feel about the ethics of trying to unmask people who are clearly trying to browse privately?
  - eatbitseveryday 10y ago
    Understanding how unmasking works by a benevolent party means we provide competition for the malevolent parties, for the benefit of those wanting to browse privately.
  - retox 10y ago
    I understand where you're coming from. In a past life I trained defence people how to build better software for their killing machines. Sometimes you just have to pay the bills.
    - 3131s 10y ago
      That argument is terrible. If you are smart enough to program weapons systems for a defense contractor then you are smart enough to make money doing something else.
      - raverbashing 10y ago
        The issue is convincing the other people that someone who does that doesn't need to be an "Angular Ninja" or know by heart the complexity of several sorting algorithms
        bartread 10y ago
        This. I understand that sometimes you might need somebody who can hit the ground running with a particular piece of technology but, most of the time, what you need is a competent engineer, regardless of whether they are currently intimately familiar with your technology du jour.
      - elastic_church 10y ago
        Only because it has no bounds, like most schools of ethics, but it is responsible financially in this society and Op doesnt work there anymore
      - retox 10y ago
        I was fresh out of uni, teaching Ada and C++ best practice, along with toolkit and processes. Literally spent my last penny getting to the job interview, then lived on credit and supermarket end-of-day markdowns for six months.
- smilliken 10y ago
  > What used to work really well were Flash cookies. I heard from an adtech company that a patent troll was granted a patent on this (incredible, using an Adobe feature as designed wins a patent), and would wait until adtech companies reached a sufficient size before suing. They had a well known reputation of settling for a few million and weren't contested.
- dorgo 10y ago
  I came across a project that combines all possible ways to store cookies (can't find it now). If user deleted one type of cookie it was recovered from other types. The same information was encoded into 10 to 15 types of cookies (like flash cookie, image cache, etags, browser storage ..). I managed to defeat this supercookie - but it was not easy.
  - kbenson 10y ago
    That sounds like evercookie[1] , which has been around for a while. 1: http://samy.pl/evercookie/ http://samy.pl/evercookie/
  - gregw134 10y ago
    I took a look at that project too. In modern chrome at least, clearing your browsing history and cookies and cache clears all those cookies.
- jfindley 10y ago
  There are a whole BUNCH of other ways available. In practice, no single thing is even close to 100% accurate, but once you have enough distinct indicators you can fingerprint to a pretty high level of accuracy (for many advertisers 80% accuracy is enough, and it's probably not worth spending lots of effort to go above 90%). Clock skew, for example, is slightly different on each device, but on a given device doesn't change that quickly. Mouse/finger movement patterns, TLS negotiation timings, etc, etc. There are even technologies able to track users across different devices, but it's been too long since I worked in this area to be able to speak confidently about the methods used to implement that.
shmolyneaux 10y ago
Under the "Tor" tab it states that the browser should be set to a window size of 1000x1000 or a multiple of 200x100. Is this to stay consistent across all Tor users? I would have thought that 1920x1080 would be fine to help stay anonymous.
- mirimir 10y ago
  Yes, you want Tor browser window sizes in as few bins as possible. On Linux, it only accepts those sizes.
- kibwen 10y ago
  Assuming the browser window is maximized, even if most screens are 1920x1080 the browser viewport area will differ based on OS, window manager, and custom settings.
- throwaway2016a 10y ago
  At least on OS X if you try to maximize it will give you a stern warning.
throwaway2016a 10y ago
I'm confused at the "Aimed mainly at Tor" part. A lot of these techniques use Javascript and the Tor browser blocks Javascript in general and warns strongly against enabling it. So it seems that part of the technique will be ineffective against most Tor users.
- eugeniub 10y ago
  What? JavaScript is enabled by default on the Tor Browser Bundle, and users are not actively encouraged to turn it off.
  - agumonkey 10y ago
    They were at some point.
  - deleted 10y ago
    [deleted]
  - deleted 10y ago
    [deleted]
  - deleted 10y ago
    [deleted]
  - mirimir 10y ago
    It's complicated. Javascript is enabled, but NoScript is installed. But NoScript by default allows all "honest" scripts, although it does block some stuff. But that's in the default "low security" mode. The goal is having websites work, so users won't give up on Tor, while providing some security. Users can increase security, but there are only three options (low, medium and high). That's to keep user profiles in fewer bins. In the "high security" mode, all scripts are blocked.
    - cookiecaper 10y ago
      I used NoScript roughly ten years ago and it was a lot of work to manually enable JavaScripts all over the place. I used it for a couple of years solid, but eventually gave up on it, since I was practically just enabling JavaScripts on every site anyway. Everyone I tried to introduce it to gave up on it after a few weeks at most. To NoScript's credit, it did block a phishing site that I otherwise would've fallen for once. That was before the era of single-page apps. Like it or not, JavaScript is mandatory for even basic functionality on the modern web. Telling users to introduce 4 clicks to load every new domain and potentially experience some significant breakages (e.g., I remember some checkout processes failing because they'd bounce the request around between scripts from processors, fraud prevention, etc.) just on the remote chance that they'll encounter malicious JavaScript is simply a non-starter. Something like Ghostery that mostly-transparently blocks things is a better proposition for ordinary adoption.
kreetx 10y ago
There's a layout bug where the nav bar covers content at >768px of with to whenever the logo and nav links are on the same row again.
shultays 10y ago
I have a few questions about finger printing. Why Tor sends window size to servers? If it has to, why it can't send closest multiple of 100 instead of real value? Why servers request size of client windows anyway? I assume so they can serve different resolution images to clients or maybe different layouts, is this correct? But then instead of sending 1920x1080, simply sending 1900x1100 would also work right? Same goes for fonts, as soon as you install a few different fonts, you are pretty much unique now. Why does a browser has to send the fonts you have? Shouldn't it be possible to only send the fonts you have? Default fonts of OSes enabled by default, and new fonts are disabled?
- jerheinze 10y ago
  > Why Tor sends window size to servers? If it has to, why it can't send closest multiple of 100 instead of real value? It's not Tor that does so, by using JS one can determine canvas size. Worse, CSS alone can be used to extract.[1] You can take a look here for a live demo.[2] [1] : https://matt.traudt.xyz/p/YF4ciVY6.html https://matt.traudt.xyz/p/YF4ciVY6.html [2] : https://system33.pw/cssmediapoc https://system33.pw/cssmediapoc
- ryacko 10y ago
  actually common browser window sizes are divisible by 128, which means that you could just bitmask all but the last few digits to achieve the same result in a few cycles
  - Walf 10y ago
    No, you actually have to display the content inside a pane of a common size because there are many ways to get screen size indirectly from content that's occupying it. Think responsive mode of developer tools.
- Vinnl 10y ago
  The browser needn't directly send info to the server; it simply has to be available to client-side Javascript. That can then, in turn, send that info to the server, which (I think) is really hard for browsers to detect.
Walf 10y ago
Surely there's some way to detect when a script touches far too many APIs such as setting several font families in succession. Then pause execution, warn the user about potential fingerprinting, and either disable script or blackhole its network requests.
- amelius 10y ago
  Or, the browser could change the output of some APIs in subtle ways.
- jerheinze 10y ago
  TBB already does that for example with HTML5 canvas element, it shows you this box: https://pbs.twimg.com/media/C1_n50BW8AACLKq.jpg https://pbs.twimg.com/media/C1_n50BW8AACLKq.jpg
eptcyka 10y ago
Why don't we just create a VM with the tor browser preinstalled ? Surely, it would be a lot harder to do fingerprinting. ETags would still make you vulnerable, but caching can also be disabled. Then you're left with cookies.
- TazeTSchnitzel 10y ago
  TAILS exists, but that doesn't necessarily prevent fingerprinting.
mrferos 10y ago
Have used several browser fingerprinting services and have tried a few of the techniques myself, they're incredibly useful for fraud prevention when said fingerprint is reported against a central database alongside with the fraud that happened to _you_. The next time said fingerprint shows up at an eCommerce site, they'll be blocked off from purchasing or at least flagged for additional verification. They're also just useful for super targeted ads. shrug
- gregw134 10y ago
  They could also be useful, in theory, for detecting session hijacking through cookie sniffing. Did somebody's device fingerprint change mid-session? It's possible that person's account has been hacked, so they should at least be made to log in again.
golergka 10y ago
I symphathise with that, but HN is a US-centric liberal echo chamber when it comes to political issues, so everything that got to do with government, intelligence and surveillance is automatically labelled 'evil'.
- dang 10y ago
  It's common for ideologically committed users to see HN as being aligned against them. But this perception is in the eye of the beholder, i.e. it's a cognitive bias. Plenty of comments make opposite claims about HN; the difference isn't in HN but in what the commenter identifies with. Edit: I've written about this in plenty of places: https://hn.algolia.com/?sort=byDate&prefix&page=0&dateRange=all&type=comment&query=by:dang%20cognitive%20bias https://hn.algolia.com/?sort=byDate&prefix&page=0&dateRange=.... Not all of those posts are about perceptions of political bias; some are about perceptions of astroturfing. But the two phenomena are variations of the same thing. We detached this comment from https://news.ycombinator.com/item?id=13930232 https://news.ycombinator.com/item?id=13930232 and marked it off-topic.
  - golergka 10y ago
    Why did you mark my comment off-topic? Wasn't it directly related to topic discussed in the parent thread?
    - dang 10y ago
      Your comment was about Hacker News, not the topic. Also, it was unsubstantive, and those are always off-topic.
- woodandsteel 10y ago
  No, that's not true. HN has lots of posts that are highly critical of the US government and also US corporations. By the way, if you don't like liberalism (democracy plus free market economics) then what are you for?
beardog 10y ago
A good fingerprinting method (only applies to devices on private networks), is using JavaScript to enumerate the devices/services on a user's network (running HTTP(S) or other services if they are in the 'safe' port range). You can also test for models/versions of a router on their network (for example, many routers allow access to static content such as images without authentication), so if a unique/uncommon image, CSS, or JavaScript URL can be accessed without authentication, then the user can be fingerprinted not just across browsers, but across devices as well (even in a VM). This is done using network timing (to test if TCP servers exist) and the onload/onerror XHR events which can be done even for 3rd party origins, by creating img or iframe elements.
suvelx 10y ago
I've noticed that the Tails and other privacy focused tools go to great lengths to look the same as other users. And as far as my understanding goes, this is somewhat tricky with things like canvas fingerprinting. Is there a reason why they want to look the same? Could the same result be achieved as looking unique every time? e.g. Instead of attempting to make every canvas fingerprint the same, instead make every fingerprint unique by introducing noise.

158 points by jerheinze 10y ago

I worked on designing tracking scripts for six months (Sorry. Fortunately they aren't in production). Fingerprinting was very difficult to pull off in practice: even with canvas fingerprinting, font enumeration, plugin enumeration, etc. most mobile phones are still indistinguishable. Desktops are easier to fingerprint because they often have unique browser plugins or a unique set of fonts installed. Even with desktops, fonts and other settings usually change within a matter of days, so its difficult to identify a user unless they're browsing from the same ip address you've seen them at before. What used to work really well were Flash cookies. Adobe had a security hole where Flash cookies weren't cleared when you cleared your regular cookies. The only way to clear your Flash cookies was to open the Flash application on your laptop and clear all content, or visit a special webpage Adobe built to help users clear their cookies. So for years marketers could store any cookies they wanted this way. This only ended when Chrome began embedding a version of Flash into the browser so Flash cookies could be deleted when other cookies are deleted. The other mechanism that was really interesting was ETag tracking. When you request a picture or other asset from a website, the website can send you an etag id which is supposed to signify the picture's version. When the client revisits the page, the client sends back the etag to confirm the version cached is the same as the version on the server. The security leak is that the etag protocol allows arbitrary text to be set as an etag, so to set an etag cookie all you have to do is place a 1x1 pixel on each page with a random GUID, and when the user revisits the page the browser will resend the tracking etag in its request for the 1x1 tracking pixel. This works for browsers with cookies disabled, and will remain when cookies are cleared. The only way to clear it is to clear all browsing history entirely, including cached images. Fortunately, Chrome now clears cached images by default when you clear your cookies.

trendia 10y ago
While you were working on it, how did you feel about the ethics of trying to unmask people who are clearly trying to browse privately?
- eatbitseveryday 10y ago
  Understanding how unmasking works by a benevolent party means we provide competition for the malevolent parties, for the benefit of those wanting to browse privately.
- retox 10y ago
  I understand where you're coming from. In a past life I trained defence people how to build better software for their killing machines. Sometimes you just have to pay the bills.
  - 3131s 10y ago
    That argument is terrible. If you are smart enough to program weapons systems for a defense contractor then you are smart enough to make money doing something else.
    - raverbashing 10y ago
      The issue is convincing the other people that someone who does that doesn't need to be an "Angular Ninja" or know by heart the complexity of several sorting algorithms
      - bartread 10y ago
        This. I understand that sometimes you might need somebody who can hit the ground running with a particular piece of technology but, most of the time, what you need is a competent engineer, regardless of whether they are currently intimately familiar with your technology du jour.
    - elastic_church 10y ago
      Only because it has no bounds, like most schools of ethics, but it is responsible financially in this society and Op doesnt work there anymore
    - retox 10y ago
      I was fresh out of uni, teaching Ada and C++ best practice, along with toolkit and processes. Literally spent my last penny getting to the job interview, then lived on credit and supermarket end-of-day markdowns for six months.
smilliken 10y ago
> What used to work really well were Flash cookies. I heard from an adtech company that a patent troll was granted a patent on this (incredible, using an Adobe feature as designed wins a patent), and would wait until adtech companies reached a sufficient size before suing. They had a well known reputation of settling for a few million and weren't contested.
dorgo 10y ago
I came across a project that combines all possible ways to store cookies (can't find it now). If user deleted one type of cookie it was recovered from other types. The same information was encoded into 10 to 15 types of cookies (like flash cookie, image cache, etags, browser storage ..). I managed to defeat this supercookie - but it was not easy.
- kbenson 10y ago
  That sounds like evercookie[1] , which has been around for a while. 1: http://samy.pl/evercookie/ http://samy.pl/evercookie/
- gregw134 10y ago
  I took a look at that project too. In modern chrome at least, clearing your browsing history and cookies and cache clears all those cookies.
jfindley 10y ago
There are a whole BUNCH of other ways available. In practice, no single thing is even close to 100% accurate, but once you have enough distinct indicators you can fingerprint to a pretty high level of accuracy (for many advertisers 80% accuracy is enough, and it's probably not worth spending lots of effort to go above 90%). Clock skew, for example, is slightly different on each device, but on a given device doesn't change that quickly. Mouse/finger movement patterns, TLS negotiation timings, etc, etc. There are even technologies able to track users across different devices, but it's been too long since I worked in this area to be able to speak confidently about the methods used to implement that.

shmolyneaux 10y ago

Under the "Tor" tab it states that the browser should be set to a window size of 1000x1000 or a multiple of 200x100. Is this to stay consistent across all Tor users? I would have thought that 1920x1080 would be fine to help stay anonymous.

mirimir 10y ago
Yes, you want Tor browser window sizes in as few bins as possible. On Linux, it only accepts those sizes.
kibwen 10y ago
Assuming the browser window is maximized, even if most screens are 1920x1080 the browser viewport area will differ based on OS, window manager, and custom settings.
throwaway2016a 10y ago
At least on OS X if you try to maximize it will give you a stern warning.

throwaway2016a 10y ago

I'm confused at the "Aimed mainly at Tor" part. A lot of these techniques use Javascript and the Tor browser blocks Javascript in general and warns strongly against enabling it. So it seems that part of the technique will be ineffective against most Tor users.

eugeniub 10y ago
What? JavaScript is enabled by default on the Tor Browser Bundle, and users are not actively encouraged to turn it off.
- agumonkey 10y ago
  They were at some point.
- deleted 10y ago
  [deleted]
- deleted 10y ago
  [deleted]
- deleted 10y ago
  [deleted]
- mirimir 10y ago
  It's complicated. Javascript is enabled, but NoScript is installed. But NoScript by default allows all "honest" scripts, although it does block some stuff. But that's in the default "low security" mode. The goal is having websites work, so users won't give up on Tor, while providing some security. Users can increase security, but there are only three options (low, medium and high). That's to keep user profiles in fewer bins. In the "high security" mode, all scripts are blocked.
  - cookiecaper 10y ago
    I used NoScript roughly ten years ago and it was a lot of work to manually enable JavaScripts all over the place. I used it for a couple of years solid, but eventually gave up on it, since I was practically just enabling JavaScripts on every site anyway. Everyone I tried to introduce it to gave up on it after a few weeks at most. To NoScript's credit, it did block a phishing site that I otherwise would've fallen for once. That was before the era of single-page apps. Like it or not, JavaScript is mandatory for even basic functionality on the modern web. Telling users to introduce 4 clicks to load every new domain and potentially experience some significant breakages (e.g., I remember some checkout processes failing because they'd bounce the request around between scripts from processors, fraud prevention, etc.) just on the remote chance that they'll encounter malicious JavaScript is simply a non-starter. Something like Ghostery that mostly-transparently blocks things is a better proposition for ordinary adoption.

kreetx 10y ago

There's a layout bug where the nav bar covers content at >768px of with to whenever the logo and nav links are on the same row again.

shultays 10y ago

I have a few questions about finger printing. Why Tor sends window size to servers? If it has to, why it can't send closest multiple of 100 instead of real value? Why servers request size of client windows anyway? I assume so they can serve different resolution images to clients or maybe different layouts, is this correct? But then instead of sending 1920x1080, simply sending 1900x1100 would also work right? Same goes for fonts, as soon as you install a few different fonts, you are pretty much unique now. Why does a browser has to send the fonts you have? Shouldn't it be possible to only send the fonts you have? Default fonts of OSes enabled by default, and new fonts are disabled?

jerheinze 10y ago
> Why Tor sends window size to servers? If it has to, why it can't send closest multiple of 100 instead of real value? It's not Tor that does so, by using JS one can determine canvas size. Worse, CSS alone can be used to extract.[1] You can take a look here for a live demo.[2] [1] : https://matt.traudt.xyz/p/YF4ciVY6.html https://matt.traudt.xyz/p/YF4ciVY6.html [2] : https://system33.pw/cssmediapoc https://system33.pw/cssmediapoc
ryacko 10y ago
actually common browser window sizes are divisible by 128, which means that you could just bitmask all but the last few digits to achieve the same result in a few cycles
- Walf 10y ago
  No, you actually have to display the content inside a pane of a common size because there are many ways to get screen size indirectly from content that's occupying it. Think responsive mode of developer tools.
Vinnl 10y ago
The browser needn't directly send info to the server; it simply has to be available to client-side Javascript. That can then, in turn, send that info to the server, which (I think) is really hard for browsers to detect.

Walf 10y ago

Surely there's some way to detect when a script touches far too many APIs such as setting several font families in succession. Then pause execution, warn the user about potential fingerprinting, and either disable script or blackhole its network requests.

amelius 10y ago
Or, the browser could change the output of some APIs in subtle ways.
jerheinze 10y ago
TBB already does that for example with HTML5 canvas element, it shows you this box: https://pbs.twimg.com/media/C1_n50BW8AACLKq.jpg https://pbs.twimg.com/media/C1_n50BW8AACLKq.jpg

eptcyka 10y ago

Why don't we just create a VM with the tor browser preinstalled ? Surely, it would be a lot harder to do fingerprinting. ETags would still make you vulnerable, but caching can also be disabled. Then you're left with cookies.

TazeTSchnitzel 10y ago
TAILS exists, but that doesn't necessarily prevent fingerprinting.

mrferos 10y ago

Have used several browser fingerprinting services and have tried a few of the techniques myself, they're incredibly useful for fraud prevention when said fingerprint is reported against a central database alongside with the fraud that happened to _you_. The next time said fingerprint shows up at an eCommerce site, they'll be blocked off from purchasing or at least flagged for additional verification. They're also just useful for super targeted ads. shrug

gregw134 10y ago
They could also be useful, in theory, for detecting session hijacking through cookie sniffing. Did somebody's device fingerprint change mid-session? It's possible that person's account has been hacked, so they should at least be made to log in again.

golergka 10y ago

I symphathise with that, but HN is a US-centric liberal echo chamber when it comes to political issues, so everything that got to do with government, intelligence and surveillance is automatically labelled 'evil'.

dang 10y ago
It's common for ideologically committed users to see HN as being aligned against them. But this perception is in the eye of the beholder, i.e. it's a cognitive bias. Plenty of comments make opposite claims about HN; the difference isn't in HN but in what the commenter identifies with. Edit: I've written about this in plenty of places: https://hn.algolia.com/?sort=byDate&prefix&page=0&dateRange=all&type=comment&query=by:dang%20cognitive%20bias https://hn.algolia.com/?sort=byDate&prefix&page=0&dateRange=.... Not all of those posts are about perceptions of political bias; some are about perceptions of astroturfing. But the two phenomena are variations of the same thing. We detached this comment from https://news.ycombinator.com/item?id=13930232 https://news.ycombinator.com/item?id=13930232 and marked it off-topic.
- golergka 10y ago
  Why did you mark my comment off-topic? Wasn't it directly related to topic discussed in the parent thread?
  - dang 10y ago
    Your comment was about Hacker News, not the topic. Also, it was unsubstantive, and those are always off-topic.
woodandsteel 10y ago
No, that's not true. HN has lots of posts that are highly critical of the US government and also US corporations. By the way, if you don't like liberalism (democracy plus free market economics) then what are you for?

beardog 10y ago

A good fingerprinting method (only applies to devices on private networks), is using JavaScript to enumerate the devices/services on a user's network (running HTTP(S) or other services if they are in the 'safe' port range). You can also test for models/versions of a router on their network (for example, many routers allow access to static content such as images without authentication), so if a unique/uncommon image, CSS, or JavaScript URL can be accessed without authentication, then the user can be fingerprinted not just across browsers, but across devices as well (even in a VM). This is done using network timing (to test if TCP servers exist) and the onload/onerror XHR events which can be done even for 3rd party origins, by creating img or iframe elements.

suvelx 10y ago

I've noticed that the Tails and other privacy focused tools go to great lengths to look the same as other users. And as far as my understanding goes, this is somewhat tricky with things like canvas fingerprinting. Is there a reason why they want to look the same? Could the same result be achieved as looking unique every time? e.g. Instead of attempting to make every canvas fingerprint the same, instead make every fingerprint unique by introducing noise.