6 ms·

I think a more robust (and privacy-preserving) way to calculate uniques ( per time-frame as well) is to make use of the local storage / indexDB in the visitor's

by __ka 7y ago

I think a more robust (and privacy-preserving) way to calculate uniques ( per time-frame as well) is to make use of the local storage / indexDB in the visitor's browser. This is how it would work: - Visitor X lands on your website. - With some JavaScript you check local storage if `dailyUnique` for today's date is set. - if yes, send `visit` signal - if not set, send `unique visit` signal and set `dailyUnique` to local storage. You can apply this to any analytics use case. Its is private and does not rely on referrers. We have been using this goal-attainment approach to do analytics at my company for quite some years.

AdriaanvRossum 7y ago
Author here. Our tool is GDPR compliant out of the box. We can't use local storage (it's legal wise the same as a cookie). Although your suggestion will provide more accurate stats, it's a trade-off we are willing to accept.
- bouncycastle 7y ago
  How about referring from a https site? AFAIK, browsers don't send a referrer from s https site.
  - michaelbuckbee 7y ago
    If the referring site is http and the site being referred to it won't send the referrer header. http --> https = no referrer header https --> https = referrer header
    - sm4rk0 7y ago
      Actually: https --> http = no referrer header
  - sm4rk0 7y ago
    > HTTPS sites will never transmit referer information to non-HTTPS sites https://developer.mozilla.org/en-US/docs/Web/Security/Referer_header:_privacy_and_security_concerns https://developer.mozilla.org/en-US/docs/Web/Security/Refere...
  - AdriaanvRossum 7y ago
    That is not really relevant because if a user comes from a different website and the referrer is not present, it will be counted as unique. Having a referrer of that other website or having no referrer does not change anything. We both treat them as unique. The only referrer we actually use is the one from the same site to the same site. Those requests are always with the same protocol (and if you not you should change that). In that case we see both the hostname and the referrer are equal thus being non unique.
- __ka 7y ago
  > it's legal wise the same as a cookie I am not sure that is the case. GDPR makes provisions for personal data that can uniquely identify users. Blanket statements like: Local storage is not allowed I think are misleading. The state is persisted in the client's machine. Unlike cookies, which get attached to all requests in the specified path, local storage items are not transmitted with the request. Furthermore, in the approach I recommended earlier, no unique identifiers are being sent with the request at all. I am pretty sure that is GDPR compliant, but would love to be pointed to legal provisions that would suggest otherwise. > it's a trade-off we are willing to accept. Referrers, in my opinion, are not reliable enough to derive uniques, and I would assume (although I would not have any numbers to back it up), that the margin of error is very significant when you consider every condition under which referrers would not be sent (some very good cases when that happens are mentioned by other people in this very thread)
  - jedimastert 7y ago
    Functionally, local storage is the same as cookies with the only real difference being it requires an active request. Any information stored can immediately moved to and from the browser to the server with fetch or whatever. "Uniquely identifying users" would work with local storage the same as a cookie.
    - true_religion 7y ago
      It could be moved, but until you actually do it it’s not tracking but merely logging.
      - yorwba 7y ago
        The same is true for cookies.
        true_religion 7y ago
        But cookies are automatically sent with every request to your server. So you would need to take special care to ensure they aren’t sent until you have permission.
- pkalinowski 7y ago
  How about using quickly expiring cookies to tell if visit is unique (no cookie) or same (cookie exist). It would get reset if external referer is recognized.
  - NullPrefix 7y ago
    Setting a short expiring cookie is still setting a cookie.
    - pleasecalllater 7y ago
      Just like setting a cookie that I didn't agree for setting other cookies :)
jedimastert 7y ago
How is this different from using a cookie, from a privacy perspective?
- thaumasiotes 7y ago
  There's no difference between a cookie and local storage -- that's what a cookie is. But there is a difference between using an identifier of 2019-12-14 vs an identifier of 175c6328-0699-4577-8364-3db5071c2173.
  - grsmvg 7y ago
    Would this pass GDPR without a cookie notice?
    - thaumasiotes 7y ago
      > ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person; > ‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction; As far as the English text of the regulation goes, it's clear that the generation, detection, and use of this record count as "processing", as long as the record itself is "personal data". It's not clear that this is the case. A record stating "this browser has visited XXXX website today" does nothing, in the absence of other records, to identify the person providing the record. But this is open to some interpretation. In particular, you might be getting this record from web requests that already identify the user by other means (perhaps they're logged in). In that case, someone could argue that the fact of the user having visited (or not) your site before on the same day is data that pertains to them specifically ("personal data"), and that your making note of it is prohibited by default under article 6 of the GDPR. The counterargument would be that when the data is reified in your use and your records, it has already become impossible to relate to any individual person. You'd have to rely on that counterargument, because article 6 won't help you at all: > 1. Processing shall be lawful only if and to the extent that at least one of the following applies: > (a) the data subject has given consent to the processing of his or her personal data for one or more specific purposes; > (b) processing is necessary for the performance of a contract to which the data subject is party or in order to take steps at the request of the data subject prior to entering into a contract; > (c) processing is necessary for compliance with a legal obligation to which the controller is subject; > (d) processing is necessary in order to protect the vital interests of the data subject or of another natural person; > (e) processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller; > (f) processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child. > Point (f) of the first subparagraph shall not apply to processing carried out by public authorities in the performance of their tasks. None of these will apply unless you are an agent of the government. ----- Thought experiment: as a hostile website operator, you decide to attack your users by filling their local storage. You generate random bytes and store them under random keys to the limit of what their browser will allow. You don't yourself know what the keys are. Is this a GDPR violation? Those random bytes are highly entropic identifiers which you processed and associated with individual users.
donohoe 7y ago
Sorry, wouldn't work. Directive 2009/136/EC (aka EU Cookie Law) is not limited to "cookies". The law itself doesn't even mention the word "cookie"! It covers localStorage and any similar features that may come along in the future.
- ricardobeat 7y ago
  The GDPR law concerns data which can be used to identify / track a person. It does mention cookies once! Quote: > Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them. This would not apply to the above method since it is not an identifier. The problem is, there is another regulation called the ePrivacy directive (from 2009, the original 'cookie law') that states you need user consent before setting any kind of cookies other than the strictly necessary. In the privacy dialog, these are allowed to be pre-selected, unlike identifying cookies, but you still need to acquire consent explicitly. Not a lawyer, this is not legal advice.

by __ka 7y ago

AdriaanvRossum 7y ago

Author here. Our tool is GDPR compliant out of the box. We can't use local storage (it's legal wise the same as a cookie). Although your suggestion will provide more accurate stats, it's a trade-off we are willing to accept.

bouncycastle 7y ago
How about referring from a https site? AFAIK, browsers don't send a referrer from s https site.
- michaelbuckbee 7y ago
  If the referring site is http and the site being referred to it won't send the referrer header. http --> https = no referrer header https --> https = referrer header
  - sm4rk0 7y ago
    Actually: https --> http = no referrer header
- sm4rk0 7y ago
  > HTTPS sites will never transmit referer information to non-HTTPS sites https://developer.mozilla.org/en-US/docs/Web/Security/Referer_header:_privacy_and_security_concerns https://developer.mozilla.org/en-US/docs/Web/Security/Refere...
- AdriaanvRossum 7y ago
  That is not really relevant because if a user comes from a different website and the referrer is not present, it will be counted as unique. Having a referrer of that other website or having no referrer does not change anything. We both treat them as unique. The only referrer we actually use is the one from the same site to the same site. Those requests are always with the same protocol (and if you not you should change that). In that case we see both the hostname and the referrer are equal thus being non unique.
__ka 7y ago
> it's legal wise the same as a cookie I am not sure that is the case. GDPR makes provisions for personal data that can uniquely identify users. Blanket statements like: Local storage is not allowed I think are misleading. The state is persisted in the client's machine. Unlike cookies, which get attached to all requests in the specified path, local storage items are not transmitted with the request. Furthermore, in the approach I recommended earlier, no unique identifiers are being sent with the request at all. I am pretty sure that is GDPR compliant, but would love to be pointed to legal provisions that would suggest otherwise. > it's a trade-off we are willing to accept. Referrers, in my opinion, are not reliable enough to derive uniques, and I would assume (although I would not have any numbers to back it up), that the margin of error is very significant when you consider every condition under which referrers would not be sent (some very good cases when that happens are mentioned by other people in this very thread)
- jedimastert 7y ago
  Functionally, local storage is the same as cookies with the only real difference being it requires an active request. Any information stored can immediately moved to and from the browser to the server with fetch or whatever. "Uniquely identifying users" would work with local storage the same as a cookie.
  - true_religion 7y ago
    It could be moved, but until you actually do it it’s not tracking but merely logging.
    - yorwba 7y ago
      The same is true for cookies.
      - true_religion 7y ago
        But cookies are automatically sent with every request to your server. So you would need to take special care to ensure they aren’t sent until you have permission.
pkalinowski 7y ago
How about using quickly expiring cookies to tell if visit is unique (no cookie) or same (cookie exist). It would get reset if external referer is recognized.
- NullPrefix 7y ago
  Setting a short expiring cookie is still setting a cookie.
  - pleasecalllater 7y ago
    Just like setting a cookie that I didn't agree for setting other cookies :)

jedimastert 7y ago

How is this different from using a cookie, from a privacy perspective?

thaumasiotes 7y ago
There's no difference between a cookie and local storage -- that's what a cookie is. But there is a difference between using an identifier of 2019-12-14 vs an identifier of 175c6328-0699-4577-8364-3db5071c2173.
- grsmvg 7y ago
  Would this pass GDPR without a cookie notice?
  - thaumasiotes 7y ago
    > ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person; > ‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction; As far as the English text of the regulation goes, it's clear that the generation, detection, and use of this record count as "processing", as long as the record itself is "personal data". It's not clear that this is the case. A record stating "this browser has visited XXXX website today" does nothing, in the absence of other records, to identify the person providing the record. But this is open to some interpretation. In particular, you might be getting this record from web requests that already identify the user by other means (perhaps they're logged in). In that case, someone could argue that the fact of the user having visited (or not) your site before on the same day is data that pertains to them specifically ("personal data"), and that your making note of it is prohibited by default under article 6 of the GDPR. The counterargument would be that when the data is reified in your use and your records, it has already become impossible to relate to any individual person. You'd have to rely on that counterargument, because article 6 won't help you at all: > 1. Processing shall be lawful only if and to the extent that at least one of the following applies: > (a) the data subject has given consent to the processing of his or her personal data for one or more specific purposes; > (b) processing is necessary for the performance of a contract to which the data subject is party or in order to take steps at the request of the data subject prior to entering into a contract; > (c) processing is necessary for compliance with a legal obligation to which the controller is subject; > (d) processing is necessary in order to protect the vital interests of the data subject or of another natural person; > (e) processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller; > (f) processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party, except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child. > Point (f) of the first subparagraph shall not apply to processing carried out by public authorities in the performance of their tasks. None of these will apply unless you are an agent of the government. ----- Thought experiment: as a hostile website operator, you decide to attack your users by filling their local storage. You generate random bytes and store them under random keys to the limit of what their browser will allow. You don't yourself know what the keys are. Is this a GDPR violation? Those random bytes are highly entropic identifiers which you processed and associated with individual users.

donohoe 7y ago

Sorry, wouldn't work. Directive 2009/136/EC (aka EU Cookie Law) is not limited to "cookies". The law itself doesn't even mention the word "cookie"! It covers localStorage and any similar features that may come along in the future.

ricardobeat 7y ago
The GDPR law concerns data which can be used to identify / track a person. It does mention cookies once! Quote: > Natural persons may be associated with online identifiers provided by their devices, applications, tools and protocols, such as internet protocol addresses, cookie identifiers or other identifiers such as radio frequency identification tags. This may leave traces which, in particular when combined with unique identifiers and other information received by the servers, may be used to create profiles of the natural persons and identify them. This would not apply to the above method since it is not an identifier. The problem is, there is another regulation called the ePrivacy directive (from 2009, the original 'cookie law') that states you need user consent before setting any kind of cookies other than the strictly necessary. In the privacy dialog, these are allowed to be pre-selected, unlike identifying cookies, but you still need to acquire consent explicitly. Not a lawyer, this is not legal advice.