4 ms·

That is easy to fix though. Just use a temporary salt. Pseudo code: if salt.day < today(): salt = {day: today(), salt: random()} ip_hash = sha

by TekMol 3y ago

That is easy to fix though. Just use a temporary salt. Pseudo code: if salt.day < today(): salt = {day: today(), salt: random()} ip_hash = sha256(ip + salt.salt)

__alexs 3y ago
Assuming you don't store the salts, this produces a value that is useless for anything but counting something like DAU. Which you could equally just do by counting them all and deleting all the data at the end of the day, or using a cardinality estimator like HLL.
- TekMol 3y ago
  DAU in regards to a given page. Have you read the article? That is what the author's goal seems to be. He wants to prevent multiple requests to the same page by the same IP counted multiple times.
  - tatersolid 3y ago
    Is that more efficiently done with an appropriate caching header on the page as it is served? Cache-Control: private, max-age=86400 This prevents repeat requests for normal browsers from hitting the server.
- dvdkon 3y ago
  That same uselessness for long-term identification of users is what makes this approach compliant with laws regulating use of PII, since what you have after a small time window isn't actually PII (unless correlated with another dataset, but that's always the case).
- SamBam 3y ago
  That's precisely all that OP is storing in the original article. They're just getting a list of hashes per day, and associated client info. They have no idea if the same user visit them on multiple days, because the hashes will be different.
kevincox 3y ago
Of course if you have multiple severs or may reboot you need to store the salt somewhere. If you are going to bother storing the salt and cleaning it up after the day is over it may be just as easy to clean the hashes at the end of the day (and keep the total count) which is equivalent. This should work unless you want to keep individual counts around for something like seeing distribution of requests per IP or similar. But in that case you could just replace the hashes with random values at the end of the day to fully anonymize them since you no longer need to increment then.

by TekMol 3y ago

That is easy to fix though. Just use a temporary salt. Pseudo code: if salt.day < today(): salt = {day: today(), salt: random()} ip_hash = sha256(ip + salt.salt)

__alexs 3y ago

Assuming you don't store the salts, this produces a value that is useless for anything but counting something like DAU. Which you could equally just do by counting them all and deleting all the data at the end of the day, or using a cardinality estimator like HLL.

TekMol 3y ago
DAU in regards to a given page. Have you read the article? That is what the author's goal seems to be. He wants to prevent multiple requests to the same page by the same IP counted multiple times.
- tatersolid 3y ago
  Is that more efficiently done with an appropriate caching header on the page as it is served? Cache-Control: private, max-age=86400 This prevents repeat requests for normal browsers from hitting the server.
dvdkon 3y ago
That same uselessness for long-term identification of users is what makes this approach compliant with laws regulating use of PII, since what you have after a small time window isn't actually PII (unless correlated with another dataset, but that's always the case).
SamBam 3y ago
That's precisely all that OP is storing in the original article. They're just getting a list of hashes per day, and associated client info. They have no idea if the same user visit them on multiple days, because the hashes will be different.

kevincox 3y ago

Of course if you have multiple severs or may reboot you need to store the salt somewhere. If you are going to bother storing the salt and cleaning it up after the day is over it may be just as easy to clean the hashes at the end of the day (and keep the total count) which is equivalent. This should work unless you want to keep individual counts around for something like seeing distribution of requests per IP or similar. But in that case you could just replace the hashes with random values at the end of the day to fully anonymize them since you no longer need to increment then.