5 ms·

Could you salt and perform a one-way hash on the IP address and store that? It would alleviate a large amount of leakage issues while still giving you uniquenes

by ghayes 8y ago

Could you salt and perform a one-way hash on the IP address and store that? It would alleviate a large amount of leakage issues while still giving you uniqueness counts.

saalweachter 8y ago
IPv4 addresses are only 32 bits, which makes building rainbow tables almost trivial.
- recursive 8y ago
  I thought salt was supposed to be unique per hashed value. Rainbow tables don't work in that case.
  - deleted 8y ago
    [deleted]
  - weaksauce 8y ago
    that only work if you had two pieces of information. username and password works because you can find the salt value associated with that username and then use that for the password hash. an ip would still require an unhashed thing to lookup to get the hash if you did it per ip address. for this you might be able to get away with using a sole salt value for all ip addresses but even then if you get hacked it would be trivial to write a script to compute the rainbow table when you steal the salt value.
  - darkr 8y ago
    For passwords, yes, this is generally best practice. Also, the salt is normally stored with the hashed password, as it’s not regarded as a secret. Modern GPUs can manage several thousand million SHA256 hashes/sec, so even with a salt per hash it’s not going to take long to get a given entry, given the 32bit address space of IPv4
    - jacobkg 8y ago
      You can use bcrypt or argon2 to make it much slower than that.
      - j16sdiz 8y ago
        but why? If I am got a DoS attack or Spam, I need the IP to find out to whom I should file abuse complain. Do we need to sanitize SMTP header too? How about shuting down DNSBL?
  - TeMPOraL 8y ago
    No matter how complex your scheme is, if IP address is the only input, it's a (mathematical) function of f: IP → hash. Since IP(v4) space is 32bit (in practice, slightly less), if you know the function f, you can trivially enumerate all inputs. From security point of view, if you use a fixed (unrelated to input) salt, the attacker will have a harder time to discover the function f (unless you store the salt next to your IP hashes). But from privacy point of view, in relationship between me (user) and you (service provider), you are the attacker. And you know your function f. Hashing IPv4 addresses, salt or not, gives me no privacy protection, since you can trivially reverse the hash - just due to small domain size. With IPv6, this problem will resolve itself somewhat; till then, I'd prefer if you encrypted those IPs with keys that have finite and short life time, in a way that a third party could audit if need be.
- gregmac 8y ago
  Yeah, though a salt would at least mean you'd have to rebuild the table for each site/database/whatever. However I'm having a hard time seeing how to really protect against this. The IP is a an identifier, so unlike password salt (where the user is the identifier) you need a way to know what the salt is to hash the IP, and it needs to be consistent. You can do a lookup table of IP-to-salt, but this either gives away your list of addresses (if only containing IPs you've seen) or is huge (entire ipv4 range), and either way doesn't prevent rainbow tables. You can have a static salt for the entire site, but again this is not really helping much against rainbow tables (beyond requiring recalculating the table, once). Is there a mitigation I'm not thinking of?
  - amenghra 8y ago
    You could encrypt instead of hash, and then have some policy (e.g. the decryption library/service/piece will only allow decrypting ciphertext newer than 30 days). If you need the ability to group ciphertexts without decrypting them, you could create a scheme which will make cryptographers cringe, but could be justified in this specific case.
    - saalweachter 8y ago
      For large sites, you also have the risk that you might be able to say something statistically useful about the plaintext. For instance, you can probably assert things like which IP blocks are likely to comprise most of the entries in the table or which IP blocks or addresses cannot be in the table. That just makes me all sorts of uncomfortable.
- bernardrubble 8y ago
  Not if you salt them.
masonicb00m 8y ago
I built an Nginx plugin to do something like this https://github.com/masonicboom/ipscrub https://github.com/masonicboom/ipscrub
sheeper 8y ago
It's not possible to one-way hash a 32-bit IP address. A hash of a 32-bit value can always be reversed because the search space is so small.
- lodi 8y ago
  Who cares if it’s trivially hackable; we’re talking about a legal checkbox that you have to tick.
  - sheeper 8y ago
    A reversable hash "could reasonably be linked" with the plaintext. You can't get around the law on technicalities. Judges are not computers.
    - lyxsus1 8y ago
      > You can't get around the law on technicalities. Simply out of curiosity, what do you mean by that? All my life experience and knowledge tells me it's exactly how you get around the law, unless court has its own agenda or strong bias.
      - elliekelly 8y ago
        https://en.wikipedia.org/wiki/I_know_it_when_I_see_it https://en.wikipedia.org/wiki/I_know_it_when_I_see_it
  - TeMPOraL 8y ago
    I do. People treating privacy protections as "legal checkbox that you have to tick" are the reason regulations like this show up in the first place.
- dorgo 8y ago
  Store only the first 16 bits of the hash maybe?
  - 8y ago

by ghayes 8y ago

Could you salt and perform a one-way hash on the IP address and store that? It would alleviate a large amount of leakage issues while still giving you uniqueness counts.

saalweachter 8y ago

IPv4 addresses are only 32 bits, which makes building rainbow tables almost trivial.

recursive 8y ago
I thought salt was supposed to be unique per hashed value. Rainbow tables don't work in that case.
- deleted 8y ago
  [deleted]
- weaksauce 8y ago
  that only work if you had two pieces of information. username and password works because you can find the salt value associated with that username and then use that for the password hash. an ip would still require an unhashed thing to lookup to get the hash if you did it per ip address. for this you might be able to get away with using a sole salt value for all ip addresses but even then if you get hacked it would be trivial to write a script to compute the rainbow table when you steal the salt value.
- darkr 8y ago
  For passwords, yes, this is generally best practice. Also, the salt is normally stored with the hashed password, as it’s not regarded as a secret. Modern GPUs can manage several thousand million SHA256 hashes/sec, so even with a salt per hash it’s not going to take long to get a given entry, given the 32bit address space of IPv4
  - jacobkg 8y ago
    You can use bcrypt or argon2 to make it much slower than that.
    - j16sdiz 8y ago
      but why? If I am got a DoS attack or Spam, I need the IP to find out to whom I should file abuse complain. Do we need to sanitize SMTP header too? How about shuting down DNSBL?
- TeMPOraL 8y ago
  No matter how complex your scheme is, if IP address is the only input, it's a (mathematical) function of f: IP → hash. Since IP(v4) space is 32bit (in practice, slightly less), if you know the function f, you can trivially enumerate all inputs. From security point of view, if you use a fixed (unrelated to input) salt, the attacker will have a harder time to discover the function f (unless you store the salt next to your IP hashes). But from privacy point of view, in relationship between me (user) and you (service provider), you are the attacker. And you know your function f. Hashing IPv4 addresses, salt or not, gives me no privacy protection, since you can trivially reverse the hash - just due to small domain size. With IPv6, this problem will resolve itself somewhat; till then, I'd prefer if you encrypted those IPs with keys that have finite and short life time, in a way that a third party could audit if need be.
gregmac 8y ago
Yeah, though a salt would at least mean you'd have to rebuild the table for each site/database/whatever. However I'm having a hard time seeing how to really protect against this. The IP is a an identifier, so unlike password salt (where the user is the identifier) you need a way to know what the salt is to hash the IP, and it needs to be consistent. You can do a lookup table of IP-to-salt, but this either gives away your list of addresses (if only containing IPs you've seen) or is huge (entire ipv4 range), and either way doesn't prevent rainbow tables. You can have a static salt for the entire site, but again this is not really helping much against rainbow tables (beyond requiring recalculating the table, once). Is there a mitigation I'm not thinking of?
- amenghra 8y ago
  You could encrypt instead of hash, and then have some policy (e.g. the decryption library/service/piece will only allow decrypting ciphertext newer than 30 days). If you need the ability to group ciphertexts without decrypting them, you could create a scheme which will make cryptographers cringe, but could be justified in this specific case.
  - saalweachter 8y ago
    For large sites, you also have the risk that you might be able to say something statistically useful about the plaintext. For instance, you can probably assert things like which IP blocks are likely to comprise most of the entries in the table or which IP blocks or addresses cannot be in the table. That just makes me all sorts of uncomfortable.
bernardrubble 8y ago
Not if you salt them.

masonicb00m 8y ago

I built an Nginx plugin to do something like this https://github.com/masonicboom/ipscrub https://github.com/masonicboom/ipscrub

sheeper 8y ago

It's not possible to one-way hash a 32-bit IP address. A hash of a 32-bit value can always be reversed because the search space is so small.

lodi 8y ago
Who cares if it’s trivially hackable; we’re talking about a legal checkbox that you have to tick.
- sheeper 8y ago
  A reversable hash "could reasonably be linked" with the plaintext. You can't get around the law on technicalities. Judges are not computers.
  - lyxsus1 8y ago
    > You can't get around the law on technicalities. Simply out of curiosity, what do you mean by that? All my life experience and knowledge tells me it's exactly how you get around the law, unless court has its own agenda or strong bias.
    - elliekelly 8y ago
      https://en.wikipedia.org/wiki/I_know_it_when_I_see_it https://en.wikipedia.org/wiki/I_know_it_when_I_see_it
- TeMPOraL 8y ago
  I do. People treating privacy protections as "legal checkbox that you have to tick" are the reason regulations like this show up in the first place.
dorgo 8y ago
Store only the first 16 bits of the hash maybe?
- 8y ago