5 ms·

I wish services didn't store IPs at all. If abuse is an issue, why not hash the IP with a nonce?

codetrotter 5y ago
IPv4 space is small so they will subpoena the nonce and find what the original IP was
kadoban 5y ago
For ipv4 is there a difference between storing IPs and storing their hash with a nonce? You can calculate the hash of every IP address in reasonable time, so it's reversible. Only benefit I can think of is you can forget the nonce and now the data is securely useless, if the nonce was secure, but that doesn't seem that useful really.
- xvector 5y ago
  I think if we use a difficult enough hash function it should be okay? With 4 billion IPv4 addresses it would take 120+ years to iterate through all of them. You could probably rotate the nonce periodically, making it effectively worthless to pre-compute any table. But this gets complicated fast.
  - rpadovani 5y ago
    Why 120 years? It is easily parallelized, and with any cloud provider you can launch hundreds of thousands of computing unit in seconds. I'd say, as a private citizen, I can create a rainbow table of the IPv4 space in half a day, more or less?
  - somethingwitty1 5y ago
    Except you are still storing the nonce/salt (not sure which you are proposing)...which means you can reverse it, so the data is subpoenable. It doesn't really buy anyone anything, in this scenario. It could help if the logs were stolen, but that isn't what is being discussed here.
  - kadoban 5y ago
    You could try to do a more difficult hash or something (bcrypt maybe?) but I don't know if it's a very good idea. I think you'd spike your latency, open yourself to DoS attacks or only minorly inconvenience anyone reversing the hashes, or some combination of those.
    - xxs 5y ago
      >(bcrypt maybe?) but I don't know if it's a very good idea b/scrypt and all other password grade hashes are slow on purpose but they are slow per each use. Imagine the processing takes 0.1s (which is on the low side of hardness) per each request - you just killed all your servers w/o any designated DoS. If you abandon the nonce and use the same salt multiple times (so the computation is amortized), it'd take a replicated cache of IP->hash and even then it still doesn't accomplish much...
- deleted 5y ago
  [deleted]
gizmo686 5y ago
There are only 2^32 possible IP addresses. You can brute-force that on a personal laptop.
- vgaldikas 5y ago
  There's even less 'usable' ones, when you exclude private ranges etc...
  - olliej 5y ago
    And if you further restrict to US service provider ranges
- xvector 5y ago
  If you use a hard hash function you cannot brute force that on a laptop - not even a tenth of that. You can, however, spin up compute instances to brute force it in a few days if you have $50k lying around.
  - zeroimpl 5y ago
    What are the odds that a website will run a computationally hard hash function on every single HTTP request just so it can log something less sensitive than an IP address?
    - josefx 5y ago
      The website could cache the hash for an hour or two.
  - xxs 5y ago
    >force that on a laptop They said: "personal computer", which could easily have 2x GPUs and 16+ cores. Heck, laptops nowadays can have a pretty good discrete GPU. Using password grade hash with a =nonce= is absolutely no way to be accomplished per each request. The nonce would have to be the same for multiple uses - hence NOT a 'nonce'. The sharing of the said non-nonce would require a form a replicated Map (or IP-sticky processing with a local map). It's rather convoluted solution for absolutely no benefit as it's still not hard to brute force. Storing it in such a way - slow hash + salt yields no benefits for debugging either, so I wonder why would you do so? Password hashes are useful for proving a match with an unknown plain text (while making it expensive to brute force) - so what would be the exact purpose of having non-nonce+IP?
uses 5y ago
Hm, I'm confused, usually the whole point of storing an IP is in case the visitor uses the platform to do something illegal, like a death threat. Without the original IP law enforcement can't subpoena the ISP, etc. But also as someone else said, if you use a nonce, and I think you mean salt, then it can be cracked nearly instantaneously anyways due to the small space of IPv4 (~4 billion).
- ed25519FUUU 5y ago
  Wouldn’t be relevant for a news website.
  - LatteLazy 5y ago
    I wouldn't be so sure, lots of papers have comment sections and they're pretty rammed with death threats on any social issue sadly...
    - ed25519FUUU 5y ago
      Oh jeeze with comments enabled anything goes.
- njharman 5y ago
  IP is not a person identifier. They should not be allowed as evidence in criminal cases.
  - gizmo686 5y ago
    There is more to investigating then gathering admissible evividence. For example, if you are talking with Alice and she says that she heard from Bob chat Charlie was in the office at a weird hour on the day in question investigators have gains nothing that they can admit as evidence [0]. However, there is nothing that anyone (defense or a third party) can do challenge this portion of the investigation other then keep it out of evidence, and the investigators are free to follow up with either Bob or Charlie to get something that would be admissible.
aneutron 5y ago
Sometimes there's a forensic purpose. For example, you want to know which servers exfiltrated your data and to which IP. Or for audit purposes (e.g. you might need to prove to some regulator no outside access was made, which is stupid but ...)
nullc 5y ago
There are only 2^32 IPv4 addresses, if you know the nonce you just try them all... no privacy provided. If you don't know the nonce, you can't match against other users-- so not useful for abuse. But I'm skeptical re: abuse uses. For commenters, sure-- you may need to store IPs to combat abuse. But for readers? At most you would need sampled data or in-memory counters (e.g. to catch high volume bots). Unfortunately, there really isn't any penalty for failing to minimize private data collection.
- xvector 5y ago
  If you use a difficult hash function that takes ~1 seconds to calculate then it would take over 120 years to iterate through the IPv4 address space. At the very least, this could cut down on dragnet surveillance
  - 542458 5y ago
    Yes, but then I’m burning a second of compute time every time I want to log something. Also, by removing unlikely candidates (IPs owned by irrelevant entities or that are not US based) you could get the search range much much smaller, and with the FBIs budget you could probably compute it all in a few days even with a 1-second hash time.
    - deleted 5y ago
      [deleted]
  - nullc 5y ago
    But then a single user clicking on links quickly would bring your webserver to its knees. So much for using those addresses to combat abuse... :) Plus the FBI could probably narrow their search to a few hundred thousand addresses (relevant ISPs, no unroutable/multicast/etc), then only use the list to confirm. Finally, if it takes 120 years on one core, it'll take 1.4 months on 1000 cores. I'm willing to be the FBI has access to more computing power than I do. ~100 CPU years isn't a particularly daunting amount of computing work, even for fairly low stakes research. That search would also decode all addresses in the logs, not just one targeted one...
GeneralMayhem 5y ago
Anti-spam/anti-abuse operations often look a lot like tracking. There's no point in knowing a single request's IP, or a single user's; to spot the patterns, you have to be able to join it with others.
8note 5y ago
There's some structure to the IP that can be somewhat useful for tracking identity. Its imperfect, but you'd expect definitely good folks to look a certain way

IPv4 space is small so they will subpoena the nonce and find what the original IP was

For ipv4 is there a difference between storing IPs and storing their hash with a nonce? You can calculate the hash of every IP address in reasonable time, so it's reversible. Only benefit I can think of is you can forget the nonce and now the data is securely useless, if the nonce was secure, but that doesn't seem that useful really.

xvector 5y ago
I think if we use a difficult enough hash function it should be okay? With 4 billion IPv4 addresses it would take 120+ years to iterate through all of them. You could probably rotate the nonce periodically, making it effectively worthless to pre-compute any table. But this gets complicated fast.
- rpadovani 5y ago
  Why 120 years? It is easily parallelized, and with any cloud provider you can launch hundreds of thousands of computing unit in seconds. I'd say, as a private citizen, I can create a rainbow table of the IPv4 space in half a day, more or less?
- somethingwitty1 5y ago
  Except you are still storing the nonce/salt (not sure which you are proposing)...which means you can reverse it, so the data is subpoenable. It doesn't really buy anyone anything, in this scenario. It could help if the logs were stolen, but that isn't what is being discussed here.
- kadoban 5y ago
  You could try to do a more difficult hash or something (bcrypt maybe?) but I don't know if it's a very good idea. I think you'd spike your latency, open yourself to DoS attacks or only minorly inconvenience anyone reversing the hashes, or some combination of those.
  - xxs 5y ago
    >(bcrypt maybe?) but I don't know if it's a very good idea b/scrypt and all other password grade hashes are slow on purpose but they are slow per each use. Imagine the processing takes 0.1s (which is on the low side of hardness) per each request - you just killed all your servers w/o any designated DoS. If you abandon the nonce and use the same salt multiple times (so the computation is amortized), it'd take a replicated cache of IP->hash and even then it still doesn't accomplish much...
deleted 5y ago
[deleted]

gizmo686 5y ago

There are only 2^32 possible IP addresses. You can brute-force that on a personal laptop.

vgaldikas 5y ago
There's even less 'usable' ones, when you exclude private ranges etc...
- olliej 5y ago
  And if you further restrict to US service provider ranges
xvector 5y ago
If you use a hard hash function you cannot brute force that on a laptop - not even a tenth of that. You can, however, spin up compute instances to brute force it in a few days if you have $50k lying around.
- zeroimpl 5y ago
  What are the odds that a website will run a computationally hard hash function on every single HTTP request just so it can log something less sensitive than an IP address?
  - josefx 5y ago
    The website could cache the hash for an hour or two.
- xxs 5y ago
  >force that on a laptop They said: "personal computer", which could easily have 2x GPUs and 16+ cores. Heck, laptops nowadays can have a pretty good discrete GPU. Using password grade hash with a =nonce= is absolutely no way to be accomplished per each request. The nonce would have to be the same for multiple uses - hence NOT a 'nonce'. The sharing of the said non-nonce would require a form a replicated Map (or IP-sticky processing with a local map). It's rather convoluted solution for absolutely no benefit as it's still not hard to brute force. Storing it in such a way - slow hash + salt yields no benefits for debugging either, so I wonder why would you do so? Password hashes are useful for proving a match with an unknown plain text (while making it expensive to brute force) - so what would be the exact purpose of having non-nonce+IP?

uses 5y ago

Hm, I'm confused, usually the whole point of storing an IP is in case the visitor uses the platform to do something illegal, like a death threat. Without the original IP law enforcement can't subpoena the ISP, etc. But also as someone else said, if you use a nonce, and I think you mean salt, then it can be cracked nearly instantaneously anyways due to the small space of IPv4 (~4 billion).

ed25519FUUU 5y ago
Wouldn’t be relevant for a news website.
- LatteLazy 5y ago
  I wouldn't be so sure, lots of papers have comment sections and they're pretty rammed with death threats on any social issue sadly...
  - ed25519FUUU 5y ago
    Oh jeeze with comments enabled anything goes.
njharman 5y ago
IP is not a person identifier. They should not be allowed as evidence in criminal cases.
- gizmo686 5y ago
  There is more to investigating then gathering admissible evividence. For example, if you are talking with Alice and she says that she heard from Bob chat Charlie was in the office at a weird hour on the day in question investigators have gains nothing that they can admit as evidence [0]. However, there is nothing that anyone (defense or a third party) can do challenge this portion of the investigation other then keep it out of evidence, and the investigators are free to follow up with either Bob or Charlie to get something that would be admissible.

aneutron 5y ago

Sometimes there's a forensic purpose. For example, you want to know which servers exfiltrated your data and to which IP. Or for audit purposes (e.g. you might need to prove to some regulator no outside access was made, which is stupid but ...)

nullc 5y ago

There are only 2^32 IPv4 addresses, if you know the nonce you just try them all... no privacy provided. If you don't know the nonce, you can't match against other users-- so not useful for abuse. But I'm skeptical re: abuse uses. For commenters, sure-- you may need to store IPs to combat abuse. But for readers? At most you would need sampled data or in-memory counters (e.g. to catch high volume bots). Unfortunately, there really isn't any penalty for failing to minimize private data collection.

xvector 5y ago
If you use a difficult hash function that takes ~1 seconds to calculate then it would take over 120 years to iterate through the IPv4 address space. At the very least, this could cut down on dragnet surveillance
- 542458 5y ago
  Yes, but then I’m burning a second of compute time every time I want to log something. Also, by removing unlikely candidates (IPs owned by irrelevant entities or that are not US based) you could get the search range much much smaller, and with the FBIs budget you could probably compute it all in a few days even with a 1-second hash time.
  - deleted 5y ago
    [deleted]
- nullc 5y ago
  But then a single user clicking on links quickly would bring your webserver to its knees. So much for using those addresses to combat abuse... :) Plus the FBI could probably narrow their search to a few hundred thousand addresses (relevant ISPs, no unroutable/multicast/etc), then only use the list to confirm. Finally, if it takes 120 years on one core, it'll take 1.4 months on 1000 cores. I'm willing to be the FBI has access to more computing power than I do. ~100 CPU years isn't a particularly daunting amount of computing work, even for fairly low stakes research. That search would also decode all addresses in the logs, not just one targeted one...

GeneralMayhem 5y ago

Anti-spam/anti-abuse operations often look a lot like tracking. There's no point in knowing a single request's IP, or a single user's; to spot the patterns, you have to be able to join it with others.

8note 5y ago

There's some structure to the IP that can be somewhat useful for tracking identity. Its imperfect, but you'd expect definitely good folks to look a certain way