4 ms·

How about deleting the IP addresses within 48 hours like 1.1.1.1 and 8.8.8.8 do? https://developers.google.com/speed/public-dns/privacy https://developers.goog

• staticfloat 6y ago
  We do have a limited retention policy for the package server logs we keep (which include client IP addresses). It's not publicly stated anywhere right now, but one reason why we need to keep IP addresses is for abuse mitigation. We have been hit in the past by users that do things like download large (100MB+) files from our package cache servers multiple times a second for days on end. This is a particularly easy case to catch (since it easily pops to the top of any analysis you'd care to run, across any timespan) but there are more subtle forms that require a longer time window of analysis (e.g. users that download once per hour, all month) that would be lost in the noise without the ability to see what's going on. This comment is not meant to serve as an official policy, just pointing out one of the reasons why we can't delete IP addresses like 1.1.1.1 and 8.8.8.8 do; because the abuse vectors for a server that serves the community large resources is very different from that of a DNS server. Most of the "abuse" we see is not malicious in nature, but is instead users that have some kind of very poorly-configured autoinstaller on a cluster. In the case of a catastrophic issue like the one mentioned above, we null-routed the IP address, reached out to the abuse contact for that IP, and worked with the user to architect a better system. Everyone is happy now, and we can continue to provide a high quality service for the community without breaking the bank.
  • edw 6y ago
    How about hashing IPs? You could still see if someone were on your abuse list if abusers.contains(hashfn(req.addr)).
    • codedokode 6y ago
      Hash of IPv4 address can be easily reverted because there is a limited number of addresses.
    • KenoFischer 6y ago
      Doesn't help for two reasons 1) If the has has enough bits to be useful for blocking, it's trivial to reverse 2) Even if it did make the IPs anonymous, we want to be able to email the NOC at whoever is sending the abusive traffic, so they can go investigate
      • j88439h84 6y ago
        > we want to be able to email the NOC at whoever is sending the abusive traffic, so they can go investigate If you block their traffic with HTTP 429 Too Many Requests, they can email you instead.
        • KenoFischer 6y ago
          We prefer not to break researchers' workflow because the group next door misconfigured their server. Happens all the time. We only sinkhole IPs if the traffic is malicious or on track to exceed or budget.
• KenoFischer 6y ago
  I don't work on this particular thing, so I can't say precisely what the planned retention period is. I suspect 48 hours is too short, since people do take weekends ;). It'll probably become clear with experience what retention periods work. DNS servers are in a very unique position of course since they essentially get your browsing history.