5 ms·

I checked the posted privacy policy - and I have no idea what your system logs and for how long. There are a few sites I could use this for, but some of them w

I checked the posted privacy policy - and I have no idea what your system logs and for how long. There are a few sites I could use this for, but some of them would also end up sending private customer ips for lookups, and for that reason I just manually check the suspicious ones and manually don't look up the ones I know have passed the 'already a member and human not hacker' gate. I keep look for something I can just add lists to my server and check against that, but not going to spend thousands on it.

marvinblum 4y ago
Same here. I would like to integrate a simple blacklist into our system. Does anybody know of a good IP lists that contains known bots, data centers, and so on? Also from a performance perspective, I can't do hundreds of millions of requests a month over the wire. That just a waste.
- AndrewCopeland 4y ago
  IPDetective collects data data from bot nets, datacenters, tor nodes, proxies and vpns. Is that what you are looking for? Also IPDetective can be used just as a detection solution rather than a prevention solution. Sometimes I use it against my nginx access logs and see how much of my traffic could be from bots.
  - marvinblum 4y ago
    I think your service would be close to what I'm looking for. But can you handle 120+ million requests per month? I would rather integrate an open-source list into our open-source tool than using a proprietary service.
    - AndrewCopeland 4y ago
      I feel you. Maybe we could work something out.
AndrewCopeland 4y ago
Thank you very much for your input. I will look into having the privacy be more verbose around the system logs and for how long I store them. > I keep look for something I can just add lists to my server and check against that, but not going to spend thousands on it. What do you mean by this? Like having the ability to create your own deny/black list of ip addresses and then you can validate against it?
- toast0 4y ago
  >> I keep look for something I can just add lists to my server and check against that, but not going to spend thousands on it. > What do you mean by this? Like having the ability to create your own deny/black list of ip addresses and then you can validate against it? They want to download your database and run queries on it locally, rather than call your API. This way they don't share data with you and don't need to worry about your data practices.
  - AndrewCopeland 4y ago
    Ahhh makes sense. I could definitely offer that it's pretty large being 250 million IP addresses. I wonder what the best format would be? Csv? SQLite?
    - stevenicr 4y ago
      Currently to make things a little smaller I am putting many ip's in with cidr notation.. whichever format you would offer I could likely find ways to convert if needed. I am still researching what is the fastest / lightest on server resources for blocking ips (mainly from data centers lately been the main problems) - ip-block-file that is ready via cpanel's WHM is what I am using on one system, and iptables/uffirewall / fail2ban on another server.. I'm using wordpress firewall plugin to block some on another site - but I think it's lighter on server resources to block before the requests hit wordpress.. I wonder if it'd be lighter to pull the file from a separate server and return yay or nay - or if it's fine loading up 250 million ips via one of the firewall options already in place. How large of a file it 250 mill ips? (non-compressed) - I suppose available RAM could also play into these things.

marvinblum 4y ago

Same here. I would like to integrate a simple blacklist into our system. Does anybody know of a good IP lists that contains known bots, data centers, and so on? Also from a performance perspective, I can't do hundreds of millions of requests a month over the wire. That just a waste.

AndrewCopeland 4y ago
IPDetective collects data data from bot nets, datacenters, tor nodes, proxies and vpns. Is that what you are looking for? Also IPDetective can be used just as a detection solution rather than a prevention solution. Sometimes I use it against my nginx access logs and see how much of my traffic could be from bots.
- marvinblum 4y ago
  I think your service would be close to what I'm looking for. But can you handle 120+ million requests per month? I would rather integrate an open-source list into our open-source tool than using a proprietary service.
  - AndrewCopeland 4y ago
    I feel you. Maybe we could work something out.

AndrewCopeland 4y ago

Thank you very much for your input. I will look into having the privacy be more verbose around the system logs and for how long I store them. > I keep look for something I can just add lists to my server and check against that, but not going to spend thousands on it. What do you mean by this? Like having the ability to create your own deny/black list of ip addresses and then you can validate against it?

toast0 4y ago
>> I keep look for something I can just add lists to my server and check against that, but not going to spend thousands on it. > What do you mean by this? Like having the ability to create your own deny/black list of ip addresses and then you can validate against it? They want to download your database and run queries on it locally, rather than call your API. This way they don't share data with you and don't need to worry about your data practices.
- AndrewCopeland 4y ago
  Ahhh makes sense. I could definitely offer that it's pretty large being 250 million IP addresses. I wonder what the best format would be? Csv? SQLite?
  - stevenicr 4y ago
    Currently to make things a little smaller I am putting many ip's in with cidr notation.. whichever format you would offer I could likely find ways to convert if needed. I am still researching what is the fastest / lightest on server resources for blocking ips (mainly from data centers lately been the main problems) - ip-block-file that is ready via cpanel's WHM is what I am using on one system, and iptables/uffirewall / fail2ban on another server.. I'm using wordpress firewall plugin to block some on another site - but I think it's lighter on server resources to block before the requests hit wordpress.. I wonder if it'd be lighter to pull the file from a separate server and return yay or nay - or if it's fine loading up 250 million ips via one of the firewall options already in place. How large of a file it 250 mill ips? (non-compressed) - I suppose available RAM could also play into these things.