9 ms·

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the meth

First, I am big fan of your articles even before I joined IPinfo, where we provide IP geolocation data service. Our geolocation methodology expands on the methodology you described. We utilize some of the publicly available datasets that you are using. However, the core geolocation data comes from our ping-based operation. We ping an IP address from multiple servers across the world and identify the location of the IP address through a process called multilateration. Pinging an IP address from one server gives us one dimension of location information meaning that based on certain parameters the IP address could be in any place within a certain radius on the globe. Then as we ping that IP from our other servers, the location information becomes more precise. After enough pings, we have a very precise IP location information that almost reaches zip code level precision with a high degree of accuracy. Currently, we have more than 600 probe servers across the world and it is expanding. The publicly available information that you are referring to is sometimes not very reliable in providing IP location data as: - They are often stale and not frequently updated. - They are not precise enough to be generally useful. - They provide location context at an large IP range level or even at organization level scale. And last but not least, there is no verification process with these public datasets. With IPv4 trade and VPN services being more and more popular we have seen evidence that in some instances inaccurate information is being injected in these datasets. We are happy and grateful to anyone who submits IP location corrections to us but we do verify these correction submissions for that reason. From my experience with our probe network, I can definitely say that it is far easier and cheaper to buy a server in New York than in any country in the middle of Africa. Location of an IP address greatly influences the value it can provide. We have a free IP to Country ASN database that you can use in your project if you like. https://ipinfo.io/developers/ip-to-country-asn-database https://ipinfo.io/developers/ip-to-country-asn-database

TheClassic 3y ago
Your comment is extremely interesting and what I was hoping to learn from the article (without an existing source of information, how do we determine the location of an IP address). Thank you!
- reincoder 3y ago
  I really appreciate. Thank you. We are very transparent about our process. If you have any questions, you can always reach out to us. We have a simplified explanation of our probe network here: https://ipinfo.io/blog/probe-network-how-we-make-sure-our-data-is-accurate/ https://ipinfo.io/blog/probe-network-how-we-make-sure-our-da... The only update is the number of servers is like 600+ now. The probe network is growing extremely rapidly. Our IP geolocation process is quite complicated, and we have a team of data engineers, infrastructure engineers, and data scientists working on various aspects of it. Therefore, our approach is users can ask us questions, and we will try our best to answer them.
  - freedomben 3y ago
    Just wanted to let you know, it's this transparency that turned me into a customer! I love your company and service, but I hate your pricing. I work with a lot of small clients/apps that paying for usage would be a no-brainer, but the defined monthly price buckets don't make any economical sense at their scale. If you added a "pay as you go" tier that a small app could reasonably start by using dollars worth of API calls per month and grow from there, I'd be spreading your seed all over the place. I'm not saying this to rag on you, just trying to provide some constructive feedback as a thank you for your info sharing!
    - reincoder 3y ago
      Thank you very much; I really appreciate your feedback. This is not the first time I have heard this. The solution is to try to take as much advantage as you can from the free tier. # Check out the free IP databases https://ipinfo.io/products/free-ip-database https://ipinfo.io/products/free-ip-database The free databases come with commercial usage permission, and because they are databases, you can make unlimited lookups from them. The databases provide full accuracy and are updated daily. They are just a subset of our IP geolocation database that only provides IP to Country information. # Complement the database with the API service If you only want city-level information, switch to the API service. Use the database to look up IP-to-country information as many times as you want. However, use the API service only when necessary. Additionally, if you include a credit link to us, we will double your API limit to 100k/month. Visit https://ipinfo.io/contact/creditlink https://ipinfo.io/contact/creditlink. # Cache data All of our API libraries have native caching support. We strongly recommend that users reduce their number of requests by caching the response. I highly recommend you check out our libraries: https://github.com/ipinfo https://github.com/ipinfo --- The only challenge with the free IP databases is that you need to host the database somewhere to lookup the IP to Country information. Having an API service with nearly unlimited lookups for IP to Country information will be fantastic. If you know someone who has an IP to Country as API service please, let me know. We only require an attribution for using our database. If you have a similar service that is popular but don't want to maintain it let us know as well, we can takeover the site and host it ourselves with the IP to Country data.
chaps 3y ago
Not gonna lie, this creeps the heck out of me.
- giantrobot 3y ago
  You might want to unplug your router then. A conceit of being connected to a network is you're connected to the network. If you can see other nodes they can see you.
  - jiveturkey 3y ago
    Is this an accepted usage of the word `conceit`? I love the construction, and it does feel like it belongs, but I'm not finding this usage. https://www.wordnik.com/words/conceit https://www.wordnik.com/words/conceit has a bunch of meanings collected from various sources. I wonder if you meant `concession`. Also, it's a false dichotomy. One can use VPN or proxies, to limit exposure or to encapsulate it. Of course, you can't get perfect location privacy.
- fragmede 3y ago
  Your IP address is LEAKING!
- reincoder 3y ago
  Thousands of people live in a zip code, while hundreds and thousands of people live in a city. We are literally giving away that data for free through our API and database. The creepiness of IP geolocation is mostly a meme. IP geolocation is mainly used in cybersecurity and marketing analytics. There are many ways to geolocate someone. I once came across a project that could estimate the country a user is from based on their writing style and grammar mistakes. For example, American people sometimes use "should of" instead of "should have". Knowing the geolocation of an IP address isn't super creepy. It's just how things work on the internet.
  - chaps 3y ago
    And you're literally advertising this project as being helpful for targeted ads. So it's pretty clear from the get go that what you consider creepy isn't what I consider creepy. And having done enough reidentification work to scare myself, "thousands of people" might as well be a couple dozen or less. I get why you're defensive and why you think it's not creepy, but calling it a "meme" is insultingingly dismissive. Just because it's "how things work on the internet" doesn't make its mass collection right. Under the same logic, any side channel attack is just "how it works", and its abuse warrants no ethical question.
voltagex_ 3y ago
Can your probes be identified and blocked?
- kube-system 3y ago
  iptables -A INPUT -p icmp -j DROP
  - chaps 3y ago
    This isn't helpful. The comment was specifically asking about the probes, not ICMP traffic.
    - kube-system 3y ago
      Anybody can do this same thing, if you're worried about this, you probably don't want inbound ICMP.
      - chaps 3y ago
        Cool. Thanks. But let's say I do.
        kube-system 3y ago
        Then there's nothing you can do. If you respond to pings, then others can take note of the responses you send.
        chaps 3y ago
        You're missing the point that the question is effectively asking for a list of hosts that they can block. Edit: they provided a method: https://news.ycombinator.com/item?id=37510063 https://news.ycombinator.com/item?id=37510063
        kube-system 3y ago
matsur 3y ago
ICMP response time not useful for “locating” an anycasted address, some of which have logical location associated with them. See https://blog.cloudflare.com/icloud-private-relay/ https://blog.cloudflare.com/icloud-private-relay/ for an example
- cuu508 3y ago
  Well, at least you can detect it is an anycast address, and mark it as such.
incolumitas 3y ago
Big fan of what articles? On https://incolumitas.com/ https://incolumitas.com/ or on https://ipapi.is/ https://ipapi.is/? Great idea with latency triangulation, I used latency information for a lot of things, especially VPN and Proxy detection. But I didn't assume you can obtain that accurate location. I am honestly impressed. But latency triangulation with 600 servers gives some very good approximation. Nice man! Some questions: - ICMP traffic is penalised/degraded by some ISP's. How do you deal with that? - In order to geolocate every IPv4 address, you need to constantly ping billions of IPv4's, how do you do that? You only ping an arbitrary IP of each allocated inetnum/NetRange? - Most IP addresses do not respond to ICMP packets. Only some servers do. How do you deal with that? Do you find the router in front of the target IP and you geolocate the closest router to the target IP (traceroute)?
- withinboredom 3y ago
  I'm very curious why you'd do VPN/proxy detection... But at a previous company I worked at that ran a very large chunk of the internet, we did indexing of nearly the entire internet (even large portions of the dark web) approximately every two weeks. There were about 500 servers doing that non-stop. So, I think it is relatively reasonable if you have 600 servers to do that.
  - meroje 3y ago
    In the business of media streaming, rightholder will require that you check for vpn and proxies in addition to countries when deciding if a given viewer will be able to stream a given media.
    - withinboredom 3y ago
      Does that actually work? That could explain an issue with a particular streaming service I use. There are currently some ongoing routing issues in BGP land and my ISP. When trying to stream, it says I’m using a proxy, so due to the incredible route my packets are taking, that might be it. What’s funny is that the only way to watch this service is to use a vpn right now.
chankstein38 3y ago
That's pretty neat! You're basically using ping triangulation!
- sib 3y ago
  Trilateration (same technique as used for mobile network location - in addition to the GPS on the phone)
Daviey 3y ago
Would you consider no-signup inspection of the data you hold on the requesters IP address? I would love to see what you have on MY IP address, and if sufficiency accurate it feels that it would be a good incentive to sign up to use commerically. It feels like it couldn't be abused by 'freeloaders', because i'd guess their use-case is viewing other peoples.
- kam 3y ago
  This is literally the most prominent thing on the https://ipinfo.io https://ipinfo.io home page.
  - Daviey 3y ago
    That's embarrassing for me... I thought that was a static image of an example. And I did look through the site looking for a search. Oops.
  - qingcharles 3y ago
    Huh, that's cool. It got my home IP about 15 miles from where I am, but still not bad. Wait - how does this work for cell IPs? A lot of cellphone v4 IPs are now shared between hundreds or thousands of devices, right?
    - reincoder 3y ago
      I work there, and I am supposed to know these things, but I don't exactly :/ It probably has something to do with important routers. What tags do we show when you visit the IP data page? The IP data page can be accessed by visiting ipinfo.io/<IP_address>. We use the generic term "data experts," but it actually consists of about 2 dozen engineers, including data engineers, data scientists, infrastructure engineers, backend engineers, and a great technical CEO working on all that. All those folks have gone on a boating trip off the coast of Spain for a retreat.....except for me. I will ask them and try to circle back with some answers.
- reincoder 3y ago
  We have a very open approach to our data. In fact, our website is extremely accessible. It is quite useful for researching IP addresses and does not require signing up. The data is largely available to view on the website. Although we display all IP address meta data on the home page, if you intend to use our website frequently, I recommend utilizing the IP data pages. You can enter IP addresses on the right side to look up information here: https://ipinfo.io/what-is-my-ip https://ipinfo.io/what-is-my-ip Additionally, we offer some enjoyable tools that you can use here: https://ipinfo.io/tools https://ipinfo.io/tools The CLI tool is particularly entertaining. You can also use our API service without signing up, with a limit of 1000 requests per day. If you do choose to sign up for a free account, you will receive 50,000 requests per month, free IP databases, a bulk lookup feature, and more.
detourdog 3y ago
I just noticed that my wifes iphone uses the same mycingular ip address while driving accross 3 states over 5 hours.l while checking mail.
- inemesitaffia 3y ago
  There's several options/techniques for doing it. But just imagine you have a permanent zero overhead VPN. I don't know if that provider terminates long running calls, but the calls would stay up too regardless of tower.
  - detourdog 3y ago
    Yes, I’m sure it is iOS anti-tracking and directly related to why firewall apps inside SIP my not know what is going on.
    - Vendan 3y ago
      More likely to be just standard Mobile IP https://en.wikipedia.org/wiki/Mobile_IP https://en.wikipedia.org/wiki/Mobile_IP. Fairly standard stuff, can cause some false positives around traveling (I've seen people get freaked out about stuff like "This person just logged in from their home state and then less then an hour later logged in from France!" when it was just mobile IP treating their phone as still in the US while they were in France on a trip, but their laptop connected over normal internet was seen as coming from France)
      - detourdog 3y ago
        this was a consistent ip address nothing to do with location and nobody was freaked out.
welder 3y ago
Great comment. I'm a big fan and customer of IPinfo, using your API in our login notification emails to say "You just logged in from Berlin, Germany. If this wasn't you click here." To provide country data for customers in their audit logs. And for anti-spam and fraud detection.
- reincoder 3y ago
  I appreciate it, sir! If you have any questions or feedback, please let us know. The challenge of being a data provider is that you can use our data in a million ways, and we don't have coverage of all. So, when you come up with questions or ideas, we can help you better. As you mentioned, audit logs. I highly recommend you look into the ASN field. The ASN identifies an organization that owns a block of IP addresses. In my experience, I have found that the combination of ASN+Country is the most valuable information you can use in spam and fraud detection. You can fake the IP geolocation information with a VPN. However, it is not as easy to fake the ASN information of the IP address. So, when you use a combination of country + ASN, you can have a robust cybersecurity system.
  - welder 3y ago
    Can you explain more how to use ASN to detect fraud and how it's different from the country detected for the IP? I thought ASN was derived from the IP, basically the route to that IP? Here's the ipinfo response for an IP used by a recent fraud signup attempt. The asn field matches country. { "city": "Mumbai", "connection": { "asn": 24560, "isp": "Bharti Airtel Ltd." }, "continent_code": "AS", "continent_name": "Asia", "country_code": "IN", "country_name": "India", "currency": { "code": "INR", "name": "Indian Rupee", "plural": "Indian rupees", "symbol": "Rs", "symbol_native": "\u099f\u0995\u09be" }, "ip": "2401:4900:1f38:7402:5569:2e45:3bb:9c0d", "latitude": 19.076000213623047, "location": { "calling_code": "91", "capital": "New Delhi", "country_flag": "https://assets.ipstack.com/flags/in.svg", "country_flag_emoji": "\ud83c\uddee\ud83c\uddf3", "country_flag_emoji_unicode": "U+1F1EE U+1F1F3", "geoname_id": 1275339, "is_eu": false, "languages": [ { "code": "hi", "name": "Hindi", "native": "\u0939\u093f\u0928\u094d\u0926\u0940" }, { "code": "en", "name": "English", "native": "English" } ] }, "longitude": 72.87770080566406, "region_code": "MH", "region_name": "Maharashtra", "time_zone": { "code": "IST", "current_time": "2023-09-15T10:52:42+05:30", "gmt_offset": 19800, "id": "Asia/Kolkata", "is_daylight_saving": false }, "type": "ipv6", "zip": "400203" } Here's the response from ipinfo.io which includes privacy fields. It's technically a proxy but might be hard to detect because it's probably a crowdsourced/botnet proxy not a public one. We don't pay for { "ip": "2401:4900:1f38:7402:5569:2e45:3bb:9c0d", "city": "Najafgarh", "region": "Delhi", "country": "IN", "loc": "28.6114,77.2982", "org": "AS24560 Bharti Airtel Ltd., Telemedia Services", "postal": "110097", "timezone": "Asia/Kolkata", "asn": { "asn": "AS24560", "name": "Bharti Airtel Ltd., Telemedia Services", "domain": "airtel.com", "route": "2401:4900:1f38::/48", "type": "isp" }, "company": { "name": "ABTS (Karnataka),", "domain": "airtel.com", "type": "isp" }, "privacy": { "vpn": false, "proxy": false, "tor": false, "relay": false, "hosting": false, "service": "" }, "abuse": { "address": "Bharti Airtel Ltd., ISP Division - Transport Network Group, 234 , Okhla Industrial Estate,, Phase III, New Delhi-110020, INDIA", "country": "IN", "email": "ip.misuse@airtel.com", "name": "ABUSE BHARTIIN", "network": "2401:4900:1f30::/44", "phone": "+000000000" } } EDIT: Oops, I confused ipinfo with ipstack. I'm actually using ipstack. Their security field also doesn't detect this IP as a proxy, which is why we only pay for Professional (no security field). { "ip": "2401:4900:1f38:7402:5569:2e45:3bb:9c0d", "type": "ipv6", "continent_code": "AS", "continent_name": "Asia", "country_code": "IN", "country_name": "India", "region_code": "MH", "region_name": "Maharashtra", "city": "Mumbai", "zip": "400203", "latitude": 19.076000213623047, "longitude": 72.87770080566406, "location": { "geoname_id": 1275339, "capital": "New Delhi", "languages": [ { "code": "hi", "name": "Hindi", "native": "\u0939\u093f\u0928\u094d\u0926\u0940" }, { "code": "en", "name": "English", "native": "English" } ], "country_flag": "https://assets.ipstack.com/flags/in.svg", "country_flag_emoji": "\ud83c\uddee\ud83c\uddf3", "country_flag_emoji_unicode": "U+1F1EE U+1F1F3", "calling_code": "91", "is_eu": false }, "time_zone": { "id": "Asia/Kolkata", "current_time": "2023-09-15T12:27:08+05:30", "gmt_offset": 19800, "code": "IST", "is_daylight_saving": false }, "currency": { "code": "INR", "name": "Indian Rupee", "plural": "Indian rupees", "symbol": "Rs", "symbol_native": "\u099f\u0995\u09be" }, "connection": { "asn": 24560, "isp": "Bharti Airtel Ltd." }, "security": { "is_proxy": false, "proxy_type": null, "is_crawler": false, "crawler_name": null, "crawler_type": null, "is_tor": false, "threat_level": "low", "threat_types": null } }
caribdude 3y ago
[dead]
EwanToo 3y ago
Have you considered making your database available for download as Parquet format so people could just copy the file to S3, Google Cloud, etc, and query it immediately with various tools? I know it can be done with CSV but it's not as smooth.
- reincoder 3y ago
  Thank you for the feature request. We usually just send users the documentation of ingesting the data in CSV or NDJSON format (Newline Delimited JSON). We don't actually get many requests for data downloads in Parquet format. I think we have a few customers where we deliver the data in parquet format directly to their cloud storage bucket. But keep an eye out for our emails if we announce the parquet data downloads. I will talk with the folks about this. BUT, there are some good news. At least for the free database, we deliver the data directly to data warehouse platforms. Not even storage buckets. And we supply a good amount of documentation. We have the free database in Snowflake, GCP, Kaggle, and Splitgraph, and we are working on a few more deals. For the free database, atleast, we are working on better things than parquet. Like literally one-click solution to bring the IP data to your data warehouse. Kaggle: https://www.kaggle.com/code/ipinfo/ipinfo-ip-to-country-asn-demo-notebook-for-kaggle https://www.kaggle.com/code/ipinfo/ipinfo-ip-to-country-asn-... Snowflake: https://app.snowflake.com/marketplace/listing/GZSTZSHKQ4QY/ipinfo-ipinfo-free-country-asn-ip-address-data https://app.snowflake.com/marketplace/listing/GZSTZSHKQ4QY/i... If you want to use our free IP database on Google Cloud or BigQuery, please send us an email (support@ipinfo.io) and mention that the DevRel sent you from HN. I can easily set you up with the free IP database in GCP/BQ.
theogravity 3y ago
How does that work with edge servers that use anycast to assume the same IP across different regions?
- SnorkelTan 3y ago
  Aren’t any cast addresses a specific subset of ips and thus knowable? Iirc, each autonomous system is allocated anycast ip space?
  - sgjohnson 3y ago
    No. Any prefix can be anycasted.
tmaly 3y ago
Are there any historical sources for geo ip info?
- reincoder 3y ago
  We don't have any free data for that. We have historical data that we sell as part of our custom enterprise deals. Historical data requests are rare, though. A time series IP database requires a substantial amount of storage and computational cost to query, as I imagine. The city level geolocation data we have is ~1.5 gb in size. IP range data is complicated to query efficiently as you need to understand data platform settings and good amount of computer network math and computer science stuff. Adding a layer of time series complexities on top of that, makes this process quite difficult. To give you some context of how IP metadata lookups work, you can check out this article https://ipinfo.io/blog/ip-address-data-in-snowflake/ https://ipinfo.io/blog/ip-address-data-in-snowflake/ Even if you keep all your database in a binary format, the computational cost is still non-negligible.
BonoboIO 3y ago
Hi, cool idea with the geolocation via latency. But I encountered 2 things using ipinfo: Hetzner Server that are in Germany in a fixed location that never moved are sometimes located in another country, for me it was once s Server placed into Moscow and once in South America. How does this happen?
- reincoder 3y ago
  If you can give me some information about the IP address or the IP range, I can take a closer look. I guess it is because of IPv4 trading or IP address shuffling. As far as I know, Hertzer, like many hosting companies, is buying IPv4 addresses around the world. Here is an article on the IPv4 trades: https://tech.marksblogg.com/ipinfo-free-ip-address-location-database.html https://tech.marksblogg.com/ipinfo-free-ip-address-location-... When a company buys an IP address block or relocates an IP block from one of its data centers to another, the location of those IP addresses changes. If your IP address is static, but we have made an error in geolocation, I would love to take a closer look. You can email our support (support@ipinfo.io) and send a link to the comment. We can discuss it further from there.
yrro 3y ago
hm, ipinfo.io tells me that I'm using a VPN even though I'm not...
- reincoder 3y ago
  Our VPN recognition is behavior-based. So, there is probably a chance that the IP address you are using is showing some of those behavior patterns. A behavior pattern could be that your IP address is being shuffled around random locations that go beyond the normal location shuffling of an ISP connection. Also, if your IP range is listed in some public datasets that belong to a VPN service, we could recognize your IP as a VPN. Please reach out to our support and let us know about this. Thanks
  - yrro 3y ago
    Will do thanks

TheClassic 3y ago

Your comment is extremely interesting and what I was hoping to learn from the article (without an existing source of information, how do we determine the location of an IP address). Thank you!

reincoder 3y ago
I really appreciate. Thank you. We are very transparent about our process. If you have any questions, you can always reach out to us. We have a simplified explanation of our probe network here: https://ipinfo.io/blog/probe-network-how-we-make-sure-our-data-is-accurate/ https://ipinfo.io/blog/probe-network-how-we-make-sure-our-da... The only update is the number of servers is like 600+ now. The probe network is growing extremely rapidly. Our IP geolocation process is quite complicated, and we have a team of data engineers, infrastructure engineers, and data scientists working on various aspects of it. Therefore, our approach is users can ask us questions, and we will try our best to answer them.
- freedomben 3y ago
  Just wanted to let you know, it's this transparency that turned me into a customer! I love your company and service, but I hate your pricing. I work with a lot of small clients/apps that paying for usage would be a no-brainer, but the defined monthly price buckets don't make any economical sense at their scale. If you added a "pay as you go" tier that a small app could reasonably start by using dollars worth of API calls per month and grow from there, I'd be spreading your seed all over the place. I'm not saying this to rag on you, just trying to provide some constructive feedback as a thank you for your info sharing!
  - reincoder 3y ago
    Thank you very much; I really appreciate your feedback. This is not the first time I have heard this. The solution is to try to take as much advantage as you can from the free tier. # Check out the free IP databases https://ipinfo.io/products/free-ip-database https://ipinfo.io/products/free-ip-database The free databases come with commercial usage permission, and because they are databases, you can make unlimited lookups from them. The databases provide full accuracy and are updated daily. They are just a subset of our IP geolocation database that only provides IP to Country information. # Complement the database with the API service If you only want city-level information, switch to the API service. Use the database to look up IP-to-country information as many times as you want. However, use the API service only when necessary. Additionally, if you include a credit link to us, we will double your API limit to 100k/month. Visit https://ipinfo.io/contact/creditlink https://ipinfo.io/contact/creditlink. # Cache data All of our API libraries have native caching support. We strongly recommend that users reduce their number of requests by caching the response. I highly recommend you check out our libraries: https://github.com/ipinfo https://github.com/ipinfo --- The only challenge with the free IP databases is that you need to host the database somewhere to lookup the IP to Country information. Having an API service with nearly unlimited lookups for IP to Country information will be fantastic. If you know someone who has an IP to Country as API service please, let me know. We only require an attribution for using our database. If you have a similar service that is popular but don't want to maintain it let us know as well, we can takeover the site and host it ourselves with the IP to Country data.

chaps 3y ago

Not gonna lie, this creeps the heck out of me.

giantrobot 3y ago
You might want to unplug your router then. A conceit of being connected to a network is you're connected to the network. If you can see other nodes they can see you.
- jiveturkey 3y ago
  Is this an accepted usage of the word `conceit`? I love the construction, and it does feel like it belongs, but I'm not finding this usage. https://www.wordnik.com/words/conceit https://www.wordnik.com/words/conceit has a bunch of meanings collected from various sources. I wonder if you meant `concession`. Also, it's a false dichotomy. One can use VPN or proxies, to limit exposure or to encapsulate it. Of course, you can't get perfect location privacy.
fragmede 3y ago
Your IP address is LEAKING!
reincoder 3y ago
Thousands of people live in a zip code, while hundreds and thousands of people live in a city. We are literally giving away that data for free through our API and database. The creepiness of IP geolocation is mostly a meme. IP geolocation is mainly used in cybersecurity and marketing analytics. There are many ways to geolocate someone. I once came across a project that could estimate the country a user is from based on their writing style and grammar mistakes. For example, American people sometimes use "should of" instead of "should have". Knowing the geolocation of an IP address isn't super creepy. It's just how things work on the internet.
- chaps 3y ago
  And you're literally advertising this project as being helpful for targeted ads. So it's pretty clear from the get go that what you consider creepy isn't what I consider creepy. And having done enough reidentification work to scare myself, "thousands of people" might as well be a couple dozen or less. I get why you're defensive and why you think it's not creepy, but calling it a "meme" is insultingingly dismissive. Just because it's "how things work on the internet" doesn't make its mass collection right. Under the same logic, any side channel attack is just "how it works", and its abuse warrants no ethical question.

voltagex_ 3y ago

Can your probes be identified and blocked?

kube-system 3y ago
iptables -A INPUT -p icmp -j DROP
- chaps 3y ago
  This isn't helpful. The comment was specifically asking about the probes, not ICMP traffic.
  - kube-system 3y ago
    Anybody can do this same thing, if you're worried about this, you probably don't want inbound ICMP.
    - chaps 3y ago
      Cool. Thanks. But let's say I do.
      - kube-system 3y ago
        Then there's nothing you can do. If you respond to pings, then others can take note of the responses you send.
        chaps 3y ago
        You're missing the point that the question is effectively asking for a list of hosts that they can block. Edit: they provided a method: https://news.ycombinator.com/item?id=37510063 https://news.ycombinator.com/item?id=37510063
        kube-system 3y ago

matsur 3y ago

ICMP response time not useful for “locating” an anycasted address, some of which have logical location associated with them. See https://blog.cloudflare.com/icloud-private-relay/ https://blog.cloudflare.com/icloud-private-relay/ for an example

cuu508 3y ago
Well, at least you can detect it is an anycast address, and mark it as such.

incolumitas 3y ago

Big fan of what articles? On https://incolumitas.com/ https://incolumitas.com/ or on https://ipapi.is/ https://ipapi.is/? Great idea with latency triangulation, I used latency information for a lot of things, especially VPN and Proxy detection. But I didn't assume you can obtain that accurate location. I am honestly impressed. But latency triangulation with 600 servers gives some very good approximation. Nice man! Some questions: - ICMP traffic is penalised/degraded by some ISP's. How do you deal with that? - In order to geolocate every IPv4 address, you need to constantly ping billions of IPv4's, how do you do that? You only ping an arbitrary IP of each allocated inetnum/NetRange? - Most IP addresses do not respond to ICMP packets. Only some servers do. How do you deal with that? Do you find the router in front of the target IP and you geolocate the closest router to the target IP (traceroute)?

withinboredom 3y ago
I'm very curious why you'd do VPN/proxy detection... But at a previous company I worked at that ran a very large chunk of the internet, we did indexing of nearly the entire internet (even large portions of the dark web) approximately every two weeks. There were about 500 servers doing that non-stop. So, I think it is relatively reasonable if you have 600 servers to do that.
- meroje 3y ago
  In the business of media streaming, rightholder will require that you check for vpn and proxies in addition to countries when deciding if a given viewer will be able to stream a given media.
  - withinboredom 3y ago
    Does that actually work? That could explain an issue with a particular streaming service I use. There are currently some ongoing routing issues in BGP land and my ISP. When trying to stream, it says I’m using a proxy, so due to the incredible route my packets are taking, that might be it. What’s funny is that the only way to watch this service is to use a vpn right now.

chankstein38 3y ago

That's pretty neat! You're basically using ping triangulation!

sib 3y ago
Trilateration (same technique as used for mobile network location - in addition to the GPS on the phone)

Daviey 3y ago

Would you consider no-signup inspection of the data you hold on the requesters IP address? I would love to see what you have on MY IP address, and if sufficiency accurate it feels that it would be a good incentive to sign up to use commerically. It feels like it couldn't be abused by 'freeloaders', because i'd guess their use-case is viewing other peoples.

kam 3y ago
This is literally the most prominent thing on the https://ipinfo.io https://ipinfo.io home page.
- Daviey 3y ago
  That's embarrassing for me... I thought that was a static image of an example. And I did look through the site looking for a search. Oops.
- qingcharles 3y ago
  Huh, that's cool. It got my home IP about 15 miles from where I am, but still not bad. Wait - how does this work for cell IPs? A lot of cellphone v4 IPs are now shared between hundreds or thousands of devices, right?
  - reincoder 3y ago
    I work there, and I am supposed to know these things, but I don't exactly :/ It probably has something to do with important routers. What tags do we show when you visit the IP data page? The IP data page can be accessed by visiting ipinfo.io/<IP_address>. We use the generic term "data experts," but it actually consists of about 2 dozen engineers, including data engineers, data scientists, infrastructure engineers, backend engineers, and a great technical CEO working on all that. All those folks have gone on a boating trip off the coast of Spain for a retreat.....except for me. I will ask them and try to circle back with some answers.
reincoder 3y ago
We have a very open approach to our data. In fact, our website is extremely accessible. It is quite useful for researching IP addresses and does not require signing up. The data is largely available to view on the website. Although we display all IP address meta data on the home page, if you intend to use our website frequently, I recommend utilizing the IP data pages. You can enter IP addresses on the right side to look up information here: https://ipinfo.io/what-is-my-ip https://ipinfo.io/what-is-my-ip Additionally, we offer some enjoyable tools that you can use here: https://ipinfo.io/tools https://ipinfo.io/tools The CLI tool is particularly entertaining. You can also use our API service without signing up, with a limit of 1000 requests per day. If you do choose to sign up for a free account, you will receive 50,000 requests per month, free IP databases, a bulk lookup feature, and more.

detourdog 3y ago

I just noticed that my wifes iphone uses the same mycingular ip address while driving accross 3 states over 5 hours.l while checking mail.

inemesitaffia 3y ago
There's several options/techniques for doing it. But just imagine you have a permanent zero overhead VPN. I don't know if that provider terminates long running calls, but the calls would stay up too regardless of tower.
- detourdog 3y ago
  Yes, I’m sure it is iOS anti-tracking and directly related to why firewall apps inside SIP my not know what is going on.
  - Vendan 3y ago
    More likely to be just standard Mobile IP https://en.wikipedia.org/wiki/Mobile_IP https://en.wikipedia.org/wiki/Mobile_IP. Fairly standard stuff, can cause some false positives around traveling (I've seen people get freaked out about stuff like "This person just logged in from their home state and then less then an hour later logged in from France!" when it was just mobile IP treating their phone as still in the US while they were in France on a trip, but their laptop connected over normal internet was seen as coming from France)
    - detourdog 3y ago
      this was a consistent ip address nothing to do with location and nobody was freaked out.

welder 3y ago

Great comment. I'm a big fan and customer of IPinfo, using your API in our login notification emails to say "You just logged in from Berlin, Germany. If this wasn't you click here." To provide country data for customers in their audit logs. And for anti-spam and fraud detection.

reincoder 3y ago
I appreciate it, sir! If you have any questions or feedback, please let us know. The challenge of being a data provider is that you can use our data in a million ways, and we don't have coverage of all. So, when you come up with questions or ideas, we can help you better. As you mentioned, audit logs. I highly recommend you look into the ASN field. The ASN identifies an organization that owns a block of IP addresses. In my experience, I have found that the combination of ASN+Country is the most valuable information you can use in spam and fraud detection. You can fake the IP geolocation information with a VPN. However, it is not as easy to fake the ASN information of the IP address. So, when you use a combination of country + ASN, you can have a robust cybersecurity system.
- welder 3y ago
  Can you explain more how to use ASN to detect fraud and how it's different from the country detected for the IP? I thought ASN was derived from the IP, basically the route to that IP? Here's the ipinfo response for an IP used by a recent fraud signup attempt. The asn field matches country. { "city": "Mumbai", "connection": { "asn": 24560, "isp": "Bharti Airtel Ltd." }, "continent_code": "AS", "continent_name": "Asia", "country_code": "IN", "country_name": "India", "currency": { "code": "INR", "name": "Indian Rupee", "plural": "Indian rupees", "symbol": "Rs", "symbol_native": "\u099f\u0995\u09be" }, "ip": "2401:4900:1f38:7402:5569:2e45:3bb:9c0d", "latitude": 19.076000213623047, "location": { "calling_code": "91", "capital": "New Delhi", "country_flag": "https://assets.ipstack.com/flags/in.svg", "country_flag_emoji": "\ud83c\uddee\ud83c\uddf3", "country_flag_emoji_unicode": "U+1F1EE U+1F1F3", "geoname_id": 1275339, "is_eu": false, "languages": [ { "code": "hi", "name": "Hindi", "native": "\u0939\u093f\u0928\u094d\u0926\u0940" }, { "code": "en", "name": "English", "native": "English" } ] }, "longitude": 72.87770080566406, "region_code": "MH", "region_name": "Maharashtra", "time_zone": { "code": "IST", "current_time": "2023-09-15T10:52:42+05:30", "gmt_offset": 19800, "id": "Asia/Kolkata", "is_daylight_saving": false }, "type": "ipv6", "zip": "400203" } Here's the response from ipinfo.io which includes privacy fields. It's technically a proxy but might be hard to detect because it's probably a crowdsourced/botnet proxy not a public one. We don't pay for { "ip": "2401:4900:1f38:7402:5569:2e45:3bb:9c0d", "city": "Najafgarh", "region": "Delhi", "country": "IN", "loc": "28.6114,77.2982", "org": "AS24560 Bharti Airtel Ltd., Telemedia Services", "postal": "110097", "timezone": "Asia/Kolkata", "asn": { "asn": "AS24560", "name": "Bharti Airtel Ltd., Telemedia Services", "domain": "airtel.com", "route": "2401:4900:1f38::/48", "type": "isp" }, "company": { "name": "ABTS (Karnataka),", "domain": "airtel.com", "type": "isp" }, "privacy": { "vpn": false, "proxy": false, "tor": false, "relay": false, "hosting": false, "service": "" }, "abuse": { "address": "Bharti Airtel Ltd., ISP Division - Transport Network Group, 234 , Okhla Industrial Estate,, Phase III, New Delhi-110020, INDIA", "country": "IN", "email": "ip.misuse@airtel.com", "name": "ABUSE BHARTIIN", "network": "2401:4900:1f30::/44", "phone": "+000000000" } } EDIT: Oops, I confused ipinfo with ipstack. I'm actually using ipstack. Their security field also doesn't detect this IP as a proxy, which is why we only pay for Professional (no security field). { "ip": "2401:4900:1f38:7402:5569:2e45:3bb:9c0d", "type": "ipv6", "continent_code": "AS", "continent_name": "Asia", "country_code": "IN", "country_name": "India", "region_code": "MH", "region_name": "Maharashtra", "city": "Mumbai", "zip": "400203", "latitude": 19.076000213623047, "longitude": 72.87770080566406, "location": { "geoname_id": 1275339, "capital": "New Delhi", "languages": [ { "code": "hi", "name": "Hindi", "native": "\u0939\u093f\u0928\u094d\u0926\u0940" }, { "code": "en", "name": "English", "native": "English" } ], "country_flag": "https://assets.ipstack.com/flags/in.svg", "country_flag_emoji": "\ud83c\uddee\ud83c\uddf3", "country_flag_emoji_unicode": "U+1F1EE U+1F1F3", "calling_code": "91", "is_eu": false }, "time_zone": { "id": "Asia/Kolkata", "current_time": "2023-09-15T12:27:08+05:30", "gmt_offset": 19800, "code": "IST", "is_daylight_saving": false }, "currency": { "code": "INR", "name": "Indian Rupee", "plural": "Indian rupees", "symbol": "Rs", "symbol_native": "\u099f\u0995\u09be" }, "connection": { "asn": 24560, "isp": "Bharti Airtel Ltd." }, "security": { "is_proxy": false, "proxy_type": null, "is_crawler": false, "crawler_name": null, "crawler_type": null, "is_tor": false, "threat_level": "low", "threat_types": null } }

caribdude 3y ago

[dead]

EwanToo 3y ago

Have you considered making your database available for download as Parquet format so people could just copy the file to S3, Google Cloud, etc, and query it immediately with various tools? I know it can be done with CSV but it's not as smooth.

reincoder 3y ago
Thank you for the feature request. We usually just send users the documentation of ingesting the data in CSV or NDJSON format (Newline Delimited JSON). We don't actually get many requests for data downloads in Parquet format. I think we have a few customers where we deliver the data in parquet format directly to their cloud storage bucket. But keep an eye out for our emails if we announce the parquet data downloads. I will talk with the folks about this. BUT, there are some good news. At least for the free database, we deliver the data directly to data warehouse platforms. Not even storage buckets. And we supply a good amount of documentation. We have the free database in Snowflake, GCP, Kaggle, and Splitgraph, and we are working on a few more deals. For the free database, atleast, we are working on better things than parquet. Like literally one-click solution to bring the IP data to your data warehouse. Kaggle: https://www.kaggle.com/code/ipinfo/ipinfo-ip-to-country-asn-demo-notebook-for-kaggle https://www.kaggle.com/code/ipinfo/ipinfo-ip-to-country-asn-... Snowflake: https://app.snowflake.com/marketplace/listing/GZSTZSHKQ4QY/ipinfo-ipinfo-free-country-asn-ip-address-data https://app.snowflake.com/marketplace/listing/GZSTZSHKQ4QY/i... If you want to use our free IP database on Google Cloud or BigQuery, please send us an email (support@ipinfo.io) and mention that the DevRel sent you from HN. I can easily set you up with the free IP database in GCP/BQ.

theogravity 3y ago

How does that work with edge servers that use anycast to assume the same IP across different regions?

SnorkelTan 3y ago
Aren’t any cast addresses a specific subset of ips and thus knowable? Iirc, each autonomous system is allocated anycast ip space?
- sgjohnson 3y ago
  No. Any prefix can be anycasted.

tmaly 3y ago

Are there any historical sources for geo ip info?

reincoder 3y ago
We don't have any free data for that. We have historical data that we sell as part of our custom enterprise deals. Historical data requests are rare, though. A time series IP database requires a substantial amount of storage and computational cost to query, as I imagine. The city level geolocation data we have is ~1.5 gb in size. IP range data is complicated to query efficiently as you need to understand data platform settings and good amount of computer network math and computer science stuff. Adding a layer of time series complexities on top of that, makes this process quite difficult. To give you some context of how IP metadata lookups work, you can check out this article https://ipinfo.io/blog/ip-address-data-in-snowflake/ https://ipinfo.io/blog/ip-address-data-in-snowflake/ Even if you keep all your database in a binary format, the computational cost is still non-negligible.

BonoboIO 3y ago

Hi, cool idea with the geolocation via latency. But I encountered 2 things using ipinfo: Hetzner Server that are in Germany in a fixed location that never moved are sometimes located in another country, for me it was once s Server placed into Moscow and once in South America. How does this happen?

reincoder 3y ago
If you can give me some information about the IP address or the IP range, I can take a closer look. I guess it is because of IPv4 trading or IP address shuffling. As far as I know, Hertzer, like many hosting companies, is buying IPv4 addresses around the world. Here is an article on the IPv4 trades: https://tech.marksblogg.com/ipinfo-free-ip-address-location-database.html https://tech.marksblogg.com/ipinfo-free-ip-address-location-... When a company buys an IP address block or relocates an IP block from one of its data centers to another, the location of those IP addresses changes. If your IP address is static, but we have made an error in geolocation, I would love to take a closer look. You can email our support (support@ipinfo.io) and send a link to the comment. We can discuss it further from there.

yrro 3y ago

hm, ipinfo.io tells me that I'm using a VPN even though I'm not...

reincoder 3y ago
Our VPN recognition is behavior-based. So, there is probably a chance that the IP address you are using is showing some of those behavior patterns. A behavior pattern could be that your IP address is being shuffled around random locations that go beyond the normal location shuffling of an ISP connection. Also, if your IP range is listed in some public datasets that belong to a VPN service, we could recognize your IP as a VPN. Please reach out to our support and let us know about this. Thanks
- yrro 3y ago
  Will do thanks