5 ms·

Is there even such a thing as a "Mississippi IP?" I.E. Are US ISPs, particularly big ones like Comcast, required to geolocate ISPs to the state where the perso

Is there even such a thing as a "Mississippi IP?" I.E. Are US ISPs, particularly big ones like Comcast, required to geolocate ISPs to the state where the person is actually in? What about mobile ones? Where I live (not US), it is extremely common to get an IP that Maxmind geolocates to a region far from where you actually live.

kube-system 1y ago
GeoIP services are not 100% accurate, but that doesn't mean they're completely useless. The law in question requires "commercially reasonable efforts"
- beefnugs 1y ago
  Remember that massive surveillance capitalism apparatus that has been created for years? Now everyone must pay for it to legally comply with whatever arbitrary bullshit no matter how expensive the data becomes
  - kube-system 1y ago
    The most popular GeoIP database has a free tier that would easily work for this. And there are many other options.
    - tzs 1y ago
      > The most popular GeoIP database has a free tier that would easily work for this The free tier does have limits on the number of API calls can you can make. But the good news is you don't have to use their API. You can download the database [1] and do all the lookups locally without having to worry about going over their API limits. It consists of 10 CSV files and is about 45 MB compressed, 380 MB uncompressed. For just identifying US states from IP address you just need 3 of the CSV files: a 207 MB file of IPv4 address information, a 120 MB file for IPv6, and a 6.7 MB file that lets you lookup by an ID that you find in one of the first two the information about the IP address location including state. It's easy to write a script to turn this into an SQL database that just contains IP ranges and the corresponding state and then use that with sqlite or whatever network database you use internally from any of your stuff that needs this information. If you don't actually need Geo IP in general and are only adding it in order to block specific states you can easily omit IPs that are not mapped to those states which would make it pretty small. The database has 3.4 million IPv4 address ranges, but only 5 359 of them are listed as being in Mississippi. There are 1.8 million address ranges in the IPv6 file, and 3 946 of them are listed as being in Mississippi. Here's how to get the Mississippi ranges from the command line, although this is kind of slow--the 3rd line took 7.5 minutes on my M2 Mac Studio and the 4th took almost 4 minutes. A proper script or program would be a lot faster. grep ,MS,Mississippi, GeoLite2-City-Locations-en.csv | cut -d , -f 1 > 1 sed -e s/^/,/ -e s/$/,/ < 1 > 2 grep -f 2 GeoLite2-City-Blocks-IPv4.csv | cut -d , -f 1 > MS-IP4.txt grep -f 2 GeoLite2-City-Blocks-IPv6.csv | cut -d , -f 1 > MS-IP6.txt Also a proper script or program would be able to look specifically at the correct field when matching the ID from the locations file to the IP range lines. The commands above just hope that things that look like location IDs don't occur in other fields in the IP range files. [1] URL=https://download.maxmind.com/geoip/databases/GeoLite2-City/download?suffix=tar.gz curl -L -u userid:license_key $URL > db.tar.gz
      - kube-system 1y ago
        My comment said "database" and not "API" :) Also there is no need to spend time parsing it yourself, there are plenty of existing libraries you can simply point at the file.
        tzs 1y ago
        > My comment said "database" and not "API" :) Sure, but it is quite common for companies offering database access to offer that via an API to query the database on their server rather than letting customers download the whole database for local use. It thus seemed worthwhile to make it clear that MaxMind does let you download the whole database. As far as libraries go sure that's a possibility. But for people who just need a simple IP to country or IP to US state lookup and need that from a variety of languages it may be overall less annoying to make your own DB from the CSV files that just handles what you need and nothing more. I've already got libraries for sqlite, MySQL, or both for every language I use where I need to do these lookups, and almost all the applications that need these lookups are already connecting to our databases. Add an IPv4_to_Country table to that database (that they are already using) and then it just a matter of doing a "select country_code from IPv4_to_Country where ip_low <= ? and ? <= ip_high" with the two '?'s replaced with the IP address we want to lookup, probably using a DB handle they already have open. Many would find that a lot easier than adding a dependency on a 3rd party GeoIP library. Beside it being one more thing on each machine that needs periodic updating (or rather N more if you are working in N different languages on that machine), I believe that most of these libraries require you to have a copy of the download database on the local machine, so that's another thing you have to keep up to date on every server. With the "make your own simple SQL DB" approach you just have to keep an up to date download from MaxMind on the machine that builds your SQL DB. After building the SQL DB you then just have to upload it to your one network DB server (e.g., your MySQL server) and all your apps that query that DB are up to date no matter what server they are on or what language they are in. If you are building an sqlite DB for some of your apps, you do have to copy that to all the servers that contain such apps, so you don't totally escape having to do updates on those machines. If making the SQL DB were hard then maybe reducing dependencies and reducing the number of things that need updates might not be worth it, but the CSV files are organized very sensibly. The scripts to make the SQL DBs are close to trivial if you've got a decent CSV parser in the language you are writing them in.
  - gruez 1y ago
    >Remember that massive surveillance capitalism apparatus that has been created for years? Now everyone must pay for it to legally comply with whatever arbitrary bullshit Calling geoip databases "surveillance capitalism" seems like a stretch. It might be used by "surveillance capitalism", but you don't really have to surveil people to build a geoip database, only scrape RIR allocation records (all public, btw) and BGP routes, do ping tests, and parse geofeeds provided by providers. None of that is "surveillance capitalism" in any meaningful sense.
    - TGower 1y ago
      If selling the physical location information of users isn't surveillance capatalism, then the term doesn't mean anything. "We don't surveil people, we just try to find out where they live and sell that data"
      - gruez 1y ago
        If that's "surveillance capitalism", what's your opinion on databases that map phone numbers to locations? eg. when you get a phone call from 217-555-1234, and it shows "Springfield, IL"? Is that "surveillance capitalism"? That's basically all geoip databases are. Moreover there's plenty of non "surveillance capitalism" uses for geoip that make it questionable to call it "surveillance capitalism". Determining the region for a site, or automatically selecting the closest store, for instance. Before the advent of anycast CDNs, it was also basically the only way to route your visitors to the closest server.
        TGower 1y ago
        Is there a single company out there making it's money selling access to an area code database? GeoIP databases are much higher resolution and use active scanning methods like ping timing. If a company was spam calling me to estimate distance based on call connection lag, yes that would be surveillance capitalism.
        gruez 1y ago
- Falkon1313 1y ago
  I wonder what is a "commercially reasonable effort" for a non-commercial website to collect, accurately verify, and securely store everyone's identity, location, and age? Personally I'd say none at all, unless the government itself provides it as a free service, takes on all the liability, and makes it simple to use. It also defines personally identifiable information as including "pseudonymous information when the information is used by a controller or processor in conjunction with additional information that reasonably links the information to an identified or identifiable individual." But it doesn't specify what it means by 'controller' or 'processor' either. If a hobbyist just sets up a forum site, with no payment processor and no identified or identifiable information required, it would seem reasonable that the law should not apply. But I'm not a lawyer. Clearly, however, attempting to comply with the law just in case, by requiring ID, would however then make it applicable, since that is personally identifiable information.
  - kube-system 1y ago
    > I wonder what is a "commercially reasonable effort" for a non-commercial website to collect, accurately verify, and securely store everyone's identity, location, and age? > Personally I'd say none at all, unless the government itself provides it as a free service, takes on all the liability, and makes it simple to use. 1. There are many commercial services that do identity verification. There are many other commercial websites that have tools to do identity verification themselves. There are industry published best practices for these types of activities. All of these are evidence that you could use to demonstrate how you are making a commercially reasonable effort. 2. It's completely irrelevant whether you consider your website "commercial" or not. The law defines which websites it applies to, based on the activities they engage in. https://law.justia.com/codes/mississippi/title-45/chapter-38/section-45-38-3/ https://law.justia.com/codes/mississippi/title-45/chapter-38... 3. Since when does the government have to give you compliance tools for free in order to require something of you? This isn't the standard for anything anywhere. Compliance with the law is often quite expensive. Honestly, buying an identity verification service is pretty cheap in the spectrum of compliance costs. > If a hobbyist just sets up a forum site, with no payment processor and no identified or identifiable information required, it would seem reasonable that the law should not apply. But I'm not a lawyer. You don't have to guess whether or not this is reasonable or not. If you read the law, you'll see that it says it only applies to sites that collect personally identifiable information. From the above link, again: > "Digital service" means a website, an application, a program, or software that collects or processes personal identifying information with Internet connectivity. > "Personal identifying information" means any information, including sensitive information, that is linked or reasonably linkable to an identified or identifiable individual. The term includes pseudonymous information when the information is used by a controller or processor in conjunction with additional information that reasonably links the information to an identified or identifiable individual. The term does not include deidentified information or publicly available information.
estimator7292 1y ago
You pretty much just plug the IP into a geolocating API and hope. There's nothing else to do. Any collateral damage is on the legislation, not any individual site or admin. As you say, IP geolocation is unreliable. Unfortunately that's the only option. If it is technologically impossible to comply with the law, you just gotta do the best you can. If someone in MI gets a weird IP, there's absolutely nothing any third party can do. That's on the ISP for not allocating an appropriate IP or the legislators for being morons.
- selimthegrim 1y ago
  MI is Michigan.
  - phinnaeus 1y ago
    Right, they might get an MS IP and be blocked :P
tallytarik 1y ago
ISPs have no obligation, although the ubiquity of sites and apps relying on IP geolocation mean that ISPs are incentivized to provide correct info these days. I run a geolocation service, and over the years we've seen more and more ISPs providing official geofeeds. The majority of medium-large ISPs in the US now provide a geofeed, for example. But there's still an ongoing problem in geofeeds being up-to-date, and users being assigned to a correct 'pool' etc. Mobile IPs are similar but are still certainly the most difficult (relative lack of geofeeds or other accurate data across providers)
- miki123211 1y ago
  Mobile IPs reflect the user's "registered area" at best, not their actual location. This is mostly because of how APNs / G-GNS / P-GW systems work. E.G. you may have an APN that puts you straight in a corporate network, and the mobile network needs you to keep using that APN when roaming. This is why your roaming IP is usually in the country you're from, not the one you're currently in. I've heard of local breakout being possible, but never actually seen it in practice.
cwbriscoe 1y ago
I live in Vancouver, WA and my IP comes back to Portland, OR.
- brewdad 1y ago
  Vancouver residents may as well be Oregonians anyway. Most of them are paying OR income tax. They do most of their shopping and entertainment in Oregon too.
  - cwbriscoe 1y ago
    I work for a Portland company at home in Vancouver so I get to skip their income tax. It's a 10-15 minute drive to the PDX area where there is a Best Buy, Ikea and other stores where I can easily skip sales if I want to.
    - hellojesus 1y ago
      You live the best life; basically only federal income taxes. The 20 min further south than you I live costs me over $30k/year.
      - cwbriscoe 1y ago
        Yeah, I was working from home anyway so it just made sense to move. The money I saved paid my rent fully and then some.
burnt-resistor 1y ago
Required? Not sure, but probably not, but they do so to monetize your metadata and provide hints to websites so they show language- and country-localized "local" versions of websites before/instead-of/as-a-fallback-to requesting location permissions.

kube-system 1y ago

GeoIP services are not 100% accurate, but that doesn't mean they're completely useless. The law in question requires "commercially reasonable efforts"

beefnugs 1y ago
Remember that massive surveillance capitalism apparatus that has been created for years? Now everyone must pay for it to legally comply with whatever arbitrary bullshit no matter how expensive the data becomes
- kube-system 1y ago
  The most popular GeoIP database has a free tier that would easily work for this. And there are many other options.
  - tzs 1y ago
    > The most popular GeoIP database has a free tier that would easily work for this The free tier does have limits on the number of API calls can you can make. But the good news is you don't have to use their API. You can download the database [1] and do all the lookups locally without having to worry about going over their API limits. It consists of 10 CSV files and is about 45 MB compressed, 380 MB uncompressed. For just identifying US states from IP address you just need 3 of the CSV files: a 207 MB file of IPv4 address information, a 120 MB file for IPv6, and a 6.7 MB file that lets you lookup by an ID that you find in one of the first two the information about the IP address location including state. It's easy to write a script to turn this into an SQL database that just contains IP ranges and the corresponding state and then use that with sqlite or whatever network database you use internally from any of your stuff that needs this information. If you don't actually need Geo IP in general and are only adding it in order to block specific states you can easily omit IPs that are not mapped to those states which would make it pretty small. The database has 3.4 million IPv4 address ranges, but only 5 359 of them are listed as being in Mississippi. There are 1.8 million address ranges in the IPv6 file, and 3 946 of them are listed as being in Mississippi. Here's how to get the Mississippi ranges from the command line, although this is kind of slow--the 3rd line took 7.5 minutes on my M2 Mac Studio and the 4th took almost 4 minutes. A proper script or program would be a lot faster. grep ,MS,Mississippi, GeoLite2-City-Locations-en.csv | cut -d , -f 1 > 1 sed -e s/^/,/ -e s/$/,/ < 1 > 2 grep -f 2 GeoLite2-City-Blocks-IPv4.csv | cut -d , -f 1 > MS-IP4.txt grep -f 2 GeoLite2-City-Blocks-IPv6.csv | cut -d , -f 1 > MS-IP6.txt Also a proper script or program would be able to look specifically at the correct field when matching the ID from the locations file to the IP range lines. The commands above just hope that things that look like location IDs don't occur in other fields in the IP range files. [1] URL=https://download.maxmind.com/geoip/databases/GeoLite2-City/download?suffix=tar.gz curl -L -u userid:license_key $URL > db.tar.gz
    - kube-system 1y ago
      My comment said "database" and not "API" :) Also there is no need to spend time parsing it yourself, there are plenty of existing libraries you can simply point at the file.
      - tzs 1y ago
        > My comment said "database" and not "API" :) Sure, but it is quite common for companies offering database access to offer that via an API to query the database on their server rather than letting customers download the whole database for local use. It thus seemed worthwhile to make it clear that MaxMind does let you download the whole database. As far as libraries go sure that's a possibility. But for people who just need a simple IP to country or IP to US state lookup and need that from a variety of languages it may be overall less annoying to make your own DB from the CSV files that just handles what you need and nothing more. I've already got libraries for sqlite, MySQL, or both for every language I use where I need to do these lookups, and almost all the applications that need these lookups are already connecting to our databases. Add an IPv4_to_Country table to that database (that they are already using) and then it just a matter of doing a "select country_code from IPv4_to_Country where ip_low <= ? and ? <= ip_high" with the two '?'s replaced with the IP address we want to lookup, probably using a DB handle they already have open. Many would find that a lot easier than adding a dependency on a 3rd party GeoIP library. Beside it being one more thing on each machine that needs periodic updating (or rather N more if you are working in N different languages on that machine), I believe that most of these libraries require you to have a copy of the download database on the local machine, so that's another thing you have to keep up to date on every server. With the "make your own simple SQL DB" approach you just have to keep an up to date download from MaxMind on the machine that builds your SQL DB. After building the SQL DB you then just have to upload it to your one network DB server (e.g., your MySQL server) and all your apps that query that DB are up to date no matter what server they are on or what language they are in. If you are building an sqlite DB for some of your apps, you do have to copy that to all the servers that contain such apps, so you don't totally escape having to do updates on those machines. If making the SQL DB were hard then maybe reducing dependencies and reducing the number of things that need updates might not be worth it, but the CSV files are organized very sensibly. The scripts to make the SQL DBs are close to trivial if you've got a decent CSV parser in the language you are writing them in.
- gruez 1y ago
  >Remember that massive surveillance capitalism apparatus that has been created for years? Now everyone must pay for it to legally comply with whatever arbitrary bullshit Calling geoip databases "surveillance capitalism" seems like a stretch. It might be used by "surveillance capitalism", but you don't really have to surveil people to build a geoip database, only scrape RIR allocation records (all public, btw) and BGP routes, do ping tests, and parse geofeeds provided by providers. None of that is "surveillance capitalism" in any meaningful sense.
  - TGower 1y ago
    If selling the physical location information of users isn't surveillance capatalism, then the term doesn't mean anything. "We don't surveil people, we just try to find out where they live and sell that data"
    - gruez 1y ago
      If that's "surveillance capitalism", what's your opinion on databases that map phone numbers to locations? eg. when you get a phone call from 217-555-1234, and it shows "Springfield, IL"? Is that "surveillance capitalism"? That's basically all geoip databases are. Moreover there's plenty of non "surveillance capitalism" uses for geoip that make it questionable to call it "surveillance capitalism". Determining the region for a site, or automatically selecting the closest store, for instance. Before the advent of anycast CDNs, it was also basically the only way to route your visitors to the closest server.
      - TGower 1y ago
        Is there a single company out there making it's money selling access to an area code database? GeoIP databases are much higher resolution and use active scanning methods like ping timing. If a company was spam calling me to estimate distance based on call connection lag, yes that would be surveillance capitalism.
        gruez 1y ago
Falkon1313 1y ago
I wonder what is a "commercially reasonable effort" for a non-commercial website to collect, accurately verify, and securely store everyone's identity, location, and age? Personally I'd say none at all, unless the government itself provides it as a free service, takes on all the liability, and makes it simple to use. It also defines personally identifiable information as including "pseudonymous information when the information is used by a controller or processor in conjunction with additional information that reasonably links the information to an identified or identifiable individual." But it doesn't specify what it means by 'controller' or 'processor' either. If a hobbyist just sets up a forum site, with no payment processor and no identified or identifiable information required, it would seem reasonable that the law should not apply. But I'm not a lawyer. Clearly, however, attempting to comply with the law just in case, by requiring ID, would however then make it applicable, since that is personally identifiable information.
- kube-system 1y ago
  > I wonder what is a "commercially reasonable effort" for a non-commercial website to collect, accurately verify, and securely store everyone's identity, location, and age? > Personally I'd say none at all, unless the government itself provides it as a free service, takes on all the liability, and makes it simple to use. 1. There are many commercial services that do identity verification. There are many other commercial websites that have tools to do identity verification themselves. There are industry published best practices for these types of activities. All of these are evidence that you could use to demonstrate how you are making a commercially reasonable effort. 2. It's completely irrelevant whether you consider your website "commercial" or not. The law defines which websites it applies to, based on the activities they engage in. https://law.justia.com/codes/mississippi/title-45/chapter-38/section-45-38-3/ https://law.justia.com/codes/mississippi/title-45/chapter-38... 3. Since when does the government have to give you compliance tools for free in order to require something of you? This isn't the standard for anything anywhere. Compliance with the law is often quite expensive. Honestly, buying an identity verification service is pretty cheap in the spectrum of compliance costs. > If a hobbyist just sets up a forum site, with no payment processor and no identified or identifiable information required, it would seem reasonable that the law should not apply. But I'm not a lawyer. You don't have to guess whether or not this is reasonable or not. If you read the law, you'll see that it says it only applies to sites that collect personally identifiable information. From the above link, again: > "Digital service" means a website, an application, a program, or software that collects or processes personal identifying information with Internet connectivity. > "Personal identifying information" means any information, including sensitive information, that is linked or reasonably linkable to an identified or identifiable individual. The term includes pseudonymous information when the information is used by a controller or processor in conjunction with additional information that reasonably links the information to an identified or identifiable individual. The term does not include deidentified information or publicly available information.

estimator7292 1y ago

You pretty much just plug the IP into a geolocating API and hope. There's nothing else to do. Any collateral damage is on the legislation, not any individual site or admin. As you say, IP geolocation is unreliable. Unfortunately that's the only option. If it is technologically impossible to comply with the law, you just gotta do the best you can. If someone in MI gets a weird IP, there's absolutely nothing any third party can do. That's on the ISP for not allocating an appropriate IP or the legislators for being morons.

selimthegrim 1y ago
MI is Michigan.
- phinnaeus 1y ago
  Right, they might get an MS IP and be blocked :P

tallytarik 1y ago

ISPs have no obligation, although the ubiquity of sites and apps relying on IP geolocation mean that ISPs are incentivized to provide correct info these days. I run a geolocation service, and over the years we've seen more and more ISPs providing official geofeeds. The majority of medium-large ISPs in the US now provide a geofeed, for example. But there's still an ongoing problem in geofeeds being up-to-date, and users being assigned to a correct 'pool' etc. Mobile IPs are similar but are still certainly the most difficult (relative lack of geofeeds or other accurate data across providers)

miki123211 1y ago
Mobile IPs reflect the user's "registered area" at best, not their actual location. This is mostly because of how APNs / G-GNS / P-GW systems work. E.G. you may have an APN that puts you straight in a corporate network, and the mobile network needs you to keep using that APN when roaming. This is why your roaming IP is usually in the country you're from, not the one you're currently in. I've heard of local breakout being possible, but never actually seen it in practice.

cwbriscoe 1y ago

I live in Vancouver, WA and my IP comes back to Portland, OR.

brewdad 1y ago
Vancouver residents may as well be Oregonians anyway. Most of them are paying OR income tax. They do most of their shopping and entertainment in Oregon too.
- cwbriscoe 1y ago
  I work for a Portland company at home in Vancouver so I get to skip their income tax. It's a 10-15 minute drive to the PDX area where there is a Best Buy, Ikea and other stores where I can easily skip sales if I want to.
  - hellojesus 1y ago
    You live the best life; basically only federal income taxes. The 20 min further south than you I live costs me over $30k/year.
    - cwbriscoe 1y ago
      Yeah, I was working from home anyway so it just made sense to move. The money I saved paid my rent fully and then some.

burnt-resistor 1y ago

Required? Not sure, but probably not, but they do so to monetize your metadata and provide hints to websites so they show language- and country-localized "local" versions of websites before/instead-of/as-a-fallback-to requesting location permissions.