9 ms·

Changing the Android captive portal page (2020)

91 points by transpute 3y ago

izacus 3y ago
The attack surface could probably be lowered a bit by configuring the HTTP server (nginx, etc.) to return 204 instead of running PHP behind it. Still, neat :)
- sp1rit 3y ago
  I don't think this has anything to do with "attack surface". This is simply to avoid google collecting ip addresses. Running nginx might be marginally faster than PHP, but I believe the latency to your server will be far greater than the time it takes the server "to spin up the php interpreter" (doesn't even happen anymore when using FastCGI).
veave 3y ago
You don't need a die(); when you're at the end of the script already...
- neoxeon 3y ago
  IIRC, calling die() or exit() explicitly when you know it's time to _end_ is a good practice to prevent erroneous or malice including.
- account42 3y ago
  You also don't need PHP to return a static status code with no content.
Raed667 3y ago
neverssl.com has been my go-to captive portal for many years now.
- pests 3y ago
  I use aol.com !
callumjones 3y ago
gstatic.com - the backbone of the internet.
- deusum 3y ago
  Just marvel at how much data/metadata they get from that little trick when multiplied by millions users.
  - throwawaaarrgh 3y ago
    I just realized, they (or a government mandate) could force a redirect under specific circumstances to a site under google.com, and if they're using the browser engine for the request, they'd get the cookies of that user as soon as they get on the internet. Maybe even trigger an exploit in the browser. This is the most fantastic way to target and exploit any single Android user anywhere in the world that I've ever heard of. Automatic, hard to avoid, easy to implement, and the user has no idea.
    - pritambaral 3y ago
      > could force a redirect under specific circumstances to a site under google.com, and if they're using the browser engine for the request, they'd get the cookies of that user as soon as they get on the internet At first, I thought this can't be true because, surely, Google marks its cookies as HTTPS-only, right? So I checked, and turns out about half the cookies google.com has in my browser are not HTTPS-only. In fact, the HTTPS-only cookies it does have seem to be the same set of cookies, just with a '__Secure-' prefix. Similarly, about half (different set) of the cookies JS accessible.
lucb1e 3y ago
I was about to go do this, because it seems cool to have a log of all the IPs that my phone has been on and not to leak this data to Google Then I remembered how much trouble I already have when my server is offline for any reason: no email, no grocery list, no backups, no VPN, no git, no link shortener, no IRC, no website with various tools like 'what is my IP' or bidirectional latency testing... self hosting is great but you're also on the hook for everything and it all has a single point of failure unless you want to run a dual-uplink server farm. And yet I am still tempted to add this 204 service, but maybe then I also don't have internet on my phone anymore because it keeps believing it's behind a captive portal when the service is offline
- bartvk 3y ago
  What do you use to self-host your grocery lists?
  - tcrenshaw 3y ago
    not OP, but I use tandoor to host my recipes and it generates a shopping list based on what I've picked to cook this week
    - lucb1e 3y ago
      this was nearly impossible to find on duckduckgo https://tandoor.dev https://tandoor.dev What a name, the only thing you get when looking for 'tandoor' or 'tandoor grocery list' are recipes for tandoori In my quest, I've also come across other promising-looking things https://kitchenowl.org https://kitchenowl.org and https://recipesage.com https://recipesage.com
  - LegitShady 3y ago
    dead trees, strained
  - lucb1e 3y ago
    Something I wrote across about eight years, trying different things and never cleaning up unused parts, so the code is a mess but this is it: https://github.com/lgommans/mpgroceries https://github.com/lgommans/mpgroceries The name (multiplayer groceries) refers to the original idea of having the software assign random items to random people, so you can both go through the store independently and not get duplicate items, for maximum efficiency, but I don't think I've used that more than once. Probably I could write a small book about all the things that you shouldn't do, like making a store layout and mapping where all the items are and then making it sort by nearest item...which, turns out, is somehow always on the other side of a wall. This routing is better than nothing, but it turns out that categorizing ("freezer", "candy", "fruit/veg") is: much simpler for the user, portable across stores, and more effective for routing as well. I've started on a rewrite with a friend, but it's very early stages and the primary purpose of the new project is to turn a list of meals into a grocery list rather than being a grocery list that you add random items to: https://codeberg.org/lucg/MealsUnveiled https://codeberg.org/lucg/MealsUnveiled The old project can also add a recipe to your grocery list, but I never wrote the code to add up amounts (it'll ignore duplicates instead)
raspyberr 3y ago
This is done on GrapheneOS by default. It also allows you to change the URLs without connecting your phone and using adb. https://grapheneos.org/faq#default-connections https://grapheneos.org/faq#default-connections
palata 3y ago
Note that ROMs like GrapheneOS or /e/OS do not connect to the Google connectivitycheck server (instead they use their own servers).
- _rdvw 3y ago
  Except /e/OS then throws that out the window because it installs and enables microG by default which immediately connects directly to Google: https://github.com/microg/GmsCore/wiki/Google-Network-Connections https://github.com/microg/GmsCore/wiki/Google-Network-Connec... And /e/OS is still 224 days behind Chromium updates: https://divestos.org/misc/ch-dates.txt https://divestos.org/misc/ch-dates.txt
  - palata 3y ago
    > Except /e/OS then throws that out the window I checked your link, I don't see the connectivity check listed there. Can you please elaborate? > And /e/OS is still 224 days behind Chromium updates Did I mention anything at all about Chromium updates? Or is it just to add a link to your website?
    - _rdvw 3y ago
      > check listed there /e/OS changes the connectivity check from Google to their own server, great, but then it still connects to Google services via microG. So why bother changing the connectivity check? > Chromium updates Your comment was recommending an operating system that is actively harmful to its users by falling behind basic security updates that every single other OS has done. edit: aside from that, what are your thoughts on their upcoming license key system for OTAs that give users a persistent identifier? https://gitlab.e.foundation/e/os/android_packages_apps_Updater/uploads/bb0e5656df992b3717d4df9c71ebf39e/Screenshot_20230713_121140.png https://gitlab.e.foundation/e/os/android_packages_apps_Updat...
      - palata 3y ago
        > but then it still connects to Google services via microG. So you actually do agree that /e/OS changes the connectivity check, don't you? > Your comment was recommending an operating system Not at all. My comment was mentioning that at least some custom ROMs I know do remove the connectivity check, which is the topic of that post ("Changing the Android captive portal page"). In that particular post, I did not recommend /e/OS, I just stated a fact. But if it's about recommendation, I would recommend against DivestOS, because the author is really toxic. That may be harmful to users, too.
c4mpute 3y ago
Note that some captive portals will break here, because they will use various kinds of redirections for everything except the well-known Google and Apple captive portal check URLs.
- account42 3y ago
  Wouldn't such redirections break Google/Apple's checks? The whole point of this URL is to test if you can actually connect to the internet - if random urls are intercepted/broken then the test should fail.
  - c4mpute 3y ago
    No, those redirections explicitly exempt Google/Apple checks by not redirecting their URLs. All other URLs are redirected to show advertisements usually, either injected into the site you wanted to visit or as a replacement, and a please-login-and-pay-here page. That devices are fooled to assume that internet is working is intentional, otherwise those devices would immediately disconnect.
    - account42 3y ago
      Yes, what you are describing is breaking the Google/Apple checks by making those devices think that Internet access is available when its not.
geokon 3y ago
Not to be conspiratorial, but captive portals always felt like a little broken piece of internet technology that's never been fixed bc it benefits Google This has a very cryptic failure mode if you travel to China. Android will just tells you you can't connect to the internet :) It's pretty easy to work around. You just go to detectportal.firefox.com/ But naturally you need to know what's gone wrong in the first place !
- izacus 3y ago
  > Not to be conspiratorial, but captive portals always felt like a little broken piece of internet technology that's never been fixed bc it benefits Google Against my better judgement and for my grim fascination, I'd love to hear a full explanation of this conspiracy theory. Can you elaborate?
- stavros 3y ago
  How do captive portals benefit Google?
  - acover 3y ago
    > But, why should I change it? > Well, on Android 6 or higher, every time you switch networks, your phone tries to access the following URL: > http://connectivitycheck.gstatic.com/generate_204 http://connectivitycheck.gstatic.com/generate_204 > As we can see, gstatic.com – Google’s servers. We might not want Google to know our new IP address or when we switched between networks. So… Why don’t we just self-host it? It’s pretty easy.
    - stavros 3y ago
      Yeah, that's not Google having a say on whether captive portals exist and deciding to keep them there, that's just Google deciding that it wants to know your IP whenever you connect to a network. I don't know why this is such a big issue, given that they have your more or less exact location at all times, but changing it is so easy that, why not.
mschuster91 3y ago
It's amazing we're still using captive portals instead of a DHCP option that's pushed to the client in 2023. Wtf.
- ajsnigrutin 3y ago
  The point of captive portals is to get the user to either click "I agree" (to the terms,..) or enter a code/pay for the internet. Putting the url in a dhcp option wouldn't change that, wouldn't work with older devices (so a redirect would still be needed), and the only change is, that you'd have two ways to reach the same captive portal page, that you'd have to configure on two (or three, with ipv6) devices/config files.
  - mschuster91 3y ago
    It would nevertheless be the cleaner option. Had this been proposed and implemented six years ago, by now almost all devices would have supported it.
    - ajsnigrutin 3y ago
      IPv6 standard came out in 1998, and a lot of devices still don't support it.
      - jeroenhd 3y ago
        I don't see a lot of devices that don't support IPv6 these days. Linux supports it, so most terrible IoT crapware works perfectly fine, and esp-idf enables it by default (solving the problem for a ton of cheap smart home crap). There are still devices that don't support hardware IPv6 acceleration but at this point even those are becoming rare.
        ajsnigrutin 3y ago
        A bunch of devices don't... IoT devices based on esp8266 or esp32 don't (from washing machines to sous vide cookers), many ip cameras, sensors, smart locks etc., don't. Yeah, linux itself might support it, but when compiling the image for those few megs of flash, developers just disable it, because well.. everyone still has ipv4 available.
_rdvw 3y ago
Or switch to a custom OS that provides controls for this and more: - my DivestOS (includes nine presets): https://divestos.org/pages/network_connections https://divestos.org/pages/network_connections - GrapheneOS (includes two presets): https://grapheneos.org/faq#default-connections https://grapheneos.org/faq#default-connections also if you do want to setup your own portal there is a simpler method than invoking php each time: RewriteEngine on RewriteRule ^/generate_204$ - [END,NE,R=204]
jeroenhd 3y ago
On the one hand, this denies Google the metadata from your connectivity check. On the other hand, using your own site uniquely identifies you on each network (negating the privacy improvements through random MAC addresses and such). I would much rather use a site I can trust. Mozilla offers a 204 service for Firefox, for example, that's a lot less obvious for trackers. You could also try to confuse the system by using iOS' 204 URL for your Android devices.
- _rdvw 3y ago
  Not all of these return 204, some just return 200 with an "OK" or "success". $ curl --verbose "https://detectportal.firefox.com" < HTTP/2 200 success
lazyweb 3y ago
I'm using Graphene, but with the default Google captive portal settings. While yes, I am technically leaking my current public IP to Google servers, what does it actually tell them apart from the IP? It's a generic user agent I believe and there's billions of (simple) HTTP requests hitting that endpoint. If you're using a stock Android (or even worse, like Samsung) it's the Play services and unkillable vendor background apps you should be worried about. I'd argue it's a lot more conspicuous to network operators if you're using non-standard captive URLs.

91 points by transpute 3y ago

The attack surface could probably be lowered a bit by configuring the HTTP server (nginx, etc.) to return 204 instead of running PHP behind it. Still, neat :)

sp1rit 3y ago
I don't think this has anything to do with "attack surface". This is simply to avoid google collecting ip addresses. Running nginx might be marginally faster than PHP, but I believe the latency to your server will be far greater than the time it takes the server "to spin up the php interpreter" (doesn't even happen anymore when using FastCGI).

veave 3y ago

You don't need a die(); when you're at the end of the script already...

neoxeon 3y ago
IIRC, calling die() or exit() explicitly when you know it's time to _end_ is a good practice to prevent erroneous or malice including.
account42 3y ago
You also don't need PHP to return a static status code with no content.

Raed667 3y ago

neverssl.com has been my go-to captive portal for many years now.

pests 3y ago
I use aol.com !

callumjones 3y ago

gstatic.com - the backbone of the internet.

deusum 3y ago
Just marvel at how much data/metadata they get from that little trick when multiplied by millions users.
- throwawaaarrgh 3y ago
  I just realized, they (or a government mandate) could force a redirect under specific circumstances to a site under google.com, and if they're using the browser engine for the request, they'd get the cookies of that user as soon as they get on the internet. Maybe even trigger an exploit in the browser. This is the most fantastic way to target and exploit any single Android user anywhere in the world that I've ever heard of. Automatic, hard to avoid, easy to implement, and the user has no idea.
  - pritambaral 3y ago
    > could force a redirect under specific circumstances to a site under google.com, and if they're using the browser engine for the request, they'd get the cookies of that user as soon as they get on the internet At first, I thought this can't be true because, surely, Google marks its cookies as HTTPS-only, right? So I checked, and turns out about half the cookies google.com has in my browser are not HTTPS-only. In fact, the HTTPS-only cookies it does have seem to be the same set of cookies, just with a '__Secure-' prefix. Similarly, about half (different set) of the cookies JS accessible.

lucb1e 3y ago

I was about to go do this, because it seems cool to have a log of all the IPs that my phone has been on and not to leak this data to Google Then I remembered how much trouble I already have when my server is offline for any reason: no email, no grocery list, no backups, no VPN, no git, no link shortener, no IRC, no website with various tools like 'what is my IP' or bidirectional latency testing... self hosting is great but you're also on the hook for everything and it all has a single point of failure unless you want to run a dual-uplink server farm. And yet I am still tempted to add this 204 service, but maybe then I also don't have internet on my phone anymore because it keeps believing it's behind a captive portal when the service is offline

bartvk 3y ago
What do you use to self-host your grocery lists?
- tcrenshaw 3y ago
  not OP, but I use tandoor to host my recipes and it generates a shopping list based on what I've picked to cook this week
  - lucb1e 3y ago
    this was nearly impossible to find on duckduckgo https://tandoor.dev https://tandoor.dev What a name, the only thing you get when looking for 'tandoor' or 'tandoor grocery list' are recipes for tandoori In my quest, I've also come across other promising-looking things https://kitchenowl.org https://kitchenowl.org and https://recipesage.com https://recipesage.com
- LegitShady 3y ago
  dead trees, strained
- lucb1e 3y ago
  Something I wrote across about eight years, trying different things and never cleaning up unused parts, so the code is a mess but this is it: https://github.com/lgommans/mpgroceries https://github.com/lgommans/mpgroceries The name (multiplayer groceries) refers to the original idea of having the software assign random items to random people, so you can both go through the store independently and not get duplicate items, for maximum efficiency, but I don't think I've used that more than once. Probably I could write a small book about all the things that you shouldn't do, like making a store layout and mapping where all the items are and then making it sort by nearest item...which, turns out, is somehow always on the other side of a wall. This routing is better than nothing, but it turns out that categorizing ("freezer", "candy", "fruit/veg") is: much simpler for the user, portable across stores, and more effective for routing as well. I've started on a rewrite with a friend, but it's very early stages and the primary purpose of the new project is to turn a list of meals into a grocery list rather than being a grocery list that you add random items to: https://codeberg.org/lucg/MealsUnveiled https://codeberg.org/lucg/MealsUnveiled The old project can also add a recipe to your grocery list, but I never wrote the code to add up amounts (it'll ignore duplicates instead)

raspyberr 3y ago

This is done on GrapheneOS by default. It also allows you to change the URLs without connecting your phone and using adb. https://grapheneos.org/faq#default-connections https://grapheneos.org/faq#default-connections

palata 3y ago

Note that ROMs like GrapheneOS or /e/OS do not connect to the Google connectivitycheck server (instead they use their own servers).

_rdvw 3y ago
Except /e/OS then throws that out the window because it installs and enables microG by default which immediately connects directly to Google: https://github.com/microg/GmsCore/wiki/Google-Network-Connections https://github.com/microg/GmsCore/wiki/Google-Network-Connec... And /e/OS is still 224 days behind Chromium updates: https://divestos.org/misc/ch-dates.txt https://divestos.org/misc/ch-dates.txt
- palata 3y ago
  > Except /e/OS then throws that out the window I checked your link, I don't see the connectivity check listed there. Can you please elaborate? > And /e/OS is still 224 days behind Chromium updates Did I mention anything at all about Chromium updates? Or is it just to add a link to your website?
  - _rdvw 3y ago
    > check listed there /e/OS changes the connectivity check from Google to their own server, great, but then it still connects to Google services via microG. So why bother changing the connectivity check? > Chromium updates Your comment was recommending an operating system that is actively harmful to its users by falling behind basic security updates that every single other OS has done. edit: aside from that, what are your thoughts on their upcoming license key system for OTAs that give users a persistent identifier? https://gitlab.e.foundation/e/os/android_packages_apps_Updater/uploads/bb0e5656df992b3717d4df9c71ebf39e/Screenshot_20230713_121140.png https://gitlab.e.foundation/e/os/android_packages_apps_Updat...
    - palata 3y ago
      > but then it still connects to Google services via microG. So you actually do agree that /e/OS changes the connectivity check, don't you? > Your comment was recommending an operating system Not at all. My comment was mentioning that at least some custom ROMs I know do remove the connectivity check, which is the topic of that post ("Changing the Android captive portal page"). In that particular post, I did not recommend /e/OS, I just stated a fact. But if it's about recommendation, I would recommend against DivestOS, because the author is really toxic. That may be harmful to users, too.

c4mpute 3y ago

Note that some captive portals will break here, because they will use various kinds of redirections for everything except the well-known Google and Apple captive portal check URLs.

account42 3y ago
Wouldn't such redirections break Google/Apple's checks? The whole point of this URL is to test if you can actually connect to the internet - if random urls are intercepted/broken then the test should fail.
- c4mpute 3y ago
  No, those redirections explicitly exempt Google/Apple checks by not redirecting their URLs. All other URLs are redirected to show advertisements usually, either injected into the site you wanted to visit or as a replacement, and a please-login-and-pay-here page. That devices are fooled to assume that internet is working is intentional, otherwise those devices would immediately disconnect.
  - account42 3y ago
    Yes, what you are describing is breaking the Google/Apple checks by making those devices think that Internet access is available when its not.

geokon 3y ago

Not to be conspiratorial, but captive portals always felt like a little broken piece of internet technology that's never been fixed bc it benefits Google This has a very cryptic failure mode if you travel to China. Android will just tells you you can't connect to the internet :) It's pretty easy to work around. You just go to detectportal.firefox.com/ But naturally you need to know what's gone wrong in the first place !

izacus 3y ago
> Not to be conspiratorial, but captive portals always felt like a little broken piece of internet technology that's never been fixed bc it benefits Google Against my better judgement and for my grim fascination, I'd love to hear a full explanation of this conspiracy theory. Can you elaborate?
stavros 3y ago
How do captive portals benefit Google?
- acover 3y ago
  > But, why should I change it? > Well, on Android 6 or higher, every time you switch networks, your phone tries to access the following URL: > http://connectivitycheck.gstatic.com/generate_204 http://connectivitycheck.gstatic.com/generate_204 > As we can see, gstatic.com – Google’s servers. We might not want Google to know our new IP address or when we switched between networks. So… Why don’t we just self-host it? It’s pretty easy.
  - stavros 3y ago
    Yeah, that's not Google having a say on whether captive portals exist and deciding to keep them there, that's just Google deciding that it wants to know your IP whenever you connect to a network. I don't know why this is such a big issue, given that they have your more or less exact location at all times, but changing it is so easy that, why not.

mschuster91 3y ago

It's amazing we're still using captive portals instead of a DHCP option that's pushed to the client in 2023. Wtf.

ajsnigrutin 3y ago
The point of captive portals is to get the user to either click "I agree" (to the terms,..) or enter a code/pay for the internet. Putting the url in a dhcp option wouldn't change that, wouldn't work with older devices (so a redirect would still be needed), and the only change is, that you'd have two ways to reach the same captive portal page, that you'd have to configure on two (or three, with ipv6) devices/config files.
- mschuster91 3y ago
  It would nevertheless be the cleaner option. Had this been proposed and implemented six years ago, by now almost all devices would have supported it.
  - ajsnigrutin 3y ago
    IPv6 standard came out in 1998, and a lot of devices still don't support it.
    - jeroenhd 3y ago
      I don't see a lot of devices that don't support IPv6 these days. Linux supports it, so most terrible IoT crapware works perfectly fine, and esp-idf enables it by default (solving the problem for a ton of cheap smart home crap). There are still devices that don't support hardware IPv6 acceleration but at this point even those are becoming rare.
      - ajsnigrutin 3y ago
        A bunch of devices don't... IoT devices based on esp8266 or esp32 don't (from washing machines to sous vide cookers), many ip cameras, sensors, smart locks etc., don't. Yeah, linux itself might support it, but when compiling the image for those few megs of flash, developers just disable it, because well.. everyone still has ipv4 available.

_rdvw 3y ago

Or switch to a custom OS that provides controls for this and more: - my DivestOS (includes nine presets): https://divestos.org/pages/network_connections https://divestos.org/pages/network_connections - GrapheneOS (includes two presets): https://grapheneos.org/faq#default-connections https://grapheneos.org/faq#default-connections also if you do want to setup your own portal there is a simpler method than invoking php each time: RewriteEngine on RewriteRule ^/generate_204$ - [END,NE,R=204]

jeroenhd 3y ago

On the one hand, this denies Google the metadata from your connectivity check. On the other hand, using your own site uniquely identifies you on each network (negating the privacy improvements through random MAC addresses and such). I would much rather use a site I can trust. Mozilla offers a 204 service for Firefox, for example, that's a lot less obvious for trackers. You could also try to confuse the system by using iOS' 204 URL for your Android devices.

_rdvw 3y ago
Not all of these return 204, some just return 200 with an "OK" or "success". $ curl --verbose "https://detectportal.firefox.com" < HTTP/2 200 success

lazyweb 3y ago

I'm using Graphene, but with the default Google captive portal settings. While yes, I am technically leaking my current public IP to Google servers, what does it actually tell them apart from the IP? It's a generic user agent I believe and there's billions of (simple) HTTP requests hitting that endpoint. If you're using a stock Android (or even worse, like Samsung) it's the Play services and unkillable vendor background apps you should be worried about. I'd argue it's a lot more conspicuous to network operators if you're using non-standard captive URLs.