5 ms·
Kubernetes Home – what do you do if your ISP changes your IP addresses?
- wutwutwat 2y agoDealing with changing residential ips is nothing new. It's interesting to see how it's still being solved for even in this overly complex k8s landscape we find ourselves in now. Back in the day we'd use free services like https://freedns.afraid.org/ https://freedns.afraid.org/ on a cron to refresh the ip every so often. I used afraid to refresh my dial up ip address, for my "hosting service" domain. The "hosting service" was an old tower pc living in the cabinet underneath a fish tank. Ops was a lot different back then... Nowadays, if you're poking holes in your firewall and exposing your ip address to the world, you're doing it wrong. We've moved away from that model. There's no need to do that and expose yourself in those ways, when you can instead tunnel out. Cloudflare/argo tunnels, or tailscale tunnels, dial out from your service and don't expose your system directly to the open internet. Clourflare will even automagically set the dns for your domain to always route through that tunnel. Your isp allocated ip address is irrelevant, and nothing ever needs it because nothing ever routes to it. Your domain routes to a cf endpoint, and your system tunnels out to it, meeting in the middle. No open ports, no firewall rules, no NAT bs. Only downside is, you're relying on and trusting services like cf and tailscale.
- joecool1029 2y agoYeah, afraid.org used to work great. I mean the service still works great but Google appears to blacklist any domain with the nameserver there, all email goes to spam. I just found this out in the past month. Kept all my records the same and just moved the nameserver to Fastmail and the problem resolved immediately. I have an unusual dynamic ip situation I've been taking advantage of with a different system. Some years back I noticed T-Mobile phone lines allow inbound connections on ipv6 (T-Mobile Home Internet unfortunately does not). I have a small weather station I run on a rpi3b I can access anywhere by using ddclient on the pi with cloudflare api key and it sets the AAAA record which is proxied by cloudflare, I leave the A record blank. If any users on ipv4 try to visit the site, cloudflare proxies it for them, works pretty reliably.
- LoganDark 2y ago> Some years back I noticed T-Mobile phone lines allow inbound connections on ipv6 HELL YEAH. This is exactly how ipv6 is meant to be implemented. They deserve some real praise and recognition for this.
- lostmsu 2y ago> T-Mobile Home Internet unfortunately does not Why did they make it so inconvenient?
- LoganDark 2y agoHome Internet is seemingly far more affordable than their phone plans. So they have other ways of encouraging you to use the phone plans.
- joecool1029 2y agoIts about the same as my per-line cost. The real reason I suspect is to try to discourage upload usage. T-mobile's main spectrum is deployed with TDD which schedules send and receive on the same frequency. The config they use allocates 80% or 90% of the time to download and is set at network level, not dynamic per device. Basically, they are way more constrained on upload capacity the way things are currently built out.
- TZubiri 2y ago[flagged]
- yjftsjthsd-h 2y ago>> ddns > Yeah don't do that if you want to be a professional with pride in their craft Why not?
- sampullman 2y agoWhat's a better option than DDNS that doesn't add a bunch of latency to every connection?
- TZubiri 2y agoStatic, dedicated ip address
- sampullman 2y agoSorry, I thought the context was an environment in which static IPs aren't an option. For example, my home internet provider offers them, but the connection is guaranteed to go down once a month for maintenance. So I think DDNS is strictly better in this situation.
- gsich 2y agoDDNS does not imply foreign domain.
- homodyne 2y ago[dead]
- npodbielski 2y agoThere are more downsides. Instead of maintaining 1 server you have to maintain server and tunnel or 2 servers and tunnel. If someone does not know how to maintain internal network DNS and dhcp then if internet is down then your services are down too, because likely there are only reachable through external domain. I agree though that someone who does know much probably should not do that. If you know what is SSH and root account, probably less is more.
- globular-toast 2y ago"Exposing your IP address to the world" seems like such an arbitrary thing to care about when you're opening a tunnel with the express intention of letting people in. No NAT bs, but you've got magic tunnel bs instead that you have no control over. And of course you're still "exposed to the world". Your IP address is public. That's the whole point. So you're going to be using a firewall regardless, what difference does one rule make really?
- TZubiri 2y agoCrazy that someone is using something as complex as k8s on a home server and without knowing basics. Newbies are better served starting with the simple stuff and then moving to the complex if needed
- homodyne 2y ago[dead]
- robertlagrant 2y agoI think it's good. K8s is well packaged (these days) and quite discoverable. I agree it's likely to throw up some problems, but it's great that it's possible.
- TZubiri 2y agoI'll be honest, never learned it, worked up from network protocols, reached docker in terms of virtualization, and of course OS fundamentals. But even in those 3 areas I haven't exhausted all the knowledge and features available. So I'm just skeptical when someone is using k8s and hasn't mastered the fundamentals. How do they know whether they should be using a high level k8 feature or a low level os feature? Happened with docker a lot to me when I didn't know better, I was learning how to set memory limits on containers and reset policies instead of learning to do it on the OS level.
- Operyl 2y ago> reached docker in terms of virtualization Except docker, on its own without something else in the stack, isn't Virtualization.
- TZubiri 2y agoNot arguing this again, go edit the wikipedia article if you are so confident
- robertlagrant 2y ago
- manofmanysmiles 2y agoHow about a wireguard tunnel from an ingress box? You still pay for one VPS, but can run everything locally and just load balance at the ingress. I just manually add configs to nginx, but there are automated tools too.
- TZubiri 2y agoLol, kind of defeats the purpose
- yjftsjthsd-h 2y agoWhat defeats what purpose? I don't run k8s out of some love of ... managing external IPs?
- _cenw 2y agoAre you even running a real homelab if you're not running MetalLB in BGP mode?! Sarcasm obviously, but it's a fun exercise, especially if you get a real (v6, probably) net to announce.
- linuxdude314 2y agoRight?! Get some free IP space from he.net tunnel broker and build your own IPv6 AnyCast network using Quagga for BGP. Kidding about building your own AnyCast network (although you really could…), but he.net tunnel broker is GOAT.
- _cenw 2y agoI wish Happy Eyeballs worked better or let me absolutely prefer IPv4, because every time I set up a free v6 tunnel I get banned about 2 days layer for pushing a few terrabytes over it, when all I wanted was to be able to SSH into every one of my containers on a cheap VPS separately. Or the tunnel is so slow, I'm degrading my entire internet connectivity.
- deathanatos 2y ago> what do you do if your ISP changes your IP addresses? I update the DNS record. Manually. It's a once in a blue moon thing, and I assume the probability of it is low enough that it will not occur when I'm so far from home that "it can wait until I get home" doesn't suffice. 15+ years or so now, and that strategy has worked just fine. … TFA's intro could do with explaining why the IP is so hard coded in the cluster, or in the router? My home router just does normal port-forwarding to my home cluster. My cluster … doesn't need to know its own IP? It just uses normal Ingress objects. (And ingress-nginx.) I'm wondering if this is partly complexity introduced by having a |cluster| > 1, and I'm just on duck tales here. Y'all have multiple non-mobile machines? (I have a desktop & a laptop. I'm not running k8s on the laptop… because it's a laptop. I … suppose I could … and deal with connectivity to the desktop with like Wireguard or something but … why?) My previous ISP offered static IP addresses, and I had one, since I had a somewhat special offer where the price wasn't terrible. It changed on me one day. They refused to fix that. I was very disappointed.
- _cenw 2y ago> Manually. It's a once in a blue moon thing, and I assume the probability of it is low enough that it will not occur when I'm so far from home that "it can wait until I get home" doesn't suffice. It's very common here (germany) to be forcefully disconnected and reassigned an IP from the pool once a day here, especially on DSL contracts - and we still got a lot of vectored to the limit DSL. It's in fact so common, that JDownloader has a client for some common routers built-in so it can automatically dodge one-click-hoster IP limits. And our cable internet companies all use CGNAT by now, good luck getting anything through that.
- kukkamario 2y agoMikroTik has dynamic DNS that is based on random unique number for their router. I just point my DNS record to that dynamic DNS address and everything just works.
- rmwaite 2y agoeero also offers this, but only if you pay for their Plus subscription.
- TZubiri 2y ago"My ISP is in total control over my external IP addresses. I don’t pay for permanent IP addresses, and while they haven’t so far changed neither my IPv4 address or my IPv6 network, it can happen. Probably by mistake, since I have no kept my current ones for three months" If you can't shell a buck or persuade your isp to reserve a static ip for you. Try to persuade their dhcp server. https://datatracker.ietf.org/doc/html/rfc2131#section-3.5 https://datatracker.ietf.org/doc/html/rfc2131#section-3.5 And, again, if you can't handle fundamentals, drop the Google level tech. You are not that deep.
- homodyne 2y agoIt's really concerning that we have people trying to use tools like Kubernetes without understanding the basics that underlie them, like networking. Post author should read Beej's Guide to Network Programming and come back when that's comfortable for them.
- vegarde 2y agoAll of that is sort of "best effort" when it comes to the home market. While an ISP should honor it and give you the same IP address (or IPv6 range), at home we're in the consumer market where it's good enough if you're able to hit youtube from your browser. I know my way around networking, I know that my networking equipment will always try to request the same IP address/IPV6 range. But I can't do shit with it if my ISP has lost my reservation and given that to someone else. And as numerous other comments here have shown, some ISPs are incompetent bordering on malicious, but sometimes, they're the best you have. My ISP is pretty decent, but that doesn't keep me from playing with "what if"-scenarios and solve things that may not ever happen for me. I learnt something from it - not the core networking stuff, there is nothing in that that was new to me, but I leart a lot about how to do things more dynamically in Kubernetes, and I had to dig into the APIs of my router to achieve what I needed. However, when I write a blog post, I also try to assume not everyone has a deep understanding of the things. If someone read my blog post and just then learnt about IPv6 addressing, that's great! Then they too would have learnt some things, even if Kubernetes is not for them.
- kuschku 2y ago
- mannyv 2y agoYou update your cluster with your new IP address. How you do that depends on your level of expertise.
- alfons_foobar 2y agoNot even necessary, just update the DNS record pointing to your home address
- mychael 2y agoThis is an example of optimizing something that shouldn't exist. They can simplify all of this by adding Cloudflare tunnel or Wireguard to proxy traffic from the outside world to a k8s Service running in the cluster.
- globular-toast 2y agoUhh. What is all this for? My IP address can change. I just use a dynamic DNS client to update my DNS record using my registrar's API. It's been this way since, like, 2001? (Well, most registrars didn't have APIs back then, but there was dyndns).
- merpkz 2y agoKubernetes admin here with ~2y experience. Since a lot of you have misconception of what this guy is doing I will try to explain. Author wrote a piece of code which will interact with network gateway to get IPv4/IPv6 network address and then update kubernetes configuration accordingly from within a container running on said cluster. That seems to be needed, because MetalLB component in use exposes kubernetes deployments in cluster via predefined IPv6 network address pool which is given from ISP, so if that changes, cluster configuration should change too. This is one of most bizarre things I have read about kubernetes this year and probably shouldn't exist outside a home testing environment, but hey, props to author for coming up with such idea.
- vegarde 2y agoOP here. Thanks. If I was a company, I would probably be in control over when my IPv6 range changes. And if my ISP is any good (I just recently switched to it), my IPv6 network should stay the same. The network range in a home setting is always given by your ISP, most likely with DHCPV6 prefix delegation, very rarely do you in a home setting dish out for a permanent IPv6 network range. Granted, most decent ISPs try to persist it, since there's no good reasons not to, and it's a strong recommendation from standardization bodies etc. But it's still just best effort, accidents happen, state get lost, and suddently you have a different network. Sure, it's probably take me less than an hour to just change everything, but we are hackers here, so what's the fun in that? At least I gravitate towards perfecting things even beyong pure needs, just because I can. At work, I have to call it a day when it gives no more significant gain, at home I am free to think "this is fine, but can I actually do it better?". If the answer is yes, and you have the time, I'd say go for it. Some people like to watch cat videos on Youtube, I prefer to tinker with getting stuff to work. Sometimes it's useful, sometimes it's just for the fun of it. I'm on my way to improve this, by the way. I plan to create a Unifi Networking Operator that can help me not only this, but to configure my Unifi Gateway and firewall rules through Kubernetes properties. It will be more logical to let my "dynamic IP" setup just change Kubernetes properties, and let the OPerator handling the Unifi Configuration of it. Overkill? Hell, yes! Fun? For me, at least? Will I learn something? Yes, I will learn how to create a Kubernetes Operator! Yeah, I'm a beginner in Kubernetes, but not in IT and sysadmining in general, I've got 30 years experience there. For now, Kubernetes is a just-for-fun project at home, but it's used to run my day-to-day home services, which makes it even more fun to improve it. We use Kubernetes where I work, but not in my area, it's not inconcieveable that my home-tinkering will be of benefit at work, some day. And yes. I run a personal blog (in my Kubernetes cluster). I try to make it a bit educational, with more or less repeatable experiments for people to pick and chose from. Some will be good, some will probably be a bad idea. But as long as there's learnings to be had, it's worth doing.
- citizenpaul 2y agoMy experience is this is no longer a problem. Ever since the US gov legalized data mining/spying/tracking I have not had my residential IP change. I think its more profitable to spy by essentially giving "free" static IPs to all customers.
- dharmab 2y agoMore likely your ISP uses CGNAT and your router's IP is not a "real" public IP address.
- from-nibly 2y agoHoly cow, I've been doing kubernetes for 8+ years at this point. No idea why your home IP address would change a single thing in kubernetes.
- advisedwang 2y agoIn the opening sentence the author says they are using external-dns to set outside DNS to point to their cluster. You need the IP address for that. (although they'd be better served by just using dynamic DNS rather than this complexity)
- from-nibly 2y agoBut why would you expose your cluster directly to the internet in the first place?
- vegarde 2y agoIt would if you are running your IPv6 and real IPv6 addresses on your K8s ingress adresses. And I do.
- davkan 2y agoI have one A record for my home ip address. This is dynamically updated by my router whenever the public IP address changes. Everything else is a CNAME pointing at the A record. Completely set and forget and supported by most of the shelf consumer routers or router OS like vyos. This is a much preferable solution to me as there are no changes to external-dns resources when the public IP changes. Granted, i don’t run a dual stack setup.
- vegarde 2y agoI run dual-stack. I used to run a dynamic DNS service, but these days I prefer to do it myself towards the API of my DNS provider. External-DNS in k8s is pretty neat that way. Changing the K8s resources automatically is not really a big deal. It's a fun exercise.
- tamishungry 2y agoHuh? I host my domain with Namecheap and it's a simple curl command to update my DNS daily on my Pi. Why all this?
- david24802 2y agoThanks for the post. I ran into the same issue with assigning IPv6 addresses to k8s pods. Wish there were easier solutions to handle the prefix changing.
- vegarde 2y agoA good advise: Never use a real range for the PODs. I run K3s, and it's not supported to change the internal ranges after creating the cluster. On the ingress, i.e. with metalLB, it's different, of course. And that is what my solution is for. I'm going to improve it in the coming weeks. It's not going to be any easier, though, it will just be more fun for me :) (I like to learn new stuff). It's probably going to be a bit more robust and logical, though, not look as hacked-together than my current script. But unless I actually get around to making a plugin architecture for it, it's still going to assume you're using a Unifi gateway.
- linuxdude314 2y agoThis could all be solved using HE.net tunnel broker for free…
- vegarde 2y agoIt could, but then all my IPv6 traffic would take a detour around the network. My ISP gives me native IPv6. Granted, it hasn't changed yet. I am pretty sure it will, some day. Probably by an accident of mine or the ISP.
- bdhcuidbebe 2y ago> what do you do if your ISP changes your IP addresses? You use dyndns
- Saris 2y agoNothing, inside the network addresses don't change, so K8S will never see anything change even if your external addresses update.