4 ms·

Tailscale is needed if you require site to site connectivity via something like Starlink. I may be putting my ignorance on display here, but I recently complet

Tailscale is needed if you require site to site connectivity via something like Starlink. I may be putting my ignorance on display here, but I recently completed a site-to-site network between two farms in rural America, no other ISP can serve these farms, and they needed to communicate cow data between the different farms. Tailscale did the majority of the heavy lifting thankfully, and we were able to get them all sorted out. I could not get Wireguard to work, and that may be down to my limitations in networking, but I was sucessful with tailscale, so make of that as you will.

howeyc 3y ago
If I had to guess, it's probably the NAT hole-punching or use of tailscale servers as an intermediary. If you signed up for a $5 VPS to forward packets you probably would have been fine.
- toomuchtodo 3y ago
  Tailscale is free for 3 users and up to 100 devices, so it's a fine solution vs running your own VM for hole punching. Consider your time value. You can even pay for Mullvad exit node access for devices in your net and configure the coordination layer accordingly.
  - incahoots 3y ago
    Free was the price, client didn't want to pony up for any additional monthly fees. In the past I would've included it into my monthly "on call fee" but now I value my free time too much to the point where clients aren't interested in having me as an emergency line. I'm sure I'm leaving money on the table, but I'm a one man show, and I have trust issues with relying on others, so here we are.
    - toomuchtodo 3y ago
      Who takes care of the VPS then? You're paying for something somewhere if you're not donating your time for free to the client.
nijave 3y ago
Looks like Starlink supports IPv6 but you'd probably want dynamic DNS updater to keep a DNS record with the correct IP in case it ever changes.
- hossbeast 3y ago
  They will even vend you a /56 if you ask for it, via dhcpv6 IA_PD.
freedomben 3y ago
Lack of Wireguard docs/tutorials is unfortunate, because I'm in the same boat. I'm using Tailscale now because I just don't know enough about linux networking to figure out why my packets aren't going where they should. I suspect it's a very simple config error, but I don't know how to debug or where to look. I like Tailscale, but I'd much rather use real wireguard and eliminate a dependency on tailscale, but I can't find guides/tutorials/etc. If anyone knows of a Wireguard walk-through to bridge two separate LANs together, I'd love to see it
- candiddevmike 3y ago
  Create a wireguard interface on each end, add the keys to them, open up the allow list for each subnet, assign a IP from a /30 (or /31) subnet on each end, set static routes to point to other end or use a dynamic routing protocol. There are tools to automate this like wg-quick and systemd-networkd, depending on your preference.
- drdaeman 3y ago
  > Lack of Wireguard docs/tutorials is unfortunate The thing about Wireguard is that it's very simple and minimal. It does just one thing, and that is - establishes a layer 3 tunnel for sending IP packets between local machine and some other peers. It doesn't do mesh, it doesn't do routing (it just knows the IPs of its direct peers and that's all it does), it doesn't do bridging - all this stuff is done by other pieces such as Linux kernel, but not Wireguard itself. > Wireguard walk-through to bridge two separate LANs Same or different subnets for those LANs? If they're different and non-intersecting, and if you don't need cross-LAN broadcast or multicast, the simplest option is to establish a Wireguard connection between those LAN's default gateway routers (assuming you can do this), and on each of those routers set up a route that sends opposite LAN's traffic to the opposite gateway (in case of iproute2: `ip route add my.other.lan.subnet/mask via my.other.lan.gw`, how to make this persistent depends on your distro). Then, on each gateway, allow packet forwarding between Wireguard and LAN interfaces (with e.g. iptables or nftables or whatever you use there). If you can't run Wireguard on gateways, the overall principle holds, but you'll need to distribute routes to your respective LANs via Wireguard-running routers through DHCP or whatever you use for routing on your LANs (e.g. OSPF). And if your LANs both have the same subnet, or if you need multicast, things get significantly trickier (plus, there's inevitable question of what should happen if two machines on different LANs have the same IP). You'll probably need to run something like L2TP or GRETAP (or something else that can encapsulate you layer 2) over Wireguard. Or maybe just use OpenVPN in TAP mode (if you want all stuff independent of any third parties) or Tailscale (because it already works).
mschuster91 3y ago
> but I recently completed a site-to-site network between two farms in rural America, no other ISP can serve these farms, and they needed to communicate cow data between the different farms. This may be verging on off-topic, but it's too good an opportunity to not deliver this very fitting reminder: in 2018 Anja Karliczek, then-Minister of Science of Germany, made headlines for the quote "5G nicht an jeder Milchkanne notwendig" [1] ("you don't need 5G on every milk jug") aka the rural areas (that had been left behind even with prior mobile phone/data and even landline DSL deployments) didn't have enough need for fast Internet access. So, to answer her question, what exactly are these farms doing that they need to exchange cow metrics? Tracking of milk production per cow? [1] https://www.spiegel.de/politik/deutschland/anja-karliczek-bremst-beim-netzausbau-5g-nicht-an-jeder-milchkanne-notwendig-a-450a7e4e-3e18-49d3-8132-c87b71c9b295 https://www.spiegel.de/politik/deutschland/anja-karliczek-br...
- incahoots 3y ago
  That's a fair question, I assume it's milk production tracking, health status of each cow, etc. That's a bit out of my wheel house. This was previously 2 different farms owned by two different families, but after 2020 a lot of these places sold off their farms due to financial hardships. I lost a ton of clientele over the course of 2020 due to mega farms buying them out for pennies on the dollar.
0x457 3y ago
I don't think you need Tailscale for this. You need tailscale when: 1) You have a lot of road-warriors 2) You need decent ACLs that isn't just IP based. 3a) You don't ability to directly connect to a site (just one needs to be accessible) 3b) You can't add 3rd site to relay traffic or punch holes in NAT. 4) You're afraid of terminals and/or text editors. Here is a good write-up on the subject: https://www.procustodibus.com/blog/2021/11/wireguard-nftables/ https://www.procustodibus.com/blog/2021/11/wireguard-nftable...
jabroni_salad 3y ago
> no other ISP I live in Iowa and the problem we have is nobody markets their stuff very well. We have a fixed wireless provider that, looking at their website, looks like they are just a cell tower climbing service. But they also lease space on just about every tall structure in the region (grain elevators etc) and can connect anything to anything. We got to them because we asked a WISP if they would do a private backhaul and the guy referred us over to somebody's cell phone. I guess they are happy to just have word of mouth and only have the website to reel in bonus work with the out of town telcos. Never had worse than next day service on any lightning strike event from this guy, which is more than I can say for mediacom or lumen lol.
bct 3y ago
Starlink's IPv4 is behind CGNAT, and their IPv6 prefixes are not stable. The good news is that it's relatively easy to punch through the NAT. This blog post describes an approach that is working well for me: https://www.jordanwhited.com/posts/wireguard-endpoint-discovery-nat-traversal/ https://www.jordanwhited.com/posts/wireguard-endpoint-discov...
- incahoots 3y ago
  I'll take a look at this, thanks for the link!
Bluecobra 3y ago
> Tailscale is needed if you require site to site connectivity via something like Starlink. You should be able setup a Wireguard instance on some cloud provider like AWS, then setup each Starlink endpoint to point to that public cloud IP. I have done this for some time on a cheap $5/mo Digital Ocean droplet to work around my old ISP giving me a RFC to 1918 addresses. Also don’t forget that if you have a line of sight between the two farms you can use the unlicensed 60Ghz spectrum and grab some cheap Ubiquiti millimeter wave gear.
wscott 3y ago
The beauty of Starlink is while they use a cgnat for ipv4 they assign you fully routable ipv6 addresses. So Tailscale has no problem making that bridge, it just uses ipv6.

howeyc 3y ago

If I had to guess, it's probably the NAT hole-punching or use of tailscale servers as an intermediary. If you signed up for a $5 VPS to forward packets you probably would have been fine.

toomuchtodo 3y ago
Tailscale is free for 3 users and up to 100 devices, so it's a fine solution vs running your own VM for hole punching. Consider your time value. You can even pay for Mullvad exit node access for devices in your net and configure the coordination layer accordingly.
- incahoots 3y ago
  Free was the price, client didn't want to pony up for any additional monthly fees. In the past I would've included it into my monthly "on call fee" but now I value my free time too much to the point where clients aren't interested in having me as an emergency line. I'm sure I'm leaving money on the table, but I'm a one man show, and I have trust issues with relying on others, so here we are.
  - toomuchtodo 3y ago
    Who takes care of the VPS then? You're paying for something somewhere if you're not donating your time for free to the client.

nijave 3y ago

Looks like Starlink supports IPv6 but you'd probably want dynamic DNS updater to keep a DNS record with the correct IP in case it ever changes.

hossbeast 3y ago
They will even vend you a /56 if you ask for it, via dhcpv6 IA_PD.

freedomben 3y ago

Lack of Wireguard docs/tutorials is unfortunate, because I'm in the same boat. I'm using Tailscale now because I just don't know enough about linux networking to figure out why my packets aren't going where they should. I suspect it's a very simple config error, but I don't know how to debug or where to look. I like Tailscale, but I'd much rather use real wireguard and eliminate a dependency on tailscale, but I can't find guides/tutorials/etc. If anyone knows of a Wireguard walk-through to bridge two separate LANs together, I'd love to see it

candiddevmike 3y ago
Create a wireguard interface on each end, add the keys to them, open up the allow list for each subnet, assign a IP from a /30 (or /31) subnet on each end, set static routes to point to other end or use a dynamic routing protocol. There are tools to automate this like wg-quick and systemd-networkd, depending on your preference.
drdaeman 3y ago
> Lack of Wireguard docs/tutorials is unfortunate The thing about Wireguard is that it's very simple and minimal. It does just one thing, and that is - establishes a layer 3 tunnel for sending IP packets between local machine and some other peers. It doesn't do mesh, it doesn't do routing (it just knows the IPs of its direct peers and that's all it does), it doesn't do bridging - all this stuff is done by other pieces such as Linux kernel, but not Wireguard itself. > Wireguard walk-through to bridge two separate LANs Same or different subnets for those LANs? If they're different and non-intersecting, and if you don't need cross-LAN broadcast or multicast, the simplest option is to establish a Wireguard connection between those LAN's default gateway routers (assuming you can do this), and on each of those routers set up a route that sends opposite LAN's traffic to the opposite gateway (in case of iproute2: `ip route add my.other.lan.subnet/mask via my.other.lan.gw`, how to make this persistent depends on your distro). Then, on each gateway, allow packet forwarding between Wireguard and LAN interfaces (with e.g. iptables or nftables or whatever you use there). If you can't run Wireguard on gateways, the overall principle holds, but you'll need to distribute routes to your respective LANs via Wireguard-running routers through DHCP or whatever you use for routing on your LANs (e.g. OSPF). And if your LANs both have the same subnet, or if you need multicast, things get significantly trickier (plus, there's inevitable question of what should happen if two machines on different LANs have the same IP). You'll probably need to run something like L2TP or GRETAP (or something else that can encapsulate you layer 2) over Wireguard. Or maybe just use OpenVPN in TAP mode (if you want all stuff independent of any third parties) or Tailscale (because it already works).

mschuster91 3y ago

> but I recently completed a site-to-site network between two farms in rural America, no other ISP can serve these farms, and they needed to communicate cow data between the different farms. This may be verging on off-topic, but it's too good an opportunity to not deliver this very fitting reminder: in 2018 Anja Karliczek, then-Minister of Science of Germany, made headlines for the quote "5G nicht an jeder Milchkanne notwendig" [1] ("you don't need 5G on every milk jug") aka the rural areas (that had been left behind even with prior mobile phone/data and even landline DSL deployments) didn't have enough need for fast Internet access. So, to answer her question, what exactly are these farms doing that they need to exchange cow metrics? Tracking of milk production per cow? [1] https://www.spiegel.de/politik/deutschland/anja-karliczek-bremst-beim-netzausbau-5g-nicht-an-jeder-milchkanne-notwendig-a-450a7e4e-3e18-49d3-8132-c87b71c9b295 https://www.spiegel.de/politik/deutschland/anja-karliczek-br...

incahoots 3y ago
That's a fair question, I assume it's milk production tracking, health status of each cow, etc. That's a bit out of my wheel house. This was previously 2 different farms owned by two different families, but after 2020 a lot of these places sold off their farms due to financial hardships. I lost a ton of clientele over the course of 2020 due to mega farms buying them out for pennies on the dollar.

0x457 3y ago

I don't think you need Tailscale for this. You need tailscale when: 1) You have a lot of road-warriors 2) You need decent ACLs that isn't just IP based. 3a) You don't ability to directly connect to a site (just one needs to be accessible) 3b) You can't add 3rd site to relay traffic or punch holes in NAT. 4) You're afraid of terminals and/or text editors. Here is a good write-up on the subject: https://www.procustodibus.com/blog/2021/11/wireguard-nftables/ https://www.procustodibus.com/blog/2021/11/wireguard-nftable...

jabroni_salad 3y ago

> no other ISP I live in Iowa and the problem we have is nobody markets their stuff very well. We have a fixed wireless provider that, looking at their website, looks like they are just a cell tower climbing service. But they also lease space on just about every tall structure in the region (grain elevators etc) and can connect anything to anything. We got to them because we asked a WISP if they would do a private backhaul and the guy referred us over to somebody's cell phone. I guess they are happy to just have word of mouth and only have the website to reel in bonus work with the out of town telcos. Never had worse than next day service on any lightning strike event from this guy, which is more than I can say for mediacom or lumen lol.

bct 3y ago

Starlink's IPv4 is behind CGNAT, and their IPv6 prefixes are not stable. The good news is that it's relatively easy to punch through the NAT. This blog post describes an approach that is working well for me: https://www.jordanwhited.com/posts/wireguard-endpoint-discovery-nat-traversal/ https://www.jordanwhited.com/posts/wireguard-endpoint-discov...

incahoots 3y ago
I'll take a look at this, thanks for the link!

Bluecobra 3y ago

> Tailscale is needed if you require site to site connectivity via something like Starlink. You should be able setup a Wireguard instance on some cloud provider like AWS, then setup each Starlink endpoint to point to that public cloud IP. I have done this for some time on a cheap $5/mo Digital Ocean droplet to work around my old ISP giving me a RFC to 1918 addresses. Also don’t forget that if you have a line of sight between the two farms you can use the unlicensed 60Ghz spectrum and grab some cheap Ubiquiti millimeter wave gear.

wscott 3y ago

The beauty of Starlink is while they use a cgnat for ipv4 they assign you fully routable ipv6 addresses. So Tailscale has no problem making that bridge, it just uses ipv6.