8 ms·

I really like wireguard, but one thing that bugs me is the fact that it's layer 3 (an ip tunnel) and has no code to support layer 2 (ethernet MAC tunnel). The d

I really like wireguard, but one thing that bugs me is the fact that it's layer 3 (an ip tunnel) and has no code to support layer 2 (ethernet MAC tunnel). The downside for me is that you have to manage static ips in the configurations (specifically it's not compatible with ipv6 slaac and NDP). There is https://git.zx2c4.com/wg-dynamic https://git.zx2c4.com/wg-dynamic but it's very experimental at the moment. The level 3-only tunnel is motivated as "the cleanest approach for ensuring authenticity and attributability of the packets" (in the whitepaper), but in fact every claim and routing algorithm described (needed since the tunnel is many-to-one) would work equally well substituting "ip address" with "mac address" (i may be missing something, but for sure it's not explicit anywhere). And indeed imho it would be less surprising to have an "allowed mac address" option in the configuration than an "allowed ip address": it's already common practice to white-list mac address of physical endpoints (in office). I'm toying with the idea of forking the driver code to adapt it to ethernet frames as i don't think it would need any big rewrite but i'm realizing my inexperience in writing kernel code.

jbverschoor 7y ago
Zerotier does a perfect job ..
- EmoSupportEmu 7y ago
  Agreed. That's my daily driver. No pun intended.
- v0tary 7y ago
  I disagree - spent a considerable amount of time with zerotier as a possible replacement of a small sized ipsec mesh (4 sites) and it failed horribly. Had commercial support, different hardware and even virtualized it. Latency was a major issue and quality of the links were erratic to say the least. Don't get me wrong, I think zerotier is great, but it's not prime time.
  - grumblez 7y ago
    Your physical network settings likely didn't allow direct connections between peers. Fix that and it'll work fine.
    - kortilla 7y ago
      If it’s replacing an ipsec mesh that’s pretty hard to believe. And if that was the issue and commercial support couldn’t even identify that as the cause, ZeroTier has bigger issues.
      - grumblez 7y ago
        If all sites are behind symmetric NATs, there's not much ZeroTier could do to help aside from telling him to assign direct mappings on the NAT/Firewall to each ZT instance. Symmetric NATs are antithetical to peer to peer communication. Many I've run across in the wild have special rules to handle IPSec which won't exist for other lesser known protocols. It's also possible the user wasn't willing or able to make network configuration changes to make those p2p connections possible. Without seeing what the user tried & support recommended, it's not really fair to throw out such baseless accusations.
        pathseeker 7y ago
        ZeroTier uses UDP. That's hardly "lesser known" than IPSec.
        grumblez 7y ago
        "lesser known" as in protocols such as IPSec, ZeroTier, WireGuard, etc. Of which IPSec has been around forever and many NATs/Firewalls have special handling rules built in, just as @api mentioned in another comment. Yes, ZeroTier uses UDP underneath, but that doesn't mean symmetric NATs don't/won't cause havoc to peer to peer protocols using UDP.
        7y ago
- lapinot 7y ago
  Yeah, i hear about that regularly but didn't look into it. I must say i'm not really happy about the whole business thing. The arch wiki says you need an account, i'm not sure if that is true but if it is, it's a non-starter for me. If you have good technical refs to prove me wrong i'd be happy to hear.
  - grumblez 7y ago
    You only need an account if you're using ZeroTier's hosted network controller service. You can run your own controller as well. The code to do so is in every instance of ZeroTier. https://github.com/zerotier/ZeroTierOne/tree/master/controller https://github.com/zerotier/ZeroTierOne/tree/master/controll...
cpach 7y ago
Out of curiosity, what is the use case for doing VPN at layer 2?
- kortilla 7y ago
  Broadcast/multicast propagation.
  - rattray 7y ago
    Can you say more to explain the utility of this to those unfamiliar with networking?
    - nightfly 7y ago
      You can send data to N hosts by only sending it once, rather than N times.
    - pathseeker 7y ago
      Many device discovery protocols work by sending out broadcast or multicast packets (either to announce themselves to devices who might be listening or to request devices to send them data). These packets are expected to go out to either everyone on the same layer-2 network (the broadcast case) or everyone who has subscribed to a particular multicast address (the multicast case). In addition to device discovery, these are frequently used for heartbeat messages to indicate that you are still alive (for high-availability protocols like VRRP).
      - rattray 7y ago
        Thanks! That's helpful/interesting and makes sense
    - thayne 7y ago
      One common use case is multicast DNS (https://en.wikipedia.org/wiki/Multicast_DNS https://en.wikipedia.org/wiki/Multicast_DNS) which uses multicast for individual hosts to publish services available on them to other hosts on the network without needing a dedicated DNS server.
api 7y ago
Substituting MAC for IP address is exactly what ZeroTier does. MACs can't be spoofed, though nodes can be designated as bridges and that allows them to impersonate MACs. There's still the issue of authenticating IPv4 IPs though, which are too small to embed anything useful into. ZeroTier has a certificate system for that but it requires the use of the rules engine to enable it.
- alanpca 7y ago
  Can you not set your MAC to whatever you want on many platforms?
  - nightfly 7y ago
    I think they're implying that MACs are authenticated the same way IP addresses are authenticated with Wireguard (you can say "only 52:54:00:7a:cc:dd can talk over this connection").
- AceJohnny2 7y ago
  > MACs can't be spoofed lol 20 years ago, in college, some folks in the dorm had fun fucking with others at the Ethernet level. Most of us only had experience with the IP level, so couldn't understand what was going on.
  - swinglock 7y ago
    ZeroTier doesn't permit spoofed MACs unless explicitly configured.
    - AceJohnny2 7y ago
      and how can they tell?
      - api 7y ago
        MACs are computed directly from cryptographic hashes. For normal ZeroTier P2P traffic the MAC and Ethernet header are elided entirely too, which saves about 14 bytes of per-frame overhead.
wmf 7y ago
It should be possible to run GRE, L2TP, or VXLAN over WireGuard although such tooling probably doesn't exist yet.
- ryanlol 7y ago
  What tooling do you need for this? Shell scripts would be the traditional approach.
  - wmf 7y ago
    Yeah, as long as the shell script has been audited. It would probably be easy to accidentally send the GRE traffic in the clear instead of through WireGuard.
    - ryanlol 7y ago
      Doesn’t seem that easy to fuck up, you just use internal WG IPs for the L2 tunnel. Packets just won’t go anywhere if WG is down.
- lapinot 7y ago
  Sure but it hurts a bit to run a tunnel on top of another tunnel, and since you have to run wireguard as-is, you still have to do the static ip thing. It's a bit insane to have ethernet > udp (l2tp) > ip > udp (wireguard) > ip > ethernet. That's at least 128 bytes overhead per frame (udp/ip: 2*48, l2tp: 4, eth: 14, wireguard: 14).
- bitbang 7y ago
  I've run VXLAN over top of wireguard connections. One advantage is that you can have multiple intermediate wireguard connections that are not visible at the VXLAN layer.
BlueTemplar 7y ago
Hmm, but MACs (and Ethernet?) are on their way out with IPv6, replaced by specifically software-set GUIDs ?
- ffk 7y ago
  Most IPv6 packets are encapsulated in Ethernet frames which use MAC addresses. What may change is the vendor specific MAC address could be replaced with a MAC address generated by a cryptographic hash to preserve privacy.
  - tinus_hn 7y ago
    Privacy from what? MAC addresses are not used on the internet, only on the local network
    - ffk 7y ago
      Not all networks are private. Also, many IPv6 addresses contain the MAC address which would effectively deanonymize you over the internet.
      - tinus_hn 7y ago
        So instead of not using the MAC address in the IPv6 (which any reasonably modern OS does because this problem is old, well known and trivially solved) you get rid of MAC addresses altogether? Just so you can have some illusion of privacy while sending your traffic through a supposedly compromised network? This is paranoia about all the wrong things, focusing on irrelevant details and ignoring what’s important.
        ffk 7y ago
        I don't believe I suggested getting rid of the MAC address altogether. Ethernet isn't going away anytime soon. The suggestion is simply this: Don't embed your MAC address into your IPv6 address because it's a unique identifier that can deanonymize you even if you shift networks.
ffk 7y ago
In most scenarios, you want to avoid L2 tunnels to reduce complexity and/or performance issues. The chain of thought typically goes like this: * Remote networks are connected via L2 tunnel. * ARP requests are broadcasted over L2 tunnel to all connected networks, introducing scalability issues * Proxy ARP is introduced to cache ARP responses * Proxy ARP may become out of date or not scale as the L2 domain grows. * BGP is introduced to keep track of and broadcast all topology changes * How do you mitigate issues caused if Proxy ARP fails? Most of these issues go away if you use IP tunnels instead of Ethernet because IP was designed to be routable. For your point on security... Whitelisting MAC addresses doesn't provide security. These are trivial to spoof. Same with IP. Please start relying on cryptographic primitives to establish workload identity instead. I highly suggest looking at SPIFFE to get started here. If you must send L2 over the VPN, please go use a L2 EVPN which is designed to handle the complexity and provide fault tolerance. There are numerous SDNs out there you can use to implement this including Tungsten Fabric and OpenDaylight. No need to complicate Wireguard to support EVPN. [edited to improve formatting of bullets and clarity of wording]
thayne 7y ago
I don't have a problem with it being layer 3 rather than layer 2. But the lack of dynamic configuration is a bit of an issue. I don't care too much about needing static ip addresses, but I do want to be able to push down dynamic routes and dns servers to clients.
- nekeo0 7y ago
  That it doesn’t act as a DHCP server or anything but what it’s for is my main reason for loving it. Logical functionality should be decoupled to ease maintenance, reduce risk from complexity. Config management tools can dump some text in a file for corporate needs. Apply it over SSH or via git. It’s just a bit of text to lay down.

jbverschoor 7y ago

Zerotier does a perfect job ..

EmoSupportEmu 7y ago
Agreed. That's my daily driver. No pun intended.
v0tary 7y ago
I disagree - spent a considerable amount of time with zerotier as a possible replacement of a small sized ipsec mesh (4 sites) and it failed horribly. Had commercial support, different hardware and even virtualized it. Latency was a major issue and quality of the links were erratic to say the least. Don't get me wrong, I think zerotier is great, but it's not prime time.
- grumblez 7y ago
  Your physical network settings likely didn't allow direct connections between peers. Fix that and it'll work fine.
  - kortilla 7y ago
    If it’s replacing an ipsec mesh that’s pretty hard to believe. And if that was the issue and commercial support couldn’t even identify that as the cause, ZeroTier has bigger issues.
    - grumblez 7y ago
      If all sites are behind symmetric NATs, there's not much ZeroTier could do to help aside from telling him to assign direct mappings on the NAT/Firewall to each ZT instance. Symmetric NATs are antithetical to peer to peer communication. Many I've run across in the wild have special rules to handle IPSec which won't exist for other lesser known protocols. It's also possible the user wasn't willing or able to make network configuration changes to make those p2p connections possible. Without seeing what the user tried & support recommended, it's not really fair to throw out such baseless accusations.
      - pathseeker 7y ago
        ZeroTier uses UDP. That's hardly "lesser known" than IPSec.
        grumblez 7y ago
        "lesser known" as in protocols such as IPSec, ZeroTier, WireGuard, etc. Of which IPSec has been around forever and many NATs/Firewalls have special handling rules built in, just as @api mentioned in another comment. Yes, ZeroTier uses UDP underneath, but that doesn't mean symmetric NATs don't/won't cause havoc to peer to peer protocols using UDP.
        7y ago
lapinot 7y ago
Yeah, i hear about that regularly but didn't look into it. I must say i'm not really happy about the whole business thing. The arch wiki says you need an account, i'm not sure if that is true but if it is, it's a non-starter for me. If you have good technical refs to prove me wrong i'd be happy to hear.
- grumblez 7y ago
  You only need an account if you're using ZeroTier's hosted network controller service. You can run your own controller as well. The code to do so is in every instance of ZeroTier. https://github.com/zerotier/ZeroTierOne/tree/master/controller https://github.com/zerotier/ZeroTierOne/tree/master/controll...

cpach 7y ago

Out of curiosity, what is the use case for doing VPN at layer 2?

kortilla 7y ago
Broadcast/multicast propagation.
- rattray 7y ago
  Can you say more to explain the utility of this to those unfamiliar with networking?
  - nightfly 7y ago
    You can send data to N hosts by only sending it once, rather than N times.
  - pathseeker 7y ago
    Many device discovery protocols work by sending out broadcast or multicast packets (either to announce themselves to devices who might be listening or to request devices to send them data). These packets are expected to go out to either everyone on the same layer-2 network (the broadcast case) or everyone who has subscribed to a particular multicast address (the multicast case). In addition to device discovery, these are frequently used for heartbeat messages to indicate that you are still alive (for high-availability protocols like VRRP).
    - rattray 7y ago
      Thanks! That's helpful/interesting and makes sense
  - thayne 7y ago
    One common use case is multicast DNS (https://en.wikipedia.org/wiki/Multicast_DNS https://en.wikipedia.org/wiki/Multicast_DNS) which uses multicast for individual hosts to publish services available on them to other hosts on the network without needing a dedicated DNS server.

api 7y ago

Substituting MAC for IP address is exactly what ZeroTier does. MACs can't be spoofed, though nodes can be designated as bridges and that allows them to impersonate MACs. There's still the issue of authenticating IPv4 IPs though, which are too small to embed anything useful into. ZeroTier has a certificate system for that but it requires the use of the rules engine to enable it.

alanpca 7y ago
Can you not set your MAC to whatever you want on many platforms?
- nightfly 7y ago
  I think they're implying that MACs are authenticated the same way IP addresses are authenticated with Wireguard (you can say "only 52:54:00:7a:cc:dd can talk over this connection").
AceJohnny2 7y ago
> MACs can't be spoofed lol 20 years ago, in college, some folks in the dorm had fun fucking with others at the Ethernet level. Most of us only had experience with the IP level, so couldn't understand what was going on.
- swinglock 7y ago
  ZeroTier doesn't permit spoofed MACs unless explicitly configured.
  - AceJohnny2 7y ago
    and how can they tell?
    - api 7y ago
      MACs are computed directly from cryptographic hashes. For normal ZeroTier P2P traffic the MAC and Ethernet header are elided entirely too, which saves about 14 bytes of per-frame overhead.

wmf 7y ago

It should be possible to run GRE, L2TP, or VXLAN over WireGuard although such tooling probably doesn't exist yet.

ryanlol 7y ago
What tooling do you need for this? Shell scripts would be the traditional approach.
- wmf 7y ago
  Yeah, as long as the shell script has been audited. It would probably be easy to accidentally send the GRE traffic in the clear instead of through WireGuard.
  - ryanlol 7y ago
    Doesn’t seem that easy to fuck up, you just use internal WG IPs for the L2 tunnel. Packets just won’t go anywhere if WG is down.
lapinot 7y ago
Sure but it hurts a bit to run a tunnel on top of another tunnel, and since you have to run wireguard as-is, you still have to do the static ip thing. It's a bit insane to have ethernet > udp (l2tp) > ip > udp (wireguard) > ip > ethernet. That's at least 128 bytes overhead per frame (udp/ip: 2*48, l2tp: 4, eth: 14, wireguard: 14).
bitbang 7y ago
I've run VXLAN over top of wireguard connections. One advantage is that you can have multiple intermediate wireguard connections that are not visible at the VXLAN layer.

BlueTemplar 7y ago

Hmm, but MACs (and Ethernet?) are on their way out with IPv6, replaced by specifically software-set GUIDs ?

ffk 7y ago
Most IPv6 packets are encapsulated in Ethernet frames which use MAC addresses. What may change is the vendor specific MAC address could be replaced with a MAC address generated by a cryptographic hash to preserve privacy.
- tinus_hn 7y ago
  Privacy from what? MAC addresses are not used on the internet, only on the local network
  - ffk 7y ago
    Not all networks are private. Also, many IPv6 addresses contain the MAC address which would effectively deanonymize you over the internet.
    - tinus_hn 7y ago
      So instead of not using the MAC address in the IPv6 (which any reasonably modern OS does because this problem is old, well known and trivially solved) you get rid of MAC addresses altogether? Just so you can have some illusion of privacy while sending your traffic through a supposedly compromised network? This is paranoia about all the wrong things, focusing on irrelevant details and ignoring what’s important.
      - ffk 7y ago
        I don't believe I suggested getting rid of the MAC address altogether. Ethernet isn't going away anytime soon. The suggestion is simply this: Don't embed your MAC address into your IPv6 address because it's a unique identifier that can deanonymize you even if you shift networks.

ffk 7y ago

In most scenarios, you want to avoid L2 tunnels to reduce complexity and/or performance issues. The chain of thought typically goes like this: * Remote networks are connected via L2 tunnel. * ARP requests are broadcasted over L2 tunnel to all connected networks, introducing scalability issues * Proxy ARP is introduced to cache ARP responses * Proxy ARP may become out of date or not scale as the L2 domain grows. * BGP is introduced to keep track of and broadcast all topology changes * How do you mitigate issues caused if Proxy ARP fails? Most of these issues go away if you use IP tunnels instead of Ethernet because IP was designed to be routable. For your point on security... Whitelisting MAC addresses doesn't provide security. These are trivial to spoof. Same with IP. Please start relying on cryptographic primitives to establish workload identity instead. I highly suggest looking at SPIFFE to get started here. If you must send L2 over the VPN, please go use a L2 EVPN which is designed to handle the complexity and provide fault tolerance. There are numerous SDNs out there you can use to implement this including Tungsten Fabric and OpenDaylight. No need to complicate Wireguard to support EVPN. [edited to improve formatting of bullets and clarity of wording]

thayne 7y ago

I don't have a problem with it being layer 3 rather than layer 2. But the lack of dynamic configuration is a bit of an issue. I don't care too much about needing static ip addresses, but I do want to be able to push down dynamic routes and dns servers to clients.

nekeo0 7y ago
That it doesn’t act as a DHCP server or anything but what it’s for is my main reason for loving it. Logical functionality should be decoupled to ease maintenance, reduce risk from complexity. Config management tools can dump some text in a file for corporate needs. Apply it over SSH or via git. It’s just a bit of text to lay down.