7 ms·

Netlink Added to FreeBSD – Unmodified Linux IP(8) Correctly Works

113 points by vermaden 4y ago

gclawes 4y ago
Oh may, I pray for this to be added to macOS. The amount of times I reach for 'ip ...' and am stymied...
- NegativeLatency 4y ago
  I'm not familiar with this, can you recommend a good source to read up on it (preferably something more engaging than a manpage)?
  - jandrese 4y ago
    The ip command has a built-in help system you can get started with "ip help", but it tends to throw a lot of fugly BNF at you and is pretty light on detail. The ip manpage is much better. It is broken up into sections which is a bummer for searching but does make it a little less daunting for reading. The nice thing about it is that since it has been around for many years now you can google search most any question and find the answer. All in all the ip command is fairly straightforward and orthogonal which is nice. Some starting commands "ip addr show" lists interfaces and their stats. It's harder to read than ifconfig -a but gives you the same information in a similar format. "ip route show" replaces netstat -rn to show the routing table but is also not as well formatted as the old command. "ip neigh show" replaces arp -a and is about the same level of legibility. "ip route add ..." replaces "route ..." command and is about the same to use. So basically it combines a bunch of commands (including ifconfig) into one and gives them all a unified interface.
    - gerdesj 4y ago
      ip is way more useful than your summary! It also supports abbreviations just like a switch does which is nice: $ip a $ip a a 10.99.0.111/24 dev re0 $ip r $ip r a default via 10.99.0.1 dev re0 The second command is: ip address add. The third is show me the routing table - ip4 is the default add -6 for ipv6. The fourth is add a default route. $ip n s That is neigh(bour) show - the arp table. On Linux, ipconfig and co are destined to quietly go away. Use ip and co instead - so much more powerful.
      - laumars 4y ago
        The abbreviation convention on networking gear isn’t a good thing. It allows for typos to become completely different command, thus introducing unexpected behaviour. I’ve heard so many horror stories (and had some of my own too) because of this.
        gerdesj 4y ago
        I don't agree with you. It is a convenience but you do need to be careful. There are few (if any) commands that can be morphed into another to cause horror. All switch command completers I know of will not complete on ambiguity. If you feel you are personally a bit random then use the try/commit mode that most decent gear allows. You package up a set of changes and try them out. You then commit them within an agreed time or the changes revert.
        laumars 4y ago
        As I said, I’ve personally been bit by feature and know of others who have. So I’m not talking about a theoretical risk. Things might have improved since I was last managing Cisco gear. try/commit wasn’t available back then and maybe those stupid commands (like “administrator-full-port-shutdown”) have been since removed to prevent ambiguous shortenings. But I did quickly learn that it’s better to be explicit rather than implicit. And frankly, I still think that’s good advice given the IaC methodologies that are now commonly practiced. If nothing else, you should be writing your config so that others can read it, and storing that config in version control.
    - iokanuon 4y ago
      >"ip addr show" lists interfaces and their stats. It's harder to read than ifconfig -a but gives you the same information in a similar format. It's worth noting that `-br` gives you a brief version of the output, and `-c` colors it, so that it's possible to do something like this: ip -br -c addr to get a very readable and colorful table-like output. There's also `-j` for JSON, and `-j -p` for pretty JSON.
    - icehawk 4y ago
      There's some caveats to comparing it to linux ifconfig and netstat: ifconfig can't express the idea that an interface can have IPv4 address, and is inconsistent with itself because it does show all of the IPv6 addresses. netstat (and route because you can't manipulate the main RIB with netstat, so now you have two problems-- er-- tools) completely elides things like source address selection as a function of the RIB, and the fact that the kernel can have more than one routing table.
      - eptcyka 4y ago
        Could you elaborate on the Mach kernel having multiple routing tables?
  - nixcraft 4y ago
    I have "Linux ip Command Examples"[1] page that explains how to use the ip command to display and configure the network parameters for host interfaces. HTH. [1] https://www.cyberciti.biz/faq/linux-ip-command-examples-usage-syntax/ https://www.cyberciti.biz/faq/linux-ip-command-examples-usag...
    - jaredhallen 4y ago
      I know ip is not only the future, but very much the present. But damn if I can ditch my ifconfig/route habits.
  - ipdashc 4y ago
    The other replies already answered you, but I just wanted to add: If you pass the -c flag to `ip`, like `ip -c addr`, it colors it! Makes it orders of magnitude easier to work with, IMO. I don't know why this isn't publicized more widely. Hence my username :)
- mhio 4y ago
  There's a python script that does a passable impression of `ip` brew install iproute2mac https://github.com/brona/iproute2mac https://github.com/brona/iproute2mac
kasabali 4y ago
Why would they want to run that abomination in FreeBSD?
- kevans91 4y ago
  IMO it was just a relatable example of a somewhat extensive user of netlink that demonstrates compatibility. Some folks may want to run it under the Linuxolator, which seems reasonable. I only have limited knowledge about netlink, all coming from a devsummit presentation by Alexander, but it seems like an actually good generic kernel interface to have.
  - stefan_ 4y ago
    It really isn't. It's like someone wanted to add protobufs for kernel interfaces, but everything has not one tenth of the usability. In fact it's the kind of thing someone would design who wanted to make it's use in C maximally painful and error prone.
    - eqvinox 4y ago
      Working with both netlink and protobufs in C for 10+ years, I gotta say the latter is much more painful than the former. Error prone I would agree though, mostly because it's too easy to implement netlink… and get it wrong in the process. NB: my experience specifically relates to C. Protobuf's bindings to other languages are better.
- chris_wot 4y ago
  What is wrong with it?
  - f1shy 4y ago
    See my answer to the parent comment.
- f1shy 4y ago
  That was exactly my thought, specially after reading another answer: > "ip addr show" lists interfaces and their stats. It's harder to read than ifconfig -a but gives you the same information in a similar format. > "ip route show" replaces netstat -rn to show the routing table but is also not as well formatted as the old command. > So basically it combines a bunch of commands (including ifconfig) into one and gives them all a unified interface. To me sounds like same but worse, and combining 10 commands into 1 seems to be totally against the unix philosophy "it should do one thing, and do it properly"
Melatonic 4y ago
Serious question - what do you all think of specifically (now) specializing in FreeBSD to work on either virtual networking stacks or storages? I have a good amount of experience already and it seems quite niche compared to general DevOps work but also does seem like a lot of big companies are using FreeBSD based setups. Dead end career or ?
- pdntspa 4y ago
  I would like to know too, I have been enjoying working with the OS and really like it for what it is, and also as a potential alternative to the Docker Devops hegemony BSD in general seems like a potential pivot into embedded space too, there's a lot of interesting electronics out there that run it
- fanf2 4y ago
  It has some advantages in that derivatives of the BSD networking code turns up all over the place, so it can be useful to know your way around it - and the Richard Stevens TCP/IP illustrated books are a very helpful guide. But I wouldn’t deliberately specialize: get a job working with BSD or Linux, and learn by doing; but keep an eye on the other kernels to see how they do it, and to get a broader basis for your skills.
- CoolCold 4y ago
  I'll reply in that way - their way of thinking is still in Sysadmins field, for good or bad. They dismiss/don't realize that the world has shifted for Dev guys doing bit of Ops and all that DevOps things. I see it used by more and more very special cases (this one from Meta employee who did networking at Yandex before), but out of general usage. Just over this year: https://redd.it/x3ewxf - IBM Cloud ditches FreeBSD https://redd.it/wf7h34 - Hetzner drops FreeBSD support https://redd.it/ui9z0m - Digital Ocean doing the same
  - nix23 4y ago
    >https://redd.it/wf7h34 https://redd.it/wf7h34 - Hetzner drops FreeBSD support Not true, i send them an email about it, the Answer was kind of: They had problem's with the Boot process on some machines, however you can send them the image and they will put i on a USB-Stick then open a web-terminal for you..for free, they still support FreeBSD but not in the Rescue-Media-Way. BTW thanks for the great Answer from Robin...Hetzner is still great ;)
yayforfreebsd 4y ago
FreeBSD never ceases to amaze. You can run Linux binaries with the emulation layer, you can of course run many Linux applications through the ports system, they also have a Linux compatibility layer for the graphics stack so they can use Linux drivers, their ZFS implementation is now tracking ZFS on Linux, and now this! This is something other BSD systems don't get. OpenBSD developers pretend Linux doesn't even exist, while FreeBSD is doing a great job catching up with Linux and doing all the Linux stuff so you can Linuxy Linux your Linux stuff in your Linux FreeBSD Linux. Linux.
- yjftsjthsd-h 4y ago
  > their ZFS implementation is now tracking ZFS on Linux I don't think that's quite right. OpenZFS is a thing that is consumed (and pushed forward) by both FreeBSD and Linux; FreeBSD isn't following Linux, they just both use the same thing. (Which is why it isn't named ZoL anymore; it isn't supposed to be Linux-centric)
  - nix23 4y ago
    Correct! It was created out of the ZoL project (because most people worked there instead of the illumos base) and now it transformed into OpenZFS with the Goal: >brings together developers from the Linux, FreeBSD, illumos, MacOS, and Windows platforms. OpenZFS is supported by a wide range of companies. https://openzfs.org/wiki/Main_Page https://openzfs.org/wiki/Main_Page
- antonios 4y ago
  FreeBSD is doing a great job catching up to Linux, yes, (not in the ZFS case though where they were the pioneers) but once you start catching up you have to, well, continue catching up in perpetuity.
  - nix23 4y ago
    Na, FreeBSD is not catching up/down to Linux..absolutely no interest..with one exception..drivers. But if you work with Illumos or *BSD's for a extended time, Linux feels dirty and bloated..just not right anymore, if you like linux good...i am happy for you but i try to avoid linux if i can (obviously not always the case...OracleDB, DB2, k8s and so on). >ZFS case though where they were the pioneers No that was OpenSolaris...and i am still sad about it. But Jails? I would say that was a real pioneering, whereas Solaris came a bit later with Zones and then a decade later Linux, you probably call that containers "a pure linux invention" today ;)
tux2bsd 4y ago
[dead]
suprjami 4y ago
At what point are BSD people going to realise they've lifted so much code from Linux they're basically running Linux? Permissive licenses work so well they'd still be stuck in the 90s if not for Linux ports
- nix23 4y ago
  There is nearly no GPL-Code in FreeBSD-Base anymore, but another question when is Linux getting a reliable COW Filesystem (NILFS2 is a exception)? You know saving files is kind of important, but since not even Linus understands COW-Filesystem's i have not much hope for Linux, and what about the kernel-queue NIH-Syndrom? Oh and when do you get a real Scheduler who's not start to hiccup on a load over 85%...yes i know you have 20 of them, and maybe in the future ULE (you know the FreeBSD one).
  - sofixa 4y ago
    There are multiple COW filesystems one can choose on Linux, the main notable ones being ZFS(yes, the same one as FreeBSD) and Btrfs.
    - nix23 4y ago
      Ah yeah btrfs...
- CoolCold 4y ago
  That's easy - will happen the next day they realize that Docker solves totally different issue that Jails solve and [they] will start thinking on why saying it's superior is over Docker makes other [people] smile. That Jails vs Docker insight event will happen on the date, not older than 10 years after `Year Of Linux On Desktop` finally comes :)
- talideon 4y ago
  Please explain to me what code has FreeBSD "lifted [...] from Linux"? Implementing a useful interface isn't "lifting code".
Uptrenda 4y ago
I tried out many different BSD 'flavours' recently and seemingly none of them had good networking config. It was the thing that stuck out to me. Everything else seemed to work perfectly but outside of using DHCP you were in a world of pain. I was trying to setup a BSD install and put Jenkins on it for testing. But I had so much trouble with networking I ended up having to virtualize it and do it that way. Surprisingly, VMWare wasn't up to the task either. It was Virtualbox that saved me in the end. Netlink = actual good networking management. Super useful tbh.
- lmz 4y ago
  This makes no sense. How would netlink improve the UI? (my guess is if you're not doing anything exotic, the UI is what you're complaining about)
- nix23 4y ago
  Can you decribe your pain with networking a bit more? Because that's a thing i never had problems with....on BSD's...hell not even on linux.
- CoolCold 4y ago
  Not big fan of Freebsd myself, but what you say makes not much sense even for me. Some examples would be nice.
Aissen 4y ago
Kudos on the work. Netlink interfaces are one of the most underdocumented part of the linux kernel userspace ABI. Current wish: more stateful netlink decoding in strace.

113 points by vermaden 4y ago

gclawes 4y ago

Oh may, I pray for this to be added to macOS. The amount of times I reach for 'ip ...' and am stymied...

NegativeLatency 4y ago
I'm not familiar with this, can you recommend a good source to read up on it (preferably something more engaging than a manpage)?
- jandrese 4y ago
  The ip command has a built-in help system you can get started with "ip help", but it tends to throw a lot of fugly BNF at you and is pretty light on detail. The ip manpage is much better. It is broken up into sections which is a bummer for searching but does make it a little less daunting for reading. The nice thing about it is that since it has been around for many years now you can google search most any question and find the answer. All in all the ip command is fairly straightforward and orthogonal which is nice. Some starting commands "ip addr show" lists interfaces and their stats. It's harder to read than ifconfig -a but gives you the same information in a similar format. "ip route show" replaces netstat -rn to show the routing table but is also not as well formatted as the old command. "ip neigh show" replaces arp -a and is about the same level of legibility. "ip route add ..." replaces "route ..." command and is about the same to use. So basically it combines a bunch of commands (including ifconfig) into one and gives them all a unified interface.
  - gerdesj 4y ago
    ip is way more useful than your summary! It also supports abbreviations just like a switch does which is nice: $ip a $ip a a 10.99.0.111/24 dev re0 $ip r $ip r a default via 10.99.0.1 dev re0 The second command is: ip address add. The third is show me the routing table - ip4 is the default add -6 for ipv6. The fourth is add a default route. $ip n s That is neigh(bour) show - the arp table. On Linux, ipconfig and co are destined to quietly go away. Use ip and co instead - so much more powerful.
    - laumars 4y ago
      The abbreviation convention on networking gear isn’t a good thing. It allows for typos to become completely different command, thus introducing unexpected behaviour. I’ve heard so many horror stories (and had some of my own too) because of this.
      - gerdesj 4y ago
        I don't agree with you. It is a convenience but you do need to be careful. There are few (if any) commands that can be morphed into another to cause horror. All switch command completers I know of will not complete on ambiguity. If you feel you are personally a bit random then use the try/commit mode that most decent gear allows. You package up a set of changes and try them out. You then commit them within an agreed time or the changes revert.
        laumars 4y ago
        As I said, I’ve personally been bit by feature and know of others who have. So I’m not talking about a theoretical risk. Things might have improved since I was last managing Cisco gear. try/commit wasn’t available back then and maybe those stupid commands (like “administrator-full-port-shutdown”) have been since removed to prevent ambiguous shortenings. But I did quickly learn that it’s better to be explicit rather than implicit. And frankly, I still think that’s good advice given the IaC methodologies that are now commonly practiced. If nothing else, you should be writing your config so that others can read it, and storing that config in version control.
  - iokanuon 4y ago
    >"ip addr show" lists interfaces and their stats. It's harder to read than ifconfig -a but gives you the same information in a similar format. It's worth noting that `-br` gives you a brief version of the output, and `-c` colors it, so that it's possible to do something like this: ip -br -c addr to get a very readable and colorful table-like output. There's also `-j` for JSON, and `-j -p` for pretty JSON.
  - icehawk 4y ago
    There's some caveats to comparing it to linux ifconfig and netstat: ifconfig can't express the idea that an interface can have IPv4 address, and is inconsistent with itself because it does show all of the IPv6 addresses. netstat (and route because you can't manipulate the main RIB with netstat, so now you have two problems-- er-- tools) completely elides things like source address selection as a function of the RIB, and the fact that the kernel can have more than one routing table.
    - eptcyka 4y ago
      Could you elaborate on the Mach kernel having multiple routing tables?
- nixcraft 4y ago
  I have "Linux ip Command Examples"[1] page that explains how to use the ip command to display and configure the network parameters for host interfaces. HTH. [1] https://www.cyberciti.biz/faq/linux-ip-command-examples-usage-syntax/ https://www.cyberciti.biz/faq/linux-ip-command-examples-usag...
  - jaredhallen 4y ago
    I know ip is not only the future, but very much the present. But damn if I can ditch my ifconfig/route habits.
- ipdashc 4y ago
  The other replies already answered you, but I just wanted to add: If you pass the -c flag to `ip`, like `ip -c addr`, it colors it! Makes it orders of magnitude easier to work with, IMO. I don't know why this isn't publicized more widely. Hence my username :)
mhio 4y ago
There's a python script that does a passable impression of `ip` brew install iproute2mac https://github.com/brona/iproute2mac https://github.com/brona/iproute2mac

kasabali 4y ago

Why would they want to run that abomination in FreeBSD?

kevans91 4y ago
IMO it was just a relatable example of a somewhat extensive user of netlink that demonstrates compatibility. Some folks may want to run it under the Linuxolator, which seems reasonable. I only have limited knowledge about netlink, all coming from a devsummit presentation by Alexander, but it seems like an actually good generic kernel interface to have.
- stefan_ 4y ago
  It really isn't. It's like someone wanted to add protobufs for kernel interfaces, but everything has not one tenth of the usability. In fact it's the kind of thing someone would design who wanted to make it's use in C maximally painful and error prone.
  - eqvinox 4y ago
    Working with both netlink and protobufs in C for 10+ years, I gotta say the latter is much more painful than the former. Error prone I would agree though, mostly because it's too easy to implement netlink… and get it wrong in the process. NB: my experience specifically relates to C. Protobuf's bindings to other languages are better.
chris_wot 4y ago
What is wrong with it?
- f1shy 4y ago
  See my answer to the parent comment.
f1shy 4y ago
That was exactly my thought, specially after reading another answer: > "ip addr show" lists interfaces and their stats. It's harder to read than ifconfig -a but gives you the same information in a similar format. > "ip route show" replaces netstat -rn to show the routing table but is also not as well formatted as the old command. > So basically it combines a bunch of commands (including ifconfig) into one and gives them all a unified interface. To me sounds like same but worse, and combining 10 commands into 1 seems to be totally against the unix philosophy "it should do one thing, and do it properly"

Melatonic 4y ago

Serious question - what do you all think of specifically (now) specializing in FreeBSD to work on either virtual networking stacks or storages? I have a good amount of experience already and it seems quite niche compared to general DevOps work but also does seem like a lot of big companies are using FreeBSD based setups. Dead end career or ?

pdntspa 4y ago
I would like to know too, I have been enjoying working with the OS and really like it for what it is, and also as a potential alternative to the Docker Devops hegemony BSD in general seems like a potential pivot into embedded space too, there's a lot of interesting electronics out there that run it
fanf2 4y ago
It has some advantages in that derivatives of the BSD networking code turns up all over the place, so it can be useful to know your way around it - and the Richard Stevens TCP/IP illustrated books are a very helpful guide. But I wouldn’t deliberately specialize: get a job working with BSD or Linux, and learn by doing; but keep an eye on the other kernels to see how they do it, and to get a broader basis for your skills.
CoolCold 4y ago
I'll reply in that way - their way of thinking is still in Sysadmins field, for good or bad. They dismiss/don't realize that the world has shifted for Dev guys doing bit of Ops and all that DevOps things. I see it used by more and more very special cases (this one from Meta employee who did networking at Yandex before), but out of general usage. Just over this year: https://redd.it/x3ewxf - IBM Cloud ditches FreeBSD https://redd.it/wf7h34 - Hetzner drops FreeBSD support https://redd.it/ui9z0m - Digital Ocean doing the same
- nix23 4y ago
  >https://redd.it/wf7h34 https://redd.it/wf7h34 - Hetzner drops FreeBSD support Not true, i send them an email about it, the Answer was kind of: They had problem's with the Boot process on some machines, however you can send them the image and they will put i on a USB-Stick then open a web-terminal for you..for free, they still support FreeBSD but not in the Rescue-Media-Way. BTW thanks for the great Answer from Robin...Hetzner is still great ;)

yayforfreebsd 4y ago

FreeBSD never ceases to amaze. You can run Linux binaries with the emulation layer, you can of course run many Linux applications through the ports system, they also have a Linux compatibility layer for the graphics stack so they can use Linux drivers, their ZFS implementation is now tracking ZFS on Linux, and now this! This is something other BSD systems don't get. OpenBSD developers pretend Linux doesn't even exist, while FreeBSD is doing a great job catching up with Linux and doing all the Linux stuff so you can Linuxy Linux your Linux stuff in your Linux FreeBSD Linux. Linux.

yjftsjthsd-h 4y ago
> their ZFS implementation is now tracking ZFS on Linux I don't think that's quite right. OpenZFS is a thing that is consumed (and pushed forward) by both FreeBSD and Linux; FreeBSD isn't following Linux, they just both use the same thing. (Which is why it isn't named ZoL anymore; it isn't supposed to be Linux-centric)
- nix23 4y ago
  Correct! It was created out of the ZoL project (because most people worked there instead of the illumos base) and now it transformed into OpenZFS with the Goal: >brings together developers from the Linux, FreeBSD, illumos, MacOS, and Windows platforms. OpenZFS is supported by a wide range of companies. https://openzfs.org/wiki/Main_Page https://openzfs.org/wiki/Main_Page
antonios 4y ago
FreeBSD is doing a great job catching up to Linux, yes, (not in the ZFS case though where they were the pioneers) but once you start catching up you have to, well, continue catching up in perpetuity.
- nix23 4y ago
  Na, FreeBSD is not catching up/down to Linux..absolutely no interest..with one exception..drivers. But if you work with Illumos or *BSD's for a extended time, Linux feels dirty and bloated..just not right anymore, if you like linux good...i am happy for you but i try to avoid linux if i can (obviously not always the case...OracleDB, DB2, k8s and so on). >ZFS case though where they were the pioneers No that was OpenSolaris...and i am still sad about it. But Jails? I would say that was a real pioneering, whereas Solaris came a bit later with Zones and then a decade later Linux, you probably call that containers "a pure linux invention" today ;)

tux2bsd 4y ago

[dead]

suprjami 4y ago

At what point are BSD people going to realise they've lifted so much code from Linux they're basically running Linux? Permissive licenses work so well they'd still be stuck in the 90s if not for Linux ports

nix23 4y ago
There is nearly no GPL-Code in FreeBSD-Base anymore, but another question when is Linux getting a reliable COW Filesystem (NILFS2 is a exception)? You know saving files is kind of important, but since not even Linus understands COW-Filesystem's i have not much hope for Linux, and what about the kernel-queue NIH-Syndrom? Oh and when do you get a real Scheduler who's not start to hiccup on a load over 85%...yes i know you have 20 of them, and maybe in the future ULE (you know the FreeBSD one).
- sofixa 4y ago
  There are multiple COW filesystems one can choose on Linux, the main notable ones being ZFS(yes, the same one as FreeBSD) and Btrfs.
  - nix23 4y ago
    Ah yeah btrfs...
CoolCold 4y ago
That's easy - will happen the next day they realize that Docker solves totally different issue that Jails solve and [they] will start thinking on why saying it's superior is over Docker makes other [people] smile. That Jails vs Docker insight event will happen on the date, not older than 10 years after `Year Of Linux On Desktop` finally comes :)
talideon 4y ago
Please explain to me what code has FreeBSD "lifted [...] from Linux"? Implementing a useful interface isn't "lifting code".

Uptrenda 4y ago

I tried out many different BSD 'flavours' recently and seemingly none of them had good networking config. It was the thing that stuck out to me. Everything else seemed to work perfectly but outside of using DHCP you were in a world of pain. I was trying to setup a BSD install and put Jenkins on it for testing. But I had so much trouble with networking I ended up having to virtualize it and do it that way. Surprisingly, VMWare wasn't up to the task either. It was Virtualbox that saved me in the end. Netlink = actual good networking management. Super useful tbh.

lmz 4y ago
This makes no sense. How would netlink improve the UI? (my guess is if you're not doing anything exotic, the UI is what you're complaining about)
nix23 4y ago
Can you decribe your pain with networking a bit more? Because that's a thing i never had problems with....on BSD's...hell not even on linux.
CoolCold 4y ago
Not big fan of Freebsd myself, but what you say makes not much sense even for me. Some examples would be nice.

Aissen 4y ago

Kudos on the work. Netlink interfaces are one of the most underdocumented part of the linux kernel userspace ABI. Current wish: more stateful netlink decoding in strace.