8 ms·

Faster Linux 2.5G Networking with Realtek RTL8125B

121 points by geerlingguy 5y ago

deleted 5y ago
[deleted]
mattst88 5y ago
I've always been confused by the situation with Realtek Linux drivers. Does anyone know why they are not contributing to the kernel directly? (Just a guess: their drivers are from the same codebase as the Windows driver and use a bunch of abstraction layers that wouldn't be acceptable in Linux)
- morning_gelato 5y ago
  Realtek does contribute directly to the kernel, but I think it depends on which internal team is handling the device support. For their PCIe 802.11ac and 802.11ax wifi cards Realtek devs wrote and upstreamed rtw88[1] and rtw89[2] respectively. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e3037485c68ec1a299ff41160d8fedbd4abc29b9 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e3ec7017f6a20d12ddd9fe23d345ebb7b8c104dd https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
  - pantalaimon 5y ago
    Wasn't rtw88 supposed to abstract away the transport so it could be used with PCIe, USB and SDIO? [0] The later two never materialized (there is the out of tree rtw88-usb [1] but it's not very active and not maintained by Realtek) [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e3037485c68ec1a299ff41160d8fedbd4abc29b9 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... [1] https://github.com/ulli-kroll/rtw88-usb https://github.com/ulli-kroll/rtw88-usb
- KarlKemp 5y ago
  Do they publish source code or binary drivers? If it's the latter, the article points at why they might not want their competitors to read their source code: the drivers matter a lot, and starting from theirs might well cut the development costs for a competing product in half.
  - gsich 5y ago
    Source code and no BLOBs.
  - segfaultbuserr 5y ago
    Realtek drivers are GPL’d, but have a reputation (especially Wi-Fi dongles) of not following the coding standard of the Linux kernel and of low code quality, making them unacceptable for the upstream kernel. A driver often goes through a lengthy cleanup process by the community, first by existing as a “staging” driver in the kernel with incomplete features and unstable code, and later the code is gradually refactored before it’s finally official.
- the_biot 5y ago
  I'm familiar with the Realtek RTL vendor driver codebase. It's a combination of two classics: a Realtek-specific hardware abstraction layer across a large number of their products (which won't fly in the Linux kernel), and abominable code quality (ditto).
formerly_proven 5y ago
I'm guessing the story is overall similar to how gigabit ethernet was ~ten years ago: Use Intel if you want working hardware and drivers that actually achieve line-rate throughput. Back then the RTL chips were a trashfire and couldn't do gigabit even on Windows. Nowadays the RTL chips can do that. And here we are again with bad / proprietary network drivers for RTL chips.
- nightfly 5y ago
  Isn't it the same for wifi? On nearly every laptop I've owned I've ended up swapping for an Intel wireless card eventually.
  - formerly_proven 5y ago
    Yes, and even the generational changes can be quite worth it - I recently swapped a 2016 Intel 8265 wifi card with a newer AX210 (- which costs basically nothing) and even without utilizing the updated standards I got 50-80 % more bandwidth under exactly the same conditions.
- gsich 5y ago
  The driver from Realtek is GPL2. Maybe the code quality is not good enough for the kernel. https://www.realtek.com/en/component/zoo/category/network-interface-controllers-10-100-1000m-gigabit-ethernet-pci-express-software https://www.realtek.com/en/component/zoo/category/network-in...
  - my123 5y ago
    They are upstream nowadays. Supported mainline by the r8169 driver.
    - gsich 5y ago
      Yeah, but according to the blog post there must be some difference. "Supported" but not "fully supported". Mainline has 8k lines, Realteks driver has 15k lines in only 1 .c file and there are some more.
daneel_w 5y ago
PCI-E cards with this chip cost about $15 today, and 2.5G Ethernet switches just over $100. Post-gigabit networking has become truly affordable.
- netr0ute 5y ago
  > just over $100 This is pretty expensive considering that you can get a 10G switch instead for around $40 more.
  - makomk 5y ago
    Believe it or not, the fact that 2.5G switches are cheaper than 10G switches at all is an improvement. There seems to be a weird lack of interest in shipping it as a consumer technology.
    - flemhans 5y ago
      Why do Ethernet standards mostly come in 10x orders of magnitude, anyway? 10 -> 100 -> 1000 -> 10G -> 100G I know there are some oddball 25G, 40G, and 2.5G out there too, but always wondered this.
      - comboy 5y ago
        https://en.wikipedia.org/wiki/IEEE_802.3 https://en.wikipedia.org/wiki/IEEE_802.3 I suspect it's about us having 10 fingers and using arabic numerals.
      - deleted 5y ago
        [deleted]
      - dsr_ 5y ago
        Think about it this way: what's the minimum improvement you'll pay a whole heap of money for? Each generation gets introduced with new switches, new NICs, and new cabling. Since 100Mb it has been possible to have link aggregation groups, where two or more cables on the same number of NICs on each side are bundled together into one bandwidth device. At some point you need to decide that it's worth doing the transition -- and 10x is a pretty good number for that. The recent oddballs are 50G, 40G, 25G and 2.5G. All of them come from the idea of taking a higher speed NIC and adding a little more hardware to get multiple PHY transceivers -- a 100G becomes 2x50 or 4x25, and a 10G becomes 4x2.5. Oddly, 40G comes from 4x10G in the other direction, and is more expensive to produce than 50G equipment. It's not very popular.
TacticalCoder 5y ago
Speaking of which, what's a reasonably cheap solution for a little 10 GBit/s home LAN (Linux)? A 4 port Microtik switch and which cards? Nothing fancy. Just something that won't bottleneck like crazy the PCIe 3.0 or 4.0 x4 lanes NVMe SSDs my machines have. I'd rather go to 10 GBit/s directly instead of 2.5 but I'll go 2.5 if it's really easier/cheaper.
- cpgxiii 5y ago
  Mikrotik makes a 4xSFP+ switch and Ubiquiti makes a 4x10GBase-T switch. The Mikrotik one can take SFP+ copper transceivers, so it allows you to mix and match twisted pair and fiber optics if you need it. Asus makes a pair of Aquantia (now Marvell) 10G PCIe x4 cards, one with SFP+ and the other with 1/2.5/5/10G Base-T. Other vendors have similar cards in PCIe x4 and Thunderbolt 2/3 formats.
  - zrail 5y ago
    I just picked up a Mikrotik CRS305 from Amazon and a pair of used Intel X520T dual SPF+ cards from eBay, plus a couple DAC (direct attach copper) cables for maybe $300 total. I'm very happy with the performance of this setup. One thing to keep in mind about the CRS305 is that it can't handle more than one or maybe two SFP+ copper modules. They use a lot of power and put out a lot of heat. I think the CRS309 has fewer restrictions.
    - csdvrx 5y ago
      Could you recommend a PCIe adapter to receive a Nokia 3fe4960ac SFP+ transceiver (AT&T fiber) and maybe one or two 2.5G or 10G modules?
      - zrail 5y ago
        Oh I have no idea. You'd need to do some digging. I picked the Intel card because it was decently priced and I knew it would be reasonably compatible with what I was doing. Sounds like maybe you want a quad SFP+ card?
drewg123 5y ago
TL;DR: Vendor's driver performs far better than what's upstream. This is what happens when the kernel community is confrontational and generally a pain in the neck to deal with. After enough friction caused by the netdev folks, the vendor just gives up on upstreaming and self-publishes their driver.
- swills 5y ago
  It's a shame you're getting down voted for this.
- caskstrength 5y ago
  FreeBSD dev dishing out Linux processes in 2021. Some things never change I guess ;) But honestly, Drew, all serious vendors of networking hardware have fully functional upstream drivers and good relationships with Jakub and Dave. However, one can't just throw mess of a code to netdev and expected it to be accepted as-is because they are a big vendor or something. Which I believe is good for users!
  - drewg123 5y ago
    I'm still peeved at what happened to my company around ~2006 or so which resulted in the exact same situation in the article. In my case: Company A submits a driver with a feature in their driver that basically triples performance for our class of device. It was implemented in a horrific, unreadable way. Companies B and C (mine) each independently re-implement that feature in a less crazy, far more readable and maintainable way. Company B gets their driver accepted with this feature (I think it was part of the initial submission). We get ours NACKed and are told to implement it for the entire kernel. This would be fine, I suppose, if companies A and B were also required to remove the feature and help implement it, but they weren't. This might not have been such a big deal if we had been a big company. However, we had 1 dev (me) doing drivers and support for Linux, Solaris, OSX, FreeBSD, ESX, etc. So in the short term, management directed that we have customers ignore the driver in the kernel and use the one from our website so we didn't have to stop development on other OSes to implement this feature for our competition. This feature was eventually implemented in a generic way (with my help), but for several kernel releases the driver on our website outperformed the in-kernel driver by a factor of 3 or more. FWIW, FreeBSD had no problem with the feature, and another driver author eventually ported it out of my driver into a general layer where it still exists to this day, and is used by almost every driver in the OS.
fulafel 5y ago
Ah consumer networking, stuck in the single gigabit speeds, 1 GigE was introduced decades ago in the last millenium.. I wonder if this is a permanent plateau in human development or if there will be some bump when we get modern 100+G ethernet that servers are currently using. Meanwhile there are notable recent perf speedups in the modern ethernet too: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=f35f821935d8df76f9c92e2431a225bdff938169 https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-n...
- Tepix 5y ago
  Can you do wired >1GBit/s with low power consumption?
  - deleted 5y ago
    [deleted]
- tehbeard 5y ago
  I've typically seen most 10gig kit to need a lot of cooling (either heatsinks or active fans), not something I'd be too happy about dealing with. Also there's like only the homelab enthusiast that needs 10gig. The rest of user can potter along fine on 1/2.5gig as we're not slinging ZFS snapshots about the place. Tl;Dr 1/2.5gig works for consumers needs, 10gig is not price competitive for our needs.
  - fulafel 5y ago
    Bandwidth stopped growing for consumer stuff when everyone moved to slow and flaky wireless. If it was available, there would be apps to use it.
    - stevenhuang 5y ago
      Bandwidth stopped growing because needs were met. > If it was available, there would be apps to use it. To an extent, since the inverse is also true: there is as of yet no market demand for such large amounts of bandwidth either. Once popular apps/usecases with heavy bandwidth requirements see wide adoption (at first limited to specialty high bandwidth networks), we will start seeing growth in the consumer bandwidth space.
baybal2 5y ago
Rtl8156 is one of few NCM USB cards which got mainstream. NCM protocol is very lightweight, but has support for DMA, and some offloading
mateo1 5y ago
Another front for improvement is USB wireless support in linux. Yes I'm hijacking this to complain but currently (almost 2022) there isn't a single 5GHz capable(802.11ac) USB dongle with in-tree drivers or support by Ubuntu afaik. Only sketchy github scripts exist. Sure, this is primarily the vendors fault and you cannot compromise security and GPL for convenience but come on.

121 points by geerlingguy 5y ago

deleted 5y ago

[deleted]

I've always been confused by the situation with Realtek Linux drivers. Does anyone know why they are not contributing to the kernel directly? (Just a guess: their drivers are from the same codebase as the Windows driver and use a bunch of abstraction layers that wouldn't be acceptable in Linux)

morning_gelato 5y ago
Realtek does contribute directly to the kernel, but I think it depends on which internal team is handling the device support. For their PCIe 802.11ac and 802.11ax wifi cards Realtek devs wrote and upstreamed rtw88[1] and rtw89[2] respectively. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e3037485c68ec1a299ff41160d8fedbd4abc29b9 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e3ec7017f6a20d12ddd9fe23d345ebb7b8c104dd https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
- pantalaimon 5y ago
  Wasn't rtw88 supposed to abstract away the transport so it could be used with PCIe, USB and SDIO? [0] The later two never materialized (there is the out of tree rtw88-usb [1] but it's not very active and not maintained by Realtek) [0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=e3037485c68ec1a299ff41160d8fedbd4abc29b9 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin... [1] https://github.com/ulli-kroll/rtw88-usb https://github.com/ulli-kroll/rtw88-usb
KarlKemp 5y ago
Do they publish source code or binary drivers? If it's the latter, the article points at why they might not want their competitors to read their source code: the drivers matter a lot, and starting from theirs might well cut the development costs for a competing product in half.
- gsich 5y ago
  Source code and no BLOBs.
- segfaultbuserr 5y ago
  Realtek drivers are GPL’d, but have a reputation (especially Wi-Fi dongles) of not following the coding standard of the Linux kernel and of low code quality, making them unacceptable for the upstream kernel. A driver often goes through a lengthy cleanup process by the community, first by existing as a “staging” driver in the kernel with incomplete features and unstable code, and later the code is gradually refactored before it’s finally official.
the_biot 5y ago
I'm familiar with the Realtek RTL vendor driver codebase. It's a combination of two classics: a Realtek-specific hardware abstraction layer across a large number of their products (which won't fly in the Linux kernel), and abominable code quality (ditto).

formerly_proven 5y ago

I'm guessing the story is overall similar to how gigabit ethernet was ~ten years ago: Use Intel if you want working hardware and drivers that actually achieve line-rate throughput. Back then the RTL chips were a trashfire and couldn't do gigabit even on Windows. Nowadays the RTL chips can do that. And here we are again with bad / proprietary network drivers for RTL chips.

nightfly 5y ago
Isn't it the same for wifi? On nearly every laptop I've owned I've ended up swapping for an Intel wireless card eventually.
- formerly_proven 5y ago
  Yes, and even the generational changes can be quite worth it - I recently swapped a 2016 Intel 8265 wifi card with a newer AX210 (- which costs basically nothing) and even without utilizing the updated standards I got 50-80 % more bandwidth under exactly the same conditions.
gsich 5y ago
The driver from Realtek is GPL2. Maybe the code quality is not good enough for the kernel. https://www.realtek.com/en/component/zoo/category/network-interface-controllers-10-100-1000m-gigabit-ethernet-pci-express-software https://www.realtek.com/en/component/zoo/category/network-in...
- my123 5y ago
  They are upstream nowadays. Supported mainline by the r8169 driver.
  - gsich 5y ago
    Yeah, but according to the blog post there must be some difference. "Supported" but not "fully supported". Mainline has 8k lines, Realteks driver has 15k lines in only 1 .c file and there are some more.

daneel_w 5y ago

PCI-E cards with this chip cost about $15 today, and 2.5G Ethernet switches just over $100. Post-gigabit networking has become truly affordable.

netr0ute 5y ago
> just over $100 This is pretty expensive considering that you can get a 10G switch instead for around $40 more.
- makomk 5y ago
  Believe it or not, the fact that 2.5G switches are cheaper than 10G switches at all is an improvement. There seems to be a weird lack of interest in shipping it as a consumer technology.
  - flemhans 5y ago
    Why do Ethernet standards mostly come in 10x orders of magnitude, anyway? 10 -> 100 -> 1000 -> 10G -> 100G I know there are some oddball 25G, 40G, and 2.5G out there too, but always wondered this.
    - comboy 5y ago
      https://en.wikipedia.org/wiki/IEEE_802.3 https://en.wikipedia.org/wiki/IEEE_802.3 I suspect it's about us having 10 fingers and using arabic numerals.
    - deleted 5y ago
      [deleted]
    - dsr_ 5y ago
      Think about it this way: what's the minimum improvement you'll pay a whole heap of money for? Each generation gets introduced with new switches, new NICs, and new cabling. Since 100Mb it has been possible to have link aggregation groups, where two or more cables on the same number of NICs on each side are bundled together into one bandwidth device. At some point you need to decide that it's worth doing the transition -- and 10x is a pretty good number for that. The recent oddballs are 50G, 40G, 25G and 2.5G. All of them come from the idea of taking a higher speed NIC and adding a little more hardware to get multiple PHY transceivers -- a 100G becomes 2x50 or 4x25, and a 10G becomes 4x2.5. Oddly, 40G comes from 4x10G in the other direction, and is more expensive to produce than 50G equipment. It's not very popular.

TacticalCoder 5y ago

Speaking of which, what's a reasonably cheap solution for a little 10 GBit/s home LAN (Linux)? A 4 port Microtik switch and which cards? Nothing fancy. Just something that won't bottleneck like crazy the PCIe 3.0 or 4.0 x4 lanes NVMe SSDs my machines have. I'd rather go to 10 GBit/s directly instead of 2.5 but I'll go 2.5 if it's really easier/cheaper.

cpgxiii 5y ago
Mikrotik makes a 4xSFP+ switch and Ubiquiti makes a 4x10GBase-T switch. The Mikrotik one can take SFP+ copper transceivers, so it allows you to mix and match twisted pair and fiber optics if you need it. Asus makes a pair of Aquantia (now Marvell) 10G PCIe x4 cards, one with SFP+ and the other with 1/2.5/5/10G Base-T. Other vendors have similar cards in PCIe x4 and Thunderbolt 2/3 formats.
- zrail 5y ago
  I just picked up a Mikrotik CRS305 from Amazon and a pair of used Intel X520T dual SPF+ cards from eBay, plus a couple DAC (direct attach copper) cables for maybe $300 total. I'm very happy with the performance of this setup. One thing to keep in mind about the CRS305 is that it can't handle more than one or maybe two SFP+ copper modules. They use a lot of power and put out a lot of heat. I think the CRS309 has fewer restrictions.
  - csdvrx 5y ago
    Could you recommend a PCIe adapter to receive a Nokia 3fe4960ac SFP+ transceiver (AT&T fiber) and maybe one or two 2.5G or 10G modules?
    - zrail 5y ago
      Oh I have no idea. You'd need to do some digging. I picked the Intel card because it was decently priced and I knew it would be reasonably compatible with what I was doing. Sounds like maybe you want a quad SFP+ card?

drewg123 5y ago

TL;DR: Vendor's driver performs far better than what's upstream. This is what happens when the kernel community is confrontational and generally a pain in the neck to deal with. After enough friction caused by the netdev folks, the vendor just gives up on upstreaming and self-publishes their driver.

swills 5y ago
It's a shame you're getting down voted for this.
caskstrength 5y ago
FreeBSD dev dishing out Linux processes in 2021. Some things never change I guess ;) But honestly, Drew, all serious vendors of networking hardware have fully functional upstream drivers and good relationships with Jakub and Dave. However, one can't just throw mess of a code to netdev and expected it to be accepted as-is because they are a big vendor or something. Which I believe is good for users!
- drewg123 5y ago
  I'm still peeved at what happened to my company around ~2006 or so which resulted in the exact same situation in the article. In my case: Company A submits a driver with a feature in their driver that basically triples performance for our class of device. It was implemented in a horrific, unreadable way. Companies B and C (mine) each independently re-implement that feature in a less crazy, far more readable and maintainable way. Company B gets their driver accepted with this feature (I think it was part of the initial submission). We get ours NACKed and are told to implement it for the entire kernel. This would be fine, I suppose, if companies A and B were also required to remove the feature and help implement it, but they weren't. This might not have been such a big deal if we had been a big company. However, we had 1 dev (me) doing drivers and support for Linux, Solaris, OSX, FreeBSD, ESX, etc. So in the short term, management directed that we have customers ignore the driver in the kernel and use the one from our website so we didn't have to stop development on other OSes to implement this feature for our competition. This feature was eventually implemented in a generic way (with my help), but for several kernel releases the driver on our website outperformed the in-kernel driver by a factor of 3 or more. FWIW, FreeBSD had no problem with the feature, and another driver author eventually ported it out of my driver into a general layer where it still exists to this day, and is used by almost every driver in the OS.

fulafel 5y ago

Ah consumer networking, stuck in the single gigabit speeds, 1 GigE was introduced decades ago in the last millenium.. I wonder if this is a permanent plateau in human development or if there will be some bump when we get modern 100+G ethernet that servers are currently using. Meanwhile there are notable recent perf speedups in the modern ethernet too: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=f35f821935d8df76f9c92e2431a225bdff938169 https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-n...

Tepix 5y ago
Can you do wired >1GBit/s with low power consumption?
- deleted 5y ago
  [deleted]
tehbeard 5y ago
I've typically seen most 10gig kit to need a lot of cooling (either heatsinks or active fans), not something I'd be too happy about dealing with. Also there's like only the homelab enthusiast that needs 10gig. The rest of user can potter along fine on 1/2.5gig as we're not slinging ZFS snapshots about the place. Tl;Dr 1/2.5gig works for consumers needs, 10gig is not price competitive for our needs.
- fulafel 5y ago
  Bandwidth stopped growing for consumer stuff when everyone moved to slow and flaky wireless. If it was available, there would be apps to use it.
  - stevenhuang 5y ago
    Bandwidth stopped growing because needs were met. > If it was available, there would be apps to use it. To an extent, since the inverse is also true: there is as of yet no market demand for such large amounts of bandwidth either. Once popular apps/usecases with heavy bandwidth requirements see wide adoption (at first limited to specialty high bandwidth networks), we will start seeing growth in the consumer bandwidth space.

baybal2 5y ago

Rtl8156 is one of few NCM USB cards which got mainstream. NCM protocol is very lightweight, but has support for DMA, and some offloading

mateo1 5y ago

Another front for improvement is USB wireless support in linux. Yes I'm hijacking this to complain but currently (almost 2022) there isn't a single 5GHz capable(802.11ac) USB dongle with in-tree drivers or support by Ubuntu afaik. Only sketchy github scripts exist. Sure, this is primarily the vendors fault and you cannot compromise security and GPL for convenience but come on.