6 ms·

My Linux server already has "full router capabilities" AND I don't have to use RouterOS to configure it (which is just a shit abstraction on top of common Linux

My Linux server already has "full router capabilities" AND I don't have to use RouterOS to configure it (which is just a shit abstraction on top of common Linux network services like iptables).

cute_boi 5y ago
I think the use case is to reduce CPU usage. Its like GPU cards, but for networking.
- xxs 5y ago
  By just reading the title you can tell it runs an Arm cpu with linux on it. Not really certain how useful that is.
  - 293984j29384 5y ago
    I'm not sure where the confusion is. OP mentioned that his Linux system can already do routing. The purpose of this card is to remove that load from the computer. The manufacture suggests it can do up to 100Gbps which isn't trivial.
    - xxs 5y ago
      >>reduce CPU usage. Its like GPU cards, It uses another CPU to do that. GPU is fundamentally different, high memory bandwidth, embarrassingly parallel, virtually no branches, and what not. That's just using a different CPU to do more CPU, and using the same OS the host already runs. Then it requires its own security maintenance (+training) and patches.
      - benou 5y ago
        Personally I think one of the real usecase for smartnic is isolation: for a cloud provider, you can rent a bare metal instance and run all your networking security stack (think encapsulation, filtering, throttling etc) on the smartnic. IOW the customer has full control of the host, but the cloud provider manages the smartnic. Incidentally, this is exactly what AWS does with their ENA adapters designed by... (ex-?)Anapurna Lab they bought some years ago (:
        jabart 5y ago
        Mikrotik uses Annapurna ARM chips.
      - jotm 5y ago
        I think the analogy was that a CPU can do a GPU's job, but a GPU will do it much faster. Pretty much all modern NICs are already using separate hardware to reduce the load on the main CPU. I.e. using a different CPU to do more CPU. Without that you're looking at sacrificing a whole core or two just to handle 1Gbps, nevermind 10+.
    - drewg123 5y ago
      This NIC can reach wire-speed (100Gbps) with Jumbo frames. To me, this suggests that it's packet-rate limited, and if so, it can really only be counted on to do 1500/9000 or ~16.6Gb/s with standard frames.
  - aseipp 5y ago
    Products like this are, generally speaking, designed for service providers, where having more available host capacity directly translates to increased revenue. Consider a cloud provider who offers virtual machines to users: the physical host machine typically is involved in whatever networking path is necessary (e.g. an SDN), as well as the control plane software for managing VMs, and other tidbits. Moving the entire networking and SDN layer off the host system and onto an accelerator card, with your own customizations to the data path, means you can take those host resources and use them for VMs instead -- effectively increasing the total amount of capacity you have available. It's not just CPU time either: things like this also effectively increase available PCIe bandwidth, memory bandwidth, etc, available to users, by moving the resources the operator needs elsewhere. There are some other benefits too, like you can run the whole security framework on a card like this. Or QoS controls. You could for example rent out the entire bare metal server to someone more or less and use a device like this to implement throttling/QoS/SDN transparently. Most of the vendors are calling these "Data Processing Units" or "Infrastructure Processing Units" or whatever, but the idea is all the same. Offloading the networking/data paths into accelerators allows you to offer more raw compute to your users. For example, Nvidia Bluefield or Intel's new Mount Evans IPU. This Mikrotik is basically the bargain-bin version of those products. Which is actually pretty cool. I could actually use a couple of 25GbE breakouts for that price...
flower-giraffe 5y ago
SRP of 199usd and 2x SFP28 25GbE. It’s not for the enterprise but I’ll get some for home.
walrus01 5y ago
Almost everyone I know that's ever used JunOS from a command line for 'serious' ISP things finds RouterOS painful and cumbersome. The way things are laid out in a hierarchy in a full system "/export" from a Mikrotik is so weird and annoying compared to a hierarchical junos configuration from a "show configuration" on a juniper router. If people want to make a real router of an x86-64 system rather than putting a mikrotik pci-e card into it (wtf, why?) I'd recommend they go with vyatta or VyOS instead, or install something like a barebones centos or debian and then add FRR to it.
- techsupporter 5y ago
  > make a real router of an x86-64 system rather than putting a mikrotik pci-e card into it (wtf, why?) I'd recommend they go with vyatta or VyOS instead One thing I've been looking for is a hardware box that can replicate what Ubiquiti's EdgeRouter Infinity does: a handful of 10Gbps SFP+ ports (sorry, I know that the term is "cages" but I just can't) and a couple of copper 1Gbps ports. So far I haven't found anything but I feel like my search will get motivated in the next couple of years since it feels like Ubiquiti has forgotten that EdgeRouter exists. Do you have any rack form factor x86-type systems you like for VyOS?
  - walrus01 5y ago
    When space permits I prefer full-size 1U systems that have dual/hotswap power supplies and room for three low profile pci-e slots, such as a Dell R630/R640 or similar. With Intel chipset 4-port 10GbE SFP+ NICs this would max out at twelve ports plus whatever is on the motherboard daughtercard for network interfaces (2 x 10GbE + 2 x 1GbE copper, or whatever). for smaller or shallow stuff, supermicro, msi, tyan, asus
  - logifail 5y ago
    > a hardware box [with] a handful of 10Gbps SFP+ [..] and a couple of copper 1Gbps ports I have a couple of (fanless!) CRS305-1G-4S+IN[0] at home, one in my study and one in the utility room. They each connect with 10GbE fibre (or DAC) to ConnectX-3 cards in my PCs and servers. [0] https://mikrotik.com/product/crs305_1g_4s_in https://mikrotik.com/product/crs305_1g_4s_in
deleted 5y ago
[deleted]
Alupis 5y ago
This Dual SFP28 (dual 25Gb cages) plus 1Gb Eth PCI-e card has an MSRP of $199, meaning a street price will be a bit under that. 10Gb NIC's run around $100... and can't do any switching or routing. As mentioned, this card can offload 100% of routing needs from the server (ie. zero CPU usage on your server to make routing decisions), can switch at line speed (well above line speed actually, rated for 100Gbps throughput), plus the server can still use one of the ports for it's own needs. Sounds pretty powerful to me. It's unlikely this is an interesting product for a home lab or business - it's likely more geared towards service providers. Still a pretty cool idea none-the-less, regardless of how you feel about routerOS.
- philjohn 5y ago
  New ones do, but 5 year old PCIe3 ones, such as the SolarFlare 7000 series which are still supported with drivers by Xilinx can be had for around $30 off ebay for home use.
- ikiris 5y ago
  What level of routing decisions do you expect to be made on a single / dual ethernet drop to a server card with no other ports? Like this is cute and all, but kind of a weird target market.
  - Alupis 5y ago
    Bunch 'O VLAN's to guest VM's without that routing consuming CPU cycles on the host hardware. RouterOS has a full API so you could automate everything. The card actually has 4 "ports". 1 virtual port dedicated to management (via PCIe passthrough), 1 GBe port, and 2 SFP28 cages. Plenty for a cloud hosting provider. Both SFP28 cages, plus the virtual 1GBe port support passthrough via PCIe to the physical server. That's just one use case. Another is a dedicated firewall for the actual physical server that's powering the thing, running at a full 25Gbps consuming zero CPU time on the server itself and zero U's of space... all for less than $199 street price. The card supports PCIe 8x passthrough, so the physical connection to the server is 64Gbps - way more than a single SFP28 cage can support. Pretty slick...
deleted 5y ago
[deleted]

My Linux server already has "full router capabilities" AND I don't have to use RouterOS to configure it (which is just a shit abstraction on top of common Linux network services like iptables).

cute_boi 5y ago

I think the use case is to reduce CPU usage. Its like GPU cards, but for networking.

xxs 5y ago
By just reading the title you can tell it runs an Arm cpu with linux on it. Not really certain how useful that is.
- 293984j29384 5y ago
  I'm not sure where the confusion is. OP mentioned that his Linux system can already do routing. The purpose of this card is to remove that load from the computer. The manufacture suggests it can do up to 100Gbps which isn't trivial.
  - xxs 5y ago
    >>reduce CPU usage. Its like GPU cards, It uses another CPU to do that. GPU is fundamentally different, high memory bandwidth, embarrassingly parallel, virtually no branches, and what not. That's just using a different CPU to do more CPU, and using the same OS the host already runs. Then it requires its own security maintenance (+training) and patches.
    - benou 5y ago
      Personally I think one of the real usecase for smartnic is isolation: for a cloud provider, you can rent a bare metal instance and run all your networking security stack (think encapsulation, filtering, throttling etc) on the smartnic. IOW the customer has full control of the host, but the cloud provider manages the smartnic. Incidentally, this is exactly what AWS does with their ENA adapters designed by... (ex-?)Anapurna Lab they bought some years ago (:
      - jabart 5y ago
        Mikrotik uses Annapurna ARM chips.
    - jotm 5y ago
      I think the analogy was that a CPU can do a GPU's job, but a GPU will do it much faster. Pretty much all modern NICs are already using separate hardware to reduce the load on the main CPU. I.e. using a different CPU to do more CPU. Without that you're looking at sacrificing a whole core or two just to handle 1Gbps, nevermind 10+.
  - drewg123 5y ago
    This NIC can reach wire-speed (100Gbps) with Jumbo frames. To me, this suggests that it's packet-rate limited, and if so, it can really only be counted on to do 1500/9000 or ~16.6Gb/s with standard frames.
- aseipp 5y ago
  Products like this are, generally speaking, designed for service providers, where having more available host capacity directly translates to increased revenue. Consider a cloud provider who offers virtual machines to users: the physical host machine typically is involved in whatever networking path is necessary (e.g. an SDN), as well as the control plane software for managing VMs, and other tidbits. Moving the entire networking and SDN layer off the host system and onto an accelerator card, with your own customizations to the data path, means you can take those host resources and use them for VMs instead -- effectively increasing the total amount of capacity you have available. It's not just CPU time either: things like this also effectively increase available PCIe bandwidth, memory bandwidth, etc, available to users, by moving the resources the operator needs elsewhere. There are some other benefits too, like you can run the whole security framework on a card like this. Or QoS controls. You could for example rent out the entire bare metal server to someone more or less and use a device like this to implement throttling/QoS/SDN transparently. Most of the vendors are calling these "Data Processing Units" or "Infrastructure Processing Units" or whatever, but the idea is all the same. Offloading the networking/data paths into accelerators allows you to offer more raw compute to your users. For example, Nvidia Bluefield or Intel's new Mount Evans IPU. This Mikrotik is basically the bargain-bin version of those products. Which is actually pretty cool. I could actually use a couple of 25GbE breakouts for that price...

flower-giraffe 5y ago

SRP of 199usd and 2x SFP28 25GbE. It’s not for the enterprise but I’ll get some for home.

walrus01 5y ago

Almost everyone I know that's ever used JunOS from a command line for 'serious' ISP things finds RouterOS painful and cumbersome. The way things are laid out in a hierarchy in a full system "/export" from a Mikrotik is so weird and annoying compared to a hierarchical junos configuration from a "show configuration" on a juniper router. If people want to make a real router of an x86-64 system rather than putting a mikrotik pci-e card into it (wtf, why?) I'd recommend they go with vyatta or VyOS instead, or install something like a barebones centos or debian and then add FRR to it.

techsupporter 5y ago
> make a real router of an x86-64 system rather than putting a mikrotik pci-e card into it (wtf, why?) I'd recommend they go with vyatta or VyOS instead One thing I've been looking for is a hardware box that can replicate what Ubiquiti's EdgeRouter Infinity does: a handful of 10Gbps SFP+ ports (sorry, I know that the term is "cages" but I just can't) and a couple of copper 1Gbps ports. So far I haven't found anything but I feel like my search will get motivated in the next couple of years since it feels like Ubiquiti has forgotten that EdgeRouter exists. Do you have any rack form factor x86-type systems you like for VyOS?
- walrus01 5y ago
  When space permits I prefer full-size 1U systems that have dual/hotswap power supplies and room for three low profile pci-e slots, such as a Dell R630/R640 or similar. With Intel chipset 4-port 10GbE SFP+ NICs this would max out at twelve ports plus whatever is on the motherboard daughtercard for network interfaces (2 x 10GbE + 2 x 1GbE copper, or whatever). for smaller or shallow stuff, supermicro, msi, tyan, asus
- logifail 5y ago
  > a hardware box [with] a handful of 10Gbps SFP+ [..] and a couple of copper 1Gbps ports I have a couple of (fanless!) CRS305-1G-4S+IN[0] at home, one in my study and one in the utility room. They each connect with 10GbE fibre (or DAC) to ConnectX-3 cards in my PCs and servers. [0] https://mikrotik.com/product/crs305_1g_4s_in https://mikrotik.com/product/crs305_1g_4s_in

deleted 5y ago

[deleted]

Alupis 5y ago

This Dual SFP28 (dual 25Gb cages) plus 1Gb Eth PCI-e card has an MSRP of $199, meaning a street price will be a bit under that. 10Gb NIC's run around $100... and can't do any switching or routing. As mentioned, this card can offload 100% of routing needs from the server (ie. zero CPU usage on your server to make routing decisions), can switch at line speed (well above line speed actually, rated for 100Gbps throughput), plus the server can still use one of the ports for it's own needs. Sounds pretty powerful to me. It's unlikely this is an interesting product for a home lab or business - it's likely more geared towards service providers. Still a pretty cool idea none-the-less, regardless of how you feel about routerOS.

philjohn 5y ago
New ones do, but 5 year old PCIe3 ones, such as the SolarFlare 7000 series which are still supported with drivers by Xilinx can be had for around $30 off ebay for home use.
ikiris 5y ago
What level of routing decisions do you expect to be made on a single / dual ethernet drop to a server card with no other ports? Like this is cute and all, but kind of a weird target market.
- Alupis 5y ago
  Bunch 'O VLAN's to guest VM's without that routing consuming CPU cycles on the host hardware. RouterOS has a full API so you could automate everything. The card actually has 4 "ports". 1 virtual port dedicated to management (via PCIe passthrough), 1 GBe port, and 2 SFP28 cages. Plenty for a cloud hosting provider. Both SFP28 cages, plus the virtual 1GBe port support passthrough via PCIe to the physical server. That's just one use case. Another is a dedicated firewall for the actual physical server that's powering the thing, running at a full 25Gbps consuming zero CPU time on the server itself and zero U's of space... all for less than $199 street price. The card supports PCIe 8x passthrough, so the physical connection to the server is 64Gbps - way more than a single SFP28 cage can support. Pretty slick...

deleted 5y ago

[deleted]