12 ms·
A PCIe network interface card that adds full router capabilities to your servers
- cute_boi 5y ago10/100/1000 Ethernet ports I wonder why they need to support 10 mbps port? Is it just because if the card supports 1000 mbps it will support 10mbps effortlessly?
- xxs 5y agoyes, 10 is just a single twisted pair and if the cable is faulty the connection can degrade to it.
- jaywalk 5y agoI have had bad cables degrade from 1000 to 100, and one time had to force a shoddy (and very temporary) connection to 10 for it to work at all. So there is definitely a use for it.
- Maxburn 5y agoI still have quite a lot of equipment in the field that is 10/half. PLC's that control commercial HVAC are expected to last the life of the building, at least until a refurb or two. Cisco has some switches that can't go down to 10, which makes it interesting when those show up on site and the HVAC system can't link up any more.
- rubatuga 5y agoNope, you need two twisted pairs!
- hnlmorg 5y ago10baseT is a single twisted pair. It’s 100baseT that requires two twisted pairs but that’s 100Mbs rather than 10Mbs. It used to be common run 10Mbs over coax too, back before Ethernet took over.
- Dylan16807 5y agoCitation needed. There's a 10BASE-T1 but this says it's very recent? 10BASE-T and 100BASE-TX are very similar except for the line encoding. One pair each way. Coax uses one line, but that's not using twisted pairs at all.
- assttoasstmgr 5y ago*-T1 Ethernet was designed by Broadcom and the car manufacturers to implement single pair ethernet for automotive applications. Specifically for things like backup cameras, ADAS, etc. The standard is less than 10 years old and has nothing to do with 10base-T. https://en.wikipedia.org/wiki/BroadR-Reach https://en.wikipedia.org/wiki/BroadR-Reach 100Base-T1 has more in common with 1000BASE-T than the legacy standards, imagine if you took a single pair from the 4 needed to do Gigabit.
- assttoasstmgr 5y agoThis is simply incorrect. 10Base-T is two pairs, one TX one RX. Source: am expert, have designed low level ethernet hardware. It amazes me how much misinformation gets posted on HN with convincing authority.
- hnlmorg 5y ago10base-T1S and 10base-T1L are single pair. Though I didn’t realise they’re a modern standard until I just looked it up. Coax is also two ”wires”, though obviously not twisted. I used to do networking professionally too. Though it looks like I’ve gotten rather rusty on the basics.
- Dylan16807 5y agoAnd it has to be a particular two. It's a very situational bonus rather than proper graceful degradation.
- jacquesm 5y agoYou get the 10 mbps capability for free because that is what the auto-negotiation protocol will use: https://en.wikipedia.org/wiki/Autonegotiation#Electrical_signals https://en.wikipedia.org/wiki/Autonegotiation#Electrical_sig...
- toast0 5y agoThat's not what that says. Autonegotiation (ab)uses the 10BaseT link pulses to communicate, but that data rate is 1Mbps. You could use the link pulses, but skip 10M support and leave out the 10M data encoding/decoding. Most likely, it's not a meaningful cost savings, although I've seen some devices that work at 10M, but don't turn on a link led.
- ACAVJW4H 5y agoSTH has a bit more information on the card https://www.servethehome.com/mikrotik-ccr2004-1g-2xs-pcie-is-a-2x-sfp28-25gbe-router-on-a-pcie-card/ https://www.servethehome.com/mikrotik-ccr2004-1g-2xs-pcie-is...
- candiddevmike 5y agoMy Linux server already has "full router capabilities" AND I don't have to use RouterOS to configure it (which is just a shit abstraction on top of common Linux network services like iptables).
- cute_boi 5y agoI think the use case is to reduce CPU usage. Its like GPU cards, but for networking.
- xxs 5y agoBy just reading the title you can tell it runs an Arm cpu with linux on it. Not really certain how useful that is.
- 293984j29384 5y agoI'm not sure where the confusion is. OP mentioned that his Linux system can already do routing. The purpose of this card is to remove that load from the computer. The manufacture suggests it can do up to 100Gbps which isn't trivial.
- xxs 5y ago>>reduce CPU usage. Its like GPU cards, It uses another CPU to do that. GPU is fundamentally different, high memory bandwidth, embarrassingly parallel, virtually no branches, and what not. That's just using a different CPU to do more CPU, and using the same OS the host already runs. Then it requires its own security maintenance (+training) and patches.
- benou 5y agoPersonally I think one of the real usecase for smartnic is isolation: for a cloud provider, you can rent a bare metal instance and run all your networking security stack (think encapsulation, filtering, throttling etc) on the smartnic. IOW the customer has full control of the host, but the cloud provider manages the smartnic. Incidentally, this is exactly what AWS does with their ENA adapters designed by... (ex-?)Anapurna Lab they bought some years ago (:
- nanochad 5y agoRouting should be done in software.
- vetinari 5y agoIt is. Just the software doing the routing is not running on your main CPU, but on the CPU bundled on the board.
- kazen44 5y agoplease define routing? actual packet forwarding should be done in hardware, because software forwarding has atrocious performance in comparison.
- wolandpe 5y agoyeah, atrocious: I can reach roughly 20Gbps routed on my not too expensive home Opnsense box with firewalling and many other features (which is absolutley unnecessary for me). No routing by a CPU is not for routing between Comcast and Google, but it has it´s place and works so well, that a lot of opensource routing projects were supported by big vendors and telcos. Also networking for containers depends on this...
- AdrianB1 5y agoThe first time when I read about it on ServeTheHome I had no idea what this can be used for. Then I saw the price and my jaw dropped, it is cheaper than a basic NIC with dual 25 Gbps ports. Together with the CPU and RAM on it, it makes a lot of sense for specific use cases and the price is appealing: for a Small or Medium Business with some servers and not a lot of dedicated network equipment, it allows to move the router/firewall inside the server case, combining it with the NIC at a good price and without eating up any of the server resources. Do you want a cheap dual-port NIC at 25Gbps? How about we add some solid router capabilities on it for no extra price?
- compsciphd 5y agoused Mellanox cx3 (qfsp, 40gbps) cards go for $30 or so on ebay and can go lower (I bought 5 a while back for $75 total).
- vetinari 5y agoHow long ago was that? I've bought recently newish dual-port (SFP+, 10Gbit) Connect-X 3 Pro at 80 GBP per piece. And that was one of the better prices.
- kube-system 5y agoThe older QSFP cards go for very cheap here in the US. $20 right now on eBay. SFPs go for more, and even more for the dual interface cards.
- compsciphd 5y agothis is the cheapest I see right now on ebay for dual port card ($35) so perhaps a bit higher than what I remember from a year ago (I guess silicon shortage effects everything). https://www.ebay.com/itm/265592690915 https://www.ebay.com/itm/265592690915
- egberts1 5y agoNot seeing any mention of Data Center Bridging Protocol there. https://en.m.wikipedia.org/wiki/Data_center_bridging https://en.m.wikipedia.org/wiki/Data_center_bridging
- wmf 5y agoYeah, at that price it may be missing some features you'd expect from a normal NIC.
- synergy20 5y agoKnowing Mikrotik for like 2 decades, it should do better than UBNT really. Mikrotik still produces great hardware, but it's totally eclipsed by Ubiquitous Networks these years. It's kind of like watching digitalocean the new cool kid playing the same tricks overtakes linode, sigh.
- core-utility 5y agoMikrotik misses the "polished" aspect still, that UBNT does well. As someone with moderate enterprise network experience, setting up RouterOS as a basic L3 switch was way more difficult than it should have been. That being said, once I was done I haven't had to think twice about the switch, it just works (which should be default, but isn't always the case).
- gh02t 5y agoTo be fair to Mikrotik if you just want basic/intermediate switch they have SwOS, which is FAR easier to set up. I also find RouterOS to be extremely unituitive, but SwOS is a breeze. I think most of their switches can run either and even dual boot.
- synergy20 5y agowhat's the goal for SwOS(new to me), replacing RouterOS?
- gh02t 5y agoIt's a dumbed down RouterOS (afaik they use the same base but I'm not certain) with a lot of features stripped out to focus on just basic switching features. It also has a vastly simplified webui. As the other commenter noted you lose a lot of the remote management features and of course it hides lots of the stuff RouterOS can do, but that's the point. If you only need set it once and be done basic switching, SwOS is vastly simpler.
- vetinari 5y ago
- SamuelAdams 5y agoI currently have a ProtectCLI vault device running PFSense for my router. I also have a TrueNas / FreeNas device (Supermicro board with Xeon 26xx processor, 2x 1Gbps ports). I’ve been wanting 10 Gbps networking for some time but I’ve been undecided how to best do that. Could I simply get this card, drop it in my FreeNas box, then plug my Arris S33 modem into the card, then the card to my network switch? Would the FreeNas host also get 10/25 Gbps virtually, or do I still need another card specifically for the FreeNas box?
- mjh2539 5y agoYour switches and all client devices would have to have 10Gbps+ NICs/be 10Gbps+ capable.
- bombcar 5y agoI got the four SPF+ port microtic, some eBay 10GB cards for my VM server and my ZFS NAS, and connected one port to each, along with one to the 10GB uplink on the old Nortel switch and one to the 10GB port on the Mac (that one is the only one that was cable ethernet instead of fibre or direct connect). Works fast and well. The fifth "management" 1GB port goes to my router, 1GB is way faster than my internet anyway.
- nimbius 5y agoim seeing a lot of "my router" and "my computer" threads so its probably worth it to say this isnt for your home network. Mikrotik is targeting larger customers with a product that handles offloading to the ASIC's on the board, which is far more performant and scalable than COTS ethernet cards or the onboard gigabit. the reason you would slap a router card in your rackmount server is because its an IOMMU passthrough to a k8s service load balancer or straight up just openstack and the push toward hyperconvergence. the switch is already virtual inside the kvm on openvswitch (has been for a decade now), but the router is still hardware and this product aims to solve that problem.
- kiallmacinnes 5y ago> this device can handle a lot: firewalls, user management and access control for home media and file servers, and even some traffic control in data centers – without the need for a stand-alone router. It's definitely got home networks as a target market. It's one of the suggested use cases.
- core-utility 5y agoIt's also worth saying that Mikrotik is a common platform for "homelabbers" who use enterprise-grade (ish) hardware in their homes. RouterOS isn't without its flaws and pain points, but Mikrotik brings high quality features into a low cost package that appeals to many. It's the lesser-known (and polished) brother of what Ubiquiti used to be.
- 5y ago
- kkielhofner 5y agoNeat concept but I wonder why the PCIe initialization delay can't be handled with an option ROM. I don't know that a fully fledged option ROM would add value but it seems like it could be a good workaround/hack to not require additional BIOS configuration or support a BIOS that doesn't allow configuration of a delay. I've seen some option ROMs take 10 seconds or more depending on the card - hardware RAID controllers being a well known example.
- wmf 5y agoMikrotik probably can't afford to develop an option ROM.
- pilsetnieks 5y agoWhat makes an option ROM so expensive?
- sigstoat 5y agoi don't know anything about mikrotik's hardware development practices and don't want to besmirch them, but some places basically just fab reference designs, or smoosh a couple of reference designs together, and have a product. it helps to keep costs down if you don't have any NRE.
- rsync 5y agoInteresting ... so if I could find a server board with no other network ports and then put this card in, I could finally build a wire-speed multi-gigabit "network slug"[1]. [1] https://john.kozubik.com/pub/NetworkSlug/tip.html https://john.kozubik.com/pub/NetworkSlug/tip.html
- runnerup 5y agoJust watch out for Amazon Sidewalk! Your consumer TV could connect to your neighbors' Amazon Echo wirelessly to continue sending screenshots (or hashes of screenshots) to Amazon and its marketing partners. [0]: https://www.washingtonpost.com/technology/2021/06/07/amazon-sidewalk-network/ https://www.washingtonpost.com/technology/2021/06/07/amazon-...
- bombcar 5y agoIn theory you could configure RouterOS to be your slug itself, and provide PCIe power and no computer at all to slug this.
- deleted 5y ago[deleted]
- 10000truths 5y agoHell, we could get rid of the PCIe connector to give it a nicer form factor. I think we're on to something here...
- sitkack 5y agoThe xilinx pcie boards with dual sfp connectors would be a good choice as well.
- cft 5y agoThey advertise it as having "full routing capabilities", but I am not sure if 4GB of RAM can keep the full ipv4/6 routing tables? Edit: never mind, it says it's primarily for home use
- Alupis 5y agoI'm not certain what you mean. 4GB of RAM is far more than plenty for nearly anything. This card isn't going to be the core router for Comcast or anything... but for what it's worth BGP definitely requires far less than 4GB of RAM, although it depends on the exact implementation of course. And this card is highly unlikely to be targeted for home use - mostly service providers doing routing within their private networks.
- deleted 5y ago[deleted]
- oneplane 5y agoEssentially it's a single board computer with two network interfaces, one on the PCIe side, one on the bracket side. This has been done before with the likes of DSL modems that weren't actually modems but just router-on-a-card that would just have a Realtek PCI chip on the bus side, which then directly had its GMII interface hooked up to a conexant DSL modem/router package which itself then connected to the actual on-board modem.
- zelon88 5y agoSo can you add more regular NICs and then use them as router ports?
- Nextgrid 5y agoNo. The ports on this "NIC" are actually connected to the router, though they can be passed through to the host if needed. The ports on another NIC would be assigned directly to the host. While I'm sure you can theoretically redirect them to this router wit a combination of VLANs and other Linux networking magic, you will be limited by your CPU and it's unlikely you'll manage more than a few Gbps.
- oneplane 5y agoNo, but at that point you are building a router anyway, so you might as well run the router software on the host directly instead of on the SBC host inside the main host. Keep in mind that most routers are just computers too. Sometimes they are low-power computers with special hardware components to offload specific tasks so you can trade power for specialisation (which also comes with a rigidity trade-off, you can't change the hardware after the fact like for new protocols).
- yewenjie 5y agoIn layperson terms, what are these 'full router capabilities' and why would one want those?
- deleted 5y ago[deleted]
- symlinkk 5y agoFor real. I’ve read through half this thread and have no idea what this is for.
- Melatonic 5y agoWhile this seems cool for some implementations there is a reason we often have separate boxes for compute / storage / routing. Some of these are much more critical to have consistently running than the others and it also means it is easier to swap out and do upgrades without having to worry about affecting the other parts of the pie. I think virtualized networking devices like routers are definitely the future but I would still much rather have it as its own separate physical box so that if some hardware fault in a server takes it down the network still functions (not to mention having them on different UPS hardware or different levels of redundancy.) And with servers getting smaller and smaller and the compute required getting more and more power friendly I do not see this as something I would like to use unless I was EXTREMELY space constrained. Where I can see this being super cool though is niche use cases like highly portable servers and whatnot for things like VFX shoots. I once was contracted to built a set of highly mobile and durable servers for mobile rendering of 8K footage. I built the servers into some super durable hard case boxes that are usually used for shipping things like expensive camera equipment, military hardware, etc. The cases even have a valve to equalize pressure in case they get pushed deep underwater (like in the event of a boat capsizing) and a very robust waterproof gasket. Of course for the servers to be running the case must be open (mainly for cooling) but it would have been interesting to network multiple of them together AND other equipment without needing a separate physical device for routing. It would also have made scaling the system much easier if each server could also act as a router - you could bring one or 10 and each could function independently of each other.
- stragies 5y agoDoes anybody know, if Openwrt for this is planned/feasible/complicated/...? ARM64 sounds like basic boot could be easy, but the CPU name (AL52400) top search hits are from the Mikrotik product page. Is something known about the rest of the components?
- wolandpe 5y agoIt´s too early to tell. This card was hit by the chip shortage, estimated delivery times here in EU begin with Mai. So it will take some time for Openwrt devs to lay hands on this. There is lot of ongoing successful work (for example RB5009) on other Mikrotik devices and the HW is fairly hackable. Anyway RouterOS is worth a try, it gives you probably much better performance than Openwrt. You can see a lot of comments like it´s being unstable and lacking features. IMHO that comes from the bad relesae timing by MT. They have released RoS 7 as stable in a state of rather being beta. It have seen lots of improvements lately. So until we can buy one Ros 7 will probably be OK for production use.
- oliwarner 5y agoAn embedded device running inside my server I cannot audit, with direct memory access to everything running? Feels like the Holy Grail of backdoors.
- wolandpe 5y agoCan you audit your CPU? Can you audit your mainboard, which was probably assembled in China and has some microcontrollers with firmware installed in China? Mikrotik is made in Europe in a democratic state.
- oliwarner 5y agoI cannot. But this is another software stack, another CPU and another mainboard with DMA access. It's a big bucket of additional weak links in my chain of security security. The whattabouttery in these replies isn't s good approach. It only takes one component to get hacked. More than doubling your surface area isn't something to do lightly.
- cduzz 5y agoA modern (server) system probably has 3-8 of these already, some of them explicitly with independent network connectivity. Trust your vendors, lock down your network, be large enough to build your stack yourself; chose any 2...
- blibble 5y agoturn on your IOMMU
- johnklos 5y agoProprietary OS from a company that has gotten caught with their proverbial security pants down around the ankles? No, thank you. When this can run non-Mikrotik open source software, this'll be great!
- wolandpe 5y agoHuhh, all big vendors have made and are still making similiar mistakes on a regular basis, just look up Cisco related CVEs. While I don´t know it for sure yet, as this card is not yet available at my favorit suppliers, this might very well work with Linux. While Mikrotik does not directly support installing Linux on their HW, they don´t lock the HW down. There is support by Openwrt for many different Mikrotik devices. Anyway I wouldn´t replace RouterOS.
- pilsetnieks 5y ago> that has gotten caught with their proverbial security pants down around the ankles So just like any other major networking provider, including opensource projects?
- teleforce 5y agoAt USD$199 I think this could be a steal. Combine this with the new low-cost Intel CPU and they could be a game changer especially at the edge and data center environment [1]. Please check out this Kamuee software router and it has the performance of 100 Gbps without using SmartNIC based open soure Quagga/FRR open source routing suite although it's proprietary technology by NTT [2]. Perhaps it's not coincident that the original Zebra open source routing suite pre-cursor to the Quagga/FRR is also written by a Japanese. Another related and promising software router technology from the startup Netris using SmartNIC [3],[4]. It aspires to provide automatic network operations platform that turns the physical network into a service like a cloud. [1] Welcome to the Intel Ice Lake D Era with the Xeon D-2700 and D-1700 series: https://news.ycombinator.com/item?id=30568011 https://news.ycombinator.com/item?id=30568011 [2] 100 GbE High Speed Software PC Router "Kamuee": https://www.ntt.com/en/lp/router/Kamuee.html https://www.ntt.com/en/lp/router/Kamuee.html [3] SmartNIC Linux router for modern data center: https://www.netris.ai/how-to-select-a-network-gateway-for-your-private-cloud/ https://www.netris.ai/how-to-select-a-network-gateway-for-yo... [4] SmartNIC Linux router for modern data center (HN discussions): https://news.ycombinator.com/item?id=30260974 https://news.ycombinator.com/item?id=30260974
- bozhark 5y agoQuagga was my top scoring word in words with friends back in the day. Played it against a friend who couldn't believe I wasn't cheating. It's an extinct Half-Striped Zebra. I was cheating.
- 71a54xd 5y agoThis is something I've been wanting for years to solve a huge painpoint in smaller colocation deployments. Basically, it sucks to have to pay extra for a 1u space and power just for a dedicated router. Virtualizing a router / trusting a hypervisor to keep a virtual networking appliance (connected to raw internet) separate is certainly not something I trust. (maybe I'm just old school?) This completely solves the issue, in a much better way than me just shipping a 2u server to a DC with a small micro router shoved inside the case and powered with a micro-usb cable haha. Can't wait to buy one of these.
- Thlom 5y agoCisco have had this in the form of the Cisco 5921 Embedded Service Router for years already.
- _8j50 5y agoSo, give an OS on a SOC PCIe access? Is it possible to flash or image the router OS with custom libre firmware/OS at least?