6 ms·
Line rate 10Gbit/s packet processing on FreeBSD with netmap
- sc68cal 15y agoAs many have said on -CURRENT: Thanks for the great work. Do you plan on getting this merged into the FreeBSD tree, and maintaining it?
- mateuszb 15y agoVery clever solution on stack bridging. +1
- smutticus 15y agoThis is impressive. How is the performance on different packet sizes? Most real world loads would require far fewer than 14.88Mpps since most packets would be equal to MTU. So I'm wondering if performance would increase or decrease with packet size.
- lrizzo 15y agoplease look at the paper linked to the url, which defines the problem and the metrics in more detail.
- smutticus 15y agoThanks. This is what I was looking for. From the linked paper: The most challenging situation is usually with the shortest packet sizes, where per-packet costs are almost unavoidably dominant over the memory copy costs. This justifies why we will run most of our tests with 64 byte packets (60 + 4 for the Ethernet CRC). In netmap, in particular, the per-byte system CPU cost is exactly zero because all data transfers are performed by the NIC, and it is up to applications to access memory, if it really needs to.
- muppetman 15y agoThis is really very interesting. I work with Juniper routers and switches all day and every day, the thing I love about them is they're built on top of FreeBSD (now called JunOS, but it still harks back to FreeBSD when you drop to the shell) Something small and light and fast, running on FreeBSD with this added would be perfect for customer CPE equipment or for non-essential routing tasks. Very impressive, I look forward to seeing where this goes.
- tptacek 15y agoThe data path on a Juniper isn't passing mbufs around the FreeBSD IP stack, is it? It's FreeBSD command-and-control, but my understanding is that the packet processing itself might as well be its own rtos.
- tryp 15y agoI don't know about juniper specifically but most multiport routers are built on packet processing asics with content addressable storage in local ram and built-in macs for each port. When a packet is encountered that requires more complex interaction than the asic can handle it's kicked up to a CPU for processing.
- Nrsolis 15y agoThis. Juniper has always used a (heavily hacked) FreeBSD core running on an embedded PC platform (Intel-based Routing Engine) for JUNOS that did all (or most) of the protocol related stuff and left the actual forwarding of packets to a specialized core of ASICs (The Packet Forwarding Engine). This has changed in recent years with some lower-level protocols being handled on the line-cards themselves on newer platforms. The vast majority of the protocols still run on the Routing Engine though.
- tptacek 15y agoYeah, sorry, I was talking about the control plane stuff. Does Juniper use any of the original FreeBSD IP stack for its control plane or exception path stuff?
- codepoet 15y agoGreat! Maybe you could compare this with http://www.ntop.org/PF_RING.html http://www.ntop.org/PF_RING.html for Linux?
- lrizzo 15y agoThe "old" pf_ring is only meant for packet capture and involves packet copies, so it is several times slower than netmap. There is a newer "Direct Network Access" (DNA) version of pf_ring which avoids copies and has the same performance of netmap, but is much more fragile because in DNA the userspace program writes directly into the NIC registers and rings (so it can crash the entire OS), whereas in netmap the NIC programming is filtered by system calls.