9 ms·

HBPF – eBPF in Hardware

88 points by losfair 5y ago

edude03 5y ago
This is super cool. Is the idea that you could use this as part of a NIC and get higher performance than running eBPF in the kernel?
- tenebrisalietum 5y ago
  Most NICs have one or more CPUs to service offload engines. This could probably happen right now.
- ilovecaching 5y ago
  There are already NICs that have this feature. XDP can run in one of three modes. The firsts mode simply runs the program in the kernel stack and is either for testing or used in the worst case where you have no driver support. The primary mode is for the driver author to write in the invocation of the eBPF program and then implement all the resulting actions into the driver. The third mode is offload, where the eBPF program is loaded into a Smart NIC, FPGA device and the packet is processed without the need for DMA. By far, the most popular form is running in the driver, which can get very close to line rate for most network cards. If you have any questions about this I'd be happy to go into more detail.
  - ciupicri 5y ago
    What NICs?
    - ilovecaching 5y ago
      Someone already mentioned it, but it's a NIC made by Netronome.
r0bot 5y ago
Hi are you the author? What would I need to study to be able to do stuff like this? Are there any specific books or courses you’d recommend? Thanks
- pastage 5y ago
  That is a broad question, going through a FPGA tutorial buying a small dev board. My first project was something similar to this, but a lot simpler.
- ilovecaching 5y ago
  If you have any questions about eBPF I'd be happy to answer them, I work on the kernel portion.
  - nimish 5y ago
    Is it possible to do stateful stuff like NAT in eBPF without having to reimplement conntrack? I think cilium does it but has to reimplement in using maps.
    - ilovecaching 5y ago
      Not in an XDP program, it runs almost immediately after a packet has been rxed, and yes, Cilium does this by reimplementing similar logic in BPF. You could request new helpers for a SCHED_CLS type program though. The helpers don't exist today, but eBPF is use case driven, and if there is a good use case for it, it can go onto the roadmap.
      - nimish 5y ago
        Thanks! I did find this: https://lists.linuxfoundation.org/pipermail/iovisor-dev/2017-September/001023.html https://lists.linuxfoundation.org/pipermail/iovisor-dev/2017... and some references to bpf_skb_ct_lookup/bpf_xdp_ct_lookup -- are these fully baked?
- mhh__ 5y ago
AshamedCaptain 5y ago
First we create a portable bytecode in order to run executable code in a hardware-independent way. Then we create hardware to run that bytecode. (No criticism intended to the author, I guess the goal is just to have fun).
- yjftsjthsd-h 5y ago
  It's happened before; https://en.m.wikipedia.org/wiki/Java_processor https://en.m.wikipedia.org/wiki/Java_processor has a whole list of JVM hardware. I suppose lisp machines might also count.
- klysm 5y ago
  I’ve thought about this kind of co-evolution before and it seems to me it always leads to very sub optimal results. The problem is making a better solution is not economically feasible because all the software already exists.
- uluyol 5y ago
  Same thing happens with IO interfaces. https://lwn.net/Articles/805235/ https://lwn.net/Articles/805235/
- Banana699 5y ago
  I mean, bytecode that runs on hardware is still hardware-independent, it's the hardware that have to conform to the bytecode's interface not the other way around. If every processor in existence had to accept x86 instruction streams as valid programs, x86 would be a hardware-independent VM. The issue would be if there are multiple bytecode formats for different hardware.
aseipp 5y ago
This is really cool. I wonder about whether or not putting the ISA itself directly in hardware is the right path forward, versus "virtualizing" it and compiling the bytecode again down to (yet another) architecture. It's a much better fit for hardware than a lot of "virtual" ISAs, though. There are some reasons you might do this, but then again, reasons not to. Netronome uses this virtual approach instead of baking eBPF into their processors, from my cursory understanding. An advantage of that is agility; FPGAs are only so agile compared to a software compiler and on top of that new evolving features have impacts on things like clock frequency, testing, etc. Beyond that there are also some big "what ifs" about eBPF; for example the fact it just uses the underlying Linux kernel memory model for concurrency. Maybe this is good. Maybe this is bad. I don't know. But it has a big impact on the underlying design of the memory subsystem for any real design. Intuitively I want to say the concurrency abstractions in eBPF should have always been much, much higher level from the start for reasons like this (for example, maybe it would have just been easier to flat-out introduce the concept of transactions, if you're already going to write a verifier to check every incoming program. This is much more difficult than what the verifier does today, but it's also a very well explored one and makes the machine memory model irrelevant. It's not the kind of stuff kernel developers do or think about, though...) But it's not, so it's still definitely something to think about. Do alternative vendors handle this already if they offload programs? There is also the current state we're in where "what is a valid eBPF program" mostly just comes down to "whatever the Linux kernel says is a valid eBPF program", so you can probably get into situations where code emitted by Clang will work in one of the two, and not the another (note: you can already get into situations where Clang will emit code from valid input that the kernel then rejects, so this probably isn't a very difficult example to construct.) This project already has some differences in ABI versus Linux, though, so... Overall this is still a really cool project, though. You'd have to do something like this to answer all the above, and it bills itself as an experiment. You could still have a lot of fun with this even ignoring all that. And it doesn't use Verilog. 10/10.
- ilovecaching 5y ago
  > for example, maybe it would have just been easier to flat-out introduce the concept of transactions, if you're already going to write a verifier to check every incoming program Could you elaborate on this? Loading the program is a one time operation. The actual evocation is dependent on the program type and its location in the kernel. I do not believe there is a blanket way to describe how eBPF interacts with the VM or its concurrency model. To your second point about verification, yes this is something that is actively being worked on. with BTF and CO-RE we've made huge strides over module loading, however, and eBPF already has many advantages over use cases where people were forced to load bespoke kernel bypass modules.
ncmncm 5y ago
A NIC company Netronome has had something like this (i.e. running physically on the NIC) as a product for a couple of years now. They have a NIC with lots of proprietary cores that with eBPF can be made to run user code. To be useful for capture it needed an extra feature, ability to grab a synchronized hardware time counter value, and deliver to a user-space buffer. Writing to user space memory, for kernel bypass, at least once was an underdeveloped eBPF feature; don't know its current condition.

88 points by losfair 5y ago

This is super cool. Is the idea that you could use this as part of a NIC and get higher performance than running eBPF in the kernel?

tenebrisalietum 5y ago
Most NICs have one or more CPUs to service offload engines. This could probably happen right now.
ilovecaching 5y ago
There are already NICs that have this feature. XDP can run in one of three modes. The firsts mode simply runs the program in the kernel stack and is either for testing or used in the worst case where you have no driver support. The primary mode is for the driver author to write in the invocation of the eBPF program and then implement all the resulting actions into the driver. The third mode is offload, where the eBPF program is loaded into a Smart NIC, FPGA device and the packet is processed without the need for DMA. By far, the most popular form is running in the driver, which can get very close to line rate for most network cards. If you have any questions about this I'd be happy to go into more detail.
- ciupicri 5y ago
  What NICs?
  - ilovecaching 5y ago
    Someone already mentioned it, but it's a NIC made by Netronome.

r0bot 5y ago

Hi are you the author? What would I need to study to be able to do stuff like this? Are there any specific books or courses you’d recommend? Thanks

pastage 5y ago
That is a broad question, going through a FPGA tutorial buying a small dev board. My first project was something similar to this, but a lot simpler.
ilovecaching 5y ago
If you have any questions about eBPF I'd be happy to answer them, I work on the kernel portion.
- nimish 5y ago
  Is it possible to do stateful stuff like NAT in eBPF without having to reimplement conntrack? I think cilium does it but has to reimplement in using maps.
  - ilovecaching 5y ago
    Not in an XDP program, it runs almost immediately after a packet has been rxed, and yes, Cilium does this by reimplementing similar logic in BPF. You could request new helpers for a SCHED_CLS type program though. The helpers don't exist today, but eBPF is use case driven, and if there is a good use case for it, it can go onto the roadmap.
    - nimish 5y ago
      Thanks! I did find this: https://lists.linuxfoundation.org/pipermail/iovisor-dev/2017-September/001023.html https://lists.linuxfoundation.org/pipermail/iovisor-dev/2017... and some references to bpf_skb_ct_lookup/bpf_xdp_ct_lookup -- are these fully baked?
mhh__ 5y ago

AshamedCaptain 5y ago

First we create a portable bytecode in order to run executable code in a hardware-independent way. Then we create hardware to run that bytecode. (No criticism intended to the author, I guess the goal is just to have fun).

yjftsjthsd-h 5y ago
It's happened before; https://en.m.wikipedia.org/wiki/Java_processor https://en.m.wikipedia.org/wiki/Java_processor has a whole list of JVM hardware. I suppose lisp machines might also count.
klysm 5y ago
I’ve thought about this kind of co-evolution before and it seems to me it always leads to very sub optimal results. The problem is making a better solution is not economically feasible because all the software already exists.
uluyol 5y ago
Same thing happens with IO interfaces. https://lwn.net/Articles/805235/ https://lwn.net/Articles/805235/
Banana699 5y ago
I mean, bytecode that runs on hardware is still hardware-independent, it's the hardware that have to conform to the bytecode's interface not the other way around. If every processor in existence had to accept x86 instruction streams as valid programs, x86 would be a hardware-independent VM. The issue would be if there are multiple bytecode formats for different hardware.

aseipp 5y ago

This is really cool. I wonder about whether or not putting the ISA itself directly in hardware is the right path forward, versus "virtualizing" it and compiling the bytecode again down to (yet another) architecture. It's a much better fit for hardware than a lot of "virtual" ISAs, though. There are some reasons you might do this, but then again, reasons not to. Netronome uses this virtual approach instead of baking eBPF into their processors, from my cursory understanding. An advantage of that is agility; FPGAs are only so agile compared to a software compiler and on top of that new evolving features have impacts on things like clock frequency, testing, etc. Beyond that there are also some big "what ifs" about eBPF; for example the fact it just uses the underlying Linux kernel memory model for concurrency. Maybe this is good. Maybe this is bad. I don't know. But it has a big impact on the underlying design of the memory subsystem for any real design. Intuitively I want to say the concurrency abstractions in eBPF should have always been much, much higher level from the start for reasons like this (for example, maybe it would have just been easier to flat-out introduce the concept of transactions, if you're already going to write a verifier to check every incoming program. This is much more difficult than what the verifier does today, but it's also a very well explored one and makes the machine memory model irrelevant. It's not the kind of stuff kernel developers do or think about, though...) But it's not, so it's still definitely something to think about. Do alternative vendors handle this already if they offload programs? There is also the current state we're in where "what is a valid eBPF program" mostly just comes down to "whatever the Linux kernel says is a valid eBPF program", so you can probably get into situations where code emitted by Clang will work in one of the two, and not the another (note: you can already get into situations where Clang will emit code from valid input that the kernel then rejects, so this probably isn't a very difficult example to construct.) This project already has some differences in ABI versus Linux, though, so... Overall this is still a really cool project, though. You'd have to do something like this to answer all the above, and it bills itself as an experiment. You could still have a lot of fun with this even ignoring all that. And it doesn't use Verilog. 10/10.

ilovecaching 5y ago
> for example, maybe it would have just been easier to flat-out introduce the concept of transactions, if you're already going to write a verifier to check every incoming program Could you elaborate on this? Loading the program is a one time operation. The actual evocation is dependent on the program type and its location in the kernel. I do not believe there is a blanket way to describe how eBPF interacts with the VM or its concurrency model. To your second point about verification, yes this is something that is actively being worked on. with BTF and CO-RE we've made huge strides over module loading, however, and eBPF already has many advantages over use cases where people were forced to load bespoke kernel bypass modules.

ncmncm 5y ago

A NIC company Netronome has had something like this (i.e. running physically on the NIC) as a product for a couple of years now. They have a NIC with lots of proprietary cores that with eBPF can be made to run user code. To be useful for capture it needed an extra feature, ability to grab a synchronized hardware time counter value, and deliver to a user-space buffer. Writing to user space memory, for kernel bypass, at least once was an underdeveloped eBPF feature; don't know its current condition.