7 ms·
BPF for storage: an exokernel-inspired approach
- hardwaresofton 6y agoI wonder if Unikernels would also see this speedup, though they're so much harder to implement/use that I assume BPF-powered storage will see mass adoption before unikernels do...
- jankotek 6y agoMany databases can bypass filesystem and use block device (partitions) directly. But modifying NVMe drivers is pretty novel approach. Maybe Redis compiled into Linux kernel is not bad idea :)
- appleflaxen 6y agoHoly crap this is a cool idea.
- nynx 6y agoExokernels coming back would be so cool. I wrote a prototype kernel/os [0] a few years back that ran wasm in kernel mode. I'd love to see that idea pursued further. [0]: https://github.com/nebulet/nebulet https://github.com/nebulet/nebulet
- jFriedensreich 6y agoI really loved the idea of nebulet! Do you think it is possible to reach the same security guarantees of eBPF with a feauture set like wasm? Other big challenges you encountered?
- nynx 6y agoI think the main issue is that eBPF cannot have loops. It restricts the programs you can write. Wasm does not have that property, so you cannot prove that it will complete.
- rkeene2 6y agoTo be clear, eBPF programs can have loops -- they're just jumps that have negative offsets (which are signed 16-bit numbers), but for security reasons many verifiers do not allow them so they can ensure that the program halts.
- smasher164 6y agoI'd like to see more proof-carrying code techniques extended to WASM. Some of the work could be handed off to the compiler, embedding the proof in the binary. This could make expensive proofs, like termination checking, more tractable.
- rubatuga 6y agoare you aware of the halting problem?
- smasher164 6y agouh..yeah. The halting problem doesn’t mean you can’t do termination checking. It just means there’s no general algorithm for deciding if a machine halts. We only care about the specific subset of programs that the compiler is able to prove properties about. https://en.m.wikipedia.org/wiki/Termination_analysis https://en.m.wikipedia.org/wiki/Termination_analysis
- wtallis 6y agoThere's also some interest in making eBPF the standard for computational storage [1]: offloading processing to the SSD controller itself. Many vendors have found that it's easy to add a few extra ARM cores or some fixed-function accelerators to a high-end enterprise SSD controller, but the software ecosystem is lacking and fragmented. This work may be a very complementary approach. Using eBPF to move some processing into the kernel should make it easier to later move it off the CPU entirely and into the storage device itself. [1] https://nvmexpress.org/bringing-compute-to-storage-with-nvm-express/ https://nvmexpress.org/bringing-compute-to-storage-with-nvm-...
- toolslive 6y agowasn't it the other way around ? The vendors got extra cores from their suppliers and started asking themselves what to do with them?
- wtallis 6y agoBy "vendors", I meant drive and controller vendors rather than server/appliance vendors. They're looking for ways to differentiate their products and offer more value add, but extending an SSD's functionality beyond mere block storage (potentially with transparent compression or encryption) requires a lot of buy-in from end users who need to write software to use such capabilities.
- toolslive 6y agoI meant the Seagates and WD's from this world. Around 2014 they also had HDDs with an extra core, where I suspect they just got that for free from they supplier with the message "look, we stopped making these single core CPUs anymore, here's a dual core for the same price".
- monocasa 6y agoWD and Seagate are their chip vendors. They design their own custom SoCs.
- musicale 6y agoThis kind of nails what has been obvious to me for a long time: eBPF is basically the revenge of exokernels.
- kijiki 6y agoEverything old is new again. https://en.wikipedia.org/wiki/Channel_I/O https://en.wikipedia.org/wiki/Channel_I/O
- dathinab 6y agoAnd then there where "in" eBPF Drivers And they where compiled on the fly Compatible forever Running non stop until the end of all time.
- cryptonector 6y agoReminds me of the old Auspex file servers, with their functional processing units. For a while CPUs got fast enough to do everything. Then they didn't, and GPUs became a thing (though they had been something of a thing back in the 80s too, like the Amiga blitter), and now we're putting CPUs in storage controllers. The pendulum has swung a few times.
- nwmcsween 6y agoWhat is disappointing is even with newer OS's they seems to ignore exokernels and go straight for older microkernel style architecture, which is 1980's tech