5 ms·

My favorite ring buffer structure is like the one described in https://www.gnuradio.org/blog/2017-01-05-buffers/ https://www.gnuradio.org/blog/2017-01-05-buffer

by rdtsc 3y ago

My favorite ring buffer structure is like the one described in https://www.gnuradio.org/blog/2017-01-05-buffers/ https://www.gnuradio.org/blog/2017-01-05-buffers/ > It asks the operating system to give it memory-mappable memory, and map twice, “back to back”. Blocks then get called with pointers to the “earliest” position of a workload within this memory region – guaranteeing that they can access all memory they were offered in a linear fashion, as if they’d actually be dealing with hardware ring buffers. It imposes limitations on hardware and OS support in a way, but I think it's pretty neat. This also used by the kernel BPF ring buffer: https://www.kernel.org/doc/html/latest/bpf/ringbuf.html https://www.kernel.org/doc/html/latest/bpf/ringbuf.html > ...data area is mapped twice contiguously back-to-back in the virtual memory. This allows to not take any special measures for samples that have to wrap around at the end of the circular buffer data area, because the next page after the last data page would be first data page again, and thus the sample will still appear completely contiguous in virtual memory

dist1ll 3y ago
Unfortunately that means the chunks are only contiguous in virtual memory. So it won't work with the DMA use case mentioned in the article, which requires contiguous physical addresses. But it's still a nice trick, I like it when people get creative with HW features.
- jnwatson 3y ago
  Theoretically, the same trick could be used to double map addresses coming from an external master via an IOMMU.
  - speed_spread 3y ago
    It is unlikely that cheap microcontrollers where DMA is most helpful will have an IOMMU.
- gpderetta 3y ago
  But the hardware only needs to see on copy of the duplicated memory and you can let it deal with the wraparound. The software can use the convenience of the double mapping.
gpderetta 3y ago
AKA the Magic Ring Buffer. It extremely convenient not having to deal with split messages, especially if you support variables size payloads.
scottlamb 3y ago
This is a cool trick, and iirc there are a few Rust crates that implement it, including slice-deque. ...but I think there are a few significant downsides, even in userspace: * The most obvious one: you have to write `unsafe`, non-portable code. The Linux, macOS, and Windows implementations are totally different from each other. * The setup and teardown of each ring is a bit expensive (few system calls). For a regular allocation, the malloc implementation typically caches that for you. Here you have to do your own pooling if you might be frequently creating them. * Using whole pages, and two mappings per ring, is wasteful in terms of not only RAM (often no big deal) but also TLB space (which often turns into significant CPU usage). If you just allocate 32 64 KiB rings from the standard allocator, on x86-64 you might be talking about a single 2 MiB huge page mapping. If you do this instead, you're talking about 1024 4 KiB page mappings.
- duped 3y ago
  Any real, production ready ring buffer should be using unsafe. I would consider anything that doesn't to be a toy. In Rust it's basically impossible to do this without MaybeUninit. You could use Option, but then you're paying a massive cost for a very easy to write and audit chunk of unsafe code.
  - scottlamb 3y ago
    I don't think it's useful to consider "uses `unsafe`" as a boolean. A one-line `unsafe` around `MaybeUninit::assume_init` isn't the same as an `unsafe` module per platform wrapping VM operations. Also, it's not that crazy for a byte-oriented buffer to start with a `vec![0u8; N]` (cheap) and not need `MaybeUninit` at all. Probably doesn't buy you that much though; you still want to be careful to not leak previous bytes. Also, you might be missing the point of my comment if you're responding to one word of "the most obvious [downside]" and not the other bullets...
    - duped 3y ago
      It's not an attack on the wording, but the correctness of your first bullet point. `unsafe` is appropriate for the initialization of a ring buffer in Rust. That's true for using `mmap` or anything in "pure" Rust using the allocator API to get the most idiomatic representation (which can't be done in safe or stable Rust). It's not one line. It's also not platform dependent, the code is the same on MacOS, Linux, and Windows the last I tried it. The rest of the bullet points are issues with scaling, which sure, are valid. But if your bottleneck is determined by the frequency at which channels get created or how many exist then I would call architecture into the question. A ringbuffer is a heavy hammer to synchronization problems. It's appropriate in many, but not many times in the same application, in my experience. This last month I've written a lock-free ring buffer to solve a problem and there's exactly one in an application that spawns millions of concurrent tasks.
deleted 3y ago
[deleted]
signa11 3y ago
ah the vm-trick ! have used it current (and previous) places of work to great effect.
- rdtsc 3y ago
  Yup. The other "official" name of it is The Magic Ring Buffer as a sibling comment mentioned.

by rdtsc 3y ago

dist1ll 3y ago

Unfortunately that means the chunks are only contiguous in virtual memory. So it won't work with the DMA use case mentioned in the article, which requires contiguous physical addresses. But it's still a nice trick, I like it when people get creative with HW features.

jnwatson 3y ago
Theoretically, the same trick could be used to double map addresses coming from an external master via an IOMMU.
- speed_spread 3y ago
  It is unlikely that cheap microcontrollers where DMA is most helpful will have an IOMMU.
gpderetta 3y ago
But the hardware only needs to see on copy of the duplicated memory and you can let it deal with the wraparound. The software can use the convenience of the double mapping.

gpderetta 3y ago

AKA the Magic Ring Buffer. It extremely convenient not having to deal with split messages, especially if you support variables size payloads.

scottlamb 3y ago

This is a cool trick, and iirc there are a few Rust crates that implement it, including slice-deque. ...but I think there are a few significant downsides, even in userspace: * The most obvious one: you have to write `unsafe`, non-portable code. The Linux, macOS, and Windows implementations are totally different from each other. * The setup and teardown of each ring is a bit expensive (few system calls). For a regular allocation, the malloc implementation typically caches that for you. Here you have to do your own pooling if you might be frequently creating them. * Using whole pages, and two mappings per ring, is wasteful in terms of not only RAM (often no big deal) but also TLB space (which often turns into significant CPU usage). If you just allocate 32 64 KiB rings from the standard allocator, on x86-64 you might be talking about a single 2 MiB huge page mapping. If you do this instead, you're talking about 1024 4 KiB page mappings.

duped 3y ago
Any real, production ready ring buffer should be using unsafe. I would consider anything that doesn't to be a toy. In Rust it's basically impossible to do this without MaybeUninit. You could use Option, but then you're paying a massive cost for a very easy to write and audit chunk of unsafe code.
- scottlamb 3y ago
  I don't think it's useful to consider "uses `unsafe`" as a boolean. A one-line `unsafe` around `MaybeUninit::assume_init` isn't the same as an `unsafe` module per platform wrapping VM operations. Also, it's not that crazy for a byte-oriented buffer to start with a `vec![0u8; N]` (cheap) and not need `MaybeUninit` at all. Probably doesn't buy you that much though; you still want to be careful to not leak previous bytes. Also, you might be missing the point of my comment if you're responding to one word of "the most obvious [downside]" and not the other bullets...
  - duped 3y ago
    It's not an attack on the wording, but the correctness of your first bullet point. `unsafe` is appropriate for the initialization of a ring buffer in Rust. That's true for using `mmap` or anything in "pure" Rust using the allocator API to get the most idiomatic representation (which can't be done in safe or stable Rust). It's not one line. It's also not platform dependent, the code is the same on MacOS, Linux, and Windows the last I tried it. The rest of the bullet points are issues with scaling, which sure, are valid. But if your bottleneck is determined by the frequency at which channels get created or how many exist then I would call architecture into the question. A ringbuffer is a heavy hammer to synchronization problems. It's appropriate in many, but not many times in the same application, in my experience. This last month I've written a lock-free ring buffer to solve a problem and there's exactly one in an application that spawns millions of concurrent tasks.

deleted 3y ago

[deleted]

signa11 3y ago

ah the vm-trick ! have used it current (and previous) places of work to great effect.

rdtsc 3y ago
Yup. The other "official" name of it is The Magic Ring Buffer as a sibling comment mentioned.