5 ms·

A Peek at GNU Radio’s Buffer Architecture

142 points by ptr 10y ago

hoytech 10y ago
The double-mapped ring buffer trick is so clever I can't help but wish I invented it. :) Here's the code for the GNU radio implementation described in the article, it's a good reference for some various ways to set up the mappings: https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/vmcircbuf_createfilemapping.cc https://github.com/gnuradio/gnuradio/blob/master/gnuradio-ru... https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/vmcircbuf_mmap_shm_open.cc https://github.com/gnuradio/gnuradio/blob/master/gnuradio-ru... https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/vmcircbuf_mmap_tmpfile.cc https://github.com/gnuradio/gnuradio/blob/master/gnuradio-ru... And here some other links to implementations I've collected: http://vrb.sourceforge.net/ http://vrb.sourceforge.net/ https://fgiesen.wordpress.com/2012/07/21/the-magic-ring-buffer/ https://fgiesen.wordpress.com/2012/07/21/the-magic-ring-buff... https://gist.github.com/rygorous/3158316 https://gist.github.com/rygorous/3158316 https://www.mikeash.com/pyblog/friday-qa-2012-02-03-ring-buffers-and-mirrored-memory-part-i.html https://www.mikeash.com/pyblog/friday-qa-2012-02-03-ring-buf... https://www.mikeash.com/pyblog/friday-qa-2012-02-17-ring-buffers-and-mirrored-memory-part-ii.html https://www.mikeash.com/pyblog/friday-qa-2012-02-17-ring-buf... https://github.com/mikeash/MAMirroredQueue https://github.com/mikeash/MAMirroredQueue http://atastypixel.com/blog/a-simple-fast-circular-buffer-implementation-for-audio-processing/ http://atastypixel.com/blog/a-simple-fast-circular-buffer-im... https://github.com/michaeltyson/TPCircularBuffer https://github.com/michaeltyson/TPCircularBuffer http://www.koanlogic.com/libu/api/html/group__rb.html http://www.koanlogic.com/libu/api/html/group__rb.html https://github.com/ulfalizer/botniklas/blob/master/src/read_msg.c https://github.com/ulfalizer/botniklas/blob/master/src/read_... https://github.com/willemt/cbuffer https://github.com/willemt/cbuffer
zump 10y ago
SDR strikes me as good in theory, but breaks down for anything practical. My core 2 duo was 100% cpu utilization simply from decoding 56kbps 2-BPSK, even with SIMD extension enabled (within GR).
- viraptor 10y ago
  Any other prefiltering / post-processing? This processing should be pretty trivial on its own. This should be fairly easy load - definitely not pegging the CPU at 100%.
- lmh 10y ago
  You may find this SDR LTE eNodeB interesting: http://bellard.org/lte/ http://bellard.org/lte/ There's also this open-source LTE library: https://github.com/srsLTE/srsLTE https://github.com/srsLTE/srsLTE
- derefr 10y ago
  SIMD is only going to get you so far; I'd expect much greater gains from running on a GPU.
- ChuckMcM 10y ago
  I run a HackRF One on an i7 (Intel NUC) and it is hard to peg the Cores. Typically when I do it is because some synthetic workload is dumping infinite work into the buffers and so everything is running full on, but generally real signals have been 20 - 30% of one or two cores.
- jcims 10y ago
  Something's terribly wrong with your flowgraph.
- nikanj 10y ago
  Core 2 duo was released in 2006, is it as bad on anything from this decade?
- femto 10y ago
  It is practical, but it's important to have the core of your code optimised. In particular, you will need a very tight "multiply-accumulate" loop, as typically used in an FIR filter. By way of real-world example, a 9600 bps P25 transceiver (with FEC) can be done on a C5500 DSP (200MHz) with about 10-20% CPU utilisation. As a further illustration, a vocoder that could not run in real-time was reduced to 5% CPU usage when a correlator (a single MAC loop in C) was replaced with the equivalent assembly that used a MAC instruction. 97% of the CPU time was being spent in that loop.
quotemstr 10y ago
tl;dr: the old magic ring buffer trick

142 points by ptr 10y ago

The double-mapped ring buffer trick is so clever I can't help but wish I invented it. :) Here's the code for the GNU radio implementation described in the article, it's a good reference for some various ways to set up the mappings: https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/vmcircbuf_createfilemapping.cc https://github.com/gnuradio/gnuradio/blob/master/gnuradio-ru... https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/vmcircbuf_mmap_shm_open.cc https://github.com/gnuradio/gnuradio/blob/master/gnuradio-ru... https://github.com/gnuradio/gnuradio/blob/master/gnuradio-runtime/lib/vmcircbuf_mmap_tmpfile.cc https://github.com/gnuradio/gnuradio/blob/master/gnuradio-ru... And here some other links to implementations I've collected: http://vrb.sourceforge.net/ http://vrb.sourceforge.net/ https://fgiesen.wordpress.com/2012/07/21/the-magic-ring-buffer/ https://fgiesen.wordpress.com/2012/07/21/the-magic-ring-buff... https://gist.github.com/rygorous/3158316 https://gist.github.com/rygorous/3158316 https://www.mikeash.com/pyblog/friday-qa-2012-02-03-ring-buffers-and-mirrored-memory-part-i.html https://www.mikeash.com/pyblog/friday-qa-2012-02-03-ring-buf... https://www.mikeash.com/pyblog/friday-qa-2012-02-17-ring-buffers-and-mirrored-memory-part-ii.html https://www.mikeash.com/pyblog/friday-qa-2012-02-17-ring-buf... https://github.com/mikeash/MAMirroredQueue https://github.com/mikeash/MAMirroredQueue http://atastypixel.com/blog/a-simple-fast-circular-buffer-implementation-for-audio-processing/ http://atastypixel.com/blog/a-simple-fast-circular-buffer-im... https://github.com/michaeltyson/TPCircularBuffer https://github.com/michaeltyson/TPCircularBuffer http://www.koanlogic.com/libu/api/html/group__rb.html http://www.koanlogic.com/libu/api/html/group__rb.html https://github.com/ulfalizer/botniklas/blob/master/src/read_msg.c https://github.com/ulfalizer/botniklas/blob/master/src/read_... https://github.com/willemt/cbuffer https://github.com/willemt/cbuffer

zump 10y ago

SDR strikes me as good in theory, but breaks down for anything practical. My core 2 duo was 100% cpu utilization simply from decoding 56kbps 2-BPSK, even with SIMD extension enabled (within GR).

viraptor 10y ago
Any other prefiltering / post-processing? This processing should be pretty trivial on its own. This should be fairly easy load - definitely not pegging the CPU at 100%.
lmh 10y ago
You may find this SDR LTE eNodeB interesting: http://bellard.org/lte/ http://bellard.org/lte/ There's also this open-source LTE library: https://github.com/srsLTE/srsLTE https://github.com/srsLTE/srsLTE
derefr 10y ago
SIMD is only going to get you so far; I'd expect much greater gains from running on a GPU.
ChuckMcM 10y ago
I run a HackRF One on an i7 (Intel NUC) and it is hard to peg the Cores. Typically when I do it is because some synthetic workload is dumping infinite work into the buffers and so everything is running full on, but generally real signals have been 20 - 30% of one or two cores.
jcims 10y ago
Something's terribly wrong with your flowgraph.
nikanj 10y ago
Core 2 duo was released in 2006, is it as bad on anything from this decade?
femto 10y ago
It is practical, but it's important to have the core of your code optimised. In particular, you will need a very tight "multiply-accumulate" loop, as typically used in an FIR filter. By way of real-world example, a 9600 bps P25 transceiver (with FEC) can be done on a C5500 DSP (200MHz) with about 10-20% CPU utilisation. As a further illustration, a vocoder that could not run in real-time was reduced to 5% CPU usage when a correlator (a single MAC loop in C) was replaced with the equivalent assembly that used a MAC instruction. 97% of the CPU time was being spent in that loop.

quotemstr 10y ago

tl;dr: the old magic ring buffer trick