3 ms·

I loved the idea of QNX. Got way excited about it. We were moving our optical food processor from dedicated DSPs to general purpose hardware, using 1394 (FireWi

I loved the idea of QNX. Got way excited about it. We were moving our optical food processor from dedicated DSPs to general purpose hardware, using 1394 (FireWire). The process isolation was awesome. The overhead of moving data through messages, not so much. In the end, we paid someone $2K to contribute isochronous mode/dma to the Linux 1394 driver and went our way with RT extensions. It was a powerful lesson (amongst others) in what I came to call “the Law of Conservation of Ugly”. In many software problems, there’s a part that just is never going to feel elegant. You can make one part of the system elegant, which often causes the inelegance surface elsewhere in the system.

jamestnz 1y ago
> what I came to call “the Law of Conservation of Ugly”. In many software problems, there’s a part that just is never going to feel elegant This may be an instance of the Waterbed Principle: in any sufficiently-complex system, suppressing or refactoring some undesirable characteristic in one area inevitably causes an undesirability to pop up somewhere else. Like there is some minimum amount of complexity/ugliness/etc that it is possible for the entire system to contain while still carrying out its essential functions, and it must leak out somewhere. https://en.wikipedia.org/wiki/Waterbed_theory https://en.wikipedia.org/wiki/Waterbed_theory
- vacuity 1y ago
  The terms I've seen used and prefer to use are "essential complexity" and "accidental complexity".
jacquesm 1y ago
I have a really neat idea to improve the message passing speed in QNX: you simply use the paging mechanism to send the message. That means there is no copying of the data at all, just a couple of page table updates. You still have the double TSS load overhead (vs 1 TSS load in a macro kernel), but that is pretty quick. But you are right that there is a price for elegance. It becomes an easier choice to make when you factor in things like latency and long term reliability / stability / correctness. Those can weigh much heavier than mere throughput.
- sahil-kang 1y ago
  I haven't seen it implemented anywhere, but that sounds like the "pagetable displacement" approach described here: https://wiki.osdev.org/IPC_Data_Copying_methods#Pagetable_displacement https://wiki.osdev.org/IPC_Data_Copying_methods#Pagetable_di... The same idea occurred to me a while ago too, which is how I originally found that link :)
  - smj-edison 1y ago
    How performant is that in practice? I thought setting pages was a fairly expensive process. Using a statically mapped circular buffer makes more sense to me at least. Disclaimer: I don't actually know what I'm talking about, lol
    - Veserv 1y ago
      You can pretty reliably do it on the order of 1 us on a modern desktop processor. If you use a level 2 sized mapping table entry of say 2 MB, that is a transfer speed on the order of 2 TB/s or ~32x faster than RAM for a single core even if you only move a single level 2 sized mapping table entry. If you transfer multiple in one go or use say a level 3 sized mapping table entry of 1 GB that would be 1 PB/s or ~16,000x faster than RAM or ~20x the full memory bandwidth of a entire H200 GPU.
    - jacquesm 1y ago
      Pretty quick, far faster than inter-process memory copy. The only way to be sure would be to set it up and to measure it, but on a 486/33 I could do this ~200K per second, on modern systems it should be a lot faster than that, more so if the processe(s) do not use FP. But I never actually tried setting up say a /dev/null implementation that used this, it would be an interesting experiment.
    - vacuity 1y ago
      To be clear, since the other replies to you don't seem to be mentioning it, the major costs of MMU page-based virtual memory are never about setting the page metadata. In any instance of remapping, TLB shootdowns and subsequent misses hurt. Page remapping is still very useful for large buffers, and other costs can be controlled based on intended usage, but smaller buffers should use other methods. (Of course I'm being vague about the cutoff for "large" and "smaller" buffers. Always benchmark!)
schoen 1y ago
Is "optical food processor" a metaphor, or is this actually a device that would cut up food items based on image feedback?
- travisgriggs 1y ago
  Usually it's about sorting. Take a lot of whatever (french fries, green beans, etc), accelerate them to something like 3 m/s, launch them off the end of a belt, scan them, looking for defects, and then use air jets to divert the defective items. Look on you tube for it. It's sort of mind boggling to see the scale at which french fries alone are produced. You see one line running at load, and then realize there are multiple lines in most plants, and there are hundreds of plants world wide. It's mind boggling. The cooler machines were specialized for fries, they use a rotating knife drum above a belt to cut defect spots from fries. I've not done that for 17 years now; the newer machines are that much cooler.
  - schoen 1y ago
    That's awesome. Thanks for the explanation. I did find several machines like this on YouTube, and it's amazing to watch. (One of them had little motor-actuated slats that could kick the defective items away, almost like a foot kicking a soccer ball!)
JohnAtQNX 1y ago
Today though, I'd argue that with full DSMP support and much more capable systems, any overhead from message passing is much less of a concern, or at least outweighed by other benefits.
kristjansson 1y ago
There's an older talk Simon Peyton Jones (IIRC?) gave about some development or other in haskell, in which he suggested that many software systems have some aspect of the swamp or the marsh into which you must eventually wade - that there's a mucky, sticky, irreducible aspect to the problem that must be dealt with somewhere, regardless of how elegant the rest of the system is. "that marsh thing" has stuck with me, and been a frequent contributor to my work and thinking. I'll happily take Law of Conservation of Ugly as a _much_ better name for the thought :)

jamestnz 1y ago

> what I came to call “the Law of Conservation of Ugly”. In many software problems, there’s a part that just is never going to feel elegant This may be an instance of the Waterbed Principle: in any sufficiently-complex system, suppressing or refactoring some undesirable characteristic in one area inevitably causes an undesirability to pop up somewhere else. Like there is some minimum amount of complexity/ugliness/etc that it is possible for the entire system to contain while still carrying out its essential functions, and it must leak out somewhere. https://en.wikipedia.org/wiki/Waterbed_theory https://en.wikipedia.org/wiki/Waterbed_theory

vacuity 1y ago
The terms I've seen used and prefer to use are "essential complexity" and "accidental complexity".

jacquesm 1y ago

I have a really neat idea to improve the message passing speed in QNX: you simply use the paging mechanism to send the message. That means there is no copying of the data at all, just a couple of page table updates. You still have the double TSS load overhead (vs 1 TSS load in a macro kernel), but that is pretty quick. But you are right that there is a price for elegance. It becomes an easier choice to make when you factor in things like latency and long term reliability / stability / correctness. Those can weigh much heavier than mere throughput.

sahil-kang 1y ago
I haven't seen it implemented anywhere, but that sounds like the "pagetable displacement" approach described here: https://wiki.osdev.org/IPC_Data_Copying_methods#Pagetable_displacement https://wiki.osdev.org/IPC_Data_Copying_methods#Pagetable_di... The same idea occurred to me a while ago too, which is how I originally found that link :)
- smj-edison 1y ago
  How performant is that in practice? I thought setting pages was a fairly expensive process. Using a statically mapped circular buffer makes more sense to me at least. Disclaimer: I don't actually know what I'm talking about, lol
  - Veserv 1y ago
    You can pretty reliably do it on the order of 1 us on a modern desktop processor. If you use a level 2 sized mapping table entry of say 2 MB, that is a transfer speed on the order of 2 TB/s or ~32x faster than RAM for a single core even if you only move a single level 2 sized mapping table entry. If you transfer multiple in one go or use say a level 3 sized mapping table entry of 1 GB that would be 1 PB/s or ~16,000x faster than RAM or ~20x the full memory bandwidth of a entire H200 GPU.
  - jacquesm 1y ago
    Pretty quick, far faster than inter-process memory copy. The only way to be sure would be to set it up and to measure it, but on a 486/33 I could do this ~200K per second, on modern systems it should be a lot faster than that, more so if the processe(s) do not use FP. But I never actually tried setting up say a /dev/null implementation that used this, it would be an interesting experiment.
  - vacuity 1y ago
    To be clear, since the other replies to you don't seem to be mentioning it, the major costs of MMU page-based virtual memory are never about setting the page metadata. In any instance of remapping, TLB shootdowns and subsequent misses hurt. Page remapping is still very useful for large buffers, and other costs can be controlled based on intended usage, but smaller buffers should use other methods. (Of course I'm being vague about the cutoff for "large" and "smaller" buffers. Always benchmark!)

schoen 1y ago

Is "optical food processor" a metaphor, or is this actually a device that would cut up food items based on image feedback?

travisgriggs 1y ago
Usually it's about sorting. Take a lot of whatever (french fries, green beans, etc), accelerate them to something like 3 m/s, launch them off the end of a belt, scan them, looking for defects, and then use air jets to divert the defective items. Look on you tube for it. It's sort of mind boggling to see the scale at which french fries alone are produced. You see one line running at load, and then realize there are multiple lines in most plants, and there are hundreds of plants world wide. It's mind boggling. The cooler machines were specialized for fries, they use a rotating knife drum above a belt to cut defect spots from fries. I've not done that for 17 years now; the newer machines are that much cooler.
- schoen 1y ago
  That's awesome. Thanks for the explanation. I did find several machines like this on YouTube, and it's amazing to watch. (One of them had little motor-actuated slats that could kick the defective items away, almost like a foot kicking a soccer ball!)

JohnAtQNX 1y ago

Today though, I'd argue that with full DSMP support and much more capable systems, any overhead from message passing is much less of a concern, or at least outweighed by other benefits.

kristjansson 1y ago

There's an older talk Simon Peyton Jones (IIRC?) gave about some development or other in haskell, in which he suggested that many software systems have some aspect of the swamp or the marsh into which you must eventually wade - that there's a mucky, sticky, irreducible aspect to the problem that must be dealt with somewhere, regardless of how elegant the rest of the system is. "that marsh thing" has stuck with me, and been a frequent contributor to my work and thinking. I'll happily take Law of Conservation of Ugly as a _much_ better name for the thought :)