28 ms·

I have an embedded real-time control project that is currently written in Rust, but runs with RTIC (https://rtic.rs/ https://rtic.rs/), a framework which is con

I have an embedded real-time control project that is currently written in Rust, but runs with RTIC (https://rtic.rs/ https://rtic.rs/), a framework which is conceptually similar (no dynamic allocation of tasks or resources) but also has some differences. RTIC is more of a framework for locks and critical sections in an interrupt based program than a full fledged RTOS. Looking through the docs, here's the main differences (for my purposes) I see: 1. In Hubris, all interrupt handlers dispatch to a software task. In RTIC, you can dispatch to a software task, but you can also run the code directly in the interrupt handler. RTIC is reliant on Cortex-M's NVIC for preemption, whereas Hubris can preempt in software (assuming it is implemented). This does increase the minimum effective interrupt latency in Hubris, and if not very carefully implemented, the jitter also. 2. Hubris compiles each task separately and then pastes the binaries together, presumably with a fancy linker script. RTIC can have everything in one source file and builds everything into one LTO'd blob. I see the Hubris method as mostly a downside (unless you want to integrate binary blobs, for example), but it might have been needed for: 3. Hubris supports Cortex-M memory protection regions. This is pretty neat and something that is mostly out of scope for RTIC (being built around primitives that allow shared memory, trying to map into the very limited number of MPU regions would be difficult at best). Of course, it's Rust, so in theory you wouldn't need the MPU protections, but if you have to run any sort of untrusted code this is definitely the winner. Hubris does support shared memory via leases, but I'm not sure how it manages to map them into the very limited 8 Cortex-M MPU regions. I'm quite interested to look at the implementation when the source code is released. Edit: I forgot to mention the biggest difference, which is that because tasks have separate stacks in Hubris, you can do blocking waits. RTIC may support async in the future but for now you must manually construct state machines.

JulianMorrison 5y ago
I don't think they link the binaries. It's more like, put them each on executable flash in separate places and the kernel just calls them. The intent here seems to be that each binary has no need (and no ability) to get all up in another binary's business. Nothing shared, except access to the RTOS.
- TD-Linux 5y ago
  It doesn't need to link any symbols, but I believe it does need to do relocations if the code isn't PIC, and to relocate the task's statically allocated RAM.
panick21_ 5y ago
Maybe they want it to be possible to have closed source and open source task to be mixed.
- steveklabnik 5y ago
  In general we plan on making the system as open sourced as we possibly can, so there's no specific thought currently being put into closed source tasks. While it could work, the build system doesn't actually support that at all right now.
  - panick21_ 5y ago
    I assumed you guys wouldn't do this, but thought this could lead to a larger adoption in this space. Alternatively I thought maybe you needed to have some closed source component from a vendor and could only include it like that.
    - TD-Linux 5y ago
      I definitely didn't mean it as a feature request - blobs aren't actually that common in embedded (esp32 and some motor driver libraries are the most common exceptions), so I don't think it's important for adoption. In fact, not supporting it enables future ergonomics improvements and code sharing between tasks, so I appreciate that it's not a driving factor in the design.
      - floatboth 5y ago
        > esp32 and […] are the most common exceptions Well, nearly anything having to do with wireless is typically blobs :/ Even Nordic has blobby SoftDevices, though you don't have to use them since Apache NimBLE exists (and rubble in Rust though that's only usable for advertising-only for now).
    - steveklabnik 5y ago
      Yeah, I hear you on the adoption thing for sure, though to be honest, right now we are laser focused on shipping Oxide's product, and so if nobody else but us uses Hubris, that is 100% okay. As our CONTRIBUTING.md mentions, we aren't yet at the stage where we're trying to grow this as an independent thing. It's true that vendors are likely to be an area where we have to deal with some things being closed source, though we're not simply accepting that as a given. This is one reason we're writing so much of our own software, and also, poking at the bits that must remain closed: https://oxide.computer/blog/lpc55 https://oxide.computer/blog/lpc55
wyldfire 5y ago
Your link does not appear to work, maybe this one [1] is intended instead? I can't resolve that name, at least. [1] https://rtic.rs/ https://rtic.rs/
monocasa 5y ago
> Hubris does support shared memory via leases, but I'm not sure how it manages to map them into the very limited 8 Cortex-M MPU regions. What I did in a similar kernel was dynamically map them from a larger table on faults, sort of like you would with a soft fill TLB. When you turn off the MPU in supervisor mode you get a sane 'map everything' mapping, leaving all 8 entries to user code. The way LDM/STM restart after faults is amenable to this model on the M series cores.
- TD-Linux 5y ago
  Neat, I didn't know that the MPU fault handler was complete enough to allow for restarts. Now that the source is available, I took a look at what hubris does - it is not actually anything fancy, just a static list of up to 8 MPU regions per task [1]. It seems that leases aren't actually shared memory, but rather just grant permission for a memcpy-like syscall [2]. This is slightly better than plain message passing as the recipient gets to decide what memory it wants to access, but is still a memcpy. [1] https://github.com/oxidecomputer/hubris/blob/8833cc1dcfdbf107369e188a035a38651ba7ab46/build/xtask/src/dist.rs#L1293 https://github.com/oxidecomputer/hubris/blob/8833cc1dcfdbf10... [2] https://hubris.oxide.computer/reference/#_borrow_read_4 https://hubris.oxide.computer/reference/#_borrow_read_4
  - elcritch 5y ago
    That's interesting, and pretty neat what you can do with Cortex M mpus. Leases seem an interesting twist to regular message passing.

JulianMorrison 5y ago

I don't think they link the binaries. It's more like, put them each on executable flash in separate places and the kernel just calls them. The intent here seems to be that each binary has no need (and no ability) to get all up in another binary's business. Nothing shared, except access to the RTOS.

TD-Linux 5y ago
It doesn't need to link any symbols, but I believe it does need to do relocations if the code isn't PIC, and to relocate the task's statically allocated RAM.

panick21_ 5y ago

Maybe they want it to be possible to have closed source and open source task to be mixed.

steveklabnik 5y ago
In general we plan on making the system as open sourced as we possibly can, so there's no specific thought currently being put into closed source tasks. While it could work, the build system doesn't actually support that at all right now.
- panick21_ 5y ago
  I assumed you guys wouldn't do this, but thought this could lead to a larger adoption in this space. Alternatively I thought maybe you needed to have some closed source component from a vendor and could only include it like that.
  - TD-Linux 5y ago
    I definitely didn't mean it as a feature request - blobs aren't actually that common in embedded (esp32 and some motor driver libraries are the most common exceptions), so I don't think it's important for adoption. In fact, not supporting it enables future ergonomics improvements and code sharing between tasks, so I appreciate that it's not a driving factor in the design.
    - floatboth 5y ago
      > esp32 and […] are the most common exceptions Well, nearly anything having to do with wireless is typically blobs :/ Even Nordic has blobby SoftDevices, though you don't have to use them since Apache NimBLE exists (and rubble in Rust though that's only usable for advertising-only for now).
  - steveklabnik 5y ago
    Yeah, I hear you on the adoption thing for sure, though to be honest, right now we are laser focused on shipping Oxide's product, and so if nobody else but us uses Hubris, that is 100% okay. As our CONTRIBUTING.md mentions, we aren't yet at the stage where we're trying to grow this as an independent thing. It's true that vendors are likely to be an area where we have to deal with some things being closed source, though we're not simply accepting that as a given. This is one reason we're writing so much of our own software, and also, poking at the bits that must remain closed: https://oxide.computer/blog/lpc55 https://oxide.computer/blog/lpc55

wyldfire 5y ago

Your link does not appear to work, maybe this one [1] is intended instead? I can't resolve that name, at least. [1] https://rtic.rs/ https://rtic.rs/

monocasa 5y ago

> Hubris does support shared memory via leases, but I'm not sure how it manages to map them into the very limited 8 Cortex-M MPU regions. What I did in a similar kernel was dynamically map them from a larger table on faults, sort of like you would with a soft fill TLB. When you turn off the MPU in supervisor mode you get a sane 'map everything' mapping, leaving all 8 entries to user code. The way LDM/STM restart after faults is amenable to this model on the M series cores.

TD-Linux 5y ago
Neat, I didn't know that the MPU fault handler was complete enough to allow for restarts. Now that the source is available, I took a look at what hubris does - it is not actually anything fancy, just a static list of up to 8 MPU regions per task [1]. It seems that leases aren't actually shared memory, but rather just grant permission for a memcpy-like syscall [2]. This is slightly better than plain message passing as the recipient gets to decide what memory it wants to access, but is still a memcpy. [1] https://github.com/oxidecomputer/hubris/blob/8833cc1dcfdbf107369e188a035a38651ba7ab46/build/xtask/src/dist.rs#L1293 https://github.com/oxidecomputer/hubris/blob/8833cc1dcfdbf10... [2] https://hubris.oxide.computer/reference/#_borrow_read_4 https://hubris.oxide.computer/reference/#_borrow_read_4
- elcritch 5y ago
  That's interesting, and pretty neat what you can do with Cortex M mpus. Leases seem an interesting twist to regular message passing.