4 ms·

I've been doing embedded development for over 20 years. The rust evangelists just don't understand embedded development. Embedded developers are very conservat

I've been doing embedded development for over 20 years. The rust evangelists just don't understand embedded development. Embedded developers are very conservative, they don't chase the shiny and believe in using old tried and true technology. C++ offers significant benefits over C in the embedded space and its been impossible to gain traction. With C you have a HAL provided by a vendor that allows very rapid bring up of the hardware which is 90% of the battle in embedded. Switching to Rust you would have to write your own. With C you have billions of lines C libraries at your disposal to bring your product to market. To use Rust you would have to write your own or deal with the shims and lose the Rust benefits. I do 4-5 embedded projects a year, using Rust and not having the HAL and C library ecosystem would put those projects ad a competitive disadvantage. I find the whole hubris and humilty thing is ridiculous. Writing a RTOS in Rust which isn't your core competency is such a waste of resources. Use FreeRTOS and ship the product.

jack_h 5y ago
I've been in the embedded space for over 10 years so I figure I'll give my thoughts on the matter. Apologies up front as this will be a little ranty. > Embedded developers are very conservative, they don't chase the shiny and believe in using old tried and true technology. This is underselling the point. They are conservative in the same sense as an electrical engineer who refuses to use ECAD software or transistors because drafting tables and vacuum tubes are tried and true. There's a difference between "[not] chas[ing] the shiny" and just being obstinate. > With C you have a HAL provided by a vendor that allows very rapid bring up of the hardware which is 90% of the battle in embedded. Switching to Rust you would have to write your own. I'm not sure why vendor supplied HALs - which can be pretty hit or miss in terms of quality - are so important for board bring-up. In the end it's a minor milestone in the development lifecycle for all but the simplest of projects. In the projects I work on practically all of the time is spent on the complexity of the firmware and how it interacts with the various networks of sensors, actuators, cellular, BLE, storage, etc. Besides, last I checked there were many people putting effort into HALs written in Rust. > With C you have billions of lines C libraries at your disposal to bring your product to market. To use Rust you would have to write your own or deal with the shims and lose the Rust benefits. First off it has not been my experience that libraries are used often. As you mention later FreeRTOS is generally the only external code I see pulled into a project, everything else is written from scratch or copy pasted from a previous project which was written from scratch or copy pasted from a previous previous project etc. Let's assume that libraries are commonly used though, how exactly do you use them? This is one point of C/C++ that is still absolutely dire. Rust has Cargo whereas C/C++ has a lot of wacky ways to deal with it such as through git submodules, or cmake, or possibly conan. Generally speaking what I see is the code being copy pasted into the project and never updated despite the growing security concerns for connected embedded devices. Of course Rust offers more than mere package management, Cargo is also the build system, it also has support for automated testing and documentation. All of these things are typically afterthoughts at best in many embedded projects. > I do 4-5 embedded projects a year, using Rust and not having the HAL and C library ecosystem would put those projects ad a competitive disadvantage. To be clear at this point in time if I were asked if an embedded project should be written in Rust my answer would be "probably not", although given 5-10 years that answer very well may change to the affirmative. The ecosystem is young and growing; as long as the growth continues it will likely be a viable alternative in the future.
- couchand 5y ago
  > Of course Rust offers more than mere package management, Cargo is also the build system, it also has support for automated testing and documentation. All of these things are typically afterthoughts at best in many embedded projects. Completely agree. I'm starting new embedded projects in Rust mostly because cargo test is a force multiplier. (well and also the expressive type system)
bcantrill 5y ago
> I find the whole hubris and humilty (sic) thing is (sic) ridiculous. Can you elaborate?
- not_the_fda 5y ago
  Oxide Computer is a startup without a product on the market yet. Embedded operating systems are a solved problem, there are many available on the market used in mission critical systems. At most companies that are trying to ship a product and have a mountain of work to be done, and someone said, "I know, lets build our own RTOS", even though there are already many perfectly fine ones out there, they would be laughed out of the room.
  - bcantrill 5y ago
    I encourage you to learn more about Hubris and about how we arrived at it -- certainly, it was nothing as you convey (and to the contrary, we tried hard to get extant systems to work). Read Cliff's blog[0] for details, but we are not lightweights and nor are we rookies, and the ideas represented in Hubris come from decades of our collective experience at the hardware/software interface. I also emphatically disagree that "embedded operating systems are a solved problem" (or indeed, that operating systems are a solved problem) -- and that in general such proclamations reveal much more about the technologist than the technology that they proclaim to be finished. [0] https://cliffle.com/blog/on-hubris-and-humility/ https://cliffle.com/blog/on-hubris-and-humility/
ahepp 5y ago
Do you think there's any potential for rust to gain a competitive advantage in safety critical systems? Or are the benefits too difficult to quantify? It seems like if something along the lines of "MISRA C" (but requiring Rust) were to become an industry best practice, we might see the Rust ecosystem develop. However, automakers are some of the most conservative companies out there when it comes to tech, and I doubt they really care about safety or formal correctness as much as being able to slap a "MISRA" sticker on the code for liability reasons. It would have to be, decades I imagine, before there's any chance of rust becoming truly widespread in the embedded world. Universities would need to start teaching with it. People who know C and nothing else would need to retire. I agree that it's unlikely to be commercially viable at a large scale any time soon (unless you can find a handful of true believers for your team).
- not_the_fda 5y ago
  I see benefits of it, but I think its a hard sell. In aviation things need certified, you can buy off the shelf compilers, RTOSs, and libraries certified for aviation. Rust doesn't have any of that. Medical devices, maybe, but you still have the problem of finding talent. If you were to build a product on Rust, you would have trouble hiring people to maintain it over the lifecycle of the product. Its hard enough hiring C and C++ developers.
  - couchand 5y ago
    Weirdly, it's already significantly easier to find Rust (or Rust-interested) developers these days than C/C++. If your main concern is being able to hire devs then Rust is a clear win. And don't sleep on Sealed Rust. They've surely got a hard road ahead, but there are some dedicated folks working on it.
    - dataangel 5y ago
      Yeah I call bullshit. By sheer numbers C/C++ programmers are much easier to find. Rust may improve your company's appeal to an interested programmer, but by the numbers C/C++ have incredibly greater supply.
abalaji 5y ago
For those who weren't familiar with some of the jargon * HAL: (Hardware Application Layer) [1] * RTOS: (Real time Operating System) [2,3] [1] https://www.beningo.com/embedded-basics-apis-vs-hals/ https://www.beningo.com/embedded-basics-apis-vs-hals/ [2] https://en.wikipedia.org/wiki/Real-time_operating_system https://en.wikipedia.org/wiki/Real-time_operating_system [3] https://en.wikipedia.org/wiki/FreeRTOS https://en.wikipedia.org/wiki/FreeRTOS
smilekzs 5y ago
I concur, and add my 2 cents on why "Use FreeRTOS and ship the product" actually works in the real world (i.e. paid job), from the perspective of a part-time firmware engineer. Obviously none of this applies if you are just having fun building your side project. 1. Unless your product is mass-produced, as long as your $work has >= 2 projects, having them share a over-spec'd MCU family (i.e. way more pins, MHz, SRAM, ... than you could imagine yourself needing), or even a single part, is very desirable. The BOM is much simpler, and given the current chip shortage situation, you don't have to track incompatible chip families. All the infra work, internal codebases, know-how, even quirks and their workarounds can be shared in-house. 2. The vendor most likely has some BSP (board support package) written in C with examples in C as well, and it most likely will support / assume / integrate with FreeRTOS. Neither of these will be optimized to the bones --- usually the situation is quite on the contrary, with O(n) loops liberally littered around. However, since you picked a overkill MCU, it will more likely than not gladly chug along, given you don't abuse them. When you FFI / shim / start from scratch with registers, usually the happy path does work after some hacking, but when it doesn't work, do you have confidence it's a HW issue, or is it rather your added indirection? With Embedded Rust's prevalent macro-heavy approach, this can be a real time sink. 3. Firmware engineering really works closer to hardware than you'd think. To avoid a catch-22 situation in the co-design, already knowing a lot about what embedded platform breaks the loop and effectively decouples HW and SW, reducing your time-to-market.
ostenning 5y ago
From your statements it seems to me you haven't really given Rust an honest evaluation. I've been using the Rust stm32 HALs offered by the open-source community and they are stellar. People are doing some really great work so people like myself can come along and knock together a highly performative bare metal application with relative ease. Obviously not all features are supported and some chips are more supported than others but its a work in progress. Regarding library support, Rust has crates.io and a first class package manager with native support for heapless environments which means that I can easily add a package designed specifically for embedded development to my project. Its pretty awesome not having to mess around with a million linker flags, copy-pasting directories etc to try to get a library to work. Ontop of that with Rusts language features there are standardized traits for peripherals like I2C, SPI etc, meaning that any library can easily consume any HAL's peripheral implementation. Basically I can install an LCD library and have it running with 1 line of code. Obviously there will be more C libraries out there and it will take time to port them over to Rust but the foundation is very very solid. That's a rebuttal for your HAL and library criticisms and I still think there are vast safety and ergonomic improvements to using Rust over C which would undoubtedly lead to more reliable code and quicker development times.
raxxorrax 5y ago
> C++ offers significant benefits over C in the embedded space and its been impossible to gain traction. Wasn't it Arduino that tried to establish C++? I could never see the advantages to be honest, but I didn't really use too much C++. Header files mainly provide semantic descriptors to otherwise pretty hard to memorize memory addresses. A format like SVD would provide the main benefit for other languages though. Even non-ARM-processor manufacturers use it. I believe many embedded devs long for "high tech features" like unit tests, which of course would require some form of formal simulators and peripherals. That problem goes far beyond the choice of language though. Validation is an extremely hard thing to do for embedded system outside of a specialized laboratory. Memory safety is almost laughingly trivial compared to that.
floatboth 5y ago
Vendor-provided SDKs are typically not good. Nordic's is one of the more decent ones and it still has a messy makefile based build system that's horrible to use. With nrf-rs you just use it as a cargo dependency. You don't have to write your own HAL when the community maintains really excellent ones, written by people who actually want to fucking use it rather than vendor employees who just have to get it out of the door.

jack_h 5y ago

I've been in the embedded space for over 10 years so I figure I'll give my thoughts on the matter. Apologies up front as this will be a little ranty. > Embedded developers are very conservative, they don't chase the shiny and believe in using old tried and true technology. This is underselling the point. They are conservative in the same sense as an electrical engineer who refuses to use ECAD software or transistors because drafting tables and vacuum tubes are tried and true. There's a difference between "[not] chas[ing] the shiny" and just being obstinate. > With C you have a HAL provided by a vendor that allows very rapid bring up of the hardware which is 90% of the battle in embedded. Switching to Rust you would have to write your own. I'm not sure why vendor supplied HALs - which can be pretty hit or miss in terms of quality - are so important for board bring-up. In the end it's a minor milestone in the development lifecycle for all but the simplest of projects. In the projects I work on practically all of the time is spent on the complexity of the firmware and how it interacts with the various networks of sensors, actuators, cellular, BLE, storage, etc. Besides, last I checked there were many people putting effort into HALs written in Rust. > With C you have billions of lines C libraries at your disposal to bring your product to market. To use Rust you would have to write your own or deal with the shims and lose the Rust benefits. First off it has not been my experience that libraries are used often. As you mention later FreeRTOS is generally the only external code I see pulled into a project, everything else is written from scratch or copy pasted from a previous project which was written from scratch or copy pasted from a previous previous project etc. Let's assume that libraries are commonly used though, how exactly do you use them? This is one point of C/C++ that is still absolutely dire. Rust has Cargo whereas C/C++ has a lot of wacky ways to deal with it such as through git submodules, or cmake, or possibly conan. Generally speaking what I see is the code being copy pasted into the project and never updated despite the growing security concerns for connected embedded devices. Of course Rust offers more than mere package management, Cargo is also the build system, it also has support for automated testing and documentation. All of these things are typically afterthoughts at best in many embedded projects. > I do 4-5 embedded projects a year, using Rust and not having the HAL and C library ecosystem would put those projects ad a competitive disadvantage. To be clear at this point in time if I were asked if an embedded project should be written in Rust my answer would be "probably not", although given 5-10 years that answer very well may change to the affirmative. The ecosystem is young and growing; as long as the growth continues it will likely be a viable alternative in the future.

couchand 5y ago
> Of course Rust offers more than mere package management, Cargo is also the build system, it also has support for automated testing and documentation. All of these things are typically afterthoughts at best in many embedded projects. Completely agree. I'm starting new embedded projects in Rust mostly because cargo test is a force multiplier. (well and also the expressive type system)

bcantrill 5y ago

> I find the whole hubris and humilty (sic) thing is (sic) ridiculous. Can you elaborate?

not_the_fda 5y ago
Oxide Computer is a startup without a product on the market yet. Embedded operating systems are a solved problem, there are many available on the market used in mission critical systems. At most companies that are trying to ship a product and have a mountain of work to be done, and someone said, "I know, lets build our own RTOS", even though there are already many perfectly fine ones out there, they would be laughed out of the room.
- bcantrill 5y ago
  I encourage you to learn more about Hubris and about how we arrived at it -- certainly, it was nothing as you convey (and to the contrary, we tried hard to get extant systems to work). Read Cliff's blog[0] for details, but we are not lightweights and nor are we rookies, and the ideas represented in Hubris come from decades of our collective experience at the hardware/software interface. I also emphatically disagree that "embedded operating systems are a solved problem" (or indeed, that operating systems are a solved problem) -- and that in general such proclamations reveal much more about the technologist than the technology that they proclaim to be finished. [0] https://cliffle.com/blog/on-hubris-and-humility/ https://cliffle.com/blog/on-hubris-and-humility/

ahepp 5y ago

Do you think there's any potential for rust to gain a competitive advantage in safety critical systems? Or are the benefits too difficult to quantify? It seems like if something along the lines of "MISRA C" (but requiring Rust) were to become an industry best practice, we might see the Rust ecosystem develop. However, automakers are some of the most conservative companies out there when it comes to tech, and I doubt they really care about safety or formal correctness as much as being able to slap a "MISRA" sticker on the code for liability reasons. It would have to be, decades I imagine, before there's any chance of rust becoming truly widespread in the embedded world. Universities would need to start teaching with it. People who know C and nothing else would need to retire. I agree that it's unlikely to be commercially viable at a large scale any time soon (unless you can find a handful of true believers for your team).

not_the_fda 5y ago
I see benefits of it, but I think its a hard sell. In aviation things need certified, you can buy off the shelf compilers, RTOSs, and libraries certified for aviation. Rust doesn't have any of that. Medical devices, maybe, but you still have the problem of finding talent. If you were to build a product on Rust, you would have trouble hiring people to maintain it over the lifecycle of the product. Its hard enough hiring C and C++ developers.
- couchand 5y ago
  Weirdly, it's already significantly easier to find Rust (or Rust-interested) developers these days than C/C++. If your main concern is being able to hire devs then Rust is a clear win. And don't sleep on Sealed Rust. They've surely got a hard road ahead, but there are some dedicated folks working on it.
  - dataangel 5y ago
    Yeah I call bullshit. By sheer numbers C/C++ programmers are much easier to find. Rust may improve your company's appeal to an interested programmer, but by the numbers C/C++ have incredibly greater supply.

abalaji 5y ago

For those who weren't familiar with some of the jargon * HAL: (Hardware Application Layer) [1] * RTOS: (Real time Operating System) [2,3] [1] https://www.beningo.com/embedded-basics-apis-vs-hals/ https://www.beningo.com/embedded-basics-apis-vs-hals/ [2] https://en.wikipedia.org/wiki/Real-time_operating_system https://en.wikipedia.org/wiki/Real-time_operating_system [3] https://en.wikipedia.org/wiki/FreeRTOS https://en.wikipedia.org/wiki/FreeRTOS

smilekzs 5y ago

I concur, and add my 2 cents on why "Use FreeRTOS and ship the product" actually works in the real world (i.e. paid job), from the perspective of a part-time firmware engineer. Obviously none of this applies if you are just having fun building your side project. 1. Unless your product is mass-produced, as long as your $work has >= 2 projects, having them share a over-spec'd MCU family (i.e. way more pins, MHz, SRAM, ... than you could imagine yourself needing), or even a single part, is very desirable. The BOM is much simpler, and given the current chip shortage situation, you don't have to track incompatible chip families. All the infra work, internal codebases, know-how, even quirks and their workarounds can be shared in-house. 2. The vendor most likely has some BSP (board support package) written in C with examples in C as well, and it most likely will support / assume / integrate with FreeRTOS. Neither of these will be optimized to the bones --- usually the situation is quite on the contrary, with O(n) loops liberally littered around. However, since you picked a overkill MCU, it will more likely than not gladly chug along, given you don't abuse them. When you FFI / shim / start from scratch with registers, usually the happy path does work after some hacking, but when it doesn't work, do you have confidence it's a HW issue, or is it rather your added indirection? With Embedded Rust's prevalent macro-heavy approach, this can be a real time sink. 3. Firmware engineering really works closer to hardware than you'd think. To avoid a catch-22 situation in the co-design, already knowing a lot about what embedded platform breaks the loop and effectively decouples HW and SW, reducing your time-to-market.

ostenning 5y ago

From your statements it seems to me you haven't really given Rust an honest evaluation. I've been using the Rust stm32 HALs offered by the open-source community and they are stellar. People are doing some really great work so people like myself can come along and knock together a highly performative bare metal application with relative ease. Obviously not all features are supported and some chips are more supported than others but its a work in progress. Regarding library support, Rust has crates.io and a first class package manager with native support for heapless environments which means that I can easily add a package designed specifically for embedded development to my project. Its pretty awesome not having to mess around with a million linker flags, copy-pasting directories etc to try to get a library to work. Ontop of that with Rusts language features there are standardized traits for peripherals like I2C, SPI etc, meaning that any library can easily consume any HAL's peripheral implementation. Basically I can install an LCD library and have it running with 1 line of code. Obviously there will be more C libraries out there and it will take time to port them over to Rust but the foundation is very very solid. That's a rebuttal for your HAL and library criticisms and I still think there are vast safety and ergonomic improvements to using Rust over C which would undoubtedly lead to more reliable code and quicker development times.

raxxorrax 5y ago

> C++ offers significant benefits over C in the embedded space and its been impossible to gain traction. Wasn't it Arduino that tried to establish C++? I could never see the advantages to be honest, but I didn't really use too much C++. Header files mainly provide semantic descriptors to otherwise pretty hard to memorize memory addresses. A format like SVD would provide the main benefit for other languages though. Even non-ARM-processor manufacturers use it. I believe many embedded devs long for "high tech features" like unit tests, which of course would require some form of formal simulators and peripherals. That problem goes far beyond the choice of language though. Validation is an extremely hard thing to do for embedded system outside of a specialized laboratory. Memory safety is almost laughingly trivial compared to that.

floatboth 5y ago

Vendor-provided SDKs are typically not good. Nordic's is one of the more decent ones and it still has a messy makefile based build system that's horrible to use. With nrf-rs you just use it as a cargo dependency. You don't have to write your own HAL when the community maintains really excellent ones, written by people who actually want to fucking use it rather than vendor employees who just have to get it out of the door.