7 ms·

Picolibc – Libc for Embedded Systems

118 points by homarp 7y ago

Ecco 7y ago
The biggest news is probably the fact that it's BSD licensed. Having some GPL parts had driven me away from newlib before.
- delinka 7y ago
  If I were in the market for such a library, I'd still have serious concerns. The GPL license still appears in the root of the repo as if the entire codebase might be subject to GPL; It appears to be a fork of newlib (GPL) and that means it might not qualify as a "clean room implementation" of GPL'd code - the legal team in a corporation is going to insist on a ton of due diligence on Picolibc before they allow it to be used.
  - AWildC182 7y ago
    Or we could stop mindlessly releasing closed source encrypted firmware?
  - derefr 7y ago
    Quoting the README: > Remove[d] unused code with non-BSD licenses. There's still a pile of unused code hanging around, but all non-BSD licensed bits have been removed to make the licensing situation clear. Picolibc is BSD licensed. From my understanding, Newlib is a GPL-licensed project that has some GPL-licensed contributions and some BSD-licensed contributions (in the sense that these aren't original code written by contributors, but rather are copy-and-paste inclusions of pieces from various GPL- and BSD-licensed codebases.) The author created Picolibc by (doing the moral equivalent of) cherry-picking out a small base of the commits to Newlib—all of which were copy-and-pastes from BSD-licensed codebases. --- In some sense, Newlib is very similar to how a Linux distro works, repackaging upstream code components (of various licenses) together. Picolibc, then, would be a lot like creating a new Linux distro, derived "from" Ubuntu, that happens to use only the packages in Ubuntu that are themselves directly copied in from upstream Debian (pretending for a moment that Ubuntu does this rather than re-signing packages with their own keys.) Is such a distro really "derived from Ubuntu"? Or is it derived "from" Debian, with an Ubuntu mirror server just serving as a pipe that some Debian packages went through on their trip "from" Debian's hands "into" the hands of the new distro? --- Either way, worries that the GPL could infect this effort are misplaced—even if it comes to a legal battle, at any point, the same codebase could be recreated (with a bit of a schlep) by just going to all the same BSD-licensed upstream sources that Newlib's contributors pulled from, and doing the same copy-and-pasting that Newlib's contributors did.
- enriquto 7y ago
  you license fanatics are so annoying...
  - bpt3 7y ago
    There's nothing annoying about understanding how you're allowed to use something created by another person, especially when legal liability is involved.
  - dang 7y ago
    Please don't post unsubstantive comments here, especially not nasty ones, regardless of how annoying other commenters are. Otherwise we all go down an annoyance spiral. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
deleted 7y ago
[deleted]
nerpderp82 7y ago
Needs to have a strong overlap with WASI.
- saagarjha 7y ago
  Why?
stock_toaster 7y ago
I didn't see this addressed in the post, but why not musl?
- zamadatix 7y ago
  The original post linked in the opening covers some of the search and reasoning.
  - justincormack 7y ago
    it says "A more lively project; still, definitely targets systems with a real Linux kernel." but actually people are using Musl for other use cases. And it is BSD licensed, and very high quality.
    - stefan_ 7y ago
      Okay, what good is any of that if musl literally does not compile without Linux headers? This is for systems that are at best a small RTOS, and there is no standard for what those operating systems offer to a libc.
      - justincormack 7y ago
        Musl upstream only supports Linux, but it is so easy to port that there are large numbers of ports to non Linux environments.
    - digikata 7y ago
      There were other considerations against musl there. I'd summarize them as targeting a desktop/server with kernel level system, or at least a mid-to-high end embedded system, has different code, memory, and optimization tradeoffs than the smaller embedded systems that seem to be targeted with picolibc.
- fra 7y ago
  For what it's worth: musl is much larger than newlib-nano or picolibc. Most of the embedded projects I've worked on could not afford to use musl.
AceJohnny2 7y ago
I wish we could do away with (parts of) libc entirely. Many of its design principles and limitations date back to its inception and don't make sense today, even on "embedded" systems. - function names limited to 6 characters (because of compiler/linker memory limits at the time?) - non-reentrency of many functions (strtok(), errno...) - dangerous string manipulation functions (see the various iterations of strcpy()) - and so much more... Of course, I'm missing the point. Libc continues on because it's standard and backwards-compatible, and lets you leverage 50 years of software ecosystem. Nevertheless, I'd love to see what a modern clean-slate implementation might look like. I'm sure there'd be much disagreement about what warrants replacement and what doesn't ;)
- mangix 7y ago
  musl?
  - saagarjha 7y ago
    musl still has many of the same issues, as it needs to remain compatible with POSIX.
- _rtld_global_ro 7y ago
  errno is thread-safe as it resides in TLS. Part of libc such as pthread is very hard, and so does the dynamic linker. The latter is supposed to handle TLS data as required by TLS/elf spec, which more or less requires a full thread/pthread implementation. As a result, it is rather difficult to break down libc, without writing the whole thing in the first place. golang might be an exception (no linkage against libc), but AFAIK it doesn't use TLS. It's a pretty hard task to tackle libc (not just string functions, obviously), maybe for the same reason there're only a few libc implementations (support full dynamic linking).
  - weberc2 7y ago
    What is TLS? > Part of libc such as pthread is very hard, and so does the dynamic linker. What does this mean? I'm having trouble parsing this as English much less making sense of the argument :) > It's a pretty hard task to tackle libc (not just string functions, obviously), maybe for the same reason there're only a few libc implementations (support full dynamic linking). Why does libc ship with string functions in the first place? What is special about strings that they are included in libc while other data structures (to my knowledge) are not? Presumably you don't need strings to implement malloc or other parts of libc?
alexhutcheson 7y ago
Most of the big chip vendors provide a libc implementation along with the proprietary compilers that they maintain for their chips. On the TI MSP430, I've had some projects that were too big to fit into flash if I used the libc (newlib) that's bundled with the open source MSP430-GCC compiler[1]. By contrast, if I compiled with the proprietary TI compiler[2] and its libc, the binary was small enough to fit. My experience is a bit dated at this point, so newlib and the GCC fork may have improved enough to reach parity by now. The dominance of GCC and LLVM on x86[-64] machines has conditioned many of us to take great open source build toolchains for granted, but embedded systems are comparatively the wild west - there are still a lot of proprietary build toolchains out there. Obviously this is a negative if you're committed to using open source tools, but one positive is that it's a niche where companies can still make money selling compilers (and therefore employing engineers who work on compilers). [1] http://www.ti.com/tool/MSP430-GCC-OPENSOURCE http://www.ti.com/tool/MSP430-GCC-OPENSOURCE [2] http://www.ti.com/tool/MSP-CGT http://www.ti.com/tool/MSP-CGT
jbuzbee 7y ago
If you missed it, this is from Keith Packard, big-time X Window System developer. https://en.wikipedia.org/wiki/Keith_Packard https://en.wikipedia.org/wiki/Keith_Packard
ChuckMcM 7y ago
This is an excellent libc for embedded systems. I am looking to add it to a self hosted tcc (Bellard's C compiler) on ARM Cortex-M systems.
snops 7y ago
An interesting non-free alternative is the SEGGER Runtime Library [1] which works with GCC, but with smaller code size than newlib-nano and significantly faster floating point emulation [2] for processors like the Cortex M3 or RISCV wih no FPU. It would be interesting to see if newlib-nano or picolibc could use these tricks. [1] https://www.segger.com/products/development-tools/runtime-library/ https://www.segger.com/products/development-tools/runtime-li... [2] https://blog.segger.com/floating-point-face-off/ https://blog.segger.com/floating-point-face-off/
pantalaimon 7y ago
Looks like it can save a good chunk of memory compared to newlib: https://github.com/RIOT-OS/RIOT/pull/12305 https://github.com/RIOT-OS/RIOT/pull/12305

118 points by homarp 7y ago

Ecco 7y ago

The biggest news is probably the fact that it's BSD licensed. Having some GPL parts had driven me away from newlib before.

delinka 7y ago
If I were in the market for such a library, I'd still have serious concerns. The GPL license still appears in the root of the repo as if the entire codebase might be subject to GPL; It appears to be a fork of newlib (GPL) and that means it might not qualify as a "clean room implementation" of GPL'd code - the legal team in a corporation is going to insist on a ton of due diligence on Picolibc before they allow it to be used.
- AWildC182 7y ago
  Or we could stop mindlessly releasing closed source encrypted firmware?
- derefr 7y ago
  Quoting the README: > Remove[d] unused code with non-BSD licenses. There's still a pile of unused code hanging around, but all non-BSD licensed bits have been removed to make the licensing situation clear. Picolibc is BSD licensed. From my understanding, Newlib is a GPL-licensed project that has some GPL-licensed contributions and some BSD-licensed contributions (in the sense that these aren't original code written by contributors, but rather are copy-and-paste inclusions of pieces from various GPL- and BSD-licensed codebases.) The author created Picolibc by (doing the moral equivalent of) cherry-picking out a small base of the commits to Newlib—all of which were copy-and-pastes from BSD-licensed codebases. --- In some sense, Newlib is very similar to how a Linux distro works, repackaging upstream code components (of various licenses) together. Picolibc, then, would be a lot like creating a new Linux distro, derived "from" Ubuntu, that happens to use only the packages in Ubuntu that are themselves directly copied in from upstream Debian (pretending for a moment that Ubuntu does this rather than re-signing packages with their own keys.) Is such a distro really "derived from Ubuntu"? Or is it derived "from" Debian, with an Ubuntu mirror server just serving as a pipe that some Debian packages went through on their trip "from" Debian's hands "into" the hands of the new distro? --- Either way, worries that the GPL could infect this effort are misplaced—even if it comes to a legal battle, at any point, the same codebase could be recreated (with a bit of a schlep) by just going to all the same BSD-licensed upstream sources that Newlib's contributors pulled from, and doing the same copy-and-pasting that Newlib's contributors did.
enriquto 7y ago
you license fanatics are so annoying...
- bpt3 7y ago
  There's nothing annoying about understanding how you're allowed to use something created by another person, especially when legal liability is involved.
- dang 7y ago
  Please don't post unsubstantive comments here, especially not nasty ones, regardless of how annoying other commenters are. Otherwise we all go down an annoyance spiral. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

deleted 7y ago

[deleted]

nerpderp82 7y ago

Needs to have a strong overlap with WASI.

saagarjha 7y ago
Why?

stock_toaster 7y ago

I didn't see this addressed in the post, but why not musl?

zamadatix 7y ago
The original post linked in the opening covers some of the search and reasoning.
- justincormack 7y ago
  it says "A more lively project; still, definitely targets systems with a real Linux kernel." but actually people are using Musl for other use cases. And it is BSD licensed, and very high quality.
  - stefan_ 7y ago
    Okay, what good is any of that if musl literally does not compile without Linux headers? This is for systems that are at best a small RTOS, and there is no standard for what those operating systems offer to a libc.
    - justincormack 7y ago
      Musl upstream only supports Linux, but it is so easy to port that there are large numbers of ports to non Linux environments.
  - digikata 7y ago
    There were other considerations against musl there. I'd summarize them as targeting a desktop/server with kernel level system, or at least a mid-to-high end embedded system, has different code, memory, and optimization tradeoffs than the smaller embedded systems that seem to be targeted with picolibc.
fra 7y ago
For what it's worth: musl is much larger than newlib-nano or picolibc. Most of the embedded projects I've worked on could not afford to use musl.

AceJohnny2 7y ago

I wish we could do away with (parts of) libc entirely. Many of its design principles and limitations date back to its inception and don't make sense today, even on "embedded" systems. - function names limited to 6 characters (because of compiler/linker memory limits at the time?) - non-reentrency of many functions (strtok(), errno...) - dangerous string manipulation functions (see the various iterations of strcpy()) - and so much more... Of course, I'm missing the point. Libc continues on because it's standard and backwards-compatible, and lets you leverage 50 years of software ecosystem. Nevertheless, I'd love to see what a modern clean-slate implementation might look like. I'm sure there'd be much disagreement about what warrants replacement and what doesn't ;)

mangix 7y ago
musl?
- saagarjha 7y ago
  musl still has many of the same issues, as it needs to remain compatible with POSIX.
_rtld_global_ro 7y ago
errno is thread-safe as it resides in TLS. Part of libc such as pthread is very hard, and so does the dynamic linker. The latter is supposed to handle TLS data as required by TLS/elf spec, which more or less requires a full thread/pthread implementation. As a result, it is rather difficult to break down libc, without writing the whole thing in the first place. golang might be an exception (no linkage against libc), but AFAIK it doesn't use TLS. It's a pretty hard task to tackle libc (not just string functions, obviously), maybe for the same reason there're only a few libc implementations (support full dynamic linking).
- weberc2 7y ago
  What is TLS? > Part of libc such as pthread is very hard, and so does the dynamic linker. What does this mean? I'm having trouble parsing this as English much less making sense of the argument :) > It's a pretty hard task to tackle libc (not just string functions, obviously), maybe for the same reason there're only a few libc implementations (support full dynamic linking). Why does libc ship with string functions in the first place? What is special about strings that they are included in libc while other data structures (to my knowledge) are not? Presumably you don't need strings to implement malloc or other parts of libc?

alexhutcheson 7y ago

Most of the big chip vendors provide a libc implementation along with the proprietary compilers that they maintain for their chips. On the TI MSP430, I've had some projects that were too big to fit into flash if I used the libc (newlib) that's bundled with the open source MSP430-GCC compiler[1]. By contrast, if I compiled with the proprietary TI compiler[2] and its libc, the binary was small enough to fit. My experience is a bit dated at this point, so newlib and the GCC fork may have improved enough to reach parity by now. The dominance of GCC and LLVM on x86[-64] machines has conditioned many of us to take great open source build toolchains for granted, but embedded systems are comparatively the wild west - there are still a lot of proprietary build toolchains out there. Obviously this is a negative if you're committed to using open source tools, but one positive is that it's a niche where companies can still make money selling compilers (and therefore employing engineers who work on compilers). [1] http://www.ti.com/tool/MSP430-GCC-OPENSOURCE http://www.ti.com/tool/MSP430-GCC-OPENSOURCE [2] http://www.ti.com/tool/MSP-CGT http://www.ti.com/tool/MSP-CGT

jbuzbee 7y ago

If you missed it, this is from Keith Packard, big-time X Window System developer. https://en.wikipedia.org/wiki/Keith_Packard https://en.wikipedia.org/wiki/Keith_Packard

ChuckMcM 7y ago

This is an excellent libc for embedded systems. I am looking to add it to a self hosted tcc (Bellard's C compiler) on ARM Cortex-M systems.

snops 7y ago

An interesting non-free alternative is the SEGGER Runtime Library [1] which works with GCC, but with smaller code size than newlib-nano and significantly faster floating point emulation [2] for processors like the Cortex M3 or RISCV wih no FPU. It would be interesting to see if newlib-nano or picolibc could use these tricks. [1] https://www.segger.com/products/development-tools/runtime-library/ https://www.segger.com/products/development-tools/runtime-li... [2] https://blog.segger.com/floating-point-face-off/ https://blog.segger.com/floating-point-face-off/

pantalaimon 7y ago

Looks like it can save a good chunk of memory compared to newlib: https://github.com/RIOT-OS/RIOT/pull/12305 https://github.com/RIOT-OS/RIOT/pull/12305