6 ms·

Musl-libc version 1.0.0 released

100 points by deoxxa 13y ago

justincormack 13y ago
I have been using Musl for various purposes for quite some time. I think my favourite things are: * The team are very helpful, and fix things fast and correctly. You can see some of the detail in this post on race conditions in glibc http://ewontfix.com/16/ http://ewontfix.com/16/ * It is very standards compliant, so it is a good portability test for your code. It is a bit like using a BSD libc, except it does have Linux specific syscalls etc. * Static linking works, which it doesnt properly in glibc. If you want to be like those trendy Go people and provide simple statically linked single file binaries, for running in containers etc then you can. And they will be small: the glibc shared library is 1.8MB (not including rt, pthread etc), while a Musl statically linked copy of Lua for example is only 160k. Static linking is very useful for all sorts of things, my most recent use was cross compiling bianries to run under qemu-user for testing. * It is BSD licensed so you can use it in completely different projects however you want, eg OSv uses it http://osv.io/ http://osv.io/ * The code is small and readable. If you want to know how things work you can just take a look, quickly find the code and understand it. Just finding the code in glibc takes ages. I keep a reference copy just for this purpose. You can use the "cross compiler" musl-gcc that ships with it to build if your code is happy being cross compiled (should be, most things just work). If you have a lot of dependencies it might be easier to use a distro, I use Sabotage in a chroot usually https://github.com/sabotage-linux/sabotage https://github.com/sabotage-linux/sabotage
- pascal_cuoq 13y ago
  Not to offend any secondary Musl contributor, but I think that your first favorite thing is all the more impressive that “The team” is essentially one person.
  - dalias 13y ago
    It started out that way, but these days we have several other developers who are pretty active. You can see from our stats on ohloh: https://www.ohloh.net/p/musl/contributors?query=&sort=commits https://www.ohloh.net/p/musl/contributors?query=&sort=commit... Also, without the work of the entire community testing and reporting issues getting various packages to work with musl, it would have taken forever to get this far. Testers and bug reporters are seriously under-appreciated!
- anonemouscoward 13y ago
  > Static linking works False. It doesn't. At the very least, VDSO support on Linux for static builds is broken. I.e., calls to time() and gettimeofday() and friends will be orders of magnitude slower when linking statically. Looking at the code I think nobody on the team was even aware that this is an important corner case. I'll stick to glibc, they at least do proper testing.
dalias 13y ago
A special treat to go with the release, that's not linked on the website yet: in-browser demo setup using jslinux: http://www.musl-libc.org/jslinux/ http://www.musl-libc.org/jslinux/
- justincormack 13y ago
  Cool, I have never run a C compiler in JavaScript before...
nathell 13y ago
Any Linux distro built against musl out there?
- dalias 13y ago
  Yet, there are quite a few. See the wiki: http://wiki.musl-libc.org/wiki/Projects_using_musl http://wiki.musl-libc.org/wiki/Projects_using_musl A few big names are in the process of adding musl-based variants or switching entirely to musl, but most of these are still experimental. Alpine hopes to finish switching over sometime this year. Aboriginal Linux too. Bedrock Linux is already using musl in their latest release, but due to the way their distro works, they're in the unique situation where they don't have to build a lot of packages against the system libc, so they have it much easier.
- ParadigmComplex 13y ago
  I'm the founder / lead developer of Bedrock Linux, which as dalias said uses musl (as of our latest release). They've been very prompt about fixing any issues we have with musl and making it very hard to find excuses not to use musl.
justincormack 13y ago
There is a very minimal Docker build with Musl showing how it can be usd to build a minimal container https://github.com/mwcampbell/vontainrr https://github.com/mwcampbell/vontainrr
- agumonkey 13y ago
  I get a 404. Maybe this https://github.com/mwcampbell/docker-muslbase https://github.com/mwcampbell/docker-muslbase ?
  - justincormack 13y ago
    Yes that's the one.. Android paste...
    - NicoJuicy 13y ago
      So you are saying that there is a bug in the Android copy-paste? :-) (even though the 2 links are completely different)
      - justincormack 13y ago
        I think the bug is in my fingers
dalias 13y ago
Hi all. I'm the original author and maintainer of musl, and I'm happy to answer any questions anyone might have about the project. Glad to see so much interest!
- stephencanon 13y ago
  No question, but congratulations! Very exciting.
- Fede_V 13y ago
  First - awesome job! I'm curious, what motivated you to start working on it?
  - dalias 13y ago
    The original idea goes back to around 2005 and frustration at the ever-growing size of glibc and poor (slow) support for UTF-8. At the time I made a prototype and used it personally. In 2010 I relaunched the project with a goal of doing it a lot better and targeting user bases who might actually be looking for a new libc; later on this turned out to include a lot of people unhappy with GPL/LGPL, so we re-licensed from LGPLv2.1+ to MIT.
- noselasd 13y ago
  Is there any overview of differences/missing pieces in respect to glibc ? (I'd expect most of them are GNU extensions or or legacy, e.g. at a glance, no SunRPC, fts.h, obstack.h, fstab.h). And how is C11 library support ?
  - dalias 13y ago
    Some of these things are covered on the comparison page I did at http://www.etalabs.net/compare_libcs.html http://www.etalabs.net/compare_libcs.html In regards to fts.h, the version in glibc is not even usable because it only works in 32-bit off_t mode, and stat() randomly fails in this mode due to modern inode numbers being 64-bit. See the recent thread on the glibc mailing list: https://sourceware.org/ml/libc-alpha/2014-03/msg00408.html https://sourceware.org/ml/libc-alpha/2014-03/msg00408.html and the bug tracker issue linked. At this point both musl and glibc only cover a very small portion of C11 additions. glibc has a GSoC project proposal (with 3+ students interested in it) to add C11 threads, which is one of the big missing areas. We've been holding off on doing so in musl in an interest of making sure we do it in a way that's ABI-compatible with glibc, so we'll probably add C11 threads in the near future too.
jokoon 13y ago
wait, but I thought premature optimization was evil !
- dalias 13y ago
  musl actually has very little in the way of optimization, premature or otherwise. Most functions are written to be as simple and direct as possible. Often but not always this gives near-optimal size and speed too. The places where more complex approaches are used are _mostly_ situations where the naive solution would have corner cases that fail or corner cases that have pathologically bad performance and could be used as a DoS attack vector. The main exception is things like memcpy, memset, etc.; if they're slow, people are generally very unhappy. BTW, one difficulty in libc is that it's hard to know what optimizations are "premature" because you can't envision everybody's usage cases without an enormous volume of experience with third-party code.
  - jokoon 13y ago
    Isn't this implementation targeting embedded platforms ? I read it's reducing its memory footprint by using static linking...
deleted 13y ago
[deleted]
Zen101 13y ago
Interesting, great update

100 points by deoxxa 13y ago

I have been using Musl for various purposes for quite some time. I think my favourite things are: * The team are very helpful, and fix things fast and correctly. You can see some of the detail in this post on race conditions in glibc http://ewontfix.com/16/ http://ewontfix.com/16/ * It is very standards compliant, so it is a good portability test for your code. It is a bit like using a BSD libc, except it does have Linux specific syscalls etc. * Static linking works, which it doesnt properly in glibc. If you want to be like those trendy Go people and provide simple statically linked single file binaries, for running in containers etc then you can. And they will be small: the glibc shared library is 1.8MB (not including rt, pthread etc), while a Musl statically linked copy of Lua for example is only 160k. Static linking is very useful for all sorts of things, my most recent use was cross compiling bianries to run under qemu-user for testing. * It is BSD licensed so you can use it in completely different projects however you want, eg OSv uses it http://osv.io/ http://osv.io/ * The code is small and readable. If you want to know how things work you can just take a look, quickly find the code and understand it. Just finding the code in glibc takes ages. I keep a reference copy just for this purpose. You can use the "cross compiler" musl-gcc that ships with it to build if your code is happy being cross compiled (should be, most things just work). If you have a lot of dependencies it might be easier to use a distro, I use Sabotage in a chroot usually https://github.com/sabotage-linux/sabotage https://github.com/sabotage-linux/sabotage

pascal_cuoq 13y ago
Not to offend any secondary Musl contributor, but I think that your first favorite thing is all the more impressive that “The team” is essentially one person.
- dalias 13y ago
  It started out that way, but these days we have several other developers who are pretty active. You can see from our stats on ohloh: https://www.ohloh.net/p/musl/contributors?query=&sort=commits https://www.ohloh.net/p/musl/contributors?query=&sort=commit... Also, without the work of the entire community testing and reporting issues getting various packages to work with musl, it would have taken forever to get this far. Testers and bug reporters are seriously under-appreciated!
anonemouscoward 13y ago
> Static linking works False. It doesn't. At the very least, VDSO support on Linux for static builds is broken. I.e., calls to time() and gettimeofday() and friends will be orders of magnitude slower when linking statically. Looking at the code I think nobody on the team was even aware that this is an important corner case. I'll stick to glibc, they at least do proper testing.

dalias 13y ago

A special treat to go with the release, that's not linked on the website yet: in-browser demo setup using jslinux: http://www.musl-libc.org/jslinux/ http://www.musl-libc.org/jslinux/

justincormack 13y ago
Cool, I have never run a C compiler in JavaScript before...

nathell 13y ago

Any Linux distro built against musl out there?

dalias 13y ago
Yet, there are quite a few. See the wiki: http://wiki.musl-libc.org/wiki/Projects_using_musl http://wiki.musl-libc.org/wiki/Projects_using_musl A few big names are in the process of adding musl-based variants or switching entirely to musl, but most of these are still experimental. Alpine hopes to finish switching over sometime this year. Aboriginal Linux too. Bedrock Linux is already using musl in their latest release, but due to the way their distro works, they're in the unique situation where they don't have to build a lot of packages against the system libc, so they have it much easier.
ParadigmComplex 13y ago
I'm the founder / lead developer of Bedrock Linux, which as dalias said uses musl (as of our latest release). They've been very prompt about fixing any issues we have with musl and making it very hard to find excuses not to use musl.

justincormack 13y ago

There is a very minimal Docker build with Musl showing how it can be usd to build a minimal container https://github.com/mwcampbell/vontainrr https://github.com/mwcampbell/vontainrr

agumonkey 13y ago
I get a 404. Maybe this https://github.com/mwcampbell/docker-muslbase https://github.com/mwcampbell/docker-muslbase ?
- justincormack 13y ago
  Yes that's the one.. Android paste...
  - NicoJuicy 13y ago
    So you are saying that there is a bug in the Android copy-paste? :-) (even though the 2 links are completely different)
    - justincormack 13y ago
      I think the bug is in my fingers

dalias 13y ago

Hi all. I'm the original author and maintainer of musl, and I'm happy to answer any questions anyone might have about the project. Glad to see so much interest!

stephencanon 13y ago
No question, but congratulations! Very exciting.
Fede_V 13y ago
First - awesome job! I'm curious, what motivated you to start working on it?
- dalias 13y ago
  The original idea goes back to around 2005 and frustration at the ever-growing size of glibc and poor (slow) support for UTF-8. At the time I made a prototype and used it personally. In 2010 I relaunched the project with a goal of doing it a lot better and targeting user bases who might actually be looking for a new libc; later on this turned out to include a lot of people unhappy with GPL/LGPL, so we re-licensed from LGPLv2.1+ to MIT.
noselasd 13y ago
Is there any overview of differences/missing pieces in respect to glibc ? (I'd expect most of them are GNU extensions or or legacy, e.g. at a glance, no SunRPC, fts.h, obstack.h, fstab.h). And how is C11 library support ?
- dalias 13y ago
  Some of these things are covered on the comparison page I did at http://www.etalabs.net/compare_libcs.html http://www.etalabs.net/compare_libcs.html In regards to fts.h, the version in glibc is not even usable because it only works in 32-bit off_t mode, and stat() randomly fails in this mode due to modern inode numbers being 64-bit. See the recent thread on the glibc mailing list: https://sourceware.org/ml/libc-alpha/2014-03/msg00408.html https://sourceware.org/ml/libc-alpha/2014-03/msg00408.html and the bug tracker issue linked. At this point both musl and glibc only cover a very small portion of C11 additions. glibc has a GSoC project proposal (with 3+ students interested in it) to add C11 threads, which is one of the big missing areas. We've been holding off on doing so in musl in an interest of making sure we do it in a way that's ABI-compatible with glibc, so we'll probably add C11 threads in the near future too.

jokoon 13y ago

wait, but I thought premature optimization was evil !

dalias 13y ago
musl actually has very little in the way of optimization, premature or otherwise. Most functions are written to be as simple and direct as possible. Often but not always this gives near-optimal size and speed too. The places where more complex approaches are used are _mostly_ situations where the naive solution would have corner cases that fail or corner cases that have pathologically bad performance and could be used as a DoS attack vector. The main exception is things like memcpy, memset, etc.; if they're slow, people are generally very unhappy. BTW, one difficulty in libc is that it's hard to know what optimizations are "premature" because you can't envision everybody's usage cases without an enormous volume of experience with third-party code.
- jokoon 13y ago
  Isn't this implementation targeting embedded platforms ? I read it's reducing its memory footprint by using static linking...

deleted 13y ago

[deleted]

Zen101 13y ago

Interesting, great update