7 ms·

Raw Linux Threads via System Calls (2015)

200 points by eatonphil 10y ago

signa11 10y ago
previous submission: https://news.ycombinator.com/item?id=9675248 https://news.ycombinator.com/item?id=9675248
- tux1968 10y ago
  The article notes that it has been augmented with a follow up piece since that original submission: http://nullprogram.com/blog/2016/09/23/ http://nullprogram.com/blog/2016/09/23/
  - signa11 10y ago
    which also was submitted (https://news.ycombinator.com/item?id=12579668 https://news.ycombinator.com/item?id=12579668) but unfortunately garnered a muted response from the masses...
antirez 10y ago
Writing a toy libc is a very fascinating exercise, if you happen to like this kind of stuff.
JoachimSchipper 10y ago
"Linux has an elegant and beautiful design when it comes to threads" - unifying threads and processes does simplify some things, but it makes it extremely hard to change a process-level property such as the effective user/group id of a process: https://ewontfix.com/17/ https://ewontfix.com/17/.
- DSMan195276 10y ago
  I'm surprised no solution in the kernel has ever been done. But that said, I suspect the big reason is that setting uid and gid with threads is going to be messy no way how you slice it. Making setuid and setgid work per-thread, and then having programs coordinate a privileged loss over all the threads is easier to do since the programmer knows where in their code is an appropriate place to do the privileged loss, rather then having one thread trigger it to happen regardless of what the other threads are currently doing. But of course, there's always the chance a thread doesn't drop it's privileges, and since libc has to emulate the POSIX API anyway it just makes things a lot messier since it has to force all the threads to call the setuid or setgid function without having them call it in code. But of course, I think it's notable that if you can (And in most cases, you probably can) you should be dropping privileges well before you start threads and the body of your program, and if you do that then this doesn't even become an issue in the first place. I would suspect that this is part of why this hasn't been added to the kernel - the user-space solutions are sufficient simply because most programs don't actually ever call these functions after they've already setup threads.
  - ajross 10y ago
    > I'm surprised no solution in the kernel has ever been done. It has, just not the one the linked article is whining about. "Process level" capability management in modern security architectures is basically never based on uid/gid. The kernel is filled with tools (too many, IMHO) for doing this. Looping over your process to try to change the IDs is the Wrong Thing, basically.
  - crest 10y ago
    Some language runtimes create threads before the application main() function is executed. Even if Linux has to retain bug-for-bug ABI compatibility there is valid reason to only provide fucked up the setid() syscalls. The problem is even worse than it looks at first glance because setid() can fail per thread and there is no sane way to recover from this in userspace. Atomic syscalls are the only sane API for this and Linux is lacking those.
faragon 10y ago
As curiosity it is great. BTW, if POSIX thread implementation is not good enough in the default C library, it should be revised instead of everyone rewriting stuff for each specific Unix-like flavour. Rationale: Platform-specific goodies are great for specific cases, but terrible for standard and multi-platform development.
- scott_s 10y ago
  I'm a little confused by your comment. This article is a pedagogical exercise: it demonstrates how thread creation works on Linux by implementing it from scratch using assembly. The author's intention is not to create a new library, but rather to show how such a library might be implemented. Nowhere does the author advocate doing this in production code over using the standard implementation of Pthreads on Linux (https://en.wikipedia.org/wiki/Native_POSIX_Thread_Library https://en.wikipedia.org/wiki/Native_POSIX_Thread_Library).
  - faragon 10y ago
    >>"The author's intention is not to create a new library, but ..." I don't know author's intention. Do you? :-) My point was a warning about how bad idea is using platform-specific non-standard APIs. As exercise, of course it is fine, that is why I up-voted this history.
    - scott_s 10y ago
      > I don't know author's intention. Do you? :-) I do not know the author's intention; I am not certain. But I'm reasonably confident. And that's a reasonable warning, just a bit of a non-sequitur. Hence my confusion.
em3rgent0rdr 10y ago
Only 15 instructions to spawn a thread isn't something to write home about considering cost of context switching is much higher.
- woodrowbarlow 10y ago
  i interpreted that statement as the author encouraging the reader to take a peek behind the curtain of something they might consider incomprehensible magic (the author is saying "look, it's not that scary!"), rather than as a claim that thread spawning is computationally cheap.
  - em3rgent0rdr 10y ago
    Ok, fair point...I see that.

200 points by eatonphil 10y ago

signa11 10y ago

previous submission: https://news.ycombinator.com/item?id=9675248 https://news.ycombinator.com/item?id=9675248

tux1968 10y ago
The article notes that it has been augmented with a follow up piece since that original submission: http://nullprogram.com/blog/2016/09/23/ http://nullprogram.com/blog/2016/09/23/
- signa11 10y ago
  which also was submitted (https://news.ycombinator.com/item?id=12579668 https://news.ycombinator.com/item?id=12579668) but unfortunately garnered a muted response from the masses...

antirez 10y ago

Writing a toy libc is a very fascinating exercise, if you happen to like this kind of stuff.

JoachimSchipper 10y ago

"Linux has an elegant and beautiful design when it comes to threads" - unifying threads and processes does simplify some things, but it makes it extremely hard to change a process-level property such as the effective user/group id of a process: https://ewontfix.com/17/ https://ewontfix.com/17/.

DSMan195276 10y ago
I'm surprised no solution in the kernel has ever been done. But that said, I suspect the big reason is that setting uid and gid with threads is going to be messy no way how you slice it. Making setuid and setgid work per-thread, and then having programs coordinate a privileged loss over all the threads is easier to do since the programmer knows where in their code is an appropriate place to do the privileged loss, rather then having one thread trigger it to happen regardless of what the other threads are currently doing. But of course, there's always the chance a thread doesn't drop it's privileges, and since libc has to emulate the POSIX API anyway it just makes things a lot messier since it has to force all the threads to call the setuid or setgid function without having them call it in code. But of course, I think it's notable that if you can (And in most cases, you probably can) you should be dropping privileges well before you start threads and the body of your program, and if you do that then this doesn't even become an issue in the first place. I would suspect that this is part of why this hasn't been added to the kernel - the user-space solutions are sufficient simply because most programs don't actually ever call these functions after they've already setup threads.
- ajross 10y ago
  > I'm surprised no solution in the kernel has ever been done. It has, just not the one the linked article is whining about. "Process level" capability management in modern security architectures is basically never based on uid/gid. The kernel is filled with tools (too many, IMHO) for doing this. Looping over your process to try to change the IDs is the Wrong Thing, basically.
- crest 10y ago
  Some language runtimes create threads before the application main() function is executed. Even if Linux has to retain bug-for-bug ABI compatibility there is valid reason to only provide fucked up the setid() syscalls. The problem is even worse than it looks at first glance because setid() can fail per thread and there is no sane way to recover from this in userspace. Atomic syscalls are the only sane API for this and Linux is lacking those.

faragon 10y ago

As curiosity it is great. BTW, if POSIX thread implementation is not good enough in the default C library, it should be revised instead of everyone rewriting stuff for each specific Unix-like flavour. Rationale: Platform-specific goodies are great for specific cases, but terrible for standard and multi-platform development.

scott_s 10y ago
I'm a little confused by your comment. This article is a pedagogical exercise: it demonstrates how thread creation works on Linux by implementing it from scratch using assembly. The author's intention is not to create a new library, but rather to show how such a library might be implemented. Nowhere does the author advocate doing this in production code over using the standard implementation of Pthreads on Linux (https://en.wikipedia.org/wiki/Native_POSIX_Thread_Library https://en.wikipedia.org/wiki/Native_POSIX_Thread_Library).
- faragon 10y ago
  >>"The author's intention is not to create a new library, but ..." I don't know author's intention. Do you? :-) My point was a warning about how bad idea is using platform-specific non-standard APIs. As exercise, of course it is fine, that is why I up-voted this history.
  - scott_s 10y ago
    > I don't know author's intention. Do you? :-) I do not know the author's intention; I am not certain. But I'm reasonably confident. And that's a reasonable warning, just a bit of a non-sequitur. Hence my confusion.

em3rgent0rdr 10y ago

Only 15 instructions to spawn a thread isn't something to write home about considering cost of context switching is much higher.

woodrowbarlow 10y ago
i interpreted that statement as the author encouraging the reader to take a peek behind the curtain of something they might consider incomprehensible magic (the author is saying "look, it's not that scary!"), rather than as a claim that thread spawning is computationally cheap.
- em3rgent0rdr 10y ago
  Ok, fair point...I see that.