3 ms·

I started recently learning Rust, and a thing I don't understand is why they decided that threads have to be done this way and not simply wrapping what is avail

by scoutt 6y ago

I started recently learning Rust, and a thing I don't understand is why they decided that threads have to be done this way and not simply wrapping what is available in the OS (I know that at some point, at the core, a Rust thread is the same thing I create with pthread_create). Like, in both Windows and Linux, threads exist as a callback function (pointer to function) with its own stack, in which a void* parameter is passed. Can this be done in Rust? Can multithreading be done without closures?

rcxdude 6y ago
What you've described with pthread_create is basically a closure, just implemented within the limitations of C and like many things in C, wildly unsafe. You can of course interact with the underlying implementation if you want (Rust is quite capable of calling into libc directly), but then you're responsible for ensuring safety. Rust provides a safe wrapper over this with a closure which is generally easier to use and harder to misuse than the equivilent C.
- scoutt 6y ago
  Thanks. > a closure which is generally easier to use I didn't experimented much with closures so I can't discuss about simplicity yet (but the article explains that "Closures can be pretty radioactive"). > within the limitations of C Threads are exposed like that by the operating system. The C implementation is just a way to have access to what the operating system exposes, so I think it should be more like a limitation of the operating system?
  - rcxdude 6y ago
    It's more the limitations of C: C doesn't have the concept of a closure, so if you are passing around a function with its state, you must pass them around separately (and generally the state must be behind a void pointer, because you can't have a generic interface). In rust and C++ you can make a function with associated state easily and pass it around, both as a raw value with a unique type (though it's difficult to name), and as a reference or heap-allocated value which can be handled generically. In all these cases you need to make sure that any pointers remain valid until the function has finished executing. In C and C++ the compiler cannot help you, in rust the compiler can catch cases where function could be called after those pointers are no longer valid. This is what causes the 'radioactive' property of closures containing references to other data (they are also radioactive in C++, just at runtime and unpredictably, as opposed to at compile time in rust). If the example code in the article was written in C++ (and the complaint about the closure type which is also present in C++ was addressed), there would be a subtle concurrency bug, because the database handle could be closed and freed before the closure function actually executes. In all these cases the solution is either making sure the closure runs before the data it needs becomes invalid (which is not generally easy to do if you're planning to use it to spawn a thread), or you move the data onto the heap, either arranging so the closure frees the data when it executes (and then guaranteeing that it executes), or using some shared reference count system to ensure it only gets freed when all code which might want to use it is done. In Rust and C++ there are mechanisms in the standard library and their closure implementations to achieve this, in C you must do it manually.
    - scoutt 6y ago
      Thanks. I'm familiar with all these concepts except for Rust closures and its relationship with multithreading. I was talking about that at the lowest (OS) level, the thread mechanics is exactly what C presents us as a thread (and all we can do with a thread). The rest (the alternative options C++ and Rust implement) is handy to have, built on top of the standard implementation, but not necessary to actually operate with threads (with tradeoffs, of course). So I don't see it as a limitation: it's just the way it is; the way the OS expose multithreading to its users through its libraries. Not having those handy features is not a limitation (in what multithread regards). What a I could probably see as a limitation for a system language perhaps it's the fact that I cannot easily reach what the OS exposes (as multithreading implementation) and that I have to use a special construct to reach it. But if in Rust, Closures are just function pointers, then I guess it's fine (and this is what I am trying to understand with your explanations and with the article).
dthul 6y ago
Note that you don't need a closure in Rust to start a thread. You can just specify a function to be called (because functions implement FnOnce).
- scoutt 6y ago
  Thanks. My question emerged from the fact that all the Rust threads examples I see are in the closure form || {}.
  - zesterer 6y ago
    Because the closure form is much more useful. You very rarely want to create a thread without first passing some state to it, and environment-passing with closures is a brilliantly ergonomic way to do this.

by scoutt 6y ago

rcxdude 6y ago

What you've described with pthread_create is basically a closure, just implemented within the limitations of C and like many things in C, wildly unsafe. You can of course interact with the underlying implementation if you want (Rust is quite capable of calling into libc directly), but then you're responsible for ensuring safety. Rust provides a safe wrapper over this with a closure which is generally easier to use and harder to misuse than the equivilent C.

scoutt 6y ago
Thanks. > a closure which is generally easier to use I didn't experimented much with closures so I can't discuss about simplicity yet (but the article explains that "Closures can be pretty radioactive"). > within the limitations of C Threads are exposed like that by the operating system. The C implementation is just a way to have access to what the operating system exposes, so I think it should be more like a limitation of the operating system?
- rcxdude 6y ago
  It's more the limitations of C: C doesn't have the concept of a closure, so if you are passing around a function with its state, you must pass them around separately (and generally the state must be behind a void pointer, because you can't have a generic interface). In rust and C++ you can make a function with associated state easily and pass it around, both as a raw value with a unique type (though it's difficult to name), and as a reference or heap-allocated value which can be handled generically. In all these cases you need to make sure that any pointers remain valid until the function has finished executing. In C and C++ the compiler cannot help you, in rust the compiler can catch cases where function could be called after those pointers are no longer valid. This is what causes the 'radioactive' property of closures containing references to other data (they are also radioactive in C++, just at runtime and unpredictably, as opposed to at compile time in rust). If the example code in the article was written in C++ (and the complaint about the closure type which is also present in C++ was addressed), there would be a subtle concurrency bug, because the database handle could be closed and freed before the closure function actually executes. In all these cases the solution is either making sure the closure runs before the data it needs becomes invalid (which is not generally easy to do if you're planning to use it to spawn a thread), or you move the data onto the heap, either arranging so the closure frees the data when it executes (and then guaranteeing that it executes), or using some shared reference count system to ensure it only gets freed when all code which might want to use it is done. In Rust and C++ there are mechanisms in the standard library and their closure implementations to achieve this, in C you must do it manually.
  - scoutt 6y ago
    Thanks. I'm familiar with all these concepts except for Rust closures and its relationship with multithreading. I was talking about that at the lowest (OS) level, the thread mechanics is exactly what C presents us as a thread (and all we can do with a thread). The rest (the alternative options C++ and Rust implement) is handy to have, built on top of the standard implementation, but not necessary to actually operate with threads (with tradeoffs, of course). So I don't see it as a limitation: it's just the way it is; the way the OS expose multithreading to its users through its libraries. Not having those handy features is not a limitation (in what multithread regards). What a I could probably see as a limitation for a system language perhaps it's the fact that I cannot easily reach what the OS exposes (as multithreading implementation) and that I have to use a special construct to reach it. But if in Rust, Closures are just function pointers, then I guess it's fine (and this is what I am trying to understand with your explanations and with the article).

dthul 6y ago

Note that you don't need a closure in Rust to start a thread. You can just specify a function to be called (because functions implement FnOnce).

scoutt 6y ago
Thanks. My question emerged from the fact that all the Rust threads examples I see are in the closure form || {}.
- zesterer 6y ago
  Because the closure form is much more useful. You very rarely want to create a thread without first passing some state to it, and environment-passing with closures is a brilliantly ergonomic way to do this.