6 ms·

Let's say that, as you recommend, I only support one compiler and one build system in my C project. What is the "easy" solution to the problem stated by the aut

Let's say that, as you recommend, I only support one compiler and one build system in my C project. What is the "easy" solution to the problem stated by the author ? How do I add multi-threading to my project in a way that allows me to run a function on all available cores in two lines of code, while making the project cross-compile to every major OS and architecture from an M1 mac ? In rust the solution is just to edit a file named "Cargo.toml" and add one line in it: rayon = "*".

Arch-TK 5y ago
>rayon = "*". Doesn't this either effectively force you to pin versions to ensure your code doesn't break or require you to constantly re-test your code to ensure there wasn't a breaking change? I don't see how that's a desirable thing. >How do I add multi-threading to my project in a way that allows me to run a function on all available cores in two lines of code The author had to rewrite the project in a different language to add this cross platform multi-threading support. I'm not saying you can do this in two lines of code, the code likely needs some modifications and some considerations, but it's definitely easier to just use one of the multiple cross-platform threading libraries available (or even to write your own) and adapt the code to use it than it is to rewrite a whole project in rust. It's a little bit disingenuous to suggest that in rust any parallel processing task in any project out there can be achieved with 2 lines of code, it's like claiming that OMP can be effectively used by any C codebase with just a 1 line #pragma. If you write your code with the intention of using rayon for your multi-threading tasks then obviously it will not require any modification to make it work with rayon because presumably you had already written your code in a way that makes it usable with rayon. If you write your C code with the intention of using it with your multi-platform threading library of choice then it won't require changes to make it use that library.
- steveklabnik 5y ago
  > Doesn't this either Cargo pins versions via Cargo.lock for you automatically. It's still not a great idea, (because when you update dependencies it will go to whatever version is the latest, which may not be compatible) and crates.io disallows packages to be uploaded that use this, but it is the easiest built-in way. If you install cargo-edit, you could "cargo add rayon" and it will add it with the latest version specifier instead, which is slightly easier and better but isn't built-in (yet, someday...) > The author had to rewrite the project in a different language to add this cross platform multi-threading support. They're not saying that, they're saying "how do I do what Rayon does for Rust in C?" OpenMP is sorta kinda an answer there, but the post is claiming it's not as good of a solution. "Rewrite it in Rust is easier than using OpenMP" is not what's being said, and you're certainly right that porting to another language entirely is not simple. > If you write your code with the intention of using rayon for your multi-threading tasks then obviously it will not require any modification to make it work with rayon because presumably you had already written your code in a way that makes it usable with rayon. The claim being made is that due to things like the borrow checker, this is often true for Rust code generally, moreso than it is with C and OpenMP. And, in Rust, if it doesn't Just Work, you'll get an error message pointing out exactly why not, whereas (in my understanding, I haven't used OpenMP) that is not going to happen in C.
  - Arch-TK 5y ago
    > The claim being made is that due to things like the borrow checker, this is often true for Rust code generally, moreso than it is with C and OpenMP. And, in Rust, if it doesn't Just Work, you'll get an error message pointing out exactly why not, whereas (in my understanding, I haven't used OpenMP) that is not going to happen in C. Firstly, nobody knows what claim is being made here other than the author of the original question. Maybe let's let lovasoa reply. Secondly, no, unless rust is the most rigid language ever invented (which as far as I am aware it's not) there most certainly IS a way to write code in a way where it may need to be modified to work with another library. I highly doubt that literally any random code unit in a random codebase could be made instantly parallel with no code changes other than calling it using rayon, this is just wishful thinking. Even not considering fundamental problems like figuring out how to chunk a workload appropriately for parallelization. The point I was making is that for any given codebase, it's possible to write code in such a way that it can be used with some tool easily such that it only takes two lines to make use of that tool, just like it's possible to not do this and require rewriting things. You can't make judgements on how easy to use a library is on the basis of the fact that someone who rewrote their code specifically to use that library was able to use it by changing two lines of code. With OpenMP it certainly can sometimes be as easy as adding a #pragma omp parallel for somewhere, or it can be much more difficult than that. Nothing stops you from engineering a project to specifically be tailored towards working well with a particular library.
    - pornel 5y ago
      For me it was easier, or at least more pleasant, to rewrite the project in Rust than to deal with C build issues for various platforms. It's not only that one thing I've got stuck with. I've been maintaining this project for years, and there have been endless issues with the build: GNU vs BSD make incompatibilities. grep doesn't support the same flags on all systems. -o /dev/null does not work in msys. Flags for MSVC, GCC, and ICC flags, including odd ones like aligning the stack so it doesn't crash on SSE loads. The arcana of static linking with compiler-private runtime libraries, for multiple compilers. __declspec. Windows charset woes. MSVC<>MinGW incompatibilities. All the combinations of uname -m. All the missing headers. pkg-config sysroots. libpng.pc is sometimes sabotaged to force use of libpng-config. Moving -lm around. Apple clang has SDK includes separate from include paths. macOS system frameworks pulling in duplicate incompatible versions of libraries I'm trying to use. .h and .so ABI mismatches. Why are these my problems? I just want to process some pixels. And regarding Rust parallel code working out of the box, it really does happen. Rust's single-threaded iterators already require a high degree of isolation. The borrow checker already has shared XOR mutable rule for all code everywhere. Mutable globals are already banned. So yeah, it often is as easy as changing iter() to par_iter(). Best of all, when that is not safe, the compiler will show you exactly where and why, even if it's due to dependency of a dependency in some deeply nested function call.

Arch-TK 5y ago

>rayon = "*". Doesn't this either effectively force you to pin versions to ensure your code doesn't break or require you to constantly re-test your code to ensure there wasn't a breaking change? I don't see how that's a desirable thing. >How do I add multi-threading to my project in a way that allows me to run a function on all available cores in two lines of code The author had to rewrite the project in a different language to add this cross platform multi-threading support. I'm not saying you can do this in two lines of code, the code likely needs some modifications and some considerations, but it's definitely easier to just use one of the multiple cross-platform threading libraries available (or even to write your own) and adapt the code to use it than it is to rewrite a whole project in rust. It's a little bit disingenuous to suggest that in rust any parallel processing task in any project out there can be achieved with 2 lines of code, it's like claiming that OMP can be effectively used by any C codebase with just a 1 line #pragma. If you write your code with the intention of using rayon for your multi-threading tasks then obviously it will not require any modification to make it work with rayon because presumably you had already written your code in a way that makes it usable with rayon. If you write your C code with the intention of using it with your multi-platform threading library of choice then it won't require changes to make it use that library.

steveklabnik 5y ago
> Doesn't this either Cargo pins versions via Cargo.lock for you automatically. It's still not a great idea, (because when you update dependencies it will go to whatever version is the latest, which may not be compatible) and crates.io disallows packages to be uploaded that use this, but it is the easiest built-in way. If you install cargo-edit, you could "cargo add rayon" and it will add it with the latest version specifier instead, which is slightly easier and better but isn't built-in (yet, someday...) > The author had to rewrite the project in a different language to add this cross platform multi-threading support. They're not saying that, they're saying "how do I do what Rayon does for Rust in C?" OpenMP is sorta kinda an answer there, but the post is claiming it's not as good of a solution. "Rewrite it in Rust is easier than using OpenMP" is not what's being said, and you're certainly right that porting to another language entirely is not simple. > If you write your code with the intention of using rayon for your multi-threading tasks then obviously it will not require any modification to make it work with rayon because presumably you had already written your code in a way that makes it usable with rayon. The claim being made is that due to things like the borrow checker, this is often true for Rust code generally, moreso than it is with C and OpenMP. And, in Rust, if it doesn't Just Work, you'll get an error message pointing out exactly why not, whereas (in my understanding, I haven't used OpenMP) that is not going to happen in C.
- Arch-TK 5y ago
  > The claim being made is that due to things like the borrow checker, this is often true for Rust code generally, moreso than it is with C and OpenMP. And, in Rust, if it doesn't Just Work, you'll get an error message pointing out exactly why not, whereas (in my understanding, I haven't used OpenMP) that is not going to happen in C. Firstly, nobody knows what claim is being made here other than the author of the original question. Maybe let's let lovasoa reply. Secondly, no, unless rust is the most rigid language ever invented (which as far as I am aware it's not) there most certainly IS a way to write code in a way where it may need to be modified to work with another library. I highly doubt that literally any random code unit in a random codebase could be made instantly parallel with no code changes other than calling it using rayon, this is just wishful thinking. Even not considering fundamental problems like figuring out how to chunk a workload appropriately for parallelization. The point I was making is that for any given codebase, it's possible to write code in such a way that it can be used with some tool easily such that it only takes two lines to make use of that tool, just like it's possible to not do this and require rewriting things. You can't make judgements on how easy to use a library is on the basis of the fact that someone who rewrote their code specifically to use that library was able to use it by changing two lines of code. With OpenMP it certainly can sometimes be as easy as adding a #pragma omp parallel for somewhere, or it can be much more difficult than that. Nothing stops you from engineering a project to specifically be tailored towards working well with a particular library.
  - pornel 5y ago
    For me it was easier, or at least more pleasant, to rewrite the project in Rust than to deal with C build issues for various platforms. It's not only that one thing I've got stuck with. I've been maintaining this project for years, and there have been endless issues with the build: GNU vs BSD make incompatibilities. grep doesn't support the same flags on all systems. -o /dev/null does not work in msys. Flags for MSVC, GCC, and ICC flags, including odd ones like aligning the stack so it doesn't crash on SSE loads. The arcana of static linking with compiler-private runtime libraries, for multiple compilers. __declspec. Windows charset woes. MSVC<>MinGW incompatibilities. All the combinations of uname -m. All the missing headers. pkg-config sysroots. libpng.pc is sometimes sabotaged to force use of libpng-config. Moving -lm around. Apple clang has SDK includes separate from include paths. macOS system frameworks pulling in duplicate incompatible versions of libraries I'm trying to use. .h and .so ABI mismatches. Why are these my problems? I just want to process some pixels. And regarding Rust parallel code working out of the box, it really does happen. Rust's single-threaded iterators already require a high degree of isolation. The borrow checker already has shared XOR mutable rule for all code everywhere. Mutable globals are already banned. So yeah, it often is as easy as changing iter() to par_iter(). Best of all, when that is not safe, the compiler will show you exactly where and why, even if it's due to dependency of a dependency in some deeply nested function call.