6 ms·

Forking had a ton of its own downsides, it’s not a free lunch either, from poor ergonomics to communications overhead it works well for somethings and very poor

Forking had a ton of its own downsides, it’s not a free lunch either, from poor ergonomics to communications overhead it works well for somethings and very poorly for others.

miraculixx 3y ago
Yes, the same is true for free threading. Yet people assume free threading is free concurrency and that's the problem.
- usrbinbash 3y ago
  Show me someone who actually knows how threads work and what writing threading code entails who assumes that. I am perfectly aware that threads are not free. Just as I am perfectly aware that a context switch between threads is less expensive that switching a new process onto the core, and that IPC requires kernel involvement.
  - commonlisp94 3y ago
    > context switch between threads is less expensive that switching a new process onto the core But the overhead of a context switch for a thread and process is very similar. The main difference is whether memory is shared by default.
    - usrbinbash 3y ago
      > is very similar. Except that the threads share the exact same virtual address space, and processes do not, which makes the thread context switch faster. And that is to say nothing about the setup and teardown process, which for a process involves copy-on-demand'ing the entire memory, but for a thread merely setting up its own stack.
      - commonlisp94 3y ago
        > Except that the threads share the exact same virtual address space, and processes do not, which makes the thread context switch faster That's what I said. But it's really not much. I'm afraid we will need numbers now to continue the conversation. If I measured would you be open to changing your opinion? Or are you committed to this topic, so that it would have no bearing?
        kerkeslager 3y ago
        Well, I don't know how you'd test that, but you should really consider testing the other half of the post you're responding to which you ignored, because that's much easier to test: Spin up and tear down a million pthreads in C, and see how long that takes and how much memory it takes. Then spin up and tear down a million processes in C and see your computer grind to a halt until you kill the process that is starting the processes, if you can even get your computer to do that without power-cycling. It's <50 lines of code for each, so I'm eagerly waiting for your response! Notably, my confidence here comes from the fact that I don't generally get into performance arguments without having actually tested what I'm saying. I've written this code before--it's what I do whenever I'm checking out a new programming language or threading library. Given the complexity of modern computers, nobody really can predict how a program will behave without testing it (except maybe in assembly) there's just too many variables. So you should stop doing that. If you decide to try the same thing in Java (the other language mentioned), probably drop the number of threads/processes down to 100,000, since Java's lightweight threads aren't quite as efficient. 100,000 processes will probably still be enough to crash your computer. I'm sure you can find some language/library which implements threads particularly inefficiently, so let's stick to pthreads/C and avoid that straw man. EDIT: Here ya go, I had ChatGPT write this one for ya: #include <stdio.h> #include <pthread.h> #include <unistd.h> void* threadFunction(void* arg) { // Sleep for 10 seconds sleep(10); pthread_exit(NULL); } int main() { int numThreads = 1000000; pthread_t threads[numThreads]; // Create threads for (int i = 0; i < numThreads; i++) { int result = pthread_create(&threads[i], NULL, threadFunction, NULL); if (result != 0) { printf("Failed to create thread %d\n", i); return 1; } } // Join threads for (int i = 0; i < numThreads; i++) { int result = pthread_join(threads[i], NULL); if (result != 0) { printf("Failed to join thread %d\n", i); return 1; } } return 0; } And... #include <stdio.h> #include <sys/types.h> #include <sys/wait.h> #include <unistd.h> int main() { int numProcesses = 1000000; pid_t childPID; // Create processes for (int i = 0; i < numProcesses; i++) { childPID = fork(); if (childPID < 0) { printf("Failed to create process %d\n", i); return 1; } else if (childPID == 0) { // Child process sleep(10); return 0; } } // Wait for all child processes to finish int status; pid_t pid; while ((pid = wait(&status)) > 0); return 0; } It looks like the latter just crashes the program without taking down my whole machine now, which is an improvement over the last time I tried this with processes.

Forking had a ton of its own downsides, it’s not a free lunch either, from poor ergonomics to communications overhead it works well for somethings and very poorly for others.

miraculixx 3y ago

Yes, the same is true for free threading. Yet people assume free threading is free concurrency and that's the problem.

usrbinbash 3y ago
Show me someone who actually knows how threads work and what writing threading code entails who assumes that. I am perfectly aware that threads are not free. Just as I am perfectly aware that a context switch between threads is less expensive that switching a new process onto the core, and that IPC requires kernel involvement.
- commonlisp94 3y ago
  > context switch between threads is less expensive that switching a new process onto the core But the overhead of a context switch for a thread and process is very similar. The main difference is whether memory is shared by default.
  - usrbinbash 3y ago
    > is very similar. Except that the threads share the exact same virtual address space, and processes do not, which makes the thread context switch faster. And that is to say nothing about the setup and teardown process, which for a process involves copy-on-demand'ing the entire memory, but for a thread merely setting up its own stack.
    - commonlisp94 3y ago
      > Except that the threads share the exact same virtual address space, and processes do not, which makes the thread context switch faster That's what I said. But it's really not much. I'm afraid we will need numbers now to continue the conversation. If I measured would you be open to changing your opinion? Or are you committed to this topic, so that it would have no bearing?
      - kerkeslager 3y ago
        Well, I don't know how you'd test that, but you should really consider testing the other half of the post you're responding to which you ignored, because that's much easier to test: Spin up and tear down a million pthreads in C, and see how long that takes and how much memory it takes. Then spin up and tear down a million processes in C and see your computer grind to a halt until you kill the process that is starting the processes, if you can even get your computer to do that without power-cycling. It's <50 lines of code for each, so I'm eagerly waiting for your response! Notably, my confidence here comes from the fact that I don't generally get into performance arguments without having actually tested what I'm saying. I've written this code before--it's what I do whenever I'm checking out a new programming language or threading library. Given the complexity of modern computers, nobody really can predict how a program will behave without testing it (except maybe in assembly) there's just too many variables. So you should stop doing that. If you decide to try the same thing in Java (the other language mentioned), probably drop the number of threads/processes down to 100,000, since Java's lightweight threads aren't quite as efficient. 100,000 processes will probably still be enough to crash your computer. I'm sure you can find some language/library which implements threads particularly inefficiently, so let's stick to pthreads/C and avoid that straw man. EDIT: Here ya go, I had ChatGPT write this one for ya: #include <stdio.h> #include <pthread.h> #include <unistd.h> void* threadFunction(void* arg) { // Sleep for 10 seconds sleep(10); pthread_exit(NULL); } int main() { int numThreads = 1000000; pthread_t threads[numThreads]; // Create threads for (int i = 0; i < numThreads; i++) { int result = pthread_create(&threads[i], NULL, threadFunction, NULL); if (result != 0) { printf("Failed to create thread %d\n", i); return 1; } } // Join threads for (int i = 0; i < numThreads; i++) { int result = pthread_join(threads[i], NULL); if (result != 0) { printf("Failed to join thread %d\n", i); return 1; } } return 0; } And... #include <stdio.h> #include <sys/types.h> #include <sys/wait.h> #include <unistd.h> int main() { int numProcesses = 1000000; pid_t childPID; // Create processes for (int i = 0; i < numProcesses; i++) { childPID = fork(); if (childPID < 0) { printf("Failed to create process %d\n", i); return 1; } else if (childPID == 0) { // Child process sleep(10); return 0; } } // Wait for all child processes to finish int status; pid_t pid; while ((pid = wait(&status)) > 0); return 0; } It looks like the latter just crashes the program without taking down my whole machine now, which is an improvement over the last time I tried this with processes.