3 ms·

> it seems like C#, Java and Go all have at least one simple and safe method to achieve this. Its only simple after you've studied double-checked locking. Init

> it seems like C#, Java and Go all have at least one simple and safe method to achieve this. Its only simple after you've studied double-checked locking. Initial attempts often lead to failure. > If you depend on global state in the parent process, how can the subprocesses even operate, since they do not have access to that state? Plenty of ways to get access. The #1 way is probably to use a database to share that state in a concurrency-safe way. sqlite3 works, though postgresql is more scalable. There are also solutions that require less resources: flock() a file and then read the shared state in a manner that's cohesive across processes. If your process dies while flock()ing something, the flock() automatically undoes (unlike mutexes where if pthread_cancelled() you could very well have a permanently locked mutex). That's why so many systems have a database + dedicated process that handles this kind of shared global state between processes. > Yes, certain resources, such as loggers, need to be global, but these should really be thread-safe anyway. Loggers are an excellent example of where opening up a pipe or socket to syslogd is far easier than trying to shoehorn in a mutex+queue across threads. > Yeah, it is easy to kill a subprocess. But let's consider the opposite: what happens if the parent process dies? How do you make sure that a long running subprocess automatically terminates? It is definitely not trivial. If the parent of a process group is terminated, SIGHUP is sent to its children. Catch the signal then terminate. So once again: processes handle both situations (parent dies, kill children. Or children die, notify parent), with SIGHUP and SIGCHLD respectively. No such signaling exists in pthread_blah world. You're (trying to) argue about the "superiority" of threads when processes have all of these issues 100% figured out, while the pthread-world is completely ignorant to these issues.

spacechild1 4y ago
> The #1 way is probably to use a database to share that state in a concurrency-safe way. sqlite3 works, though postgresql is more scalable. > There are also solutions that require less resources: flock() a file and then read the shared state in a manner that's cohesive across processes. Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-) > (unlike mutexes where if pthread_cancelled() you could very well have a permanently locked mutex). Again, there is almost never a good reason to use pthread_cancel() in the first place. > opening up a pipe or socket to syslogd In my projects, logging means "print to stderr or write to a file" :-) > If the parent of a process group is terminated, SIGHUP is sent to its children. Catch the signal then terminate. So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows? > No such signaling exists in pthread_blah world This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChannel/#show_close https://openframeworks.cc/documentation/utils/ofThreadChanne.... Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions.
- dragontamer 4y ago
  > Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-) When it comes to understanding locks, concurrency, and parallelism with shared data between threads... yeah. In my experience, the database is heavy lifting but absolutely ensures that most issues are taken care of. But as I stated earlier: lighter weight solutions, such as flock() exist for a reason. If the database is too heavy (note: sqlite3 is extremely lightweight, so I bet it works for most cases), then flock() a file and read/writing to it works too. What flock() gets you is that 100% certainty about cleanups upon strange exit cases that threads do not get you. A flock() always cleans itself up on process termination. No guarantees about mutexes (or other issues) on thread-cancellations or other thread-related issues. (I dunno, oom killer) > Again, there is almost never a good reason to use pthread_cancel() in the first place. On the contrary. The pthread community knows that pthread_cancel() is poorly behaved and constantly tells beginner programmers not to use it. "There's no good reason" because everyone knows that the number of traps in using that function are legion. Its never worthwhile to use that function because it just leads to severely buggy behavior in practice. > So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows? ... you know that Win32 doesn't support pthreads, right? And C++ std::thread doesn't support anything that we've talked about either. To answer your question: Win32 job objects. Every reasonable modern OS supports the concept of sessions (Windows just calls them job objects instead). The OS-level (be it session leaders / session groups in Linux, or Job objects in Windows) is the correct solution to this problem. EDIT: Found the function name for ya: https://learn.microsoft.com/en-us/windows/win32/api/jobapi2/nf-jobapi2-terminatejobobject https://learn.microsoft.com/en-us/windows/win32/api/jobapi2/... > This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChanne https://openframeworks.cc/documentation/utils/ofThreadChanne.... > Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions. And you damn well know that under a thread-kill or thread-cancel scenario, this code stops functioning. While all the code I talked above will function correctly even in the worst-case "kill -9" SIGKILL. Whatever underlying synchronization that channel is using to synchronize thread access will stop working if a pthread_mutex_lock() is called, but its corresponding pthread_mutex_unlock() fails to be called due to cancellation or other issue.

spacechild1 4y ago

> The #1 way is probably to use a database to share that state in a concurrency-safe way. sqlite3 works, though postgresql is more scalable. > There are also solutions that require less resources: flock() a file and then read the shared state in a manner that's cohesive across processes. Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-) > (unlike mutexes where if pthread_cancelled() you could very well have a permanently locked mutex). Again, there is almost never a good reason to use pthread_cancel() in the first place. > opening up a pipe or socket to syslogd In my projects, logging means "print to stderr or write to a file" :-) > If the parent of a process group is terminated, SIGHUP is sent to its children. Catch the signal then terminate. So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows? > No such signaling exists in pthread_blah world This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChannel/#show_close https://openframeworks.cc/documentation/utils/ofThreadChanne.... Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions.

dragontamer 4y ago
> Wow. And that is somehow easier than using, say, a concurrent collection? (I have never used a database in my life, so we obviously come from very different angles :-) When it comes to understanding locks, concurrency, and parallelism with shared data between threads... yeah. In my experience, the database is heavy lifting but absolutely ensures that most issues are taken care of. But as I stated earlier: lighter weight solutions, such as flock() exist for a reason. If the database is too heavy (note: sqlite3 is extremely lightweight, so I bet it works for most cases), then flock() a file and read/writing to it works too. What flock() gets you is that 100% certainty about cleanups upon strange exit cases that threads do not get you. A flock() always cleans itself up on process termination. No guarantees about mutexes (or other issues) on thread-cancellations or other thread-related issues. (I dunno, oom killer) > Again, there is almost never a good reason to use pthread_cancel() in the first place. On the contrary. The pthread community knows that pthread_cancel() is poorly behaved and constantly tells beginner programmers not to use it. "There's no good reason" because everyone knows that the number of traps in using that function are legion. Its never worthwhile to use that function because it just leads to severely buggy behavior in practice. > So you need to make a process group... What if your code should be cross platform? Do you know how to do this on Windows? ... you know that Win32 doesn't support pthreads, right? And C++ std::thread doesn't support anything that we've talked about either. To answer your question: Win32 job objects. Every reasonable modern OS supports the concept of sessions (Windows just calls them job objects instead). The OS-level (be it session leaders / session groups in Linux, or Job objects in Windows) is the correct solution to this problem. EDIT: Found the function name for ya: https://learn.microsoft.com/en-us/windows/win32/api/jobapi2/nf-jobapi2-terminatejobobject https://learn.microsoft.com/en-us/windows/win32/api/jobapi2/... > This kind of signaling does not exist because it is not necessary. Tasks either periodically check a boolean flag or get notified via the queue itself. Here is a random simple example: https://openframeworks.cc/documentation/utils/ofThreadChanne https://openframeworks.cc/documentation/utils/ofThreadChanne.... > Also, I'm not sure why you keep talking about pthreads... Modern programming languages have their own (portable) threading abstractions. And you damn well know that under a thread-kill or thread-cancel scenario, this code stops functioning. While all the code I talked above will function correctly even in the worst-case "kill -9" SIGKILL. Whatever underlying synchronization that channel is using to synchronize thread access will stop working if a pthread_mutex_lock() is called, but its corresponding pthread_mutex_unlock() fails to be called due to cancellation or other issue.