7 ms·
In my opinion, it's libraries that secretly use threads under the covers (but still access state shared with the rest of the program like the malloc heap) that
by caf 10y ago
In my opinion, it's libraries that secretly use threads under the covers (but still access state shared with the rest of the program like the malloc heap) that are what's dangerous.
- Johnny_Brahms 10y agoguh. Those bugs aren't fun to hunt down. I don't think you really understand the importance of avoiding global or semi-global states until you have to do concurrent stuff, or when you have to start being serious about tests.
- pcwalton 10y agoI dunno--under that definition every concurrent GC is "dangerous".
- chubot 10y agoWhat libraries need to do GC? That is a language issue and not a library issue. When you are embedding a language like Go in a C program, its runtime becomes a library. But that's the example the proves the rule: there are (or were) well-known problems with that. Libraries shouldn't be doing GC.
- dozzie 10y ago> What libraries need to do GC? That is a language issue and not a library issue. Boehm GC disagrees.
- chubot 10y agoYes. Libraries should provide algorithms and data structures, and parameterize a threading interface if necessary (though I think it's usually not necessary). Applications can then instantiate their own threads and pass them to the library (i.e. dependency injection.) So if you have 10 different concurrent libraries in an application, then you don't end up with 10 different threading paradigms or 10 different thread pools. This is a matter of application architecture -- libraries shouldn't bake in a specific policy.
- TickleSteve 10y agotrue... ...this applies not just to threading, but other things that are not in the domain of the library such as memory allocation strategy or communication mechanism. Libraries should be small, focused and decoupled. This is the way to proper re-usability.
- amag 10y agoLike left-pad?
- TickleSteve 10y agojeez, no.... extremes of everything are terrible. I suppose another rule should be that there libraries should encapsulate a sufficient amount of complexity to make their existence worthwhile.... unlike left-pad.
- gnaritas 10y agoThat's a function deployed as a library, not a real library; the word library implies many functions.
- prodigal_erik 10y agoThere's nothing really wrong with packaging small isolated pieces of code. The debacle came from consumers' bad habit of randomly upgrading their dependencies without doing full regression testing (partly because the tooling encourages this terrible practice).
- djsumdog 10y agoI know it's very different in JVM land, but many Akka libraries do something similar where they take am actor system, allowing the developer to share or separate the actor systems.
- rlpb 10y ago> Threads are inherently the right way to do concurrency for more complex tasks. True, but they're more often used as a crutch where a select-style loop and/or state machine would be more suitable, leaving race conditions in their wake.
- eridius 10y agoI completely disagree. The only part of this that seems dangerous is the fact that it can turn a single-threaded program into a multi-threaded one, and the only real problem there is it makes fork-without-exec unsafe. But fork-without-exec is basically unsafe anyway unless you can absolutely guarantee that nothing has ever spawned a thread. Meanwhile, preventing libraries from using threading under the covers basically means throwing away lot of performance gains for no good reason.
- adrianratnapala 10y agoThis is the very first time I have heard about fork-without-exec as dangerous, but it is not the first time I have heard about having to be careful about libraries and hidden threads. The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction.
- eridius 10y ago> The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction. That's not true at all. Threads aren't that complicated, and there's lots of ways of using them that aren't complicated and don't leak anything. As a trivial example, if I want to process a bunch of data, it might be faster to process it in parallel, and I can use a thread to do that without causing any observable difference in behavior (beyond potentially turning a single-threaded program into a multithreaded one). And if I'm on a platform like OS X or iOS, I can use higher-level threading APIs like libdispatch that make it even simpler.
- adrianratnapala 10y agoThis is exaclty the kind of reasoning that makes library writers stumpble into using abstractions that turn out leaky. The whole point of the OP is to show one (of many) machanisms through which it does cause an observable change in behaviour. And your example of a big calculation in a library is particurly bad, because the user can threadify that herself if she wants to. #include <thread.h> #include <yourlib.h> #include "my_eventloop.h" void launch_da_ting() { std::thread ting_doer([]{ Result rez = yourlib::do_da_ting(); my_event_loop::post([rez](){get_rez(rez);}); }); ting_doer.detach(); } Now it's true those five lines would become a single function call if your library forced the user to user a particular dispatching library. But I think that would be poor seperation of concerns.
- pjmlp 10y agoWhat is dangerous is that the interactions between threads and processes aren't part of POSIX standard and its behavior is OS specific across UNIX implementations. This isn't an issue on non-UNIX OSes.
- wruza 10y agoHow many non-unixes do have fork_without_exec()?
- pjmlp 10y agoMost non-UNIX OSes only have exec(), not fork(). Hence why fork() is just kind-of implemented on their POSIX compatibility layers. In UNIX it is only an issue, because IEEE doesn't want to set in stone in POSIX how threads, signals and processes should interact, due to the way threads were added to UNIX.
- cperciva 10y agoNon-unix systems don't have exec; they have spawn.
- the_mitsuhiko 10y agoWindows has exec.
- wruza 10y agoExec is substitution of current process with new image without closing fds or changing pid. Which winapi function does that?
- the_mitsuhiko 10y ago_wexec and friends. It internally uses CreateProcess obviously.
- 10y ago
- the_mitsuhiko 10y agoLibraries should not be disallowed from spawning threads. Threads are inherently the right way to do concurrency for more complex tasks. fork() is a tremendously complex call and has huge implications on all things in the OS. For many years now I have the rule that new code must not use fork() and I am living a happier life. It's a bad call and it has no use if you have good threading support in the language. There are many programming languages that do not support fork() andy more or heavily restrict the use.
- DSMan195276 10y agoI would add, in many/most cases using fork() without calling exec() just doesn't make sense. Even at the system call level you have clone(), which is how actual threading is implemented under the covers. And really, if you need a thread, just get a thread. I see extremely few or no advantages to doing fork() instead of spawning a new thread normally using whatever interface is provided (except in the case of exec() where fork() or vfork() is necessary).