4 ms·

I completely disagree. The only part of this that seems dangerous is the fact that it can turn a single-threaded program into a multi-threaded one, and the only

by eridius 10y ago

I completely disagree. The only part of this that seems dangerous is the fact that it can turn a single-threaded program into a multi-threaded one, and the only real problem there is it makes fork-without-exec unsafe. But fork-without-exec is basically unsafe anyway unless you can absolutely guarantee that nothing has ever spawned a thread. Meanwhile, preventing libraries from using threading under the covers basically means throwing away lot of performance gains for no good reason.

adrianratnapala 10y ago
This is the very first time I have heard about fork-without-exec as dangerous, but it is not the first time I have heard about having to be careful about libraries and hidden threads. The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction.
- eridius 10y ago
  > The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction. That's not true at all. Threads aren't that complicated, and there's lots of ways of using them that aren't complicated and don't leak anything. As a trivial example, if I want to process a bunch of data, it might be faster to process it in parallel, and I can use a thread to do that without causing any observable difference in behavior (beyond potentially turning a single-threaded program into a multithreaded one). And if I'm on a platform like OS X or iOS, I can use higher-level threading APIs like libdispatch that make it even simpler.
  - adrianratnapala 10y ago
    This is exaclty the kind of reasoning that makes library writers stumpble into using abstractions that turn out leaky. The whole point of the OP is to show one (of many) machanisms through which it does cause an observable change in behaviour. And your example of a big calculation in a library is particurly bad, because the user can threadify that herself if she wants to. #include <thread.h> #include <yourlib.h> #include "my_eventloop.h" void launch_da_ting() { std::thread ting_doer([]{ Result rez = yourlib::do_da_ting(); my_event_loop::post([rez](){get_rez(rez);}); }); ting_doer.detach(); } Now it's true those five lines would become a single function call if your library forced the user to user a particular dispatching library. But I think that would be poor seperation of concerns.
    - eridius 10y ago
      It's leaky in one extremely minor way that affects almost no programs whatsoever. It's extremely rare for anyone to do fork-without-exec, and anyone doing that already has a large number of restrictions they must be aware of. And no, the user can't threadify that themselves. Your sample code is putting the library call on a thread. That's not at all what I was talking about. I was talking about a library using multiple threads (or, more likely, some abstraction over a thread pool that handles scheduling chunks of work on threads for you, such as libdispatch on Apple platforms) in order to process a bunch of data in parallel, data that only the library knows about. The user makes a single call into the library, the library processes a bunch of data in parallel, and then returns to the user. From the user's perspective it's a synchronous call, but it runs in a fraction of the time it would if the library wasn't allowed to use threading internally.
chubot 10y ago
You can have your cake and eat it too -- there's no throwing away of performance gains. Libraries can use threads, but not directly or "silently". They should take threads as parameters. Then the application knows when they're being used and the application programmer can reason about it.
- amelius 10y ago
  Yes, in theory. In practice, every library developer/maintainer uses different guidelines.
  - hughperkins 10y ago
    In practice, a lot of library developers/maintainers read this list, or know someone who does.
- gpderetta 10y ago
  compilers and runtimes can spawn threads implicitly when required (think of openmp, cilk+, or even autopar). Are you saying that libraries shouldn't use these facilities?
  - lmm 10y ago
    If you're spawning threads implicitly then you're no longer a library, you're a framework. Sometimes a framework is a good fit, but frameworks don't compose (if you try to use two frameworks in the same application you're gonna have a bad time) so there is a cost. But in any case, better a framework that's explicitly a framework than a "library" which can't actually be used freely in general-purpose code.
    - gpderetta 10y ago
      would you consider a parallelized BLAS a framework?
      - 10y ago

by eridius 10y ago

adrianratnapala 10y ago

This is the very first time I have heard about fork-without-exec as dangerous, but it is not the first time I have heard about having to be careful about libraries and hidden threads. The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction.

eridius 10y ago
> The thing is that any thread you launch is a big piece of machinery and your attempt to hide it is inevitably going to be a leaky abstraction. That's not true at all. Threads aren't that complicated, and there's lots of ways of using them that aren't complicated and don't leak anything. As a trivial example, if I want to process a bunch of data, it might be faster to process it in parallel, and I can use a thread to do that without causing any observable difference in behavior (beyond potentially turning a single-threaded program into a multithreaded one). And if I'm on a platform like OS X or iOS, I can use higher-level threading APIs like libdispatch that make it even simpler.
- adrianratnapala 10y ago
  This is exaclty the kind of reasoning that makes library writers stumpble into using abstractions that turn out leaky. The whole point of the OP is to show one (of many) machanisms through which it does cause an observable change in behaviour. And your example of a big calculation in a library is particurly bad, because the user can threadify that herself if she wants to. #include <thread.h> #include <yourlib.h> #include "my_eventloop.h" void launch_da_ting() { std::thread ting_doer([]{ Result rez = yourlib::do_da_ting(); my_event_loop::post([rez](){get_rez(rez);}); }); ting_doer.detach(); } Now it's true those five lines would become a single function call if your library forced the user to user a particular dispatching library. But I think that would be poor seperation of concerns.
  - eridius 10y ago
    It's leaky in one extremely minor way that affects almost no programs whatsoever. It's extremely rare for anyone to do fork-without-exec, and anyone doing that already has a large number of restrictions they must be aware of. And no, the user can't threadify that themselves. Your sample code is putting the library call on a thread. That's not at all what I was talking about. I was talking about a library using multiple threads (or, more likely, some abstraction over a thread pool that handles scheduling chunks of work on threads for you, such as libdispatch on Apple platforms) in order to process a bunch of data in parallel, data that only the library knows about. The user makes a single call into the library, the library processes a bunch of data in parallel, and then returns to the user. From the user's perspective it's a synchronous call, but it runs in a fraction of the time it would if the library wasn't allowed to use threading internally.

chubot 10y ago

You can have your cake and eat it too -- there's no throwing away of performance gains. Libraries can use threads, but not directly or "silently". They should take threads as parameters. Then the application knows when they're being used and the application programmer can reason about it.

amelius 10y ago
Yes, in theory. In practice, every library developer/maintainer uses different guidelines.
- hughperkins 10y ago
  In practice, a lot of library developers/maintainers read this list, or know someone who does.
gpderetta 10y ago
compilers and runtimes can spawn threads implicitly when required (think of openmp, cilk+, or even autopar). Are you saying that libraries shouldn't use these facilities?
- lmm 10y ago
  If you're spawning threads implicitly then you're no longer a library, you're a framework. Sometimes a framework is a good fit, but frameworks don't compose (if you try to use two frameworks in the same application you're gonna have a bad time) so there is a cost. But in any case, better a framework that's explicitly a framework than a "library" which can't actually be used freely in general-purpose code.
  - gpderetta 10y ago
    would you consider a parallelized BLAS a framework?
    - 10y ago