3 ms·

What I don't understand is that neither exceptions nor segfaults are handled gracefully when using multiprocessing. The former do not have the full traceback, w

What I don't understand is that neither exceptions nor segfaults are handled gracefully when using multiprocessing. The former do not have the full traceback, while the latter aren't even detected and cause a deadlock. This requires running the code again in single processing mode to figure out where the bug is. The new concurrent.futures module has the same problem.

nine_k 13y ago
If you follow the monadic route and return a Maybe... Stop. Let me reformulate. If you follow the standard AJAX route and return either a {'success': True, 'payload': ...} or {'success': False, 'error': ...} from your workers, your error handling will be fine.
- andreasvc 13y ago
  That would not give me a traceback, and it would not catch segfaults or ctrl-C. It's a different style of programming that would work around part of the issue (only the exceptions, and only by checking for errors manually), whereas what perplexes me is that multiprocessing and concurrent.futures are not designed to handle these error conditions themselves.
  - nine_k 13y ago
    Why, it would give you a traceback if your worker process collects it, as a reasonable worker would. Since workers are probably non-interactive, they can't exactly receive a Ctrl+C. The dispatcher process, OTOH, will. I suppose a pool should kill all processes on SystemExit. When you think about more details, you grow more and more sophisticated solution that best suits your needs. But the point of the post is different: it's a dead simple way to start. Starting is often the hardest part.
    - andreasvc 13y ago
      Collecting tracebacks is the sort of plumbing you would expect a 'batteries included' library to do. I didn't expect the workers to receive the ctrl-C, I'd expect the main process to terminate, along with its children; none of that happens. I think it's clear that these sorts of details belong in the library. Everybody could hack together their own traceback collector, segfault handlers etc., but it's difficult to get right and in a scripting language you should be able to forget about such things.
  - wc- 13y ago
    Check out the combo of sys.exc_info and traceback.format_tb, I've used these two functions to generate stack traces for failed worker jobs in the past with pretty good success. edit: I guess I didn't really answer the segfault issue. Beyond checking the return code of the child process to see if it segfault'ed (so you can log it or retry it) I'm not sure of how you could get more informative failing info.
    - andreasvc 13y ago
      Indeed, you can pass the traceback manually as a string. With the segfaults you can use the 'faulthandler' module to dump information about them, and I see that starting with Python 3.3 they correctly detect child processes that have died: http://hg.python.org/cpython/rev/6d6099f7fe89 http://hg.python.org/cpython/rev/6d6099f7fe89

nine_k 13y ago

If you follow the monadic route and return a Maybe... Stop. Let me reformulate. If you follow the standard AJAX route and return either a {'success': True, 'payload': ...} or {'success': False, 'error': ...} from your workers, your error handling will be fine.

andreasvc 13y ago
That would not give me a traceback, and it would not catch segfaults or ctrl-C. It's a different style of programming that would work around part of the issue (only the exceptions, and only by checking for errors manually), whereas what perplexes me is that multiprocessing and concurrent.futures are not designed to handle these error conditions themselves.
- nine_k 13y ago
  Why, it would give you a traceback if your worker process collects it, as a reasonable worker would. Since workers are probably non-interactive, they can't exactly receive a Ctrl+C. The dispatcher process, OTOH, will. I suppose a pool should kill all processes on SystemExit. When you think about more details, you grow more and more sophisticated solution that best suits your needs. But the point of the post is different: it's a dead simple way to start. Starting is often the hardest part.
  - andreasvc 13y ago
    Collecting tracebacks is the sort of plumbing you would expect a 'batteries included' library to do. I didn't expect the workers to receive the ctrl-C, I'd expect the main process to terminate, along with its children; none of that happens. I think it's clear that these sorts of details belong in the library. Everybody could hack together their own traceback collector, segfault handlers etc., but it's difficult to get right and in a scripting language you should be able to forget about such things.
- wc- 13y ago
  Check out the combo of sys.exc_info and traceback.format_tb, I've used these two functions to generate stack traces for failed worker jobs in the past with pretty good success. edit: I guess I didn't really answer the segfault issue. Beyond checking the return code of the child process to see if it segfault'ed (so you can log it or retry it) I'm not sure of how you could get more informative failing info.
  - andreasvc 13y ago
    Indeed, you can pass the traceback manually as a string. With the segfaults you can use the 'faulthandler' module to dump information about them, and I see that starting with Python 3.3 they correctly detect child processes that have died: http://hg.python.org/cpython/rev/6d6099f7fe89 http://hg.python.org/cpython/rev/6d6099f7fe89