4 ms·

No concurrency? asyncio is great for I/O bound network stuff!

• colonwqbang 6y ago
  "No parallelism" is probably what was meant.
  • dragonwriter 6y ago
    > "No parallelism" is probably what was meant. Which is still wrong, of course, but "no in-process (or in-single-runtime-instance) parallelism" would be correct, as would "forking inconvenient parallelism".
    • colonwqbang 6y ago
      Posix fork() doesn't really count, if that's what you mean...
      • dragonwriter 6y ago
        Why doesn't Python's multiprocessing module (which uses fork by default on Unix) count? It literally exists for parallelism.
        • int_19h 6y ago
          It doesn't use fork() on macOS anymore, because some of Apple's own APIs get broken by its use. Pretty much any app that uses both fork and threads, has to jump through many hoops to make the two work together well. And this applies to all the libraries that it uses, directly or indirectly - if any library spawns a thread and does some locking in it, you get all kinds of hard-to-debug deadlocks if you try to fork. So unless you have very good perf reasons to need fork, I would strongly recommend multiprocessing.set_start_method("spawn") on all platforms. No obscure bugs, and it'll also behave the same everywhere, so things will be more portable. Code using multiprocessing that's written to rely on fork semantics can be very difficult to port later.
          • guenthert 6y ago
            You wouldn't fork() for performance, but for security reasons.
        • deleted 6y ago
          [deleted]
        • colonwqbang 6y ago
          It's understood that you can have "parallelism" by running two copies of your program using basic system facilities like fork(), or even by buying several computers and running one instance of your program on each of them. That's not what is meant by a language "supporting parallelism". If it was, then every language ever designed supports parallelism and so the term is meaningless. To claim that a language "supports parallelism", it has to do something more to facilitate parallel programming. I would say that parallel threads of computation with shared memory and system resources is the bare minimum. You can go the extra mile and support transactional memory or other "nice" abstractions which make parallel programming easier. Saying that Python support parallelism because it has a fork() wrapper is like saying that Posix shell is a strongly typed language because it has strings and string is a type.
    • FartyMcFarter 6y ago
      It's not wrong. If running two processes counts as parallelism, then everything does parallelism, and it becomes pointless to talk about it.
      • xapata 6y ago
        Then one should talk about how convenient the related abstractions are. I like the concurrent.futures library.
        • colonwqbang 6y ago
          Concurrency is not the same as parallelism. Python has good concurrency support, I agree. Python (C Python) does not support parallelism however due to its Big Interpreter Lock which actively prevents any parallelism in Python code. This was probably a conscious design decision on the part of C Python implementers and perhaps a good one. But we should not claim that Python is something which (actively and by design) it's not.
          • xapata 6y ago
            I use `concurrent.futures.ProcessPoolExecutor` fairly often. I handle inter-process communication usually through a database or message queue, expecting that someday I'll want to go clustered instead of just single-machine multiprocessing. I've been burned by implementing multithreading and then needing to overhaul to clustered enough times to stop doing it.