8 ms·

> Instead, many reach for multiprocessing, but spawning processes is expensive Agreed. > and communicating across processes often requires making expensive co

• crabbone 1y ago
  [flagged]
• ogrisel 1y ago
  You cannot share arbitrarily structured objects in the `ShareableList`, only atomic scalars and bytes / strings. If you want to share structured Python objects between instances, you have to pay the cost of `pickle.dump/pickle.dump` (CPU overhead for interprocess communication) + the memory cost of replicated objects in the processes.
  • tomrod 1y ago
    I can fit a lot of json into bytes/strings though?
    • cjbgkagh 1y ago
      Perhaps flatbuffers would be better?
      • tomrod 1y ago
        I love learning from folks on HN -- thanks! Will check it out.
        • notpushkin 1y ago
          Take a look at https://capnproto.org/ https://capnproto.org/ as well, while at it. Neither solve the copying problem, though.
          • deleted 1y ago
            [deleted]
          • frollogaston 1y ago
            Ah, I forgot capnproto doesn't let you edit a serialized proto in-memory, it's read-only. In theory this should be possible as long as you're not changing the length of anything, but I'm not surprised such trickery is unsupported. So this doesn't seem like a versatile solution for sharing data structs between two Python processes. You're gonna have to reserialize the whole thing if one side wants to edit, which is basically copying.
      • tinix 1y ago
        let me introduce you to quickle.
    • vlovich123 1y ago
      That’s even worse than pickle.
      • tomrod 1y ago
        pickle pickles to pickle binary, yeah? So can stream that too with an io Buffer :D
    • frollogaston 1y ago
      If all your state is already json-serializable, yeah. But that's just as expensive as copying if not more, hence what cjbgkagh said about flatbuffers.
      • frollogaston 1y ago
        oh nvm, that doesn't solve this either
    • reliabilityguy 1y ago
      What’s the point? The whole idea is to share an object, and not to serialize them whether it’s json, pickle, or whatever.
      • tomrod 1y ago
        I mean, the answer to this is pretty straightforward -- because we can, not because we should :)
  • notpushkin 1y ago
    We need a dataclass-like interface on top of a ShareableList.
    • notpushkin 1y ago
      Actually, ShareableList feels like a tuple really (as it’s impossible to change its length). If we could mix ShareableList and collections.namedtuple together, it would get us 90% there (99.9% if we use typing.NamedTuple). Unfortunately, I can’t decipher either one [1, 2] from the first glance – maybe if I get some more sleep? [1]: https://github.com/python/cpython/blob/3.13/Lib/collections/__init__.py#L358 https://github.com/python/cpython/blob/3.13/Lib/collections/... [2]: https://github.com/python/cpython/blob/3.13/Lib/typing.py#L2979 https://github.com/python/cpython/blob/3.13/Lib/typing.py#L2...
  • sgarland 1y ago
    So don’t do that? Send data to workers as primitives, and have a separate process that reads the results and serializes it into whatever form you want.
• modeless 1y ago
  Yeah I've had great success sharing numpy arrays this way. Explicit sharing is not a huge burden, especially when compared with the difficulty of debugging problems that occur when you accidentally share things between threads. People vastly overstate the benefit of threads over multiprocessing and I don't look forward to all the random segfaults I'm going to have to debug after people start routinely disabling the GIL in a library ecosystem that isn't ready. I wonder why people never complained so much about JavaScript not having shared-everything threading. Maybe because JavaScript is so much faster that you don't have to reach for it as much. I wish more effort was put into baseline performance for Python.
  • dhruvrajvanshi 1y ago
    > I wonder why people never complained so much about JavaScript not having shared-everything threading. Maybe because JavaScript is so much faster that you don't have to reach for it as much. I wish more effort was put into baseline performance for Python. This is a fair observation. I think a part of the problem is that the things that make GIL less python hard are also the things that make faster baseline performance hard. I.e. an over reliance of the ecosystem on the shape of the CPython data structures. What makes python different is that a large percentage of python code isn't python, but C code targeting the CPython api. This isn't true for a lot of other interpreted languages.
  • com2kid 1y ago
    > I wonder why people never complained so much about JavaScript not having shared-everything threading. Maybe because JavaScript is so much faster that you don't have to reach for it as much. I wish more effort was put into baseline performance for Python. Nobody sane tries to do math in JS. Backend JS is recommended for situations where processing is minimal and it is mostly lots of tiny IO requests that need to be shunted around. I'm a huge JS/Node proponent and if someone says they need to write a backend service that crunches a lot of numbers, I'll recommend choosing a different technology! For some reason Python peeps keep trying to do actual computations in Python...
    • frollogaston 1y ago
• chubot 1y ago
  Spawning processes generally takes much less than 1 ms on Unix Spawning a PYTHON interpreter process might take 30 ms to 300 ms before you get to main(), depending on the number of imports It's 1 to 2 orders of magnitude difference, so it's worth being precise This is a fallacy with say CGI. A CGI in C, Rust, or Go works perfectly well. e.g. sqlite.org runs with a process PER REQUEST - https://news.ycombinator.com/item?id=3036124 https://news.ycombinator.com/item?id=3036124
  • Sharlin 1y ago
    Unix is not the only platform though (and is process creation fast on all Unices or just Linux?) The point about interpreter init overhead is, of course, apt.
    • btilly 1y ago
      Process creation should be fast on all Unices. If it isn't, then the lowly shell script (heavily used in Unix) is going to perform very poorly.
      • kragen 1y ago
        While I think you've been using Unix longer than I have, shell scripts are known for performing very poorly, and on PDP-11 Unix (where perhaps shell scripts were most heavily used, since Perl didn't exist yet) fork() couldn't even do copy-on-write; it had to literally copy the process's entire data segment, which in most cases also contained a copy of its code. Moving to paged machines like the VAX and especially the 68000 family made it possible to use copy-on-write, but historically speaking, Linux has often been an order of magnitude faster than most other Unices at fork(). However, I think people mostly don't use those Unices anymore. I imagine the BSDs have pretty much caught up by now. https://news.ycombinator.com/item?id=44009754 https://news.ycombinator.com/item?id=44009754 gives some concrete details on fork() speed on current Linux: 50μs for a small process, 700μs for a regular process, 1300μs for a venti Python interpreter process, 30000–50000μs for Python interpreter creation. This is on a CPU of about 10 billion instructions per second per core, so forking costs on the order of ½–10 million instructions.
• pyronik19 1y ago
  [dead]
• isignal 1y ago
  Processes can die independently so the state of a concurrent shared memory data structure when a process dies while modifying this under a lock can be difficult to manage. Postgres which uses shared memory data structures can sometimes need to kill all its backend processes because it cannot fully recover from such a state. In contrast, no one thinks about what happens if a thread dies independently because the failure mode is joint.
  • wongarsu 1y ago
    > In contrast, no one thinks about what happens if a thread dies independently because the failure mode is joint. In Rust if a thread holding a mutex dies the mutex becomes poisoned, and trying to acquire it leads to an error that has to be handled. As a consequence every rust developer that touches a mutex has to think about that failure mode. Even if in 95% of cases the best answer is "let's exit when that happens". The operating system tends to treat your whole process as one and shot down everything or nothing. But a thread can still crash in its own due to unhandled oom, assertion failures or any number of other issues
    • jcalvinowens 1y ago
      > But a thread can still crash in its own due to unhandled oom, assertion failures or any number of other issues That's not really true on POSIX. Unless you're doing nutty things with clone(), or you actually have explicit code that calls pthread_exit() or gettid()/pthread_kill(), the whole process is always going to die at the same time. POSIX signal dispositions are process-wide, the only way e.g. SIGSEGV kills a single thread is if you write an explicit handler which actually does that by hand. Unhandled exceptions usually SIGABRT, which works the same way. ** Just to expand a bit: there is a subtlety in that, while dispositions are process-wide, one individual thread does indeed take the signal. If the signal is handled, only that thread sees -EINTR from a blocking syscall; but if the signal is not handled, the default disposition affects all threads in the process simultaneously no matter which thread is actually signalled.
      • wahern 1y ago
• tinix 1y ago
  shared memory only works on dedicated hardware. if you're running in something like AWS fargate, there is no shared memory. have to use the network and file system which adds a lot of latency, way more than spawning a process. copying processes through fork is a whole different problem. green threads and an actor model will get you much further in my experience.
  • bradleybuda 1y ago
    Fargate is just a container runtime. You can fork processes and share memory like you can in any other Linux environment. You may not want to (because you are running many cheap / small containers) but if your Fargate containers are running 0.25 vCPUs then you probably don't want traditional multiprocessing or multithreading...
    • tinix 1y ago
      Go try it and report back. Fargate isn't just ECS and plain containers. You cannot use shared memory in fargate, there is literally no /dev/shm. See "sharedMemorySize" here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html https://docs.aws.amazon.com/AmazonECS/latest/developerguide/... > If you're using tasks that use the Fargate launch type, the sharedMemorySize parameter isn't supported.
  • sgarland 1y ago
    Well don’t use Fargate, there’s your problem. Run programs on actual servers, not magical serverless bullshit.
    • tinix 1y ago
      > Well don’t use Fargate, there’s your problem. Run programs on actual servers, not magical serverless bullshit. That kind of absolutism misses the point of why serverless architectures like Fargate exist. It might feel satisfying, but it closes the door on understanding why stateless and ephemeral workloads exist in the first place. I get the frustration, but dismissing a production architecture outright ignores the constraints and trade-offs that lead teams to adopt it in the first place. It's worth asking: if so many teams are using this shit in production, at scale, with real stakes, what do they know that might be missing from my current mental model? Serverless, like any abstraction, isn't magic. It's a tool with defined trade-offs, and resource/process isolation is one of them. If you're running containerized workloads at scale, optimizing for organizational velocity, security boundaries, and multi-tenant isolation, these constraints aren't bullshit, they're literally design parameters and intentional boundaries. It's easy to throw shade from a distance, but the operational reality of running modern systems, especially in regulated or high-scale environments, looks very different from a home lab or startup sandbox.
• perlgeek 1y ago
  > Never understood why this isn’t used more frequently. Can you throw a JSON-serializable data structure (lists, dict, strings, numbers) into SharedMemory? What about regular instance of random Python classes? If the answer is "no", that explains why it's not done more often. The examples in the docs seem to pass byte strings and byte arrays around, which is far less convenient than regular data structures.
  • dragonwriter 1y ago
    > Can you throw a JSON-serializable data structure (lists, dict, strings, numbers) into SharedMemory? You can throw a JSON-serialized data structure into SharedMemory, sure, since you can store strings. > The examples in the docs seem to pass byte strings and byte arrays around The examples in the docs largely use ShareableList, which itself can contain any of int, float, bool, str, bytes, and None-type values.