6 ms·

> As an observation, performance optimized code is almost always effectively single-threaded these days, even when using all the cores on a CPU to very efficien

• Galanwe 6y ago
  > rely on process-based parallelism which is basically strictly worse. Why is that worse? I very seldomly use threads for concurrency, it creates monolithic binaries that are hard to maintain, configure, and understand. I much prefer a process based architecture with mmap'd shared memories for interprocess communications.
  • snovv_crash 6y ago
    Doesn't this just move the hard part, the maintenance, configuration and understanding, to a different (and equally complex) abstraction layer? Personally, I'd prefer to have a single binary that I start with some arguments, then need to also have a launch script, probably in a different language, which needs to coordinate all the starting, stopping, shared state etc. But, most of my work has been at the workstation level. Maybe it's different once you start needing clusters? The issue is that workstations have grown extremely powerful over the last few years with 50+ cores, 1000s of GPU 'cores', and hundreds of GB of memory. Clusters now bring the same headaches, in hardware, of multiprocess design.
    • Galanwe 6y ago
      > Doesn't this just move the hard part, the maintenance, configuration and understanding, to a different (and equally complex) abstraction layer? That's true indeed, but I find it more manageable that way. To me, managing independent processes instead of threads is especially powerful when the lifecycle of the concurrent work can vary. A typical example that happens quite often is when you have some kind of producer/consumer application. Say, you need to receive some data from a socket, and then process it. If implemented with threads, it becomes quite messy very fast. You end up with a big blob binary that spawns whatever the hell it wants and need to engineer some complex configuration file to tell it how many workers you want, etc. You also need some notification to this process to tell him to increase or decrease its amount of workers, etc. With independent process, it can be much more manageable. You can have a "collector" process that reads the socket and places messages in a shared memory, and if you want more/less workers, you just spawn worker processes to read from the shared memory.
      • maccard 6y ago
        You can make exactly this argument the other way around: If implemented with processes it becomes quite messy very fast. You end up with a bunch of processes that do whatever the hell they want, and need some complex orchestrator script to tell it how many processes you want, and need some notifications to the orchestrator to increase or decrease the number of workers With threads, you can have a main thread that reads the socket and places messages in a shared queue, and if you want more/less workers you just spawn worker threads to read from the queue.
        • Galanwe 6y ago
          > if you want more/less workers you just spawn worker threads to read from the queue. My point is precisely that's its not trivial to do that. You now need some signal mechanism to tell your main process to spawn or kill workers. That's an additional layer of complexity. Things get even worse if you want to add new types of workers that were unplanned. Say I want a new type of worker that is just a forwarder, or a new type of worker that stores the queue on disk for replaying it. With a monolithic thread based design, you now need to stop your whole main and worker threads and start a newly built process that support these workers. With a process based design, you just spawn a new worker process and point it to the existing shared memory without any interruption.
• memco 6y ago
  I was really struck by a comment Jonathan Blow made on stream recently: he said he’s never written a parallel for loop in his whole career. I seem to recall the implication being that they’re often not really necessary for performant code. There’s also been some discussion lately about issues with asynchronous code both in Rust and Python. Point being that parallelism still had a ways to go before it’s proven it’s usefulness. However, I agree with you that it would be nice to see more language tootling to make it simpler since i work on some bits of code that I think could benefit from parallelization but the amount of work I’d have to put in mean it’s a very low priority given the savings.
  • pixel_fcker 6y ago
    This says much more about the type of programs Jonathan Blow tends to write than anything else.
  • Jweb_Guru 6y ago
    Parallelism and asynchronous code are not the same, and in the case of Rust they are very much not the same. Parallel for provides massive advantages for many things including game programming (from experience) so with all due respect I think this says more about Jonathan Blow than it does anything about "parallelism still needing to prove itself."
  • CJefferson 6y ago
    The world is full of highly parallel programs getting useful work done. Most graphics, AI and compression libraries (picking 3 easy examples I've worked on) parallelize well, and can usually make use of all the cores you can throw at them. Jonathan Blow makes good games, but chooses not to make particularly CPU intensive ones. That's fine, but that's also his choice.
    • mariusor 6y ago
      He's also currently building one of the fastest compilers around. It's unreasonable to consider that he never encountered use cases where parallelism makes sense.
• IgorPartola 6y ago
  In what way is process based parallelism strictly worse? On Linux a process and a thread in the kernel are both the same thing. The main difference between the two is that threads share memory by default whereas a process would need to explicitly mmap a chunk of memory to share with another process. This means that with threads you get to save RAM because of application code not having to take up more space (except memory is effectively deduplicated already because those are read only blocks) and with threads you have to explicitly guard against trampling over each others’ memory, whereas with processes it’s safety by default and you have to make an explicit choice to share memory, limiting the number of places you can forget to add or check a lock. I will grant you that languages where standard libraries by default shoehorn you into using sockets to communicate between processes do by default introduce more overhead than just using threads. But any serious language will have at least one queue implementation that’s based on very fast primitives over shared virtual memory. I say this as someone whose done projects using a number of different types of parallelism, including process and thread, and can find strengths for all these. I just think treatment of process based parallelism is undeserved: it can be very efficient if done well, and no worse than a lot of the other methods for a large number of use cases.
  • Shoop 6y ago
    I do not think your model of threads and processes is correct. Processes have different address spaces whereas threads share an address space. Context switching between threads is much cheaper than context switching between processes because you do not have to swap page tables and do a tlb flush. tlb flushes are extremely expensive. I also think you are misunderstanding how mmap works. mmap is not related to thread spawning.
    • mypalmike 6y ago
      Not the parent, but... mmap is related to thread spawning (well, process forking) in that using the MAP_SHARED flag will result in a pointer that is valid shared memory for both the parent process and any forked processes.
    • gmueckl 6y ago
• icedchai 6y ago
  Some languages make it very simple to do parallel operations. Java and Scala parallel streams, for example, are like one line of code. Obviously that won't work for all use cases, but when it does, it is simple. Heavily multi threaded code is difficult to write correctly. Do it wrong, you wind up with race conditions, data corruption, dead locks because a thread pool or other resource is exhausted, thread leaks because you didn't shut something down correctly. The problems go on and on.
  • snovv_crash 6y ago
    Use a map-reduce style pattern for your work allocation and you lose some small amount of efficiency, but the design becomes much much easier. You can even mix and match different types of work in your reduce stage to keep the different types of hardware busy.
• jandrewrogers 6y ago
  Parallelism in 2021 should not be tightly coupled across threads if performance matters, the limitations of that model are well-understood. There is no way to make that comparatively efficient; the CPU cache waste alone ensures that. Nothing you can do with thread support in a programming language will be competitive with e.g. a purpose-built scheduler + native coroutines. That’s right up against the theoretical limit of what is possible in terms of throughput and it doesn’t have any thread overhead. It does introduce the problem of load shedding across cores but that’s solve for all practical purposes. I’ve been writing parallel code at the largest scales most of my career. The state-of-the-art architectures are all, effectively, single-threaded with latency-hiding. This model has a lot of mechanical sympathy with real silicon which is why it is used. It is also pleasantly simple in practice.
  • andi999 6y ago
    Thanks. What is latency hiding?
    • Jweb_Guru 6y ago
      Latency hiding is a a way to substantially increase throughput by queuing massive numbers of requests or tasks while waiting on expensive resources (e.g. main memory access can have latency in the hundreds of cycles for GPUs). By scheduling enough tasks or enough requests that can be dispatched in parallel (or very soon after one another), after an initial delay you may be able to process the queued requests very quickly (possibly once per clock cycle or two) providing similar overall performance to running the same set of tasks sequentially with very low latency. However, such long pipelines are very prone to stalling, especially if there are data dependencies that prevent loads from being issued early, so getting maximum performance out of code on architecture that heavily exploits latency hiding techniques can require a lot of very specific domain knowledge.
    • jandrewrogers 6y ago
      There are many operations on data that are relatively slow from a CPU’s perspective — filling a cache line, page faulting, cache coherency, acquiring a lock, waiting on I/O to complete, etc. All of these add latency by stalling execution. In conventional software, when these events occur you simply stall execution, possibly triggering a context switch (which is very expensive). In many types of modern systems, these events are extremely frequent. Latency hiding is a technique where 1) most workloads are trivially decomposed into independent components that can be executed separately and 2) you can infer or directly determine when any particular operation will stall. There are many ways to execute these high latency operations in an asynchronous and non-blocking way such that you can immediate work on some other part of the workload. The “latency-hiding” part is that the CPU is rarely stalled, always switching to a part of the workload that is immediately runnable if possible so that the CPU is never stalled and always doing real, constructive work. Latency-hiding optimizes for throughput, maximizing utilization of the CPU, but potentially increasing the latency of specific sub-operations by virtue of reordering the execution schedule to “hide” the latency of operations that would stall the processor. For many workloads, the latency of the sub-operations doesn’t matter, only the throughput of the total operation. The real advantage of latency-hiding architectures is that you can approach the theoretical IPC of the silicon in real software. There are exotic CPU architectures explicitly designed for latency hiding, mostly used in supercomputing. Cray/Tera MTA architecture is probably the canonical example as well as the original Xeon Phi. As a class, latency-hiding CPU architectures are sometimes referred to as “barrel processors”. In the case of Cray MTA, the CPU can track 128 separate threads of execution in hardware and automatically switch to a thread that is immediately runnable at each clock cycle. Thread coordination is effectively “free”. In software, switching between logical threads of execution is much more by inference but often sees huge gains in throughput. The only caveat is that you can’t ignore tail latencies in the design — a theoretically optimal latency-hiding architecture may defer execution of an operation indefinitely.
• _ph_ 6y ago
  There are a few tasks, where process-based parallelism is working fine. Like a forking server for handling network requests. This is quite efficient and works even if the handling function is not thread save. Obviously, you get memory safety as well. Unfortunately, that is only a certain subsection of problems and usually you want to be able to use parallel computations on the function call level. There the support for parallel computations of Rust or Go shines. When at each point in the program flow you can decide to go parallel.