6 ms·

The implication of what mTCP is doing: Only threads on the same CPU core can communicate at the highest speeds. Isn't this situation broken? Shouldn't we have b

The implication of what mTCP is doing: Only threads on the same CPU core can communicate at the highest speeds. Isn't this situation broken? Shouldn't we have better hardware support for communicating threads? Why in modern software, are threads communicating through CPU cache invalidation?

dooglius 7y ago
Moving to e.g. shared scratchpad memory would be a major paradigm shift, you'd have to do a lot of coordination between CPU vendors, OS/kernel developers, high-performance library writeslrs, etc to make it happen.
- stcredzero 7y ago
  How did mainframe I/O channels work?
  - michaelwilson 7y ago
    As I recalled, you wrote channel instructions, or chains of channel instructions, as "CCW"s (Channel Command Words), which you dispatched to the channels via a "SIO" or "SIOF" instruction (Start I/O, or Start I/O Fast). Once the operation started, the SIO(F) returned to you and the channel operated asynchronously from the CPU, moving the data directly in or out of memory. I think SIOF returned as soon as the channel received the instruction, whereas SIO waited for the instruction actually began. So the channel actually added as sort of co-processor with the same access to memory as other CPUS. I'm pretty sure the channel could use virtual addresses, but the memory of course had to be fixed (pagefixed) during the operation.
    - stcredzero 7y ago
      How about another set of registers, which can be written by one CPU, and read by the others? Could channels be implemented as co-processors with access to memory, going through the cache? I know there are instructions to fix cache lines. I suppose some of this could be accomplished by compilers on current hardware, if they had information about CPU cores being targeted.
      - MaxBarraclough 7y ago
        > How about another set of registers, which can be written by one CPU, and read by the others? The L3 cache is doing something similar. Whatever you come up with has to cope with each core implementing out-of-order execution, so it can't be both trivially simple and ultra-fast. The L2 and L1 caches are more local to the particular core, and so are faster. More generally, it seems to me unlikely that it would ever make sense to trade off against per-core performance. > I suppose some of this could be accomplished by compilers on current hardware, if they had information about CPU cores being targeted. Are we talking about a new software abstraction, or a performance-enhancement on existing hardware? The latter seems unlikely to me - the parallelism folks would've thought of it.
- dragontamer 7y ago
  > Moving to e.g. shared scratchpad memory would be a major paradigm shift Shared scratchpad memory is (slowly) happening in the GPGPU world, at least in very limited circumstances. But yeah, I think that's why GPGPU programmers are managing to get better scaling than classical CPUs, because GPGPUs are more willing to toy with the memory model, and GPGPU programmers are willing to use those highly-specialized communication features.
  - dooglius 7y ago
    I don't know much about GPGPU-land, but all of the difficulties I foresee have to do with multiprocessing/sharing/context switching... my guess is that only one logical program can use the scratchpad at a time?
    - stcredzero 7y ago
      all of the difficulties I foresee have to do with multiprocessing/sharing/context switching... I'd like to be able to park a high performance actor on a particular CPU, and have it consuming its "inbox" queue, no context switches necessary. my guess is that only one logical program can use the scratchpad at a time? How about registers or a scratchpad which has async pub/sub semantics? One CPU can write, and the rest can read? A mechanism like that would use the same hardware which already supports cache coherency, but would let programmers forget about a lot of potential headaches.
    - dragontamer 7y ago
      > I don't know much about GPGPU-land, but all of the difficulties I foresee have to do with multiprocessing/sharing/context switching... my guess is that only one logical program can use the scratchpad at a time? Well.. that one logical program can have 1024 SIMD-threads (maybe 16x (AMD) or 32x (NVidia) "actual" threads... where an "actual thread" is a "ticking program counter" by my definition), but yeah, its "one logical program" from the perspective of the GPU. Even more specific than "one logical program", but "one threadgroup". So if you have a program that spins up 4096 SIMD-threads, only 1024 of them at a time can actually share a particular Shared Memory. (The GPU will allocate say, 10kB to the different "thread groups", but Threadgroup 0-1023 can touch its own 10kB block, while Threadgroup 1024-2047 can only touch another 10kB block) GPU Shared Memory can be "split" to different programs. So one program can reserve 10kB, while a 2nd program can reserve 20kB, and then both can run on one GPU-unit. But the two programs are unable to touch each other's shared memory.
tempguy9999 7y ago
Forgive me if I'm getting the wrong end of the stick but > Only threads on the same CPU core can communicate at the highest speeds. Isn't this situation broken? Why broken? If they're on the same core then they share the L1/L2 cache which is as fast as it can get. > Shouldn't we have better hardware support for communicating threads? Peeps who do chips aren't stupid. It's probably laws of physics rather than incompetence that stopping it, and if it isn't that then it's economics. > Why in modern software, are threads communicating through CPU cache invalidation? I can't see that mentioned in the article. But if it works, well, cache line snooping is, I guess, exceedingly fast so why not.
- stcredzero 7y ago
  Why broken? If they're on the same core then they share the L1/L2 cache which is as fast as it can get. Why can't I have fast, low instruction cost communication between any two CPUs, regardless of which core it's on? In other words, why do I have to do that bookkeeping to have super fast communication between threads/actors? Peeps who do chips aren't stupid. It's probably laws of physics rather than incompetence that stopping it, and if it isn't that then it's economics. The RISC revolution was in part due to "peeps who do chips" getting stuck in their ways, succumbing to CISC group-think, and forgetting to "play Moneyball" with price/performance. I don't think communication between threads was so much an issue in the past. But if it works, well, cache line snooping is, I guess, exceedingly fast so why not. Is it really worth sacrificing an entire cache line to do it? Are there other communication mechanisms which could achieve the same or higher speeds with a smaller component count? I don't see how it would violate the laws of physics if the answers were no and yes.
  - tempguy9999 7y ago
    > Why can't I have fast, low instruction cost communication between any two CPUs, regardless of which core it's on? Fizzics, specifically a mixture of C and some electrical crap about signal propagation which I don't unnerstand. Einstein, he say no. > The RISC revolution was in part true I suppose. Even dumber things were reputed to have been done in the design of the SPARC chips. > Is it really worth sacrificing an entire cache line to do it? First show me where this is mentioned in the doc. Second, yeah, why not. If you can pass 512 bits of info between threads in a few cycles, sounds ok to me. If you want another mechanism, be prepared to pay for it. Memory has been around for ages, it's well understood, and bloody fast. It'd be stupid not to use it. Look, you can't just have stuff cos you want it. You've given no reason for needing it, or that you understand the tradeoffs at the hardware level, or any appreciation of the blinding speed of current chips. How would faster XYZ improve your life? Go and read <https://www.amazon.co.uk/Computer-Architecture-Quantitative-Approach-Kaufmann/dp/0128119055/> https://www.amazon.co.uk/Computer-Architecture-Quantitative-... then maybe you'll understand your plaint is falling on uninterested ears (mine anyway). Suggest something constructive based on the realities of hardware then I'll talk.

dooglius 7y ago

Moving to e.g. shared scratchpad memory would be a major paradigm shift, you'd have to do a lot of coordination between CPU vendors, OS/kernel developers, high-performance library writeslrs, etc to make it happen.

stcredzero 7y ago
How did mainframe I/O channels work?
- michaelwilson 7y ago
  As I recalled, you wrote channel instructions, or chains of channel instructions, as "CCW"s (Channel Command Words), which you dispatched to the channels via a "SIO" or "SIOF" instruction (Start I/O, or Start I/O Fast). Once the operation started, the SIO(F) returned to you and the channel operated asynchronously from the CPU, moving the data directly in or out of memory. I think SIOF returned as soon as the channel received the instruction, whereas SIO waited for the instruction actually began. So the channel actually added as sort of co-processor with the same access to memory as other CPUS. I'm pretty sure the channel could use virtual addresses, but the memory of course had to be fixed (pagefixed) during the operation.
  - stcredzero 7y ago
    How about another set of registers, which can be written by one CPU, and read by the others? Could channels be implemented as co-processors with access to memory, going through the cache? I know there are instructions to fix cache lines. I suppose some of this could be accomplished by compilers on current hardware, if they had information about CPU cores being targeted.
    - MaxBarraclough 7y ago
      > How about another set of registers, which can be written by one CPU, and read by the others? The L3 cache is doing something similar. Whatever you come up with has to cope with each core implementing out-of-order execution, so it can't be both trivially simple and ultra-fast. The L2 and L1 caches are more local to the particular core, and so are faster. More generally, it seems to me unlikely that it would ever make sense to trade off against per-core performance. > I suppose some of this could be accomplished by compilers on current hardware, if they had information about CPU cores being targeted. Are we talking about a new software abstraction, or a performance-enhancement on existing hardware? The latter seems unlikely to me - the parallelism folks would've thought of it.
dragontamer 7y ago
> Moving to e.g. shared scratchpad memory would be a major paradigm shift Shared scratchpad memory is (slowly) happening in the GPGPU world, at least in very limited circumstances. But yeah, I think that's why GPGPU programmers are managing to get better scaling than classical CPUs, because GPGPUs are more willing to toy with the memory model, and GPGPU programmers are willing to use those highly-specialized communication features.
- dooglius 7y ago
  I don't know much about GPGPU-land, but all of the difficulties I foresee have to do with multiprocessing/sharing/context switching... my guess is that only one logical program can use the scratchpad at a time?
  - stcredzero 7y ago
    all of the difficulties I foresee have to do with multiprocessing/sharing/context switching... I'd like to be able to park a high performance actor on a particular CPU, and have it consuming its "inbox" queue, no context switches necessary. my guess is that only one logical program can use the scratchpad at a time? How about registers or a scratchpad which has async pub/sub semantics? One CPU can write, and the rest can read? A mechanism like that would use the same hardware which already supports cache coherency, but would let programmers forget about a lot of potential headaches.
  - dragontamer 7y ago
    > I don't know much about GPGPU-land, but all of the difficulties I foresee have to do with multiprocessing/sharing/context switching... my guess is that only one logical program can use the scratchpad at a time? Well.. that one logical program can have 1024 SIMD-threads (maybe 16x (AMD) or 32x (NVidia) "actual" threads... where an "actual thread" is a "ticking program counter" by my definition), but yeah, its "one logical program" from the perspective of the GPU. Even more specific than "one logical program", but "one threadgroup". So if you have a program that spins up 4096 SIMD-threads, only 1024 of them at a time can actually share a particular Shared Memory. (The GPU will allocate say, 10kB to the different "thread groups", but Threadgroup 0-1023 can touch its own 10kB block, while Threadgroup 1024-2047 can only touch another 10kB block) GPU Shared Memory can be "split" to different programs. So one program can reserve 10kB, while a 2nd program can reserve 20kB, and then both can run on one GPU-unit. But the two programs are unable to touch each other's shared memory.

tempguy9999 7y ago

Forgive me if I'm getting the wrong end of the stick but > Only threads on the same CPU core can communicate at the highest speeds. Isn't this situation broken? Why broken? If they're on the same core then they share the L1/L2 cache which is as fast as it can get. > Shouldn't we have better hardware support for communicating threads? Peeps who do chips aren't stupid. It's probably laws of physics rather than incompetence that stopping it, and if it isn't that then it's economics. > Why in modern software, are threads communicating through CPU cache invalidation? I can't see that mentioned in the article. But if it works, well, cache line snooping is, I guess, exceedingly fast so why not.

stcredzero 7y ago
Why broken? If they're on the same core then they share the L1/L2 cache which is as fast as it can get. Why can't I have fast, low instruction cost communication between any two CPUs, regardless of which core it's on? In other words, why do I have to do that bookkeeping to have super fast communication between threads/actors? Peeps who do chips aren't stupid. It's probably laws of physics rather than incompetence that stopping it, and if it isn't that then it's economics. The RISC revolution was in part due to "peeps who do chips" getting stuck in their ways, succumbing to CISC group-think, and forgetting to "play Moneyball" with price/performance. I don't think communication between threads was so much an issue in the past. But if it works, well, cache line snooping is, I guess, exceedingly fast so why not. Is it really worth sacrificing an entire cache line to do it? Are there other communication mechanisms which could achieve the same or higher speeds with a smaller component count? I don't see how it would violate the laws of physics if the answers were no and yes.
- tempguy9999 7y ago
  > Why can't I have fast, low instruction cost communication between any two CPUs, regardless of which core it's on? Fizzics, specifically a mixture of C and some electrical crap about signal propagation which I don't unnerstand. Einstein, he say no. > The RISC revolution was in part true I suppose. Even dumber things were reputed to have been done in the design of the SPARC chips. > Is it really worth sacrificing an entire cache line to do it? First show me where this is mentioned in the doc. Second, yeah, why not. If you can pass 512 bits of info between threads in a few cycles, sounds ok to me. If you want another mechanism, be prepared to pay for it. Memory has been around for ages, it's well understood, and bloody fast. It'd be stupid not to use it. Look, you can't just have stuff cos you want it. You've given no reason for needing it, or that you understand the tradeoffs at the hardware level, or any appreciation of the blinding speed of current chips. How would faster XYZ improve your life? Go and read <https://www.amazon.co.uk/Computer-Architecture-Quantitative-Approach-Kaufmann/dp/0128119055/> https://www.amazon.co.uk/Computer-Architecture-Quantitative-... then maybe you'll understand your plaint is falling on uninterested ears (mine anyway). Suggest something constructive based on the realities of hardware then I'll talk.