5 ms·

Because it is not a tool or a language problem but a network reliability problem?

stcredzero 10y ago
Also, our current hardware has no good mechanism for low overhead/latency communication between threads running on different cores. Instead, HFT programmers have to hack something together like Disruptor and insert unused arrays of longint to prevent false sharing.
- zzzcpan 10y ago
  I think the hardware is ok, it's just that most programmers are not that good at writing software on that level. Running an event loop per core and amortizing synchronization is not a rocket science.
  - stcredzero 10y ago
    I think the hardware is ok, it's just that most programmers are not that good at writing software on that level. If it becomes the case that the hardware has changed in such a way that most programmers now have a hard time writing code that compiles efficiently to it, the hardware has changed in the wrong way! Well, that is the case over the past few decades! Hardware has changed in the wrong way.
    - di4na 10y ago
      Or the compilers we use did not evolved... which mean language have not evolved...
      - stcredzero 10y ago
        I read it as closer to "640k ought to be enough for anyone" than simply having features in a language/compiler. C'mon, if you're inserting unused stuff into structures to get stuff on separate cache lines, something is a little too hacky here!
- CyberDildonics 10y ago
  > Also, our current hardware has no good mechanism for low overhead/latency communication between threads running on different cores What is wrong with shared memory? Also if HFT programmers can do it, wouldn't that imply that the hardware can do it?
  - millstone 10y ago
    Here's an illustration; this may be somewhat dated (not sure how this works with e.g. shared L3 caches) but it should give you a flavor of why shared memory sucks for messaging. Consider the simplest example of a busy-loop, which is likely close to the fastest you can get. Core1 sends a message to Core2 by writing to an address that Core2 is repeatedly reading from. Core1 first has to gain exclusive access to the address, by broadcasting an invalidate on the bus. Core2 thus discards its cache line. Next Core1 writes to the address by modifying the data in its L1 cache (or store buffer or worse). Core2 then reads from the address. Since Core1 holds the modified cache line, it is required to snoop the read. Core1 tells Core2 to wait and then retry. This happens repeatedly until Core1's write lands in main memory. Now Core2 can read the memory. So: 1. Messaging requires a round trip to memory, a bunch of cache line invalidation, and other nonsense. Messaging involves multiple transitions to and from shared state. 2. Your CPUs already have a very fast message bus to implement MESI, but it is unavailable to you as a client programmer. It should be obvious from this that we could build a much faster CPU messaging architecture if we cared to.
    - pcwalton 10y ago
      x86 CPUs use MESIF or MOESI, not MESI. Those protocols don't require round trips to main memory.
  - infinite8s 10y ago
    They can do it only by understanding the limitations of the underlying hardware - ie slots as big as a cache line to avoid false sharing, power of 2 ring buffer size to for fast slot calculation, etc.
- pcwalton 10y ago
  > Also, our current hardware has no good mechanism for low overhead/latency communication between threads running on different cores. Shared memory is extremely fast on modern x86.
  - marcosdumay 10y ago
    It is also full of synchronization problems. Both for the layer above and bellow it. Hardware message interfaces would go a good deal into improving the situation.
    - jandrese 10y ago
      It comes down to cache invalidation, which is one of the Hard Problems of computer science.
      - marcosdumay 10y ago
        But it's much better if you only have to deal with it in software, only when it's strictly needed. Currently, we just turn any random problem into a cache invalidation one, and hope it makes our life easier.
  - hyperpape 10y ago
    Is millstone wrong, or are y'all just using different definitions? https://news.ycombinator.com/item?id=12264017 https://news.ycombinator.com/item?id=12264017
    - Jweb_Guru 10y ago
      Millstone is wrong (and pcwalton explained why).
coldtea 10y ago
Nope, that's not it. For one, scaling and parallelism are hard even for programs not depending on the network (in the sense you mean, e.g. being web/internet services) at all (number crunching, simulations, video processing, etc). Second, algorithms can be inherently hard (or even mathematically) impossible to parallelize by themselves -- no network or even hardware required.

Also, our current hardware has no good mechanism for low overhead/latency communication between threads running on different cores. Instead, HFT programmers have to hack something together like Disruptor and insert unused arrays of longint to prevent false sharing.

zzzcpan 10y ago
I think the hardware is ok, it's just that most programmers are not that good at writing software on that level. Running an event loop per core and amortizing synchronization is not a rocket science.
- stcredzero 10y ago
  I think the hardware is ok, it's just that most programmers are not that good at writing software on that level. If it becomes the case that the hardware has changed in such a way that most programmers now have a hard time writing code that compiles efficiently to it, the hardware has changed in the wrong way! Well, that is the case over the past few decades! Hardware has changed in the wrong way.
  - di4na 10y ago
    Or the compilers we use did not evolved... which mean language have not evolved...
    - stcredzero 10y ago
      I read it as closer to "640k ought to be enough for anyone" than simply having features in a language/compiler. C'mon, if you're inserting unused stuff into structures to get stuff on separate cache lines, something is a little too hacky here!
CyberDildonics 10y ago
> Also, our current hardware has no good mechanism for low overhead/latency communication between threads running on different cores What is wrong with shared memory? Also if HFT programmers can do it, wouldn't that imply that the hardware can do it?
- millstone 10y ago
  Here's an illustration; this may be somewhat dated (not sure how this works with e.g. shared L3 caches) but it should give you a flavor of why shared memory sucks for messaging. Consider the simplest example of a busy-loop, which is likely close to the fastest you can get. Core1 sends a message to Core2 by writing to an address that Core2 is repeatedly reading from. Core1 first has to gain exclusive access to the address, by broadcasting an invalidate on the bus. Core2 thus discards its cache line. Next Core1 writes to the address by modifying the data in its L1 cache (or store buffer or worse). Core2 then reads from the address. Since Core1 holds the modified cache line, it is required to snoop the read. Core1 tells Core2 to wait and then retry. This happens repeatedly until Core1's write lands in main memory. Now Core2 can read the memory. So: 1. Messaging requires a round trip to memory, a bunch of cache line invalidation, and other nonsense. Messaging involves multiple transitions to and from shared state. 2. Your CPUs already have a very fast message bus to implement MESI, but it is unavailable to you as a client programmer. It should be obvious from this that we could build a much faster CPU messaging architecture if we cared to.
  - pcwalton 10y ago
    x86 CPUs use MESIF or MOESI, not MESI. Those protocols don't require round trips to main memory.
- infinite8s 10y ago
  They can do it only by understanding the limitations of the underlying hardware - ie slots as big as a cache line to avoid false sharing, power of 2 ring buffer size to for fast slot calculation, etc.
pcwalton 10y ago
> Also, our current hardware has no good mechanism for low overhead/latency communication between threads running on different cores. Shared memory is extremely fast on modern x86.
- marcosdumay 10y ago
  It is also full of synchronization problems. Both for the layer above and bellow it. Hardware message interfaces would go a good deal into improving the situation.
  - jandrese 10y ago
    It comes down to cache invalidation, which is one of the Hard Problems of computer science.
    - marcosdumay 10y ago
      But it's much better if you only have to deal with it in software, only when it's strictly needed. Currently, we just turn any random problem into a cache invalidation one, and hope it makes our life easier.
- hyperpape 10y ago
  Is millstone wrong, or are y'all just using different definitions? https://news.ycombinator.com/item?id=12264017 https://news.ycombinator.com/item?id=12264017
  - Jweb_Guru 10y ago
    Millstone is wrong (and pcwalton explained why).

coldtea 10y ago

Nope, that's not it. For one, scaling and parallelism are hard even for programs not depending on the network (in the sense you mean, e.g. being web/internet services) at all (number crunching, simulations, video processing, etc). Second, algorithms can be inherently hard (or even mathematically) impossible to parallelize by themselves -- no network or even hardware required.