4 ms·

exactly, a typical REST server spends a lot of compute time in json encoding and decoding. which could be sped up significantly if there is better memory bandwi

exactly, a typical REST server spends a lot of compute time in json encoding and decoding. which could be sped up significantly if there is better memory bandwidth

glhaynes 3y ago
My first guess would've been that JSON encode/decode wouldn't be memory bandwith-bound because it involves so many operations on single bytes at a time, most of which would be in the cache after the page has been brought in initially. Of course maybe much of that is done with SIMD these days? Anyway, I have near-zero experience in this area so I'm mostly just posting this hoping for someone to explain why I'm wrong.
- densh 3y ago
  See https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson
  - gpderetta 3y ago
    I don't see how the link clarifies things. simdjson is fast enough that JSON decoding can approach single-core bandwidth capacity, but it doesn't follow that a JSON-heavy load is bandwitdh constrained. In fact the existence of simdjson points to json decoding being compute bound.
    - densh 3y ago
      It’s a bit more nuanced than that. Scalar code that does lots of branching and loads one value at a time (e.g. single character) often can’t hit memory at peak bandwidth rate. On the other hand well vectorized code that loads vectors of multiple values at a time can properly saturate the available hardware resources.
      - glhaynes 3y ago
        The earlier post was referring to "a typical REST server" in a context of the claim that memory bandwidth "is becoming the ultimate spec". simdjson (impressive as it may be) existing as an interesting project seems likely to imply that most JSON parsing isn't done with such performant methods on "the typical REST server". EDIT: Seeing that it's used in many projects including Node.js shifts me back to thinking more highly of the claim that memory bandwidth is becoming the ultimate spec!
deleted 3y ago
[deleted]

exactly, a typical REST server spends a lot of compute time in json encoding and decoding. which could be sped up significantly if there is better memory bandwidth

glhaynes 3y ago

My first guess would've been that JSON encode/decode wouldn't be memory bandwith-bound because it involves so many operations on single bytes at a time, most of which would be in the cache after the page has been brought in initially. Of course maybe much of that is done with SIMD these days? Anyway, I have near-zero experience in this area so I'm mostly just posting this hoping for someone to explain why I'm wrong.

densh 3y ago
See https://github.com/simdjson/simdjson https://github.com/simdjson/simdjson
- gpderetta 3y ago
  I don't see how the link clarifies things. simdjson is fast enough that JSON decoding can approach single-core bandwidth capacity, but it doesn't follow that a JSON-heavy load is bandwitdh constrained. In fact the existence of simdjson points to json decoding being compute bound.
  - densh 3y ago
    It’s a bit more nuanced than that. Scalar code that does lots of branching and loads one value at a time (e.g. single character) often can’t hit memory at peak bandwidth rate. On the other hand well vectorized code that loads vectors of multiple values at a time can properly saturate the available hardware resources.
    - glhaynes 3y ago
      The earlier post was referring to "a typical REST server" in a context of the claim that memory bandwidth "is becoming the ultimate spec". simdjson (impressive as it may be) existing as an interesting project seems likely to imply that most JSON parsing isn't done with such performant methods on "the typical REST server". EDIT: Seeing that it's used in many projects including Node.js shifts me back to thinking more highly of the claim that memory bandwidth is becoming the ultimate spec!

deleted 3y ago

[deleted]