6 ms·

HN is collectively schizophrenic on this topic. On the one hand, the comments say that nobody really needs anything fancier than JSON. On the other hand, othe

HN is collectively schizophrenic on this topic. On the one hand, the comments say that nobody really needs anything fancier than JSON. On the other hand, other comments say that if JSON parsing shows up in your profiles, you should switch off JSON!

gameswithgo 7y ago
that just reflects that actual engineers and not actual engineers frequent hn
userbinator 7y ago
HN (and a lot of other places, really) has a large population of web or otherwise "web-heritage" developers, as well as those whose ventures into computing started beside/before the web. I suspect the JSON promotion comes mostly from the former group, while the latter, of which I am a member, laments it roughly as much as we did the "XML craze" of the late 90s and early 2000s.
- cm2187 7y ago
  And a tiny minority I am proudly part of, who watches these debates about xml vs json or tabs vs whitespace with the same amusement than Mike Judge!
  - xxs 7y ago
    C'mon spaces vs tab is a serious one and it makes sense... unlike xml, json shizz.
    - saagarjha 7y ago
      Spaces versus tabs is a stupid debate because "tabs for indentation, spaces for alignment" is clearly the correct answer.
- xxs 7y ago
  Indeed, hand rolled fixed point network marshaling, preferably with delta encoding for time series alike data (or data that oscillates) is couple of orders of magnitude faster and but also significantly "smaller" than a double-string-double, counterpart. Getting ~1 byte per data point of time series is quite achievable.
  - vbezhenar 7y ago
    How do you encode floating point delta into one byte?
    - thedance 7y ago
      You can't, that's why they said fixed point. Time series data should probably never use floats, for the exact reason that delta-encoding is not effective.
      - scottlamb 7y ago
        "Can't" is too strong a word. Depending on what your numbers represent, I'd think you could do one or more of: * have a shorter exponent and/or mantissa to make each number shorter and the overall series more repetitive. (maybe even skip the mantissa if you only care about order of magnitude! call it the opposite of fixed-point, or just storing the logarithm of the value, whatever terminology works for you...) * skip the sign bit for a non-negative series (e.g., deltas of a non-decreasing series), * use a variable-width bitstream encoding like exp-Golomb to represent the exponent and/or mantissa, * use run-length encoding if it's repetitive (after delta-encoding), * etc. One byte per value (or less!) is surely possible in some cases. But I've never done it myself and agree fixed-point sounds much more pleasant. One of my projects represents deltas of durations in units of 90,000ths of a second, varint-encoded. I much prefer that to dealing with float seconds.
        thedance 7y ago
        There's also the XOR compression scheme described by Facebook: https://www.vldb.org/pvldb/vol8/p1816-teller.pdf https://www.vldb.org/pvldb/vol8/p1816-teller.pdf
scottlamb 7y ago
I think if you're a little more generous and say "you rarely need" rather than "nobody really needs", there's no contradiction. Most of the time, you're not using much CPU in total or your server is doing something much heavier than parsing and serialization. Either way, JSON is fine. Improving its performance is nice but not worth a ton of effort. Once in a while, it's significant, and you switch to a binary format if you can. And like anything, it depends on what you're doing. If you're doing lots of server-to-server RPCs on big distributed systems like within Google, it's much more reasonable to start with protobuf or the like than with JSON. And not just for performance reasons; having the generated code helps a lot with maintainability. (Some JSON libraries have something similar; some don't.)
- zozbot234 7y ago
  Hexfloats are not a binary format, they're as text based as anything else. And they get rid of significant bottlenecks in float parsing.
  - thedance 7y ago
    But the existence of the hex float format, and the fact that it accepts either 0x or 0X, upper or lowercase characters for A-F, dot or comma for the point depending on locale, and either p or P for the exponent, all contribute to the reasons why strtod is so slow. The faster library in this article handles none of those things. It also does not handle NaN or INF.
XMPPwocky 7y ago
Couldn't it just be different users posting those?
- mandelbrotwurst 7y ago
  That's exactly what this person is saying / what they intended by the phrase "collectively schizophrenic".
  - XMPPwocky 7y ago
    Okay, but then "people disagree about this" seems a bit less interesting?

gameswithgo 7y ago

that just reflects that actual engineers and not actual engineers frequent hn

userbinator 7y ago

HN (and a lot of other places, really) has a large population of web or otherwise "web-heritage" developers, as well as those whose ventures into computing started beside/before the web. I suspect the JSON promotion comes mostly from the former group, while the latter, of which I am a member, laments it roughly as much as we did the "XML craze" of the late 90s and early 2000s.

cm2187 7y ago
And a tiny minority I am proudly part of, who watches these debates about xml vs json or tabs vs whitespace with the same amusement than Mike Judge!
- xxs 7y ago
  C'mon spaces vs tab is a serious one and it makes sense... unlike xml, json shizz.
  - saagarjha 7y ago
    Spaces versus tabs is a stupid debate because "tabs for indentation, spaces for alignment" is clearly the correct answer.
xxs 7y ago
Indeed, hand rolled fixed point network marshaling, preferably with delta encoding for time series alike data (or data that oscillates) is couple of orders of magnitude faster and but also significantly "smaller" than a double-string-double, counterpart. Getting ~1 byte per data point of time series is quite achievable.
- vbezhenar 7y ago
  How do you encode floating point delta into one byte?
  - thedance 7y ago
    You can't, that's why they said fixed point. Time series data should probably never use floats, for the exact reason that delta-encoding is not effective.
    - scottlamb 7y ago
      "Can't" is too strong a word. Depending on what your numbers represent, I'd think you could do one or more of: * have a shorter exponent and/or mantissa to make each number shorter and the overall series more repetitive. (maybe even skip the mantissa if you only care about order of magnitude! call it the opposite of fixed-point, or just storing the logarithm of the value, whatever terminology works for you...) * skip the sign bit for a non-negative series (e.g., deltas of a non-decreasing series), * use a variable-width bitstream encoding like exp-Golomb to represent the exponent and/or mantissa, * use run-length encoding if it's repetitive (after delta-encoding), * etc. One byte per value (or less!) is surely possible in some cases. But I've never done it myself and agree fixed-point sounds much more pleasant. One of my projects represents deltas of durations in units of 90,000ths of a second, varint-encoded. I much prefer that to dealing with float seconds.
      - thedance 7y ago
        There's also the XOR compression scheme described by Facebook: https://www.vldb.org/pvldb/vol8/p1816-teller.pdf https://www.vldb.org/pvldb/vol8/p1816-teller.pdf

scottlamb 7y ago

I think if you're a little more generous and say "you rarely need" rather than "nobody really needs", there's no contradiction. Most of the time, you're not using much CPU in total or your server is doing something much heavier than parsing and serialization. Either way, JSON is fine. Improving its performance is nice but not worth a ton of effort. Once in a while, it's significant, and you switch to a binary format if you can. And like anything, it depends on what you're doing. If you're doing lots of server-to-server RPCs on big distributed systems like within Google, it's much more reasonable to start with protobuf or the like than with JSON. And not just for performance reasons; having the generated code helps a lot with maintainability. (Some JSON libraries have something similar; some don't.)

zozbot234 7y ago
Hexfloats are not a binary format, they're as text based as anything else. And they get rid of significant bottlenecks in float parsing.
- thedance 7y ago
  But the existence of the hex float format, and the fact that it accepts either 0x or 0X, upper or lowercase characters for A-F, dot or comma for the point depending on locale, and either p or P for the exponent, all contribute to the reasons why strtod is so slow. The faster library in this article handles none of those things. It also does not handle NaN or INF.

XMPPwocky 7y ago

Couldn't it just be different users posting those?

mandelbrotwurst 7y ago
That's exactly what this person is saying / what they intended by the phrase "collectively schizophrenic".
- XMPPwocky 7y ago
  Okay, but then "people disagree about this" seems a bit less interesting?