4 ms·

Long story short: don't use JSON numbers to represent money or monetary rates. Always use decimals encoded as string. It's surprising how many APIs fall short o

Long story short: don't use JSON numbers to represent money or monetary rates. Always use decimals encoded as string. It's surprising how many APIs fall short of this basic bar.

jillesvangurp 3y ago
Depends on the language. On the JVM you are fine. With Javascript, doing math on big numbers is probably going to end in tears unless you know what you are doing. Either way, have some tests for this and make sure your code is doing what you expect. Encoding numbers as string because you are using a language and parser that can't deal with numbers properly (even 64 bit doubles), is a bit of a hack. Basically the rest of the world giving up because Javascript can't get its shit together is not a great plan.
- wruza 3y ago
  Accounting for the lowest common denominator that has a huge share in it is always a great plan. Every trading platform out there uses "+-ddd.ddd" format, even binary-born protocols completely unrelated to js used it since forever.
incorrecthorse 3y ago
No. Use integers to store the smallest money decimal, and store the currency name alongside.
- sjducb 3y ago
  What happens if you’re sure that four decimal places is the smallest, then suddenly a partner system starts sending you 6 decimal places?
  - Macha 3y ago
    Precision should be part of the spec for integrations. With the integer multiple of minimal unit, that makes it clear in the API what it is. e.g. it doesn't make sense to support billing in sub-currency unit amounts just by allowing it in your API definition, as you're going to need to batch that until you get a billable amount which is larger than the fee for issuing a bill. Even for something like $100,000.1234, the bank doesn't let you do a transfer for 0.34c. For cases where sub-currency unit billing is a thing, it should be agreed what the minimal unit is (e.g. advertising has largely standardised on millicents)
    - worddepress 3y ago
      Implementation of fixed point decimals, using multiple integer representations encoded within a floating point system. Nice.
      - Macha 3y ago
        Well I mean, if your minimal unit is 1c, then a price like $22.56 should be encoded as 2256 cents. If you're doing ads and going for millicents, something like $0.01234 should be encoded as 1234 millicents. Obviously you have to agree on what you're measuring in the API, you can't have some values be millicents and others cents.
        worddepress 3y ago
        Yeah I am more laughing that once encoded in JSON as { "p": 2256, "dp": 2 } you are using 2 floating point numbers. But JSON, and indeed JS wasn't designed.
        Macha 3y ago
        To be clear, I wasn't advocating for flexible decimal points. There is no "dp" parameter in the solution I was proposing. It's just documented in the API that "price" is denominated in cents (or satoshis or whatever you want)
    - isleyaardvark 3y ago
      Just a note that precision is a part of the standard if you’re using the ISO 4217 standard, which defines minor unit fractions. https://en.wikipedia.org/wiki/ISO_4217 https://en.wikipedia.org/wiki/ISO_4217 Or choose a different standard, I don’t know what else is out there, but you probably should choose an existing one.
- mrloba 3y ago
  Then you should store the time as well, because the number of decimals in a currency can change (see ISK). Also, some systems disagree on the number of decimals, so be careful. And of course prices can have more decimals. And then you have cryptocurrencies, so make sure you use bigints
- yawaramin 3y ago
  You store it as an integer, but as we just saw in the OP, for general interop with any system that parses JSON you have to assume that it will be parsed as a double. So to avoid precision loss you are going to have to store it as a string anyway. At that point it's upto you whether you want to reinvent the wheel and implement all the required arithmetic operations for your new fixed-point type. Or you could just use the existing decimal type that ships on almost every mature platform: Java, C#, Python, Ruby, etc.
  - zerocrates 3y ago
    In dollars, what do you get up to with a double of cents without precision loss? It's in the trillions, I figure? So a very large space of applications where smallest-denomination-as-JSON-number is going to be fine.
    - akvadrako 3y ago
      Prices can certainly have more decimals that cents. If you just store cents you can't represent them. You either have to guess at the beginning the smallest unit or store the precision along with it. Just use strings, it's much simpler.
magicalhippo 3y ago
We've used XML for interchange of order-like data. Customers have started demanding JSON, so I built a tool to generate XML <-> JSON converters, along with JSON Schema file, based on an XSD, so we could continue to use our existing XML infrastructure on the inside. I must admit I totally forgot about the JSON number issue. Our files include fields for various monetary amounts and similar, and in XML we just used "xs:decimal". Most will be less than a million and requires less than four decimal digits. But I guess someone might decide to use it for the equivalent of a space station or similar, ya never know...
kag0 3y ago
Yeah I disagree. As the article said > RFC 8259 raises the important point that ultimately implementations decide what a JSON number is. Any implementation dealing with money or monetary rates should know that it needs to deal with precision and act accordingly. If you want to use JavaScript to work with money, you need to get a library that allows you to represent high precision numbers. It's not unreasonable to also expect that you get a JSON parsing library that supports the same. oh, TIL that you can support large numbers with the default JavaScript JSON library https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON?ref=codesnippet.io#using_json_numbers https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
- yawaramin 3y ago
  The only problem with this attitude is that JSON APIs are meant to be interoperable, and as the OP showed, you can't rely on the systems you interoperate with to uniformly have the same understanding of JSON numbers, and misinterpreting numbers because of system incompatibilities will cause some really bad headaches that are totally avoidable by just forcing everyone to interop in terms of decimal numbers encoded as strings.

Long story short: don't use JSON numbers to represent money or monetary rates. Always use decimals encoded as string. It's surprising how many APIs fall short of this basic bar.

jillesvangurp 3y ago

Depends on the language. On the JVM you are fine. With Javascript, doing math on big numbers is probably going to end in tears unless you know what you are doing. Either way, have some tests for this and make sure your code is doing what you expect. Encoding numbers as string because you are using a language and parser that can't deal with numbers properly (even 64 bit doubles), is a bit of a hack. Basically the rest of the world giving up because Javascript can't get its shit together is not a great plan.

wruza 3y ago
Accounting for the lowest common denominator that has a huge share in it is always a great plan. Every trading platform out there uses "+-ddd.ddd" format, even binary-born protocols completely unrelated to js used it since forever.

incorrecthorse 3y ago

No. Use integers to store the smallest money decimal, and store the currency name alongside.

sjducb 3y ago
What happens if you’re sure that four decimal places is the smallest, then suddenly a partner system starts sending you 6 decimal places?
- Macha 3y ago
  Precision should be part of the spec for integrations. With the integer multiple of minimal unit, that makes it clear in the API what it is. e.g. it doesn't make sense to support billing in sub-currency unit amounts just by allowing it in your API definition, as you're going to need to batch that until you get a billable amount which is larger than the fee for issuing a bill. Even for something like $100,000.1234, the bank doesn't let you do a transfer for 0.34c. For cases where sub-currency unit billing is a thing, it should be agreed what the minimal unit is (e.g. advertising has largely standardised on millicents)
  - worddepress 3y ago
    Implementation of fixed point decimals, using multiple integer representations encoded within a floating point system. Nice.
    - Macha 3y ago
      Well I mean, if your minimal unit is 1c, then a price like $22.56 should be encoded as 2256 cents. If you're doing ads and going for millicents, something like $0.01234 should be encoded as 1234 millicents. Obviously you have to agree on what you're measuring in the API, you can't have some values be millicents and others cents.
      - worddepress 3y ago
        Yeah I am more laughing that once encoded in JSON as { "p": 2256, "dp": 2 } you are using 2 floating point numbers. But JSON, and indeed JS wasn't designed.
        Macha 3y ago
        To be clear, I wasn't advocating for flexible decimal points. There is no "dp" parameter in the solution I was proposing. It's just documented in the API that "price" is denominated in cents (or satoshis or whatever you want)
  - isleyaardvark 3y ago
    Just a note that precision is a part of the standard if you’re using the ISO 4217 standard, which defines minor unit fractions. https://en.wikipedia.org/wiki/ISO_4217 https://en.wikipedia.org/wiki/ISO_4217 Or choose a different standard, I don’t know what else is out there, but you probably should choose an existing one.
mrloba 3y ago
Then you should store the time as well, because the number of decimals in a currency can change (see ISK). Also, some systems disagree on the number of decimals, so be careful. And of course prices can have more decimals. And then you have cryptocurrencies, so make sure you use bigints
yawaramin 3y ago
You store it as an integer, but as we just saw in the OP, for general interop with any system that parses JSON you have to assume that it will be parsed as a double. So to avoid precision loss you are going to have to store it as a string anyway. At that point it's upto you whether you want to reinvent the wheel and implement all the required arithmetic operations for your new fixed-point type. Or you could just use the existing decimal type that ships on almost every mature platform: Java, C#, Python, Ruby, etc.
- zerocrates 3y ago
  In dollars, what do you get up to with a double of cents without precision loss? It's in the trillions, I figure? So a very large space of applications where smallest-denomination-as-JSON-number is going to be fine.
  - akvadrako 3y ago
    Prices can certainly have more decimals that cents. If you just store cents you can't represent them. You either have to guess at the beginning the smallest unit or store the precision along with it. Just use strings, it's much simpler.

magicalhippo 3y ago

We've used XML for interchange of order-like data. Customers have started demanding JSON, so I built a tool to generate XML <-> JSON converters, along with JSON Schema file, based on an XSD, so we could continue to use our existing XML infrastructure on the inside. I must admit I totally forgot about the JSON number issue. Our files include fields for various monetary amounts and similar, and in XML we just used "xs:decimal". Most will be less than a million and requires less than four decimal digits. But I guess someone might decide to use it for the equivalent of a space station or similar, ya never know...

kag0 3y ago

Yeah I disagree. As the article said > RFC 8259 raises the important point that ultimately implementations decide what a JSON number is. Any implementation dealing with money or monetary rates should know that it needs to deal with precision and act accordingly. If you want to use JavaScript to work with money, you need to get a library that allows you to represent high precision numbers. It's not unreasonable to also expect that you get a JSON parsing library that supports the same. oh, TIL that you can support large numbers with the default JavaScript JSON library https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON?ref=codesnippet.io#using_json_numbers https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

yawaramin 3y ago
The only problem with this attitude is that JSON APIs are meant to be interoperable, and as the OP showed, you can't rely on the systems you interoperate with to uniformly have the same understanding of JSON numbers, and misinterpreting numbers because of system incompatibilities will cause some really bad headaches that are totally avoidable by just forcing everyone to interop in terms of decimal numbers encoded as strings.