3 ms·

Regarding correctness, will it display uint64 numbers without truncating them? That's my biggest pet peeve with jq currently.

necubi 3y ago
Unfortunately JSON numbers are 64 bit floats, so if you're standards compliant you have to treat them as such, which gives you 53 bits of precision for integers. Also hey, been a while ;) Edit: I stand corrected, the latest spec (rfc8259) only formally specifies the textual format, but not the semantics of numbers. However, it does have this to say: > This specification allows implementations to set limits on the range/and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. In practice, most implementations treat JSON as a subset of Javascript, which implies that numbers are 64-bit floats.
- Groxx 3y ago
  JSON does not define a precision for numbers, so: it's often float64 (but note -0 is allowed, but NaN and +/-Inf are not), but it depends on your language, parser config, etc. Many will produce higher precision but parse as float64 by default. But maximally-compatible JSON systems should always handle arbitrary precision.
- matt_kantor 3y ago
  I'm being pedantic here, but JSON numbers are sequences of digits and ./+/-/e/E. Whether to parse those sequences into 64-bit floats or something else is left up to the implementation. However what you say is good practice anyway. The spec (RFC 8259) has this note on interoperability: > This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.
- rdtsc 3y ago
  > Unfortunately JSON numbers are 64 bit floats, so if you're standards compliant you have to treat them as such, Are you sure? Looking at https://www.json.org/json-en.html https://www.json.org/json-en.html I don't see anything about 64 bit floats.
- lopatin 3y ago
  I thought the JSON spec says that numbers can have an arbitrary amount of digits. Also, what!! Hey! Miss you man.
re 3y ago
I believe this has improved in jq 1.7: https://github.com/jqlang/jq/releases/tag/jq-1.7 https://github.com/jqlang/jq/releases/tag/jq-1.7 > Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate.
- anonymoushn 3y ago
  This is still broken in jq 1.7 for sufficiently long exponents
  - re 3y ago
    From a quick test it looks like it supports exponents up to 9 digits long (i.e. 1.0e999999999), which, frankly, seems pretty reasonable; it's hard for me to imagine a use case where you'd want to represent numbers larger than that.
wwader 3y ago
jq 1.7 do preserve large integers but will truncate if any operation is done on them. Unfortunetly it currently truncates to a decimal64 which is a bit confusing, this will be fixed in next release where it follow the suggestion from the JSON spec and truncates to binary64 (double) https://github.com/jqlang/jq/pull/2949 https://github.com/jqlang/jq/pull/2949

necubi 3y ago

Unfortunately JSON numbers are 64 bit floats, so if you're standards compliant you have to treat them as such, which gives you 53 bits of precision for integers. Also hey, been a while ;) Edit: I stand corrected, the latest spec (rfc8259) only formally specifies the textual format, but not the semantics of numbers. However, it does have this to say: > This specification allows implementations to set limits on the range/and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. In practice, most implementations treat JSON as a subset of Javascript, which implies that numbers are 64-bit floats.

Groxx 3y ago
JSON does not define a precision for numbers, so: it's often float64 (but note -0 is allowed, but NaN and +/-Inf are not), but it depends on your language, parser config, etc. Many will produce higher precision but parse as float64 by default. But maximally-compatible JSON systems should always handle arbitrary precision.
matt_kantor 3y ago
I'm being pedantic here, but JSON numbers are sequences of digits and ./+/-/e/E. Whether to parse those sequences into 64-bit floats or something else is left up to the implementation. However what you say is good practice anyway. The spec (RFC 8259) has this note on interoperability: > This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.
rdtsc 3y ago
> Unfortunately JSON numbers are 64 bit floats, so if you're standards compliant you have to treat them as such, Are you sure? Looking at https://www.json.org/json-en.html https://www.json.org/json-en.html I don't see anything about 64 bit floats.
lopatin 3y ago
I thought the JSON spec says that numbers can have an arbitrary amount of digits. Also, what!! Hey! Miss you man.

re 3y ago

I believe this has improved in jq 1.7: https://github.com/jqlang/jq/releases/tag/jq-1.7 https://github.com/jqlang/jq/releases/tag/jq-1.7 > Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate.

anonymoushn 3y ago
This is still broken in jq 1.7 for sufficiently long exponents
- re 3y ago
  From a quick test it looks like it supports exponents up to 9 digits long (i.e. 1.0e999999999), which, frankly, seems pretty reasonable; it's hard for me to imagine a use case where you'd want to represent numbers larger than that.

wwader 3y ago

jq 1.7 do preserve large integers but will truncate if any operation is done on them. Unfortunetly it currently truncates to a decimal64 which is a bit confusing, this will be fixed in next release where it follow the suggestion from the JSON spec and truncates to binary64 (double) https://github.com/jqlang/jq/pull/2949 https://github.com/jqlang/jq/pull/2949