34 ms·

Denormal and subnormal are synonyms as far as I know and can find. That first sentence is awkwardly phrased: "subnormal numbers are the subset of denormalized

by mkl 2y ago

Denormal and subnormal are synonyms as far as I know and can find. That first sentence is awkwardly phrased: "subnormal numbers are the subset of denormalized numbers (sometimes called denormals)" would be better as "subnormal numbers (sometimes called denormals) are the subset of denormalized numbers". It does imply there are other "denormalized" numbers though; not sure what they are.

jcranmer 2y ago
Terminology is confusing! A floating-point number is generally defined as (-1)^sign * (mantissa) * base^exponent, where mantissa is a sequence of digits in the base. A normalized floating-point number has mantissa be a fixed number of digits for a given floating-point type. A denormalized floating-point number is one with a different number of digits, where the exponent is the minimum possible value. An unnormalized floating-point number is one with a different number of digits, but the exponent is not the minimum possible value. (This doesn't occur in the standard IEEE 754 types, but the PPC and the x86 long double floating-point types have these numbers). There's also noncanonical floating-point numbers, which are different representations for the same value. These occur, for example, with IEEE 754 decimal FP, as well as many pre-IEEE 754 types (where instead of subnormals, you tend to get noncanonical representations of 0). There's also another kind of scenario where you have multiple values that are all still canonical--for example, 0.0 and 0.00 are different canonical decimal FP values in IEEE 754 (they have different exponents, and I'm not going to attempt to remember what they are off the top of my head), and of course -0 and +0 are distinct values of 0 that work differently.
- Asraelite 2y ago
  You didn't cover what exactly "subnormal" means
  - jcranmer 2y ago
    In all practical terms, subnormal is the same as denormal (a non-normalized number with the smallest possible exponent). I think this is a situation where there's confusion over precision of terminology, so things have shifted to a new term which is consistently used. Subnormal is now the new preferred terminology for denormal, I think out of confusion as to whether or not unnormal numbers were also denormal (I have never heard the term so used, but I'm also not a great connoisseur of non-IEEE 754 floating-point). There's a similar attempt to shift use of the term "mantissa" to "significand", since a mantissa implies that the number is in the range [0, 1), whereas the usual implementation in a floating point type is to use the range [1, base).
- librasteve 2y ago
  this is wrong(!) > A normalized floating-point number has mantissa be a fixed number of digits for a given floating-point type. well, no. in fact... > A normalized [binary] floating-point number has mantissa which is bit shifted so that the first bit to the right of the binary point is '1'. (not a great look in an expert explainer comment)
  - labcomputer 2y ago
    Erm, no, not in 754 floating point. The significand of a normalized number starts with an implied 1 to the left of the decimal point, which isn’t explicitly represented. In other words, the significand represents the range [1, 2) for normalized numbers. In fact, that’s reason we need subnormal numbers: they are numbers for which the implied first bit is not 1, so the significand represents the range [0, 1). Subnormal numbers therefore do not have a fixed number of digits (precision). As subnormals approach zero, the precision gets smaller.
    - librasteve 2y ago
      fair point - I did reference this from a general article and it did not specify P754 - I guess this has an implied 1 and can save one bit of data path … but the definition of normalized is that the mantissa is shifted to be aligned to the binary point and not that is have a fixed number of digits since it will often be packed with trailing zeros
      - labcomputer 2y ago
        > but the definition of normalized is that the mantissa is shifted to be aligned to the binary point And that’s the key point. A subnormal number does not have the significand/mantissa aligned to start with an (implied) 1. It has leading zeros. Thus the number of digits of a subnormal number is not fixed. 1.0000x10^0 has the same precision as 1.0000x10^-99. 1.0000x10^-99 does not have the same precision as 0.0010x10^-99. That is why the original poster wrote the subnormals have a not-fixed number of digits. But if you’re going to go around writing snarky “gotcha” comments, at least try to get your facts straight.
stephencanon 2y ago
For IEEE 754-2019 basic formats, subnormal and denormal are synonyms. There are formats (like x87 80-bit as implemented in pre-387 hardware¹) that are not 754 basic formats which admit non-normal encodings that are not subnormal (i.e. not smaller in magnitude than the smallest normal number). Some people used to call these "denormal" as well (even though the x87 docs call them "unnormal"), which caused confusion. "Subnormal" is unambiguous. In decimal IEEE 754 formats there are also non-canonical encodings, but these are neither subnormal nor denormal (decimal encodings are not, in general, normalized the same way binary floating-point is). ----- ¹ on 387 and later these are invalid encodings, and behave like a signaling NaN.

by mkl 2y ago

jcranmer 2y ago

Terminology is confusing! A floating-point number is generally defined as (-1)^sign * (mantissa) * base^exponent, where mantissa is a sequence of digits in the base. A normalized floating-point number has mantissa be a fixed number of digits for a given floating-point type. A denormalized floating-point number is one with a different number of digits, where the exponent is the minimum possible value. An unnormalized floating-point number is one with a different number of digits, but the exponent is not the minimum possible value. (This doesn't occur in the standard IEEE 754 types, but the PPC and the x86 long double floating-point types have these numbers). There's also noncanonical floating-point numbers, which are different representations for the same value. These occur, for example, with IEEE 754 decimal FP, as well as many pre-IEEE 754 types (where instead of subnormals, you tend to get noncanonical representations of 0). There's also another kind of scenario where you have multiple values that are all still canonical--for example, 0.0 and 0.00 are different canonical decimal FP values in IEEE 754 (they have different exponents, and I'm not going to attempt to remember what they are off the top of my head), and of course -0 and +0 are distinct values of 0 that work differently.

Asraelite 2y ago
You didn't cover what exactly "subnormal" means
- jcranmer 2y ago
  In all practical terms, subnormal is the same as denormal (a non-normalized number with the smallest possible exponent). I think this is a situation where there's confusion over precision of terminology, so things have shifted to a new term which is consistently used. Subnormal is now the new preferred terminology for denormal, I think out of confusion as to whether or not unnormal numbers were also denormal (I have never heard the term so used, but I'm also not a great connoisseur of non-IEEE 754 floating-point). There's a similar attempt to shift use of the term "mantissa" to "significand", since a mantissa implies that the number is in the range [0, 1), whereas the usual implementation in a floating point type is to use the range [1, base).
librasteve 2y ago
this is wrong(!) > A normalized floating-point number has mantissa be a fixed number of digits for a given floating-point type. well, no. in fact... > A normalized [binary] floating-point number has mantissa which is bit shifted so that the first bit to the right of the binary point is '1'. (not a great look in an expert explainer comment)
- labcomputer 2y ago
  Erm, no, not in 754 floating point. The significand of a normalized number starts with an implied 1 to the left of the decimal point, which isn’t explicitly represented. In other words, the significand represents the range [1, 2) for normalized numbers. In fact, that’s reason we need subnormal numbers: they are numbers for which the implied first bit is not 1, so the significand represents the range [0, 1). Subnormal numbers therefore do not have a fixed number of digits (precision). As subnormals approach zero, the precision gets smaller.
  - librasteve 2y ago
    fair point - I did reference this from a general article and it did not specify P754 - I guess this has an implied 1 and can save one bit of data path … but the definition of normalized is that the mantissa is shifted to be aligned to the binary point and not that is have a fixed number of digits since it will often be packed with trailing zeros
    - labcomputer 2y ago
      > but the definition of normalized is that the mantissa is shifted to be aligned to the binary point And that’s the key point. A subnormal number does not have the significand/mantissa aligned to start with an (implied) 1. It has leading zeros. Thus the number of digits of a subnormal number is not fixed. 1.0000x10^0 has the same precision as 1.0000x10^-99. 1.0000x10^-99 does not have the same precision as 0.0010x10^-99. That is why the original poster wrote the subnormals have a not-fixed number of digits. But if you’re going to go around writing snarky “gotcha” comments, at least try to get your facts straight.

stephencanon 2y ago

For IEEE 754-2019 basic formats, subnormal and denormal are synonyms. There are formats (like x87 80-bit as implemented in pre-387 hardware¹) that are not 754 basic formats which admit non-normal encodings that are not subnormal (i.e. not smaller in magnitude than the smallest normal number). Some people used to call these "denormal" as well (even though the x87 docs call them "unnormal"), which caused confusion. "Subnormal" is unambiguous. In decimal IEEE 754 formats there are also non-canonical encodings, but these are neither subnormal nor denormal (decimal encodings are not, in general, normalized the same way binary floating-point is). ----- ¹ on 387 and later these are invalid encodings, and behave like a signaling NaN.