5 ms·

Hmm good point. My plan was to allow concepts such as inf, NaN and negative 0 to be available in the abstract, and then convertible to a concrete implementation

Hmm good point. My plan was to allow concepts such as inf, NaN and negative 0 to be available in the abstract, and then convertible to a concrete implementation such as ieee754. But in actual fact it's mostly muddying the waters, and those kinds of values would be better represented as functions that produce ieee754 values rather than abstract concepts that don't really work abstractly anyway (I was thinking that as an abstract concept it could be passed to a general function for any given floating type implementation such as binary float, decimal float, bfloat, etc). >> unsigned: limited to positive integers and 0 > signed: limited to positive and negative integers, and 0 (but excluding -0) > These are floating point values pretending to be integers. You should have actual unlimited precision integers with constraints like explicit maximum and minimum values. The idea here is to have all numbers be treated as reals (in the mathematical sense) for abstract calculation purposes, and then use the pseudo-types "signed" and "unsigned" to represent common invariants on those reals (such as "can only be 0 or an integer" or "can only be 0 or a positive integer"). For example, ('a'~'z'{chars_per_record}){char_count / chars_per_record} (where char_count and chars_per_record are bound from another part of the document) is a common enough method used to envelope data in binary formats. But if char_count = 100 and chars_per_second = 3, you can't accept a fractional count (i.e. it would be an indication that the data in this document is corrupt or malformed). Only the functions `uint`, `sint`, and `float` would convert a number into an actual expression (provided their invariants are respected).

HelloNurse 4y ago
> The idea here is to have all numbers be treated as reals (in the mathematical sense) No. You can't represent, compare etc. most real numbers, so they aren't any good for computational purposes. Practically usable number types include integers and things that have sufficiently number-like behaviour (e.g. fields) and can be represented with a finite number of integers: rational numbers, rational numbers with field extensions, vector spaces, finite sets, intervals, etc. In a language for grammars you can probably stop at integers; general purpose programming languages might want more (for instance, Python has rational numbers) but advanced number types can usually be implemented with libraries.
- kstenerud 4y ago
  I think maybe I'm not understanding you correctly... In most cases for binary formats an integer type will be expected for 99% of cases. Something like `uint(8,a/b)` will only generate an alternates set within the realm of integers, so a rule containing such an expression as this could only match when this particular byte matches a/b, which can only happen when a/b results in an integer from 0 to 255. For all a/b where this isn't true, the rule will not match. But support for matching float encodings is still needed, such as for `reading = float(32,0~1);`, which would happily match an ieee754 32-bit binary float approximation of 0.3 or 0.8 or anything from 0 to 1 that can be represented in this float format.
  - HelloNurse 4y ago
    You are talking about the limited precision numbers in data, not about appropriate numbers for use in your language to express counts, repetitions, aggregate sizes etc. They are two related but categorically different kinds of entity, like screws and screwdrivers or sheet music and audio files. Grammar rules aren't supposed to work "for 99% of cases" depending on complex invisible constraints such as whether finite precision causes floating point arithmetic to depart from exact integer arithmetic.
    - kstenerud 4y ago
      I'm not saying that grammar rules are supposed to work for only 99% of cases; that comment was referring to the fact that most binary data of interest for binding (counts etc) will be stored in the document as integers. But I'm still not understanding what you're saying. All counts, repetitions, aggregate sizes etc are going to come from limited precision data in the document being examined (except in cases where you're passing in a numeric literal). In some cases these values will be used directly to produce expressions via functions and repetition, and in others they will be used in calculations that are then passed to functions or repetition to produce expressions. The grammar rules only match expressions, which are bit patterns, not numbers. Some of those bit patterns are arrived at through functions such as `float()`, taking a number range to give an expression composed of a set of possible (ieee754 binary) bit patterns. The bit patterns that the `float(32,1~10)` expression will match are far more numerous than the bit patterns that the `uint(32,1~10)` expression will match (0x00000001, 0x00000002, ... 0x0000000a), even though they both are taking in the same numeric range (each picks out only the values it can represent as discrete bit patterns from the range). `uint(32,1.5)` is a malformed grammar because it violates the invariant of `uint()`, whereas `uint(32,2)` is fine (giving an expression of the big endian bit pattern 00000000000000000000000000000010).

HelloNurse 4y ago

> The idea here is to have all numbers be treated as reals (in the mathematical sense) No. You can't represent, compare etc. most real numbers, so they aren't any good for computational purposes. Practically usable number types include integers and things that have sufficiently number-like behaviour (e.g. fields) and can be represented with a finite number of integers: rational numbers, rational numbers with field extensions, vector spaces, finite sets, intervals, etc. In a language for grammars you can probably stop at integers; general purpose programming languages might want more (for instance, Python has rational numbers) but advanced number types can usually be implemented with libraries.

kstenerud 4y ago
I think maybe I'm not understanding you correctly... In most cases for binary formats an integer type will be expected for 99% of cases. Something like `uint(8,a/b)` will only generate an alternates set within the realm of integers, so a rule containing such an expression as this could only match when this particular byte matches a/b, which can only happen when a/b results in an integer from 0 to 255. For all a/b where this isn't true, the rule will not match. But support for matching float encodings is still needed, such as for `reading = float(32,0~1);`, which would happily match an ieee754 32-bit binary float approximation of 0.3 or 0.8 or anything from 0 to 1 that can be represented in this float format.
- HelloNurse 4y ago
  You are talking about the limited precision numbers in data, not about appropriate numbers for use in your language to express counts, repetitions, aggregate sizes etc. They are two related but categorically different kinds of entity, like screws and screwdrivers or sheet music and audio files. Grammar rules aren't supposed to work "for 99% of cases" depending on complex invisible constraints such as whether finite precision causes floating point arithmetic to depart from exact integer arithmetic.
  - kstenerud 4y ago
    I'm not saying that grammar rules are supposed to work for only 99% of cases; that comment was referring to the fact that most binary data of interest for binding (counts etc) will be stored in the document as integers. But I'm still not understanding what you're saying. All counts, repetitions, aggregate sizes etc are going to come from limited precision data in the document being examined (except in cases where you're passing in a numeric literal). In some cases these values will be used directly to produce expressions via functions and repetition, and in others they will be used in calculations that are then passed to functions or repetition to produce expressions. The grammar rules only match expressions, which are bit patterns, not numbers. Some of those bit patterns are arrived at through functions such as `float()`, taking a number range to give an expression composed of a set of possible (ieee754 binary) bit patterns. The bit patterns that the `float(32,1~10)` expression will match are far more numerous than the bit patterns that the `uint(32,1~10)` expression will match (0x00000001, 0x00000002, ... 0x0000000a), even though they both are taking in the same numeric range (each picks out only the values it can represent as discrete bit patterns from the range). `uint(32,1.5)` is a malformed grammar because it violates the invariant of `uint()`, whereas `uint(32,2)` is fine (giving an expression of the big endian bit pattern 00000000000000000000000000000010).