3 ms·

As much as I like Rust, I don’t think that it would have solved the Mars Climate Orbiter problem. That was caused by one party writing numbers out into a CSV fi

by db48x 2y ago

As much as I like Rust, I don’t think that it would have solved the Mars Climate Orbiter problem. That was caused by one party writing numbers out into a CSV file in one unit, and a different party reading the CSV file but assuming that the numbers were in a different unit. Both parties could have been using Rust, and using types to encode physical units, and the problem could still have happened.

pessimizer 2y ago
Unless the unit was part of the type, and why wouldn't it be? A unit error is a type error. You don't have to say that something is 5 long, you can say that it's 5 inches long. And the function can be made only to accept a length in inches, and you can implement :Into<Inches> on your length type for an automagic conversion.
- nurettin 2y ago
  That wouldn't be enough unless the csv somehow kept serialized column type in the header and the deserializer knew how to interpret it.
- db48x 2y ago
  Of course, but that doesn’t solve the problem. The problem here is that two separate organizations with completely different software were communicating via CSV file. There was a contractual obligation between them that required them to use specific units, but no technical collaboration. The problem was not in their software, but in the management and engineering discipline of the two groups.
tialaramex 2y ago
Yeah, even an extremely powerful units type system like C++ mp-units doesn't fix "I assumed this number was feet and you assumed it was millimetres". But both the mp-units and this lighter weight Rust approach would at least make you write that down in the software. There's some chance the mistake is realised, whereas when it's just an organisational assumption that just doesn't happen.
westurner 2y ago
W3C CSVW supports per-column schema. Serialize a dict containing a value with uncertainties and/or Pint (or astropy.units) and complex values to JSON, then read it from JSON back to the same types. Handle datetimes, complex values, and categoricals "CSVW: CSV on the Web" https://github.com/jazzband/tablib/issues/305 https://github.com/jazzband/tablib/issues/305 7 Columnar metadata header rows for a CSV: column label, property URI, datatype, quantity/unit, accuracy, precision, significant figures https://wrdrd.github.io/docs/consulting/linkedreproducibility#csv-csvw-and-metadata-rows https://wrdrd.github.io/docs/consulting/linkedreproducibilit... CSV on the Web: A Primer > 6. Advanced Use > 6.1 How do you support units of measure? https://www.w3.org/TR/tabular-data-primer/#units-of-measure https://www.w3.org/TR/tabular-data-primer/#units-of-measure You can specify units in a CSVW file with QUDT, an RDFS schema and vocabulary for Quantities, Units, Dimensions, and Types Schema.org has StructuredValue and rdfs:subPropertyOf like QuantitativeValue and QuantitativeValueDistribution: https://schema.org/StructuredValue https://schema.org/StructuredValue There are linked data schema for units, and there are various in-code in-RAM typed primitive and compound type serialization libraries for various programming languages; but they're not integrated, so we're unable to share data with units between apps and fall back to CSVW. There are zero-copy solutions for sharing variables between cells of e.g. a polyglot notebook with input cells written in more than one programing language without reshaping or serializing.

by db48x 2y ago

pessimizer 2y ago

Unless the unit was part of the type, and why wouldn't it be? A unit error is a type error. You don't have to say that something is 5 long, you can say that it's 5 inches long. And the function can be made only to accept a length in inches, and you can implement :Into<Inches> on your length type for an automagic conversion.

nurettin 2y ago
That wouldn't be enough unless the csv somehow kept serialized column type in the header and the deserializer knew how to interpret it.
db48x 2y ago
Of course, but that doesn’t solve the problem. The problem here is that two separate organizations with completely different software were communicating via CSV file. There was a contractual obligation between them that required them to use specific units, but no technical collaboration. The problem was not in their software, but in the management and engineering discipline of the two groups.

tialaramex 2y ago

Yeah, even an extremely powerful units type system like C++ mp-units doesn't fix "I assumed this number was feet and you assumed it was millimetres". But both the mp-units and this lighter weight Rust approach would at least make you write that down in the software. There's some chance the mistake is realised, whereas when it's just an organisational assumption that just doesn't happen.

westurner 2y ago

W3C CSVW supports per-column schema. Serialize a dict containing a value with uncertainties and/or Pint (or astropy.units) and complex values to JSON, then read it from JSON back to the same types. Handle datetimes, complex values, and categoricals "CSVW: CSV on the Web" https://github.com/jazzband/tablib/issues/305 https://github.com/jazzband/tablib/issues/305 7 Columnar metadata header rows for a CSV: column label, property URI, datatype, quantity/unit, accuracy, precision, significant figures https://wrdrd.github.io/docs/consulting/linkedreproducibility#csv-csvw-and-metadata-rows https://wrdrd.github.io/docs/consulting/linkedreproducibilit... CSV on the Web: A Primer > 6. Advanced Use > 6.1 How do you support units of measure? https://www.w3.org/TR/tabular-data-primer/#units-of-measure https://www.w3.org/TR/tabular-data-primer/#units-of-measure You can specify units in a CSVW file with QUDT, an RDFS schema and vocabulary for Quantities, Units, Dimensions, and Types Schema.org has StructuredValue and rdfs:subPropertyOf like QuantitativeValue and QuantitativeValueDistribution: https://schema.org/StructuredValue https://schema.org/StructuredValue There are linked data schema for units, and there are various in-code in-RAM typed primitive and compound type serialization libraries for various programming languages; but they're not integrated, so we're unable to share data with units between apps and fall back to CSVW. There are zero-copy solutions for sharing variables between cells of e.g. a polyglot notebook with input cells written in more than one programing language without reshaping or serializing.