3 ms·

I agree with everything; for users that are experts in their domain (not computer science), if you don't use text in the frontend that they can parse with littl

I agree with everything; for users that are experts in their domain (not computer science), if you don't use text in the frontend that they can parse with little (but very inefficient) code in their scripting language of choice, then, they will be reinventing some even more difficult/complex/yet-another way to handle text that you will have to support in the backend for performance - and in the backend I choose binary formats (although my experience, without focusing too much in any other aspects of performance than retrieval, because most uses with large amounts of data require specific filtering/reordering/new memory layout for efficient algorithms, etc.). And, counterintuitively, when you do that, and you have hundreds of files which are hundreds of megabytes where some columns are integers, and you see the size of your binary data you wonder... why is my binary data few times larger?...

okamiueru 5y ago
That is a good point. It is a good observation that just binary encoding numbers does not mean you have an efficient serialized representation. And, that sometimes that can be so bad that even encoding numbers as serialized text beats it out. I think what threw me off was the context of this discussion, which was 3MF file format. For such cases, the usefulness of humans opening up the data in text editors is somewhat contrived. I believe that tradeoff for the 3MF format was a mistake, and will likely hinder broad adoption. I'm however mostly concerned with the extra processing required in order to decode the human readable files. We're talking a likely speedup of a factor of 50-100. For models that push this limit, imagine a file that takes 60 seconds to process. This isn't entirely unreasonable for a model with insane level of details. If they instead had gone for a binary file representation, the load time would likely be less than a second. Failing to recognize this tradeoff for human readable text... is unfortunate.

okamiueru 5y ago

That is a good point. It is a good observation that just binary encoding numbers does not mean you have an efficient serialized representation. And, that sometimes that can be so bad that even encoding numbers as serialized text beats it out. I think what threw me off was the context of this discussion, which was 3MF file format. For such cases, the usefulness of humans opening up the data in text editors is somewhat contrived. I believe that tradeoff for the 3MF format was a mistake, and will likely hinder broad adoption. I'm however mostly concerned with the extra processing required in order to decode the human readable files. We're talking a likely speedup of a factor of 50-100. For models that push this limit, imagine a file that takes 60 seconds to process. This isn't entirely unreasonable for a model with insane level of details. If they instead had gone for a binary file representation, the load time would likely be less than a second. Failing to recognize this tradeoff for human readable text... is unfortunate.