3 ms·

If you’re reading or writing a file format, whether it’s text, floating point or some other encoding, you don’t have to store the data in memory the same way wh

If you’re reading or writing a file format, whether it’s text, floating point or some other encoding, you don’t have to store the data in memory the same way when you work with it. …which frees you to write your data to disk however you want. And given that freedom to choose any serialization format, you definitely should not manually be figuring out every byte you write — you probably write an abstraction, so wherever complexity there is should be completely transparent because you unit tested the hell out of your sterilization routines. …which to me makes the whole complexity point moot, and which also makes the whole “text is better for memory minimization” argument also moot. I do think text formats have a place — primarily readability and interoperability — but to choose it for purely reducing your file sizes sounds insane to me.

ggrrhh_ta 5y ago
of course... I never thought that text files to reduce file sizes is sane... Maybe because I don't want to write in a wishy-washy way as if I was stepping into a mine field I came across like that [look at my first comment... I think I started as "I know it is counterintuitive" - I wanted to point point out the binary (as naturally interpreted: storing floats/doubles) does not necessarily mean smaller file sizes for files generated by humans.
okamiueru 5y ago
There is a good reason for why one would want to store data the same way as when you work on it. It removes the need to process the data itself. A common approach is to follow it up with a fast compression algorithm when storage space is important. The difference in post-processing when having to decode text is around two orders of magnitude, which is very significant. In the case of the 3MF format in question in this article, I find the tradeoff for human readable files to be a very strange one. The benefit of that at the cost of say waiting a minute to load a file that would otherwise take a second, is a puzzling one. But, it's all a tradeoff after all. If humans work with the files, it is a strong argument work with text. When software stores and loads files, it will likely be a poor choice to store data as text. But, you are right. The file size consideration is not all that relevant. However, what was mostly discussed was the argument that serializing text that represents numbers is more space efficient. Which as I mentioned a few times, can only be the case when the straight serialization of the numbers is done poorly.

ggrrhh_ta 5y ago

of course... I never thought that text files to reduce file sizes is sane... Maybe because I don't want to write in a wishy-washy way as if I was stepping into a mine field I came across like that [look at my first comment... I think I started as "I know it is counterintuitive" - I wanted to point point out the binary (as naturally interpreted: storing floats/doubles) does not necessarily mean smaller file sizes for files generated by humans.

okamiueru 5y ago

There is a good reason for why one would want to store data the same way as when you work on it. It removes the need to process the data itself. A common approach is to follow it up with a fast compression algorithm when storage space is important. The difference in post-processing when having to decode text is around two orders of magnitude, which is very significant. In the case of the 3MF format in question in this article, I find the tradeoff for human readable files to be a very strange one. The benefit of that at the cost of say waiting a minute to load a file that would otherwise take a second, is a puzzling one. But, it's all a tradeoff after all. If humans work with the files, it is a strong argument work with text. When software stores and loads files, it will likely be a poor choice to store data as text. But, you are right. The file size consideration is not all that relevant. However, what was mostly discussed was the argument that serializing text that represents numbers is more space efficient. Which as I mentioned a few times, can only be the case when the straight serialization of the numbers is done poorly.