4 ms·

>Hello Microsoft programmers, a memory dump is not a file format Wait until you see the Office formats without the extra x in the extension.

mnd999 4y ago
That was a deliberate attempt to make it impossible for competitors to read and write their format wasn’t it?
- jve 4y ago
  I don't know the exact year that file format was made, but you had to think about performance more than wasting so much memory and cpu cycles to have a fancy file format, no? Ofcourse that era Microsoft tools didn't think much about interoperability outside of Microsoft ecosystem. But to me it seems that them having some XML based file format at that time would be silly and have disadvantages. XML actually was started on '96 and published on '98 https://en.wikipedia.org/wiki/XML https://en.wikipedia.org/wiki/XML .doc seems to be used starting from '89 > The format used in earlier, pre-97 ("1.0" 1989 through "7.0" 1995) versions of Word are less known https://en.wikipedia.org/wiki/Doc_(computing) https://en.wikipedia.org/wiki/Doc_(computing)
  - Someone 4y ago
    Agree. I think most software of the era did that kind of thing. Most of it wasn’t popular enough and/or got abandoned soon enough for ‘the internet’ to notice, though. If your software runs in 256kB, writing to a floppy disk, you pack your data structures, use bit fields where possible, and you’re not going to waste CPU time, RAM and disk space to write a format that’s easy to understand and extend. (Of course, they extended it, anyways, as everybody would)
- ccurrens 4y ago
  I attended a seminar on the office binary file formats about 10 years ago at MS. The reason it was done was for performance reasons, including the wonky layout that made it quicker to save and read the file from slow media like floppy discs.
  - jve 4y ago
    I also remember reading about that somwhere, sometime... loading... ah, here it is: https://www.joelonsoftware.com/2008/02/19/why-are-the-microsoft-office-file-formats-so-complicated-and-some-workarounds/ https://www.joelonsoftware.com/2008/02/19/why-are-the-micros... > The file format is contorted, where necessary, to make common operations fast. For example, Excel 95 and 97 have something called “Simple Save” which they use sometimes as a faster variation on the OLE compound document format, which just wasn’t fast enough for mainstream use. Word had something called Fast Save. To save a long document quickly, 14 out of 15 times, only the changes are appended to the end of the file, instead of rewriting the whole document from scratch. On the hard drives of the day, this meant saving a long document took one second instead of thirty. (It also meant that deleted data in a document was still in the file. This turned out to be not what people wanted.)
    - int_19h 4y ago
      The underlying file format, COM Structured Storage, is basically filesystem-in-a-file, and works much like FAT. So, bits of deleted data would be floating around even without any performance hacks used by the app itself.
- bonzini 4y ago
  No, it's just that if you have to convert you need more memory to load and save, and then you have less memory for the document itself.
- zaphar 4y ago
  In the era that these formats were started it was fairly commmon to treat your file format as a dump of memory. It wasn't about preventing competitors from reading it. It was more about the speed of loading it and the lower overhead. This was before the landscaped had changed so much that the security implications of a format you didn't parse before loading it and the hardware landscape had changed so much that there was no longer a detectable performance hit on parsing a format before loading it. You had interchange formats which were designed to be used between systems and had a defined parseable formats but if your format was meant to be used by you system alone and not shared then it was believed that a memory dump was strictly faster and better for the the user experience. We've since learned a whole about why that is not a good idea but it wasn't always obvious in the before times.
- p_l 4y ago
  They were formats designed to be fast on floppy drives which were major storage format at the time, and most importantly, designed as "work in progress" format. For interchange, you were supposed to use other formats - for example RTF with Word, which was kept in sync with DOC capabilities all the way to Word 2003 which was the last version that used DOC (2007 uses DOCX and no longer maintains DOC+RTF combo in sync with internal features). However, saving back the file if you did small change in large RTF file took ages in comparison.
- pianoben 4y ago
  Not really; it was an attempt at making fast and responsive software. If you can just blit a file into memory, why not? It's a hell of a lot faster than parsing and validating bytes and then converting them into your runtime data structures. It's a lot faster in the other direction, too. When all the world runs one architecture, why wouldn't you do that?
severak_cz 4y ago
yeah, such a classical Microsoft move
- naikrovek 4y ago
  it was a very common approach at the time.

mnd999 4y ago

That was a deliberate attempt to make it impossible for competitors to read and write their format wasn’t it?

jve 4y ago
I don't know the exact year that file format was made, but you had to think about performance more than wasting so much memory and cpu cycles to have a fancy file format, no? Ofcourse that era Microsoft tools didn't think much about interoperability outside of Microsoft ecosystem. But to me it seems that them having some XML based file format at that time would be silly and have disadvantages. XML actually was started on '96 and published on '98 https://en.wikipedia.org/wiki/XML https://en.wikipedia.org/wiki/XML .doc seems to be used starting from '89 > The format used in earlier, pre-97 ("1.0" 1989 through "7.0" 1995) versions of Word are less known https://en.wikipedia.org/wiki/Doc_(computing) https://en.wikipedia.org/wiki/Doc_(computing)
- Someone 4y ago
  Agree. I think most software of the era did that kind of thing. Most of it wasn’t popular enough and/or got abandoned soon enough for ‘the internet’ to notice, though. If your software runs in 256kB, writing to a floppy disk, you pack your data structures, use bit fields where possible, and you’re not going to waste CPU time, RAM and disk space to write a format that’s easy to understand and extend. (Of course, they extended it, anyways, as everybody would)
ccurrens 4y ago
I attended a seminar on the office binary file formats about 10 years ago at MS. The reason it was done was for performance reasons, including the wonky layout that made it quicker to save and read the file from slow media like floppy discs.
- jve 4y ago
  I also remember reading about that somwhere, sometime... loading... ah, here it is: https://www.joelonsoftware.com/2008/02/19/why-are-the-microsoft-office-file-formats-so-complicated-and-some-workarounds/ https://www.joelonsoftware.com/2008/02/19/why-are-the-micros... > The file format is contorted, where necessary, to make common operations fast. For example, Excel 95 and 97 have something called “Simple Save” which they use sometimes as a faster variation on the OLE compound document format, which just wasn’t fast enough for mainstream use. Word had something called Fast Save. To save a long document quickly, 14 out of 15 times, only the changes are appended to the end of the file, instead of rewriting the whole document from scratch. On the hard drives of the day, this meant saving a long document took one second instead of thirty. (It also meant that deleted data in a document was still in the file. This turned out to be not what people wanted.)
  - int_19h 4y ago
    The underlying file format, COM Structured Storage, is basically filesystem-in-a-file, and works much like FAT. So, bits of deleted data would be floating around even without any performance hacks used by the app itself.
bonzini 4y ago
No, it's just that if you have to convert you need more memory to load and save, and then you have less memory for the document itself.
zaphar 4y ago
In the era that these formats were started it was fairly commmon to treat your file format as a dump of memory. It wasn't about preventing competitors from reading it. It was more about the speed of loading it and the lower overhead. This was before the landscaped had changed so much that the security implications of a format you didn't parse before loading it and the hardware landscape had changed so much that there was no longer a detectable performance hit on parsing a format before loading it. You had interchange formats which were designed to be used between systems and had a defined parseable formats but if your format was meant to be used by you system alone and not shared then it was believed that a memory dump was strictly faster and better for the the user experience. We've since learned a whole about why that is not a good idea but it wasn't always obvious in the before times.
p_l 4y ago
They were formats designed to be fast on floppy drives which were major storage format at the time, and most importantly, designed as "work in progress" format. For interchange, you were supposed to use other formats - for example RTF with Word, which was kept in sync with DOC capabilities all the way to Word 2003 which was the last version that used DOC (2007 uses DOCX and no longer maintains DOC+RTF combo in sync with internal features). However, saving back the file if you did small change in large RTF file took ages in comparison.
pianoben 4y ago
Not really; it was an attempt at making fast and responsive software. If you can just blit a file into memory, why not? It's a hell of a lot faster than parsing and validating bytes and then converting them into your runtime data structures. It's a lot faster in the other direction, too. When all the world runs one architecture, why wouldn't you do that?

severak_cz 4y ago

yeah, such a classical Microsoft move

naikrovek 4y ago
it was a very common approach at the time.